Method and system for automatic identification and desensitization of sensitive data in communication logs

Abstract

The present application relates to the technical field of automatic identification and desensitization processing, in particular to a method and system for automatic identification and desensitization processing of sensitive data in communication logs, comprising the following steps: S1, collecting communication data in the communication logs, performing semantic analysis on the communication data to determine the corresponding industry scene, searching for sensitive samples based on the industry scene and constructing a basic sensitive database; S2, dividing the sensitive samples in the basic sensitive database into sensitive types; performing three-layer semantic analysis on the communication data through industry training of an NLP model, realizing accurate scene positioning in combination with industry key terms and business scene identification, and constructing a basic sensitive database based on the sensitive samples searched and adapted according to the scene, effectively solving the problem of poor scene adaptability in the prior art, making the sensitive data identification more in line with the business characteristics and sensitive type requirements of specific industries, and significantly improving the identification hit rate of industry-specific sensitive data.

Description

Technical Field

[0001] This invention relates to the field of data exchange technology, and more specifically, to a method and system for the automatic identification and desensitization of sensitive data in communication logs. Background Technology

[0002] In today's era of rapid development of digital communication technology, communication logs, as the core carrier for recording communication behavior and data interaction processes, are widely used in various fields such as enterprise operation and maintenance management, business process traceability, and compliance auditing. The communication data they contain includes both structured communication addresses, timestamps, and other information, as well as unstructured dialogue texts, business instructions, and other content.

[0003] Existing sensitive data identification and de-identification technologies often rely on fixed regular expression rules or general keyword libraries, which are insufficient to handle sensitive data with implicit associations or format distortions in communication logs. This results in high false positive and false negative rates. Misjudgments may include non-sensitive data in the de-identification scope, leading to decreased data usability, while false negatives may result in the leakage of sensitive information. Furthermore, existing technologies lack deep adaptability to industry scenarios. General identification models do not consider the specific business characteristics and differences in sensitive data types within specific industries, resulting in poor identification performance in particular industry scenarios. For example, industry-specific sensitive data such as transaction serial numbers in the financial sector and medical record numbers in the medical sector are difficult to identify accurately. Therefore, this paper proposes an automatic identification and de-identification method and system for sensitive data in communication logs. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for automatic identification and desensitization of sensitive data in communication logs, so as to solve the problems mentioned in the background art.

[0005] To address the aforementioned technical problems, one objective of this invention is to provide a method for automatically identifying and de-identifying sensitive data in communication logs, comprising the following steps: S1. Collect communication data from communication logs, perform semantic analysis on the communication data to determine the corresponding industry scenarios, retrieve sensitive samples based on industry scenarios, and build a basic sensitive database; S2. Divide the sensitive samples in the basic sensitive database into sensitive types, set weight coefficients according to the sample proportion of each sensitive type, determine the corresponding standard samples for each sensitive type, filter communication data according to sensitive type, select labeled data by combining the weight coefficient of sensitive type and the corresponding data quantity, and segment the communication log with the labeled data as nodes to obtain multiple data segments. S3. Set the corresponding accuracy threshold according to the weight coefficient of each sensitivity type. Within the same sensitivity type and corresponding data segment, compare the data similarity between the communication data and the standard sample of that type. Combine the accuracy threshold to determine the matching accuracy between the data and the sensitivity type, and remove the sensitive type markers that do not match accurately. S4. Perform correlation analysis between the accurately matched sensitive communication data and the non-sensitive communication data, adjust the sensitivity type of the non-sensitive communication data according to the correlation results, and update the basic sensitive database by combining the accurately matched sensitive data in each data segment, and perform sensitivity identification on subsequent communication data based on the updated database. S5. Configure exclusive desensitization rules for each sensitive type, and call the corresponding rules to perform desensitization processing on the communication data.

[0006] As a further improvement to this technical solution, in step S1, a connection is established with the communication management terminal, and then the communication log and the communication data contained therein are collected at the communication management terminal; By analyzing the text content, semantic relationships, and business characteristics of communication data using natural language processing technology, key industry terms and business scenario identifiers are extracted, and the corresponding industry scenarios of communication logs are accurately located based on key terms and business scenario identifiers. We acquire sensitive samples that are highly compatible with industry scenarios through big data retrieval, and then integrate them to build a basic sensitive database.

[0007] As a further improvement to this technical solution, in step S2, the sensitive samples in the basic sensitive database are classified into sensitive types. By analyzing the data characteristics and business attributes of each sensitive sample one by one, samples with the same characteristics and attributes are classified into the same sensitive type. The sample size corresponding to each sensitive type is combined with the total sample size for a percentage analysis. The higher the percentage of a sensitive type's sample size, the greater the weight coefficient assigned to that sensitive type. Conversely, the lower the percentage of a sensitive type's sample size, the smaller the weight coefficient assigned to that sensitive type. Extract the sensitive sample logs corresponding to each sensitive sample, perform log similarity analysis between the sensitive sample logs and the communication logs collected by S1, obtain the log similarity between each sensitive sample log and the communication log, and then select the sensitive sample with the highest log similarity as the standard sample for each sensitive type.

[0008] As a further improvement to this technical solution, in step S2, feature attribute analysis is performed on each sensitive type based on sensitive samples and standard samples to obtain the feature attributes corresponding to each sensitive type. The communication data in the communication log is compared with the feature attributes of the sensitive type. The communication data that matches the feature attributes is classified into the corresponding sensitive type. Conversely, when the communication data does not match the feature attributes of each sensitive type, the next communication data is compared until all communication data has been compared. Calculate the product of the number of communication data of the same sensitivity type and the corresponding weight coefficient, sort them from largest to smallest according to the product value, and select the communication data with the first ranking as the labeled data; The communication log is segmented into multiple data segments by using labeled data as nodes. The range of each data segment is defined as starting from the current node and ending at the next node, excluding the next node. The next node serves as the starting node of the new data segment. The process is repeated iteratively to complete the segmentation of the entire log.

[0009] As a further improvement to this technical solution, in step S3, a corresponding accurate threshold is set according to the weight coefficient of each sensitivity type. The method is achieved through linear function mapping. The minimum accurate threshold corresponding to the maximum value of the weight coefficient is set, and the maximum accurate threshold corresponding to the minimum value of the weight coefficient is set. For the weight coefficient in the middle range, the corresponding accurate threshold is calculated by linear interpolation algorithm, thereby completing the accurate threshold corresponding to each sensitivity coefficient. Within the same sensitive type and corresponding data segment, semantic feature analysis is performed on communication data and standard samples to obtain the semantic features of communication data and standard samples. Then, the semantic features of communication data are combined with the semantic features of standard samples to perform feature fit analysis, and the data similarity is obtained based on the feature fit. The data similarity is compared with the accuracy threshold. When the data similarity is greater than the accuracy threshold, it is determined that the communication data matches the corresponding sensitive type accurately, and the sensitive type mark is retained. When the similarity value is less than the accuracy threshold, the match is deemed inaccurate, the sensitive type tag corresponding to the communication data is directly removed, and it is not included in the subsequent sensitive processing flow.

[0010] As a further improvement to this technical solution, in step S4, the accurately matched communication data and the non-sensitive communication data are analyzed for contextual association, business logic association, and data feature association to determine whether the non-sensitive communication data and the accurately matched sensitive communication data belong to the same communication session, the same business process, and have a direct data dependency relationship. If the judgment results simultaneously meet the criteria of being in the same communication session, the same business process, and having a direct data dependency relationship, then the two are determined to be associated and matched, and the non-sensitive communication data is adjusted to the same sensitive type as the accurately matched sensitive communication data. Conversely, if there is no simultaneous fulfillment of the same communication session, the same business process, and a direct data dependency relationship, it is determined that the two are not related and the original type of the non-sensitive communication data remains unchanged.

[0011] As a further improvement to this technical solution, the basic sensitive database is updated based on the association matching results and the accurately matched sensitive data, with the following steps: Supplement each data segment with accurately matched communication data and associated matching communication data as new sensitive samples; Delete sensitive samples from the original database that do not match the actual matching results; Based on the number of newly added and deleted samples, update the sample proportion and corresponding weight coefficient of each sensitive type; Re-screen the standard samples for each sensitive type; Then, the updated basic sensitive database is invoked to perform a sensitive identification process on the newly generated communication logs.

[0012] As a further improvement to this technical solution, in step S5, a differentiated and exclusive desensitization rule is set for each sensitive type through the communication management terminal; Among them, the specific desensitization rules include desensitization methods, desensitization intensity, and data retention rules; Identify the sensitive types corresponding to each communication data, call the dedicated desensitization rules corresponding to that sensitive type, and complete the desensitization process.

[0013] The second objective of this invention is to provide an automatic identification and desensitization system for sensitive data in communication logs, including any of the above-mentioned methods for automatic identification and desensitization of sensitive data in communication logs, comprising a database establishment module, a data segmentation module, a data removal module, a sensitive identification module, and a desensitization processing module. The database establishment module is used to collect communication data from communication logs, perform semantic analysis on the communication data to determine the corresponding industry scenarios, retrieve sensitive samples based on the industry scenarios, and build a basic sensitive database. The data segmentation module is used to classify sensitive samples in the basic sensitive database into sensitive types, set weight coefficients according to the sample proportion of each sensitive type, determine the corresponding standard samples for each sensitive type, filter communication data according to sensitive type, select labeled data by combining the weight coefficient of sensitive type and the corresponding data quantity, and segment the communication log with the labeled data as nodes to obtain multiple data segments. The data removal module is used to set the corresponding accuracy threshold according to the weight coefficient of each sensitivity type, compare the communication data with the standard sample of the same sensitivity type within the same sensitivity type and the corresponding data segment, determine the matching accuracy between the data and the sensitivity type in combination with the accuracy threshold, and remove the sensitive type markers that do not match accurately. The sensitive identification module is used to perform correlation analysis between accurately matched sensitive communication data and non-sensitive communication data, adjust the sensitivity type of non-sensitive communication data according to the correlation results, update the basic sensitive database by combining accurately matched sensitive data in each data segment, and perform sensitive identification on subsequent communication data based on the updated database. The desensitization processing module is used to configure exclusive desensitization rules for each sensitive type and call the corresponding rules to perform desensitization processing on the communication data.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. A method and system for automatic identification and de-identification of sensitive data in communication logs. This method uses an industry-trained NLP model to perform three-layer semantic parsing of communication data, combines key industry terms and business scenario identifiers to achieve accurate scenario positioning, and then builds a basic sensitive database based on scenario-adapted sensitive samples. This effectively solves the problem of poor scenario adaptability of existing technologies, making sensitive data identification more in line with the business characteristics and sensitive type requirements of specific industries, and significantly improving the identification hit rate of industry-specific sensitive data.

[0015] 2. A method and system for automatic identification and de-identification of sensitive data in communication logs. By setting dynamic weight coefficients based on sample proportions and combining the weight coefficients with the data quantity to select labeled data for segmented processing of communication logs, the identification of sensitive data focuses on areas associated with high-priority sensitive data, avoiding the inefficiency caused by blindly identifying all logs. At the same time, the dynamic adjustment mechanism of the weight coefficients ensures that the identification priority of different sensitive types matches the actual business needs, improving the overall targeting and efficiency of identification and processing.

[0016] 3. A method and system for automatic identification and desensitization of sensitive data in communication logs. This method establishes a negative correlation mapping relationship between weight coefficients and accuracy thresholds through linear interpolation algorithm, and achieves semantic depth matching between communication data and standard samples by combining cosine similarity algorithm. Then, it determines the association matching relationship between sensitive and non-sensitive data through triple association conditions. This not only effectively eliminates mislabeled data in the initial screening and reduces the misjudgment rate of sensitive type determination, but also accurately mines hidden associated sensitive data, expands the coverage of sensitive data, and achieves a dual improvement in identification accuracy and coverage. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the automatic identification and desensitization process for sensitive data in communication logs according to the present invention. Figure 2 This is a flowchart of S1 of the present invention; Figure 3 This is a flowchart of S2 of the present invention; Figure 4 This is a flowchart of S3 of the present invention; Figure 5 This is a flowchart of S4 of the present invention; Figure 6 This is a flowchart of S5 of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] like Figures 1-6 As shown, one of the objectives of this invention is to provide a method for automatically identifying and de-identifying sensitive data in communication logs, comprising the following steps: S1. Collect communication data from communication logs, perform semantic analysis on the communication data to determine the corresponding industry scenarios, retrieve sensitive samples based on industry scenarios and build a basic sensitive database; complete the input of raw data and the reserve of basic sensitive samples to provide materials and judgment criteria for subsequent sensitive type classification and data screening; In S1, a connection is established with the communication management terminal, and then the communication logs and the communication data contained therein are collected at the communication management terminal; Establish a stable data transmission connection with the communication management terminal. Based on the preset communication protocol (preferably TCP / IP), collect complete communication log files in batches from the log storage module of the communication management terminal, and synchronously extract the structured communication data (such as the addresses of the communicating parties and timestamps) and unstructured communication data (such as dialogue text and business instructions) contained in the logs. By analyzing the text content, semantic relationships, and business characteristics of communication data using natural language processing technology, key industry terms and business scenario identifiers are extracted, and the corresponding industry scenarios of communication logs are accurately located based on key terms and business scenario identifiers. We employ industry-specific trained Natural Language Processing (NLP) models as the optimal parsing tool to perform three-layer parsing on communication data. The first layer extracts core vocabulary and sentence logic from the communication data. The second layer constructs a dependency network between words to identify business action directions. The third layer extracts industry-specific business identifiers. By integrating the three-layer parsing results, a set of key industry terms and a set of business scenario identifiers are extracted. Through dual matching of terms and identifiers, the target industry scenario corresponding to the communication log is accurately located. We acquire sensitive samples that are highly compatible with industry scenarios through big data retrieval, and then integrate them to build a basic sensitive database.

[0020] Based on the target industry scenario, a set of scenario keywords is constructed. A weighted keyword retrieval algorithm is used to retrieve sensitive sample big data. The retrieved sensitive samples are deduplicated and format standardized. After removing invalid samples, they are classified and stored according to the preliminary form of sensitive data, and finally the basic sensitive database is constructed.

[0021] S2. Sensitive samples in the basic sensitive database are classified into sensitive types. Weighting coefficients are set according to the sample proportion of each sensitive type. At the same time, standard samples are determined for each sensitive type. Communication data is screened according to sensitive type. Labeled data is selected by combining the weighting coefficient of sensitive type and the corresponding data quantity. The communication log is segmented using the labeled data as nodes to obtain multiple data segments. The unordered samples in the basic sensitive database are transformed into structured features. Data segments are divided by labeled data nodes to achieve precise regional processing of sensitive identification. In S2, sensitive samples in the basic sensitive database are classified into sensitive types. By analyzing the data characteristics and business attributes of each sensitive sample one by one, samples with the same characteristics and attributes are classified into the same sensitive type. The core data features (such as format features, length features, and data structure features) and business attributes (such as business process and related business object) of each sensitive sample in the basic sensitive database are extracted one by one using the feature attribute clustering algorithm. Sensitive samples with completely consistent data features and business attributes are classified into the same sensitive type, and a mapping table between sensitive type and feature attribute is generated. The sample size for each sensitivity type is combined with the total sample size for a percentage analysis. The higher the percentage of a sensitivity type's sample size, the greater the weight coefficient assigned to that sensitivity type; conversely, the lower the percentage of a sensitivity type's sample size, the smaller the weight coefficient assigned to that sensitivity type. The formula is as follows: ; in, The weight coefficient for the k-th sensitivity type is... Let k be the percentage of samples with the kth sensitive type. The total number of sensitive types, with a total weight coefficient of 1; Extract the sensitive sample logs corresponding to each sensitive sample, perform log similarity analysis between the sensitive sample logs and the communication logs collected by S1, obtain the log similarity between each sensitive sample log and the communication logs, and then select the sensitive sample with the highest log similarity as the standard sample for that sensitive type in each sensitive type, as shown in the following formula:

[0022] in, Let be the similarity value between the i-th sample log in the k-th sensitive type and the communication log to be processed. Let i be the semantic feature vector of the i-th sample log in the k-th sensitivity type. Select the semantic feature vector of the communication log to be processed. The sensitive sample corresponding to the sample log with the largest value is used as the standard sample.

[0023] In S2, feature attribute analysis is performed on each sensitivity type based on sensitive samples and standard samples to obtain the corresponding feature attributes of each sensitivity type. For each sensitive type, the core features of all sensitive samples and standard samples under that type are integrated, and a feature fusion algorithm is used to analyze the feature attributes. The extracted dimensions include data format features (such as length, character type, and verification rules), semantic features (such as business-related words and contextual references), and business attribute features (such as business process and related objects). These are then integrated to form a set of exclusive feature attributes for each sensitive type. The communication data in the communication log is compared with the feature attributes of the sensitive type. The communication data that matches the feature attributes is classified into the corresponding sensitive type. Conversely, when the communication data does not match the feature attributes of each sensitive type, the next communication data is compared until all communication data has been compared. ; in, Let d be the feature matching degree between the d-th communication data and the t-th sensitive type. This is the format feature weight, with a value of 0.5. This represents the semantic feature weight, with a value of 0.3. This represents the weight of the business attribute feature, with a value of 0.2. For format feature matching degree, For semantic feature matching degree, For business attribute feature matching degree; If the value is ≥0.8, then the communication data is determined to belong to the t-th sensitive type. Calculate the product of the number of communication data of the same sensitivity type and the corresponding weight coefficient, sort them from largest to smallest product value, and select the communication data with the highest product value as the labeled data. The formula is as follows: ; in, For the product result of the t-th sensitive type, Let be the number of communication data categorized under the t-th sensitivity type. Let be the weight coefficient for the t-th sensitivity type. The final selected marker data; The communication log is segmented into multiple data segments by using labeled data as nodes. The range of each data segment is defined as starting from the current node and ending at the next node, excluding the next node. The next node serves as the starting node of the new data segment. The process is repeated iteratively to complete the segmentation of the entire log.

[0024] S3. Set the corresponding accurate threshold according to the weight coefficient of each sensitive type. Within the same sensitive type and corresponding data segment, compare the data similarity between the communication data and the standard sample of that type. Combine the accurate threshold to determine the matching accuracy between the data and the sensitive type, and remove the sensitive type labels that do not match accurately. Through similarity comparison and threshold determination, remove the mislabeled data in the initial screening in S2 to ensure the accuracy of sensitive type determination. In S3, the corresponding accurate threshold is set according to the weight coefficient of each sensitivity type; This is achieved through linear function mapping. The minimum accurate threshold corresponding to the maximum weight coefficient is set, and the maximum accurate threshold corresponding to the minimum weight coefficient is set. For weight coefficients in the intermediate range, their corresponding accurate thresholds are calculated using a linear interpolation algorithm, thus completing the accurate thresholds for each sensitivity coefficient. The formula is as follows: ; in, The accurate threshold corresponding to the k-th sensitivity type. For the exact maximum threshold value for all sensitive types, The minimum accurate threshold value for all sensitive types. The weight coefficient for the k-th sensitivity type is... This represents the maximum weight coefficient for all sensitive types. This represents the minimum weight coefficient for all sensitive types.

[0025] Within the same sensitive type and corresponding data segment, semantic feature analysis is performed on communication data and standard samples to obtain the semantic features of communication data and standard samples. Then, the semantic features of communication data are combined with the semantic features of standard samples to perform feature fit analysis, and the data similarity is obtained based on the feature fit. The data similarity is compared with the accuracy threshold. When the data similarity is greater than the accuracy threshold, it is determined that the communication data matches the corresponding sensitive type accurately, and the sensitive type mark is retained. When the similarity value is less than the accuracy threshold, the match is deemed inaccurate, the sensitive type tag corresponding to the communication data is directly removed, and it is not included in the subsequent sensitive processing flow.

[0026] S4. Perform correlation analysis between accurately matched sensitive communication data and non-sensitive communication data, adjust the sensitivity type of non-sensitive communication data according to the correlation results, update the basic sensitive database by combining accurately matched sensitive data in each data segment, and perform sensitivity identification on subsequent communication data based on the updated database; expand the coverage of sensitive data (mining hidden related sensitive data), and achieve the self-learning and adaptive capabilities of the solution through iterative optimization of the database. In S4, the accurate matching communication data and the non-sensitive communication data are analyzed for contextual association, business logic association and data feature association to determine whether the non-sensitive communication data and the accurate matching sensitive communication data belong to the same communication session, the same business process and have a direct data dependency relationship. If the judgment results simultaneously meet the criteria of being in the same communication session, the same business process, and having a direct data dependency relationship, then the two are determined to be associated and matched, and the non-sensitive communication data is adjusted to the same sensitive type as the accurately matched sensitive communication data. Conversely, if there is no simultaneous fulfillment of the same communication session, the same business process, and a direct data dependency relationship, it is determined that the two are not related and the original type of the non-sensitive communication data remains unchanged.

[0027] The basic sensitive database is updated based on the correlation matching results and the accurately matched sensitive data. The steps are as follows: Supplement each data segment with accurately matched communication data and associated matching communication data as new sensitive samples; Delete sensitive samples in the original database that do not match the actual matching results (such as mislabeled non-sensitive samples or old samples with invalid features). Based on the number of newly added and deleted samples, update the sample proportion and corresponding weight coefficient of each sensitive type; For each sensitive type, the sample logs in the updated sample set are compared with the communication logs to be processed, and the sample with the highest similarity is selected as the standard sample for that type. Then, the updated basic sensitive database is invoked to perform a sensitive identification process on the newly generated communication logs.

[0028] S5. Configure dedicated de-identification rules for each sensitive data type, and call the corresponding rules to perform de-identification processing on the communication data. Perform differentiated de-identification operations for different sensitive data types to maximize data availability while ensuring data security. In S5, the communication management terminal sets differentiated and exclusive de-identification rules for each sensitive type based on its data characteristics, industry scenario requirements, and compliance requirements. Among them, the specific desensitization rules include desensitization methods, desensitization intensity, and data retention rules; The desensitization method is selected based on the characteristics of the sensitive type (such as masking, format preservation encryption, numerical perturbation, entity replacement, etc.). Desensitization intensity is divided according to sensitivity level (e.g., core sensitive types are set to high intensity, and ordinary sensitive types are set to medium and low intensity). Data retention rules should clearly define the core features that need to be retained after anonymization (such as format, check bits, and semantic relevance). Identify the sensitive types corresponding to each communication data, call the dedicated desensitization rules corresponding to that sensitive type, and complete the desensitization process.

[0029] The second objective of this invention is to provide an automatic identification and desensitization system for sensitive data in communication logs, including any of the above-mentioned methods for automatic identification and desensitization of sensitive data in communication logs, comprising a database establishment module, a data segmentation module, a data removal module, a sensitive identification module, and a desensitization processing module. The database establishment module is used to collect communication data from communication logs, perform semantic analysis on the communication data to determine the corresponding industry scenarios, retrieve sensitive samples based on industry scenarios, and build a basic sensitive database. The data segmentation module is used to classify sensitive samples in the basic sensitive database into sensitive types. It sets weight coefficients based on the sample proportion of each sensitive type, determines the corresponding standard samples for each sensitive type, filters communication data according to sensitive type, selects labeled data by combining the weight coefficients of sensitive types and the corresponding data quantity, and segments the communication log with the labeled data as nodes to obtain multiple data segments. The data removal module is used to set the corresponding accuracy threshold according to the weight coefficient of each sensitivity type. Within the same sensitivity type and corresponding data segment, the communication data is compared with the standard sample of that type. The accuracy threshold is combined to determine the matching accuracy between the data and the sensitivity type, and the sensitive type markers with inaccurate matching are removed. The sensitive identification module is used to perform correlation analysis between accurately matched sensitive communication data and non-sensitive communication data, adjust the sensitivity type of non-sensitive communication data according to the correlation results, update the basic sensitive database by combining accurately matched sensitive data in each data segment, and perform sensitive identification on subsequent communication data based on the updated database. The desensitization module is used to configure exclusive desensitization rules for each sensitive type and call the corresponding rules to perform desensitization processing on communication data.

[0030] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for automatic identification and desensitization of sensitive data in communication logs, characterized in that: Includes the following steps: S1. Collect communication data from communication logs, perform semantic analysis on the communication data to determine the corresponding industry scenarios, retrieve sensitive samples based on industry scenarios, and build a basic sensitive database; S2. Divide the sensitive samples in the basic sensitive database into sensitive types, set weight coefficients according to the sample proportion of each sensitive type, determine the corresponding standard samples for each sensitive type, filter communication data according to sensitive type, select labeled data by combining the weight coefficient of sensitive type and the corresponding data quantity, and segment the communication log with the labeled data as nodes to obtain multiple data segments. S3. Set the corresponding accuracy threshold according to the weight coefficient of each sensitivity type. Within the same sensitivity type and corresponding data segment, compare the data similarity between the communication data and the standard sample of that type. Combine the accuracy threshold to determine the matching accuracy between the data and the sensitivity type, and remove the sensitive type markers that do not match accurately. S4. Perform correlation analysis between the accurately matched sensitive communication data and the non-sensitive communication data, adjust the sensitivity type of the non-sensitive communication data according to the correlation results, and update the basic sensitive database by combining the accurately matched sensitive data in each data segment, and perform sensitivity identification on subsequent communication data based on the updated database. S5. Configure exclusive desensitization rules for each sensitive type, and call the corresponding rules to perform desensitization processing on the communication data.

2. The method for automatic identification and desensitization of sensitive data in communication logs according to claim 1, characterized in that: In step S1, a connection is established with the communication management terminal, and then the communication log and the communication data contained therein are collected at the communication management terminal. By analyzing the text content, semantic relationships, and business characteristics of communication data using natural language processing technology, key industry terms and business scenario identifiers are extracted, and the corresponding industry scenarios of communication logs are accurately located based on key terms and business scenario identifiers. We acquire sensitive samples that are highly compatible with industry scenarios through big data retrieval, and then integrate them to build a basic sensitive database.

3. The method for automatic identification and desensitization of sensitive data in communication logs according to claim 1, characterized in that: In S2, the sensitive samples in the basic sensitive database are classified into sensitive types. By analyzing the data characteristics and business attributes of each sensitive sample one by one, samples with the same characteristics and attributes are classified into the same sensitive type. The sample size corresponding to each sensitive type is combined with the total sample size for a percentage analysis. The higher the percentage of a sensitive type's sample size, the greater the weight coefficient assigned to that sensitive type. Conversely, the lower the percentage of a sensitive type's sample size, the smaller the weight coefficient assigned to that sensitive type. Extract the sensitive sample logs corresponding to each sensitive sample, perform log similarity analysis between the sensitive sample logs and the communication logs collected by S1, obtain the log similarity between each sensitive sample log and the communication log, and then select the sensitive sample with the highest log similarity as the standard sample for each sensitive type.

4. The method for automatic identification and desensitization of sensitive data in communication logs according to claim 3, characterized in that: In S2, feature attribute analysis is performed on each sensitivity type based on sensitive samples and standard samples to obtain the feature attributes corresponding to each sensitivity type. The communication data in the communication log is compared with the feature attributes of the sensitive type. The communication data that matches the feature attributes is classified into the corresponding sensitive type. Conversely, when the communication data does not match the feature attributes of each sensitive type, the next communication data is compared until all communication data has been compared. Calculate the product of the number of communication data of the same sensitivity type and the corresponding weight coefficient, sort them from largest to smallest according to the product value, and select the communication data with the first ranking as the labeled data; The communication log is segmented into multiple data segments by using labeled data as nodes. The range of each data segment is defined as starting from the current node and ending at the next node, excluding the next node. The next node serves as the starting node of the new data segment. The process is repeated iteratively to complete the segmentation of the entire log.

5. The method for automatic identification and desensitization of sensitive data in communication logs according to claim 1, characterized in that: In step S3, the corresponding accurate threshold is set according to the weight coefficient of each sensitivity type; The method is achieved through linear function mapping. The minimum accurate threshold corresponding to the maximum value of the weight coefficient is set, and the maximum accurate threshold corresponding to the minimum value of the weight coefficient is set. For the weight coefficient in the middle range, the corresponding accurate threshold is calculated by linear interpolation algorithm, thereby completing the accurate threshold corresponding to each sensitivity coefficient. Within the same sensitive type and corresponding data segment, semantic feature analysis is performed on communication data and standard samples to obtain the semantic features of communication data and standard samples. Then, the semantic features of communication data are combined with the semantic features of standard samples to perform feature fit analysis, and the data similarity is obtained based on the feature fit. The data similarity is compared with the accuracy threshold. When the data similarity is greater than the accuracy threshold, it is determined that the communication data matches the corresponding sensitive type accurately, and the sensitive type mark is retained. When the similarity value is less than the accuracy threshold, the match is deemed inaccurate, the sensitive type tag corresponding to the communication data is directly removed, and it is not included in the subsequent sensitive processing flow.

6. The method for automatic identification and desensitization of sensitive data in communication logs according to claim 1, characterized in that: In step S4, the accurately matched communication data and the non-sensitive communication data are analyzed for contextual association, business logic association, and data feature association to determine whether the non-sensitive communication data and the accurately matched sensitive communication data belong to the same communication session, the same business process, and have a direct data dependency relationship. If the judgment results simultaneously meet the criteria of being in the same communication session, the same business process, and having a direct data dependency relationship, then the two are determined to be associated and matched, and the non-sensitive communication data is adjusted to the same sensitive type as the accurately matched sensitive communication data. Conversely, if there is no simultaneous fulfillment of the same communication session, the same business process, and a direct data dependency relationship, it is determined that the two are not related and the original type of the non-sensitive communication data remains unchanged.

7. The method for automatic identification and desensitization of sensitive data in communication logs according to claim 6, characterized in that: The basic sensitive database is updated based on the association matching results and the accurately matched sensitive data, as follows: Supplement each data segment with accurately matched communication data and associated matching communication data as new sensitive samples; Delete sensitive samples from the original database that do not match the actual matching results; Based on the number of newly added and deleted samples, update the sample proportion and corresponding weight coefficient of each sensitive type; Re-screen the standard samples for each sensitive type; Then, the updated basic sensitive database is invoked to perform a sensitive identification process on the newly generated communication logs.

8. The method for automatic identification and desensitization of sensitive data in communication logs according to claim 2, characterized in that: In S5, a differentiated, exclusive desensitization rule is set for each sensitive type through the communication management terminal; Among them, the specific desensitization rules include desensitization methods, desensitization intensity, and data retention rules; Identify the sensitive types corresponding to each communication data, call the dedicated desensitization rules corresponding to that sensitive type, and complete the desensitization process.

9. An automatic identification and de-identification system for sensitive data in communication logs, used to implement the automatic identification and de-identification method for sensitive data in communication logs as described in any one of claims 1-8, characterized in that: It includes a database creation module, a data segmentation module, a data removal module, a sensitive data identification module, and a data anonymization module; The database establishment module is used to collect communication data from communication logs, perform semantic analysis on the communication data to determine the corresponding industry scenarios, retrieve sensitive samples based on the industry scenarios, and build a basic sensitive database. The data segmentation module is used to classify sensitive samples in the basic sensitive database into sensitive types, set weight coefficients according to the sample proportion of each sensitive type, determine the corresponding standard samples for each sensitive type, filter communication data according to sensitive type, select labeled data by combining the weight coefficient of sensitive type and the corresponding data quantity, and segment the communication log with the labeled data as nodes to obtain multiple data segments. The data removal module is used to set the corresponding accuracy threshold according to the weight coefficient of each sensitivity type, compare the communication data with the standard sample of the same sensitivity type within the same sensitivity type and the corresponding data segment, determine the matching accuracy between the data and the sensitivity type in combination with the accuracy threshold, and remove the sensitive type markers that do not match accurately. The sensitive identification module is used to perform correlation analysis between accurately matched sensitive communication data and non-sensitive communication data, adjust the sensitivity type of non-sensitive communication data according to the correlation results, update the basic sensitive database by combining accurately matched sensitive data in each data segment, and perform sensitive identification on subsequent communication data based on the updated database. The desensitization processing module is used to configure exclusive desensitization rules for each sensitive type and call the corresponding rules to perform desensitization processing on the communication data.

Images