Sensitive data identification method and device, electronic equipment, medium and program product

Abstract

The invention discloses a sensitive data identification method and device, electronic equipment, a medium and a program product, and relates to the technical field of data security, and the sensitive data identification method comprises the following steps: segmenting to-be-identified text data to obtain a plurality of short sentences; counting specific vocabularies not in a preset common word library in each short sentence, and recording a time residence matrix of the word frequency corresponding to the time residence matrix; performing word segmentation on each short sentence according to a preset word segmentation rule, and determining various word segmentation paths corresponding to each short sentence; and calculating a joint probability corresponding to each word segmentation path based on the word frequency of each time residence matrix, taking the word segmentation path with the highest joint probability as a target word segmentation path, and performing sensitive word retrieval on each word segmentation to obtain a sensitive data recognition result. According to the technical scheme, the problem that the accuracy of sensitive data recognition is affected due to the fact that errors are likely to occur when Chinese texts are processed through a traditional sensitive word matching algorithm is solved.

Description

Technical Field

[0001] The present application relates to the field of data security technology, and in particular to sensitive data identification methods, devices, electronic devices, computer-readable storage media and computer program products. Background Art

[0002] At present, the sensitive word matching algorithm is usually used for sensitive identification of text data. This algorithm matches words based on alphabetic languages. Because there are spaces between words, the meaning of each word is relatively accurate and standardized, so there is rarely any ambiguity. However, due to the characteristics of Chinese, there are no clear word boundaries. Multiple consecutive Chinese characters form more diverse semantics, and the intrinsic meaning of each Chinese character may be different in different words. For a single Chinese character, it may form a word with the previous character or with the following character. This feature makes the word segmentation of Chinese text data complicated, and incorrect word segmentation will cause the semantics of the sentence to deviate. Therefore, due to inaccurate word segmentation, the traditional sensitive word matching algorithm will make mistakes when processing Chinese text, affecting the accuracy of sensitive data identification.

[0003] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0004] The main purpose of this application is to provide a sensitive data identification method, device, electronic device, computer-readable storage medium and computer program product, aiming to solve the technical problem that traditional sensitive word matching algorithms are prone to errors when processing Chinese text, affecting the accuracy of sensitive data identification.

[0005] To achieve the above objectives, the present application proposes a sensitive data identification method, which includes:

[0006] Based on punctuation marks in the text data to be recognized, segmenting the text data to be recognized to obtain a plurality of short sentences;

[0007] Counting the specific words in each of the short sentences that are not in the preset common word library, and recording the word frequency of each of the specific words in the corresponding time residence matrix;

[0008] According to the preset word segmentation rules, each short sentence is segmented separately to determine multiple word segmentation paths corresponding to each short sentence;

[0009] Based on the word frequencies of each of the time residence matrices, the joint probabilities corresponding to each of the word segmentation paths are calculated, and the word segmentation path with the highest joint probability is used as the target word segmentation path;

[0010] All the segmented words of the text data to be identified are obtained according to the target segmented word path of each of the short sentences, and sensitive word retrieval is performed on each of the segmented words to obtain sensitive data identification results.

[0011] In one embodiment, the step of segmenting each short sentence according to a preset segmentation rule and determining a plurality of segmentation paths corresponding to each short sentence includes:

[0012] Based on the word segmentation rule, determining candidate attributes of each word in the short sentence, wherein the candidate attributes include one or more of the first word, the middle word, the last word, and the single word, and the word segmentation rule at least includes: the first word of the short sentence is the first word or the single word, and the last word of the short sentence is the last word or the single word;

[0013] According to the multiple alternative attributes corresponding to each character, the short sentence is divided into paths according to the word segmentation rules to obtain multiple word segmentation paths, wherein the word segmentation rules at least include: the next character after the first character or the middle character is the middle character or the last character, and the next character after the last character or the single character is the first character or the single character.

[0014] In one embodiment, the step of calculating the joint probabilities corresponding to the segmentation paths based on the word frequencies of the time residence matrices, and taking the segmentation path with the highest joint probability as the target segmentation path includes:

[0015] The word frequencies of the time-residence matrices and the joint probabilities corresponding to the word segmentation paths are respectively input into a preset hidden semi-Markov model to calculate the joint probabilities corresponding to each word segmentation path;

[0016] The word segmentation path with the highest joint probability among the word segmentation paths is selected to obtain the target word segmentation path.

[0017] In one embodiment, before the step of inputting the word frequencies of each of the time residence matrices and the joint probabilities corresponding to each of the word segmentation paths into a preset hidden semi-Markov model, the method further includes:

[0018] By searching each of the segmented words in a preset original word library, the meaningless segmented words that do not exist in the original word library are determined;

[0019] The segmentation paths including the meaningless segmentations in each segmentation path are deleted to obtain updated segmentation paths, and the step of inputting the word frequencies of each time-residence matrix and the joint probabilities corresponding to each segmentation path into a preset hidden semi-Markov model is performed based on the updated segmentation paths.

[0020] In one embodiment, the sensitive data identification result is a sensitivity level, and the steps of obtaining all the segmented words of the text data to be identified according to the target segmented word path of each of the short sentences, performing sensitive word retrieval on each of the segmented words, and obtaining the sensitive data identification result include:

[0021] Dividing each of the short sentences according to the target word segmentation paths corresponding to each of the short sentences to obtain a plurality of word segments corresponding to each of the short sentences;

[0022] Based on the preset sensitive word classification table, the sensitivity level of each word is determined in turn to obtain the sensitivity level corresponding to each word;

[0023] The highest sensitivity level among the sensitivity levels corresponding to the word segments is determined as the sensitivity level corresponding to the text data to be identified.

[0024] In one embodiment, the sensitive data identification method further includes:

[0025] Collect corresponding vocabulary data from multiple training material files to build an original vocabulary;

[0026] Based on the preset general material files and professional material files, counting the number of files containing each vocabulary and each word in the original word library;

[0027] Calculate the inverse text frequency index corresponding to each word or word according to the number of files containing each word and each word and the total number of the training material files;

[0028] Vocabulary or single words whose inverse text frequency index is less than a preset inverse text frequency index threshold are selected to form a preset common word library.

[0029] In addition, to achieve the above purpose, the present application also proposes a sensitive data identification device, the sensitive data identification device comprising:

[0030] A short sentence segmentation module, used for segmenting the text data to be identified based on punctuation marks in the text data to be identified, to obtain a plurality of short sentences;

[0031] A vocabulary statistics module, used for counting the specific words in each of the short sentences that are not in the preset common vocabulary, and recording the word frequency of each of the specific words in the corresponding time residence matrix;

[0032] The short sentence segmentation module is used to segment each short sentence according to the preset segmentation rules and determine the multiple segmentation paths corresponding to each short sentence;

[0033] A path selection module, used to calculate the joint probability corresponding to each of the segmentation paths based on the word frequency of each of the time residence matrices, and take the segmentation path with the highest joint probability as the target segmentation path;

[0034] The sensitive search module is used to obtain all the segmented words of the text data to be identified according to the target segmented word path of each short sentence, perform sensitive word search on each segmented word, and obtain sensitive data identification results.

[0035] In addition, to achieve the above-mentioned purpose, the present application also proposes an electronic device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the sensitive data identification method as described above.

[0036] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the sensitive data identification method described above are implemented.

[0037] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the sensitive data identification method described above.

[0038] The present application proposes a sensitive data identification method. First, based on the punctuation marks in the text data to be identified, the text data to be identified is segmented to obtain multiple short sentences. Then, specific words in each of the short sentences that are not in a preset common vocabulary are counted, and the word frequency of each of the specific words is recorded in a corresponding time residence matrix. Then, according to a preset word segmentation rule, each short sentence is segmented separately to determine a plurality of word segmentation paths corresponding to each short sentence. Then, based on the word frequency of each of the time residence matrices, the joint probability corresponding to each of the word segmentation paths is calculated, and the word segmentation path with the highest joint probability is used as the target word segmentation path. Finally, according to the target word segmentation path of each of the short sentences, all the word segments of the text data to be identified are obtained, and sensitive word retrieval is performed on each of the word segments to obtain a sensitive data identification result. The technical solution of the present application proposes a method for screening specific words based on a preset common vocabulary, and performing joint probability calculation based on the word frequency of the specific words recorded in the time residence matrix combined with multiple word segmentation paths determined by preset word segmentation rules. Compared with the traditional word segmentation method, the frequency influence of specific words in the text is taken into account, and the optimal target word segmentation path that is relatively more in line with Chinese vocabulary habits can be obtained, and the semantic deviation of the text to be identified caused by ambiguity caused by random word segmentation is minimized. Therefore, sensitive data identification is performed after word segmentation through the technical solution of the present application, which substantially improves the accuracy of sensitive data identification by improving the word segmentation accuracy of sensitive words. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0040] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0041] Figure 1 A flowchart of the first embodiment of the sensitive data identification method of the present application is provided;

[0042] Figure 2 Schematic diagram of semantic sequence and text sequence in the embodiment of the present application;

[0043] Figure 3 A schematic diagram of allocating optional attributes to each word in a short sentence based on a word segmentation rule according to an embodiment of the present application;

[0044] Figure 4 This is a schematic diagram of the structure of a sensitive data identification device in an embodiment of the present application;

[0045] Figure 5 Schematic diagram of the device structure of the hardware operating environment involved in the sensitive data identification method in the embodiment of the present application.

[0046] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0047] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0048] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0049] The execution subject of this embodiment may be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, a server, etc., or an electronic device or a control device capable of realizing the above functions. The following takes the server as an execution subject as an example to illustrate this embodiment and the following embodiments.

[0050] Existing sensitive word matching algorithms are all based on matching words in alphabetic languages. Since there are spaces between words, the meaning of each word is relatively accurate and standardized, and there is generally little ambiguity. Due to the characteristics of Chinese, there are no clear word boundaries, but multiple consecutive Chinese characters form a variety of semantics, and the intrinsic meaning of each Chinese character in different words is not exactly the same. A character may form a word with the previous character or the following character. This feature makes word segmentation complicated, and traditional sensitive word matching algorithms may have errors when processing Chinese. Taking the judgment of the degree of sensitivity in the files of the telecommunications industry support system as an example, "number" is a typical personal information and has a high sensitivity mark in the sensitive word library. When processing the following sentence "On the third terminal of Port A", the usual full-text search method will determine that the sentence contains the word "number". But in fact, the semantics of the sentence does not contain "number", so according to the existing sensitive word search technology, the information will be misjudged as sensitive, and there is a defect of low accuracy in sensitive data recognition.

[0051] In order to overcome the technical defects existing in the above-mentioned sensitive word matching algorithm, the embodiment of the present application provides a sensitive data identification method. Taking into account the negative impact of unreasonable word segmentation on sensitive word matching, the technical solution of the embodiment of the present application proposes an improved word segmentation method based on Chinese semantics and context. Taking into account that there may be multiple word segmentations for the same Chinese sentence, specific words are screened through a preset common word library, and the probability of each word segmentation path conforming to the Chinese context is calculated in combination with the word frequency of specific words, and then the most reasonable word segmentation path is inferred, thereby effectively reducing the errors generated in the word segmentation process.

[0052] Reference Figure 1 , Figure 1 This is a flowchart of the first embodiment of the sensitive data identification method of the present application, and the sensitive data identification method includes:

[0053] Step S10, segmenting the text data to be recognized based on punctuation marks in the text data to be recognized to obtain a plurality of short sentences;

[0054] First, the text data to be identified that needs to be used for sensitive data identification is obtained. The text data to be identified can be directly obtained or converted from other forms of data, such as extracted from audio data or video data. When the text data to be identified has punctuation marks, the entire text data to be identified can be divided into multiple short sentences based on the functions of various punctuation marks (such as commas, periods, and semicolons) during preliminary sentence segmentation for further processing.

[0055] Step S20, counting the specific words in each short sentence that are not in the preset common word library, and recording the word frequency of each specific word in the corresponding time residence matrix;

[0056] Among them, the preset common vocabulary is a pre-established vocabulary consisting of non-specific words and common words. Non-specific words and common words refer to words that are widely used in various fields and scenarios, such as pronouns (I, you, he, etc.), prepositions (and, with, on, etc.), and daily expressions (hello, thank you, goodbye, etc.). These types of words are generally not closely related to sensitive words and have little effect on the understanding of semantics. Therefore, for non-specific words and common words in the preset common vocabulary in short sentences, no further processing is required. For specific words that are not in the preset common vocabulary, their frequency of occurrence in the text data to be identified is counted, and the corresponding word frequency is recorded in its corresponding time residence matrix to facilitate the determination of the possibility of various specific words.

[0057] It is understandable that if the frequency of a specific word is higher, the possibility that the specific word is the real meaning to be expressed by the short sentence is higher, and the accuracy of the word segmentation method containing the specific word is higher. In the embodiment of the present application, by recording the frequency of each specific word, a reliable data reference benchmark is provided for the short sentence word segmentation.

[0058] Step S30, segmenting each short sentence according to a preset segmentation rule, and determining multiple segmentation paths corresponding to each short sentence;

[0059] It is understandable that when segmenting a short sentence, certain rules need to be followed so that the segmentation and segmentation path obtained are logical. For example, when segmenting a short sentence, the attributes of each word in the short sentence can be analyzed in detail. For example, the attributes of each word can be the first word (referring to the first word of the vocabulary), the middle word (referring to the middle word of the vocabulary), the last word (referring to the last word of the vocabulary), and the single word (referring to the vocabulary formed by a single word). The segmentation rule can stipulate that each word needs to follow the provisions of the first word, the middle word (can be one or more), and the last word when forming a vocabulary to ensure the smooth progress of the segmentation.

[0060] In the specific word segmentation process, we can analyze all possible word segmentation paths corresponding to each short sentence based on the word segmentation rules. These word segmentation paths are all possible word segmentation results of the short sentence. Each word segmentation path corresponds to different semantics. We need to further determine which word segmentation path is more likely and can better express the true semantics of the short sentence.

[0061] Step S40, based on the word frequencies of each time residence matrix, calculate the joint probabilities corresponding to each word segmentation path, and use the word segmentation path with the highest joint probability as the target word segmentation path;

[0062] The time residence matrix includes the word frequencies corresponding to various specific words, and the word frequencies of these specific words respectively affect the size of the joint probability of various word segmentation paths.

[0063] Specifically, for a single short sentence, it has multiple different segmentations in corresponding segmentation paths. If the segmentation of each segmentation path includes a specific word, its word frequency in the time residence matrix can be determined. The higher the word frequency, the more likely the specific word is to be the true meaning of the short sentence. The higher the accuracy of the specific word, the higher the joint probability of the segmentation path corresponding to the specific word.

[0064] Finally, the target word segmentation path with the highest joint probability is used to represent the word segmentation path that is closest to the real semantics that the text wants to express among the multiple word segmentation paths corresponding to each short sentence.

[0065] In the embodiment of the present application, when selecting the word segmentation path, the influence of the word frequency of specific words in the full text of the text data to be identified is taken into account, which further improves the accuracy of word segmentation and makes the final target word segmentation path more accurate, providing an accurate text basis for sensitive data identification.

[0066] Step S50, obtaining all the segmented words of the text data to be recognized according to the target segmented word path of each short sentence, performing sensitive word search on each segmented word respectively, and obtaining a sensitive data recognition result.

[0067] After determining the target segmentation path for each short sentence in the text data to be identified, all the segmentations of the text data to be identified can be obtained according to the target segmentation path corresponding to each short sentence, and then the sensitivity of each segmentation can be judged in turn according to the pre-defined sensitive word identification detection rules to obtain the sensitive data identification result.

[0068] The sensitive data identification results may include the sensitive words in the text data to be identified, such as which sensitive words are included and the distribution of sensitive words. If the sensitivity level of sensitive words is predefined, the sensitivity level of each word in the text data to be identified can be further obtained, and a more detailed sensitive data analysis report can be generated.

[0069] In another feasible embodiment, after obtaining the sensitive data identification results, the embodiment of the present application can also define a protection rule library for different stages of the data life cycle. After completing the data classification and grading according to the sensitive types, the corresponding security protection measures are automatically completed according to the definition of the protection rule library.

[0070] Exemplarily, the sensitive data life cycle in the embodiment of the present application may include: data collection, identifying the accessed data source to prevent data source counterfeiting. Logging and security auditing of the collection behavior. Temporary storage files and cache areas used in the collection process are included in the unified security management scope; data transmission, control the data transmission between systems, use encryption or encrypted channels to transmit data, use reliable identity authentication measures for both parties of the transmission, and prevent tampering during data transmission; data storage, provide data integrity protection and identity authentication mechanism, record data storage and access, restrict external mobile storage media, and perform the same security management measures on the backup data; data use, assign permissions according to the principle of minimization of permissions, log and approve the access and operation of data. Provide dedicated monitoring solutions and complete emergency plans to improve the handling ability and response speed of dangerous behaviors; data sharing requires user authorization, authentication and logging of access to external interfaces, use a secure sharing method to prevent unauthorized access, and adopt technical means such as digital watermarks to ensure that shared data can be traced. Continuously monitor the security situation of shared data; for data destruction, establish a data destruction management process, conduct effect verification and physical destruction to ensure that the data is completely deleted and cannot be recovered.

[0071] The embodiment of the present application can automatically complete the above-mentioned corresponding security protection measures according to the data classification and grading judgment results and the definition of the protection rule library, avoid human intervention in the process, and greatly reduce the possibility of data leakage caused by human intention or negligence.

[0072] Further, in a feasible embodiment, the step of segmenting each short sentence according to a preset segmentation rule and determining a plurality of segmentation paths corresponding to each short sentence includes:

[0073] Step S31, based on the word segmentation rule, determining the candidate attributes of each word in the short sentence, wherein the candidate attributes include one or more of the first word, the middle word, the last word and the single word, and the word segmentation rule at least includes: the first word of the short sentence is the first word or the single word, and the last word of the short sentence is the last word or the single word;

[0074] Step S32, according to the multiple alternative attributes corresponding to each character, the short sentence is divided into paths according to the word segmentation rules to obtain multiple word segmentation paths, wherein the word segmentation rules at least include: the next character after the first character or the middle character is the middle character or the last character, and the next character after the last character or the single character is the first character or the single character.

[0075] It is understandable that for each word in a short sentence, there are four possible alternative attributes for word segmentation: the beginning of a word, that is, the first word, can be expressed by B (Begin); the middle part of a word (a word with more than 3 words), that is, the middle word, can be expressed by M (Middle); the end of a word, that is, the last word, can be expressed by E (End); a word that is not part of a word but a single word, that is, a single word, can be expressed by S (Separate). In the embodiment of the present application, word segmentation is performed by combining a preset common word library and probability calculation to obtain a variety of possible word segmentation paths corresponding to each short sentence.

[0076] It should be noted that, in accordance with the preset word segmentation rules, each word must be complete, so B or M can only be followed by M or E (otherwise the word does not end); E or S can only be followed by B or S (otherwise the word does not begin).

[0077] On the basis of the above rules, each short sentence is segmented respectively, and all possible segmentation paths corresponding to each short sentence are listed.

[0078] In a feasible embodiment, the step of calculating the joint probabilities corresponding to the word segmentation paths based on the word frequencies of the time residence matrices, and taking the word segmentation path with the highest joint probability as the target word segmentation path includes:

[0079] Step S31, inputting the word frequency of each time residence matrix and the joint probability corresponding to each word segmentation path into a preset hidden semi-Markov model, and calculating the joint probability corresponding to each word segmentation path;

[0080] Step S32, selecting the segmentation path with the highest joint probability among the segmentation paths to obtain the target segmentation path.

[0081] In the embodiment of the present application, the idea of ​​hidden Markov model is combined to calculate the joint probability of each segmentation path. Generally, the semantics of natural language conforms to the hidden Markov model. Markov model is a discrete time and discrete state random process with Markov property in mathematics. In this process, when the current state is known, the predicted future state has nothing to do with the past state and is only related to the current state. Suppose a random process {X n , n=0,1,2,…}, if it takes a finite number of values ​​(called the state of the process, denoted as 0,1,2,…), for any n>0 and state i,j,i 0 ,i 1 ,…,i n-1 There is P(X n+1 =j|X 0 =i 0 , X 1=i 1 , …, X n-1 =i n-1 , X n =i n )=P(X n+1 =j|X n =i), then this process is called a Markov process. n The Markov chain is called a Markov chain. The Markov model is generally written as: λ = (π, A, B), where π is the initial probability distribution vector, A is the state transition probability matrix, and B is the observation value probability matrix.

[0082] Since the content semantics of the short sentence in the embodiment of the present application is y n Unobservable (hidden), observable is the text expression X n , so it is generally believed that the natural language random variable y n Satisfies the Markov property, that is, the state of y at time n is only related to the value of its state at the previous time (y n-1 ), and the state of the observable sequence x at time n is only determined by the hidden state yn at time n. Under this assumption, HMM (Hidden Markov Model) can infer the content semantics y by considering the calculation of the joint probability P(y,x) n actual distribution.

[0083] The Markov property of natural language is an ideal situation. In fact, if we observe each sentence as an independent entity and fail to consider the context, it may lead to errors in semantic understanding. For example, if we follow the Markov property, the understanding of "the tennis auction is over" has nothing to do with the previous text. It may be understood as "tennis / auction / over" or "tennis rackets / sold out". In fact, we need to observe the context based on the content (context) before the sentence and analyze the scene in which the sentence is located. If the words "tennis" and "auction" are mentioned many times in the previous text, it is considered that the semantic scene is located at the auction site and it should be understood as "tennis / auction / over"; if the word "racket" is mentioned many times in the previous text, it is considered that the semantic scene is to buy rackets and it should be understood as "tennis rackets / sold out".

[0084] Therefore, the embodiment of the present application needs to be expanded on the basis of the hidden Markov model to determine the value of the state of y at time n and the state at the previous time (y n-1 ) and is also affected by the value of the state at a finite number of earlier times (y n-i (i=2,3,4,…,k)) and the state of the observable sequence x at time n is determined by the hidden state y at time n. nThis model is also called Hidden Semi-Markov Model (HSMM). HSMM can be expressed as λ = (π, A, B, P), where π, A, and B have the same meaning as in the conventional HMM model, and P is the probability distribution describing the state residence time. Unlike the conventional HMM where one state (semantics) corresponds to one observation symbol (text), in HSMM, one state (semantics) corresponds to a segment of observation symbols (text). Based on HSMM, taking into account the actual situation of Chinese semantics, the inverse text index can be introduced as the probability distribution of the state residence time, and the frequency of specific words is recorded in the time residence matrix to serve as the basis for calculating the joint probability of the hidden state yn.

[0085] For example, refer to Figure 2 The semantic sequence and text sequence shown, where y is a word segmentation sequence, including y 1 To 5 ; x is a literal sequence, including x 1 to x 5 , that is, the word is the beginning (B), middle (M), end (E) or single word (S) of a word. Each word segmentation sequence is reflected as text and is affected by the previous finite number of word segments. Each y shown in the figure i Only affected by i-3 ,y i-2 ,y i-1 The influence of is for the convenience of display. In the actual calculation process, the actual influence step size can far exceed the influence step size shown in the figure.

[0086] Specifically, the mathematical expression for calculating the joint probability of each segmentation path is:

[0087]

[0088] Among them, P(x 1 ,y 1 ,x 2 ,y 2 ……x n ,y n ) refers to the word segmentation path y 1 To n The joint probability of .

[0089] In another feasible embodiment, O can be defined as (O 1 ,O 2 ,O 3 ,…O n ) represents the output result, O i = {B, M, E, S}, λ = (λ 1 ,λ 2 ,λ 3,…λ n ) represents the input, i.e. λ i = Each word in the input. Find the probability of each word segmentation in this sentence, that is, find P(O|λ)=P(O 1 ,O 2 ,O 3 ,…O n |λ 1 ,λ 2 ,λ 3 ,…λ n ).

[0090] According to the independence principle of Hidden Markov, the possible output of each word is only related to this word, that is: P(O 1 ,…,O n |λ 1 ,…,λ n )=∏P(O k |λ k ), where k = 1,...,n.

[0091] Finally, according to Bayes' theorem, the mathematical expression of the joint probability corresponding to the output result is calculated as:

[0092]

[0093] Among them, P(O|λ) is the joint probability of output result O when input λ, P(λ,O) is the probability of output O when input λ, P(λ) is the probability of input λ, P(O) is the total probability of output O, and P(λO) is the probability of input λ when output result O.

[0094] In the embodiment of the present application, by calculating the specific vocabulary in the non-preset common vocabulary library matched between B and E (or S) under each path corresponding to each short sentence, and the word frequency of the specific vocabulary, a corresponding probability score is given, and the probability score is adjusted according to the above calculation results to adjust the inverse text word frequency count matrix (high-frequency specific vocabulary) P. Then, by calculating the probabilities of all paths, the path with the largest joint probability is the target word segmentation path, and the semantics corresponding to the target word segmentation path is the true semantics of the short sentence that best conforms to the Chinese vocabulary habits.

[0095] In a feasible embodiment, before the step of inputting the word frequencies of each of the time residence matrices and the joint probabilities corresponding to each of the word segmentation paths into a preset hidden semi-Markov model, the method further includes:

[0096] Step A10, by searching each segmented word in a preset original word library, determining meaningless segmented words that do not exist in the original word library;

[0097] Step A20: Delete the word segmentation paths that include meaningless word segments in each word segmentation path to obtain updated word segmentation paths, and perform the step of inputting the word frequencies of each time residence matrix and the joint probabilities corresponding to each of the word segmentation paths into a preset implicit semi-Markov model based on the updated word segmentation paths.

[0098] In the process of word segmentation, the situation of meaningless word segments needs to be considered. The original word library described in Step A10 is a word library formed in advance by collecting a large amount of data. The data involved includes materials such as "Ci Hai" and "Modern Chinese Dictionary", and professional vocabulary in the fields applied for sensitive data identification (such as telecommunications support, security, etc. in the communication field) is added. Finally, a word library is formed. It can be understood that the words in the original word library are all meaningful words. If the formed word segments cannot be found in the original word library, it means that such words belong to meaningless word segments.

[0099] For example, for the short sentence "On the third pier of Port A in Area A", after word segmentation, words such as "Port A" and "Pier 3" may be obtained. These words do not exist in the original word library and are regarded as meaningless word segments.

[0100] After determining the meaningless word segments, the formed word segmentation paths can be screened. If a word segmentation path includes meaningless word segments, this path can be ignored without calculating the joint probability of this path, saving computing resources and improving processing efficiency.

[0101] For the convenience of understanding, as an example, taking the short sentence "On the third pier of Port A in Area A" as an example, the word segmentation method in the embodiment of the present application will be described in detail.

[0102] First, a short sentence can only start with B (the first character) or S (a single character) and can only end with E (the last character) or S (a single character). The characters in the middle follow the rule that after B or M, it can only be M or E; after E or S, it can only be B or S. List all possible word segmentation paths, such as Figure 3 As shown, different colors represent different word segmentation paths.

[0103] Perform word frequency statistics on the rare words that appear in the text data. It is very likely that words such as "Area A" and "pier" will appear multiple times in this article. These two words are not in the preset common word library. Statistically calculate the word frequencies of these two words and fill the corresponding time residence matrices.

[0104] Define O=(O 1 ,O 2 ,O 3 ,…O 8 ) to represent the output result, O n ={B,M,E,S}, λ=(λ 1 ,λ 2, λ 3 , … λ 8 ) represents the input, i.e., λ 1 = "A", λ 2 = "B", … λ 8 = "C". Calculate the probability of each word segmentation path for this sentence, i.e., find P(O|λ) = P(O 1 , O 2 , O 3 , … O 8 |λ 1 , λ 2 , λ 3 , … λ 8 ).

[0105] According to the joint probability calculation formula of the hidden Markov model, calculate the joint probability of each word segmentation path

[0106] Select the one with the largest joint probability as the most appropriate word segmentation path ( Figure 3 the red line in).

[0107] Among them, "A Port, No. 3, Dock, C" ( Figure 3 the red line in) and "A Port, 3, Number, Head" ( Figure 3 the green line in) both contain three words from the original word library and a single character. Other word segmentation methods ( Figure 3 the blue line in) must contain more single characters or non-existent words. In the word segmentation combination of "A Port, No. 3, Dock, C", "C" is a locative preposition and has a relatively high probability as a single character. In the word segmentation combination of "A Port, 3, Number, Head", "3" is a number and has a relatively low probability as a single character. According to the time residence matrix, "Dock" has appeared in the previous text, so the word frequency in the time residence matrix is relatively high. According to the Markov model, although the common usage degree (probability) of "Number" and "Head" exceeds that of a word like "No. 3", the common usage degree (probability) of "3" as a single character is relatively low, and "Dock" as an uncommon word has appeared in this text data, resulting in the joint probability of "A Port, No. 3, Dock, C" being greater than that of "A Port, 3, Number, Head". Finally, "A Port, No. 3, Dock, C" is selected as the target word segmentation path and it is determined that no sensitive words are included

[0108] The target word segmentation path of the final word segmentation is: BME / BE / BE / S, corresponding to the text: A Port, No. 3, Dock, C. According to the above target word segmentation path, this short sentence does not contain sensitive words such as "Number", and will not be determined as sensitive data. Compared with the traditional method of sensitive word matching and retrieval, the technical solution of this embodiment of the present application greatly reduces the misjudgment rate of sensitive data recognition

[0109] In a feasible embodiment, the sensitive data identification result is a sensitivity level, and the steps of obtaining all the segmented words of the text data to be identified according to the target segmented word path of each of the short sentences, performing sensitive word retrieval on each of the segmented words, and obtaining the sensitive data identification result include:

[0110] Step S51, dividing each short sentence according to the target word segmentation path corresponding to each short sentence, and obtaining a plurality of word segments corresponding to each short sentence;

[0111] Step S52: Based on the preset sensitive word classification table, the sensitivity level of each word is determined in turn to obtain the sensitivity level corresponding to each word;

[0112] Step S53: determine the highest sensitivity level among the sensitivity levels corresponding to the word segments as the sensitivity level corresponding to the text data to be identified.

[0113] After determining the target segmentation path for each short sentence, one or more segmentations corresponding to each short sentence are determined according to the segmentation method in the target segmentation path. In this case, the segmentations obtained are closest to the semantics to be expressed by the original text data to be recognized, and can bring higher accuracy in data recognition.

[0114] For example, the sensitive word classification table in the embodiment of the present application can divide the sensitivity level into 4 levels, and each category, subcategory and range are shown in the following table:

[0115]

[0116]

[0117] Natural speech processing technology can be used to identify the part of speech and understand the meaning of each segmented word to determine the data category to which each segmented word belongs, and then combined with the above table to determine the sensitivity level corresponding to each segmented word.

[0118] Finally, the sensitivity level of the word with the highest sensitivity level among all the word segments is used as the sensitivity level of the entire text data to be identified, completing the entire process of sensitive data identification. It should be noted that the sensitivity level of the text data to be identified can be used as a basis for subsequent sensitive data processing, and text data of different sensitivity levels are processed differently.

[0119] In another feasible embodiment, the sensitive data identification method further includes:

[0120] Step B10, collecting corresponding vocabulary data from multiple training material files to construct an original vocabulary;

[0121] Step B20, based on the preset general material files and professional material files, counting the number of files containing each vocabulary and each single word in the original word library;

[0122] Step B30, calculating the inverse text frequency index corresponding to each word or word according to the number of files containing each word and each word and the total number of training material files;

[0123] Step B30, selecting words or single characters whose inverse text frequency index is less than a preset inverse text frequency index threshold to form a preset common word library.

[0124] The embodiment of the present application also discloses a method for constructing an original vocabulary and a preset common vocabulary. Specifically, first construct an original vocabulary with a large amount of data, that is, an IDF (Inverse Document Frequency) vocabulary, extract vocabulary from material files such as "Cihai" and "Modern Chinese Dictionary", and add professional vocabulary from training material files in the fields of telecommunications support, security, etc. to form an original vocabulary. Exemplarily, the vocabulary used in the fields of telecommunications support, security, etc. is combined with the vocabulary in material files such as "Cihai" and "Modern Chinese Dictionary" to obtain an original vocabulary with a vocabulary of about 400,000. Then, for a large number of general material files (files with no obvious field tendency) and professional material files (files in professional fields such as telecommunications support and security), the frequency of occurrence of various words or words in the vocabulary is counted to determine the number of files corresponding to each word or word. Among them, the number of files corresponding to a vocabulary or word refers to the number of times the vocabulary or word appears in all files. If it appears once or multiple times in a file, it is counted as 1. For example, the word "port" appears twice in file A, five times in file B, and once in file C. Then the number of files corresponding to the word "port" (files A, B, C) is 3.

[0125] The IDF index of each word is calculated using the IDF calculation formula. Specifically, the IDF calculation formula is as follows:

[0126]

[0127] The total number of training files refers to the total number of general material files and professional material files, and the number of specific vocabulary files refers to the number of files corresponding to each vocabulary or word.

[0128] As an example, in the embodiment of the present application, words with an IDF index lower than 1.5 are selected as non-specific common words (which can also be understood as general common words) to form a preset common word library, that is, the preset inverse text frequency index threshold is 1.5. It should be noted that when IDF = 1.5 ≈ lg32, it means that the word appears in at least 3.1% of the data files in the full training file, then the word is a non-specific common word. In actual application scenarios, the IDF index threshold can be adjusted between 0.7 (the word appears in at least 20% of the data files) and 2.3 (the word appears in at least 0.5% of the database files) according to the universality or specificity of the sensitive data identification system.

[0129] The technical solution of the embodiment of the present application is equivalent to proposing a strategy for realizing Chinese sensitive data identification through an implicit semi-Markov model, which specifically includes constructing a general + telecommunications security-specific original vocabulary, and counting the frequency of occurrence of each word and word in the vocabulary through historical material files, and forming an IDF inverse text vocabulary (i.e., a preset general vocabulary). When performing word segmentation, the probability that each word in the text data to be identified is the first word (B), middle word (M), tail word (E) or single word (S) of the word is calculated, so as to infer the most reasonable word segmentation path, and finally perform sensitive word matching, and finally confirm the method of sensitive information. Moreover, in the implicit semi-Markov model, by introducing the IDF inverse text vocabulary occurrence probability as a parameter in the time resident matrix, the method of effectively segmenting and selecting ambiguous word segments for the current text content is avoided, so as to avoid the semantic understanding of text data due to errors in word segmentation. The technical solution of the embodiment of the present application can accurately segment words according to complete sentences, and correctly understand the part of speech and meaning of each word, so as to ensure the accuracy of sensitive word recognition. After completing the identification of sensitive information data, the sensitive information content is classified and graded, and complete full life cycle protection requirements and means are given to ensure the compliance of data management and eliminate the possibility of sensitive data getting out of control.

[0130] In practice, using the text data to be identified that has been historically accumulated in the telecommunications safety production system, the traditional full-word matching algorithm is used, and the confidence level of sensitive word detection (actual sensitive words detected / all detected words) is only 93.8%, with an error rate of up to 6.2%. However, using the sensitive data identification method of the embodiment of the present application, the confidence level of sensitive word detection can reach 99.7%, with an error rate of only 0.3%. This shows that the sensitive data identification method of the embodiment of the present application has a significant effect on improving the accuracy of sensitive data identification.

[0131] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the sensitive data identification method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0132] The present application also provides a sensitive data identification device, referring to Figure 4 , the sensitive data identification device comprises:

[0133] A short sentence segmentation module 10 is used to segment the text data to be identified based on punctuation marks in the text data to be identified, thereby obtaining a plurality of short sentences;

[0134] A vocabulary statistics module 20, for counting the specific words in each of the short sentences that are not in the preset common vocabulary, and recording the word frequency of each of the specific words in a corresponding time residence matrix;

[0135] The short sentence segmentation module 30 is used to segment each short sentence according to a preset segmentation rule and determine multiple segmentation paths corresponding to each short sentence;

[0136] A path selection module 40 is used to calculate the joint probability corresponding to each of the word segmentation paths based on the word frequency of each of the time residence matrices, and take the word segmentation path with the highest joint probability as the target word segmentation path;

[0137] The sensitive search module 50 is used to obtain all the segmented words of the text data to be identified according to the target segmented word path of each of the short sentences, and perform sensitive word search on each of the segmented words to obtain sensitive data identification results.

[0138] In one embodiment, the short sentence segmentation module 30 is further used for:

[0139] Based on the word segmentation rule, determining candidate attributes of each word in the short sentence, wherein the candidate attributes include one or more of the first word, the middle word, the last word, and the single word, and the word segmentation rule at least includes: the first word of the short sentence is the first word or the single word, and the last word of the short sentence is the last word or the single word;

[0140] According to the multiple alternative attributes corresponding to each character, the short sentence is divided into paths according to the word segmentation rules to obtain multiple word segmentation paths, wherein the word segmentation rules at least include: the next character after the first character or the middle character is the middle character or the last character, and the next character after the last character or the single character is the first character or the single character.

[0141] In one embodiment, the path selection module 40 is further used for:

[0142] The word frequencies of the time-residence matrices and the joint probabilities corresponding to the word segmentation paths are respectively input into a preset hidden semi-Markov model to calculate the joint probabilities corresponding to each word segmentation path;

[0143] The word segmentation path with the highest joint probability among the word segmentation paths is selected to obtain the target word segmentation path.

[0144] In one embodiment, the path selection module 40 is further used for:

[0145] By searching each of the segmented words in a preset original word library, the meaningless segmented words that do not exist in the original word library are determined;

[0146] The segmentation paths including the meaningless segmentations in each segmentation path are deleted to obtain updated segmentation paths, and the step of inputting the word frequencies of each time-residence matrix and the joint probabilities corresponding to each segmentation path into a preset hidden semi-Markov model is performed based on the updated segmentation paths.

[0147] In one embodiment, the sensitive data identification result is a sensitivity level, and the sensitive retrieval module 50 is further used to:

[0148] Dividing each of the short sentences according to the target word segmentation paths corresponding to each of the short sentences to obtain a plurality of word segments corresponding to each of the short sentences;

[0149] Based on the preset sensitive word classification table, the sensitivity level of each word is determined in turn to obtain the sensitivity level corresponding to each word;

[0150] The highest sensitivity level among the sensitivity levels corresponding to the word segments is determined as the sensitivity level corresponding to the text data to be identified.

[0151] In one embodiment, the sensitive data identification device further includes a word library construction module, and the word library construction module is used to:

[0152] Collect corresponding vocabulary data from multiple training material files to build an original vocabulary;

[0153] Based on the preset general material files and professional material files, counting the number of files containing each vocabulary and each word in the original word library;

[0154] Calculate the inverse text frequency index corresponding to each word or word according to the number of files containing each word and each word and the total number of the training material files;

[0155] Vocabulary or single words whose inverse text frequency index is less than a preset inverse text frequency index threshold are selected to form a preset common word library.

[0156] The sensitive data identification device provided by the present application adopts the sensitive data identification method in the above embodiment, which can solve the technical problem that the traditional sensitive word matching algorithm is prone to errors when processing Chinese text, affecting the accuracy of sensitive data identification. Compared with the prior art, the beneficial effects of the sensitive data identification device provided by the present application are the same as the beneficial effects of the sensitive data identification method provided by the above embodiment, and the other technical features in the sensitive data identification device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0157] The present application provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the sensitive data identification method in the above-mentioned embodiment one.

[0158] Reference below Figure 5 , which shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present application. The electronic devices in the embodiments of the present application may include but are not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0159] like Figure 5 As shown, the electronic device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the electronic device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows an electronic device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0160] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0161] The electronic device provided by the present application adopts the sensitive data identification method in the above embodiment, which can solve the technical problem that the traditional sensitive word matching algorithm is prone to errors when processing Chinese text, affecting the accuracy of sensitive data identification. Compared with the prior art, the beneficial effects of the electronic device provided by the present application are the same as the beneficial effects of the sensitive data identification method provided by the above embodiment, and the other technical features in the electronic device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0162] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0163] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0164] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the sensitive data identification method in the above-mentioned embodiment.

[0165] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0166] The computer-readable storage medium may be included in the electronic device, or may exist independently without being installed in the electronic device.

[0167] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by an electronic device, the electronic device: based on punctuation marks in the text data to be identified, segment the text data to be identified to obtain multiple short sentences; count the specific words in each of the short sentences that are not in a preset common vocabulary, and record the word frequency of each of the specific words in a corresponding time residence matrix; according to a preset word segmentation rule, segment each short sentence separately, and determine a plurality of word segmentation paths corresponding to each short sentence; based on the word frequency of each of the time residence matrices, calculate the joint probability corresponding to each of the word segmentation paths, and use the word segmentation path with the highest joint probability as the target word segmentation path; obtain all the word segments of the text data to be identified according to the target word segmentation path of each of the short sentences, perform sensitive word retrieval on each of the word segments, and obtain sensitive data recognition results.

[0168] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0169] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0170] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0171] The readable storage medium provided in the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned sensitive data identification method, and can solve the technical problem that the traditional sensitive word matching algorithm is prone to errors when processing Chinese text, affecting the accuracy of sensitive data identification. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the sensitive data identification method provided in the above-mentioned embodiment, and will not be repeated here.

[0172] The present application also provides a computer program product, including a computer program, which implements the steps of the sensitive data identification method as described above when executed by a processor.

[0173] The computer program product provided by the present application can solve the technical problem that the traditional sensitive word matching algorithm is prone to errors when processing Chinese text, which affects the accuracy of sensitive data recognition. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the sensitive data recognition method provided by the above embodiment, which will not be repeated here.

[0174] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A sensitive data identification method, characterized in that: The sensitive data identification method comprises: Based on punctuation marks in the text data to be recognized, segmenting the text data to be recognized to obtain a plurality of short sentences; Counting the specific words in each of the short sentences that are not in the preset common word library, and recording the word frequency of each of the specific words in the corresponding time residence matrix; According to the preset word segmentation rules, each short sentence is segmented separately to determine multiple word segmentation paths corresponding to each short sentence; Based on the word frequencies of each of the time residence matrices, the joint probabilities corresponding to each of the word segmentation paths are calculated, and the word segmentation path with the highest joint probability is used as the target word segmentation path; All the segmented words of the text data to be identified are obtained according to the target segmented word path of each of the short sentences, and sensitive word retrieval is performed on each of the segmented words to obtain sensitive data identification results.

2. The sensitive data identification method according to claim 1, characterized in that: The step of segmenting each short sentence according to the preset segmentation rules and determining multiple segmentation paths corresponding to each short sentence includes: Based on the word segmentation rule, determining candidate attributes of each word in the short sentence, wherein the candidate attributes include one or more of the first word, the middle word, the last word, and the single word, and the word segmentation rule at least includes: the first word of the short sentence is the first word or the single word, and the last word of the short sentence is the last word or the single word; According to the multiple alternative attributes corresponding to each character, the short sentence is divided into paths according to the word segmentation rules to obtain multiple word segmentation paths, wherein the word segmentation rules at least include: the next character after the first character or the middle character is the middle character or the last character, and the next character after the last character or the single character is the first character or the single character.

3. The sensitive data identification method according to claim 1, characterized in that: The step of calculating the joint probabilities corresponding to the word segmentation paths based on the word frequencies of the time residence matrices, and taking the word segmentation path with the highest joint probability as the target word segmentation path includes: The word frequencies of the time-residence matrices and the joint probabilities corresponding to the word segmentation paths are respectively input into a preset hidden semi-Markov model to calculate the joint probabilities corresponding to each word segmentation path; The word segmentation path with the highest joint probability among the word segmentation paths is selected to obtain the target word segmentation path.

4. The sensitive data identification method according to claim 3, characterized in that: Before the step of inputting the word frequencies of each of the time residence matrices and the joint probabilities corresponding to each of the word segmentation paths into a preset hidden semi-Markov model, the method further includes: By searching each of the segmented words in a preset original word library, the meaningless segmented words that do not exist in the original word library are determined; The segmentation paths including the meaningless segmentations in each segmentation path are deleted to obtain updated segmentation paths, and the step of inputting the word frequencies of each time-residence matrix and the joint probabilities corresponding to each segmentation path into a preset hidden semi-Markov model is performed based on the updated segmentation paths.

5. The sensitive data identification method according to claim 1, characterized in that: The sensitive data identification result is a sensitivity level. The steps of obtaining all the segmented words of the text data to be identified according to the target segmented word path of each of the short sentences, and performing sensitive word retrieval on each of the segmented words to obtain the sensitive data identification result include: Dividing each of the short sentences according to the target word segmentation paths corresponding to each of the short sentences to obtain a plurality of word segments corresponding to each of the short sentences; Based on the preset sensitive word classification table, the sensitivity level of each word is determined in turn to obtain the sensitivity level corresponding to each word; The highest sensitivity level among the sensitivity levels corresponding to the word segments is determined as the sensitivity level corresponding to the text data to be identified.

6. The sensitive data identification method according to any one of claims 1 to 5, characterized in that: The sensitive data identification method further includes: Collect corresponding vocabulary data from multiple training material files to build an original vocabulary; Based on the preset general material files and professional material files, counting the number of files containing each vocabulary and each word in the original word library; Calculate the inverse text frequency index corresponding to each word or word according to the number of files containing each word and each word and the total number of the training material files; Vocabulary or single words whose inverse text frequency index is less than a preset inverse text frequency index threshold are selected to form a preset common word library.

7. A sensitive data identification device, characterized in that: The sensitive data identification device comprises: A short sentence segmentation module, used for segmenting the text data to be identified based on punctuation marks in the text data to be identified, to obtain a plurality of short sentences; A vocabulary statistics module, used for counting the specific words in each of the short sentences that are not in the preset common vocabulary, and recording the word frequency of each of the specific words in the corresponding time residence matrix; The short sentence segmentation module is used to segment each short sentence according to the preset segmentation rules and determine the multiple segmentation paths corresponding to each short sentence; A path selection module, used to calculate the joint probability corresponding to each of the segmentation paths based on the word frequency of each of the time residence matrices, and take the segmentation path with the highest joint probability as the target segmentation path; The sensitive search module is used to obtain all the segmented words of the text data to be identified according to the target segmented word path of each short sentence, perform sensitive word search on each segmented word, and obtain sensitive data identification results.

8. An electronic device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the sensitive data identification method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the sensitive data identification method as described in any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the sensitive data identification method according to any one of claims 1 to 6 are implemented.

Images