Context-aware based sensitive information identification method, system, device and medium

Abstract

The application provides a context-aware sensitive information identification method, system, device and medium, and relates to the technical field of data security. In the method, a preset sliding window is used to cut the to-be-identified data stream into a continuous text slice sequence and generate a global coordinate index; a corresponding prompt word template is dynamically assembled based on metadata characteristics; the text slice and the prompt word template are input into a preset context-aware model to output sensitive entities, local start coordinates, local end coordinates and confidence scores; after coordinate conversion, the sensitive entities corresponding to the same global position in the overlapping area of adjacent text slices are de-duplicated to generate a sensitive entity list; and the sensitive entity list is desensitized using a preset security protection strategy, and the desensitized sensitive entity list, global end coordinates and global coordinate index are used for stream reorganization to obtain an identified desensitized data stream, thereby improving the accuracy of sensitive information identification and data desensitization.

Description

Technical Field

[0001] This application relates to the field of data security technology, specifically to a method, system, device, and medium for identifying sensitive information based on context awareness. Background Technology

[0002] With the diversification of enterprise data types, the forms of sensitive information are no longer limited to fields with strong rule characteristics such as ID card numbers and bank card numbers. Instead, they are increasingly appearing in instant messaging, code comments, and office documents in unstructured and semantically implicit ways, such as project codes, architecture descriptions, and weak password hints, posing new challenges to data security protection.

[0003] In related technologies, the identification and de-identification of sensitive information in unstructured long text data streams typically employs a cascaded processing approach combining rule-based initial screening and model verification. Specifically, when processing long text, rule matching is first performed on the long text data stream, followed by model verification on the matched data stream. Because this processing method requires sensitive information to first trigger existing regular expressions or keyword rules before it can enter the subsequent large language model for verification, if variant expressions not included in the rule base are used (e.g., writing a phone number as "138-phone"), or if the sensitive information itself lacks a fixed format (e.g., a comment stating "the password is the boss's birthday followed by 123"), the rule layer will directly miss it, preventing this part of the data from entering the model's judgment stage. This results in missed detections of sensitive information, reducing the accuracy of sensitive information identification and de-identification. Summary of the Invention

[0004] This application provides a context-aware sensitive information identification method, system, device, and medium that can improve the accuracy of sensitive information identification and data desensitization.

[0005] The technical solution of this application embodiment is as follows: In a first aspect, embodiments of this application provide a context-aware sensitive information identification method, the method comprising: The original data stream to be identified is obtained, and the data stream to be identified is divided into a continuous sequence of text slices using a preset sliding window. A global coordinate index of each text slice in the data stream to be identified is generated. The head of the (N+1)th text slice in the text slice sequence overlaps with the Nth text slice. Identify the metadata features of the data stream to be identified, and dynamically assemble the corresponding prompt word template based on the metadata features. The prompt word template contains sensitive entity types and does not contain preset regular expression rules. The text slice and the prompt word template are input into a preset context-aware model to perform semantic reasoning, and the sensitive entities in the text slice that match the preset sensitive keywords, the local start coordinates and local end coordinates of each sensitive entity in the text slice, and the confidence score corresponding to the sensitive entity are output. Based on the global coordinate index, the local start coordinates and local end coordinates of each sensitive entity are converted into global start coordinates and global end coordinates. Based on the confidence score, the global start coordinates and global end coordinates, the sensitive entities corresponding to the same global position in the overlapping area of ​​adjacent text slices are deduplicated to generate a list of sensitive entities. The sensitive entity list is de-identified using a preset security protection strategy. Based on the de-identified sensitive entity list, the global termination coordinates, and the global coordinate index, a streaming reconstruction is performed to obtain the identified de-identified data stream.

[0006] In the above technical solution, the original data stream to be identified is first acquired. A preset sliding window is used to segment the data stream into a continuous sequence of text slices, and a global coordinate index for each text slice in the data stream is generated. The (N+1)th text slice in the sequence overlaps with the Nth text slice, avoiding the limitations of long text data streams on model processing and providing data support for subsequent processing. The metadata features of the data stream to be identified are then identified. Based on these metadata features, corresponding prompt word templates are dynamically assembled. These prompt word templates contain sensitive entity types but do not contain preset regular expression rules. Dynamic prompt word construction enables adaptive recognition of multimodal data. The text slices and prompt word templates are input into a preset context-aware model for semantic reasoning, outputting the sensitive entities in the text slices that match preset sensitive keywords, and the sensitive entities for each sensitive keyword. By extracting the local start and end coordinates of sensitive entities within text slices and their corresponding confidence scores, the problem of missed detection of implicit and unformatted sensitive information is solved. Based on a global coordinate index, the local start and end coordinates of each sensitive entity are converted into global start and end coordinates. Then, based on the confidence scores, global start and end coordinates, duplicate sensitive entities corresponding to the same global position in the overlapping areas of adjacent text slices are deduplicated to generate a list of sensitive entities. By deduplicating due to conflicts in overlapping areas, the accuracy of sensitive entity identification is further improved. The sensitive entity list is anonymized using a preset security protection strategy. Based on the anonymized sensitive entity list, global end coordinates, and global coordinate index, streaming reconstruction is performed to obtain the identified anonymized data stream, achieving high-precision streaming anonymization.

[0007] In some embodiments of this application, the step of segmenting the data stream to be identified into a continuous sequence of text slices using a preset sliding window and generating a global coordinate index for each text slice in the data stream to be identified includes: The data stream to be identified is segmented using a sliding window with a preset overlap step size to obtain multiple segmented texts; Each of the segmented texts is corrected to obtain multiple text slices, and an index number is added to each text slice to generate a global coordinate index; A continuous sequence of text slices is formed based on each of the text slices.

[0008] In some embodiments of this application, the step of correcting each of the segmented texts to obtain multiple text slices includes: In each of the segmented texts, based on the text content of the original data stream to be identified, the N+1th segmented text is advanced by the overlap step to form a corrected segmented text, where N is an integer ≥1; Determine whether the text corresponding to the overlap step of the corrected segmented text is complete; In the case of completeness, the corrected segmented text is taken as the text slice; In the case of incompleteness, based on the text content of the original data stream to be identified, the corrected N+1th segmented text is advanced by the overlap step to form a corrected segmented text, and the corrected segmented text is used as the text slice.

[0009] In some embodiments of this application, the step of deduplicating sensitive entities corresponding to the same global position within the overlapping area of ​​adjacent text slices based on the confidence score, the global start coordinates, and the global end coordinates, and generating a list of sensitive entities, includes: Obtain a first sensitive entity and a second sensitive entity within the overlapping region, wherein the first sensitive entity comes from the Nth text slice and the second sensitive entity comes from the (N+1)th text slice; Compare the confidence score of the Nth text slice with the confidence score of the (N+1)th text slice, select the sensitive entity corresponding to the text slice with the highest confidence score, and generate a list of sensitive entities.

[0010] In some embodiments of this application, the stream reassembly of the de-identified sensitive entity list, the global termination coordinates, and the global coordinate index to obtain the identified de-identified data stream includes: If the text slice is not the last slice, a safe commit boundary is determined based on the global termination coordinates of the text slice and the overlap step size; Obtain a global output cursor, which is located after the security commit boundary. Lock the data range to be processed between the global output cursor and the security commit boundary, and map the de-identified sensitive entity list to the data range to be processed. If the data within the data range to be processed falls within the range of sensitive entities, the identified desensitized data will be output. If the data within the data range to be processed does not fall within the range of sensitive entities, the data to be identified is output directly. After the output is completed, the global output cursor is advanced to the position of the safe commit boundary, so that the processing of the next text slice continues directly from the position of the global output cursor. If the text slice is the last slice, the identified de-identified data stream is output.

[0011] In some embodiments of this application, determining the safe commit boundary based on the global termination coordinates of the text slice and the overlap step size includes: The safe submission limit is obtained by subtracting the global termination coordinate from the overlap step size.

[0012] In some embodiments of this application, the step of dynamically assembling the corresponding prompt word template based on the metadata features includes: The metadata features are matched with preset prompt types to obtain matching results; If the matching result indicates a key identification type, select a security prompt word corresponding to the key identification type from the preset sensitive identification template and assemble them to form a prompt word template; If the matching result indicates an identity recognition type, select identity information prompt words corresponding to the identity recognition type from the preset sensitive recognition template and assemble them to form a prompt word template.

[0013] Secondly, embodiments of this application provide a context-aware sensitive information identification system, the system comprising: The data acquisition module is used to acquire the original data stream to be identified, divide the data stream to be identified into a continuous sequence of text slices using a preset sliding window, and generate a global coordinate index for each text slice in the data stream to be identified. The head of the (N+1)th text slice in the text slice sequence overlaps with the Nth text slice. A dynamic generation module is used to identify the metadata features of the data stream to be identified, and dynamically assemble the corresponding prompt word template based on the metadata features. The prompt word template contains sensitive entity types and does not contain preset regular expression rules. The data analysis module is used to input the text slice and the prompt word template into a preset context-aware model to perform semantic reasoning, and output the sensitive entities in the text slice that match the preset sensitive keywords, the local start coordinates and local end coordinates of each sensitive entity in the text slice, and the confidence score corresponding to the sensitive entity. The data processing module is used to convert the local start coordinates and local end coordinates of each sensitive entity into global start coordinates and global end coordinates based on the global coordinate index, and to deduplicate the sensitive entities corresponding to the same global position in the overlapping area of ​​adjacent text slices according to the confidence score, the global start coordinates and the global end coordinates, and generate a list of sensitive entities. The desensitization generation module is used to desensitize the sensitive entity list using a preset security protection strategy, and perform streaming reconstruction based on the desensitized sensitive entity list, the global termination coordinates, and the global coordinate index to obtain the identified desensitized data stream.

[0014] Thirdly, embodiments of this application provide an electronic device including a processor, a memory, a user interface, a communication bus, and a network interface. The processor, the memory, the user interface, and the network interface are respectively connected to the communication bus. The memory is used to store instructions. The user interface and the network interface are used to communicate with other devices. The processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method described in any one of the first aspects.

[0015] Fourthly, embodiments of this application provide a computer-readable storage medium storing instructions that, when executed, perform the method described in any one of the methods provided in the first aspect above.

[0016] In summary, one or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages: 1. The method first acquires the original data stream to be recognized, then uses a preset sliding window to segment the data stream into a continuous sequence of text slices, generating a global coordinate index for each text slice within the data stream. The (N+1)th text slice overlaps with the Nth text slice, avoiding the limitations of long text streams on model processing and providing data support for subsequent processing. It identifies the metadata features of the data stream, dynamically assembling corresponding prompt word templates based on these features. These prompt word templates contain sensitive entity types but do not include preset regular expression rules. This dynamic prompt word construction enables adaptive recognition of multimodal data. The text slices and prompt word templates are input into a preset context-aware model for semantic reasoning, outputting the sensitive entities in the text slices that match preset sensitive keywords, and each sensitive... By extracting entities from the full data stream using local start and end coordinates and confidence scores corresponding to sensitive entities within text slices, the problem of missed detection of implicit, unformatted sensitive information is solved. Based on a global coordinate index, the local start and end coordinates of each sensitive entity are converted into global start and end coordinates. Then, based on the confidence score, global start and end coordinates, duplicate sensitive entities corresponding to the same global position within overlapping areas of adjacent text slices are deduplicated, generating a sensitive entity list. Deduplication through conflict resolution in overlapping areas further improves the accuracy of sensitive entity identification. A preset security protection strategy is used to de-identify the sensitive entity list. Based on the de-identified sensitive entity list, global end coordinates, and global coordinate index, streaming reassembly is performed to obtain the identified de-identified data stream, achieving high-precision streaming de-identification. Therefore, this effectively solves the problem of low accuracy in sensitive information identification and de-identification in related technologies.

[0017] 2. By correcting the text segments segmented by the sliding window, the breakage of sensitive information caused by slicing can be avoided, thereby improving the accuracy of sensitive information identification.

[0018] 3. By performing conflict resolution in overlapping areas, the list of sensitive entities is made unique, further improving the accuracy of desensitization.

[0019] 4. By dynamically adjusting the output through global output flow, the system ensures that ordinary business texts that are not confidential are sent in their original form, while sensitive information is anonymized and output without changing any of the original text's general formatting, thus achieving accuracy in the output text stream. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating a context-aware sensitive information identification method provided in one embodiment of this application; Figure 2 This is a schematic diagram of existing technology; Figure 3 yes Figure 1 A flowchart illustrating a sub-step of step S100; Figure 4 This is an overall schematic diagram of a context-aware sensitive information identification method provided in one embodiment of this application; Figure 5 This is a schematic diagram of the structure of a context-aware sensitive information identification system provided in one embodiment of this application; Figure 6 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application. Detailed Implementation

[0021] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0022] In the description of the embodiments of this application, the words "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design that is described as "for example" or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design options. Rather, the use of the words "for example" or "for instance" is intended to present the relevant concepts in a specific manner.

[0023] In the description of the embodiments of this application, the term "multiple" means two or more. For example, multiple systems means two or more systems, and multiple screen terminals means two or more screen terminals. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0024] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the proper functioning of these embodiments acquired.

[0025] In related technologies, models have limitations in processing long, unstructured text, requiring segmentation. If sensitive information happens to be located at the segmentation edge, it will be physically truncated, resulting in incomplete context for the input model. This semantic fragmentation not only reduces the accuracy of large language models but may also cause sensitive entities that should be identified to be ignored by the model due to incomplete information, further reducing the accuracy of desensitization.

[0026] Based on this, embodiments of this application provide a context-aware sensitive information identification method, system, electronic device, and readable storage medium. The context-aware sensitive information identification method first acquires the original data stream to be identified, then uses a preset sliding window to segment the data stream into a continuous sequence of text slices, and generates a global coordinate index for each text slice in the data stream. The (N+1)th text slice in the text slice sequence overlaps with the Nth text slice, avoiding the limitations of long text data streams on model processing, and the overlapping area prevents sensitive information breaks, providing data support for subsequent processing. Next, the method identifies the metadata features of the data stream to be identified, and dynamically assembles corresponding prompt word templates based on these metadata features. The prompt word templates contain sensitive entity types but do not contain preset regular expression rules. Dynamic prompt word construction enables adaptive identification of multimodal data. Finally, the text slices and prompt word templates are input into a preset context-aware model. This system performs semantic reasoning and outputs sensitive entities in text slices that match preset sensitive keywords, along with the local start coordinates, local end coordinates, and confidence scores of each sensitive entity within the text slice. By extracting entities from the entire data stream, it solves the problem of missed detection of implicit and unformatted sensitive information. Based on a global coordinate index, the local start and end coordinates of each sensitive entity are converted into global start and end coordinates. Then, based on the confidence scores, global start coordinates, and global end coordinates, duplicate sensitive entities corresponding to the same global position in the overlapping areas of adjacent text slices are deduplicated, generating a list of sensitive entities. By deduplicating due to conflicts in overlapping areas, the accuracy of sensitive entity identification is further improved. The list of sensitive entities is anonymized using preset security protection strategies. Based on the anonymized sensitive entity list, global end coordinates, and global coordinate index, streaming reassembly is performed to obtain the identified anonymized data stream, achieving high-precision streaming anonymization.

[0027] It should be noted that this context-aware sensitive information identification method is used for enterprise data security management. It can de-identify office documents and code in internal data, and can also be used for compliance checks of data display, as well as for security management of user identity information, communication dialogue and work order processing in financial service records.

[0028] The technical solutions provided in the embodiments of this application will be further described below with reference to the accompanying drawings.

[0029] Reference Figure 1 , Figure 1This is a flowchart illustrating the context-aware sensitive information identification method provided in this application. The context-aware sensitive information identification method is applied to a context-aware sensitive information identification system and is executed by a processor in an electronic device or readable storage medium. Figure 2 As shown, this is a cascaded processing method combining rule-based initial screening and model verification used in existing technologies. Compared with existing technologies, the context-aware sensitive information identification method can improve the accuracy of identifying sensitive information and data anonymization. This context-aware sensitive information identification method includes steps S100, S200, S300, S400, and S500.

[0030] Step S100: Obtain the original data stream to be identified, divide the data stream to be identified into a continuous sequence of text slices using a preset sliding window, and generate a global coordinate index for each text slice in the data stream to be identified. The head of the (N+1)th text slice in the text slice sequence overlaps with the Nth text slice.

[0031] In one embodiment, the original data stream to be identified refers to the continuous text data generated by the enterprise network, terminal, or application layer that needs to be detected for sensitive information. This can be a long or short text data stream, such as Git code commit records, instant messaging chat logs, or office documents. It is acquired in real-time via an API interface and stored in memory. Then, the original data stream to be identified is read from memory to provide data support for subsequent de-identification and recognition. It should be noted that the continuous data collected by the API interface can also be image and audio data. After data reading, Optical Character Recognition (OCR) technology or image recognition algorithms are used to convert the image content into text. In the case of audio data, a speech recognition algorithm is used to convert the speech into text for processing, and finally, the de-identified text is output. All of the above algorithms are existing technologies and will not be elaborated upon here.

[0032] The following detailed explanation uses the original data stream to be identified as a long text data stream.

[0033] The preset sliding window is a processing mechanism and parameter set used to slide across long text and extract data at fixed character lengths or token counts. The size of the sliding window is set. Global coordinate indices are used to record the absolute character positions of text slices in the initial, unsegmented "raw data stream to be recognized." Overlapping regions are text areas containing identical content in two adjacent text slices.

[0034] like Figure 3As shown, a preset sliding window is used to segment the data stream to be recognized into a continuous sequence of text slices, and a global coordinate index of each text slice in the data stream to be recognized is generated, including but not limited to the following steps: Step S110: Use a sliding window with a preset overlap step size to slide and segment the data stream to be identified to obtain multiple segmented texts.

[0035] In one embodiment, the sliding window is configured with a preset overlap step and sliding window size. For example, the window length is 2000 characters, the overlap step is 200 characters, and the coordinate index starts from 0. The sliding window is aligned with the acquired data stream to be recognized from index 0. The sliding window ensures a 200-character overlap area for sliding segmentation, resulting in segmented text. Each segmented text is 2000 characters in size. The global coordinate index of the first text corresponds to the interval [0, 2000]. The global coordinate index of the second text corresponds to the interval [1800, 4000], thus ensuring an overlap area between the first and second texts. This process is repeated to obtain multiple segmented texts. Segmentation avoids the limitations of long text input models while preserving information about text boundary transitions.

[0036] Step S120: Correct each segmented text to obtain multiple text slices, and add an index number to each text slice to generate a global coordinate index.

[0037] Specifically, each segmented text is corrected to obtain multiple text slices, including but not limited to the following steps: Step S121: In each segmented text, based on the text content of the original data stream to be identified, the N+1th segmented text is advanced by the overlap step to form a corrected segmented text, where N is an integer ≥1.

[0038] In some possible embodiments of this application, regarding text segmentation, if the text is segmented at a sensitive entity or keyword identification point, it may lead to missed identification of the sensitive entity or keyword. To avoid missing identification of sensitive entities at the segmentation point, in each segmented text, based on the text content of the original data stream to be identified, adjacent texts are segmented to create a break between the previous segmented text and the next segmented text. The next segmented text, i.e., the N+1th segmented text, is advanced by an overlap step, changing the starting position of the subsequent segmentation of the next segmented text, so that the N+1th segmented text is a complete text content at the previous segmentation point, forming a corrected segmented text. According to the example of step S110 above, the overlap step setting has a large number of characters, which can completely cover the relevant content that needs to be desensitized. By correcting the segmented text as described above, it is ensured that the edges of each slice entering the context-aware model are still natural and grammatically continuous, improving the accuracy of semantic reasoning of the model, and thus improving the accuracy of sensitive entity identification.

[0039] Step S122: Determine whether the text corresponding to the overlap step of the corrected segmented text is complete.

[0040] In some possible embodiments of this application, determining completeness involves checking for lexical integrity, using lightweight regular expression matching or symbol matching algorithms for boundary validity detection. The preceding character of the starting local boundary character segment of the corrected segmented text is extracted to determine if it is a space, punctuation mark, or newline character. If it is a text concatenation, the output is False, indicating incompleteness; if it is a symbol, the output is True, indicating completeness. This determination avoids the problem of incomplete content resulting from improper parameter settings in the extracted overlapping area.

[0041] Step S123: In the complete case, the corrected segmented text is used as a text slice.

[0042] In some possible embodiments of this application, if the current returned state is True, it indicates that the sentence is complete after advancing one overlap step. The corrected segmented text is then used as a text slice for subsequent input into the context-aware model for semantic reasoning, thereby improving the accuracy of sensitive identification.

[0043] Step S124: In the case of incompleteness, based on the text content of the original data stream to be identified, the corrected N+1th segmented text is advanced by the overlap step to form a corrected segmented text, and the corrected segmented text is used as a text slice.

[0044] In some possible embodiments of this application, if the current returned state is False, it means that after looking further ahead, the boundary is still stuck on an unresolved half-word. At this point, a second-order iterative correction mechanism is triggered. The original text content of the data stream to be identified is accessed again. Based on the first correction, the N+1th segmented text is advanced forward by the overlap step, and the reinforced state forms the latest corrected segmented text. This corrected segmented text is grammatically complete and compliant, and is used as a text slice. As a flexible fault-tolerant mechanism, by expanding the search back to the historical end to find the complete boundary, it can be input into the context-aware model for semantic reasoning, improving the accuracy of sensitive identification. It should be noted that for commonly used characters that need to be desensitized, the overlap step is set reasonably, and a single correction is sufficient to satisfy the semantic integrity of the segmented text. Regarding the distinction of semantic integrity, since the characters to be desensitized are usually short in adjacent text slices, setting the overlap step to 200 characters ensures that after correction, each subsequent slice can include complete semantic information to be identified. For example, the desensitization of an ID card is much shorter than 200 characters.

[0045] In one embodiment, an index number is added to each text slice, and a global coordinate index is generated as follows: a list of tuples is constructed, and a global coordinate index is built for each corrected segmented text. An example of a global coordinate index tuple is [{SliceID:1,Global_Range:[0,1600]},...]. If a second correction is performed, a corrected global coordinate index is generated. By establishing the index, the accuracy of subsequent streaming reconstruction is ensured.

[0046] Step S130: Form a continuous text slice sequence based on each text slice.

[0047] In one embodiment, text slices are stored in a queue with a first-in, first-out (FIFO) characteristic, thus sequentially sorting the text slices according to their global coordinate indices. The text slices and their global coordinate indices are concatenated and stored in the created queue, forming a continuous sequence of text slices, which is then encapsulated into an iterable sequence object. When subsequently input into a context-aware model, this allows for the processing of continuous text slices, providing a structured arrangement foundation for feature comparison and manifold reconstruction. Notably, the head of the (N+1)th text slice in the sequence overlaps with the Nth text slice. This ensures the continuity of the segmented text while also addressing the model's limitation on long texts.

[0048] Step S200: Identify the metadata features of the data stream to be identified, and dynamically assemble the corresponding prompt word template based on the metadata features. The prompt word template contains sensitive entity types and does not contain preset regular expression rules.

[0049] In one embodiment, metadata features are used to describe the attributes of the original data stream to be identified. These can be represented as data source, data type, file format, or business scenario tags. Business scenario tags include HR email addresses, Git Commits, system logs, etc. Based on the received data stream to be identified, the context protocol header carried by the data stream is read, and metadata feature fields are extracted to prepare for subsequent generation of prompt words. Different types of sensitive prompt word samples are obtained from storage. Based on different types, various prompt words under that type are selected, enabling dynamic construction of prompt words. The prompt word template differs from rule-based matching; it includes sensitive entity types but does not contain preset regular expression rules. This prompt word template is used as input to the context-aware model to guide the large model in sensitive word identification.

[0050] Specifically, the corresponding prompt word template is dynamically assembled based on metadata features, including but not limited to the following steps: Step S210: Match the metadata features with the preset prompt types to obtain the matching results.

[0051] In one embodiment, the preset prompt types are pre-registered categories in the system, selectable and configurable, suitable for different types of data, including types such as Git code documents, HR emails, and customer service communications. Each prompt type corresponds one-to-one with a preset sensitive identification module for dynamic assembly of prompt words. A preset similarity matching algorithm is used to match the metadata features with the prompt types, obtaining a matching result. This result indicates whether the metadata features and prompt types match successfully. A similarity value is calculated using the similarity algorithm. If the similarity value is greater than or equal to a preset similarity threshold, it indicates a successful match, and the matching result is the prompt type corresponding to the successful match. If the similarity value is less than the preset similarity threshold, it indicates that none of the currently set prompt types have matched successfully, triggering manual identification of the metadata feature type. Then, manual configuration of prompt words is performed based on the sensitive identification template for future matching. Alternatively, the prompts can be categorized into other types, each corresponding to a general prompt word template, to generate dynamically assembled prompt word templates. The preset similarity algorithm can be a text similarity algorithm or a cosine similarity algorithm, and the preset similarity threshold can be 0.8, etc.

[0052] It should be noted that the pre-configured prompt types are diverse, capable of meeting the dynamic prompt word construction needs in different situations. Among them, the most commonly used are key recognition type and identity recognition type. For example, during the similarity matching process, the prompt types are stored in the form of a list or dictionary, such as [key recognition type, identity recognition type, other types]. By performing matching, prompt types greater than or equal to the similarity threshold are taken as the matching results, so as to generate prompt word templates subsequently.

[0053] Step S220: If the matching result indicates a key identification type, select a security prompt word corresponding to the key identification type from the preset sensitive identification template and assemble it to form a prompt word template.

[0054] In one embodiment, the focus of the key identification type is to protect against the leakage of hard-coded verification credentials due to oversights in program code or internal development communications. When the matching result indicates a key identification type, it signifies that the sensitive identification is for document security desensitization. A security prompt corresponding to the key identification type is selected from a preset sensitive identification template. For example, its content is set as: "Focusing on and extracting whether there are any non-fixed hidden API call password configurations, cloud application AK / SKs, or plaintext storage or information containing obvious descriptive hints of weak passwords, etc., related to high-risk system access keys." Then, through a string formatting assembly slot, such as using Template.format(), the extracted security prompt is integrated into the prompt framework set by the context-aware model, and combined and assembled into a prompt template for the context-aware model. This achieves on-demand intelligent distribution and adaptive reorganization of prompt engineering in the AI ​​era, allowing the model to concentrate its computing power and weight on tasks involving the leakage verification of code passwords and credentials without a specific format, greatly improving the probability of implicit credential lookup.

[0055] Step S230: If the matching result indicates an identity recognition type, select identity information prompt words corresponding to the identity recognition type from the preset sensitive recognition template and assemble them to form a prompt word template.

[0056] In one embodiment, the focus of the identity recognition type is on the humanities and financial personal compliance defense type categories that target the review of exposed personal characteristic elements. When the matching result indicates an identity recognition type, it means that the sensitive recognition is aimed at protecting personal information privacy. An identity information prompt word corresponding to the identity recognition type is selected from a preset sensitive recognition template. For example, its content is set to: focus on extracting data with real personal physical definition privacy characteristics, such as the natural person's full name, social welfare digital identification number, address, and patient's medical history, but carefully identify and ignore publicly available commercial business complaint hotline numbers. Then, through a string formatting assembly slot, such as using Template.format(), the extracted identity information prompt word is integrated into the prompt word framework set by the context-aware model, and combined and assembled into a prompt word template for the context-aware model. This enables the true identification of the judgment boundary of the number in the context of the large model, effectively eliminating the problem of falsely blocking publicly available public data calls, making the de-identification filtering protection logic highly compatible with the scenario, and greatly improving the phenomenon of false de-identification.

[0057] Step S300: Input the text slice and prompt word template into the preset context-aware model to perform semantic reasoning, and output the sensitive entities in the text slice that match the preset sensitive keywords, the local start coordinates and local end coordinates of each sensitive entity in the text slice, and the confidence score corresponding to the sensitive entity.

[0058] In one embodiment, the context-aware model is a large language model, which is a Transformer architecture model with a self-attention mechanism. Existing large models are applicable. A pre-defined context-aware model is called via an API interface to understand natural language context. Sensitive entities are concrete, confidential text identified by the model from text slices, such as a Wi-Fi password being the boss's birthday plus 123. Local start / end coordinates refer to the character position index of the sensitive entity within the current truncated text slice, for example, the 10th to 25th character in a 2000-character slice. The confidence score is a probability value between 0.0 and 1.0, indicating the large model's level of confidence in its output.

[0059] The text slice processed in step S100 is used as context content and concatenated with the prompt word template generated in step S200. The combined content is then sent to the deployed context-aware model via an HTTP / RPC request. The large model performs long-distance dependency semantic analysis based on a self-attention mechanism within a deep neural network. After inference, the model returns a structured result in JSON format. The returned JSON is parsed to extract specific sensitive words. Sensitive words matching preset sensitive keywords are identified as sensitive entities. The relative physical location of each sensitive entity within its slice is defined, including local start coordinates (Local_Start=50), local end coordinates (Local_End=68), and the probability confidence score (e.g., confidence score=0.98) indicating that the word belongs to sensitive information. Directly executing the model to discover information replaces the existing execution verification process, eliminating the funnel effect caused by errors in the initial rule screening. This atomically merges context analysis and entity coordinate localization, enabling the accurate identification and numerical quantification of fragmented sensitive features hidden within unstructured long texts.

[0060] Step S400: Based on the global coordinate index, the local start coordinates and local end coordinates of each sensitive entity are converted into global start coordinates and global end coordinates. Based on the confidence score, global start coordinates and global end coordinates, the sensitive entities corresponding to the same global position in the overlapping area of ​​adjacent text slices are deduplicated to generate a list of sensitive entities.

[0061] In one embodiment, the global start / end coordinates are calculated by adding the global offset of the local coordinates to the value of the slice to which it belongs, representing the true and unique absolute position of the sensitive entity in the original longest data stream. Sensitive entities at the same global position are identified because of overlapping regions; the same sensitive word is identified once at the end of slice 1 and once at the beginning of slice 2, and these two identifications, after being mapped back to the original data, have overlapping absolute global position intervals. According to the global coordinate index, for an entity in the A-th text slice, its global start coordinate = global start coordinate of slice A + local start coordinate; its global end coordinate = global start coordinate of slice A + local end coordinate. This allows the position of the sensitive entity in the original data stream to be identified, facilitating subsequent deduplication and streaming reassembly processing.

[0062] In one embodiment, based on the confidence score, global start coordinates, and global end coordinates, sensitive entities corresponding to the same global location within the overlapping area of ​​adjacent text slices are deduplicated to generate a list of sensitive entities, including but not limited to the following steps: Step S410: Obtain the first sensitive entity and the second sensitive entity within the overlapping area, wherein the first sensitive entity comes from the Nth text slice and the second sensitive entity comes from the N+1th text slice.

[0063] Specifically, the first and second sensitive entities represent two sequential entity results generated by the context-aware model after independently inferring the preceding slice containing the tail of the overlapping region and the following slice containing the head of the overlapping region, respectively, for the recognition of the same string of text. Adjacent text slices are represented as the Nth text slice and the (N+1)th text slice. The first sensitive entity is configured to come from the Nth text slice, and the second sensitive entity is configured to come from the (N+1)th text slice. The correspondence between them is not elaborated here.

[0064] By performing global coordinate conflict addressing, the received entity list is traversed. When a first sensitive entity from the Nth text slice is found to fall exactly within the physical range calculated by the overlap step, and the (N+1)th subsequent text slice is immediately scanned, a second sensitive entity is found whose global coordinates, when mapped back, also fall within the physical range of the overlap step, it is determined that these two entities have the attribute of spatial interval intersection on the original data segment. This allows the extraction of instance data for the first and second sensitive entities, preparing for subsequent comparisons. Through coordinate interval intersection comparison, the same sensitive entity, which is forced to emerge in different slice-by-slice processing due to the existence of overlapping windows, is accurately captured, providing a dataset to be processed for subsequent redundant fusion.

[0065] For example, the first sensitive entity identified is the partially incomplete suffix 67890, with global coordinates of [1950, 2000]. The second sensitive entity identified is the complete number: 123456789012, with global coordinates of [1945, 2005]. The global coordinates of the two entities intersect, indicating that there is a duplicate or conflicting identification of sensitive entities. The first and second sensitive entities are extracted and subjected to sensitive deduplication and fusion processing.

[0066] Step S420: Compare the confidence score of the Nth text slice with the confidence score of the (N+1)th text slice, select the sensitive entity corresponding to the text slice with the highest confidence score, and generate a list of sensitive entities.

[0067] Specifically, based on the first and second sensitive entities selected in step S410, corresponding confidence scores are obtained. First, the first sensitive entity is obtained, and the model returns a confidence score of 0.61. Next, the second sensitive entity is obtained. Due to text segmentation correction, the second sensitive entity is included in the overlapping area of ​​the subsequent segments, displaying the complete semantic picture, and the model returns a confidence score of 0.99. The two confidence scores are compared, and the sensitive entity corresponding to the text segment with the highest confidence score is selected. Sensitive entities with lower confidence scores are removed from the sensitive entity list, resulting in a unique list of sensitive entities. By performing conflict resolution in the overlapping area, the list of sensitive entities is made unique, avoiding missed identification and further improving the accuracy of desensitization.

[0068] Step S500: De-identify the sensitive entity list using a preset security protection strategy, and perform streaming reconstruction based on the de-identified sensitive entity list, global termination coordinates, and global coordinate index to obtain the identified de-identified data stream.

[0069] In one embodiment, the preset security protection strategy addresses the processing methods for information with different sensitivity levels, including masking, hash encryption, and risk interception. The sensitive entity list is categorized by security level, determined by a lookup table, and different security protection strategies are employed for different security levels. For example, high-risk sensitive entities are encrypted using hash encryption or full masking, while medium-risk entities are partially masked. Sensitive entities in the resulting unique sensitive entity list are replaced using the security protection strategy to generate a de-identified sensitive entity list, thereby protecting the identified sensitive information.

[0070] In another embodiment, streaming reconstruction is performed based on the de-identified sensitive entity list, global termination coordinates, and global coordinate index to obtain the identified de-identified data stream, including but not limited to the following steps: Step S510: If the text slice is not the last slice, determine the safe commit boundary based on the global termination coordinates and overlap step of the text slice.

[0071] In one embodiment, the security submission limit refers to a virtual boundary coordinate point calculated in real time during the reassembly and output of data. Data to the left of this point is secure data that has been de-identified and confirmed without omissions, while data to the right is in the area to be verified. During reassembly, the input order pointer of the original data stream is compared with the global coordinate index in the de-identification list. The buffer cursor is moved backward from global coordinate 0, i.e., reassembly begins from the first slice. If the text slice is not the first slice, the security submission limit is first determined based on the global termination coordinate and overlap step size of the text slice. Specifically, the global termination coordinate of the current text slice is read from the record cache, the previously used overlap step size value is read from the global configuration file, and the global termination coordinate is subtracted from the overlap step size to obtain the security submission limit. For example, if the global termination coordinate is 2000 and the overlap step size is 200, the coordinate point corresponding to its security submission limit is 1800. By determining the security submission limit, a security benchmark is established, separating the securely confirmed segments from the unresolved segments, so as to continuously output an accurate de-identified data stream.

[0072] Step S520: Obtain the global output cursor. After the global output cursor reaches the security commit boundary, lock the data range to be processed between the global output cursor and the security commit boundary, and map the de-identified sensitive entity list to the data range to be processed.

[0073] In one embodiment, the global output cursor represents a pointer to the physical location where data has been securely output and reached as it is being processed and sent outwards. Based on the position of the global output cursor and the position of the security commit boundary, a data range to be processed is defined. For example, if the global output cursor was previously at position 1000, the security commit boundary is 1800, and the data range to be processed is [1000, 1800], all global start and global end coordinates falling within the physical location range of [1000, 1800] are searched for and replaced encrypted entity record structures. These are then cached in the processing memory pool to achieve spatial mapping, thereby mapping the de-identified sensitive entity list to the data range to be processed. By dynamically defining the range and cutting out the current secure working area, character overwriting and confusion in overlapping areas are avoided.

[0074] Step S530: If the data within the data range to be processed falls within the range of sensitive entities, the identified desensitized data is output.

[0075] In one embodiment, the sensitive entity range is defined by the global start coordinates and global end coordinates of the sensitive entity. If data falls within the sensitive entity range of the data to be processed, it indicates that during the process of scanning the original string character by character with the cursor to prepare to send this part of the work area, the absolute position of the currently scanned character is between the start and end coordinates of the entity to be replaced. Subsequent characters are then replaced with desensitized data. For example, when the pointer advances to position 1200 in the interval [1000, 1800], a query reveals that positions 1200-1211 are within the sensitive entity range, which contains desensitized sensitive entities. The identified desensitized data is then output. By using the pointer and cursor, confirmed sensitive information is masked to prevent plaintext exposure and ensure data security.

[0076] Step S540: If the data within the data range to be processed does not fall within the range of sensitive entities, the data to be identified is directly output.

[0077] In one embodiment, the character-by-character scanning process continues. If the data in the data range to be processed does not fall within the scope of sensitive entities, it indicates that the data stream to be identified does not include sensitive entities, and plaintext output can be performed. The data to be identified is directly output, ensuring that ordinary business text that does not involve confidentiality is sent without loss and without changing the original text layout, thus meeting the principle of minimizing the impact of desensitization.

[0078] Step S550: After the output is completed, advance the global output cursor to the safe commit boundary so that the processing of the next text slice continues directly from the position of the global output cursor. If the text slice is the last slice, output the identified de-identified data stream.

[0079] In one embodiment, after the current text segment is output, the global output cursor is advanced to the safe commit boundary, completing the pointer locking and forward migration update task. This allows the processing of the next text segment to continue directly from the position of the global output cursor, uniquely identifying sensitive entities in overlapping areas. The above process is applied sequentially to text segments that are not the last. When a text segment is the last, the identified de-identified data stream is output, achieving de-identified output of all data streams to be identified.

[0080] It should be noted that in the above process, when encountering coordinate points without records, the original data to be identified is copied intact. When the cursor advances to coincide with the global starting coordinates recorded in the list of de-identified sensitive entities, the copying is paused, and the de-identified ciphertext string is read from the de-identification list and directly written into the output pipeline. Then, the control cursor jumps directly to the position of the "global ending coordinates" to continue searching downwards to complete the data, thus achieving a streaming return without duplication and smooth output. By dynamically adjusting the output through the global output cursor, it is ensured that ordinary business texts that do not involve confidentiality are sent as is, while sensitive information is de-identified and output without changing any ordinary formatting of the original text, thus achieving the accuracy of the output text stream.

[0081] For example, the streaming reconstruction process is the original stream: [ABCDEFGH], slice 1 is [ABCDE], slice 2 is [DEFGH], the overlap step is 2, and the sensitive entities are identified as E and F.

[0082] For slice 1, the global termination coordinate points to E. After calculation, the safe commit boundary points to position C. The global output cursor is after C. Outputs A, B, and C are safe. The data range to be processed includes [DE].

[0083] For slice 2, [DEFGH] inherits the overlap DE, the global termination coordinate points to F, and the global output cursor starts scanning from D. The area to be processed is [EF], which is not within the sensitive entity range, so it is output safely. E and F are within the sensitive entity range, so de-sensitized data is output. After the above processing, each text slice is processed to achieve streaming reconstruction. It should be noted that by subtracting the overlap step size from the global termination coordinate to obtain the safe submission boundary, even if a sensitive entity is broken in the current slice, it will still be identified in the next slice, thus achieving de-sensitized output of all sensitive entities.

[0084] like Figure 4 The diagram shown is an overall schematic of a context-aware sensitive information identification method provided in this application embodiment. For long texts, the text is first segmented by a sliding window, and the segmented text is corrected to obtain text slices. Dynamic prompt word templates are generated, and the text slices and prompt word templates are input into a context-aware model for inference. Overlapping deduplication and fusion are performed, and finally, a streaming strategy is executed and reorganized to output a desensitized data stream, thereby improving the accuracy of sensitive information identification and data desensitization.

[0085] The above steps will be explained below through specific embodiments.

[0086] Example 1: Processing an extremely long log document (exceeding the model's token limit) containing a long ID number or UUID that spans the split boundary.

[0087] Original data stream to be identified: ...User ID: 8829102-XJ, ID card number: 000000123456789012, address: XX city.... Slice length is 100 characters, overlap step is 20 characters.

[0088] Slice A: ... User ID: 8829102-XJ, ID number: 0000001234567890 (the last 12 is truncated, and 90 is in the overlapping area), Slice B: 789012, Address: XX City... (7890 is repeated at the beginning, and the complete second half 12 is in this slice), the real ID number is 000000123456789012.

[0089] Inference for slice A: The context-aware model identifies 0000001234567890 as an ID card number with a confidence level of 0.7. Inference for slice B: The context-aware model sees the incomplete suffix 789012, which may be unrecognizable or identified as a regular number.

[0090] Slice B is corrected by adding 20 characters to the beginning, resulting in: "...ID number: 000000123456789012, address...". The context-aware model sees the complete "000000123456789012" in slice B and successfully identifies it as an ID number with a confidence level of 0.99. During deduplication, sensitive entities from slice A are discarded, while those from slice B are retained. This achieves accurate sensitive entity recognition even when the text is interrupted during segmentation.

[0091] Example 2: When handling customer complaints, customer service personnel sent the customer's mobile phone number and the company's external complaint hotline in the same conversation.

[0092] The data stream to be identified is Mr. Li's, and we have received your registered mobile phone number 12311112222. A specialist will contact you shortly. If you have any questions, please call our complaint hotline 88888888 or mobile phone 12311113333 (24-hour service).

[0093] Contextual characteristics: "Your registered mobile phone number". Judgment: High risk, pertains to customer privacy. "Complaint hotline". Judgment: Low risk, pertains to public business information. "24-hour on-call service". Judgment: Low risk, pertains to public business mobile phone number. High-risk numbers are masked, while low-risk numbers are displayed directly. This distinguishes between mobile phone numbers and service hotlines, avoiding a blanket ban.

[0094] Example 3: In the DevSecOps process, a developer submitted a piece of Python code containing debugging information to a Git repository. This code did not contain an explicit password string, but instead included a weak password hint described in natural language.

[0095] The input data stream to be identified is DB_HOST='192.168.x.x', and the database password is the password for the company's front desk WIFI, with 0000 added to the end.

[0096] The data source metadata was identified as GitLab_Commit. The "Code Security Audit" sensitive identification template was then invoked. This template defines the IP address and weak password, and the extracted entities are as follows: Entity 1:{'Type':'IP_ADDRESS','Value':'192.168.x.x','Risk':'Low'}; Entity 2: {'Type':'WEAK_PASSWORD_HINT','Value':'The database password is the company's front desk WIFI password, followed by 0000','Risk':'Critical'}, can extract weak passwords through context recognition.

[0097] like Figure 5As shown, this application embodiment provides a context-aware sensitive information recognition system 100. This system 100 acquires the original data stream to be recognized through a data acquisition module 110, divides the data stream into a continuous sequence of text slices using a preset sliding window, and generates a global coordinate index for each text slice in the data stream. The (N+1)th text slice in the text slice sequence overlaps with the Nth text slice. Then, a dynamic generation module 120 identifies the metadata features of the data stream and dynamically assembles corresponding prompt word templates based on these features. The prompt word templates contain sensitive entity types but do not contain preset regular expression rules. Finally, a data analysis module 130 inputs the text slices and prompt word templates into a preset context-aware... The model performs semantic reasoning and outputs sensitive entities in text slices that match preset sensitive keywords, the local start coordinates and local end coordinates of each sensitive entity within the text slice, and the confidence score corresponding to the sensitive entity. The data processing module 140 converts the local start coordinates and local end coordinates of each sensitive entity into global start coordinates and global end coordinates based on the global coordinate index. Then, based on the confidence score, global start coordinates, and global end coordinates, it removes duplicates of sensitive entities corresponding to the same global position within the overlapping areas of adjacent text slices, generating a list of sensitive entities. Finally, the desensitization generation module 150 desensitizes the sensitive entity list using a preset security protection strategy. Based on the desensitized sensitive entity list, global end coordinates, and global coordinate index, it performs streaming reassembly to obtain the identified desensitized data stream.

[0098] It should be noted that the data acquisition module 110 is connected to the dynamic generation module 120, the dynamic generation module 120 is connected to the data analysis module 130, the data analysis module 130 is connected to the data processing module 140, and the data processing module 140 is connected to the de-identification generation module 150. The aforementioned context-aware sensitive information recognition method is applied to a context-aware sensitive information recognition system 100. The system acquires the original data stream to be recognized, divides it into a continuous sequence of text slices using a preset sliding window, and generates a global coordinate index for each text slice within the data stream. The (N+1)th text slice overlaps with the Nth text slice, avoiding the limitations of long text streams on model processing and providing data support for subsequent processing. The system identifies the metadata features of the data stream and dynamically assembles corresponding prompt word templates based on these features. These prompt word templates contain sensitive entity types but do not contain preset regular expression rules. This dynamic prompt word construction enables adaptive recognition of multimodal data. Finally, the text slices and prompt word templates are input into a preset context-aware model for semantic reasoning. The system outputs sensitive entities in text slices that match preset sensitive keywords, along with the local start coordinates, local end coordinates, and confidence scores of each sensitive entity within the text slice. By extracting entities from the entire data stream, it solves the problem of missed detection of implicit and unformatted sensitive information. Based on a global coordinate index, the local start and end coordinates of each sensitive entity are converted into global start and end coordinates. Then, based on the confidence scores, global start and end coordinates, duplicate sensitive entities corresponding to the same global position within the overlapping areas of adjacent text slices are deduplicated, generating a list of sensitive entities. By deduplicating due to conflicts in overlapping areas, the accuracy of sensitive entity identification is further improved. The list of sensitive entities is anonymized using preset security protection strategies. Based on the anonymized sensitive entity list, global end coordinates, and global coordinate index, streaming reconstruction is performed to obtain the identified anonymized data stream, achieving high-precision streaming anonymization.

[0099] It should also be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0100] This application also discloses an electronic device. (See reference...) Figure 6 , Figure 6This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. The electronic device 500 may include: at least one processor 501, at least one network interface 504, a user interface 503, a memory 505, and at least one communication bus 502.

[0101] The communication bus 502 is used to enable communication between these components.

[0102] The user interface 503 may include a display screen and a camera. Optionally, the user interface 503 may also include a standard wired interface and a wireless interface.

[0103] The network interface 504 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0104] The processor 501 may include one or more processing cores. The processor 501 connects to various parts of the server using various interfaces and lines, and performs various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 505, and by calling data stored in memory 505. Optionally, the processor 501 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array. The processor 501 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and Modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also be implemented as a separate chip without being integrated into the processor 501.

[0105] The memory 505 may include random access memory (RAM) or read-only memory. Optionally, the memory 505 may include a non-transitory computer-readable storage medium. The memory 505 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 505 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 505 may also be at least one storage device located remotely from the aforementioned processor 501. (Refer to...) Figure 6 The memory 505, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an application program based on a context-aware sensitive information identification method.

[0106] exist Figure 6 In the illustrated electronic device 500, the user interface 503 is mainly used to provide an input interface for the user and to acquire user input data; while the processor 501 can be used to call an application program stored in the memory 505 for a context-aware sensitive information identification method. When executed by one or more processors 501, the electronic device 500 performs one or more methods as described in the above embodiments. It should be noted that, for the foregoing method embodiments, for the sake of simplicity, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0107] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0108] In the various embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some service interface; the indirect coupling or communication connection between apparatuses or units may be electrical or other forms.

[0109] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0110] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0111] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0112] The above are merely exemplary embodiments of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Other embodiments of this disclosure will be readily apparent to those skilled in the art upon consideration of the specification and practice of the disclosure herein.

[0113] This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

Claims

1. A context-aware sensitive information identification method, characterized in that, The method includes: The original data stream to be identified is obtained, and the data stream to be identified is divided into a continuous sequence of text slices using a preset sliding window. A global coordinate index of each text slice in the data stream to be identified is generated. The head of the (N+1)th text slice in the text slice sequence overlaps with the Nth text slice. Identify the metadata features of the data stream to be identified, and dynamically assemble the corresponding prompt word template based on the metadata features. The prompt word template contains sensitive entity types and does not contain preset regular expression rules. The text slice and the prompt word template are input into a preset context-aware model to perform semantic reasoning, and the sensitive entities in the text slice that match the preset sensitive keywords, the local start coordinates and local end coordinates of each sensitive entity in the text slice, and the confidence score corresponding to the sensitive entity are output. Based on the global coordinate index, the local start coordinates and local end coordinates of each sensitive entity are converted into global start coordinates and global end coordinates. Based on the confidence score, the global start coordinates and global end coordinates, the sensitive entities corresponding to the same global position in the overlapping area of ​​adjacent text slices are deduplicated to generate a list of sensitive entities. The sensitive entity list is de-identified using a preset security protection strategy. Based on the de-identified sensitive entity list, the global termination coordinates, and the global coordinate index, a streaming reconstruction is performed to obtain the identified de-identified data stream.

2. The method according to claim 1, characterized in that, The step of segmenting the data stream to be identified into a continuous sequence of text slices using a preset sliding window and generating a global coordinate index for each text slice in the data stream to be identified includes: The data stream to be identified is segmented using a sliding window with a preset overlap step size to obtain multiple segmented texts; Each of the segmented texts is corrected to obtain multiple text slices, and an index number is added to each text slice to generate a global coordinate index; A continuous sequence of text slices is formed based on each of the text slices.

3. The method according to claim 2, characterized in that, The step of correcting each of the segmented texts to obtain multiple text slices includes: In each of the segmented texts, based on the text content of the original data stream to be identified, the N+1th segmented text is advanced by the overlap step to form a corrected segmented text, where N is an integer ≥1; Determine whether the text corresponding to the overlap step of the corrected segmented text is complete; In the case of completeness, the corrected segmented text is taken as the text slice; In the case of incompleteness, based on the text content of the original data stream to be identified, the corrected N+1th segmented text is advanced by the overlap step to form a corrected segmented text, and the corrected segmented text is used as the text slice.

4. The method according to claim 3, characterized in that, The step of deduplicating sensitive entities corresponding to the same global position within the overlapping area of ​​adjacent text slices based on the confidence score, the global start coordinate, and the global end coordinate, and generating a sensitive entity list, includes: Obtain a first sensitive entity and a second sensitive entity within the overlapping region, wherein the first sensitive entity comes from the Nth text slice and the second sensitive entity comes from the (N+1)th text slice; Compare the confidence score of the Nth text slice with the confidence score of the (N+1)th text slice, select the sensitive entity corresponding to the text slice with the highest confidence score, and generate a list of sensitive entities.

5. The method according to claim 2, characterized in that, The sensitive entity list based on the de-identification, the global termination coordinates, and the global coordinate index are reassembled in a streaming manner to obtain the identified de-identified data stream, including: If the text slice is not the last slice, a safe commit boundary is determined based on the global termination coordinates of the text slice and the overlap step size; Obtain a global output cursor, which is located after the security commit boundary. Lock the data range to be processed between the global output cursor and the security commit boundary, and map the de-identified sensitive entity list to the data range to be processed. If the data within the data range to be processed falls within the range of sensitive entities, the identified desensitized data will be output. If the data within the data range to be processed does not fall within the range of sensitive entities, the data to be identified is output directly. After the output is completed, the global output cursor is advanced to the position of the safe commit boundary, so that the processing of the next text slice continues directly from the position of the global output cursor. If the text slice is the last slice, the identified de-identified data stream is output.

6. The method according to claim 5, characterized in that, The determination of the safe submission boundary based on the global termination coordinates and the overlap step size of the text slice includes: The safe submission limit is obtained by subtracting the global termination coordinate from the overlap step size.

7. The method according to claim 1, characterized in that, The dynamic assembly of the corresponding prompt word template based on the metadata features includes: The metadata features are matched with preset prompt types to obtain matching results; If the matching result indicates a key identification type, select a security prompt word corresponding to the key identification type from the preset sensitive identification template and assemble them to form a prompt word template; If the matching result indicates an identity recognition type, select identity information prompt words corresponding to the identity recognition type from the preset sensitive recognition template and assemble them to form a prompt word template.

8. A context-aware sensitive information identification system, characterized in that, The system includes: The data acquisition module is used to acquire the original data stream to be identified, divide the data stream to be identified into a continuous sequence of text slices using a preset sliding window, and generate a global coordinate index for each text slice in the data stream to be identified. The head of the (N+1)th text slice in the text slice sequence overlaps with the Nth text slice. A dynamic generation module is used to identify the metadata features of the data stream to be identified, and dynamically assemble the corresponding prompt word template based on the metadata features. The prompt word template contains sensitive entity types and does not contain preset regular expression rules. The data analysis module is used to input the text slice and the prompt word template into a preset context-aware model to perform semantic reasoning, and output the sensitive entities in the text slice that match the preset sensitive keywords, the local start coordinates and local end coordinates of each sensitive entity in the text slice, and the confidence score corresponding to the sensitive entity. The data processing module is used to convert the local start coordinates and local end coordinates of each sensitive entity into global start coordinates and global end coordinates based on the global coordinate index, and to deduplicate the sensitive entities corresponding to the same global position in the overlapping area of ​​adjacent text slices according to the confidence score, the global start coordinates and the global end coordinates, and generate a list of sensitive entities. The desensitization generation module is used to desensitize the sensitive entity list using a preset security protection strategy, and perform streaming reconstruction based on the desensitized sensitive entity list, the global termination coordinates, and the global coordinate index to obtain the identified desensitized data stream.

9. An electronic device, characterized in that, The device includes a processor, a memory, a user interface, a communication bus, and a network interface. The processor, the memory, the user interface, and the network interface are respectively connected to the communication bus. The memory is used to store instructions. The user interface and the network interface are used to communicate with other devices. The processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed, perform the method as described in any one of claims 1-7.

Images