Sensitive information detection method and device, electronic equipment and storage medium

Semantic segmentation processing and synchronous asynchronous detection methods are performed through sliding windows, which solves the problems of semantic information loss and slow response speed in sensitive information detection in streaming output of large models, and realizes efficient sensitive information detection and improves user experience.

CN119005179APending Publication Date: 2024-11-22CHINA TELECOM ARTIFICIAL INTELLIGENCE TECHNOLOGY (BEIJING) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410707727.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-06-03
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The streaming output of large models has problems such as semantic information loss and slow response speed in sensitive information detection, resulting in poor user experience and traditional detection methods cannot meet the needs of fast response.

Method used

The sliding window is used to perform semantic segmentation processing, combined with synchronous and asynchronous detection methods, the first sensitive word detection is first performed, and then the second sensitive word detection is performed on the semantic paragraph to ensure the accuracy and response speed of the detection results.

Benefits of technology

It realizes efficient detection of sensitive information, ensures user experience, and avoids the problems of semantic information loss and slow response speed, and meets the different demands of detection speed and output speed.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119005179A_ABST
    Figure CN119005179A_ABST
Patent Text Reader

Abstract

The invention discloses a sensitive information detection method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining input information of a target object, carrying out the first sensitive word detection of the input information, and transmitting the input information to a preset large model to obtain output information when the detection passes; the output information is stored in an output cache in sequence, semantic segmentation processing is carried out according to the storage content of each time, and then the output information is transmitted to a semantic segmentation queue; obtaining a semantic paragraph from the semantic segmentation queue through the sliding window, and taking the semantic paragraph in the sliding window at the current moment as a target semantic paragraph; and carrying out second sensitive word detection on the target semantic paragraph, carrying out sensitive information detection on the target semantic paragraph when the detection is passed, pushing the target semantic paragraph to the target object when the detection is passed, and then continuously detecting subsequent paragraphs until all the semantic paragraphs are pushed to the target object. The method can efficiently realize sensitive information detection, and can be widely applied to the technical field of data processing.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a sensitive information detection method, device, electronic device and storage medium. Background Art

[0002] As a cutting-edge AI technology that has received much attention today, large-scale model technology has shown broad application prospects in various fields. However, the output results of most large-scale models are often uncertain before they are generated, which means that the results may be unpredictable. There are certain risks in directly displaying these output results to users, because sensitive information may be leaked, which may lead to policy risks or public opinion influences, which may affect the development of large-scale model technology and related industries.

[0003] At the same time, there are several major problems in the large-scale model streaming output detection:

[0004] 1. Streaming output is in units of words or characters, while traditional sensitive information detection is usually performed on whole sentences or paragraphs. Directly applying it to streaming output cannot meet the requirements of the detection interface. Excessive segmentation may lead to loss of semantic information, thus affecting the accuracy of the detection results. If the entire large model is output and directly tested, it may cause users to wait too long, which has a significant impact on the user experience.

[0005] 2. Large-scale model output is continuous. In order to ensure user experience, the results need to be output to the user page quickly. However, traditional sensitive information detection usually has complex full-process detection logic and slow response speed, which cannot meet the needs of rapid response. Summary of the invention

[0006] The present invention proposes a sensitive information detection method, device, electronic device and storage medium, aiming to solve one of the technical problems in the related art at least to a certain extent.

[0007] On the one hand, an embodiment of the present invention provides a sensitive information detection method, including:

[0008] Obtain input information of the target object, and perform first sensitive word detection on the input information;

[0009] When the result of the first sensitive detection is passed, the input information is transmitted to the preset large model for processing to obtain output information; the output information is in the form of streaming output;

[0010] The output information of the streaming output is sequentially stored in the output buffer, the output information is semantically segmented according to the content stored in the output buffer each time, and the segmented output information is transmitted to the semantic segmentation queue;

[0011] Obtain semantic paragraphs from the semantic segmentation queue through a sliding window, and use the semantic paragraph in the sliding window at the current moment as the target semantic paragraph;

[0012] Detect the second sensitive words in the target semantic paragraph;

[0013] When the result of the second sensitive detection is passed, sensitive information detection is performed on the target semantic paragraph;

[0014] When the result of sensitive information detection is passed, the target semantic paragraph is pushed to the target object, the semantic paragraph in the sliding window at the next moment is used as the target semantic paragraph, and then the step of performing the second sensitive word detection on the target semantic paragraph is returned until all semantic paragraphs corresponding to the output information are pushed to the target object.

[0015] Optionally, performing first sensitive word detection on the input information includes at least one of the following:

[0016] Traverse and match the input information with the preset sensitive word library;

[0017] Transfer the input information to the preset sensitive word detection tool.

[0018] Optionally, the method further comprises the following steps:

[0019] When the result of the first sensitive detection is failure, the preset information is sent to the target object.

[0020] Optionally, semantic segmentation processing is performed on the output information according to the content stored in the output buffer each time, and the segmented output information is transmitted to the semantic segmentation queue, including the following steps:

[0021] Output the current large model to the output buffer in a streaming manner as the target content;

[0022] Perform segmentation symbol matching on the target content;

[0023] When there is no segmentation symbol in the target content, the target content is transferred to the semantic segmentation queue and appended to the end of the previous semantic paragraph;

[0024] When there are segmentation symbols in the target content, a new semantic paragraph is opened in the semantic segmentation queue based on the segmentation symbols;

[0025] The content stored in the output buffer of the next big model stream output is used as the target content, and then the target content is returned to the step of segmentation symbol matching to continue semantic segmentation processing of the output information.

[0026] Optionally, before the step of performing sensitive information detection on the target semantic paragraph, the method further includes the following steps:

[0027] The semantic paragraphs within a preset range before the target semantic paragraph in the semantic segmentation queue are added to the target semantic paragraph.

[0028] Optionally, the method further comprises the following steps:

[0029] When the result of the second sensitivity detection is failure, determining the sensitive words in the target semantic paragraph according to the result of the second sensitivity detection;

[0030] The sensitive words in the target semantic paragraph are desensitized, and the desensitized target semantic paragraph is pushed to the target object.

[0031] Optionally, the method further comprises the following steps:

[0032] When the result of sensitive information detection is failure, all semantic paragraphs pushed to the target object are displayed and rolled back, and then the preset information is sent to the target object.

[0033] On the other hand, an embodiment of the present invention provides a sensitive information detection device, including:

[0034] The first module is used to obtain input information of the target object and perform a first sensitive word detection on the input information;

[0035] The second module is used for transmitting the input information to a preset large model for processing to obtain output information when the result of the first sensitive detection is passed; the output information is in the form of streaming output;

[0036] The third module is used to store the output information of the streaming output in the output buffer in sequence, perform semantic segmentation processing on the output information according to the content stored in the output buffer each time, and transmit the segmented output information to the semantic segmentation queue;

[0037] The fourth module is used to obtain a semantic paragraph from the semantic segmentation queue through a sliding window, and use the semantic paragraph in the sliding window at the current moment as the target semantic paragraph;

[0038] The fifth module is used to detect the second sensitive words in the target semantic paragraph;

[0039] The sixth module is used to perform sensitive information detection on the target semantic paragraph when the result of the second sensitive detection is passed;

[0040] The seventh module is used to push the target semantic paragraph to the target object when the result of sensitive information detection is passed, use the semantic paragraph in the sliding window at the next moment as the target semantic paragraph, and then return to execute the fifth module until all semantic paragraphs corresponding to the output information are pushed to the target object.

[0041] Optionally, the first module is specifically used for at least one of the following:

[0042] Traverse and match the input information with the preset sensitive word library;

[0043] Transfer the input information to the preset sensitive word detection tool.

[0044] Optionally, the device further comprises:

[0045] The eighth module sends preset information to the target object when the result of the first sensitive detection is failure.

[0046] Optionally, the third module is specifically used for:

[0047] Output the current large model to the output buffer in a streaming manner as the target content;

[0048] Perform segmentation symbol matching on the target content;

[0049] When there is no segmentation symbol in the target content, the target content is transferred to the semantic segmentation queue and appended to the end of the previous semantic paragraph;

[0050] When there are segmentation symbols in the target content, a new semantic paragraph is opened in the semantic segmentation queue based on the segmentation symbols;

[0051] The content stored in the output buffer of the next big model stream output is used as the target content, and then the target content is returned to the step of segmentation symbol matching to continue semantic segmentation processing of the output information.

[0052] Optionally, the device further comprises:

[0053] The ninth module is used to add the semantic paragraphs in the semantic segmentation queue that are within a preset range before the target semantic paragraph to the target semantic paragraph. The ninth module is executed before the sixth module executes the step of performing sensitive information detection on the target semantic paragraph.

[0054] Optionally, the device further comprises:

[0055] A tenth module is used to determine the sensitive words in the target semantic paragraph according to the result of the second sensitivity detection when the result of the second sensitivity detection is failed;

[0056] The eleventh module is used to desensitize the sensitive words in the target semantic paragraph and push the desensitized target semantic paragraph to the target object.

[0057] Optionally, the device further comprises:

[0058] The twelfth module is used to display and roll back all semantic paragraphs pushed to the target object when the result of sensitive information detection is failed, and then send preset information to the target object.

[0059] On the other hand, an embodiment of the present invention provides an electronic device, including: a processor and a memory; the memory is used to store programs; the processor executes the program to implement the above-mentioned sensitive information detection method.

[0060] On the other hand, an embodiment of the present invention provides a computer storage medium, which stores a program executable by a processor, and the program executable by the processor is used to implement the above-mentioned sensitive information detection method when executed by the processor.

[0061] The embodiment of the present invention obtains input information of a target object and performs a first sensitive word detection on the input information; when the result of the first sensitive detection is passed, the input information is transmitted to a preset large model for processing to obtain output information; the output information is in the form of streaming output; the output information of the streaming output is sequentially stored in an output cache, the output information is semantically segmented according to the content stored in the output cache each time, and the segmented output information is transmitted to a semantic segmentation queue; a semantic paragraph is obtained from the semantic segmentation queue through a sliding window, and the semantic paragraph in the sliding window at the current moment is used as the target semantic paragraph; a second sensitive word detection is performed on the target semantic paragraph; when the result of the second sensitive detection is passed, a sensitive information detection is performed on the target semantic paragraph; when the result of the sensitive information detection is passed, the target semantic paragraph is pushed to the target object, the semantic paragraph in the sliding window at the next moment is used as the target semantic paragraph, and then the step of performing a second sensitive word detection on the target semantic paragraph is returned until all semantic paragraphs corresponding to the output information are pushed to the target object. The embodiment of the present invention continuously organizes semantic paragraphs for output information, and then combines sensitive word detection and sensitive information detection of the target paragraph to implement a synchronous and asynchronous detection method to meet the different demands of detection speed and output speed. The embodiment of the present invention can efficiently realize sensitive information detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] The accompanying drawings are used to provide a further understanding of the technical solution of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the technical solution of the present invention and do not constitute a limitation on the technical solution of the present invention.

[0063] Figure 1 This is a schematic diagram of an implementation environment for sensitive information detection provided by an embodiment of the present invention;

[0064] Figure 2 is a schematic diagram of an example process of a sensitive information detection method provided by an embodiment of the present invention;

[0065] Figure 3 A schematic diagram of the expansion process of the first sensitive word detection provided by an embodiment of the present invention;

[0066] Figure 4A schematic diagram of another process example of a sensitive information detection method provided in an embodiment of the present invention;

[0067] Figure 5 A schematic diagram of the expanded flow of semantic segmentation processing provided by an embodiment of the present invention;

[0068] Figure 6 A schematic diagram of another process example of a sensitive information detection method provided in an embodiment of the present invention;

[0069] Figure 7 A schematic diagram of another process example of a sensitive information detection method provided in an embodiment of the present invention;

[0070] Figure 8 A schematic diagram of the overall process principle of sensitive information detection provided by an embodiment of the present invention;

[0071] Fig. 9 A schematic diagram of the structure of a sensitive information detection device provided by an embodiment of the present invention;

[0072] Fig.10 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0073] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0074] It should be noted that, although the functional modules are divided in the system schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the system or the order in the flowchart. The terms "first / S100", "second / S200", etc. in the specification, claims and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0075] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0076] It is understandable that the sensitive information detection method provided in the embodiment of the present invention can be applied to any computer device with data processing and computing capabilities, and this computer device can be various types of terminals or servers. When the computer device in the embodiment is a server, the server is an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Optionally, the terminal is a smart phone, a tablet computer, a laptop computer, a desktop computer, etc., but is not limited to this.

[0077] To facilitate the understanding of the technical solution of the present invention, the technical feature terms that may appear in the embodiments of the present invention are first explained:

[0078] Streaming output: It is a data processing method that transmits and processes data in the form of streams and keeps it flowing. This processing method can complete data processing during the flow process, so it can process data in real time and improve data processing efficiency.

[0079] Rollback: refers to the act of restoring a program or data to the most recent correct version when an error occurs. In distributed business compensation design, rollback is to return to the state before the service call through transaction compensation.

[0080] like Figure 1 FIG. 1 is a schematic diagram of an implementation environment provided by an embodiment of the invention. Figure 1 The implementation environment includes at least one terminal 102 and a server 101. The terminal 102 and the server 101 can be connected to a network wirelessly or wired to complete data transmission and exchange.

[0081] Server 101 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as big data and artificial intelligence platforms.

[0082] In addition, the server 101 can also be a node server in the blockchain network. Blockchain is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, and encryption algorithm.

[0083] The terminal 102 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto. The terminal 102 and the server 101 may be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiment of the present invention.

[0084] Based on the example Figure 1 In the implementation environment shown, an embodiment of the present invention provides a sensitive information detection method. The following is explained using the example of the sensitive information detection method being applied in the server 101. It can be understood that the sensitive information detection method can also be applied in the terminal 102.

[0085] Reference Figure 2 , Figure 2 The flowchart of the sensitive information detection method applied to the server provided in the embodiment of the present invention, the execution subject of the sensitive information detection method can be any of the aforementioned computer devices (including servers or terminals). Figure 2 , the method comprises the following steps:

[0086] S100, obtaining input information of a target object, and performing a first sensitive word detection on the input information;

[0087] It should be noted that in some embodiments, Figure 3 As shown, performing a first sensitive word detection on the input information may include at least one of the following: S101, traversing and matching the input information with a preset sensitive word library; S102, transmitting the input information to a preset sensitive word detection tool.

[0088] It should be noted that the sensitive word detection is packaged based on the existing sensitive information detection system, and the detection points are split. The specific detection means are not the main content of the present invention and will not be described here.

[0089] S200, when the result of the first sensitive detection is passed, the input information is transmitted to the preset large model for processing to obtain output information;

[0090] Among them, the output information is in the form of streaming output;

[0091] It should be noted that the large model refers to a large model in the application scenario of the embodiment of the present invention. The embodiment of the present invention detects the input and output data of the model, and does not limit the architectural composition and data processing logic of the model, which will not be repeated here.

[0092] Among them, in some embodiments, such as Figure 4As shown, the method may further include the following steps: S800, when the result of the first sensitive detection is failure, sending preset information to the target object.

[0093] For example, in some specific implementations, the input information is tested for common sensitive words, and if the test result fails, the system preset answer words are directly returned. Specifically, in some implementations, an asynchronous detection task (sensitive information detection) can also be submitted for the input information, and the result is written into the detection result queue after the asynchronous detection is completed.

[0094] S300, storing the output information of the streaming output in the output buffer in sequence, performing semantic segmentation processing on the output information according to the content stored in the output buffer each time, and transmitting the segmented output information to the semantic segmentation queue;

[0095] It should be noted that in some embodiments, Figure 5 As shown, the output information is semantically segmented according to the content stored in the output cache each time, and the segmented output information is transmitted to the semantic segmentation queue, which may include the following steps: S301, the content stored in the output cache streamed by the large model at the current moment is used as the target content; S302, the segmentation symbol is matched for the target content; S303, when there is no segmentation symbol in the target content, the target content is transmitted to the semantic segmentation queue and attached to the end of the previous semantic paragraph; S304, when there is a segmentation symbol in the target content, a new semantic paragraph is opened in the semantic segmentation queue based on the segmentation symbol; S305, the content stored in the output cache streamed by the large model at the next moment is used as the target content, and then the step of matching the segmentation symbol for the target content is returned to continue the semantic segmentation of the output information.

[0096] Exemplarily, in some specific implementations, after the user input is detected, the large model is called to answer the user's question. The results of the large model are presented in the form of streaming output. After obtaining the streaming output results, it is semantically segmented through the output buffer and then output processed. Among them, the semantic segmentation adopts a regular segmentation method. If there is no matching segmentation symbol in the current output buffer, the current output is appended to the end of the previous semantics, otherwise a new semantic paragraph is opened.

[0097] S400, obtaining a semantic paragraph from a semantic segmentation queue through a sliding window, and taking the semantic paragraph in the sliding window at the current moment as a target semantic paragraph;

[0098] Among them, it should be noted that in the synchronous inspection task (second sensitive word detection), each time the content is detected, a fixed sliding window method is used to obtain the content to be inspected, avoiding some common anti-sensitive word detection methods.

[0099] S500, performing second sensitive word detection on the target semantic paragraph;

[0100] It should be noted that, in some embodiments, performing a second sensitive word detection on a target semantic paragraph may include at least one of the following: traversing and matching the target semantic paragraph with a preset sensitive word library; transmitting the target semantic paragraph to a preset sensitive word detection tool.

[0101] It should be noted that the sensitive word detection is packaged based on the existing sensitive information detection system, and the detection points are split. The specific detection means are not the main content of the present invention and will not be described here.

[0102] For example, in some specific implementations, the core detection processing logic is first called to read the semantic paragraphs of the current output position sliding window from the semantic segmentation queue to perform synchronous sensitive word detection.

[0103] S600: When the result of the second sensitivity detection is passed, sensitive information detection is performed on the target semantic paragraph;

[0104] Exemplarily, in some specific implementations, if the synchronous detection result passes, the detection processing program will submit an asynchronous detection task (sensitive information detection).

[0105] It should be noted that both sensitive word detection and sensitive information detection are packaged based on the existing sensitive information detection system, and the detection points are split. The specific detection means are not the main content of the present invention and will not be described here.

[0106] Before the step of performing sensitive information detection on the target semantic paragraph, the method may further include the following steps: adding semantic paragraphs within a preset range before the target semantic paragraph in the semantic segmentation queue to the target semantic paragraph.

[0107] Exemplarily, in some specific implementations, if the synchronous detection result passes, the detection processing program will submit an asynchronous detection task. The detection content starts from the semantic paragraph of the last asynchronous detection to the previous position, and ends at the current latest complete paragraph. It should be noted that in the asynchronous detection task, the detection content overlaps with the last detection content by one semantic paragraph. This step mainly takes into account that semantic paragraph segmentation may lead to context loss, affecting the detection of sensitive word information.

[0108] Among them, in some embodiments, such as Figure 6 As shown, the method may also include the following steps: S900, when the result of the second sensitive detection is failure, determining the sensitive words in the target semantic paragraph according to the result of the second sensitive detection; S1000, desensitizing the sensitive words in the target semantic paragraph, and pushing the desensitized target semantic paragraph to the target object.

[0109] For example, in some specific implementations, if the detection result fails, the sensitive words are replaced with asterisks for shielding output.

[0110] S700. When the result of sensitive information detection is passed, the target semantic paragraph is pushed to the target object, the semantic paragraph in the sliding window at the next moment is used as the target semantic paragraph, and then the step of performing the second sensitive word detection on the target semantic paragraph is returned until all semantic paragraphs corresponding to the output information are pushed to the target object.

[0111] Among them, in some embodiments, such as Figure 7 As shown, the method may further include the following steps: S1100, when the result of sensitive information detection is failure, all semantic paragraphs pushed to the target object are displayed and rolled back, and then preset information is sent to the target object.

[0112] For example, in some specific implementations, after completing the detection process of the current paragraph, the output process verifies the detection result. If the detection result fails, the user output terminal is notified to roll back the display of the current answer, cancel all current answers, and display the preset sensitive information detection failure words.

[0113] In order to explain the principle of the technical solution of the present invention in detail, the overall process of the present invention is described below in combination with some specific embodiments. It is easy to understand that the following is an explanation of the technical principle of the present invention and cannot be regarded as a limitation of the present invention.

[0114] First of all, it should be noted that the present invention aims to solve the problem that traditional sensitive information detection methods cannot be directly used in large-scale model streaming output scenarios and have a high efficiency. At the same time, in view of the problems existing in the detection of large-scale model streaming output, it is necessary to conduct strict sensitive information detection on the input and output of large models, and only after the detection is passed can the results be displayed to customers. This measure is intended to ensure information security and compliance to promote the healthy development of large-scale model technology.

[0115] For example, for traditional sensitive word detection, the embodiment of the present invention splits its functions and decomposes the task originally handled by a single interface into two steps. First, one interface is specifically responsible for the detection of the traditional sensitive word library. This interface is only responsible for the detection of common sensitive words, so the speed is relatively fast and a synchronous detection method is adopted. The other interface is responsible for the detection of comprehensive sensitive information. Its internal processing flow is relatively complex, so the overall speed is slow and an asynchronous detection method is adopted. The main processing flow of the present invention is as follows: Figure 8 As shown, the core processing steps are as follows:

[0116] 1. After obtaining user input, the input processing step will start the detection process to perform user input detection. This detection is divided into two steps: first, submit an asynchronous detection task, and write the result to the detection result queue after the asynchronous detection is completed; second, perform synchronous common sensitive word detection. If the synchronous detection result fails, the system preset answer will be directly returned.

[0117] 2. After the user input is detected, the large model is called to answer the user's question. The results of the large model are presented in the form of streaming output. After obtaining the streaming output results, it is semantically segmented through the output buffer and then output processed.

[0118] 3. The output processing program first calls the core detection processing logic, reads the semantic paragraph of the current output position sliding window from the semantic segmentation queue, and performs synchronous sensitive word detection. If the detection result fails, the sensitive word is replaced with an asterisk to block the output.

[0119] 4. If the synchronous detection result is passed, the detection processing program will submit the asynchronous detection task. The detection content starts from the semantic paragraph of the last asynchronous detection to the previous position, and ends at the latest complete paragraph. The content is submitted to the sensitive word information detection program for detection. After the asynchronous detection is completed, the detection conclusion is written back to the detection result queue.

[0120] 5. After the detection process is completed, the output process verifies the detection result. If the detection result fails, the user output terminal is notified to roll back the current answer, cancel all current answers, and display the preset sensitive information detection failure words.

[0121] The above main processing steps are completed in the core detection processing module, which is also the main content of the present invention. In addition, the following is a brief supplement to some core processing logic:

[0122] 1. Semantic segmentation adopts regular segmentation. If there is no matching segmentation symbol in the current output buffer, the current output will be appended to the end of the previous semantics, otherwise a new semantic paragraph will be opened.

[0123] 2. In the synchronous inspection task, a fixed sliding window is used to obtain the content for inspection each time, avoiding some common anti-sensitive word detection methods.

[0124] 3. In the asynchronous detection task, the detected content has a semantic paragraph overlap with the previous detected content. This step is mainly considered that semantic paragraph segmentation may cause context loss and affect the detection of sensitive word information.

[0125] 4. Sensitive word detection and sensitive information detection are both packaged based on the existing sensitive information detection system, and the key points of detection are separated. The specific detection means are not the main content of the present invention.

[0126] The following is an explanation of interception using several common interception scenarios. This explanation is only for example reference and does not represent the specific input and output of any large model.

[0127] 1. Scenario 1: User input contains sensitive words:

[0128] a) User input: What is the process of XX? (XX is a preset sensitive word)

[0129] b) Submit the asynchronous check task of user input in core step 1 and perform synchronous check at the same time.

[0130] c) During the synchronous check, it is detected that the sensitive word "XX" exists in the user input, and the check result is placed in the check result queue as input check failure.

[0131] d) The output processing program checks the detection result queue before outputting content to the user, identifies that the input check fails, and replaces the large model output with the preset words: "I cannot answer your question" to output to the user.

[0132] 2. Scenario 2: The large model output contains sensitive words:

[0133] a) User output: What is the application principle of YY?

[0134] b) Assume that the large model outputs in sequence:

[0135] i.YY;

[0136] ii.

[0137] iii. Application principles;

[0138] iv. As follows:;

[0139] v.Implemented based on XX.

[0140] c) Each output of the large model will be stored in the output buffer, and detection will be performed during the output. In this example, the content submitted for each output detection is as follows:

[0141] i.YY;

[0142] ii. YY;

[0143] iii. Application principle of YY;

[0144] iv. The application principle of YY is as follows:;

[0145] v. The application principle of YY is as follows: based on XX;

[0146] d) In the last test, the sensitive word "XX" will be detected, and then the output result will block "XX". The final output result is as follows

[0147] i.YY's method flow is as follows: based on ** implementation.

[0148] 3. Scenario 3: Large model output contains sensitive information:

[0149] a) User input: How to implement YY’s method flow?

[0150] b) Assume that the total output of the large model is as follows:

[0151] i.YY;

[0152] ii.

[0153] iii. Methodology flow;

[0154] iv. As follows:;

[0155] v.First step;

[0156] vii. Should ZZ;

[0157] viii. Step 2;

[0158] ix. Should XX;

[0159] c) The output of the large model will be semantically segmented, and the segmentation queue is as follows:

[0160] i. The exercise method of this fitness method is as follows:;

[0161] ii. The first step should be ZZ;

[0162] iii. The second step should be XX;

[0163] d) Synchronous semantic detection was performed and no sensitive information was detected.

[0164] e) Perform asynchronous semantic detection, with a sliding semantic window size of 2. The content of each inspection is as follows:

[0165] i. The exercise method of this fitness method is as follows:

[0166] ii. The exercise method of this fitness method is as follows: \nThe first step should be ZZ;

[0167] iii. The first step should be ZZ\nThe second step should be XX;

[0168] f) Assuming that the sensitive information detection detects that it is sensitive information content in the second step, the detection result will be recorded in the detection result queue after the second detection. Before the next output, the output program detects that there is output that has not passed the inspection, and the output program will output an instruction to overwrite the result to the user page.

[0169] g) After the user page receives the overwrite instruction, the current answer will be cleared and the preset words will be displayed: "I cannot answer your question."

[0170] In summary, the present invention adopts a method of multiple continuous detections and text semantic segmentation, and combines synchronous and asynchronous detections to solve the user experience problem. In this way, it can not only ensure the user experience of large-scale model applications, but also ensure complete security monitoring, and effectively eliminate various risks and hidden dangers. Specifically, the present invention can not only quickly detect common sensitive words, but also detect complex sensitive information, and can maintain user experience without excessively demanding detection speed; at the same time, the embodiment of the present invention performs semantic segmentation and division for streaming output, and the detection content will adopt a semantic end overlap method for each detection, which can ensure that the original semantics are not lost as much as possible; and the present invention can realize detection while outputting, avoiding user waiting time and improving user experience.

[0171] Compared with the prior art, the main features of the embodiments of the present invention are: 1. Abandoning the traditional synchronous detection method, multiple checks are performed in a synchronous and asynchronous manner. 2. The detected content is segmented into semantic paragraphs, and each time the detected content is overlapped to a certain extent, the semantic information across the terminals will not be lost.

[0172] Compared with the prior art, the present invention has at least the following beneficial effects:

[0173] 1. The present invention adopts a detection method combining synchronous and asynchronous to meet the different demands of detection speed and output speed.

[0174] 2. The synchronous detection of the present invention adopts the content of the sliding paragraph window for inspection, and the asynchronous detection adopts the content of overlapping paragraphs for inspection, so as to solve the problem of losing complete semantics in the case of streaming output of single words or phrases.

[0175] 3. The present invention combines asynchronous detection notification with a user output terminal display rollback solution to solve the problem of delayed output of asynchronous detection results.

[0176] On the other hand, Fig. 9As shown, an embodiment of the present invention provides a sensitive information detection device 900, including: a first module 910, used to obtain input information of a target object and perform a first sensitive word detection on the input information; a second module 920, used to transmit the input information to a preset large model for processing to obtain output information when the result of the first sensitive detection is passed; the output information is in the form of streaming output; a third module 930, used to store the output information of the streaming output in an output cache in sequence, perform semantic segmentation processing on the output information according to the storage content of each time in the output cache, and transmit the segmented output information to a semantic segmentation queue; a fourth module 940, Used to obtain semantic paragraphs from the semantic segmentation queue through a sliding window, and use the semantic paragraph in the sliding window at the current moment as the target semantic paragraph; the fifth module 950 is used to perform a second sensitive word detection on the target semantic paragraph; the sixth module 960 is used to perform a sensitive information detection on the target semantic paragraph when the result of the second sensitive detection is passed; the seventh module 970 is used to push the target semantic paragraph to the target object when the result of the sensitive information detection is passed, use the semantic paragraph in the sliding window at the next moment as the target semantic paragraph, and then return to execute the fifth module until all semantic paragraphs corresponding to the output information are pushed to the target object.

[0177] In some embodiments, the first module may be used for at least one of the following: traversing and matching the input information with a preset sensitive word library; transmitting the input information to a preset sensitive word detection tool.

[0178] In some embodiments, the device may further include: an eighth module, which sends preset information to the target object when the result of the first sensitive detection is failure.

[0179] In some embodiments, the third module can be used to: use the content stored in the output cache streamed by the large model at the current moment as the target content; match the segmentation symbols on the target content; when there is no segmentation symbol in the target content, transfer the target content to the semantic segmentation queue and append it to the end of the previous semantic paragraph; when there is a segmentation symbol in the target content, start a new semantic paragraph in the semantic segmentation queue based on the segmentation symbol; use the content stored in the output cache streamed by the large model at the next moment as the target content, and then return to the step of matching the segmentation symbols on the target content, and continue to perform semantic segmentation processing on the output information.

[0180] In some embodiments, the device may further include: a ninth module, configured to add semantic paragraphs within a preset range before the target semantic paragraph in the semantic segmentation queue to the target semantic paragraph. The ninth module is executed before the sixth module executes the step of detecting sensitive information on the target semantic paragraph.

[0181] In some embodiments, the device may also include: a tenth module, which is used to determine the sensitive words in the target semantic paragraph according to the result of the second sensitive detection when the result of the second sensitive detection is failed; an eleventh module, which is used to desensitize the sensitive words in the target semantic paragraph and push the desensitized target semantic paragraph to the target object.

[0182] In some embodiments, the device may further include: a twelfth module, which is used to display and roll back all semantic paragraphs pushed to the target object when the result of the sensitive information detection is failed, and then send preset information to the target object.

[0183] The contents of the method embodiments of the present invention are all applicable to the device embodiments. The functions specifically implemented by the device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0184] On the other hand, an embodiment of the present invention further provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above sensitive information method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0185] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0186] like Fig.10 As shown, Fig.10 A specific example of the hardware structure of an electronic device according to an embodiment is shown, and the electronic device includes:

[0187] The processor 1001 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention;

[0188] The memory 1002 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1002 can store an operating system and other applications. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1002, and the processor 1001 calls and executes the network node population optimization method of the embodiment of the present invention;

[0189] Input / output interface 1003, used to implement information input and output;

[0190] The communication interface 1004 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0191] A bus 1005 , which transmits information between various components of the device (e.g., the processor 1001 , the memory 1002 , the input / output interface 1003 , and the communication interface 1004 );

[0192] The processor 1001 , the memory 1002 , the input / output interface 1003 and the communication interface 1004 are connected to each other in communication within the device via the bus 1005 .

[0193] The electronic device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0194] The contents of the method embodiments of the present invention are all applicable to the electronic device embodiments. The functions specifically implemented by the electronic device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0195] Another aspect of an embodiment of the present invention further provides a computer-readable storage medium, wherein the storage medium stores a program, and the program is executed by a processor to implement the above method.

[0196] It should be noted that the computer-readable medium shown in the embodiment of the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (Compact Disc Read to Only Memory, CD to ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0197] The contents of the method embodiments of the present invention are all applicable to the computer-readable storage medium embodiments. The functions specifically implemented by the computer-readable storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0198] The embodiment of the present invention also discloses a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device can read the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the above method.

[0199] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0200] It should be noted that, although several modules of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to an embodiment of the present invention, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0201] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD to ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation of the present invention.

[0202] In some selectable embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part for which is described as a larger operation is performed independently.

[0203] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise specified, one or more of the functions and / or features can be integrated into a single physical device and / or software module, or one or more functions and / or features can be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present invention. More specifically, in view of the properties, functions and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional skills of the engineer. Therefore, those skilled in the art can implement the present invention set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0204] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0205] The logic and / or steps represented in the flowchart or otherwise described herein, for example, may be considered as an ordered list of executable instructions for implementing logical functions, and may be embodied in any computer-readable medium for use by an instruction execution device, apparatus or device (such as a computer-based device, a device including a processor, or other device that can fetch instructions from an instruction execution device, apparatus or device and execute the instructions), or in conjunction with such instruction execution device, apparatus or device. For purposes of this specification, "computer-readable medium" may be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution device, apparatus or device, or in conjunction with such instruction execution device, apparatus or device.

[0206] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0207] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution device. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0208] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0209] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

[0210] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present invention.

Claims

1. A sensitive information detection method, characterized in that: The following steps are involved: Acquire input information of a target object, and perform first sensitive word detection on the input information; When the result of the first sensitive detection is passed, the input information is transmitted to a preset large model for processing to obtain output information; the output information is in the form of streaming output; The output information of the streaming output is sequentially stored in the output buffer, the output information is semantically segmented according to the content stored in the output buffer each time, and the segmented output information is transmitted to the semantic segmentation queue; Obtaining a semantic paragraph from the semantic segmentation queue through a sliding window, and taking the semantic paragraph in the sliding window at the current moment as a target semantic paragraph; Performing second sensitive word detection on the target semantic paragraph; When the result of the second sensitive detection is passed, performing sensitive information detection on the target semantic paragraph; When the result of the sensitive information detection is passed, the target semantic paragraph is pushed to the target object, the semantic paragraph in the sliding window at the next moment is used as the target semantic paragraph, and then the step of performing a second sensitive word detection on the target semantic paragraph is returned until all semantic paragraphs corresponding to the output information are pushed to the target object.

2. The sensitive information detection method according to claim 1, characterized in that: The performing first sensitive word detection on the input information includes at least one of the following: Traversing and matching the input information with a preset sensitive word library; The input information is transmitted to a preset sensitive word detection tool.

3. The sensitive information detection method according to claim 1 or 2, characterized in that: The method further comprises the following steps: When the result of the first sensitive detection is failure, preset information is sent to the target object.

4. The sensitive information detection method according to claim 1, characterized in that: The step of performing semantic segmentation processing on the output information according to the content stored in the output buffer each time, and transmitting the segmented output information to the semantic segmentation queue comprises the following steps: Outputting the large model at the current moment in a streaming manner to the stored content in the output buffer as the target content; Perform segmentation symbol matching on the target content; When the segmentation symbol does not exist in the target content, the target content is transferred to the semantic segmentation queue and appended to the end of the previous semantic paragraph; When the segmentation symbol exists in the target content, opening a new semantic paragraph in the semantic segmentation queue based on the segmentation symbol; The stored content in the output buffer streamed by the large model at the next moment is used as the target content, and then the step of matching the segmentation symbols of the target content is returned to continue the semantic segmentation processing of the output information.

5. The sensitive information detection method according to claim 1, characterized in that: Before the step of performing sensitive information detection on the target semantic paragraph, the method further includes the following steps: The semantic paragraphs within a preset range before the target semantic paragraph in the semantic segmentation queue are added to the target semantic paragraph.

6. The sensitive information detection method according to claim 1, characterized in that: The method further comprises the following steps: When the result of the second sensitivity detection is failure, determining the sensitive words in the target semantic paragraph according to the result of the second sensitivity detection; The sensitive words in the target semantic paragraph are desensitized, and the desensitized target semantic paragraph is pushed to the target object.

7. The sensitive information detection method according to any one of claims 1 to 6, characterized in that: The method further comprises the following steps: When the result of the sensitive information detection is failure, all semantic paragraphs pushed to the target object are displayed and rolled back, and then preset information is sent to the target object.

8. A sensitive information detection device, characterized in that: include: The first module is used to obtain input information of a target object and perform a first sensitive word detection on the input information; The second module is used for transmitting the input information to a preset large model for processing to obtain output information when the result of the first sensitive detection is passed; the output information is in the form of streaming output; The third module is used to store the output information of the streaming output in the output buffer in sequence, perform semantic segmentation processing on the output information according to the stored content in the output buffer each time, and transmit the segmented output information to the semantic segmentation queue; The fourth module is used to obtain a semantic paragraph from the semantic segmentation queue through a sliding window, and use the semantic paragraph in the sliding window at a current moment as a target semantic paragraph; A fifth module is used to detect a second sensitive word on the target semantic paragraph; A sixth module, configured to perform sensitive information detection on the target semantic paragraph when the result of the second sensitive detection is passed; The seventh module is used to push the target semantic paragraph to the target object when the result of the sensitive information detection is passed, use the semantic paragraph in the sliding window at the next moment as the target semantic paragraph, and then return to execute the fifth module until all semantic paragraphs corresponding to the output information are pushed to the target object.

9. An electronic device, characterized in that: including a processor and a memory; The memory is used to store programs; The processor executes the program to implement the method according to any one of claims 1 to 7.

10. A computer storage medium storing a program executable by a processor, characterized in that: The program executable by the processor is used to implement the method according to any one of claims 1 to 7 when executed by the processor.