Content security identification method based on integration of multiple large language models

Abstract

The invention discloses a content security identification method based on integration of multiple large language models, and belongs to the technical field of data processing. The method comprises the steps of receiving a query text, carrying out word segmentation processing on the query text, generating query text segmented words, obtaining a sensitive data text matched with the query text segmented words in a sensitive database, splicing the query text and the sensitive data text based on a preset prompt word template, generating security identification prompt words of the query text, and sending the security identification prompt words to the sensitive database. And inputting the security identification cue word into the at least two large language models, obtaining output results of the large language models, integrating the output results, and determining a security identification result of the query text. According to the method, the sensitive data text is recalled through the sensitive database, the safe recognition cue words of the large language models are enhanced, and the accuracy of sensitive word recognition is improved through integrated processing of the multiple large language models.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a content security identification method based on the integration of multiple large language models. Background Art

[0002] The large language model can extract suspected sensitive words in the input text through text recognition, and perform part-of-speech analysis and review on the suspected sensitive words in a matching manner with the sensitive word library to achieve secure identification of the content of the input text.

[0003] In related technologies, the extraction of sensitive words mainly relies on the reasoning ability of the pre-trained large language model for sensitive word recognition. However, due to the large number of variants and metaphors in the input text, the large language model has a certain probability of hallucination when recognizing sensitive words and cannot correctly recognize sensitive words.

[0004] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0005] The main purpose of this application is to provide a content security identification method based on the integration of multiple large language models, aiming to solve the technical problem that content security identification of input text based on large language models cannot accurately identify sensitive words in text sentences.

[0006] To achieve the above objectives, the present application provides a content security identification method based on the integration of multiple large language models, wherein the method comprises the following steps: Receive a query text, and perform word segmentation processing on the query text to generate query text word segments; Obtaining sensitive data text that matches the query text segmentation in a sensitive database; Based on a preset prompt word template, the query text and the sensitive data text are spliced ​​to generate a security identification prompt word for the query text; Inputting the security identification prompt word into at least two large language models, and obtaining output results of the large language models; The output results are integrated to determine a security identification result of the query text.

[0007] In one embodiment, after the step of integrating the output results and determining the security identification result of the query text, the step further includes: When the security identification result is that a sensitive word exists, obtaining a target prompt word template; The format requirement text in the target prompt word template is concatenated with the query text to generate a sensitive word detection prompt word; Inputting the sensitive word detection prompt word into the large language model, and obtaining the detection text output by the large language model based on the output format corresponding to the format requirement text; Based on the output format, sensitive text in the detected text is identified.

[0008] In one embodiment, after the step of identifying the sensitive text in the detected text based on the output format, the method further includes: Obtaining sensitive words in the sensitive text; Calculating the similarity between the sensitive words and the sensitive data in the sensitive database, and selecting the sensitive words whose similarity is higher than a similarity threshold as target sensitive words; The target sensitive word is updated into the sensitive database.

[0009] In one embodiment, the step of obtaining sensitive words in the sensitive text includes: Obtaining sensitive data pairs of sensitive words and sensitive sentences in the sensitive text; Using a word segmentation tool, the sensitive sentence is segmented to obtain sensitive sentence segmentations; Based on the sensitive word segmentation and the sensitive sentence segmentation, the sensitive words in the sensitive text are determined.

[0010] In one embodiment, the step of integrating the output results and determining the security identification result of the query text includes: Generate security identification information by logically connecting the output results; The security identification result of the query text is determined according to the security identification information.

[0011] In one embodiment, the step of obtaining, in a sensitive database, a sensitive data text that matches the query text segmentation includes: Based on the query text segmentation, sensitive data is obtained from the sensitive database through inverted index retrieval; Calculating target similarity between the query text segmentation and the sensitive data; Sort the sensitive data according to the target similarity and determine the sequence number of the sensitive data; According to the number of sensitive data to be selected, the sensitive data whose serial number is less than the number to be selected is selected as the target sensitive data, and the sensitive data text corresponding to the target sensitive data is obtained.

[0012] In one embodiment, the step of calculating the target similarity between the query text segmentation and the sensitive data includes: Matching the query text segmentation with the sensitive data, and determining the word frequency and inverse document frequency of morphemes in the sensitive data based on the matching results; Calculating a relevance score of the morpheme based on the word frequency and the inverse document frequency; A weighted sum calculation is performed on the relevance scores of the morphemes to obtain the target similarity of the sensitive data.

[0013] In one embodiment, the step of splicing the query text and the sensitive data text based on a preset prompt word template to generate a security identification prompt word for the query text includes: Acquire the prompt word template, wherein the prompt word template includes output format requirement text, so that the output result of the large language model is a Boolean value; The query text and the sensitive data text are filled into corresponding text areas in the prompt word template to generate the security identification prompt word.

[0014] One or more technical solutions proposed in this application have at least the following technical effects: The embodiment of the present application, after receiving a query text input by a user, obtains sensitive data text that matches the query text segmentation in a sensitive database through word segmentation processing of the query text, and splices the query text and the sensitive data text based on a preset prompt word template to generate security identification prompt words for the query text, thereby enhancing the security identification prompt words of the large language model through retrieval and recall of sensitive data, improving the recognition ability of sensitive words, and inputting the security identification prompt words into at least two large language models, obtaining the output results of the large language models, and determining the security identification results of the query text by integrating the output results, thereby improving the accuracy of sensitive word recognition by integrating the processing of multiple large language models. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0017] Figure 1 This is a flowchart of the first embodiment of the content security identification method based on the integration of multiple large language models of this application; Figure 2This is a flow chart of a second embodiment of the content security identification method based on the integration of multiple large language models of the present application; Figure 3 This is a flowchart of a third embodiment of the content security identification method based on the integration of multiple large language models of the present application; Figure 4 A brief flowchart of the content security identification method based on the integration of multiple large language models in this application; Figure 5 This is a structural diagram of a content security identification device based on the integration of multiple large language models in the hardware operating environment involved in the embodiment of the present application.

[0018] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0019] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0020] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0021] The main solution of the embodiment of the present application is: receiving a query text, and performing word segmentation processing on the query text to generate query text word segments; obtaining sensitive data text matching the query text word segments in a sensitive database; splicing the query text and the sensitive data text based on a preset prompt word template to generate a security identification prompt word for the query text; inputting the security identification prompt word into at least two large language models, and obtaining output results of the large language models; integrating the output results to determine the security identification result of the query text.

[0022] In the related technology, the large language model can extract suspected sensitive words in the input text through text recognition, and perform part-of-speech analysis and review to achieve safe content recognition of the input text, which mainly relies on the reasoning ability of the pre-trained large language model for sensitive word recognition. Therefore, when there are many variants and metaphors in the text sentences in the input text, there is a certain probability that the large language model will have hallucinations when identifying sensitive words and fail to correctly identify sensitive words.

[0023] After receiving a query text input by a user, the present application obtains sensitive data text that matches the query text segmentation in a sensitive database through word segmentation processing of the query text, and based on a preset prompt word template, splices the query text and the sensitive data text to generate security identification prompt words for the query text, thereby enhancing the security identification prompt words of the large language model through retrieval and recall of sensitive data, improving the recognition ability of sensitive words, and inputting the security identification prompt words into at least two large language models, obtaining the output results of the large language models, and determining the security identification results of the query text by integrating the output results, thereby improving the accuracy of sensitive word recognition by integrating the processing of multiple large language models.

[0024] In order to better understand the above technical solution, exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0025] It should be noted that the execution subject of this embodiment can be a text content security identification system, or a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device that can realize the above functions, a content security identification device based on the integration of multiple large language models, etc., and this embodiment does not specifically limit this. The following takes the text content security identification system as an example to illustrate this embodiment and the following embodiments.

[0026] Based on this, the embodiment of the present application provides a content security identification method based on the integration of multiple large language models, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the content security identification method based on the integration of multiple large language models in this application.

[0027] In this embodiment, the content security identification method based on the integration of multiple large language models includes steps S10 to S50: Step S10: receiving a query text, and performing word segmentation processing on the query text to generate query text word segments; In this embodiment, the query text is text data to be securely identified. The text content security identification system can receive the content to be identified input by the user, or obtain the input text of a system with text information review requirements such as a large language model and a text dialogue system as the query text. The security identification system divides the continuous query text string into units with semantic meanings through tokenization in natural language processing, that is, query text tokenization, thereby identifying the boundaries of words in the query text. Among them, the text content security identification system can perform tokenization processing based on rules, statistics, or deep learning.

[0028] Exemplarily, a security identification system for text content can use open source word segmentation tools such as jieba to perform word segmentation on user query text, generate query text word segmentations, and retrieve sensitive data in the index library. The sensitive data may optionally include sensitive words, sensitive obscure sentences and other information.

[0029] Step S20: Obtain sensitive data text that matches the query text segmentation in the sensitive database; In this embodiment, the text content security identification system is configured with a sensitive database for storing sensitive words, phrases and related semantic information to assist in text security identification. The sensitive data may include sensitive words, sensitive phrases, obscure expressions, etc., which are used to determine whether the text contains unsafe content. The security identification system uses a retrieval algorithm to determine whether there is similarity or correlation between the query text segmentation and the data in the sensitive database to determine the sensitive information related to the query text for further analysis.

[0030] As an optional implementation method of obtaining sensitive data text, step S20 includes steps S21 to S24: Step S21: Based on the query text segmentation, sensitive data is obtained from the sensitive database through inverted index retrieval; Step S22: Calculate the target similarity between the query text segmentation and the sensitive data; Step S23: Sort the sensitive data according to the target similarity and determine the sequence number of the sensitive data; Step S24: According to the number of sensitive data to be selected, the sensitive data whose serial number is less than the number to be selected is selected as the target sensitive data, and the sensitive data text corresponding to the target sensitive data is obtained.

[0031] In this embodiment, the security identification system can perform an inverted index search on the query text. Among them, the inverted index search is used to quickly retrieve documents containing specific terms by mapping the terms to the list of documents containing the terms. The text security identification system uses the query text segmentation generated by the segmentation process as the search keyword and submits it to the sensitive database for retrieval. The sensitive database can be implemented based on the inverted index technology. The system calculates the target similarity between the query text segmentation and the data in the sensitive database through a retrieval algorithm such as the Okapi BM25 algorithm, and obtains sensitive text data that matches the query text segmentation.

[0032] Specifically, the security identification system matches the query text segmentation with the sensitive data, and determines the word frequency and inverse document frequency of the morphemes in the sensitive data based on the matching results. Among them, the word frequency counts the number of times each morpheme or each query text segmentation appears in the sensitive data document, and the inverse document frequency is used to measure the rarity of the morpheme. The security identification system can calculate the relevance score of the morpheme based on the word frequency and inverse document frequency, and perform a weighted sum calculation on the relevance scores of all morphemes to obtain the target similarity of the sensitive data. Based on the number of candidates for selection, that is, the number of sensitive data selected, the security identification system will select the sensitive data with a sequence number less than or equal to the number of candidates in the sorting of sensitive data based on the target similarity as the target sensitive data.

[0033] For example, taking the Okapi BM25 algorithm as an example, the Okapi BM25 algorithm correlation score formula is: , Where: IDF (q i ) is the inverse document frequency of term qi. TF(q i , d) is the frequency of word qi in document d. k1 and b are adjustment parameters, usually k1=1.2 and b=0.75. Length(d) is the length of document d, and Average Length is the average length of all documents. Score the relevance of the document.

[0034] As another optional implementation method for obtaining sensitive data text, the query text segmentation and sensitive data can also be mapped into feature vectors, and the vector distance between the feature vectors is calculated through the L2 Euclidean distance, and the target similarity of the sensitive data is determined based on the vector distance. Therefore, based on the target similarity ranking being less than or equal to the number of sensitive data to be selected, the sensitive data is selected and the corresponding sensitive data text is obtained.

[0035] Step S30: Based on a preset prompt word template, the query text and the sensitive data text are spliced ​​to generate a security identification prompt word for the query text; In this embodiment, a text structure, namely a prompt word template, is predefined in the security identification system to guide the large language model to perform security identification. The prompt word template contains specific instructions and formats to inform the large language model of the content to be identified and the output requirements. The query text and sensitive text data are spliced ​​based on the preset prompt word template to generate input text, which is submitted to the large language model to perform security identification.

[0036] Specifically, the prompt word template usually contains a fixed instruction part and a dynamic input part. The system embeds the query text and sensitive text data into the corresponding position of the template to generate a security recognition prompt word. The generated security recognition prompt word needs to meet the input requirements of the large language model to ensure that the large language model can understand and accurately perform security recognition tasks.

[0037] As an optional implementation, step S30 includes steps S31-S32: Step S31: obtaining the prompt word template, wherein the prompt word template includes an output format requirement text, so that the output result of the large language model is a Boolean value; Step S32: Fill the query text and the sensitive data text into the corresponding text area in the prompt word template to generate the security identification prompt word.

[0038] In this embodiment, the prompt word template also includes the output format requirement text in the instruction part of the template, so that the output result of the large language model is a Boolean value.

[0039] Optionally, the prompt word template may also include placeholders for query text and sensitive data text, namely, query text placeholder and sensitive data text placeholder. The security identification prompt word is generated by replacing the query text placeholder and sensitive data text placeholder with the query text and sensitive data text, and filling the query text and sensitive data text into the corresponding text area in the prompt word template.

[0040] For example, multiple sensitive data texts and user query texts are spliced ​​and assembled into a context for large language model recognition reasoning. The prompt word template example is: {"role":"system","content":"Please check whether the following content exists in the query text: Contains sensitive word data text; Output requirements: If any of the above unsafe contents exist in the user input, output '0'; if not, output '1'. Ensure that the output is only '0' or '1' without any other redundant contents. "}, {"role":"user","content":"Unsafe words and text reference example:{content}\nUnsafe words and text reference example:{content}\nQuery text: <query>"}.

[0041] Among them, role represents role, system represents system, and user represents user. "role":"system" and "role":"user" are used to represent the dialog identity of the prompt word, indicating that the current dialog identity is system and user respectively. content represents the content of the prompt word, which is used to indicate the part of the prompt word that contains the actual content. <query>Represents the query text, which is the text content that actually needs to be judged. The system will determine whether it contains sensitive content based on the rules.

[0042] As another optional implementation, the large language model may also be output in the form of text. The security identification system identifies the output text to determine whether sensitive words exist.

[0043] Step S40: inputting the security identification prompt word into at least two large language models, and obtaining output results of the large language models; In this embodiment, a large language model trained based on deep learning technology can understand and generate natural language text. The large language model has powerful language understanding and generation capabilities by learning a large amount of text data, and can be used for a variety of natural language processing tasks, including text security recognition.

[0044] Specifically, the text security recognition system inputs the generated security recognition prompt words into at least two different large language models. These large language models can be models with different architectures, different training data or different fields to ensure security recognition of query texts from multiple angles. Each large language model processes the security recognition prompt words based on its own training and understanding capabilities and outputs the recognition results. The system collects the output results of all large language models to provide data for subsequent integrated processing.

[0045] Exemplarily, the text security identification system uses multiple different large language models, including but not limited to at least three different large language models such as gemma-2-9b-it, glm-4-9b-chat, and Qwen2-7B-Instruct, to perform security assessments on the text content and generate preliminary security assessment results, where 0 indicates that the text content is unsafe and 1 indicates that the text content is safe.

[0046] Step S50: Integrate the output results to determine the security identification result of the query text.

[0047] In this embodiment, the text security identification system integrates the output results of multiple large language models to generate the final query text security identification result. The purpose of integration is to improve the accuracy and reliability of the identification results through the synergy of multiple models. After the integration process, the final conclusion of whether the query text is safe can be selected as a label or a Boolean value.

[0048] As an optional implementation, step S50 includes steps S51-S52: Step S51: Generate security identification information by logically connecting the output results; Step S52: Determine the security identification result of the query text according to the security identification information.

[0049] In this embodiment, the text security identification system may use an "OR" rule voting mechanism for the integration method of security identification results. If any model result is determined to be unsafe, the overall result is unsafe, that is, 0.

[0050] As another optional implementation, using different large language model combinations and fields may have different effects. Other integration methods such as linear regression weighting may also be used to determine the security recognition results.

[0051] The embodiment of the present application, after receiving a query text input by a user, obtains sensitive data text that matches the query text segmentation in a sensitive database through word segmentation processing of the query text, and splices the query text and the sensitive data text based on a preset prompt word template to generate security identification prompt words for the query text, thereby enhancing the security identification prompt words of the large language model through retrieval and recall of sensitive data, improving the recognition ability of sensitive words, and inputting the security identification prompt words into at least two large language models, obtaining the output results of the large language models, and determining the security identification results of the query text by integrating the output results, thereby improving the accuracy of sensitive word recognition by integrating the processing of multiple large language models.

[0052] Based on the same inventive concept, the present application also provides a second embodiment, referring to Figure 2 , Figure 2 This is a flow chart of the second embodiment of the content security identification method based on the integration of multiple large language models in this application.

[0053] In this embodiment, the content security identification method based on the integration of multiple large language models further includes steps S61 to S64: Step S61: when the security identification result is that a sensitive word exists, obtaining a target prompt word template; Step S62: splicing the format requirement text in the target prompt word template with the query text to generate a sensitive word detection prompt word; Step S63: inputting the sensitive word detection prompt word into the large language model, and obtaining the detection text output by the large language model based on the output format corresponding to the format requirement text; Step S64: Based on the output format, identifying sensitive text in the detected text.

[0054] In this embodiment, the text content security identification system further includes a target prompt word template for guiding the large language model to identify sensitive text in the query text.

[0055] Optionally, the target prompt word template includes format requirement text to guide the large language model to output the detection text based on the expected output format, so that the security identification system of the text content can identify sensitive text therein based on the detection text and perform corresponding data processing actions.

[0056] For example, when the security identification result of the text content is unsafe, the security identification system constructs a sensitive word detection prompt word through the target prompt word template and the query text to identify unsafe content in the query text, such as sensitive words or sensitive sentences. The query text is spliced ​​and assembled into a context for the large language model to answer reasoning. The prompt word template example is: {"role":"system","content":"Analyze the provided text content, identify and extract all sensitive words and the shortest sentences. The output should follow the JSON array format, where each element represents a discovered sensitive item. For example, the format is: \[{"keyword":"Example of sensitive word","sentence":"Example of the shortest sentence containing sensitive words"}\]."}, {"role":"user","content":" <query>"}.

[0057] Among them, "keyword" indicates the identified sensitive word, and "sentence" indicates the shortest sentence containing the sensitive word. role indicates role, system indicates system, and user indicates user. "role": "system" and "role": "user" are used to indicate the dialog identity of the prompt word, indicating that the current dialog identity is system and user respectively. content indicates the content of the prompt word, which is used to indicate the part of the prompt word that contains the actual content. <query>Represents the query text, which is the text content that actually needs to be judged. The system will determine whether it contains sensitive content based on the rules.

[0058] The embodiment of the present application further guides the large language model to search for sensitive words in the query text through prompt words, and can locate the source of the problem when the large language model determines whether the text is safe.

[0059] Since the system introduced in the second embodiment of the present application is a system used to implement the method of the first embodiment of the present application, based on the method introduced in the first embodiment of the present application, the person skilled in the art can understand the specific structure and deformation of the system, so it is not repeated here. All systems used in the method of the first embodiment of the present application belong to the scope of protection of this application.

[0060] Based on the same inventive concept, the present application also provides a third embodiment, referring to Figure 3 , Figure 3 This is a flowchart of the third embodiment of the content security identification method based on the integration of multiple large language models in this application.

[0061] In this embodiment, the content security identification method based on the integration of multiple large language models further includes steps S65 to S67: Step S65: Acquire sensitive words in the sensitive text; In this embodiment, the sensitive text includes sensitive words and sensitive sentences. Based on the detection text output by the large language model, the security identification system of the text content can directly identify the sensitive words and sensitive sentences therein without further semantic analysis.

[0062] As an optional implementation, the sensitive text includes a plurality of sensitive participles determined from the detection text, and the text security identification system directly obtains the sensitive participles from the sensitive text.

[0063] As another optional implementation, the text security identification system obtains sensitive data pairs of sensitive words and sensitive sentences in sensitive texts, performs word segmentation on sensitive sentences through a word segmentation tool to obtain sensitive sentence segmentations, and determines sensitive words in sensitive texts based on the sensitive words and sensitive sentence segmentations.

[0064] Step S66: Calculate the similarity between the sensitive words and the sensitive data in the sensitive database, and select the sensitive words whose similarity is higher than a similarity threshold as target sensitive words; Step S67: Update the target sensitive word into the sensitive database.

[0065] In this embodiment, after determining sensitive words based on high similarity, the sensitive database can add sensitive words and corresponding sensitive sentences to the sensitive database. Among them, by supplementing the same or similar sensitive data, the form of sensitive data can be expanded to expand the information surface of the unsafe information index and improve accuracy.

[0066] For example, the array of unsafe content is processed one by one. The processing flow is to use the word segmentation tool to segment the sentence, generate the entry data, and then select the paragraph text with the highest similarity score from the text in the sensitive entry index library. If the similarity is greater than the similarity threshold, the entry data of the sentence is written into the sensitive entry index library, otherwise it does not need to be added to the sensitive entry index library.

[0067] Since the system introduced in the third embodiment of the present application is a system used to implement the method of the first embodiment of the present application, based on the method introduced in the first embodiment of the present application, the person skilled in the art can understand the specific structure and deformation of the system, so it is not repeated here. All systems used in the method of the first embodiment of the present application belong to the scope of protection of the present application.

[0068] For example, to help understand the implementation process of the content security identification method based on the integration of multiple large language models obtained by combining this embodiment with the above-mentioned embodiment 1, please refer to Figure 4 , Figure 4 A brief flowchart of a content security identification method based on the integration of multiple large language models is provided. Specifically: The text security identification system receives the query text, obtains the sensitive data text that matches the query text segmentation of the query text from the sensitive database, and splices the query text and the sensitive data text based on the preset prompt word template to generate the security identification prompt word of the query text. By inputting the security identification prompt word into multiple large language models and obtaining the output results of the large language models, the security identification results of the query text are determined in the form of integrated output results.

[0069] Furthermore, when the security identification result is that a sensitive word exists, the text security identification system obtains the target prompt word template, splices it with the query text, generates a sensitive word detection prompt word, inputs the sensitive word detection prompt word into the large language model, and obtains the detection text output by the large language model. Based on the output format, the sensitive text in the detection text is identified. The text security identification system calculates the similarity between the sensitive text and the sensitive data in the sensitive database, and updates the high-similarity sensitive data with a similarity higher than the similarity threshold to the sensitive database, thereby completing the iteration of sensitive data.

[0070] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the content security identification method based on the integration of multiple large language models in the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0071] The present application provides a content security identification device based on the integration of multiple large language models, the device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the content security identification method based on the integration of multiple large language models in the above-mentioned embodiment one.

[0072] Reference below Figure 5 , which shows a schematic diagram of the structure of a content security identification device based on the integration of multiple large language models suitable for implementing the embodiment of the present application. The content security identification device based on the integration of multiple large language models in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The content security identification device based on the integration of multiple large language models shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0073] like Figure 5 As shown, a content security identification device based on the integration of multiple large language models may include a processing device 1001 (e.g., a core processor, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. Various programs and data required for the operation of a content security identification device based on the integration of multiple large language models are also stored in RAM1004. The processing device 1001, ROM1002, and RAM1004 are connected to each other via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 can allow the content security identification device based on the integration of multiple large language models to communicate wirelessly or wired with other devices to exchange data. Although the figure shows a content security identification device based on the integration of multiple large language models with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have alternatively.

[0074] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0075] The content security identification device based on the integration of multiple large language models provided by the present application adopts the content security identification method based on the integration of multiple large language models in the above-mentioned embodiment, which can solve the technical problem that the content security identification of the input text based on the large language model cannot accurately identify sensitive words in the text sentence. Compared with the prior art, the beneficial effects of the content security identification device based on the integration of multiple large language models provided by the present application are the same as the beneficial effects of the content security identification method based on the integration of multiple large language models provided by the above-mentioned embodiment, and the other technical features of the content security identification device based on the integration of multiple large language models are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0076] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0077] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0078] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, the computer-readable program instructions being used to execute the content security identification method based on the integration of multiple large language models in the above-mentioned embodiment.

[0079] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0080] The computer-readable storage medium may be included in a content security identification device based on the integration of multiple large language models; or may exist independently without being assembled into a content security identification device based on the integration of multiple large language models.

[0081] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by a content security identification device based on the integration of multiple large language models, the content security identification device based on the integration of multiple large language models: receives a query text, and performs word segmentation processing on the query text to generate query text word segmentations; obtains sensitive data text that matches the query text word segmentations in a sensitive database; splices the query text and the sensitive data text based on a preset prompt word template to generate security identification prompt words for the query text; inputs the security identification prompt words into at least two large language models, and obtains output results of the large language models; integrates the output results to determine the security identification results of the query text.

[0082] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0083] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0084] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0085] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned content security identification method based on the integration of multiple large language models, which can solve the technical problem that the content security identification of input text based on the large language model cannot accurately identify sensitive words in the text sentence. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the content security identification method based on the integration of multiple large language models provided in the above-mentioned embodiment, and will not be repeated here.

[0086] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.< / query> < / query> < / query> < / query>

Claims

1. A content security identification method based on the integration of multiple large language models, characterized in that: The method comprises the following steps: Receive a query text, and perform word segmentation processing on the query text to generate query text word segments; Obtaining sensitive data text that matches the query text segmentation in a sensitive database; Based on a preset prompt word template, the query text and the sensitive data text are spliced ​​to generate a security identification prompt word for the query text; Inputting the security identification prompt word into at least two large language models, and obtaining output results of the large language models; The output results are integrated to determine a security identification result of the query text.

2. The method according to claim 1, characterized in that After the step of integrating the output results and determining the security identification result of the query text, the method further includes: When the security identification result is that a sensitive word exists, obtaining a target prompt word template; The format requirement text in the target prompt word template is concatenated with the query text to generate a sensitive word detection prompt word; Inputting the sensitive word detection prompt word into the large language model, and obtaining the detection text output by the large language model based on the output format corresponding to the format requirement text; Based on the output format, sensitive text in the detected text is identified.

3. The method according to claim 2, characterized in that After the step of identifying the sensitive text in the detected text based on the output format, the method further includes: Obtaining sensitive words in the sensitive text; Calculating the similarity between the sensitive words and the sensitive data in the sensitive database, and selecting the sensitive words whose similarity is higher than a similarity threshold as target sensitive words; The target sensitive word is updated into the sensitive database.

4. The method according to claim 3, characterized in that The step of obtaining sensitive words in the sensitive text includes: Obtaining sensitive data pairs of sensitive words and sensitive sentences in the sensitive text; Using a word segmentation tool, the sensitive sentence is segmented to obtain sensitive sentence segmentations; Based on the sensitive word segmentation and the sensitive sentence segmentation, the sensitive words in the sensitive text are determined.

5. The method according to claim 1, characterized in that The step of integrating the output results and determining the security identification result of the query text comprises: Generate security identification information by logically connecting the output results; The security identification result of the query text is determined according to the security identification information.

6. The method according to claim 1, characterized in that The step of obtaining the sensitive data text matching the query text segmentation in the sensitive database includes: Based on the query text segmentation, sensitive data is obtained from the sensitive database through inverted index retrieval; Calculating target similarity between the query text segmentation and the sensitive data; Sort the sensitive data according to the target similarity and determine the sequence number of the sensitive data; According to the number of sensitive data to be selected, the sensitive data whose serial number is less than the number to be selected is selected as the target sensitive data, and the sensitive data text corresponding to the target sensitive data is obtained.

7. The method according to claim 6, characterized in that The step of calculating the target similarity between the query text segmentation and the sensitive data comprises: Matching the query text segmentation with the sensitive data, and determining the word frequency and inverse document frequency of morphemes in the sensitive data based on the matching results; Calculating a relevance score of the morpheme based on the word frequency and the inverse document frequency; A weighted sum calculation is performed on the relevance scores of the morphemes to obtain the target similarity of the sensitive data.

8. The method according to claim 1, characterized in that The step of splicing the query text and the sensitive data text based on a preset prompt word template to generate a security identification prompt word for the query text includes: Acquire the prompt word template, wherein the prompt word template includes output format requirement text, so that the output result of the large language model is a Boolean value; The query text and the sensitive data text are filled into corresponding text areas in the prompt word template to generate the security identification prompt word.

Images