Content Security Recognition Method Based on the Integration of Multiple Large Language Models

Abstract

The present application discloses a content security recognition method based on the integration of multiple large language models, belonging to the technical field of data processing. The method includes: receiving a query text, performing word segmentation processing on the query text to generate query text word segments, obtaining sensitive data texts matching the query text word segments in a sensitive database, splicing the query text and the sensitive data texts based on a preset prompt word template to generate a security recognition prompt word for the query text, inputting the security recognition prompt word into at least two large language models, obtaining the output results of the large language models, integrating the output results, and determining the security recognition result of the query text. The present application recalls sensitive data texts through a sensitive database, enhances the security recognition prompt words of the large language models, and improves the accuracy of sensitive word recognition through the integrated processing of multiple large language models.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and particularly to a content security recognition method based on the integration of multiple large language models. Background Art

[0002] Large language models can, through text recognition, extract suspected sensitive words in the input text, and through a matching method with a sensitive word library, perform part-of-speech analysis and review on the suspected sensitive words to achieve content security recognition of the input text.

[0003] In related technologies, the extraction of sensitive words mainly relies on the inference ability of pre-trained large language models for sensitive word recognition. However, due to the large number of variants and metaphors in text sentences in the input text, there is a certain probability that the large language model will have hallucinations when recognizing sensitive words and cannot correctly recognize sensitive words.

[0004] The above content is only used to assist in understanding the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main purpose of this application is to provide a content security recognition method based on the integration of multiple large language models, aiming to solve the technical problem that the content security recognition of the input text based on the large language model cannot accurately recognize sensitive words in text sentences.

[0006] To achieve the above purpose, this application provides a content security recognition method based on the integration of multiple large language models, and the method includes the following steps:

[0007] Receive a query text, and perform word segmentation on the query text to generate query text word segments;

[0008] In a sensitive database, obtain sensitive data texts that match the query text word segments;

[0009] Based on a preset prompt word template, splice the query text and the sensitive data texts to generate a security recognition prompt word for the query text;

[0010] Input the security recognition prompt word into at least two large language models, and obtain the output results of the large language models;

[0011] Integrate the output results to determine the security recognition result of the query text.

[0012] In one embodiment, after the step of integrating the output results to determine the security recognition result of the query text, the method further includes:

[0013] When the security recognition result indicates the existence of sensitive words, obtain a target prompt word template;

[0014] Concatenate the format requirement text in the target prompt template with the query text to generate a sensitive word detection prompt;

[0015] Input the sensitive word detection prompt into the large language model, and obtain the detected text output by the large language model based on the output format corresponding to the format requirement text;

[0016] Based on the output format, identify sensitive text in the detected text.

[0017] In one embodiment, after the step of identifying sensitive text in the detected text based on the output format, the method further includes:

[0018] Obtain sensitive words in the sensitive text;

[0019] Calculate the similarity between the sensitive words and the sensitive data in the sensitive database, and select the sensitive words with a similarity higher than the similarity threshold as target sensitive words;

[0020] Update the target sensitive words to the sensitive database.

[0021] In one embodiment, the step of obtaining sensitive words in the sensitive text includes:

[0022] Obtain sensitive data pairs of sensitive word segmentation and sensitive sentences in the sensitive text;

[0023] Use a word segmentation tool to perform word segmentation on the sensitive sentence to obtain sensitive sentence word segmentation;

[0024] Based on the sensitive word segmentation and the sensitive sentence word segmentation, determine the sensitive words in the sensitive text.

[0025] In one embodiment, the step of integrating the output results to determine the security recognition result of the query text includes:

[0026] Connect the output results through logical OR to generate security recognition information;

[0027] Based on the security recognition information, determine the security recognition result of the query text.

[0028] In one embodiment, the step of obtaining sensitive data text that matches the query text word segmentation in the sensitive database includes:

[0029] Based on the query text word segmentation, retrieve sensitive data in the sensitive database through an inverted index;

[0030] Calculate the target similarity between the query text segmentation and the sensitive data;

[0031] Sort the sensitive data according to the target similarity, and determine the serial number of the sensitive data;

[0032] According to the number of candidate sensitive data, select the sensitive data with the serial number less than the number of candidate as the target sensitive data, and obtain the sensitive data text corresponding to the target sensitive data.

[0033] In one embodiment, the step of calculating the target similarity between the query text segmentation and the sensitive data includes:

[0034] Match the query text segmentation with the sensitive data, and determine the word frequency and inverse document frequency of the morphemes in the sensitive data based on the matching result;

[0035] Calculate the correlation score of the morpheme based on the word frequency and the inverse document frequency;

[0036] Perform a weighted sum calculation on the correlation scores of the morphemes to obtain the target similarity of the sensitive data.

[0037] In one embodiment, the step of splicing the query text and the sensitive data text based on a preset prompt word template to generate a security recognition prompt word for the query text includes:

[0038] Obtain the prompt word template, which includes output format requirement text to make the output result of the large language model a boolean value;

[0039] Fill the query text and the sensitive data text into the corresponding text areas in the prompt word template to generate the security recognition prompt word.

[0040] One or more technical solutions proposed in this application have at least the following technical effects:

[0041] In the embodiment of this application, after receiving the query text input by the user, through the word segmentation process of the query text, obtain the sensitive data text that matches the query text segmentation in the sensitive database, and based on a preset prompt word template, splice the query text and the sensitive data text to generate a security recognition prompt word for the query text. Thus, through the retrieval and recall of sensitive data, enhance the security recognition prompt word of the large language model, improve the recognition ability of sensitive words, and input the security recognition prompt word into at least two large language models, and obtain the output results of the large language models. By integrating the output results, determine the security recognition result of the query text. Thus, through the integrated processing method of multiple large language models, improve the accuracy of sensitive word recognition. Brief Description of the Drawings

[0042] The drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with this application, and are used together with the specification to explain the principles of this application.

[0043] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0044] Figure 1 It is a schematic flowchart of the first embodiment of the content security recognition method based on the integration of multiple large language models of this application;

[0045] Figure 2 It is a schematic flowchart of the second embodiment of the content security recognition method based on the integration of multiple large language models of this application;

[0046] Figure 3 It is a schematic flowchart of the third embodiment of the content security recognition method based on the integration of multiple large language models of this application;

[0047] Figure 4 It is a schematic flowchart of the brief content security recognition method based on the integration of multiple large language models of this application;

[0048] Figure 5 It is a schematic structural diagram of the content security recognition device based on the integration of multiple large language models in the hardware operating environment related to the solution of the embodiment of this application.

[0049] The implementation, functional features, and advantages of the purpose of this application will be further described with reference to the embodiments and the drawings. Detailed Embodiments

[0050] It should be understood that the specific embodiments described here are only used to explain this application and are not used to limit this application.

[0051] To better understand the above technical solutions, the following will describe the above technical solutions in detail in combination with the drawings of the specification and specific embodiments.

[0052] The main solution of the embodiment of this application is: receiving a query text, performing word segmentation processing on the query text to generate query text word segmentation; obtaining sensitive data text matching the query text word segmentation in a sensitive database; based on a preset prompt word template, splicing the query text and the sensitive data text to generate a security recognition prompt word for the query text; inputting the security recognition prompt word into at least two large language models, and obtaining the output results of the large language models; integrating the output results to determine the security recognition result of the query text.

[0053] In related technologies, a large language model can, through text recognition, extract suspected sensitive words in the input text, perform part-of-speech analysis and review, and achieve content security recognition of the input text, mainly relying on the inference ability of the pre-trained large language model for sensitive word recognition. Therefore, in the case of many variants and metaphors in the text sentences of the input text, there is a certain probability that the large language model will have hallucinations when recognizing sensitive words and cannot correctly recognize sensitive words.

[0054] In this application, after receiving the query text input by the user, through word segmentation processing of the query text, obtaining sensitive data text matching the query text word segmentation in a sensitive database, and based on a preset prompt word template, splicing the query text and the sensitive data text to generate a security recognition prompt word for the query text, thereby enhancing the security recognition prompt word of the large language model through retrieval and recall of sensitive data, improving the ability to recognize sensitive words, and inputting the security recognition prompt word into at least two large language models, and obtaining the output results of the large language models, and determining the security recognition result of the query text by integrating the output results, thereby improving the accuracy of sensitive word recognition through the integrated processing of multiple large language models.

[0055] To better understand the above technical solution, the exemplary embodiments of this application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of this application and to be able to fully convey the scope of this application to those skilled in the art.

[0056] It should be noted that the execution subject of this embodiment can be a security recognition system for text content, or a computing service device with data processing, network communication, and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of implementing the above functions, a content security recognition device integrated based on multiple large language models, etc. This embodiment does not make specific limitations. The following takes the security recognition system for text content as an example to illustrate this embodiment and the following embodiments.

[0057] Based on this, the embodiments of the present application provide a content security recognition method based on the integration of multiple large language models. Refer to Figure 1 , Figure 1 which is a schematic flowchart of the first embodiment of the content security recognition method based on the integration of multiple large language models of the present application.

[0058] In this embodiment, the content security recognition method based on the integration of multiple large language models includes steps S10 to S50:

[0059] Step S10: Receive a query text, and perform word segmentation processing on the query text to generate query text word segments;

[0060] In this embodiment, the query text is the text data to be subjected to security recognition. The content security recognition system can receive the content to be recognized input by the user, or obtain the input text of systems such as large language models and text dialogue systems that have text information review requirements as the query text. The security recognition system uses word segmentation (Tokenization) in natural language processing to split the continuous query text string into units with semantic meanings, that is, query text word segments, so as to identify the boundaries of words in the query text. Among them, the content security recognition system can perform word segmentation processing based on rule-based word segmentation, statistics-based word segmentation, or deep learning-based word segmentation, etc.

[0061] Exemplarily, the content security recognition system can use open-source word segmentation tools such as jieba to perform word segmentation processing on the user query text, generate query text word segments, and retrieve sensitive data in the index library. The sensitive data can optionally include information such as sensitive words and sensitive implicit sentences, etc.

[0062] Step S20: In the sensitive database, obtain the sensitive data text that matches the query text word segments;

[0063] In this embodiment, in the content security recognition system, a sensitive database for storing sensitive words, phrases, and related semantic information is configured to assist in text security recognition. The sensitive data therein can include sensitive words, sensitive phrases, implicit expressions, etc., and is used to determine whether the text contains insecure content. The security recognition system uses a retrieval algorithm to determine whether there is similarity or correlation between the query text word segments and the data in the sensitive database to determine the sensitive information related to the query text for further analysis.

[0064] As an optional implementation manner for obtaining the sensitive data text, step S20 includes steps S21 to S24:

[0065] Step S21: Based on the query text word segments, retrieve through an inverted index to obtain sensitive data in the sensitive database;

[0066] Step S22: Calculate the target similarity between the query text segmentation and the sensitive data;

[0067] Step S23: Sort the sensitive data according to the target similarity, and determine the serial number of the sensitive data;

[0068] Step S24: According to the number of sensitive data to be selected, select the sensitive data with the serial number less than the number of candidates as the target sensitive data, and obtain the sensitive data text corresponding to the target sensitive data.

[0069] In this embodiment, the security identification system can perform an inverted index retrieval on the query text. Among them, the inverted index retrieval is used to reduce information by mapping terms to a list of documents containing the terms, and is used to quickly retrieve documents containing specific terms. The text security identification system uses the query text segmentation generated by the word segmentation process as the retrieval keyword and submits it to the sensitive database for retrieval. The sensitive database can be implemented based on the inverted index technology. The system calculates the target similarity between the query text segmentation and the data in the sensitive database through retrieval algorithms such as the Okapi BM25 algorithm, and obtains the sensitive text data matching the query text segmentation.

[0070] Specifically, the security identification system matches the query text segmentation with the sensitive data, and determines the term frequency and inverse document frequency of the morphemes in the sensitive data based on the matching result. Among them, the term frequency counts the number of times each morpheme or each query text segmentation appears in the sensitive data document, and the inverse document frequency is used to measure the rarity of the morpheme. The security identification system can calculate the correlation score of the morpheme based on the term frequency and the inverse document frequency, and perform a weighted sum calculation on the correlation scores of all morphemes to obtain the target similarity of the sensitive data. The security identification system will select the sensitive data with the serial number less than or equal to the number of candidates in the sorting of the sensitive data based on the target similarity as the target sensitive data based on the number of sensitive data to be selected, that is, the number of selected sensitive data.

[0071] Exemplarily, taking the Okapi BM25 algorithm as an example, the formula for the correlation score of the Okapi BM25 algorithm is:

[0072] ,

[0073] where: IDF(q i ) is the inverse document frequency of the word qi. TF(q i , d) is the term frequency of the word qi in the document d. k1 and b are adjustment parameters, usually k1 = 1.2 and b = 0.75. Length(d) is the length of the document d, and Average Length is the average length of all documents. It is the document relevance score.

[0074] As another alternative implementation for obtaining sensitive data text, the query text can also be tokenized and mapped to a feature vector with the sensitive data. By using the L2 Euclidean distance, the vector distance between the feature vectors is calculated, and the target similarity of the sensitive data is determined based on this vector distance. Then, based on the target similarity ranking being less than or equal to the number of sensitive data candidates, the sensitive data is selected to obtain the corresponding sensitive data text.

[0075] Step S30: Based on a preset prompt template, concatenate the query text and the sensitive data text to generate a security recognition prompt for the query text.

[0076] In this embodiment, a text structure, i.e., a prompt template, is predefined in the security recognition system to guide the large language model for security recognition. The prompt template contains specific instructions and formats to inform the large language model of the content to be recognized and the output requirements. By concatenating the query text and the sensitive text data based on the preset prompt template to generate an input text, it is submitted to the large language model to perform the security recognition action.

[0077] Specifically, the prompt template usually contains a fixed instruction part and a dynamic input part. The system embeds the query text and the sensitive text data into the corresponding positions of the template to generate a security recognition prompt. The generated security recognition prompt needs to meet the input requirements of the large language model to ensure that the large language model can understand and accurately execute the security recognition task.

[0078] As an alternative implementation, step S30 includes steps S31 to S32:

[0079] Step S31: Obtain the prompt template, which contains a text for output format requirements to make the output result of the large language model a boolean value.

[0080] Step S32: Fill the query text and the sensitive data text into the corresponding text areas in the prompt template to generate the security recognition prompt.

[0081] In this implementation, in the instruction part of the prompt template, there is also a text for output format requirements, so that the output result of the large language model is a boolean value.

[0082] Optionally, the prompt template can also contain placeholders for the query text and the sensitive data text, namely the query text placeholder and the sensitive data text placeholder. By replacing the query text placeholder and the sensitive data text placeholder with the query text and the sensitive data text, the query text and the sensitive data text are filled into the corresponding text areas in the prompt template to generate the security recognition prompt.

[0083] Exemplarily, multiple sensitive data texts and user query texts are concatenated and assembled into context for large language model recognition and inference. Among them, an example of the prompt template is:

[0084] {"role": "system", "content": "Please determine whether the following content exists in the query text:

[0085] Contains sensitive word data text;

[0086] Output requirement: If any of the above unsafe content exists in the user input, output '0'; if not, output '1', ensuring that the output is only '0' or '1' without any other extra content."},

[0087] {"role": "user", "content": "Reference examples of unsafe words and texts: {content}\nReference examples of unsafe words and texts: {content}\nQuery text: <query>"}。

[0088] Among them, "role" represents the role, "system" represents the system, "user" represents the user. "role": "system" and "role": "user" are used to indicate the dialogue identity of the prompt words, representing that the current dialogue identity is the system and the user respectively. "content" represents the prompt word content, which is used to indicate the part containing the actual content in the prompt word. <query>Represents the query text, which is the actual text content to be judged. The system will determine whether it contains sensitive content according to the rules.

[0089] As another alternative implementation, the large language model can also be output in text form. The security recognition system determines whether there are sensitive words by recognizing the output text.

[0090] Step S40: Input the security recognition prompt word into at least two large language models and obtain the output results of the large language models;

[0091] In this embodiment, the large language model trained based on deep learning technology can understand and generate natural language text. By learning a large amount of text data, the large language model has powerful language understanding and generation capabilities and can be used for a variety of natural language processing tasks, including text security recognition.

[0092] Specifically, the text security recognition system inputs the generated security recognition prompt words into at least two different large language models respectively. These large language models can be models with different architectures, different training data or different domains to ensure the security recognition of the query text from multiple perspectives. Each large language model processes the security recognition prompt word according to its own training and understanding ability and outputs the recognition result. The system collects the output results of all large language models to provide data for subsequent integrated processing.

[0093] Exemplarily, the text security recognition system uses multiple different large language models, including but not limited to at least 3 different large language models such as gemma-2-9b-it, glm-4-9b-chat, Qwen2-7B-Instruct, etc. to conduct security assessment on the text content respectively and generate preliminary security assessment results, where 0 indicates that the text content is insecure and 1 indicates that the text content is secure.

[0094] Step S50: Integrate the output results to determine the security recognition result of the query text.

[0095] In this embodiment, the text security recognition system comprehensively processes the output results of multiple large language models to generate the final security recognition result of the query text. The purpose of integration is to improve the accuracy and reliability of the recognition result through the collaborative effect of multiple models. After the integrated processing, the conclusion of whether the query text is safe is finally determined, and this conclusion can be optionally a label or a boolean value.

[0096] As an alternative implementation, step S50 includes steps S51~S52:

[0097] Step S51: Connect the output results through logical OR to generate security recognition information;

[0098] Step S52: Determine the security recognition result of the query text according to the security recognition information.

[0099] In this embodiment, for the integration method of the text security recognition system regarding the security recognition result, a voting mechanism using the "OR" rule can be used. That is, if any one of the model results is determined to be insecure, the overall result is insecure, which is 0.

[0100] As another alternative embodiment, the effects may be different when using different combinations of large language models and domains. Other integration methods such as linear regression weighting can also be used to determine the security recognition result.

[0101] In the embodiment of the present application, after receiving the query text input by the user, through the word segmentation processing of the query text, sensitive data texts matching the word segmentation of the query text are obtained from the sensitive database, and based on a preset prompt word template, the query text and the sensitive data texts are spliced to generate a security recognition prompt word for the query text. Thus, through the retrieval and recall of sensitive data, the security recognition prompt word of the large language model is enhanced, the recognition ability for sensitive words is improved, and the security recognition prompt word is input into at least two large language models, and the output results of the large language models are obtained. By integrating the output results, the security recognition result of the query text is determined. Thus, through the integrated processing method of multiple large language models, the accuracy of sensitive word recognition is improved.

[0102] Based on the same inventive concept, the present application also provides a second embodiment, referring to Figure 2 , Figure 2 is a schematic flowchart of the second embodiment of the content security recognition method based on the integration of multiple large language models of the present application.

[0103] In this embodiment, the content security recognition method based on the integration of multiple large language models further includes steps S61 to S64:

[0104] Step S61: When the security recognition result indicates the existence of sensitive words, obtain the target prompt word template;

[0105] Step S62: Splice the format requirement text in the target prompt word template with the query text to generate a sensitive word detection prompt word;

[0106] Step S63: Input the sensitive word detection prompt word into the large language model and obtain the detection text output by the large language model based on the output format corresponding to the format requirement text;

[0107] Step S64: Identify the sensitive text in the detection text based on the output format.

[0108] In this embodiment, the security recognition system for text content further includes a target prompt word template for guiding the large language model to recognize sensitive text existing in the query text.

[0109] Optionally, the target prompt word template includes format requirement text to guide the large language model to output the detection text based on the expected output format, so that the security recognition system for text content can recognize sensitive text therein based on the detection text and perform corresponding data processing actions.

[0110] Exemplarily, when the security recognition result of the security recognition system for text content is insecure, a sensitive word detection prompt word is constructed through the target prompt word template and the query text to identify insecure content in the query text, such as sensitive words or sensitive sentences. The query text is spliced and assembled into context for the large language model to answer and reason. An example of the prompt word template is as follows:

[0111] {"role": "system", "content": "Analyze the provided text content, identify and extract all sensitive vocabulary and the shortest sentences therein. The output result should follow the JSON array format, and each element represents a discovered sensitive item, in the format of, for example: [{"keyword": "example of sensitive word", "sentence": "example of the shortest sentence containing the sensitive word"}]."}, {"role": "user", "content": " <query>"}。

[0112] Among them, "keyword" represents the identified sensitive word, and "sentence" represents the shortest sentence containing the sensitive word. role represents the role, system represents the system, user represents the user. "role": "system" and "role": "user" are used to indicate the dialogue identities of the prompting words, representing the current dialogue identities as the system and the user respectively. content represents the content of the prompting word, which is used to indicate the part containing the actual content in the prompting word. <query>It represents the query text, which is the actual text content to be judged. The system will determine whether it contains sensitive content according to the rules.

[0113] In the embodiment of the present application, the prompting words are used to further guide the large language model to query sensitive words in the query text, so that the source of the problem can be located when the large language model determines whether the text is safe.

[0114] Since the system introduced in the second embodiment of the present application is the system adopted for implementing the method of the first embodiment of the present application, based on the method introduced in the first embodiment of the present application, those skilled in the art can understand the specific structure and variations of the system, so it will not be elaborated here. Any system adopted by the method of the first embodiment of the present application falls within the scope of protection of the present application.

[0115] Based on the same inventive concept, the present application also provides a third embodiment. Refer to Figure 3 , Figure 3 which is a schematic flowchart of the third embodiment of the content security recognition method based on the integration of multiple large language models of the present application.

[0116] In this embodiment, the content security recognition method based on the integration of multiple large language models further includes steps S65 to S67:

[0117] Step S65: Obtain the sensitive words in the sensitive text;

[0118] In this embodiment, the sensitive text contains sensitive word segments and sensitive sentences. Based on the detection text output by the large language model, the text content security recognition system can directly identify the sensitive word segments and sensitive sentences therein without further semantic analysis.

[0119] As an optional implementation manner, the sensitive text contains multiple sensitive word segments determined from the detection text, and the text security recognition system directly obtains the sensitive word segments in the sensitive text.

[0120] As another optional implementation manner, the text security recognition system obtains the sensitive data pairs of the sensitive word segments and sensitive sentences in the sensitive text, performs word segmentation processing on the sensitive sentences through a word segmentation tool to obtain sensitive sentence word segments, and determines the sensitive words in the sensitive text based on the sensitive word segments and sensitive sentence word segments.

[0121] Step S66: Calculate the similarity between the sensitive words and the sensitive data in the sensitive database, and select the sensitive words with a similarity higher than the similarity threshold as the target sensitive words;

[0122] Step S67: Update the target sensitive words to the sensitive database.

[0123] In this embodiment, after determining sensitive words based on high similarity, the sensitive database can add the sensitive words and the corresponding sensitive sentences to the sensitive database. Among them, through the supplement of the same or similar sensitive data, the form of sensitive data can be extended to expand the information surface of the insecure information index and improve the accuracy.

[0124] Exemplarily, each array of insecure content is processed one by one. The processing flow is to use a word segmentation tool to segment the sentence to generate entry data, and then select the paragraph text with the highest similarity score in the text of the sensitive entry index library. If the similarity is greater than the similarity threshold, the entry data of the sentence is written into the sensitive entry index library; otherwise, it does not need to be supplemented to the sensitive entry index library.

[0125] Since the system introduced in Embodiment 3 of this application is the system adopted for implementing the method of Embodiment 1 of this application, based on the method introduced in Embodiment 1 of this application, those skilled in the art can understand the specific structure and variations of the system, so it will not be elaborated here. Any system adopted by the method of Embodiment 1 of this application belongs to the scope protected by this application.

[0126] Exemplarily, to help understand the implementation process of the content security recognition method based on the integration of multiple large language models obtained by combining this embodiment with the above-mentioned Embodiment 1, please refer to Figure 4 , Figure 4 which provides a schematic diagram of the brief process of a content security recognition method based on the integration of multiple large language models. Specifically:

[0127] The text security recognition system receives a query text, obtains a sensitive data text that matches the query text word segmentation in the sensitive database, and based on a preset prompt word template, splices the query text and the sensitive data text to generate a security recognition prompt word for the query text. By inputting the security recognition prompt word into multiple large language models and obtaining the output results of the large language models, the security recognition result of the query text is determined in the way of integrating the output results.

[0128] Further, when the security recognition result indicates the existence of sensitive words, the text security recognition system obtains a target prompt word template, splices it with the query text to generate a sensitive word detection prompt word, inputs the sensitive word detection prompt word into the large language model, and obtains the detection text output by the large language model. Based on this output format, the sensitive text in the detection text is recognized. The text security recognition system calculates the similarity between the sensitive text and the sensitive data in the sensitive database, and updates the highly similar sensitive data with a similarity higher than the similarity threshold to the sensitive database, thereby completing the iteration of the sensitive data.

[0129] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the content security recognition method based on the integration of multiple large language models of this application. Any simple transformation in more forms based on this technical concept is within the protection scope of this application.

[0130] This application provides a content security recognition device based on the integration of multiple large language models. The device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the content security recognition method based on the integration of multiple large language models in Embodiment 1 above.

[0131] The following refers to Figure 5 , which shows a schematic structural diagram of a content security recognition device suitable for implementing the embodiments of this application based on the integration of multiple large language models. The content security recognition device based on the integration of multiple large language models in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistant), PADs (Portable Application Description), PMPs (Portable Media Player), in-vehicle terminals (such as in-vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The content security recognition device based on the integration of multiple large language models shown is only an example and should not impose any limitations on the functions and usage scope of the embodiments of this application.

[0132] As Figure 5 As shown, the content security recognition device based on the integration of multiple large language models may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM: Read Only Memory) 1002 or the program loaded from the storage device 1003 into the random access memory (RAM: Random Access Memory) 1004. In the RAM 1004, various programs and data required for the operation of the content security recognition device based on the integration of multiple large language models are also stored. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. The input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the content security recognition device based on the integration of multiple large language models to communicate with other devices wirelessly or wiredly to exchange data. Although the figure shows a content security recognition device based on the integration of multiple large language models with various systems, it should be understood that it is not required to implement or have all the shown systems. Instead, more or fewer systems can be implemented or had.

[0133] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program codes for performing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above functions defined in the method of the embodiments disclosed in the present application are executed.

[0134] The content security recognition device based on the integration of multiple large language models provided by this application adopts the content security recognition method based on the integration of multiple large language models in the above embodiments, and can solve the technical problem that the content security recognition of the input text based on the large language model cannot accurately recognize sensitive words in the text sentences. Compared with the prior art, the beneficial effects of the content security recognition device based on the integration of multiple large language models provided by this application are the same as those of the content security recognition method based on the integration of multiple large language models provided by the above embodiments, and other technical features in the content security recognition device based on the integration of multiple large language models are the same as the features disclosed in the method of the previous embodiment, and will not be elaborated here.

[0135] It should be understood that each part disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0136] The above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art within the technical scope disclosed in this application can easily think of changes or substitutions, which should all be covered within the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claimed rights.

[0137] This application provides a computer-readable storage medium with computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the content security recognition method based on the integration of multiple large language models in the above embodiments.

[0138] The computer-readable storage medium provided by this application can be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or flash memory, optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this embodiment, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, device, or component. The program code contained on the computer-readable storage medium can be transmitted by any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0139] The above computer-readable storage medium can be included in a content security recognition device integrated based on multiple large language models; it can also exist independently and not be assembled into a content security recognition device integrated based on multiple large language models.

[0140] The above computer-readable storage medium carries one or more programs. When the above one or more programs are executed by a content security recognition device integrated based on multiple large language models, the content security recognition device integrated based on multiple large language models is caused to: receive a query text, perform word segmentation processing on the query text to generate query text word segments; in a sensitive database, obtain sensitive data texts that match the query text word segments; based on a preset prompt word template, splice the query text and the sensitive data texts to generate a security recognition prompt word for the query text; input the security recognition prompt word into at least two large language models, and obtain the output results of the large language models; integrate the output results to determine the security recognition result of the query text.

[0141] Computer program code for performing the operations of this application can be written in one or more programming languages or combinations thereof. The above-mentioned programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (for example, by connecting through the Internet using an Internet service provider).

[0142] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0143] The modules described in the embodiments of this application can be implemented in software or in hardware. Among them, the name of the module does not constitute a limitation to the unit itself in some cases.

[0144] The readable storage medium provided by this application is a computer-readable storage medium, and the computer-readable storage medium stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned content security recognition method based on the integration of multiple large language models, which can solve the technical problem that the content security recognition of the input text based on the large language model cannot accurately identify sensitive words in the text sentence. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by this application are the same as those of the content security recognition method based on the integration of multiple large language models provided by the above embodiments, and will not be elaborated here.

[0145] The above are only some embodiments of this application, and thus do not limit the patent scope of this application. Any equivalent structural transformation made under the technical concept of this application by using the content of the specification and drawings of this application, or direct / indirect application in other related technical fields, is included in the patent protection scope of this application.< / query> < / query> < / query> < / query>

Claims

1. A content security recognition method based on the integration of multiple large language models, characterized in that, The method described above includes the following steps: Receive a query text, perform word segmentation on the query text, and generate query text word segments; In a sensitive database, obtain sensitive data texts that match the query text word segments; Based on a preset prompt word template, splice the query text and the sensitive data texts to generate a security recognition prompt word for the query text; Input the security recognition prompt word into at least two large language models, and obtain the output results of the large language models; Integrate the output results to determine the security recognition result of the query text; When the security recognition result indicates the existence of sensitive words, obtain a target prompt word template; Splice the format requirement text in the target prompt word template with the query text to generate a sensitive word detection prompt word, where the format requirement text is used to guide the large language model to output a detection text based on an expected output format to identify sensitive text therein and perform corresponding data processing actions; Input the sensitive word detection prompt word into the large language model, and obtain the detection text output by the large language model based on the output format corresponding to the format requirement text; Based on the output format, identify the sensitive text in the detection text.

2. The method according to claim 1, wherein After the step of identifying the sensitive text in the detection text based on the output format, it further includes: Obtain the sensitive words in the sensitive text; Calculate the similarity between the sensitive words and the sensitive data in the sensitive database, and select the sensitive words with a similarity higher than the similarity threshold as target sensitive words; Update the target sensitive words to the sensitive database.

3. The method according to claim 2, wherein The step of obtaining the sensitive words in the sensitive text includes: Obtain the sensitive data pairs of sensitive word segments and sensitive sentences in the sensitive text; Use a word segmentation tool to perform word segmentation on the sensitive sentence to obtain sensitive sentence word segments; Based on the sensitive word segments and the sensitive sentence word segments, determine the sensitive words in the sensitive text.

4. The method according to claim 1, characterized in that The step of integrating the output results to determine the security recognition result of the query text includes: Connect the output results through logical OR to generate security recognition information; Based on the security recognition information, determine the security recognition result of the query text.

5. The method according to claim 1, characterized in that, The step of obtaining sensitive data texts that match the query text word segments in the sensitive database includes: Based on the query text word segments, retrieve sensitive data in the sensitive database through an inverted index; Calculate the target similarity between the query text word segments and the sensitive data; Sort the sensitive data according to the target similarity to determine the serial numbers of the sensitive data; According to the number of candidate sensitive data, select the sensitive data with serial numbers less than the number of candidate sensitive data as target sensitive data, and obtain the sensitive data texts corresponding to the target sensitive data.

6. The method according to claim 5, characterized in that The step of calculating the target similarity between the query text word segments and the sensitive data includes: Match the query text word segments with the sensitive data, and determine the word frequency and inverse document frequency of the morphemes in the sensitive data based on the matching results; Calculate the correlation score of the morpheme based on the word frequency and the inverse document frequency; Perform a weighted sum calculation on the correlation scores of the morphemes to obtain the target similarity of the sensitive data.

7. The method according to claim 1, wherein The step of splicing the query text and the sensitive data text based on a preset prompt template to generate a security identification prompt for the query text includes: Obtain the prompt template, which includes text requirements for the output format, so that the output result of the large language model is a boolean value; Fill the query text and the sensitive data text into the corresponding text areas in the prompt template to generate the security identification prompt.

Images