Network security defense method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses a network security defense method and device, electronic equipment and a storage medium. The method comprises the steps of obtaining to-be-processed question data; cue words of the to-be-processed question data are obtained, aggressive cue words are filtered out, and to-be-detected cue words are obtained; determining a question normal form range and a question content range corresponding to the to-be-detected cue word; if the question normal form range and the question content range meet preset conditions, inputting the to-be-processed question data into a specified language model, the specified language model being obtained by training on the basis of aggressive question answer data and non-aggressive question answer data; and obtaining answer content which is output by the specified language model and corresponds to the to-be-processed question data. According to the method, aggressive cue words in the to-be-processed question data can be filtered out before the answer content corresponding to the to-be-processed question data is obtained through the specified language model, so that information potential safety hazards can be reduced to a certain extent.

Description

Technical Field

[0001] The present application relates to the field of information security technology, and more specifically, to a network security defense method, device, electronic device and storage medium. Background Art

[0002] In the field of network security, prompt words are usually used when implementing downstream sub-tasks such as detection, analysis, and interpretation through big models and prompt engineering methods. Prompt words may be used maliciously when used, which is manifested in prompt word injection and the use of risky prompt words to maliciously guide big models, thereby affecting the effectiveness of big models on various downstream sub-tasks; moreover, maliciously guided content can easily affect users' perception of business capabilities or leak key information, causing great information security risks. Summary of the invention

[0003] In view of the above problems, the present application proposes a network security defense method, device, electronic device and storage medium to improve the above problems.

[0004] In a first aspect, an embodiment of the present application provides a network security defense method, the method comprising: obtaining question data to be processed; obtaining prompt words of the question data to be processed, and filtering out offensive prompt words to obtain prompt words to be detected; determining a question paradigm range and a question content range corresponding to the prompt words to be detected; if the question paradigm range and the question content range meet preset conditions, inputting the question data to be processed into a specified language model, the specified language model being trained based on offensive question answer data and non-offensive question answer data; obtaining answer content corresponding to the question data to be processed output by the specified language model.

[0005] Furthermore, the step of obtaining prompt words of the question data to be processed and filtering out offensive prompt words to obtain prompt words to be detected includes: obtaining prompt words of the question data to be processed; comparing the prompt words with preset prompt words to obtain the emotional polarity of the prompt words, wherein the preset prompt words include emotional polarity identifiers; deleting prompt words corresponding to offensive emotional polarities, and using the remaining prompt words as prompt words to be detected.

[0006] Furthermore, comparing the prompt word with a preset prompt word to obtain the sentiment polarity of the prompt word includes: taking the sentiment polarity corresponding to the preset prompt word having the highest matching degree with the prompt word as the sentiment polarity of the prompt word.

[0007] Furthermore, the method of obtaining prompt words of the question data to be processed and filtering out offensive prompt words to obtain prompt words to be detected includes: obtaining prompt words of the question data to be processed; inputting the prompt words into a sentiment polarity detection model, and obtaining the sentiment polarity output by the sentiment polarity detection model as the sentiment polarity of the prompt words, wherein the sentiment polarity detection model is trained based on offensive prompt word data with labels; deleting prompt words with offensive sentiment polarity, and using the remaining prompt words as prompt words to be detected.

[0008] Furthermore, if the question paradigm range and the question content range meet preset conditions, the question data to be processed is input into a specified language model, including: if the question paradigm range belongs to a specified paradigm range and the question content range belongs to a specified business range, the question data to be processed is input into a specified language model.

[0009] Furthermore, before obtaining the answer content output by the specified language model corresponding to the question data to be processed, it also includes: obtaining the answer content to be processed output by the specified language model; filtering the answer content to be processed; obtaining the answer content corresponding to the question data to be processed output by the specified language model includes: using the filtered answer content as the answer content corresponding to the question data to be processed.

[0010] Furthermore, if the question paradigm range and the question content range meet preset conditions, before inputting the question data to be processed into the designated language model, it also includes: obtaining non-aggressive question and answer paradigm data; inputting the non-aggressive question and answer paradigm data into the language model to be trained to train the language model so that the language model understands and learns the question and answer paradigm within the non-aggressive range, and uses the trained model as the first language model; obtaining aggressive question and answer paradigm data; inputting the aggressive question and answer paradigm data into the first language model to perform adversarial training on the first language model so that the first language model understands and learns to identify and prevent aggressive questions, and uses the trained model as the designated language model.

[0011] In a second aspect, an embodiment of the present application provides a network security defense device, the device comprising: a data acquisition module, used to acquire question data to be processed; a prompt word processing module, used to acquire prompt words of the question data to be processed, and filter out offensive prompt words to obtain prompt words to be detected; a range determination module, used to determine the question paradigm range and question content range corresponding to the prompt words to be detected; a security defense module, used to input the question data to be processed into a specified language model if the question paradigm range and question content range meet preset conditions, the specified language model being trained based on offensive question answer data and non-offensive question answer data; a result acquisition module, used to obtain the answer content corresponding to the question data to be processed output by the specified language model.

[0012] In a third aspect, the present application provides an electronic device comprising one or more processors and a memory; one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the method described in the first aspect above.

[0013] In a fourth aspect, the present application provides a computer-readable storage medium, in which a program code is stored, wherein the method described in the first aspect above is executed when the program code is run.

[0014] The present application provides a network security defense method, device, electronic device and storage medium, which obtains question data to be processed; obtains prompt words of the question data to be processed, and filters out offensive prompt words to obtain prompt words to be detected; determines the question paradigm range and question content range corresponding to the prompt words to be detected; if the question paradigm range and question content range meet the preset conditions, inputs the question data to be processed into a specified language model, and the specified language model is trained based on offensive question answer data and non-offensive question answer data; obtains the answer content corresponding to the question data to be processed output by the specified language model. Thus, through the above method, it is possible to filter out offensive prompt words in the question data to be processed before obtaining the answer content corresponding to the question data to be processed through the specified language model, thereby reducing the situation where the specified language model is maliciously guided, playing a certain defensive role, and thus reducing information security risks to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0016] Figure 1 A method flow chart of a network security defense method proposed in an embodiment of the present application is shown.

[0017] Figure 2 A method flow chart of a network security defense method proposed in another embodiment of the present application is shown.

[0018] Figure 3 A method flow chart of a network security defense method proposed in yet another embodiment of the present application is shown.

[0019] Figure 4 A method flow chart of a network security defense method proposed in yet another embodiment of the present application is shown.

[0020] Figure 5 A structural block diagram of a network security defense device proposed in an embodiment of the present application is shown.

[0021] Figure 6 A structural block diagram of an electronic device for executing a network security defense method according to an embodiment of the present application is shown.

[0022] Figure 7 It is a storage unit of an embodiment of the present application for storing or carrying a program code for implementing a network security defense method according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0024] In the field of network security, when using big models and prompt engineering methods to achieve downstream sub-tasks such as detection, analysis, and interpretation, the robustness of simply using prompt words is poor, and there may be malicious use when using them, which is specifically manifested in prompt word injection, using risky prompt words to maliciously guide big models, and other behaviors, thereby affecting the effectiveness of the big model on various downstream sub-tasks, and the maliciously guided content is likely to affect users' perception of business capabilities and leak key information, etc., causing great information security risks.

[0025] In order to optimize the above-mentioned problems, a network security defense method, device, electronic device and storage medium provided in the embodiments of the present application are proposed. The method can filter out offensive prompt words in the question data to be processed before obtaining the answer content corresponding to the question data to be processed through the specified language model, thereby reducing the situation where the specified language model is maliciously guided, playing a certain defensive role, and further reducing information security risks to a certain extent.

[0026] In order to facilitate a better understanding of the solution described in the embodiments of the present application, the following briefly describes the relevant terms involved in the description of the embodiments of the present application:

[0027] Large model: Large Language Model l (LLM), which is a pre-trained model and is referred to as the large model in this application. The large model is an artificial intelligence model composed of a neural network with hundreds of millions or billions of parameters. It is trained on a large amount of text data, learns the complex patterns in the language data, and can perform a wide range of natural language understanding and generation tasks.

[0028] Prompt: Also considered as an "instruction" in terms of function, it is a form or template designed to make the large model perform well on downstream tasks. As a clue or prompt, it is added as additional text to the large model input to help the large model "recall" what it has "learned" during training, so that it can better "understand" the user's questions and give answers that better match human intentions.

[0029] Prompt Engineering: refers to the method of guiding the large model to generate the target effect answer by designing and developing prompt words without updating the model parameters.

[0030] Prompt Injection: An attack method that aims to hijack the model output and intentionally change the originally set intention behavior by using certain prompts. This attack may lead to (1) prompt word jailbreak: a common example is manipulating the model to generate some negative content. (2) prompt word leakage, a common example is obtaining model training data and illegally obtaining private information.

[0031] Few-shot Learning: The ability of a machine learning model to learn and generalize from a small number of samples.

[0032] It should be noted that the network security defense method provided in the embodiment of the present application can be applied to human-computer interaction scenarios of the human-computer question-and-answer type, or other types of human-computer interaction scenarios, and is not specifically limited. The implementation method of the present application takes the human-computer interaction scenario as an example to illustrate the human-computer interaction scenario, and the model corresponding to the human-computer interaction scenario is a human-computer interaction model, which can be a large model of the human-computer question-and-answer type, and the human-computer interaction model can be composed of various detection modules (for example, multi-model integration).

[0033] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0034] See also Figure 1 An embodiment of the present application provides a network security defense method, which can be applied to electronic devices. A human-machine question-and-answer model (machine question-and-answer model) is configured in the electronic device. The application field of the human-machine question-and-answer model is not limited. For example, it can be a human-machine question-and-answer model in the fields of medical care, catering, education, transportation, etc. The user can obtain the corresponding answer content by asking questions to the human-machine question-and-answer model. The method includes:

[0035] Step S110: Obtain question data to be processed.

[0036] The question data to be processed can be understood as the question content input by the user to the human-computer interaction model, and the question content can be input in the form of text input, voice input, or a combination of text input and voice input.

[0037] Optionally, if the input method of the question data to be processed is text input, it can be user-defined input content, or it can be input content by selecting preset text content; if the input method of the question data to be processed is voice input, the voice data can be converted into text content as input content.

[0038] As an implementation method, if the user asks the human-computer interaction model a question once, the question content of this inquiry can be used as question data to be processed; if the user asks the human-computer interaction model multiple times (for example, continuous questions), the question content of multiple inquiries can all be used as question data to be processed.

[0039] Step S120: Obtain prompt words of the question data to be processed, and filter out offensive prompt words to obtain prompt words to be detected.

[0040] In the implementation mode of the present application, when the human-computer interaction model obtains the question data to be processed, it will not immediately give the corresponding answer content, but will first obtain the prompt words of the question data to be processed, and filter out the offensive prompt words in the obtained prompt words, and use the prompt words obtained after filtering as the prompt words to be detected. Among them, the offensive prompt words may include prompt words with malicious orientation, prompt words with illegal or extreme speech, and prompt words with word order problems, etc. Each question data to be processed may include at least one prompt word. By filtering out the offensive prompt words in the question data to be processed, the situation where the human-computer interaction model is maliciously guided can be reduced.

[0041] Step S130: Determine the scope of the question paradigm and the scope of the question content corresponding to the prompt word to be detected.

[0042] Among them, the question paradigm scope can be configured as which parts the question data needs to be composed of. If it does not belong to this paradigm scope, the answer will be refused; the question content scope can be configured as the question data needs to belong to a certain business scope. If it does not belong to the business scope, the answer will be refused. That is, when designing prompt words, the downstream sub-task scope of the human-computer interaction model can be strictly defined in terms of business, so that only questions related to the business scenario are retained, thereby achieving defense against malicious guidance.

[0043] In this embodiment, after filtering out the offensive prompt words in the question data to be processed, in order to further improve the information security defense capability of the human-computer interaction model, the range of question paradigms and question content corresponding to the prompt words to be detected can be determined to determine whether the current question content exceeds the business scope of the human-computer interaction model. Optionally, if the range of question paradigms and question content corresponding to the prompt words to be detected exceed the business scope of the human-computer interaction model, the human-computer interaction model can refuse to answer.

[0044] The scope of questioning paradigms can include questions within a reasonable scope and questions that are offensive. Among them, questions that are offensive may include jailbreaking, injection, leakage and other types. Optionally, jailbreaking can be understood as the user maliciously guiding the human-computer interaction model to step out of the pre-set task scope, for example, imposing an identity, guiding the human-computer interaction model to answer content that is irrelevant to the task, etc.; injection can be understood as turning the user's question content into executable instructions and tampering with the execution content of the human-computer interaction model; leakage can be understood as the user obtaining the details, parameters and other information of the human-computer interaction model in some way. Questions within a reasonable scope do not include questions with offensive prompt words.

[0045] As an implementation method, the concepts and defense purposes of prompt words for known attack types such as jailbreak, injection, and leakage can be defined in advance in the prompt words, thereby guiding the human-computer interaction model to understand the scope of the question paradigm, complete downstream subtasks in response to user questions in the direction of defense, or refuse to answer questions that exceed the scope of the question paradigm.

[0046] As a specific implementation method, some reasonable range of questions and offensive questions can be obtained, and then the ability of small sample learning of the large model can be used to guide the human-computer interaction model to better understand the scope of the question paradigm, complete downstream subtasks in response to user questions in a defensive direction, or refuse to answer questions that are beyond the scope of the question paradigm. In this way, the scope of the question paradigm of the question content can be determined by the human-computer interaction model's ability to understand the scope of the question paradigm.

[0047] As an implementation method for the design process of the scope of question content, business data and real question content can be obtained as training samples, and the ability of large models to learn from small samples can be used to guide the human-computer interaction model to better understand the scope of question content, complete downstream subtasks for user questions in a defensive direction, or refuse to answer questions that are beyond the business scope. In this way, the scope of question content can be determined by the human-computer interaction model's ability to understand the scope of question content.

[0048] Step S140: If the question paradigm range and question content range meet preset conditions, the question data to be processed is input into a specified language model, where the specified language model is trained based on aggressive question answer data and non-aggressive question answer data.

[0049] Among them, the designated language model can be understood as the question-answering module in the human-computer interaction model, and the designated language model is trained based on offensive question-answering data and non-offensive question-answering data. The question data to be processed will only be input into the designated language model when the offensive prompt words in the question data to be processed are filtered out, and the question paradigm range and question content range corresponding to the prompt words to be detected obtained after filtering meet the preset conditions (it should be noted that the question data to be processed here represents the data from which the offensive prompt words have been filtered out, and the question paradigm range and question content range meet the preset conditions), thereby achieving better information security defense.

[0050] Among them, the preset conditions may include that the question paradigm range belongs to the specified paradigm range and the question content range belongs to the specified business range. Here, the specific content of the specified business range may not be limited, and further, the specific content of the specified paradigm range may not be limited.

[0051] As an implementation mode, if the question paradigm range belongs to a specified paradigm range and the question content range belongs to a specified business range, the question data to be processed may be input into a specified language model.

[0052] Step S150: Obtaining answer content output by the specified language model corresponding to the question data to be processed.

[0053] As an implementation method, in order to improve the professionalism and standardization of the answer content of the human-computer interaction model, before obtaining the answer content corresponding to the question data to be processed output by the specified language model, the answer content to be processed output by the specified language model can be obtained, and then the answer content to be processed can be filtered, that is, the answer content is identified when it is output by the specified language model, so as to further avoid the situation where the large model is maliciously guided to cause leakage or other illegal purposes.

[0054] In this manner, when obtaining the answer content corresponding to the question data to be processed output by the specified language model, the filtered answer content can be used as the answer content corresponding to the question data to be processed.

[0055] The present embodiment provides a network security defense method, which obtains question data to be processed; obtains prompt words of the question data to be processed, and filters out offensive prompt words to obtain prompt words to be detected; determines the question paradigm range and question content range corresponding to the prompt words to be detected; if the question paradigm range and question content range meet the preset conditions, inputs the question data to be processed into a specified language model, the specified language model is trained based on offensive question answer data and non-offensive question answer data; obtains the answer content corresponding to the question data to be processed output by the specified language model. Thus, through the above method, it is possible to filter out offensive prompt words in the question data to be processed before obtaining the answer content corresponding to the question data to be processed through the specified language model, thereby reducing the situation where the specified language model is maliciously guided, playing a certain defensive role, and further reducing information security risks to a certain extent.

[0056] See also Figure 2 Another embodiment of the present application provides a network security defense method, which can be applied to an electronic device, and the method includes:

[0057] Step S210: Obtain question data to be processed.

[0058] The specific implementation of step S210 may refer to the relevant description of step S110 in the aforementioned embodiment, which will not be repeated here.

[0059] Step S220: Obtain prompt words of the question data to be processed.

[0060] The specific implementation of step S220 may refer to the relevant description of step S120 in the aforementioned embodiment, which will not be repeated here.

[0061] Step S230: Compare the prompt word with a preset prompt word to obtain the emotion polarity of the prompt word, wherein the preset prompt word includes an emotion polarity identifier.

[0062] In the process of filtering the offensive prompt words in the question data to be processed, as an implementation method, the prompt words of the question data to be processed can be directly compared (matched) with the preset prompt words to obtain the emotional polarity of the prompt words, and the preset prompt words include pre-marked emotional polarity identifiers. The specific type of emotional polarity may not be limited, for example, it may include positive, illegal, sensitive, malicious and other types of emotional polarity.

[0063] As an implementation method, the emotion polarity corresponding to the preset prompt word with the highest matching degree (which can be understood as content similarity here, and the specific method of obtaining content similarity is not repeated here) with the prompt word can be used as the emotion polarity of the prompt word.

[0064] Step S240: Delete the prompt words corresponding to the aggressive emotional polarity, and use the remaining prompt words as prompt words to be detected.

[0065] As a way to reduce the malicious guidance of the model, after obtaining the emotional polarity of the prompt words, the prompt words corresponding to the offensive emotional polarity can be deleted, and the remaining prompt words can be used as prompt words to be detected.

[0066] For example, assuming that the question data to be processed includes prompt word A, prompt word B, prompt word C and prompt word D, and assuming that prompt word A and prompt word C are both offensive prompt words, then prompt word A and prompt word C in the question data to be processed can be deleted, and then prompt word B and prompt word D can be used as prompt words to be detected.

[0067] Step S250: Determine the question pattern range and question content range corresponding to the prompt word to be detected.

[0068] The specific implementation of step S250 may refer to the relevant description of step S130 in the aforementioned embodiment, which will not be repeated here.

[0069] Step S260: If the question paradigm range and question content range meet preset conditions, the question data to be processed is input into a specified language model, and the specified language model is trained based on aggressive question answer data and non-aggressive question answer data.

[0070] The specific implementation of step S260 may refer to the relevant description of step S140 in the aforementioned embodiment, which will not be repeated here.

[0071] Step S270: Obtaining the answer content output by the specified language model corresponding to the question data to be processed.

[0072] The specific implementation of step S270 may refer to the relevant description of step S150 in the aforementioned embodiment, which will not be repeated here.

[0073] The present embodiment provides a network security defense method, which obtains question data to be processed; obtains prompt words of the question data to be processed, compares the prompt words with preset prompt words to obtain the emotional polarity of the prompt words, and the preset prompt words include emotional polarity identifiers; deletes prompt words corresponding to offensive emotional polarity, and uses the remaining prompt words as prompt words to be detected; determines the range of question paradigms and question content corresponding to the prompt words to be detected; if the range of question paradigms and question content meets the preset conditions, inputs the question data to be processed into a specified language model, and the specified language model is trained based on offensive question answer data and non-offensive question answer data; obtains the answer content corresponding to the question data to be processed output by the specified language model. Thus, through the above method, it is possible to filter out the offensive prompt words in the question data to be processed before obtaining the answer content corresponding to the question data to be processed through the specified language model, thereby reducing the situation where the specified language model is maliciously guided, playing a certain defensive role, and thus reducing information security risks to a certain extent.

[0074] At the same time, by directly comparing the prompt words of the question data to be processed with the preset prompt words to obtain the emotional polarity of the prompt words, it is possible to quickly determine the aggressive prompt words, and then quickly filter out the aggressive prompt words, thereby improving the efficiency of network security defense.

[0075] See also Figure 3 Another embodiment of the present application provides a network security defense method, which can be applied to an electronic device, and the method includes:

[0076] Step S310: Obtain question data to be processed.

[0077] The specific implementation of step S310 may refer to the relevant description of step S110 in the aforementioned embodiment, which will not be repeated here.

[0078] Step S320: Obtain prompt words for the question data to be processed.

[0079] The specific implementation of step S320 may refer to the relevant description of step S120 in the aforementioned embodiment, which will not be repeated here.

[0080] Step S330: inputting the prompt word into a sentiment polarity detection model, and obtaining the sentiment polarity output by the sentiment polarity detection model as the sentiment polarity of the prompt word, wherein the sentiment polarity detection model is trained based on offensive prompt word data with labels.

[0081] As an implementation method, a model for detecting the sentiment polarity of prompt words can be pre-trained with labeled offensive prompt word data. During the training process, in addition to being able to learn to judge the sentiment polarity of the training data, the model can also predict the sentiment polarity of data that has never been seen. When the recall rate of the trained model meets user needs, the obtained model can be used as a sentiment polarity detection model.

[0082] Furthermore, in the process of filtering the offensive prompt words in the question data to be processed, the prompt words can be input into the trained sentiment polarity detection model, and the sentiment polarity output by the sentiment polarity detection model can be obtained as the sentiment polarity of the prompt words.

[0083] Step S340: Delete the prompt words with an aggressive sentiment polarity, and use the remaining prompt words as prompt words to be detected.

[0084] After obtaining the sentiment polarity of the prompt words of the question data to be processed, if there are prompt words with offensive sentiment polarity, the prompt words with offensive sentiment polarity can be deleted, and the remaining prompt words are used as prompt words to be detected.

[0085] Step S350: Determine the scope of the question paradigm and the scope of the question content corresponding to the prompt word to be detected.

[0086] The specific implementation of step S350 may refer to the relevant description of step S130 in the aforementioned embodiment, which will not be repeated here.

[0087] Step S360: If the question paradigm range and question content range meet preset conditions, the question data to be processed is input into a specified language model, and the specified language model is trained based on aggressive question answer data and non-aggressive question answer data.

[0088] The specific implementation of step S360 may refer to the relevant description of step S140 in the aforementioned embodiment, which will not be repeated here.

[0089] Step S370: Obtaining the answer content output by the specified language model corresponding to the question data to be processed.

[0090] The specific implementation of step S370 may refer to the relevant description of step S150 in the aforementioned embodiment, which will not be repeated here.

[0091] The present embodiment provides a network security defense method, which is obtained by obtaining question data to be processed; obtaining prompt words of the question data to be processed, inputting the prompt words into a sentiment polarity detection model, and obtaining the sentiment polarity output by the sentiment polarity detection model as the sentiment polarity of the prompt word, wherein the sentiment polarity detection model is trained based on offensive prompt word data with labels; deleting prompt words with offensive sentiment polarity, and using the remaining prompt words as prompt words to be detected; determining the range of question paradigms and the range of question content corresponding to the prompt words to be detected; if the range of question paradigms and the range of question content meet the preset conditions, inputting the question data to be processed into a designated language model, wherein the designated language model is trained based on offensive question answer data and non-offensive question answer data; and obtaining the answer content corresponding to the question data to be processed output by the designated language model. Thus, the above method can filter out the offensive prompt words in the question data to be processed before obtaining the answer content corresponding to the question data to be processed through the designated language model, thereby reducing the situation where the designated language model is maliciously guided, playing a certain defensive role, and further reducing information security risks to a certain extent.

[0092] At the same time, the sentiment polarity detection model obtained through pre-training can judge the sentiment polarity of the prompt words of the question data to be processed, and then filter out the offensive prompt words, so as to make timely judgments on illegal or extreme remarks and questions in the question content, thereby minimizing the situation where the large model is maliciously guided.

[0093] See also Figure 4 Another embodiment of the present application provides a network security defense method, which can be applied to electronic devices, and the method includes:

[0094] Step S410: Acquire non-aggressive question-answering paradigm data.

[0095] Among them, non-aggressive can be understood as non-malicious and normal. As an implementation method, non-aggressive question-answering paradigm data in some subtasks of the human-computer interaction model can be obtained in advance as training data for the language model in the human-computer interaction model.

[0096] Step S420: Input the non-aggressive question-answer paradigm data into the language model to be trained to train the language model so that the language model can understand and learn the question-answer paradigm within the non-aggressive range, and use the trained model as the first language model.

[0097] As an implementation method, non-offensive question-answering paradigm data may be input into a language model to be trained to train the language model so that the language model understands and learns non-offensive question-answering paradigms, and then the trained model is used as the first language model.

[0098] Step S430: Obtaining aggressive question-answering paradigm data.

[0099] Furthermore, in order to improve the first language model's ability to identify and prevent attacks, offensive question-answering paradigm data can be obtained to train the first language model, so that the first language model can not only understand and learn to identify non-offensive question-answering paradigms, and answer non-offensive question content; but also understand and learn to identify and prevent aggressive question content, and refuse to answer offensive question content.

[0100] Optionally, the refusal to answer here may include not answering or answering with specified content, and the specific content of the specified content may not be limited. For example, the specified content may be "Your input is incorrect, please re-enter."

[0101] Step S440: Input the aggressive question-answering paradigm data into the first language model to perform adversarial training on the first language model, so that the first language model can understand and learn to identify and prevent aggressive questions, and use the trained model as the designated language model.

[0102] As an implementation method, aggressive question-answering paradigm data may be input into a first language model to perform adversarial training on the first language model, so that the first language model can understand and learn to identify and prevent aggressive questions, and then use the trained model as the designated language model.

[0103] It should be noted that, in this embodiment, the contents of step S410 to step S440 may be executed before the contents of step S450-step S470, may be executed at the same time as the contents of step S450-step S470 are executed, or may be executed after the contents of step S450-step S470 are executed and before step S480, and the specific execution order may not be limited.

[0104] Step S450: Obtain question data to be processed.

[0105] The specific implementation of step S450 may refer to the relevant description of step S110 in the aforementioned embodiment, which will not be repeated here.

[0106] Step S460: Obtain prompt words of the question data to be processed, and filter out offensive prompt words to obtain prompt words to be detected.

[0107] The specific implementation of step S460 may refer to the relevant description of step S120 in the aforementioned embodiment, which will not be repeated here.

[0108] Step S470: Determine the question pattern range and question content range corresponding to the prompt word to be detected.

[0109] The specific implementation of step S470 may refer to the relevant description of step S130 in the aforementioned embodiment, which will not be repeated here.

[0110] Step S480: If the question paradigm range and question content range meet preset conditions, the question data to be processed is input into a specified language model, and the specified language model is trained based on aggressive question answer data and non-aggressive question answer data.

[0111] The specific implementation of step S480 may refer to the relevant description of step S140 in the aforementioned embodiment, which will not be repeated here.

[0112] Step S490: Obtaining the answer content output by the specified language model corresponding to the question data to be processed.

[0113] The specific implementation of step S490 may refer to the relevant description of step S150 in the aforementioned embodiment, which will not be repeated here.

[0114] The human-computer interaction model in the implementation manner of the present application can reduce the situation where the model is maliciously guided by at least one of the three aspects of pre-training a specified language model, pre-setting the question paradigm range and the question content range, and judging the emotional polarity of the prompt word through prompt engineering, so as to more effectively defend against network information security.

[0115] The present embodiment provides a network security defense method, which includes obtaining non-offensive question-answering paradigm data; inputting the non-offensive question-answering paradigm data into a language model to be trained to train the language model, so that the language model understands and learns the question-answering paradigm within the non-offensive range, and using the trained model as the first language model; obtaining aggressive question-answering paradigm data; inputting the aggressive question-answering paradigm data into the first language model to perform adversarial training on the first language model, so that the first language model understands and learns to identify and prevent offensive questions, and using the trained model as the designated language model; obtaining question data to be processed; obtaining prompt words of the question data to be processed, and filtering out the offensive prompt words to obtain prompt words to be detected; determining the question paradigm range and question content range corresponding to the prompt words to be detected; if the question paradigm range and question content range meet preset conditions, inputting the question data to be processed into a designated language model, the designated language model being trained based on the aggressive question-answering data and the non-offensive question-answering data; and obtaining the answer content corresponding to the question data to be processed output by the designated language model. Therefore, through the above method, it is possible to filter out offensive prompt words in the question data to be processed before obtaining the answer content corresponding to the question data to be processed through the specified language model, thereby reducing the situation where the specified language model is maliciously guided, playing a certain defensive role, and further reducing information security risks to a certain extent.

[0116] At the same time, by pre-training a specified language model for obtaining the answer content corresponding to the question data, the efficiency of network security defense can be improved to a certain extent.

[0117] See also Figure 5 The embodiment of the present application provides a network security defense device 500, the device 500 comprising:

[0118] The data acquisition module 510 is used to acquire the question data to be processed.

[0119] The prompt word processing module 520 is used to obtain the prompt words of the question data to be processed, and filter out the offensive prompt words to obtain the prompt words to be detected.

[0120] As an implementation method, the prompt word processing module 520 can be used to obtain the prompt words of the question data to be processed; compare the prompt words with preset prompt words to obtain the emotional polarity of the prompt words, and the preset prompt words include emotional polarity identifiers; delete the prompt words corresponding to the aggressive emotional polarity, and use the remaining prompt words as prompt words to be detected.

[0121] Furthermore, comparing the prompt word with a preset prompt word to obtain the sentiment polarity of the prompt word includes: taking the sentiment polarity corresponding to the preset prompt word having the highest matching degree with the prompt word as the sentiment polarity of the prompt word.

[0122] As another implementation, the prompt word processing module 520 can be used to obtain the prompt words of the question data to be processed; input the prompt words into the sentiment polarity detection model, and obtain the sentiment polarity output by the sentiment polarity detection model as the sentiment polarity of the prompt words, wherein the sentiment polarity detection model is trained based on labeled offensive prompt word data; delete the prompt words with offensive sentiment polarity, and use the remaining prompt words as the prompt words to be detected.

[0123] The range determination module 530 is used to determine the range of the question paradigm and the range of the question content corresponding to the prompt word to be detected.

[0124] The security defense module 540 is used to input the question data to be processed into a specified language model if the question paradigm range and the question content range meet preset conditions. The specified language model is trained based on aggressive question answer data and non-aggressive question answer data.

[0125] As an implementation manner, the security defense module 540 may be used to input the to-be-processed question data into a specified language model if the question paradigm range belongs to a specified paradigm range and the question content range belongs to a specified business range.

[0126] As an implementation mode, the device 500 may further include a model training module, which is used to obtain non-offensive question-answer paradigm data before inputting the question data to be processed into the designated language model if the question paradigm range and the question content range meet the preset conditions; input the non-offensive question-answer paradigm data into the language model to be trained to train the language model so that the language model understands and learns the question-answer paradigm within the non-offensive range, and uses the trained model as the first language model; obtain aggressive question-answer paradigm data; input the aggressive question-answer paradigm data into the first language model to perform adversarial training on the first language model so that the first language model understands and learns to identify and prevent aggressive questions, and uses the trained model as the designated language model.

[0127] The result acquisition module 550 is used to obtain the answer content output by the specified language model and corresponding to the question data to be processed.

[0128] As an implementation mode, the device 500 may further include a content filtering module, which is used to obtain the answer content to be processed output by the specified language model before obtaining the answer content corresponding to the question data to be processed output by the specified language model; and filter the answer content to be processed. In this way, the result acquisition module 550 can be used to use the filtered answer content as the answer content corresponding to the question data to be processed.

[0129] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0130] In several embodiments provided in the present application, the coupling or direct coupling or communication connection between the modules shown or discussed may be an indirect coupling or communication connection through some interfaces, devices or modules, which may be electrical, mechanical or other forms.

[0131] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.

[0132] The following will be combined Figure 6 An electronic device provided by the present application is described.

[0133] See also Figure 6 Based on the above network security defense method and device, the embodiment of the present application also provides an electronic device 100 that can execute the above network security defense method. The electronic device 100 includes a memory 102, one or more (only one is shown in the figure) processors 104 coupled to each other, and a communication module 106, and the memory 102 and the processor 104 are connected by a communication line. Among them, the memory 102 stores a program that can execute the content of the above embodiment. The processor 104 can execute the program stored in the memory 102. The communication module 106 is used to provide a communication interface for the electronic device 100 to receive the video frame to be processed.

[0134] Among them, the processor 104 may include one or more processing cores. The processor 104 uses various interfaces and lines to connect various parts of the entire electronic device 100, and executes various functions and processes data of the electronic device 100 by running or executing instructions, programs, code sets or instruction sets stored in the memory 102, and calling data stored in the memory 102. Optionally, the processor 104 can be implemented in at least one hardware form of digital signal processing (DSP), field programmable gate array (FPGA), and programmable logic array (PLA). The processor 104 can integrate one or a combination of a central processing unit (CPU), a graphics processor (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 104, but may be implemented separately through a communication chip.

[0135] The memory 102 may include a random access memory (RAM) or a read-only memory (ROM). The memory 102 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 102 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the aforementioned embodiments, etc. The data storage area may also store data (such as a phone book, audio and video data, chat record data) created by the electronic device 100 during use.

[0136] Please refer to Figure 7 , which shows a structural block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable storage medium 600 stores program codes, which can be called by a processor to execute the method described in the above method embodiment.

[0137] The computer readable storage medium 600 can be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. Optionally, the computer readable storage medium 600 includes a non-transient computer readable medium (non-transitory computer-readable storage medium). The computer readable storage medium 600 has storage space for program code 610 that performs any method step in the above method. These program codes can be read from or written to one or more computer program products. The program code 610 can be compressed, for example, in an appropriate form.

[0138] In summary, the present application provides a network security defense method, device, electronic device and storage medium, which obtains the question data to be processed; obtains the prompt words of the question data to be processed, and filters out the offensive prompt words to obtain the prompt words to be detected; determines the question paradigm range and question content range corresponding to the prompt words to be detected; if the question paradigm range and question content range meet the preset conditions, the question data to be processed is input into the specified language model, and the specified language model is trained based on the offensive question answer data and the non-offensive question answer data; obtains the answer content corresponding to the question data to be processed output by the specified language model. Thus, through the above method, it is possible to filter out the offensive prompt words in the question data to be processed before obtaining the answer content corresponding to the question data to be processed through the specified language model, thereby reducing the situation where the specified language model is maliciously guided, playing a certain defensive role, and thus reducing information security risks to a certain extent.

[0139] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A network security defense method, characterized in that: The method comprises: Get the question data to be processed; Obtaining prompt words of the question data to be processed, and filtering out offensive prompt words to obtain prompt words to be detected; Determine the scope of question paradigm and question content corresponding to the prompt word to be detected; If the question paradigm range and the question content range meet the preset conditions, the question data to be processed is input into a specified language model, where the specified language model is trained based on the aggressive question answer data and the non-aggressive question answer data; Obtain answer content output by the specified language model corresponding to the question data to be processed.

2. The method according to claim 1, characterized in that The step of obtaining prompt words of the question data to be processed and filtering out offensive prompt words to obtain prompt words to be detected includes: Obtaining a prompt word of the question data to be processed; Comparing the prompt word with a preset prompt word to obtain the emotional polarity of the prompt word, wherein the preset prompt word includes an emotional polarity identifier; Delete the prompt words corresponding to the aggressive emotional polarity, and use the remaining prompt words as the prompt words to be detected.

3. The method according to claim 2, characterized in that The step of comparing the prompt word with a preset prompt word to obtain the emotional polarity of the prompt word includes: The emotion polarity corresponding to the preset prompt word with the highest matching degree with the prompt word is used as the emotion polarity of the prompt word.

4. The method according to claim 1, characterized in that The step of obtaining prompt words of the question data to be processed and filtering out offensive prompt words to obtain prompt words to be detected includes: Obtaining a prompt word of the question data to be processed; Inputting the prompt word into a sentiment polarity detection model, and obtaining the sentiment polarity output by the sentiment polarity detection model as the sentiment polarity of the prompt word, wherein the sentiment polarity detection model is trained based on offensive prompt word data with labels; Delete the prompt words with aggressive emotional polarity, and use the remaining prompt words as the prompt words to be detected.

5. The method according to claim 1, characterized in that: If the question paradigm range and the question content range meet the preset conditions, inputting the question data to be processed into a specified language model includes: If the question paradigm range belongs to the specified paradigm range and the question content range belongs to the specified business range, the to-be-processed question data is input into the specified language model.

6. The method according to claim 1, characterized in that Before obtaining the answer content output by the specified language model and corresponding to the question data to be processed, the method further includes: Obtaining the answer content to be processed output by the specified language model; Filtering the answer content to be processed; The obtaining the answer content output by the specified language model and corresponding to the question data to be processed includes: The filtered answer content is used as the answer content corresponding to the question data to be processed.

7. The method according to any one of claims 1 to 6, characterized in that: If the scope of the question paradigm and the scope of the question content meet the preset conditions, before inputting the question data to be processed into the specified language model, the method further includes: Obtain non-aggressive question-answer paradigm data; Inputting the non-aggressive question-answering paradigm data into a language model to be trained to train the language model, so that the language model understands and learns the question-answering paradigm within a non-aggressive range, and using the trained model as a first language model; Obtain data on aggressive question-answering paradigms; The aggressive question-answering paradigm data is input into the first language model to perform adversarial training on the first language model, so that the first language model can understand and learn to identify and prevent aggressive questions, and the trained model is used as the designated language model.

8. A network security defense device, characterized in that: The device comprises: A data acquisition module is used to acquire the question data to be processed; A prompt word processing module, used for obtaining prompt words of the question data to be processed, and filtering out offensive prompt words to obtain prompt words to be detected; A range determination module, used to determine the range of question paradigms and question content corresponding to the prompt word to be detected; A security defense module, configured to input the to-be-processed question data into a specified language model if the question paradigm range and the question content range meet preset conditions, wherein the specified language model is trained based on the offensive question answer data and the non-offensive question answer data; The result acquisition module is used to obtain the answer content output by the specified language model and corresponding to the question data to be processed.

9. An electronic device, characterized in that: including one or more processors and memory; One or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, wherein when the program codes are executed by a processor, the method according to any one of claims 1 to 7 is executed.

Images