Data enhancement method and device

Abstract

The invention provides a data enhancement method and device, is applied to the technical field of network security, and is used for performing effective data enhancement on a test set and improving richness and applicability of a copy text. The method comprises the following steps: acquiring a plurality of to-be-detected texts; inputting each to-be-detected text into a large language model for processing, and obtaining a reply text of each to-be-detected text output by the large language model; evaluating the reply text of each to-be-detected text according to a first preset rule to obtain a first score of each to-be-detected text; inputting each to-be-detected text into the evaluation model for processing, and obtaining a second score of the evaluation model for each to-be-detected text; determining a harmful score of each to-be-detected text based on the first score and the second score of each to-be-detected text; obtaining at least one to-be-detected text of which the harmful score is greater than a preset threshold value; and copying the at least one to-be-detected text to obtain at least one copy text.

Description

Technical Field

[0001] The present application relates to the field of network security technology, and in particular to a data enhancement method and device. Background Art

[0002] In the current information age, large language models are increasingly threatened by various security attacks. However, the test sets used to test and evaluate large language models are relatively small and generally of low quality, which cannot fully cover potential security threats. In addition, since harmful texts that attack large language models are often hidden, specific, and different, the effect of data enhancement by only screening the existing test sets through a single model is poor, and it is impossible to effectively deal with the ever-changing security challenges.

[0003] In view of this, how to effectively enhance the data of the test set and improve the richness and applicability of the imitation text is an urgent problem to be solved. Summary of the invention

[0004] The present application provides a data enhancement method and device for realizing effective data enhancement of a test set and improving the richness and applicability of imitation text.

[0005] In a first aspect, an embodiment of the present application provides a data enhancement method, which is applied to any electronic device with processing capabilities, and the method includes:

[0006] Acquire multiple texts to be tested, where the multiple texts to be tested are harmful texts that pose a security threat to the large language model;

[0007] Input each of the multiple texts to be detected into the large language model for processing, and obtain a reply text of each text to be detected output by the large language model; evaluate the reply text of each text to be detected according to a first preset rule to obtain a first score for each text to be detected; the first score is used to indicate the concealment degree of each text to be detected;

[0008] Each text to be detected is input into the evaluation model for processing, and a second score of each text to be detected is obtained by the evaluation model; the evaluation model is used to evaluate the maliciousness of each text to be detected;

[0009] Determine the harmfulness score of each to-be-detected text based on the first score and the second score of each to-be-detected text;

[0010] At least one text to be detected whose harmfulness score is greater than a preset threshold is obtained; and at least one text to be detected is imitated to obtain at least one imitated text.

[0011] Optionally, the reply text of each text to be detected is evaluated according to a first preset rule to obtain a first score for each text to be detected, including: determining whether the reply text contains a preset refusal to answer word, the refusal to answer the question or request contained in the text to be detected; if contained, the first score is set to a first value, the first value is used to indicate that the concealment level of the text to be detected is low; if not contained, the reply text is input into a classification model for classification processing, the classification model is used to identify whether the reply text is a normal reply text; if the result output by the classification model is that the reply text is a normal reply text, the first score is set to a second value; the second value is used to indicate that the concealment level of the text to be detected is high; if the result output by the classification model is that the reply text is a normal reply text, the first score is set to the first value.

[0012] Optionally, the large language model is a plurality of large language models with different performances; the reply text of each text to be detected is evaluated according to a first preset rule to obtain a first score for each text to be detected, including: according to the first preset rule, the reply text of each large language model in the plurality of large language models to each text to be detected is evaluated to obtain a third score for the text to be detected by each large language model; according to the weight of each large language model and the third score for each text to be detected, the first score for each text to be detected is obtained by weighted average calculation; the weight of each large language model is determined by the performance of each large language model.

[0013] Optionally, each text to be detected is input into the evaluation model for processing to obtain a second score of each text to be detected by the evaluation model, including: inputting each text to be detected into the evaluation model, calculating the correlation between each text to be detected and the violation policy through evaluation according to a preset violation policy, evaluating each text to be detected according to the correlation, and obtaining a second score for each text to be detected.

[0014] Optionally, determining the harmfulness score of each text to be detected based on the first score and the second score of each text to be detected includes: calculating a weighted average of the first score and the second score of each text to be detected to obtain the harmfulness score of each text to be detected.

[0015] Optionally, at least one text to be detected is imitated to obtain at least one imitated text, including: inputting each text to be detected in the at least one text to be detected into a topic generation model, performing topic analysis and identification on each text to be detected through the topic generation model, and determining the topic of each text to be detected; extracting the encoding method of each text to be detected based on a regular expression; inputting each text to be detected into a large language model, performing semantic analysis on each text to be detected through the large language model, and extracting the semantic logic and language expression in each text to be detected; inputting each text to be detected into a syntactic analysis model, performing syntactic analysis on each text to be detected through the syntactic analysis model, and extracting the syntactic structure of each text to be detected; and imitating each text to be detected based on at least one of the topic, encoding method, language logic, language expression method and syntactic structure.

[0016] In a second aspect, an embodiment of the present application provides a data enhancement device, including:

[0017] The acquisition module is used to: obtain multiple texts to be tested, where the multiple texts to be tested are harmful texts that pose a security threat to the large language model;

[0018] An evaluation module is used to: input each of the multiple texts to be detected into a large language model for processing, and obtain a reply text of each text to be detected output by the large language model; evaluate the reply text of each text to be detected according to a first preset rule to obtain a first score for each text to be detected; the first score is used to indicate the concealment degree of each text to be detected; input each text to be detected into an evaluation model for processing, and obtain a second score of each text to be detected by the evaluation model; the evaluation model is used to evaluate the maliciousness of each text to be detected; and determine the harmfulness score of each text to be detected based on the first score and the second score of each text to be detected;

[0019] The imitation writing module is used to: obtain at least one text to be detected whose harmfulness score is greater than a preset threshold; imitate the at least one text to be detected to obtain at least one imitated text.

[0020] Optionally, when the evaluation module evaluates the reply text of each text to be detected according to the first preset rule and obtains the first score of each text to be detected, it is specifically used to: determine whether the reply text contains a preset refusal to answer words, the refusal to answer the questions or requirements contained in the text to be detected; if contained, the first score is set to a first value, the first value is used to indicate that the concealment level of the text to be detected is low; if not contained, the reply text is input into the classification model for classification processing, the classification model is used to identify whether the reply text is a normal reply text; if the result output by the classification model is that the reply text is a normal reply text, the first score is set to a second value; the second value is used to indicate that the concealment level of the text to be detected is high; if the result output by the classification model is that the reply text is a normal reply text, the first score is set to a second value; the second value is used to indicate that the concealment level of the text to be detected is high; if the result output by the classification model is that the reply text is an abnormal reply text, the first score is set to the first value.

[0021] Optionally, the large language model is a plurality of large language models with different performances; when the evaluation module evaluates the reply text of each text to be detected according to the first preset rule to obtain the first score of each text to be detected, the evaluation module is also used to: evaluate the reply text of each large language model in the plurality of large language models to each text to be detected according to the first preset rule to obtain the third score of each large language model for the text to be detected; obtain the first score of each text to be detected by weighted average calculation according to the weight of each large language model and the third score of each text to be detected; the weight of each large language model is determined by the performance of each large language model.

[0022] Optionally, when the evaluation module inputs each text to be detected into the evaluation model for processing and obtains the second score of the evaluation model for each text to be detected, it is specifically used to: input each text to be detected into the evaluation model, calculate the correlation between each text to be detected and the violation policy through evaluation according to the preset violation policy, evaluate each text to be detected according to the correlation, and obtain the second score of each text to be detected.

[0023] Optionally, when determining the harmfulness score of each text to be detected based on the first score and the second score of each text to be detected, the evaluation module is specifically used to: calculate a weighted average of the first score and the second score of each text to be detected to obtain the harmfulness score of each text to be detected.

[0024] Optionally, when the imitation module imitates at least one text to be detected and obtains at least one imitation text, it is specifically used to: input each text to be detected in the at least one text to be detected into a topic generation model, perform topic analysis and identification on each text to be detected through the topic generation model, and determine the topic of each text to be detected; extract the encoding method of each text to be detected based on a regular expression; input each text to be detected into a large language model, perform semantic analysis on each text to be detected through the large language model, and extract the semantic logic and language expression in each text to be detected; input each text to be detected into a syntactic analysis model, perform syntactic analysis on each text to be detected through the syntactic analysis model, and extract the syntactic structure of each text to be detected; imitate each text to be detected based on at least one of the topic, encoding method, language logic, language expression method and syntactic structure.

[0025] In a third aspect, an embodiment of the present application provides an electronic device, comprising at least one processor, wherein the at least one processor is used to execute a computer program stored in a memory so that a method as in the first aspect or any optional implementation of the first aspect is implemented.

[0026] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which is used to store instructions. When the instructions are executed, the method in the first aspect or any optional implementation of the first aspect is implemented.

[0027] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program code, which, when executed on a computer, enables the method in the first aspect or any optional implementation of the first aspect to be implemented.

[0028] In an embodiment of the present application, each text to be detected is processed based on a large language model, a reply text of each text to be detected output by the large language model is obtained, and the reply text of each text to be detected is evaluated according to a first preset rule, so as to determine whether the large language model can identify the text to be detected as a harmful text, and then the concealment degree of each text to be detected can be determined, and the concealment degree of each text to be detected is quantified by a first score; the maliciousness degree of each text to be detected is quantified based on the evaluation model to obtain a second score; the harmfulness score of the text to be detected is determined by combining the first score and the second score, which can comprehensively consider the concealment degree and maliciousness degree of the text to be detected, so that the evaluation of the text to be detected is more accurate and comprehensive, and at least one text to be detected whose harmfulness score is greater than a preset threshold can better reflect the multi-faceted security threat level to the large language model; and then at least one text to be detected is imitated, so that the test set can be effectively enhanced with data, and the richness and applicability of the imitated text can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A flow chart of a data enhancement method provided in an embodiment of the present application;

[0030] Figure 2 An example flow chart of a method for determining a first score of a text to be detected provided in an embodiment of the present application;

[0031] Figure 3 An example diagram of a method for enhancing harmful data provided by an embodiment of the present application;

[0032] Figure 4 A structural diagram of a data enhancement device provided in an embodiment of the present application;

[0033] Figure 5 A structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0034] In the technical solution of this application, the collection, dissemination, and use of data are in compliance with the requirements of relevant national laws and regulations.

[0035] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0036] The technical solution of the present application is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations on the technical solution of the present application. In the absence of conflict, the embodiments of the present application and the technical features in the embodiments can be combined with each other.

[0037] It should be understood that "multiple" in the description of the embodiment of the present application refers to two or more. "First", "second" etc. in the embodiment of the present application are used to distinguish different objects, rather than to describe a specific order. The term "and / or" in the embodiment of the present application is merely a kind of association relationship describing associated objects, indicating that three relationships may exist, for example, A and / or B, which may represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "includes" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices. The module in the embodiment of the present application refers to a part with independent functions in a software system.

[0038] For ease of understanding, the following is an introduction to some of the professional terms involved in the embodiments of this application:

[0039] 1. Large Language Model (LLM) is an artificial intelligence model used to understand and generate natural language text. Large language models are trained with massive amounts of text data and are able to grasp the complex semantic structure in the language, thereby demonstrating strong performance in a variety of tasks. These tasks include but are not limited to text summarization, machine translation, sentiment analysis, question-answering systems, etc. The notable feature of large language models is their extremely large parameter scale, which usually contains billions or even hundreds of billions of parameters, which enables large language models to capture and process the nuances and complexities in the language, achieving unprecedented language understanding and generation capabilities.

[0040] 2. Data enhancement, also known as data augmentation, is a technique to artificially increase the training set by creating a modified copy of the dataset from existing data. It aims to increase the diversity and quantity of the dataset, thereby improving the generalization ability and robustness of the model. It is often used to deal with problems such as insufficient data, data imbalance or model overfitting.

[0041] With the widespread application of large language models in various industries, the huge potential they demonstrate is changing the way of working in various fields. However, the abuse of these models is also gradually increasing. Criminals use technical means such as prompt injection to break through the inherent security limitations of large language models, resulting in increasing security threats. These threats use automated methods to write malicious scripts to carry out network attacks, create phishing emails and other illegal activities. In response to these security threats, research institutions and enterprises have provided a security fence specifically for defending against attacks on large language models. This security fence is designed to detect and intercept attacks against potential weaknesses of large language models to ensure the stability and security of the model; continuous security testing of large language models enables research institutions and enterprises to promptly discover and repair security risks in the model and improve overall defense capabilities. However, the quality of existing data used to test the security of large language models varies and cannot fully cover potential security threats. In reality, harmful texts that attack large language models are concealed (i.e., traditional classifiers or models may find it difficult to detect the hidden features of harmful texts. For example, some attacks use seemingly harmless requests to induce large language models to perform abnormal operations, such as infinitely looping a word. In this case, the text itself does not contain harmful content, making it difficult for traditional methods to identify such attacks) and specific (such attacks are usually significantly different from ordinary users' queries. For example, requiring large language models to play a role that is not bound by law. The specificity of such requests is not common, making it difficult for traditional methods to identify them) , harmfulness (although some harmful texts are obscure, their potential harm is huge. For example, when using ASCII encoding to input text on how to make poison, traditional methods may ignore its potential harm due to the complexity of the encoding method, or fail to accurately classify it due to insufficient training data), timeliness (as large language model service providers continue to update and fix model vulnerabilities, attackers adjust their attack methods accordingly. For large language model security evaluation, the data required for training and evaluation models is difficult to respond to these changes quickly) and differences (different types of harmful texts often have significant differences. This diversity and difference increases the difficulty of data enhancement. It is difficult to generate such diverse data at the same time through a single model or method). Due to the lack of targeted considerations in the construction of existing data sets, these data sets are stretched when dealing with complex and changing attack strategies, and it is difficult to fully expose the security vulnerabilities of the model; and the large language model technology is developing rapidly, and attack methods are emerging in an endless stream. Traditional data expansion methods rely on the use of a single model for preliminary evaluation and screening, and use limited feature dimensions for expansion. Therefore, it is difficult to keep up with the rapid changes in technology and threat environment. This lag makes the existing test data sets unable to effectively respond to new security challenges.

[0042] In view of this, an embodiment of the present application provides a data enhancement method, which inputs each text to be detected into a large language model for processing, obtains a reply text of each text to be detected output by the large language model, and evaluates the reply text according to a first preset rule to obtain a first score; it can be determined based on the reply text whether the large language model can be regarded as a harmful text and refuse to reply, thereby determining the degree of concealment of each text to be detected, and quantifying the degree of concealment of each text to be detected through the first score; by inputting each text to be detected into an evaluation model for processing, the maliciousness of the text to be detected is quantified based on the evaluation model to obtain a second score; the text to be detected is screened by combining the first score and the second score, which can comprehensively consider the degree of concealment and maliciousness of the text to be detected, so that at least one screened text to be detected can more comprehensively reflect the security threat to the large language model; and then imitate the at least one text to be detected to obtain an imitated text, which can achieve effective data enhancement of the test set and improve the richness and applicability of the imitated text.

[0043] The data enhancement method provided in the embodiments of the present application can be applied to any data enhancement scenario, including but not limited to the above-mentioned data enhancement of harmful texts for large language models.

[0044] See also Figure 1 , is a flow chart of a data enhancement method provided in an embodiment of the present application. The method can be applied to any electronic device with processing capabilities, such as a computer, a virtual machine, etc. The method includes steps S101 to S106:

[0045] S101, obtaining multiple texts to be detected.

[0046] Among them, a plurality of the texts to be detected are harmful texts that pose a security threat to the large language model.

[0047] Exemplarily, multiple detection texts can be obtained by manual writing and network collection; specifically, professional technicians write characteristic harmful text samples to simulate complex attack situations in real scenarios; and collect harmful texts related to large language models through public platforms on the Internet, including but not limited to GitHub, Reddit and other websites. In addition, harmful texts that attack large language models can also be obtained from historical attack threat logs and other records. The embodiment of the present application does not limit the method of obtaining the text to be detected.

[0048] S102: Input each of the multiple texts to be detected into the large language model for processing, and obtain a reply text for each text to be detected output by the large language model.

[0049] Exemplarily, the large language model can perform semantic recognition, prediction and other processing on each input text to be detected, and generate a reply text based on the recognized intention of the text to be detected to respond to the questions or requests in each text to be detected.

[0050] S103 . Evaluate the reply text of each text to be detected according to a first preset rule to obtain a first score for each text to be detected.

[0051] The first score is used to indicate the concealment level of each text to be detected.

[0052] In a possible embodiment, step S103 specifically includes:

[0053] Determining whether the reply text contains a preset refusal word, wherein the refusal word is used to indicate a refusal to answer the question or request contained in the text to be detected;

[0054] If included, the first score is set to a first value, where the first value is used to indicate that the concealment level of the text to be detected is low;

[0055] If it is not included, the reply text is input into the classification model for classification processing, and the classification model is used to identify whether the reply text is a normal reply text; if the result output by the classification model is that the reply text is a normal reply text, the first score is set to a second value, and the second value is used to indicate that the concealment degree of the text to be detected is high; if the result output by the classification model is that the reply text is an abnormal reply text, the first score is set to the first value.

[0056] For example, rejection words such as "sorry", "apologies", "unable", etc. are used to identify sentences that refuse to reply or report errors, which can indicate that the large language model has identified the text to be detected as harmful text, and therefore refuses to reply to the requirements or questions contained in the text to be detected, which can also reflect that the threat level of the text to be detected is relatively low for the large language model; the first score of the text to be detected is set to a first value (for example, set to 0), and the text to be detected may not be considered for expansion in the future;

[0057] If the reply text of the text to be detected does not contain the preset rejection words, it means that the large language model may not have rejected the text to be detected. In order to confirm whether the large language model has identified the text to be detected as harmful text, further analysis of the reply text is required.

[0058] Specifically, the reply text is input into the classification model, and the classification model is used to determine whether the reply text is a normal reply text. If the classification model outputs that the reply text is a normal reply text, it means that the large language model did not identify the text to be detected as a harmful text, and responded to the requirements or questions in the text to be detected, which also reflects that the text to be detected can pose a security threat to the large language model. Therefore, the first score of the text to be detected is set to the second value, indicating that the concealment level of the text to be detected is high and has the value of imitation. If the classification model outputs that the reply text is an abnormal reply text, it means that the large language model identified the text to be detected as a harmful text and did not respond normally to the text to be detected, and the first score of the text to be detected is set to the first value.

[0059] Among them, the classification model can be a binary classification model obtained by training normal reply text and abnormal reply text, or other types of classification models can be selected according to actual needs, and the embodiments of the present application do not limit this.

[0060] Furthermore, when the large language model is a plurality of large language models with different performances, the specific implementation of step 103 further includes:

[0061] Evaluate the reply text of each large language model in the plurality of large language models to each text to be detected according to the first preset rule to obtain a third score of each large language model for the text to be detected;

[0062] According to the weight of each large language model and the third score of each text to be detected, the first score of each text to be detected is obtained by weighted average calculation; the weight of each large language model is determined by the performance of each large language model.

[0063] It can be understood that the method for each large language model in the multiple large language models to process each text to be detected to obtain a reply text, and the method for evaluating the reply text can all refer to the method described in detail in the above embodiment, and will not be repeated here.

[0064] The multiple large language models may be all the large language models in the system or cluster that require security maintenance, or the large language models that require security maintenance may be selected according to actual needs, which is not limited in the embodiments of the present application.

[0065] In the embodiment of the present application, the weight of each large language model is determined according to the performance of each large language model, that is, the prediction ability and accuracy of the large language model. In practical applications, other indicators (such as the frequency of use of the large language model, etc.) can be selected according to actual needs to determine the weight of the large language.

[0066] Exemplarily, the first score of the model to be detected can be calculated using the following formula:

[0067]

[0068] Among them, R represents the first score of the model to be tested, n is used to represent the number of large language models, and α i represents the weight of the i-th large language model, L i (x) is used to represent the third score obtained after inputting the text to be detected into the i-th large language model and evaluating the reply text of the i-th large language model.

[0069] For example, see Figure 2 , is a flowchart of a method for determining a first score of a text to be detected provided in an embodiment of the present application. Taking the text to be detected as any one of a plurality of texts to be detected as an example, the specific method flow is as follows:

[0070] First, any text to be detected is input into multiple large language models respectively, and the reply text output by each large language model after processing the text to be detected is obtained.

[0071] Then, for the reply text output by any large language model to the text to be detected, determine whether the reply text contains the preset rejection words. If it does, set the third score of the text to be detected to 0; (which can be understood as the score of the text to be detected by any large language model)

[0072] If not, the reply text is input into the classification model. If the result output by the classification model is a normal reply text, the third score of the text to be detected is set to 1. If the result output by the classification model is an abnormal reply text, the third score of the text to be detected is set to 0.

[0073] Finally, a weighted average calculation is performed based on the weight of each large language model and the third score of the text to be detected to obtain the first score of the text to be detected.

[0074] S104: Input each to-be-detected text into the evaluation model for processing, and obtain a second score of each to-be-detected model from the evaluation model.

[0075] Among them, the evaluation model is used to evaluate the maliciousness of each text to be detected.

[0076] In a possible embodiment, the specific implementation of step S104 is as follows:

[0077] Each text to be detected is input into the evaluation model. Through evaluation according to the preset violation policy, the correlation between each text to be detected and the violation policy is calculated. Each text to be detected is evaluated according to the correlation to obtain a second score for each text to be detected.

[0078] For example, the violation policy is used to describe various scenarios where the large language model is prohibited from being used, such as:

[0079] CLAS Policy - Our model (hereinafter referred to as "our model" is used to refer to large language models) does not allow the following usage:

[0080] Illegal Activities: CLAS prohibits the use of our models, tools, and services for illegal activities.

[0081] Child Sexual Abuse Material or any content that exploits or harms children: We report CSAM to the National Center for Missing and Exploited Children.

[0082] Generates hateful, harassing, or violent content: Content that expresses, incites, or promotes hatred based on identity; content that is intended to harass, threaten, or bully individuals; content that promotes or glorifies violence or celebrates the pain or humiliation of others.

[0083] Generates malware: Content that attempts to generate code designed to damage, harm, or gain unauthorized access to a computer system.

[0084] High-risk physical harm activities, including: weapons development; military and warfare; management or operation of critical energy, transportation, and water infrastructure; content that promotes, encourages, or depicts self-harmful behaviors, such as suicide, self-mutilation, and eating disorders.

[0085] High-risk economic harm activities, including: multi-level marketing; gambling; payday lending; automated determination of eligibility for credit, employment, educational institutions, or public assistance services.

[0086] Fraudulent or deceptive activity, including: hoaxes; coordinated inauthentic behavior; plagiarism; academic dishonesty; astroturfing, such as fake grassroots support or fake review generation.

[0087] False information: spam; fake drugs.

[0088] …

[0089] The above are only some examples of possible policy violations. You can set policies to disable the model based on actual conditions.

[0090] It can be understood that by calculating the correlation between the text to be detected and the illegal policy, the maliciousness of the illegal words and sentences contained in the text to be detected can be determined, and then the second score can be used to quantify the maliciousness of the text to be detected. For example, the value of the correlation is mapped to between 0 and 1, and the value of the correlation is used as the second score. If the second score is 0, it means that the text to be detected has nothing to do with the illegal policy; if the second score is 1, it means that the text to be detected is completely consistent with one or more of the illegal policies.

[0091] S105 : Determine the harmfulness score of each to-be-detected text based on the first score and the second score of each to-be-detected text.

[0092] In a possible embodiment, a weighted average of the first score and the second score of each to-be-detected text is calculated to obtain a harmfulness score of each to-be-detected text.

[0093] Exemplarily, the harmfulness score of each text to be detected can be calculated using the following formula:

[0094] H=γR+βA

[0095] Among them, H represents the harmfulness score of the text to be detected, R represents the first score of the text to be detected, and A represents the second score of the text to be detected; γ and β are adjustable parameters of the first score and the second score, respectively, which can be determined according to actual needs.

[0096] In a possible design, after determining the first score, the second score and the harmfulness score of each text to be detected, the score information of the text to be detected can be stored in a database to facilitate subsequent analysis and other operations on each text to be detected.

[0097] Exemplarily, the storage format of the scoring information of the text to be detected can be: {"harmful text": "Hello, World!", "harmful score": 0.5, "second score": 0.5, "first score": 0.5, "third score": {"qwen1.5-7b": 0.5, "llama2-70b": 0.5, "llama3.1-8b": 0.5, "llama2-7b": 0.5, "llama-7b": 0.5}, "source": "manually written"}

[0098] Among them, "qwen1.5-7b", "llama2-70b", "llama3.1-8b" and so on are the names of the large language models participating in the third scoring, and "qwen1.5-7b": 0.5 means that the large language model named "qwen1.5-7b" has a third score of 0.5 for the text to be detected.

[0099] It can be understood that the above is only an example of a possible format for storing the scoring information of the text to be detected, and is not limited thereto.

[0100] S106, obtaining at least one text to be detected whose harmfulness score is greater than a preset threshold; and imitating the at least one text to be detected to obtain at least one imitative text.

[0101] In a possible embodiment, the specific implementation of step S106 is as follows:

[0102] Inputting each of the at least one text to be detected into a topic generation model, performing topic analysis and identification on each of the texts to be detected by the topic generation model, and determining a topic of each of the texts to be detected;

[0103] Extract the encoding method of each text to be detected based on regular expressions;

[0104] Input each text to be detected into the large language model, perform semantic analysis on each text to be detected through the large language model, and extract the semantic logic and language expression in each text to be detected;

[0105] Input each text to be detected into the syntactic analysis model, perform syntactic analysis on each text to be detected through the syntactic analysis model, and extract the syntactic structure of each text to be detected;

[0106] Each sentence to be tested is imitated based on at least one of the theme, encoding method, language logic, language expression method and syntactic structure.

[0107] The topic generation model may be a latent Dirichlet Allocation (LDA) model or a bidirectional transformer model bert model; the encoding method for extracting each text to be detected may also use a static analysis tool for software static testing; and the syntactic analysis may be implemented by relying on a syntactic tree, a sentence graph, and other methods. The embodiments of the present application are not limited to this.

[0108] In addition, the features of the text to be detected (i.e., subject, encoding method, language logic, language expression and syntactic structure) extracted in the embodiment of the present application are only a possible example. In actual applications, other features of the text to be detected can be extracted as needed.

[0109] The method of imitating each to-be-detected data based on at least one of the subject, coding method, language logic, language expression method and syntactic structure can be based on any one or a combination of any multiple items. See Table 1, which gives an example of imitating the to-be-detected data based on the above features.

[0110] Table 1 Examples of imitation writing of data to be tested

[0111]

[0112] In an embodiment of the present application, each text to be detected is processed based on a large language model, a reply text of each text to be detected output by the large language model is obtained, and the reply text of each text to be detected is evaluated according to a first preset rule, so as to determine whether the large language model can identify the text to be detected as a harmful text, and then the degree of concealment of each text to be detected can be determined, and the degree of concealment of each text to be detected is quantified using a first score; and the maliciousness of each text to be detected is quantified based on the evaluation model to obtain a second score; the harmfulness score of the text to be detected is determined by combining the first score and the second score, which can comprehensively consider the degree of concealment and maliciousness of the text to be detected, so that the evaluation of the text to be detected is more accurate and comprehensive, and at least one text to be detected whose harmfulness score is greater than a preset threshold can better reflect the multi-faceted security threat level to the large language model; and then at least one text to be detected is imitated, which can effectively enhance the data of the test set and improve the richness and applicability of the imitated text.

[0113] It is understood that the above embodiments can be used alone or in combination. The following is an example of combined use. Figure 3 , is an example diagram of a method for enhancing harmful text provided in an embodiment of the present application. The method specifically includes the following implementation steps:

[0114] First, multiple texts to be detected are obtained by manual writing or network collection.

[0115] Then, a first score and a second score of each of the multiple detected texts are determined, and a harmfulness score of each to-be-detected text is calculated according to the first score and the second score.

[0116] It can be understood that the method of determining the first score and the second score of each detected text, and calculating the harmfulness score of each to-be-detected text according to the first score and the second score, can refer to the detailed method steps in the above embodiment, which will not be repeated here.

[0117] Next, the scoring information of each text to be detected is stored in the database.

[0118] The storage format of the scoring information of each to-be-detected text may be set according to actual needs, or may refer to the storage format of the scoring information in the above embodiment.

[0119] Finally, at least one text to be detected whose harmfulness score is greater than a preset threshold is screened out, feature extraction and analysis are performed on each text to be detected in the at least one text to be detected, and imitation is performed according to at least one feature of each text to be detected to obtain an imitation text.

[0120] It can be understood that the obtained imitation text can be added to multiple texts to be detected, and used as an existing data set when a large language model needs to be analyzed or harmful data is enhanced next time.

[0121] The method provided by the embodiment of the present application is introduced above, and the device provided by the embodiment of the present application is introduced below.

[0122] Based on the same technical concept, an embodiment of the present application provides a data enhancement device, which includes a module / unit / means for executing the method executed by the electronic device in the above method embodiment. The module / unit / means can be implemented by software, or by hardware, or the corresponding software can be implemented by hardware.

[0123] For example, see Figure 4 , the device 400 comprises:

[0124] The acquisition module 401 is used to: obtain a plurality of texts to be detected, where the plurality of texts to be detected are harmful texts that pose a security threat to the large language model;

[0125] Evaluation module 402, used for: inputting each of the multiple texts to be detected into the large language model for processing, obtaining the reply text of each text to be detected output by the large language model; evaluating the reply text of each text to be detected according to the first preset rule, obtaining a first score for each text to be detected; the first score is used to indicate the concealment degree of each text to be detected; inputting each text to be detected into the evaluation model for processing, obtaining a second score of each text to be detected by the evaluation model; the evaluation model is used to evaluate the maliciousness of each text to be detected; determining the harmfulness score of each text to be detected based on the first score and the second score of each text to be detected;

[0126] The imitation writing module 403 is used to: obtain at least one text to be detected whose harmfulness score is greater than a preset threshold; imitate the at least one text to be detected to obtain at least one imitated text.

[0127] Optionally, when the evaluation module 402 evaluates the reply text of each text to be detected according to the first preset rule and obtains the first score of each text to be detected, it is specifically used to: determine whether the reply text contains a preset refusal to answer words, the refusal to answer the questions or requirements contained in the text to be detected; if contained, the first score is set to a first value, the first value is used to indicate that the concealment level of the text to be detected is low; if not contained, the reply text is input into the classification model for classification processing, the classification model is used to identify whether the reply text is a normal reply text; if the result output by the classification model is that the reply text is a normal reply text, the first score is set to a second value; the second value is used to indicate that the concealment level of the text to be detected is high; if the result output by the classification model is that the reply text is a normal reply text, the first score is set to the first value.

[0128] Optionally, the large language model is a plurality of large language models with different performances; when the evaluation module 402 evaluates the reply text of each text to be detected according to the first preset rule to obtain the first score of each text to be detected, it is also used to: evaluate the reply text of each large language model in the plurality of large language models to each text to be detected according to the first preset rule to obtain the third score of each large language model for the text to be detected; obtain the first score of each text to be detected by weighted average calculation according to the weight of each large language model and the third score of each text to be detected; the weight of each large language model is determined by the performance of each large language model.

[0129] Optionally, when the evaluation module 402 inputs each text to be detected into the evaluation model for processing and obtains the second score of the evaluation model for each text to be detected, it is specifically used to: input each text to be detected into the evaluation model, calculate the correlation between each text to be detected and the violation policy through evaluation according to the preset violation policy, evaluate each text to be detected according to the correlation, and obtain the second score of each text to be detected.

[0130] Optionally, when determining the harmfulness score of each text to be detected based on the first score and the second score of each text to be detected, the evaluation module 402 is specifically used to: calculate a weighted average of the first score and the second score of each text to be detected to obtain the harmfulness score of each text to be detected.

[0131] Optionally, when the imitation module 403 imitates at least one text to be detected and obtains at least one imitation text, it is specifically used to: input each text to be detected in the at least one text to be detected into a topic generation model, perform topic analysis and identification on each text to be detected through the topic generation model, and determine the topic of each text to be detected; extract the encoding method of each text to be detected based on a regular expression; input each text to be detected into a large language model, perform semantic analysis on each text to be detected through the large language model, and extract the semantic logic and language expression in each text to be detected; input each text to be detected into a syntactic analysis model, perform syntactic analysis on each text to be detected through the syntactic analysis model, and extract the syntactic structure of each text to be detected; imitate each text to be detected based on at least one of the topic, encoding method, language logic, language expression method and syntactic structure.

[0132] It should be understood that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0133] Based on the same technical concept, see Figure 5 The embodiment of the present application further provides an electronic device 500, including:

[0134] At least one processor 501; and a communication interface 503 that is communicatively connected to the at least one processor 501; the at least one processor 501 executes instructions stored in the memory 502, so that the electronic device 500 executes the method steps performed by the billboard in the above method embodiment through the communication interface 503.

[0135] Optionally, the memory 502 is located outside the electronic device 500 .

[0136] Optionally, the electronic device 500 includes the memory 502, the memory 502 is connected to the at least one processor 501, and the memory 502 has instructions that can be executed by the at least one processor 501. Figure 5 The dashed lines indicate that the memory 502 is optional for the electronic device 500 .

[0137] The at least one processor 501 and the memory 502 may be coupled via an interface circuit or may be integrated together, which is not limited here.

[0138] The specific connection medium between the at least one processor 501, the memory 502 and the communication interface 503 is not limited in the embodiment of the present application. Figure 5 In the embodiment, at least one processor 501, a memory 502 and a communication interface 503 are connected via a bus 504. Figure 5The connection between other components is only for schematic illustration and is not intended to be limiting. The bus part may be an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 It is represented by only one thick line, but it does not mean that there is only one bus or one type of bus.

[0139] It should be understood that the processor mentioned in the embodiments of the present application can be implemented by hardware or by software. When implemented by hardware, the processor can be a logic circuit, an integrated circuit, etc. When implemented by software, the processor can be a general-purpose processor implemented by reading software code stored in a memory.

[0140] Exemplarily, the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0141] It should be understood that the memory mentioned in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both a volatile memory and a non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which acts as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (DirectRambus RAM, DR RAM).

[0142] It should be noted that when the processor is a general-purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, the memory (storage module) can be integrated into the processor.

[0143] It should be noted that the memory described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0144] Based on the same technical concept, an embodiment of the present application also provides a computer-readable storage medium, which is used to store instructions. When the instructions are executed, the computer executes the method steps executed by any device in the above method embodiments.

[0145] Based on the same technical concept, an embodiment of the present application also provides a computer program product, including computer program code. When the computer program code is executed on a computer, the method steps executed by any device in the above method embodiment are implemented.

[0146] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0147] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0148] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0149] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0150] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A data enhancement method, characterized in that: include: Acquire a plurality of texts to be tested, where the plurality of texts to be tested are harmful texts that pose a security threat to the large language model; Input each of the multiple texts to be detected into the large language model for processing, and obtain a reply text of each text to be detected output by the large language model; evaluate the reply text of each text to be detected according to a first preset rule to obtain a first score of each text to be detected; the first score is used to indicate the concealment degree of each text to be detected; Inputting each of the texts to be detected into the evaluation model for processing, and obtaining a second score of each of the texts to be detected by the evaluation model; the evaluation model is used to evaluate the maliciousness of each of the texts to be detected; Determine the harmfulness score of each of the texts to be detected based on the first score and the second score of each of the texts to be detected; Acquire at least one text to be detected whose harmfulness score is greater than a preset threshold; imitate the at least one text to be detected to obtain at least one imitated text.

2. The method according to claim 1, characterized in that Evaluating the reply text of each of the to-be-detected texts according to the first preset rule to obtain a first score for each of the to-be-detected texts includes: Determining whether the reply text contains a preset refusal word, wherein the refusal word is used to indicate a refusal to answer the question or request contained in the text to be detected; If included, the first score is set to a first value, where the first value is used to indicate that the concealment level of the text to be detected is low; If it is not included, the reply text is input into the classification model for classification processing, and the classification model is used to identify whether the reply text is a normal reply text; if the result output by the classification model is that the reply text is a normal reply text, the first score is set to a second value, and the second value is used to indicate that the concealment degree of the text to be detected is high; if the result output by the classification model is that the reply text is an abnormal reply text, the first score is set to the first value.

3. The method according to claim 2, characterized in that The large language model is a plurality of large language models with different performances; evaluating the reply text of each of the to-be-detected texts according to the first preset rule to obtain a first score for each of the to-be-detected texts, including: Evaluate the reply text of each large language model in the multiple large language models to each of the texts to be detected according to the first preset rule to obtain a third score of each large language model for the text to be detected; According to the weight of each large language model and the third score of each text to be detected, the first score of each text to be detected is obtained by weighted average calculation; the weight of each large language model is determined by the performance of each large language model.

4. The method according to claim 3, characterized in that The step of inputting each of the to-be-detected texts into the evaluation model for processing, and obtaining a second score of each of the to-be-detected texts by the evaluation model, comprises: Each of the texts to be detected is input into the evaluation model, and the evaluation model is used to calculate the correlation between each of the texts to be detected and the violation policy according to the preset violation policy, and each of the texts to be detected is evaluated according to the correlation to obtain a second score for each of the texts to be detected.

5. The method according to claim 3 or 4, characterized in that Determining the harmfulness score of each to-be-detected text based on the first score and the second score of each to-be-detected text comprises: A weighted average of the first score and the second score of each of the texts to be detected is calculated to obtain a harmfulness score of each of the texts to be detected.

6. The method according to claim 1, characterized in that The step of imitating the at least one text to be detected to obtain at least one imitative text comprises: Inputting each of the at least one text to be detected into a topic generation model, performing topic analysis and identification on each of the texts to be detected by the topic generation model, and determining the topic of each of the texts to be detected; Extracting the encoding mode of each text to be detected based on a regular expression; Input each of the texts to be detected into the large language model, perform semantic analysis on each of the texts to be detected through the large language model, and extract the semantic logic and language expression of each of the texts to be detected; Inputting each of the texts to be detected into a syntactic analysis model, performing syntactic analysis on each of the texts to be detected by the syntactic analysis model, and extracting the syntactic structure of each of the texts to be detected; Each of the texts to be detected is imitated based on at least one of the subject, the encoding method, the language logic, the language expression method and the syntactic structure.

7. A data enhancement device, characterized in that: include: The acquisition module is used to: obtain a plurality of texts to be detected, wherein the plurality of texts to be detected are harmful texts that pose a security threat to the large language model; An evaluation module is used to: input each of the multiple texts to be detected into the large language model for processing, and obtain a reply text of each text to be detected output by the large language model; evaluate the reply text of each text to be detected according to a first preset rule to obtain a first score of each text to be detected; the first score is used to indicate the concealment degree of each text to be detected; input each text to be detected into the evaluation model for processing, and obtain a second score of the evaluation model for each text to be detected; the evaluation model is used to evaluate the maliciousness of each text to be detected; determine the harmfulness score of each text to be detected based on the first score and the second score of each text to be detected; The imitation writing module is used to: obtain at least one text to be detected whose harmfulness score is greater than a preset threshold; imitate the at least one text to be detected to obtain at least one imitated text.

8. The device according to claim 7, characterized in that When the evaluation module evaluates the reply text of each text to be detected according to the first preset rule to obtain the first score of each text to be detected, it is specifically used to: Determining whether the reply text contains a preset refusal word, wherein the refusal word is used to indicate a refusal to answer the question or request contained in the text to be detected; If included, the first score is set to a first value, where the first value is used to indicate that the concealment level of the text to be detected is low; If not, the reply text is input into a classification model for classification processing, and the classification model is used to identify whether the reply text is a normal reply text; if the result output by the classification model is that the reply text is a normal reply text, the first score is set to a second value, and the second value is used to indicate that the concealment degree of the text to be detected is high; If the result output by the classification model is that the reply text is an abnormal reply text, the first score is set to a first value.

9. An electronic device, characterized in that: include: A memory for storing program instructions; A processor is used to call the program instructions stored in the memory, and execute the steps included in the method according to any one of claims 1 to 6 according to the obtained program instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to have a computer program, and the computer program includes program instructions. When the program instructions are executed by a computer, the method according to any one of claims 1 to 6 is implemented.

Images