Data augmentation method and apparatus

Abstract

The present application provides a data augmentation method and apparatus, applied to the technical field of network security, and used for effective data augmentation of a test set, thereby improving the richness and applicability of paraphrased texts. The method comprises: acquiring a plurality of texts to be tested; inputting each text into a large language model for processing, and acquiring a response text of each text outputted by the large language model; according to a first preset rule, evaluating the response text of each text to obtain a first score of each text; inputting each text into an evaluation model for processing, and acquiring a second score of each text from the evaluation model; on the basis of the first score and the second score of each text, determining a harmfulness score of each text; acquiring at least one text having a harmfulness score greater than a preset threshold; and paraphrasing the at least one text to be tested to obtain at least one paraphrased text.

Description

A data augmentation method and apparatus

[0001] Cross-references to related applications

[0002] This application claims priority to Chinese Patent Application No. 202411516698.2, filed on October 29, 2024, entitled "A Data Augmentation Method and Apparatus", the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of cybersecurity technology, and in particular to a data augmentation method and apparatus. Background Technology

[0004] In the current information age, large language models are increasingly threatened by various security attacks. However, the test sets used to test and evaluate large language models are few and generally of low quality, which cannot fully cover potential security threats. In addition, since harmful texts that attack large language models often have characteristics such as concealment, special features and differences, data augmentation by simply screening existing test sets using a single model is ineffective and cannot effectively address the ever-changing security challenges.

[0005] Therefore, how to effectively augment the test set to improve the richness and applicability of the imitation text is an urgent problem to be solved. Summary of the Invention

[0006] This application provides a data augmentation method and apparatus for effectively augmenting a test set, thereby improving the richness and applicability of the imitated text.

[0007] In a first aspect, embodiments of this application provide a data augmentation method applicable to any electronic device with processing capabilities, the method comprising:

[0008] Multiple texts to be detected are obtained, and these texts are harmful texts that pose a security threat to the large language model.

[0009] Each of the multiple texts to be detected is input into a large language model for processing, and the response text of each text to be detected is obtained from the output of the large language model; the response text of each text to be detected is evaluated according to the first preset rule to obtain the first score of each text to be detected; the first score is used to indicate the degree of concealment of each text to be detected.

[0010] Each text to be detected is input into the evaluation model for processing, and a second score is obtained from the evaluation model for each text to be detected; the evaluation model is used to assess the degree of malice of each text to be detected.

[0011] The harmfulness score of each text to be detected is determined based on the first and second scores of each text to be detected;

[0012] Obtain at least one text to be detected with a harmful score greater than a preset threshold; perform a parody on at least one text to be detected to obtain at least one parody text.

[0013] Optionally, the response text for each text to be detected is evaluated according to a first preset rule to obtain a first score for each text to be detected, including: determining whether the response text contains a preset rejection word, which is used to indicate that the question or request contained in the text to be detected is rejected; if it contains a rejection word, the first score is set to a first value, which is used to indicate that the concealment level of the text to be detected is low; if it does not contain a rejection word, the response text is input into a classification model for classification processing, which is used to identify whether the response text is a normal response text; if the output result of the classification model is that the response text is a normal response text, the first score is set to a second value; the second value is used to indicate that the concealment level of the text to be detected is high; if the output result of the classification model is that the response text is an abnormal response text, the first score is set to the first value.

[0014] Optionally, the large language model can be multiple large language models with different performance levels; the response text of each text to be detected is evaluated according to the first preset rule to obtain a first score for each text to be detected, including: evaluating the response text of each of the multiple large language models to each text to be detected according to the first preset rule to obtain a third score for each large language model to be detected; and calculating the first score for each text to be detected by weighted average based on the weight of each large language model and the third score for each text to be detected; the weight of each large language model is determined by the performance of each large language model.

[0015] Optionally, each text to be detected is input into the evaluation model for processing to obtain a second score from the evaluation model for each text to be detected. This includes: inputting each text to be detected into the evaluation model, calculating the correlation between each text to be detected and the violation policy according to the preset violation policy, evaluating each text to be detected based on the correlation, and obtaining a second score for each text to be detected.

[0016] Optionally, a harmful score for each text to be detected is determined based on a first score and a second score, including: calculating a weighted average of the first score and the second score for each text to be detected to obtain a harmful score for each text to be detected.

[0017] Optionally, at least one text to be detected is imitated to obtain at least one imitated text, including: inputting each text to be detected into a topic generation model, performing topic analysis and recognition on each text to be detected through the topic generation model to determine the topic of each text to be detected; extracting the encoding method of each text to be detected based on regular expressions; inputting each text to be detected into a large language model, performing semantic analysis on each text to be detected through the large language model to extract the semantic logic and language expressions in each text to be detected; inputting each text to be detected into a syntactic analysis model, performing syntactic analysis on each text to be detected through the syntactic analysis model to extract the syntactic structure of each text to be detected; and imitating each text to be detected based on at least one of topic, encoding method, language logic, language expression method, and syntactic structure.

[0018] Secondly, embodiments of this application provide a data enhancement apparatus, including:

[0019] The acquisition module is used to: acquire multiple texts to be detected, which are harmful texts that pose a security threat to the large language model;

[0020] The evaluation module is used for: inputting each of the multiple texts to be detected into a large language model for processing, and obtaining the response text of each text to be detected output by the large language model; evaluating the response text of each text to be detected according to a first preset rule to obtain a first score for each text to be detected; the first score is used to indicate the degree of concealment of each text to be detected; inputting each text to be detected into an evaluation model for processing, and obtaining a second score of each text to be detected from the evaluation model; the evaluation model is used to evaluate the degree of malice of each text to be detected; and determining the harmfulness score of each text to be detected based on the first score and the second score.

[0021] The imitation module is used to: obtain at least one text to be detected with a harmful score greater than a preset threshold; and imitate at least one text to be detected to obtain at least one imitation text.

[0022] Optionally, when the evaluation module evaluates the response text of each text to be detected according to the first preset rule and obtains a first score for each text to be detected, it specifically determines whether the response text contains a preset rejection word, which is used to indicate that the question or request contained in the text to be detected is rejected; if it contains such a word, the first score is set to a first value, which is used to indicate that the concealment level of the text to be detected is low; if it does not contain such a word, the response text is input into a classification model for classification processing, which is used to identify whether the response text is a normal response text; if the classification model outputs that the response text is a normal response text, the first score is set to a second value, which is used to indicate that the concealment level of the text to be detected is high; if the classification model outputs that the response text is an abnormal response text, the first score is set to the first value.

[0023] Optionally, the large language model can be multiple large language models with different performance levels. When the evaluation module evaluates the response text of each text to be detected according to the first preset rule to obtain the first score of each text to be detected, it is also used to: evaluate the response text of each of the multiple large language models to each text to be detected according to the first preset rule to obtain the third score of each large language model to the text to be detected; calculate the first score of each text to be detected by weighted average based on the weight of each large language model and the third score of each text to be detected; the weight of each large language model is determined by the performance of each large language model.

[0024] Optionally, when the evaluation module inputs each text to be detected into the evaluation model for processing and obtains the second score of each text to be detected by the evaluation model, it is specifically used to: input each text to be detected into the evaluation model, calculate the correlation between each text to be detected and the violation policy according to the preset violation policy through evaluation, evaluate each text to be detected based on the correlation, and obtain the second score of each text to be detected.

[0025] Optionally, when determining the harmfulness score of each text to be detected based on the first score and the second score of each text to be detected, the evaluation module is specifically used to: calculate the weighted average of the first score and the second score of each text to be detected to obtain the harmfulness score of each text to be detected.

[0026] Optionally, when the imitation module imitates at least one text to be detected to obtain at least one imitated text, it specifically performs the following: inputting each text to be detected into a topic generation model, performing topic analysis and recognition on each text to determine the topic of each text; extracting the encoding method of each text to be detected based on regular expressions; inputting each text to be detected into a large language model, performing semantic analysis on each text to be detected through the large language model, and extracting the semantic logic and language expressions in each text to be detected; inputting each text to be detected into a syntactic analysis model, performing syntactic analysis on each text to be detected through the syntactic analysis model, and extracting the syntactic structure of each text to be detected; and imitating each text to be detected based on at least one of the following: topic, encoding method, language logic, language expression method, and syntactic structure.

[0027] Thirdly, embodiments of this application provide an electronic device including at least one processor, which, when executing a computer program stored in a memory, causes the method as described in the first aspect or any optional implementation of the first aspect to be implemented.

[0028] Fourthly, embodiments of this application provide a computer-readable storage medium for storing instructions that, when executed, cause the method as described in the first aspect or any optional implementation of the first aspect to be implemented.

[0029] Fifthly, embodiments of this application provide a computer program product, including computer program code, which, when executed on a computer, causes the method as described in the first aspect or any optional implementation of the first aspect to be implemented.

[0030] In this embodiment, each text to be detected is processed based on a large language model to obtain the response text output by the large language model for each text to be detected. The response text for each text to be detected is evaluated according to a first preset rule to determine whether the large language model can identify the text to be detected as harmful text, and thus determine the degree of concealment of each text to be detected. A first score is used to quantify the degree of concealment of each text to be detected. A second score is obtained by quantifying the degree of malice of each text to be detected based on an evaluation model. The harmful score of the text to be detected is determined by combining the first score and the second score. This can comprehensively consider the degree of concealment and malice of the text to be detected, making the evaluation of the text to be detected more accurate and comprehensive. At least one text to be detected with a harmful score greater than a preset threshold can better reflect the degree of security threat to the large language model in many aspects. Furthermore, at least one text to be detected is paraphrased, which can effectively augment the test set and improve the richness and applicability of the paraphrased text. Attached Figure Description

[0031] Figure 1 is a flowchart of a data augmentation method provided in an embodiment of this application;

[0032] Figure 2 is a flowchart illustrating a method for determining a first score of a text to be detected according to an embodiment of this application.

[0033] Figure 3 is an example diagram of a method for harmful data augmentation provided in an embodiment of this application;

[0034] Figure 4 is a structural diagram of a data enhancement device provided in an embodiment of this application;

[0035] Figure 5 is a structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0036] The data collection, dissemination, and use in this application all comply with relevant national laws and regulations.

[0037] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.

[0038] The technical solution of this application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of this application and the specific features in the embodiments are detailed descriptions of the technical solution of this application, rather than limitations on the technical solution of this application. In the absence of conflict, the embodiments of this application and the technical features in the embodiments can be combined with each other.

[0039] It should be understood that "multiple" in the description of the embodiments of this application refers to two or more. "First," "second," etc., in the embodiments of this application are used to distinguish different objects, not to describe a specific order. The term "and / or" in the embodiments of this application is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, the term "comprising" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices. In the embodiments of this application, a module refers to a part of a software system that has independent functionality.

[0040] For ease of understanding, the following is a description of some of the technical terms used in the embodiments of this application:

[0041] 1. A Large Language Model (LLM) is an artificial intelligence model used to understand and generate natural language text. Trained on massive amounts of text data, LLMs can master the complex semantic structures of language, thus exhibiting powerful performance in various tasks. These tasks include, but are not limited to, text summarization, machine translation, sentiment analysis, and question answering systems. A significant characteristic of LLMs is their extremely large parameter scale, typically containing billions or even hundreds of billions of parameters. This allows LLMs to capture and process the subtle nuances and complexities of language, achieving unprecedented language understanding and generation capabilities.

[0042] 2. Data augmentation, also known as data enhancement, is a technique that artificially increases the training set by creating modified copies of the existing dataset. It aims to increase the diversity and quantity of the dataset, thereby improving the model's generalization ability and robustness. It is often used to address problems such as insufficient data, imbalanced data, or model overfitting.

[0043] With the widespread application of large language models across various industries, their immense potential is transforming the way we work. However, the abuse of these models is also gradually increasing. Criminals are using techniques such as prompt injection to bypass the inherent security limitations of large language models, leading to escalating security threats. These threats involve automated malicious scripting for cyberattacks, phishing emails, and other illegal activities. To address these security threats, research institutions and enterprises have developed a security guard specifically designed to defend against attacks targeting large language models. This guard aims to detect and block attacks that exploit potential weaknesses in large language models, ensuring their stability and security. Continuous security testing of large language models allows research institutions and enterprises to promptly identify and fix security vulnerabilities, improving overall defense capabilities. However, the quality of existing data used to test the security of large language models varies, failing to comprehensively cover potential security threats. In reality, malicious texts attacking large language models exhibit characteristics such as concealment (i.e., traditional classifiers or models may struggle to detect the hidden features of malicious text. For example, some attacks induce large language models to perform abnormal operations through seemingly harmless requests, such as infinitely looping a word. In such cases, the text itself does not contain harmful content, making it difficult for traditional methods to identify these attacks) and specificity (these attacks typically differ significantly from ordinary user queries; for example, requesting the large language model to play a role unbound by law. The specificity of such requests is uncommon, making them difficult for traditional methods to identify). The data augmentation process suffers from several key characteristics: (1) Harmfulness (some harmful texts, though subtle, pose significant potential dangers; for example, texts using ASCII encoding to describe how to make poison might be overlooked by traditional methods due to the complexity of the encoding, or fail to be accurately classified due to insufficient training data); (2) Timeliness (as large language model service providers continuously update and patch model vulnerabilities, attackers adjust their attack methods accordingly, making it difficult for the data needed to train and evaluate large language models to quickly respond to these changes); and (3) Diversity (different types of harmful texts often exhibit significant differences, increasing the difficulty of data augmentation; it is difficult to generate such diverse data simultaneously using a single model or method). Because existing datasets lack targeted considerations during construction, they are ill-equipped to handle complex and ever-changing attack strategies, failing to fully expose model vulnerabilities. Furthermore, the rapid development of large language model technology and the proliferation of attack methods mean that traditional data augmentation methods, relying on initial evaluation and screening using a single model and expanding with limited feature dimensions, struggle to keep pace with the rapid changes in technology and the threat environment. This lag makes existing test datasets ineffective in addressing new security challenges.

[0044] In view of this, embodiments of this application provide a data augmentation method. This method involves inputting each text to be detected into a large language model for processing, obtaining the response text output by the large language model for each text to be detected, evaluating the response text according to a first preset rule to obtain a first score, determining whether the large language model can identify the text to be detected as harmful and reject the response based on the response text, thereby determining the degree of concealment of each text to be detected, and quantifying the degree of concealment of each text to be detected through the first score; inputting each text to be detected into an evaluation model for processing, quantifying the degree of malice of the text to be detected based on the evaluation model to obtain a second score; combining the first and second scores to filter texts to be detected, comprehensively considering both the degree of concealment and the degree of malice of the texts to be detected, so that at least one text to be detected selected can more comprehensively reflect the security threat to the large language model; and then, based on at least one text to be detected, imitation texts are obtained, which can effectively augment the test set and improve the richness and applicability of the imitation texts.

[0045] The data augmentation method provided in this application can be applied to any data augmentation scenario, including but not limited to the data augmentation of harmful text in large language models described above.

[0046] Referring to Figure 1, which is a flowchart of a data augmentation method provided in an embodiment of this application, the method can be applied to any electronic device with processing capabilities, such as a computer, virtual machine, etc.; the method includes steps S101 to S106:

[0047] S101. Obtain multiple texts to be detected.

[0048] Among them, several of the texts to be detected were harmful texts that posed a security threat to large language models.

[0049] For example, multiple detection texts can be obtained through both manual writing and online collection. Specifically, malicious text samples with distinctive features can be written by professional technicians to simulate complex attack scenarios in real-world situations. Harmful texts related to the large language model can also be collected through publicly available platforms on the internet, including but not limited to websites such as GitHub and Reddit. Furthermore, harmful texts attacking the large language model can also be obtained from historical attack threat logs and other records. This application does not limit the method of obtaining the text to be detected.

[0050] S102. Input each of the multiple texts to be detected into the large language model for processing, and obtain the response text of each text to be detected output by the large language model.

[0051] For example, a large language model can perform semantic recognition, prediction, and other processing on each input text to be detected, and generate response text based on the identified intent of the text to be detected, which is used to respond to the questions or requests in each text to be detected.

[0052] S103. Evaluate the response text of each text to be detected according to the first preset rule to obtain the first score of each text to be detected.

[0053] The first score is used to indicate the degree of concealment of each text to be detected.

[0054] In one possible embodiment, step S103 specifically includes:

[0055] Determine whether the response text contains a preset rejection word, the rejection word being used to indicate that the question or request contained in the text to be detected is refused;

[0056] If included, the first score is set to a first value, which indicates that the concealment level of the text to be detected is low;

[0057] If not included, the response text is input into a classification model for classification processing. The classification model is used to identify whether the response text is a normal response text. If the classification model outputs that the response text is a normal response text, the first score is set to the second value, which is used to indicate that the concealment level of the text to be detected is high. If the classification model outputs that the response text is an abnormal response text, the first score is set to the first value.

[0058] For example, rejection words such as "sorry," "excuse me," and "unable" are used to identify statements that refuse to respond or report errors. This can indicate that the large language model has identified the text to be detected as harmful text, and therefore refuses to respond to the requests or questions contained in the text. It can also reflect that the text to be detected poses a low threat to the large language model. The first score of the text to be detected is set to the first value (e.g., set to 0), and subsequent expansion of the text to be detected can be disregarded.

[0059] If the response text to the text to be detected does not contain the preset rejection words, it means that the large language model may not have rejected the text to be detected. In order to confirm whether the large language model has identified the text to be detected as harmful text, further analysis of the response text is required.

[0060] Specifically, the response text is input into the classification model. Based on the classification model, it is determined whether the response text is a normal response. If the classification model outputs that the response text is a normal response, it means that the large language model did not identify the text to be detected as harmful text and responded to the requirements or questions in the text. This also reflects that the text to be detected poses a security threat to the large language model. Therefore, the first score of the text to be detected is set to the second value, indicating that the text to be detected has a high degree of concealment and has the value of being imitated. If the classification model outputs that the response text is an abnormal response, it means that the large language model identified the text to be detected as harmful text and did not respond normally to it. In this case, the first score of the text to be detected is set to the first value.

[0061] The classification model can be a binary classification model trained with normal and abnormal response texts, or other types of classification models can be selected according to actual needs. This application embodiment does not limit this.

[0062] Furthermore, when the large language model consists of multiple large language models with different performance levels, the specific implementation of step 103 also includes:

[0063] According to the first preset rule, the response text of each of the multiple large language models to each text to be detected is evaluated, and the third score of each large language model to the text to be detected is obtained.

[0064] The first score of each text to be detected is calculated by weighted average based on the weight of each large language model and the third score of each text to be detected; the weight of each large language model is determined by the performance of each large language model.

[0065] It is understood that the methods for processing each text to be detected by each of the multiple large language models to obtain the response text, and the methods for evaluating the response text, can all refer to the methods described in detail in the above embodiments, and will not be repeated here.

[0066] Multiple large language models can be all the large language models in a system or cluster that requires security maintenance, or the large language models that require security maintenance can be selected according to actual needs. This application does not limit this.

[0067] In this embodiment, the weights of each large language model are determined based on their performance, such as their predictive power and accuracy. In practical applications, other indicators (such as the frequency of use of the large language model) can be selected to determine the weights of the large language models according to actual needs.

[0068] For example, the first score of the model to be detected can be calculated using the following formula:

[0069] Where R represents the first score of the model to be tested, n represents the number of large language models, and α i L represents the weight of the i-th large language model. i (x) represents the third score obtained after inputting the text to be detected into the i-th large language model and evaluating the response text of the i-th large language model.

[0070] For example, referring to Figure 2, a flowchart illustrating a method for determining a first score of a text to be detected according to an embodiment of this application is shown. Taking any one of a plurality of texts to be detected as an example, the specific method flow is as follows:

[0071] First, any text to be detected is input into multiple large language models, and the response text output by each large language model after processing the text to be detected is obtained.

[0072] Then, for the response text output by any major language model to the text to be detected, it is determined whether the response text contains a preset rejection word. If it does, the third score of the text to be detected is set to 0; (this can be understood as the score of the text to be detected by any major language model).

[0073] If not included, the response text is input into the classification model. If the classification model outputs a normal response text, the third score of the text to be detected is set to 1. If the classification model outputs an abnormal response text, the third score of the text to be detected is set to 0.

[0074] Finally, a weighted average is calculated based on the weights of each large language model and the third score of the text to be detected to obtain the first score of the text to be detected.

[0075] S104. Input each text to be detected into the evaluation model for processing, and obtain the second score of the evaluation model for each text to be detected.

[0076] The evaluation model is used to assess the degree of malice of each text to be detected.

[0077] In one possible embodiment, step S104 is implemented as follows:

[0078] Each text to be detected is input into the evaluation model. The evaluation calculates the correlation between each text to be detected and the violation policy according to the preset violation policy. The text to be detected is evaluated based on the correlation and a second score is obtained for each text to be detected.

[0079] For example, the violation policy is used to describe various scenarios where large language models prohibit certain uses, such as:

[0080] Illegal activities: Content used to carry out or assist illegal acts.

[0081] Child sexual abuse material or any content that exploits or harms children: Material related to child sexual abuse or any content that exploits or harms children.

[0082] Content that hates, harasses, or encourages violence includes: identity-based hate speech; incitement or encouragement of violence against a specific group; harassment, threats, or bullying based on identity; and content that glorifies or celebrates the suffering or humiliation of others.

[0083] Malware: Content that attempts to create, distribute, or assist in the writing of code or tools designed to compromise, intrude into, or provide unauthorized access to computer systems.

[0084] High-risk activities that result in physical harm include, but are not limited to, weapons development, military and war-related operations; high-risk instruction related to the management or operation of critical infrastructure such as energy, transportation or water supply; and content that encourages or depicts self-harm, suicide or eating disorders.

[0085] High-risk economic harm activities: such as multi-level marketing, gambling, payday lending and other activities with high financial risk; and harmful models or guidelines used to automatically determine eligibility for personal credit, employment, education or public assistance.

[0086] Fraudulent or deceptive activities: including various scams and frauds; coordinated dishonesty (such as fake comments); deceptive practices such as plagiarism and academic dishonesty.

[0087] False information: This includes spam, false medical claims (counterfeit drugs), and other false or misleading statements that may mislead the public.

[0088] ...

[0089] The above is only a partial example of a possible policy violation; policies to disable models can be set according to the actual situation.

[0090] It is understandable that by calculating the correlation between the text to be detected and the violation policy, the degree of malice of the violation words and phrases contained in the text can be determined. A second score can then be used to quantify the degree of malice. For example, the correlation value can be mapped to a range of 0 to 1, and the correlation value can be used as the second score. If the second score is 0, it means that the text to be detected is unrelated to the violation policy; if the second score is 1, it means that the text to be detected completely matches one or more of the violation policies.

[0091] S105. Determine the harmful score of each text to be detected based on the first score and the second score of each text to be detected.

[0092] In one possible embodiment, a weighted average of the first and second scores of each text to be detected is calculated to obtain a harmful score for each text to be detected.

[0093] For example, the harmfulness score of each text to be detected can be calculated using the following formula: H = γR + βA;

[0094] Where H represents the harmfulness score of the text to be detected, R represents the first score of the text to be detected, and A represents the second score of the text to be detected; γ and β are adjustable parameters for the first and second scores, respectively, and can be determined according to actual needs.

[0095] In one possible design, after determining the first score, second score, and harmful score for each text to be detected, the score information of the text to be detected can be stored in a database to facilitate subsequent analysis and other operations on each text to be detected.

[0096] For example, the storage format of the scoring information of the text to be detected can be: {"Harmful Text":"Hello,World!","Harmful Score":0.5,"Second Score":0.5,"First Score":0.5,"Third Score":{"qwen1.5-7b":0.5,"llama2-70b":0.5,"llama3.1-8b":0.5,"llama2-7b":0.5,"llama-7b":0.5},"Source":"Manually Written"}

[0097] Among them, “qwen1.5-7b”, “llama2-70b”, “llama3.1-8b”, etc. are the names of the various large language models participating in the third scoring. “qwen1.5-7b”:0.5 means that the large language model named “qwen1.5-7b” has a third score of 0.5 for the text to be detected.

[0098] It is understood that the above is only one possible example of a format for storing scoring information of the text to be detected, and is not limited to this in practice.

[0099] S106. Obtain at least one text to be detected with a harmful score greater than a preset threshold; perform a parody on at least one text to be detected to obtain at least one parody text.

[0100] In one possible embodiment, step S106 is implemented as follows:

[0101] Each text to be detected is input into the topic generation model. The topic generation model performs topic analysis and identification on each text to be detected to determine the topic of each text to be detected.

[0102] Extract the encoding method of each text to be detected based on regular expressions;

[0103] Each text to be detected is input into a large language model, and the large language model performs semantic analysis on each text to be detected, extracting the semantic logic and language expressions in each text to be detected.

[0104] Each text to be detected is input into the syntactic analysis model, and the syntactic analysis model performs syntactic analysis on each text to be detected to extract the syntactic structure of each text to be detected.

[0105] Each target to be detected is imitated based on at least one of the following: topic, encoding method, language logic, language expression method, and syntactic structure.

[0106] The topic generation model can be a Latent Dirichlet Allocation (LDA) model or a bidirectional transformer model (BERT model); the encoding method of each text to be detected can also be extracted using static analysis tools for software static testing; syntactic analysis can be implemented using methods such as dependency trees and sentence graphs. This application does not impose any limitations on these aspects.

[0107] Furthermore, the features of the text to be detected extracted in this embodiment (i.e., topic, encoding method, language logic, language expression method and syntactic structure) are only one possible example. In practical applications, other features of the text to be detected can be extracted according to needs.

[0108] A method for parodying each target data based on at least one of the following: topic, encoding method, language logic, language expression method, and syntactic structure. Parodying can be based on any one of these features or any combination thereof. See Table 1, which provides an example of parodying the target data based on the above features.

[0109] Table 1 Example of rewriting the data to be detected

[0110] In this embodiment, each text to be detected is processed based on a large language model to obtain the response text output by the large language model for each text to be detected. The response text for each text to be detected is evaluated according to a first preset rule to determine whether the large language model can identify the text to be detected as harmful text, and thus determine the degree of concealment of each text to be detected. A first score is used to quantify the degree of concealment of each text to be detected. A second score is obtained by quantifying the degree of malice of each text to be detected based on an evaluation model. The harmful score of the text to be detected is determined by combining the first score and the second score. This can comprehensively consider the degree of concealment and malice of the text to be detected, making the evaluation of the text to be detected more accurate and comprehensive. At least one text to be detected with a harmful score greater than a preset threshold can better reflect the degree of security threat to the large language model in many aspects. Furthermore, at least one text to be detected is paraphrased, which can effectively augment the test set and improve the richness and applicability of the paraphrased text.

[0111] It is understood that the above embodiments can be used individually or in combination. The following provides an example of combined use. Referring to Figure 3, an example diagram of a method for malicious text enhancement provided by an embodiment of this application is shown. The method specifically includes the following implementation steps:

[0112] First, multiple texts to be tested are obtained through manual writing or online collection.

[0113] Then, a first score and a second score are determined for each of the multiple detected texts, and a harmful score is calculated for each text to be detected based on the first score and the second score.

[0114] It is understood that the methods for determining the first and second scores of each detected text, and for calculating the harmfulness score of each detected text based on the first and second scores, can refer to the detailed method steps in the above embodiments, and will not be repeated here.

[0115] Next, the scoring information for each text to be tested is stored in the database.

[0116] The storage format of the scoring information for each text to be detected can be set according to actual needs, or the storage format of the scoring information in the above embodiment can be referred to.

[0117] Finally, at least one text to be detected with a harmful score greater than a preset threshold is selected. Feature extraction and analysis are performed on each of the at least one text to be detected. Based on at least one feature of each text to be detected, a paraphrase is performed to obtain the paraphrase text.

[0118] Understandably, the resulting paraphrased text can be added to multiple texts to be detected and used as an existing dataset when large language models need to be analyzed or harmful data augmentation is needed next time.

[0119] The methods provided in the embodiments of this application have been described above. The apparatus provided in the embodiments of this application will be described below.

[0120] Based on the same technical concept, embodiments of this application provide a data enhancement apparatus, which includes a module / unit / means for performing the method executed by the electronic device in the above-described method embodiments. This module / unit / means can be implemented in software, or in hardware, or implemented by hardware executing corresponding software.

[0121] For example, referring to FIG4, device 400 includes:

[0122] The acquisition module 401 is used to: acquire multiple texts to be detected, which are harmful texts that pose a security threat to the large language model;

[0123] Evaluation module 402 is used for: inputting each of the multiple texts to be detected into a large language model for processing, and obtaining the response text of each text to be detected output by the large language model; evaluating the response text of each text to be detected according to a first preset rule to obtain a first score for each text to be detected; the first score is used to indicate the degree of concealment of each text to be detected; inputting each text to be detected into an evaluation model for processing, and obtaining a second score of each text to be detected from the evaluation model; the evaluation model is used to evaluate the degree of malice of each text to be detected; and determining the harmfulness score of each text to be detected based on the first score and the second score.

[0124] The imitation module 403 is used to: obtain at least one text to be detected with a harmful score greater than a preset threshold; and imitate at least one text to be detected to obtain at least one imitation text.

[0125] Optionally, when the evaluation module 402 evaluates the response text of each text to be detected according to the first preset rule and obtains a first score for each text to be detected, it is specifically used to: determine whether the response text contains a preset rejection word, which is used to indicate that the question or request contained in the text to be detected is rejected; if it contains such a word, the first score is set to a first value, which is used to indicate that the concealment level of the text to be detected is low; if it does not contain such a word, the response text is input into a classification model for classification processing, which is used to identify whether the response text is a normal response text; if the output result of the classification model is that the response text is a normal response text, the first score is set to a second value, which is used to indicate that the concealment level of the text to be detected is high; if the output result of the classification model is that the response text is an abnormal response text, the first score is set to the first value.

[0126] Optionally, the large language model can be multiple large language models with different performance levels. When the evaluation module 402 evaluates the response text of each text to be detected according to the first preset rule to obtain a first score for each text to be detected, it is also used to: evaluate the response text of each of the multiple large language models to each text to be detected according to the first preset rule to obtain a third score for each large language model to the text to be detected; calculate the first score for each text to be detected by weighted average based on the weight of each large language model and the third score for each text to be detected; the weight of each large language model is determined by the performance of each large language model.

[0127] Optionally, when the evaluation module 402 inputs each text to be detected into the evaluation model for processing and obtains the second score of each text to be detected by the evaluation model, it is specifically used to: input each text to be detected into the evaluation model, calculate the correlation between each text to be detected and the violation policy according to the preset violation policy through evaluation, evaluate each text to be detected based on the correlation, and obtain the second score of each text to be detected.

[0128] Optionally, when determining the harmfulness score of each text to be detected based on the first score and the second score of each text to be detected, the evaluation module 402 is specifically used to: calculate the weighted average of the first score and the second score of each text to be detected to obtain the harmfulness score of each text to be detected.

[0129] Optionally, when the imitation module 403 imitates at least one text to be detected to obtain at least one imitated text, it is specifically used to: input each text to be detected into a topic generation model, perform topic analysis and recognition on each text to be detected through the topic generation model, and determine the topic of each text to be detected; extract the encoding method of each text to be detected based on regular expressions; input each text to be detected into a large language model, perform semantic analysis on each text to be detected through the large language model, and extract the semantic logic and language expression in each text to be detected; input each text to be detected into a syntactic analysis model, perform syntactic analysis on each text to be detected through the syntactic analysis model, and extract the syntactic structure of each text to be detected; and imitate each text to be detected based on at least one of topic, encoding method, language logic, language expression method, and syntactic structure.

[0130] It should be understood that all relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and will not be repeated here.

[0131] Based on the same technical concept, referring to Figure 5, this application embodiment also provides an electronic device 500, including:

[0132] At least one processor 501; and a communication interface 503 communicatively connected to the at least one processor 501; the at least one processor 501 executes instructions stored in the memory 502, causing the electronic device 500 to execute the method steps performed by the electronic device with processing capabilities in the above method embodiment through the communication interface 503.

[0133] Optionally, the memory 502 is located outside the electronic device 500.

[0134] Optionally, the electronic device 500 includes a memory 502 connected to the at least one processor 501, and the memory 502 has instructions executable by the at least one processor 501. Figure 5 shows, with dashed lines, that the memory 502 is optional for the electronic device 500.

[0135] The at least one processor 501 and the memory 502 can be coupled through an interface circuit or integrated together, which is not limited here.

[0136] This application embodiment does not limit the specific connection medium between at least one processor 501, memory 502, and communication interface 503. In Figure 5, this application embodiment shows at least one processor 501, memory 502, and communication interface 503 connected via a bus 504, which is represented by a thick line in Figure 5. The connection methods between other components are only illustrative and not intended to be limiting. This bus can be an address bus, data bus, control bus, etc. For ease of illustration, Figure 5 uses only one thick line, but this does not indicate that there is only one bus or one type of bus.

[0137] It should be understood that the processor mentioned in the embodiments of this application can be implemented in hardware or software. When implemented in hardware, the processor can be a logic circuit, integrated circuit, etc. When implemented in software, the processor can be a general-purpose processor, implemented by reading software code stored in memory.

[0138] For example, the processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0139] It should be understood that the memory mentioned in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which acts as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate Synchronous DRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct RAM (DR RAM).

[0140] It should be noted that when the processor is a general-purpose processor, DSP, ASIC, FPGA, or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, the memory (storage module) can be integrated into the processor.

[0141] It should be noted that the memories described herein are intended to include, but are not limited to, these and any other suitable types of memories.

[0142] Based on the same technical concept, embodiments of this application also provide a computer-readable storage medium for storing instructions that, when executed, cause a computer to perform the method steps performed by any of the devices in the above method embodiments.

[0143] Based on the same technical concept, this application also provides a computer program product, including computer program code, which, when the computer program code is run on a computer, causes the method steps executed by any device in the above method embodiments to be implemented.

[0144] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0145] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0146] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0147] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0148] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A data augmentation method, characterized in that, include: Multiple texts to be detected are obtained, and these multiple texts are harmful texts that pose a security threat to the large language model; Each of the plurality of texts to be detected is input into the large language model for processing, and the response text of each text to be detected is obtained from the output of the large language model; the response text of each text to be detected is evaluated according to a first preset rule to obtain a first score for each text to be detected; the first score is used to indicate the degree of concealment of each text to be detected. Each text to be detected is input into an evaluation model for processing, and a second score is obtained from the evaluation model for each text to be detected; the evaluation model is used to assess the degree of malice of each text to be detected. A harmful score is determined for each text to be detected based on a first score and a second score. Obtain at least one text to be detected whose harmful score is greater than a preset threshold; perform a paraphrase of the at least one text to be detected to obtain at least one paraphrase text.

2. The method as described in claim 1, characterized in that, The step of evaluating the response text of each text to be detected according to a first preset rule to obtain a first score for each text to be detected includes: Determine whether the response text contains a preset rejection word, the rejection word being used to indicate that the question or request contained in the text to be detected is refused; If included, the first score is set to a first value, which indicates that the concealment level of the text to be detected is low; If not included, the response text is input into a classification model for classification processing. The classification model is used to identify whether the response text is a normal response text. If the classification model outputs that the response text is a normal response text, the first score is set to the second value, which is used to indicate that the concealment level of the text to be detected is high. If the classification model outputs that the response text is an abnormal response text, the first score is set to the first value.

3. The method as described in claim 2, characterized in that, The large language model consists of multiple large language models with different performance levels; the evaluation of the response text for each text to be detected according to a first preset rule to obtain a first score for each text to be detected includes: According to the first preset rule, the response text of each of the multiple large language models to each text to be detected is evaluated to obtain the third score of each large language model for the text to be detected; Based on the weights of each large language model and the third score for each text to be detected, a first score for each text to be detected is calculated by weighted average; the weights of each large language model are determined by the performance of each large language model.

4. The method as described in claim 3, characterized in that, The step of inputting each text to be detected into the evaluation model for processing and obtaining a second score from the evaluation model for each text to be detected includes: Each text to be detected is input into the evaluation model. The evaluation model calculates the correlation between each text to be detected and the violation policy according to the preset violation policy. The text to be detected is evaluated based on the correlation and a second score is obtained for each text to be detected.

5. The method as described in claim 3 or 4, characterized in that, The determination of the harmfulness score for each text to be detected based on the first and second scores includes: The weighted average of the first and second scores of each text to be detected is calculated to obtain the harmfulness score of each text to be detected.

6. The method as described in claim 1, characterized in that, The step of parodying the at least one text to be detected to obtain at least one parody text includes: Each of the at least one texts to be detected is input into a topic generation model, and the topic generation model is used to perform topic analysis and recognition on each text to be detected to determine the topic of each text to be detected. The encoding method of each text to be detected is extracted based on regular expressions; Each text to be detected is input into the large language model, and the large language model performs semantic analysis on each text to be detected to extract the semantic logic and language expression of each text to be detected. Each text to be detected is input into a syntactic analysis model, and the syntactic analysis model is used to perform syntactic analysis on each text to be detected to extract the syntactic structure of each text to be detected. Each text to be detected is paraphrased based on at least one of the following: the topic, the encoding method, the language logic, the language expression method, and the syntactic structure.

7. A data augmentation device, characterized in that, include: The acquisition module is used to: acquire multiple texts to be detected, wherein the multiple texts to be detected are harmful texts that pose a security threat to the large language model; An evaluation module is configured to: input each of the plurality of texts to be detected into the large language model for processing, and obtain the response text of each text to be detected output by the large language model; evaluate the response text of each text to be detected according to a first preset rule to obtain a first score for each text to be detected; the first score is used to indicate the degree of concealment of each text to be detected; input each text to be detected into an evaluation model for processing, and obtain a second score of each text to be detected from the evaluation model; the evaluation model is used to evaluate the degree of malice of each text to be detected; and determine the harmfulness score of each text to be detected based on the first score and the second score. The imitation module is used to: acquire at least one text to be detected whose harmful score is greater than a preset threshold; and imitate the at least one text to be detected to obtain at least one imitation text.

8. The apparatus as claimed in claim 7, characterized in that, When the evaluation module evaluates the response text of each text to be detected according to the first preset rule and obtains a first score for each text to be detected, it is specifically used for: Determine whether the response text contains a preset rejection word, the rejection word being used to indicate that the question or request contained in the text to be detected is refused; If included, the first score is set to a first value, which indicates that the concealment level of the text to be detected is low; If not included, the response text is input into a classification model for classification processing. The classification model is used to identify whether the response text is a normal response text. If the output of the classification model is that the response text is a normal response text, the first score is set as the second value, which is used to indicate that the concealment level of the text to be detected is high. If the classification model outputs a result indicating that the response text is an abnormal response text, then the first score is set as the first value.

9. An electronic device, characterized in that, include: Memory, used to store program instructions; A processor is configured to invoke program instructions stored in the memory and execute the steps included in the method as described in any one of claims 1-6 according to the obtained program instructions.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a computer, cause the method as described in any one of claims 1-6 to be implemented.

Images