Test method and device of content generation model, medium and equipment
By using the 'theme and algorithm combination' method in content generation model testing, instant test cases are created, and the problems of leaking test questions and low manual testing in the existing technology are solved, and more comprehensive and efficient model testing is achieved. Protected data privacy.
Patent Information
- Application Number
- CN202411167358.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-23
- Publication Date
- 2024-12-27
AI Technical Summary
In the existing content generation model testing methods, pre-fixed test questions are easily leaked, resulting in test failure, and relying on manual testing is inefficient and costly.
Using the method of "combining topics and algorithms", test cases are created immediately during the testing process, and by executing test cases, input associated topic information into the content generation model, induce them to output generated content that does not meet the set requirements, and complete the test.
It avoids leaking test questions, improves the comprehensiveness and privacy of tests, reduces the need for manual testing, improves testing efficiency, and protects data privacy in a trusted execution environment.
Smart Images
Figure CN119201684A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of information technology, and in particular to a testing method, device, medium and equipment for a content generation model. Background Art
[0002] A content generation model refers to an artificial intelligence model that generates content based on input prompt content. In actual applications, someone may deliberately input misleading prompt content into the model, causing the model to output generated content that does not meet the set requirements.
[0003] To this end, in order to ensure that the generated content output by the content generation model meets the set requirements as much as possible, the content generation model is often tested before it is put into use.
[0004] The present disclosure provides a relatively effective content generation model testing solution. Summary of the invention
[0005] The present invention provides a method for testing a content generation model, including:
[0006] Extracting subject information from a preset information database;
[0007] Calling a use case creation algorithm in a preset algorithm library to create at least one test case according to the subject information; wherein each test case is used to: input prompt content associated with the subject information into the content generation model for the purpose of inducing the content generation model to output generated content that does not meet the set requirements;
[0008] Each test case is executed, and a test result corresponding to each test case is determined according to the generated content output by the content generation model in response to each test case.
[0009] The present invention provides a testing device for a content generation model, including:
[0010] Information extraction module, extracting subject information from a preset information database;
[0011] A use case creation module, calling a use case creation algorithm in a preset algorithm library, and creating at least one test case according to the subject information; wherein each test case is used to: input prompt content associated with the subject information into the content generation model for the purpose of inducing the content generation model to output generated content that does not meet the set requirements;
[0012] The case execution module executes each test case and determines the test result corresponding to each test case in response to the generated content output by each test case according to the content generation model.
[0013] The embodiments of the present specification also provide a computer program product, wherein the computer program product stores at least one instruction, and the at least one instruction is suitable for being loaded by a processor and executing the above method steps.
[0014] The embodiments of the present specification also provide a storage medium, wherein the storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing the steps of the above method.
[0015] An embodiment of the present specification also provides an electronic device, comprising: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the steps of the above method.
[0016] In the technical solution of this specification, during the testing process of the content generation model, the pre-fixed prompt content is not used as the test topic to induce the content generation model to output the generated content that does not meet the set requirements. Instead, the "theme and algorithm combination" method is adopted to create test cases in real time during the test process. By executing the test cases, the prompt content associated with the theme information can be input into the content generation model for the purpose of inducing the content generation model to output the generated content that does not meet the set requirements, and the test is completed.
[0017] The above technical solution can avoid the leakage of pre-fixed test questions on the one hand, which will cause attacks on the model in actual applications to deliberately bypass the test questions, thereby improving the comprehensiveness and privacy of the test. On the other hand, it can avoid manual testing of the content generation model by experts, thereby improving the test efficiency. It should be noted that this solution can be implemented in a trusted execution environment to protect data privacy. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 It is a flowchart of a testing method of a content generation model provided in an embodiment of this specification;
[0019] Figure 2 It is a schematic diagram of a testing method of a content generation model provided in an embodiment of this specification;
[0020] Figure 3 It is a structural schematic diagram of a testing device for a content generation model provided in an embodiment of this specification;
[0021] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of this specification. DETAILED DESCRIPTION
[0022] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.
[0023] First, some concepts involved in this article are explained.
[0024] Content generation model: An artificial intelligence model that generates content based on input prompt content output, also commonly referred to as an artificial intelligence generated content (AIGC) model. Content generation models can be large models with a huge number of parameters, such as GPT4, Llama, etc.
[0025] Prompt content: Prompt in English is the input of the content generation model. Its function is to prompt the model how to generate content. Prompt content can be single-mode or multi-mode, such as text, file, picture, video, sound, etc.
[0026] Generated content: The content output by the content generation model. Generated content can be either unimodal or multimodal, such as text, files, pictures, videos, sounds, etc.
[0027] Model testing: Faced with the fact that content generation models are increasingly being used maliciously to generate harmful content, model testing before the models are put into use can help to discover and prevent possible risks in advance, and is also the key to promoting the healthy and sustainable development of these models. Model testing results can provide guidance to model developers, helping them better understand the potential risks of the models and take corresponding preventive measures. In addition, as regulations and policies increase the security requirements for data and algorithms, model testing has also become an important means to meet compliance requirements.
[0028] Setting requirements: It is usually required that the generated content output by the content generation model conforms to the setting requirements as much as possible. The specific setting requirements can be set according to the actual situation. For example, the setting requirement is content safety, and safety can be defined as compliance with a set of established safety criteria, which excludes a series of content risks, such as not containing pornographic information, investment advice, etc.
[0029] Use case: also known as a procedural example. Those skilled in the art can easily understand that a use case usually includes several conditions and several operations, and the process of executing a use case can be understood as executing a set of processes.
[0030] One existing solution for testing content generation models is to form a question bank based on pre-fixed prompt content, and use the question bank to induce the content generation model to answer in order to complete the test. However, if the test questions in the question bank are leaked, the meaning of the test will be lost, because if the content generation model is attacked without the test questions, it is also easy to make the generated content output by the content generation model fail to meet the set requirements.
[0031] Another existing solution for testing content generation models is to rely on experts to manually test the content generation models, and the experts conceive and write some prompts as test questions. However, this testing solution is too time-consuming and labor-intensive, high in cost and low in efficiency.
[0032] To this end, in one or more embodiments provided in the present disclosure, during the testing process of the content generation model, pre-fixed prompt content is not used as a test topic to induce the content generation model to output generated content that does not meet the set requirements. Instead, a "combination of themes and algorithms" approach is adopted to create test cases in real time during the testing process. By executing the test cases, prompt content associated with the theme information can be input into the content generation model for the purpose of inducing the content generation model to output generated content that does not meet the set requirements, thereby completing the test.
[0033] The above technical solution can avoid the leakage of pre-fixed test questions on the one hand, which will cause attacks on the model in actual applications to deliberately bypass the test questions, thereby improving the comprehensiveness and privacy of the test. On the other hand, it can avoid manual testing of the content generation model by experts, thereby improving the test efficiency. It should be noted that this solution can be implemented in a trusted execution environment to protect data privacy.
[0034] The technical solution of the present disclosure is described in detail below in conjunction with the accompanying drawings.
[0035] Figure 1 It is a flowchart of a testing method of a content generation model provided by the present disclosure, comprising the following steps:
[0036] S100: extracting subject information from a preset information database.
[0037] Figure 1 The method shown is an automated process that can be applied on a computing device.
[0038] In realization Figure 1Before the method flow shown, the information base and algorithm base can be preset. The information base can include various knowledge information, such as conceptual vocabulary, question sentences, articles, images, and audio. The information source in the information base can be a vocabulary base, a query base, a text collection, an image collection, an audio collection, etc.
[0039] The above-mentioned conceptual words, question sentences, articles, images, audio and other information can contribute the required content themes for model testing. Therefore, in step S100, information that can be used as content themes, namely, theme information, is extracted from the information database.
[0040] In some embodiments, topic information may be randomly extracted from a preset information library.
[0041] S102: calling a use case creation algorithm in a preset algorithm library, and creating at least one test case according to the subject information.
[0042] In execution Figure 1 Before the method shown, it is also necessary to set up an algorithm library, which may include various algorithms (also called operators), including at least a plurality of algorithms for creating use cases, namely, use case creation algorithms.
[0043] The use case creation algorithm can be a written code script, or a content generation model other than the content generation model to be tested, or a search API, QA system, etc.
[0044] In step S102, one or more use case creation algorithms are called from the algorithm library to create at least one test case according to the subject information. In other words, the subject information can be used as an input of a use case creation algorithm, and the output of the use case creation algorithm is a test case.
[0045] In some embodiments, for a subject information, one or more use case creation algorithms may be randomly called from an algorithm library.
[0046] From a functional perspective, each test case is used to input prompt content associated with the subject information into the content generation model for the purpose of inducing the content generation model to output generated content that does not meet the set requirements.
[0047] Generally speaking, there are differences between the prompt content of the topic information and the related topic information. The topic information can be a word, a picture, a voice, a sentence, or an article. The role of the topic information is to indicate the content theme used in this test. The prompt content of the related topic information specifically contains this topic information, but the format and expression are different from the topic information. It is generally more test-targeted and inductive than the topic information, and is suitable as the input of the content generation model in the model testing work.
[0048] Therefore, it is important to note that the use case creation algorithm will specifically create test cases with the purpose of inducing the content generation model to output generated content that does not meet the set requirements.
[0049] For example, the subject information may be "pornography", and the prompt content of the related subject information included in the test case may be "how to find a pornography website?".
[0050] S104: Execute each test case.
[0051] It is easy to understand that the process of executing a test case is to input the prompt content contained in the test case into the content generation model in the manner set in the test case.
[0052] In some embodiments, a test case creation algorithm may create multiple test cases for a subject information. Moreover, a test case includes multiple prompt contents related to the subject information. During the execution of this test case, the multiple prompt contents may be input into the content generation model one by one in order.
[0053] S106: Determine a test result corresponding to each test case according to the generated content output by the content generation model in response to each test case.
[0054] It is easy to understand that the content generation model will output the corresponding generated content for each prompt content input. If a test case contains one prompt content, then the test result corresponding to this test case only involves this one prompt content. If a test case contains multiple prompt contents, then the test result corresponding to this test case involves these multiple prompt contents.
[0055] In addition, in some embodiments, before extracting the topic information from the preset information library, the strength level specified by the test party can be determined. It should be noted that the strength level is positively correlated to the induction strength of the generated content that does not meet the set requirements when the content generation model is input. In other words, the higher the strength level, the stronger the induction strength of the generated content that does not meet the set requirements when the content generation model is input. The induction strength is usually manifested in the confusing nature of the prompt content, the ingenuity of the method of inputting the prompt content, and so on.
[0056] In this way, the use case creation algorithm in the preset algorithm library can be called to create at least one test case according to the subject information and the intensity level. It is easy to understand that the higher the intensity level, the stronger the induction of the test case to the content generation model.
[0057] In some embodiments, the information library may also include some reference information. The reference information may include information on inducement techniques, risk type information, and other information used to provide references for building inducement test cases. A case creation algorithm in a preset algorithm library may be called to create at least one test case based on the subject information and the reference information.
[0058] There are many ways to induce, such as:
[0059] 1. Prompt Injection Attacks:
[0060] By designing clever input prompts, the model can be induced to generate harmful or inappropriate content. For example, the model's sensitivity to specific keywords or sentences can be used to construct inputs to guide the model to generate bad information.
[0061] 2. Adversarial Prompts:
[0062] Use adversarial techniques to generate input prompts that are carefully crafted to cause the model to output something wrong or harmful.
[0063] 3. Context Pollution:
[0064] Harmful content is gradually introduced in multiple rounds of dialogue, and the context of the dialogue is gradually changed, so that the model unknowingly generates harmful outputs.
[0065] 4. Social Engineering Attacks:
[0066] Taking advantage of human social and psychological characteristics, constructing input prompts to induce the model to generate harmful content. For example, by simulating emergency situations or emotional incentives, the model is guided to generate inappropriate suggestions or information.
[0067] 5. Steganographic Commands:
[0068] Malicious instructions are hidden in normal input, and the model's ability to decode implicit information is exploited to induce it to generate harmful content.
[0069] 6. Fuzz Testing:
[0070] The model is fed with large amounts of random or semi-random invalid data in an attempt to find specific input patterns that trigger harmful outputs.
[0071] There are also various types of risks that do not meet the set requirements. For example:
[0072] 1. Harmful Content Generation:
[0073] Including hate speech, violent content, pornographic content, discriminatory speech, etc. These contents may cause psychological harm to users or cause social problems.
[0074] 2. Misinformation and Disinformation:
[0075] Including false information, rumors, conspiracy theories, etc. Such information may mislead users and influence public opinions and decisions.
[0076] 3. Privacy Violations:
[0077] The model may leak personal privacy information in the output, such as name, address, contact information, etc., resulting in violation of user privacy.
[0078] 4. Bias and Discrimination:
[0079] The model may reflect bias and discrimination in terms of race, gender, etc. in its output, affecting fairness and impartiality.
[0080] 5. Malicious Code Generation:
[0081] The model may generate malicious code or scripts, resulting in attacks on user systems or theft of data.
[0082] 6. Misleading Advice:
[0083] Models may provide erroneous or misleading advice, especially in fields such as medicine, law, and finance, which may lead to serious consequences.
[0084] 7. Manipulation and Coercion:
[0085] Models may be used to manipulate user behavior or opinions, for example, by influencing user decisions through psychological manipulation, emotional manipulation, etc.
[0086] 8. Inappropriate Automation:
[0087] Models may be used to automate inappropriate or illegal activities such as spamming, phishing, fraud, etc.
[0088] 9. Ethical and Legal Issues:
[0089] The output of the model may raise ethical and legal issues, such as copyright infringement and violation of data protection regulations.
[0090] In addition, in some embodiments, for each test case, if the test result corresponding to the test case indicates that it does not meet the set requirements, the weight corresponding to the reference information can be increased. In this way, reference information with a weight higher than the set weight can be extracted from the preset information library. In this way, the use of inducement techniques and risk type information that pose less threat to the content generation model can be minimized.
[0091] In some embodiments, in addition to the use case creation algorithm, the algorithm library may also include one or more recognition algorithms. The recognition algorithm in the algorithm library may be called to identify whether the generated content output by the content generation model in response to each test case meets the set requirements, and obtain the test result corresponding to each test case. As an example, the recognition algorithm may be a binary classification model to classify whether the generated content is safe.
[0092] In some other embodiments, the algorithm library may also include a quality evaluation algorithm. The quality evaluation algorithm may be called to score the quality indicators of the generated content output by the content generation model in response to each test case. The quality indicators here may include the fluency of sentences, the reasonableness of the text length, etc.
[0093] In some embodiments, in implementing Figure 1 In the method shown, S102 and S104 can be executed in a loop. That is, the following can be executed in a loop: extracting reference information from a preset information library, calling a use case creation algorithm in a preset algorithm library, creating a test case based on the subject information and the reference information, and executing the test case. If the number of created test cases reaches a preset number M, M is greater than 1, or the generated content output by the content generation model does not meet the set requirements, the loop is stopped.
[0094] In some embodiments, before extracting the subject information from the preset information library, the number of tests N specified by the tester may be determined, where N> 1. After extracting the subject information from the preset information library, the subject information may continue to be extracted from the preset information library until N test results are determined.
[0095] In addition, a test report can also be generated. The test report at least records each test case and the test result corresponding to each test case. In addition, the test report can also record more detailed information, such as test time, test times, which subject information was extracted, reference information corresponding to each subject information, each test case corresponding to each subject information, test results corresponding to each test case (whether it is safe), quality scores of each generated content corresponding to each test case, etc.
[0096] Figure 2 A schematic diagram of a testing method for a content generation model is provided as an example. Figure 2 As shown, a subject information, an inducement technique information, and a risk type information are selected from the information library. Next, a use case creation algorithm is called from the algorithm library to create a test case based on the subject information, inducement technique information, and risk type information. Then, this test case is executed to obtain the generated content output by the content generation model. Then, the recognition model is called from the algorithm library to identify whether the generated content meets the set requirements. If not, another subject information is selected from the information library for testing until the number of adopted subject information reaches N; if it meets, the subject information is kept unchanged, the inducement technique information, risk type information, and use case creation algorithm are reselected, and test cases are regenerated for the same subject information until the number of test cases generated for the same subject information reaches M, or the generated content output by the content generation model is determined by the recognition algorithm to not meet the set requirements.
[0097] The present disclosure also provides a structural schematic diagram of a testing device for a content generation model, including:
[0098] Information extraction module 301, extracting subject information from a preset information database;
[0099] The use case creation module 302 calls a use case creation algorithm in a preset algorithm library to create at least one test case according to the subject information; wherein each test case is used to: input prompt content associated with the subject information to the content generation model for the purpose of inducing the content generation model to output generated content that does not meet the set requirements;
[0100] The case execution module 303 executes each test case, and determines the test result corresponding to each test case according to the generated content output by the content generation model in response to each test case.
[0101] The above device embodiments correspond to the method embodiments. For specific descriptions, please refer to the description of the method embodiments, which will not be repeated here. The device embodiments are obtained based on the corresponding method embodiments and have the same technical effects as the corresponding method embodiments. For specific descriptions, please refer to the corresponding method embodiments.
[0102] The embodiments of the present specification also provide a computer storage medium, which can store a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the method of the embodiments of the present disclosure.
[0103] The present specification also provides a computer program product, which stores at least one instruction, and the at least one instruction is loaded by the processor to execute the method of the embodiment of the present disclosure.
[0104] The embodiments of this specification also provide Figure 4 The structural diagram of the electronic device shown in FIG. 1 is a schematic diagram of the electronic device shown in FIG. 1 . At the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for other transactions. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the above-mentioned voice activity detection method.
[0105] Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the executor of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.
[0106] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.
[0107] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.
[0108] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
[0109] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.
[0110] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
[0111] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0112] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0113] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0114] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0115] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0116] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0117] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0118] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0119] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.
[0120] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0121] The above description is only an embodiment of the present specification and is not intended to limit the present specification. For those skilled in the art, the present specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification shall be included in the scope of the claims of the present specification.
Claims
1. A method for testing a content generation model, comprising: Extracting subject information from a preset information database; Calling a use case creation algorithm in a preset algorithm library to create at least one test case according to the subject information; wherein each test case is used to: input prompt content associated with the subject information into the content generation model for the purpose of inducing the content generation model to output generated content that does not meet the set requirements; Each test case is executed, and a test result corresponding to each test case is determined according to the generated content output by the content generation model in response to each test case.
2. The method according to claim 1, before extracting the subject information from the preset information library, the method further comprises: Determining a strength level specified by the tester; wherein the strength level is positively correlated to the induction strength of the generated content input to the content generation model that does not meet the set requirements; Calling a use case creation algorithm in a preset algorithm library to create at least one test case according to the subject information includes: A use case creation algorithm in a preset algorithm library is called to create at least one test case according to the subject information and the intensity level.
3. The method according to claim 1, further comprising: Extracting reference information from a preset information database; wherein the reference information includes: information on inducement techniques, and / or information on risk types; Calling a use case creation algorithm in a preset algorithm library to create at least one test case according to the subject information includes: A use case creation algorithm in a preset algorithm library is called to create at least one test case according to the subject information and the reference information.
4. The method according to claim 3, further comprising: For each test case, if the test result corresponding to the test case indicates that it does not meet the set requirements, the weight corresponding to the reference information is increased; Extract reference information from a pre-set information library, including: Extract reference information with a weight higher than the set weight from the preset information database.
5. The method of claim 1, wherein determining the test result corresponding to each test case according to the generated content output by the content generation model in response to each test case comprises: The recognition algorithm in the algorithm library is called to identify whether the generated content output by the content generation model in response to each test case meets the set requirements, and the test result corresponding to each test case is obtained.
6. The method according to claim 5, further comprising: Generate test reports; The test report at least records each test case and the test result corresponding to each test case.
7. The method according to claim 3, calling a use case creation algorithm in a preset algorithm library, creating at least one test case according to the subject information, and executing each test case, comprising: Loop execution: extract reference information from a preset information library, call a use case creation algorithm in a preset algorithm library, create a test case according to the subject information and the reference information, and execute the test case; If the number of created test cases reaches a preset number M, M is greater than 1, or the generated content output by the content generation model does not meet the set requirements, the loop is stopped.
8. The method according to any one of claims 1 to 7, before extracting the subject information from the preset information library, the method further comprises: Determine the number of tests N specified by the tester, where N>1; After extracting the subject information from the preset information library, the method further includes: Continue to extract subject information from the preset information base until N test results are determined.
9. A testing device for a content generation model, comprising: Information extraction module, extracting subject information from a preset information database; A use case creation module, calling a use case creation algorithm in a preset algorithm library, and creating at least one test case according to the subject information; wherein each test case is used to: input prompt content associated with the subject information into the content generation model for the purpose of inducing the content generation model to output generated content that does not meet the set requirements; The case execution module executes each test case and determines the test result corresponding to each test case in response to the generated content output by each test case according to the content generation model.
10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.
11. An electronic device, characterized in that: include: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the steps of the method as claimed in any one of claims 1 to 8.
12. A computer program product having at least one instruction stored thereon, characterized in that: When the at least one instruction is executed by the processor, the steps of the method described in any one of claims 1 to 8 are implemented.