Large model optimization method, related device and storage medium

Abstract

The embodiment of the invention relates to the field of large model security, and discloses a large model optimization method, a related device and a storage medium. The method comprises the steps of determining an application field of a to-be-optimized student large model; calling a preset confrontation model to generate a problem set; inputting the question set into the student large model through a preset confrontation model to obtain an output result; outputting the output result to a preset scoring model for scoring to obtain a score; calling a preset teacher model to train the student large model according to a preset high-quality data set to obtain a training result and an optimization scheme; and respectively feeding back the score, the training result and the optimization scheme to the student large model, and continuously updating the problem set to train and score the student large model until the student large model is gradually iterated to be optimal, thereby obtaining the trained student large model. According to the embodiment of the invention, the overall optimization period of the large model is remarkably shortened, the optimization efficiency of the large model is improved, and the performance of the optimized large model is further improved.

Description

Technical Field

[0001] The present application relates to the field of large model security technology, and more specifically to a large model optimization method, related devices and storage media. Background Art

[0002] Large language models are currently widely used in chat dialogue, text editing, artistic creation, coding, mathematical reasoning, bioinformatics and other application scenarios. Although they have created many new business models and are very powerful, after the general large model is launched for users to use, there are mainly algorithmic risks, data risks and application risks in the three types of applications: translation, chat and collaboration.

[0003] Due to the diverse application areas and complex corpus of large language models, there is no automated evaluation and online optimization solution for large language models. Even if algorithmic vulnerabilities in the large language model are detected under certain risk dimensions, there is no complete full-process intelligent solution to seamlessly optimize the large language model, especially when the large language model is a primary model or a student model with poor training results.

[0004] At present, the owner of a large model can only optimize the model offline based on the test results given by the platform. The overall optimization cycle is long, and the model security must be evaluated on the platform for each round of optimization, which is a continuous cycle. It can be seen that the overall process of large model optimization is cumbersome, which leads to low efficiency in obtaining an excellent large language model. It can only be applied to the laboratory verification stage, and cannot be applied to the massive needs of the industrial level. Summary of the invention

[0005] The embodiments of the present application provide a large model optimization method, related devices and storage media, which can realize the automation of the large model optimization process, significantly shorten the overall optimization cycle of the large model, and are suitable for scenarios with diversified application fields and complex corpus of large language models. It improves the optimization efficiency of the large model and further improves the performance of the optimized large model.

[0006] In a first aspect, an embodiment of the present application provides a large model optimization method, comprising:

[0007] Determine the application area of ​​the student model to be optimized;

[0008] Calling a preset adversarial model to generate a question set for adversarial training of the student model according to the application field, wherein the question set includes at least one preset question generated based on text adversarial technology, and each question in the question set is a triple data, and the triple data is a mapping relationship between a question, an excellent answer, and a bad answer;

[0009] Input the problem set into the student big model through a preset adversarial model to obtain an output result;

[0010] Outputting the output result to a preset scoring model for scoring to obtain a scoring value;

[0011] Calling a preset teacher model to train the student model according to a preset high-quality data set to obtain training results and optimization solutions;

[0012] The scoring score, the training result and the optimization plan are respectively fed back to the student big model to be optimized, so that it can continuously reinforce the learning of the difference between the excellent answers and the bad answers corresponding to the question set, and continuously update the question set to train and score the student big model, until the question-answering ability of the student big model is gradually iterated to the optimal, and the trained student big model is obtained.

[0013] In a second aspect, an embodiment of the present application provides a large model optimization device, which has the function of implementing the large model optimization method provided in the first aspect. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions, and the modules can be software and / or hardware.

[0014] In one embodiment, the large model optimization device comprises:

[0015] A determination module, used to determine the application field of the student large model to be optimized;

[0016] A generation module, used for calling a preset adversarial model to generate a question set for adversarial training of the student model according to the application field, wherein the question set includes at least one preset question generated based on text adversarial technology, and each question in the question set is a triple data, and the triple data is a mapping relationship between a question, an excellent answer, and a bad answer;

[0017] An input module, used for inputting the problem set into the student big model through a preset adversarial model to obtain an output result;

[0018] A scoring model is used to output the output result to a preset scoring model for scoring to obtain a scoring value;

[0019] A tutoring module is used to call a preset teacher model to train the student model according to a preset high-quality data set to obtain training results and optimization solutions;

[0020] The training module is used to feed back the scoring score, the training result and the optimization plan to the student big model to be optimized, so that it can continuously strengthen the learning of the difference between the excellent answers and the bad answers corresponding to the question set, and continuously update the question set to train and score the student big model, until the question-answering ability of the student big model is gradually iterated to the optimal, and the trained student big model is obtained.

[0021] In some embodiments of the present application, the large model optimization device includes:

[0022] A model training module, used for obtaining training sets of the multiple application fields, wherein the training sets include multiple preset triple samples, each preset triple sample includes a mapping relationship between a question, an excellent answer, and a bad answer;

[0023] A preset adversarial model is trained based on the training set, so that the preset adversarial model has the function of generating a preset number of triple samples with preset multiple application fields and preset attack effects.

[0024] In some embodiments of the present application, the model training module is specifically used for:

[0025] Acquire a plurality of problems in the plurality of application fields;

[0026] Obtaining a set of answers to the multiple questions, where each question in the answer set corresponds to multiple answers;

[0027] Based on the preset prior knowledge, determine the best answer and the bad answer among the multiple answers corresponding to each question;

[0028] Based on each question, the best answer and the bad answer corresponding to each question, a triplet sample is constructed to obtain the training set.

[0029] In some embodiments of the present application, the model training module is specifically used for:

[0030] Acquire a plurality of problems in the plurality of application fields;

[0031] Obtaining a set of answers to the multiple questions, where each question in the answer set corresponds to multiple answers;

[0032] Obtain the scores of the preset experts for multiple answers corresponding to each question;

[0033] Determine the best answer and the bad answer corresponding to each question according to the score;

[0034] Based on each question, the best answer and the bad answer corresponding to each question, a triplet sample is constructed to obtain the training set.

[0035] In some embodiments of the present application, the model training module is also used to:

[0036] Get the value data of the preset data volume;

[0037] Get the preset amount of Anthropic data;

[0038] Selecting a first preset proportion of value data and a first preset proportion of Anthropic data to train an initial scoring model to obtain the scoring model;

[0039] Selecting a second preset proportion of value data and a second preset proportion of anthropic data to train an initial teacher model to obtain the teacher model;

[0040] The sum of the first preset ratio and the second preset ratio is 1.

[0041] In some embodiments of the present application, the model training module is specifically used for:

[0042] Get the preset amount of initial Anthropic data;

[0043] The initial Anthropic data having value errors are optimized to obtain correct Anthropic data.

[0044] In a third aspect, an embodiment of the present application provides a computing device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the large model optimization method described in the first aspect when executing the computer program.

[0045] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which includes instructions, which, when executed on a computer, enables the computer to execute the large model optimization method as described in the first aspect.

[0046] In a fifth aspect, an embodiment of the present application provides a chip, which includes a processor coupled to a transceiver of a terminal device, and is used to execute the technical solution provided in the first aspect of the embodiment of the present application.

[0047] In a sixth aspect, an embodiment of the present application provides a chip system, the chip system comprising:

[0048] A communication interface for inputting and / or outputting information;

[0049] A processor is used to execute a computer executable program so that a device equipped with the chip system executes the large model optimization method as described in any one of the first aspects.

[0050] In a possible design, the chip system further includes a memory, which is used to store program instructions and data required by the terminal. The chip system can be composed of a chip, or can include a chip and other discrete devices.

[0051] In the seventh aspect, an embodiment of the present application provides a computer program product comprising instructions, wherein the computer program product includes program instructions, and when the program instructions are executed on a computer or a processor, the computer or the processor executes the large model optimization method as described in any one of the first aspects.

[0052] Compared with the prior art, in the embodiment of the present application, after determining the application field of the student big model to be optimized, the preset adversarial model is called to generate a question set for adversarial training of the student big model according to the application field, and then the preset scoring model is input for scoring. Since each question in the question set is a triple data, the triple data is a mapping relationship between questions, excellent answers and poor answers. Based on the scoring results of the scoring model, high-quality questions corresponding to the results with high scoring values ​​in the question set and optimization plans corresponding to the results with low scoring values ​​can be determined. The big model after training the student big model according to the high-quality data set and the first optimization plan has higher accuracy and better capabilities in specific application fields.

[0053] Compared with the prior art, in the embodiment of the present application, an adversarial model is used to generate a question set for adversarial training of the student model, and the question set is input into the student model to be optimized for multiple rounds of question-answer iterative training, and the question-answering results are scored with the trained scoring model, and the scoring results are fed back to the student model to be optimized, so that it can continuously strengthen the learning of the key points and differences between good and bad answers until the question-answering ability is gradually iterated, while the teacher model performs multiple rounds of question-answering training on the student model to be optimized, and scores each round of question-answering results output by the student model to be optimized with a pre-trained scoring model, and then feeds the scoring results back to the student model. The large model is generated so that it can continuously strengthen the learning of the key points and differences between good and bad answers, realize the automation of the large model optimization process, and significantly shorten the overall optimization cycle of the large model. It is suitable for large language model application fields with diversified and complex corpus scenarios. In scenarios where there are algorithm loopholes in certain risk dimensions based on training based on a single adversarial model, by combining the gradual iteration of the scoring model and the teacher model from different angles, the loopholes in some risk dimensions are made up, so that the student large model question and answer ability is gradually iterated to the optimal, which improves the optimization efficiency of the large model. The scoring model is better able to learn excellent answers and bad answers through the question setting of triple data, further improving the performance of the optimized large model. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0055] Figure 1 is a schematic diagram of a scenario of a model optimization system provided in an embodiment of the present application;

[0056] Figure 2 It is a schematic diagram of an embodiment of the large model optimization method provided in the embodiments of the present application;

[0057] Figure 3 It is a schematic diagram of an embodiment of triple data in the embodiment of the present application;

[0058] Figure 4 It is a schematic diagram of an embodiment of the model optimization result provided in the embodiments of the present application;

[0059] Figure 5 It is a schematic diagram of a scenario of a model optimization model provided in an embodiment of the present application;

[0060] Figure 5a A schematic diagram of an evaluation parameter configuration of a large model security evaluation method according to an embodiment of the present application;

[0061] Figure 5b A schematic diagram showing an evaluation result of a large model security evaluation method according to an embodiment of the present application;

[0062] Figure 5c A schematic diagram showing another evaluation result of the large model security evaluation method according to an embodiment of the present application;

[0063] Figure 5d A schematic diagram of a horizontal comparison between different large models of the large model security assessment method of an embodiment of the present application;

[0064] Figure 5e A schematic diagram of a large model and risk category selection for a large model security assessment method according to an embodiment of the present application;

[0065] Figure 6 A schematic diagram of the structure of a large model optimization device according to an embodiment of the present application;

[0066] Figure 7 A schematic diagram of the structure of a mobile phone in an embodiment of the present application;

[0067] Figure 8 A schematic diagram of the structure of a server in an embodiment of the present application.

[0068] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts. DETAILED DESCRIPTION

[0069] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0070] In the following description, the specific embodiments of the present application will be described with reference to the steps and symbols performed by one or more computers, unless otherwise specified. Therefore, these steps and operations will be mentioned several times as being performed by a computer, and the computer execution referred to in the embodiments of the present application includes the operation of a computer processing unit by an electronic signal representing data in a structured form. This operation converts the data or maintains it at a location in the memory system of the computer, which can be reconfigured or otherwise changed in a manner familiar to testers in the field. The data structure maintained by the data is the physical location of the memory, which has specific characteristics defined by the data format. However, the principle of the present application is described with the above text, which does not represent a limitation, and testers in the field will understand that the various steps and operations described below can also be implemented in hardware.

[0071] The terms "module" or "unit" used in the embodiments of the present application can be regarded as software objects executed on the computing system. The different components, modules, engines and services described in the embodiments of the present application can be regarded as implementation objects on the computing system. The devices and methods described in the embodiments of the present application are preferably implemented in software, but can also be implemented in hardware, all within the scope of protection of the present application.

[0072] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0073] A large language model is a language model with a huge number of parameters that is trained based on a large amount of data (usually using large-scale self-supervised learning). Large language models are mainly implemented based on the Transformer machine learning method. The Transformer model is a deep learning model that uses a self-attention mechanism. This mechanism can assign different weights according to the importance of each part of the input data. Typical large model applications include OpenAI's ChatGPT.

[0074] Various methods and software of generative artificial intelligence have been widely used, which has brought about potential risks and hidden dangers. In addition to the leakage of personal information, negative news, false news and speeches of key figures produced by generative artificial intelligence tools are difficult to distinguish between true and false, spread rapidly, and are seriously misleading to public opinion, affecting national security and social stability. They may also become a new means of cybercrime and infringe on the legitimate rights and interests of citizens.

[0075] The general big model is currently widely used in chat dialogue, text editing, artistic creation, coding, mathematical reasoning, bioinformatics and other application scenarios. Although it has created many new business models and is very powerful, after the general big model is launched for users to use, there are mainly algorithmic risks, data risks and application risks in the three types of applications: translation, chat and collaboration.

[0076] Algorithmic risk: refers to the inherent defects of these large models at the bottom layer regardless of the scenario in which they are used. These defects mainly come from the mathematical characteristics of neural networks. When normal users use general large models, the risks that these models may expose are called intrinsic risks. This includes various attributes such as interpretability, value alignment, and cognitive ability. When malicious users maliciously attack general large models, the risks that these models may expose are called external risks. This includes prompt injection, adversarial attacks, model theft, etc. The solution to these algorithmic risks often requires deep algorithm optimization. For example, by injecting confusing instructions and data into the model based on the "prompt injection" method, users can obtain unauthorized permissions when using it, so that the model ignores the preset "translation" and "hello" tasks and gives random answers. For example, adding slight perturbations to the text based on the "adversarial attack" method will cause the model to make mistakes and have low reliability.

[0077] Data risk: refers to the fact that if the database is not handled properly, the general large model may expose defects in the corpus, thereby bringing risks in content security, privacy leakage, and credibility. The solution to these data risks often requires restricting the content and access rights of the database. 1. Due to the complex source of the corpus, it may contain attitudes and prejudices of certain human communities, which can easily cause the models trained based on these corpora to be biased and discriminatory; 2. Due to the complex source of the corpus, it may contain negative content, which can easily cause the models trained based on these corpora to lead users to have mental health problems; 3. Due to the complex source of the corpus, it may contain erroneous content, which can easily cause the models trained based on these corpora to output false / wrong information.

[0078] Application risks: If abused by users, general large models may be used maliciously, thus bringing application risks. Solving these data risks often requires managing user behavior. For example, criminals commit fraud by writing fraud scripts and copywriting. Such large models can enhance the attack capabilities of fraudsters and criminals, which is extremely harmful to society.

[0079] Based on this, the embodiments of the present application provide a large model optimization method, related devices and storage media, which are applied to application scenarios such as chat dialogue, text editing, artistic creation, code writing, mathematical reasoning, bioinformatics, etc., and can perform automated evaluation of large language models for all-round evaluation and perform online optimization, effectively improving the optimization efficiency of large language models.

[0080] See also Figure 1 , Figure 1 A schematic diagram of a scenario of a model optimization system provided in an embodiment of the present application, wherein the model optimization system may include a computing device 100, which is connected via a network, and a large model optimization device is integrated in the computing device 100. In the embodiment of the present application, the computing device 100 may be a terminal device or a server. The computing device 100 may be connected to a user terminal 200 for collecting a training set through a network.

[0081] In the embodiment of the present application, when the computing device 100 is a server, the server can be an independent server, or a server network or server cluster composed of servers, for example, the server described in the embodiment of the present application includes but is not limited to a computer, a network host, a single network server, a plurality of network server sets or a cloud server composed of a plurality of servers. Wherein, the cloud server is composed of a large number of computers or network servers based on cloud computing (Cloud Computing). In the embodiment of the present application, communication between the server and the client can be achieved by any communication mode, including but not limited to, mobile communication based on the third generation partnership project (3rd Generation Partnership Project, 3GPP), long term evolution (Long Term Evolution, LTE), Worldwide Interoperability for Microwave Access (Worldwide Interoperability for Microwave Access, WiMAX), or computer network communication based on TCP / IP protocol suite (TCP / IP Protocol Suite, TCP / IP), user datagram protocol (User Datagram Protocol, UDP) protocol, etc.

[0082] It is understood that when the computing device 100 used in the embodiments of the present application is a terminal device, the terminal device may be a device including both receiving hardware and transmitting hardware, that is, a device having receiving and transmitting hardware capable of performing two-way communication on a two-way communication link. Such terminal devices may include: cellular or other communication devices having a single-line display or a multi-line display or a cellular or other communication device without a multi-line display. The specific computing device 100 may be a desktop terminal or a mobile terminal, and the computing device 100 may be one of a mobile phone, a tablet computer, a laptop computer, etc.

[0083] The terminal device involved in the embodiments of the present application may also refer to a device that provides voice and / or data connectivity to a user, a handheld device with a wireless connection function, or other processing devices connected to a wireless modem. For example, a mobile phone (or "cellular" phone) and a computer with a mobile terminal, for example, a portable, pocket-sized, handheld, computer-built-in or vehicle-mounted mobile device, which exchanges voice and / or data with a wireless access network. For example, a personal communication service (English full name: Personal Communication Service, English abbreviation: PCS) phone, a cordless phone, a session initiation protocol (SIP) phone, a wireless local loop (Wireless Local Loop, English abbreviation: WLL) station, a personal digital assistant (English full name: Personal Digital Assistant, English abbreviation: PDA) and other devices.

[0084] Those skilled in the art will understand that Figure 1 The application environment shown in the figure is only one application scenario of the present application solution and does not constitute a limitation on the application scenario of the present application solution. Other application environments may also include Figure 1 More or fewer computing devices, or computing device network connection relationships, such as Figure 1 Only one computing device is shown. It can be understood that the model optimization system may also include one or more other computing devices, or / and one or more other computing devices connected to the computing device 100 through a network, which is not specifically limited here.

[0085] In addition, if Figure 1 As shown, the model optimization system may also include a memory 300 for storing data, such as storing triple data, high-quality data sets, model optimization solutions, etc.

[0086] It should be noted that Figure 1 The scenario diagram of the model optimization system shown is merely an example. The model optimization system and scenario described in the embodiment of the present application are intended to more clearly illustrate the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided in the embodiment of the present application. A person of ordinary skill in the art can appreciate that with the evolution of the model optimization system and the emergence of new business scenarios, the technical solution provided in the embodiment of the present application is equally applicable to similar technical problems.

[0087] The solution provided in the embodiments of the present application involves technologies such as artificial intelligence (AI), computer vision (CV), and machine learning (ML), which are specifically described by the following embodiments:

[0088] Among them, AI is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0089] AI technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0090] CV is a science that studies how to make machines "see". To put it more specifically, it refers to machine vision that uses cameras and computers to replace human eyes to identify, track and measure targets, and further performs graphic processing to make computer processing into images that are more suitable for human eye observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multi-dimensional data. Computer vision technology usually includes adversarial disturbance generation, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and map construction, and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.

[0091] The following describes it in detail with reference to specific embodiments.

[0092] In this embodiment, the description will be made from the perspective of a large model optimization device, which can be specifically integrated in the computing device 100.

[0093] The present application provides a large model optimization method, which includes: determining the application field of a student large model to be optimized; calling a preset adversarial model to generate a question set for adversarial training of the student large model according to the application field, the question set including at least one preset question generated based on text adversarial technology, each question in the question set is a triple data, and the triple data is a mapping relationship between a question, an excellent answer and a poor answer; inputting the question set to the student large model through the preset adversarial model to obtain an output result; outputting the output result to a preset scoring model for scoring to obtain a scoring value; calling a preset teacher model to train the student large model according to a preset high-quality data set to obtain a training result and an optimization plan; feeding back the scoring value, the training result and the optimization plan to the student large model to be optimized, so that it continuously reinforces the learning of the difference between the excellent answer and the poor answer corresponding to the question set, and continuously updates the question set to train and score the student large model until the question-answering ability of the student large model is gradually iterated to the optimal, and a trained student large model is obtained.

[0094] See also Figure 2 , is a flow chart of an embodiment of a large model optimization method in an embodiment of the present application, and the large model optimization method includes the following steps 201 to 207:

[0095] 201. Determine the application areas of the student model to be optimized.

[0096] Large models refer to ultra-large-scale deep learning models, machine learning models with a large number of parameters and complex structures, which can be applied to process large-scale data and complex problems. Traditional machine learning models, such as logistic regression, decision trees, and naive Bayes, are small in scale and can only process small amounts of data, while deep learning models can contain millions of parameters and process massive amounts of data. Ultra-large-scale deep learning models can even reach tens of billions of parameters and require supercomputers for training.

[0097] In the embodiment of the present application, the student big model to be optimized is a big model to be optimized that is applied to students, and the application field may be a learning field, for example, it may be an application field such as mental health, laws and regulations, sensitivity, etc.

[0098] 202. Call the preset adversarial model to generate a problem set for adversarial training of the student large model according to the application field.

[0099] The question set includes at least one preset question generated based on text adversarial technology. Each question in the question set is a triple data, which is a mapping relationship between the question, the excellent answer and the bad answer. The questions in the question set are all questions in this application field.

[0100] 203. Input the problem set into the student model through the preset adversarial model to obtain the output result.

[0101] In an embodiment of the present application, a question set is input into the student model through a preset adversarial model to obtain an output result. Specifically, the questions in the question set are input into the student model through the preset adversarial model to obtain an output result of each question (i.e., the output answer to the question). The output result of each question in the question set constitutes the output result.

[0102] 204. Output the output result to a preset scoring model for scoring to obtain a scoring value.

[0103] Among them, the scoring model can be a common reinforcement learning reward constraint (Reward Models). Reward Models are generally classification or recognition models trained by information and labeled data fed back by humans or AI red team models. Its input can be only the output result of the AI ​​model, or it can also add human feedback signals here, and the output is reward signals of different strengths for reinforcement learning training.

[0104] 205. Call the preset teacher model to train the student large model according to the preset high-quality data set to obtain the training results and optimization plan.

[0105] Among them, high-quality question sets are question sets that match questions with excellent answers.

[0106] The teacher model can be a reinforcement learning model based on human feedback (RLHF) or a reinforcement learning model based on AI feedback (RLAIF).

[0107] Among them, RLHF refers to the use of a large amount of manually annotated data to train the AI ​​system model in the reinforcement learning stage. The preset reward model generates reward signals (Reward Signals) of different strengths according to the output results of the AI ​​model and the annotated data to guide the AI ​​model to converge in the expected direction, and a safer AI system model is obtained after the training. The effect of the model after training under this technology depends largely on the scale and quality of the manually annotated data. RLHF helps to solve the problems of hallucinations and harmful outputs.

[0108] RLAIF is a variant of RLHF. The feedback information source of the reward model is partially or completely changed from human feedback to agent models (Agent Models) automatically provided. The agent model is a model that meets safety standards and is aligned after pre-training. It acts as a "supervisor" to provide feedback signals for the reinforcement training of the new model. Among them, fine-tuned models that can provide AI feedback signals include OpenAI's InstructGPT, DeepMind's Automatically Red-LM+Red Clf, and Anthropic's Constitutional-AI, etc.

[0109] It should be noted that in the embodiments of the present application, the scoring model and the teacher model are decoupled, that is, the scoring model can be used as the input for teacher model training, or it can simply evaluate the question and answer pairs. Taking the scoring model as the reward model and the teacher model as the RLHF training framework as an example, the reward model can be used as the input for RLHF training framework training, or it can simply evaluate the question and answer pairs.

[0110] 206. Feedback the scoring scores, training results and optimization plans to the student big model to be optimized, so that it can continuously strengthen its learning to distinguish between excellent answers and bad answers corresponding to the question set, and continuously update the question set to train and score the student big model, until the question-answering ability of the student big model is gradually iterated to the optimal level, and the trained student big model is obtained.

[0111] Compared with the prior art, in the embodiment of the present application, an adversarial model is used to generate a question set for adversarial training of the student model, and the question set is input into the student model to be optimized for multiple rounds of question-answer iterative training, and the question-answering results are scored with the trained scoring model, and the scoring results are fed back to the student model to be optimized, so that it can continuously strengthen the learning of the key points and differences between good and bad answers until the question-answering ability is gradually iterated, while the teacher model performs multiple rounds of question-answering training on the student model to be optimized, and scores each round of question-answering results output by the student model to be optimized with a pre-trained scoring model, and then feeds the scoring results back to the student model. The large model is generated so that it can continuously strengthen the learning of the key points and differences between good and bad answers, realize the automation of the large model optimization process, and significantly shorten the overall optimization cycle of the large model. It is suitable for large language model application fields with diversified and complex corpus scenarios. In scenarios where there are algorithm loopholes in certain risk dimensions based on training based on a single adversarial model, by combining the gradual iteration of the scoring model and the teacher model from different angles, the loopholes in some risk dimensions are made up, so that the student large model question and answer ability is gradually iterated to the optimal, which improves the optimization efficiency of the large model. The scoring model is better able to learn excellent answers and bad answers through the question setting of triple data, further improving the performance of the optimized large model.

[0112] In some embodiments of the present application, the adversarial model can be trained through some training sets of triple samples to generate a massive, targeted, and attack-effective triple problem set. Specifically, the method also includes: obtaining training sets for multiple application fields, the training sets include multiple preset triple samples, each preset triple sample includes a mapping relationship between questions, excellent answers, and bad answers; based on the training set, the preset adversarial model is trained so that the preset adversarial model has the function of generating a preset number of triple samples, preset multiple application fields, and preset attack effects, such as Figure 3 Shown is an example graph of triplet data.

[0113] In an embodiment of the present application, a preset adversarial model is trained based on a training set of triple samples, so that it has the ability to generate massive, targeted, and attack-effective triple problem sets, which are used as input for the student model to be optimized to train the large student model.

[0114] In the embodiment of the present application, there are multiple ways to obtain the training sets of the multiple application fields, which are described below with examples:

[0115] (1) Obtaining training sets based on prior knowledge

[0116] At this time, the step of obtaining training sets for the multiple application fields may include: obtaining multiple questions for the multiple application fields; obtaining a set of answers to the multiple questions, in which each question corresponds to multiple answers; based on preset prior knowledge, determining the best answer and the bad answer among the multiple answers corresponding to each question; constructing a triplet sample based on each question, the best answer and the bad answer corresponding to each question, to obtain the training set.

[0117] (2) Obtaining training sets based on expert confirmation

[0118] At this time, obtaining the training sets for the multiple application fields may include: obtaining multiple questions for the multiple application fields; obtaining a set of answers to the multiple questions, in which each question corresponds to multiple answers; obtaining scores from preset experts for the multiple answers corresponding to each question; determining the best answer and the bad answer corresponding to each question based on the scores; and constructing a triplet sample based on each question, the best answer and the bad answer corresponding to each question to obtain the training set.

[0119] In an embodiment of the present application, the questions corresponding to the output results with low scoring values ​​of the scoring model can also be optimized to a high-quality set of questions that meet the requirements, and then added to a preset high-quality data set for subsequent student large model training. Specifically, the method in an embodiment of the present application can also include: determining a first output result with a scoring value not lower than a preset value, and adding the question corresponding to the first output result to a preset high-quality question set. Determine a second output result with a scoring value lower than a preset value, and input the question corresponding to the second output result into a preset teacher model to obtain a problem optimization plan corresponding to the second output result, and according to the problem optimization plan, optimize the excellent answers corresponding to the second output result in the problem set to obtain an optimized problem set; update the high-quality data set according to the optimized problem set.

[0120] Furthermore, in an embodiment of the present application, the data for training the scoring model and the teacher model include value data and Anthropic data, wherein the value data is pre-collected data that conforms to good values, and the Anthropic data is data generated by an Anthropic model (such as Claude). The value data and the Anthropic data are parallel and there is no inclusion relationship.

[0121] In the embodiment of the present application, the scoring model and the teacher model can be trained by obtaining a preset amount of value data and anthropic data. Specifically, the following steps can be included: obtaining a preset amount of value data; obtaining a preset amount of anthropic data; selecting a first preset ratio of value data and a first preset ratio of anthropic data to train the initial scoring model, and obtaining the scoring model; selecting a second preset ratio of value data and a second preset ratio of anthropic data to train the initial teacher model, and obtaining the teacher model; wherein the sum of the first preset ratio and the second preset ratio is 1.

[0122] Since there are some erroneous data in Anthropic data, such as the reversal of correct values ​​and incorrect values, and there are a large number of neutral data that cannot be well distinguished, the initially collected Anthropic data need to be optimized to obtain the final Anthropic data that meets the requirements. Specifically, obtaining a preset amount of Anthropic data includes: obtaining a preset amount of initial Anthropic data; optimizing the data with erroneous values ​​in the initial Anthropic data to obtain the correct Anthropic data.

[0123] For example, in the Anthropic dataset, the training set is 161k and the test set is 8.55k, which is consistent with good values. The value dataset is expected to have 5k data. The value dataset is based on public documents and generates controversial questions, answers that are consistent with good values, and answers that are inconsistent with good values ​​through prompt design.

[0124] Construct data samples: Based on public documents, generate controversial questions through prompt design, as well as answers that are consistent with good values ​​and answers that are inconsistent with good values.

[0125] For the scoring model: it can include the Anthropic dataset and the values ​​dataset. For example, the Anthropic dataset has 161k training sets and 8.55k test sets. The data in the Anthropic dataset all conform to good values, while the values ​​dataset is expected to have 5k data. The values ​​dataset can be based on public documents and can generate controversial questions, answers that conform to good values, and answers that do not conform to good values ​​through prompt design.

[0126] Taking the Reward Model as an example, the Reward Model training uses the deepspeed framework, and the RewardModel scoring accuracy is:

[0127] $$\frac{|\{c>r|c,r\in S\}|}{|S|}$$

[0128] Among them, S represents the verification set, c represents chosen, and r represents rejected.

[0129] The training set and test set of the Reward Model can be divided in a ratio of 9:1.

[0130] After testing and verification by the inventor, the Reward Model does not work well when the model scale is small and the amount of data is small. As the amount of data increases, the acc of the model increases.

[0131] For the teacher model: the RLHF training framework can be used. The training in the RLHF stage only requires question (instruction) data, not answer data. Therefore, in addition to reusing the questions in the Anthropic dataset and the values ​​dataset, you can also supplement the questions pre-generated by the T-800 model to achieve better generalization.

[0132] The teacher model training uses the deepspeed framework. Currently, the deepspeed framework only perfectly supports OPT in the RLHF stage. Since Deepspeed only supports the same type of actor model and critic model (tokenizer problem), the deepspeed code needs to be improved to support different types of models for RLHF.

[0133] The evaluation of the teacher model includes qualitative evaluation and quantitative evaluation, as follows:

[0134] Qualitative evaluation: The evaluation of the RLHF stage is the same as that of the SFT stage, and requires humans or ChatGPT as evaluators to conduct the evaluation.

[0135] Quantitative evaluation: Use the Reward Model to score the model output before and after RLHF and compare the scores.

[0136] In the embodiment of this application, the value data set is divided into nine categories, including sensitive, legal, mental health, public safety, ethics, privacy protection, role simulation, value orientation, medical health, etc. At present, the first phase of the project mainly focuses on three categories: sensitive, legal, and mental health. In terms of the nature of the data set, there are two types in the embodiment of this application:

[0137] First, authoritative basic data sets. Used to train language models to have basic and good value-answering capabilities. Including: sensitive categories. Legal and regulatory categories, covering my country's constitution, laws, administrative regulations, supervision regulations, judicial interpretations, and policies and regulations in the field of cybersecurity and informatization. Mental health categories, covering national authoritative psychology textbooks, etc.

[0138] Second, high-quality data set tuning. It is used to train large models to produce answers that meet good value requirements in terms of sensitive and controversial issues. The main approach is to set up sensitive and controversial topic question and answer pairs for the three main areas of sensitivity, laws and regulations, and mental health. Among them: In the sensitive field, according to 17 dimensions, potentially controversial questions and answers are constructed. Among them, authoritative experts from the Academy of Social Sciences are invited to take the lead in constructing the answer content. In the field of laws and regulations, according to six dimensions such as civil law, commercial law, international law, criminal law, criminal procedure law, and counter-espionage law, potentially controversial questions and answers are constructed. In the field of mental health, according to eight dimensions such as abnormal psychology, moral psychology, healthy psychology, social psychology, career development, psychological counseling, religious psychology, and developmental psychology, potentially controversial questions and answers are constructed.

[0139] 2. Develop a teacher model (Reward model) that conforms to good values

[0140] First, based on the authoritative basic data sets and high-quality tuning data sets that have been constructed, a "Q-CA-RA triplet" is formed, namely Question (Q), Chosen Answer (CA), and Rejected Answer (RA). Second, based on the "Q-CA-RA triplet", through supervised training and other technologies, a good value language model teacher model (Reward model) is developed and constructed. Third, relying on the teacher model (Reward model), through reinforcement learning and other technologies, the language model product is trained, corrected, tuned, and improved to achieve the goal of making its output content conform to good values.

[0141] At present, the application test shows that the data set of this application and the teacher model developed based on the data set have achieved good tuning effects after acting on the existing language model. Taking the Vicuna model as an example, this application can currently achieve two results:

[0142] First, for sensitive questions, the trained model no longer adopts the strategy of "answering positive content first, then answering negative content", but only answers positive content.

[0143] Second, for sensitive issues, the answers are in line with my country's authoritative and correct multi-dimensional perspectives.

[0144] At present, the related work of this application is still advancing and iterating rapidly. The data set is growing every day, and the teacher model is also constantly adjusting strategies and methods based on feedback during training to continuously optimize the coaching effectiveness.

[0145] In the embodiments of the present application, the triplet samples can be divided into a test indicator set and an attack instruction set. The corresponding test indicator set and attack instruction set are different for different application fields of the student big model. Among them, the test indicator set includes evaluation indicators corresponding to multiple risk categories. In order to improve the reliability of the evaluation of the big model to be tested and increase the diversity of risk categories, in some embodiments, the evaluation indicators corresponding to the risk categories in the test indicator set include malicious use, harmful language, violation of national security, erroneous knowledge and information, violation of good values, prejudice and discrimination, information leakage, and at least one of deployment risks. There are different evaluation indicators corresponding to the big models to be tested in different application fields. Of course, the risk categories can also include other categories, which are not limited here.

[0146] Among them, as shown in Table 1 below, harmful language includes: content that encourages or suggests self-harm, pushes or suggests users to perform immoral or illegal acts, pornographic language, violent, offensive content, harassing, derogatory and hateful content, instructions on how to find illegal content, inflammatory speech, personal attacks or malicious slander, damage to social morals or values, and at least one of impolite language.

[0147] Prejudice and discrimination include at least one of: social stereotypes, individual or group prejudice, and exclusionary norms.

[0148] Information leakage includes: leakage of private information, leakage of privacy by correctly inferring private information, and at least one of leakage or inference of sensitive information.

[0149] Malicious uses include: spreading false information, fraud / scam, spreading malicious websites, promoting illegal products, and promoting at least one of extreme speech.

[0150] Violations of national security include at least one of the following: damaging the national image, damaging national interests, inciting ethnic hatred or regional discrimination, undermining social stability, and supporting terrorist activities.

[0151] Violations of good values ​​include at least one of: individualism, materialism, lack of morality, and spiritual emptiness.

[0152] Table 1 Risk categories

[0153]

[0154]

[0155] As shown in Table 2 below, the attack instruction set includes attack instructions corresponding to multiple attack categories. In order to improve the reliability of the evaluation of the large model to be tested and increase the diversity of attack categories, in some embodiments, the attack instructions corresponding to the attack categories in the attack instruction set include at least one of target hijacking, unsafe inquiry, injection attack, developer mode, negative inducement, role playing, and prompt leakage. There are different attack instructions corresponding to the large models to be tested in different application fields. Of course, the risk category can also include other categories, which are not limited here.

[0156] Table 2 Attack instructions

[0157]

[0158] The test indicator set and attack instruction set may be determined by:

[0159] Method 1: The large model security assessment device randomly determines the test indicator set and attack instruction set according to the application field of the large model to be tested.

[0160] Method 2: The large model security assessment device determines the test indicator set and attack instruction set based on the assessment parameters of the large model to be tested and the application field of the large model to be tested configured through the configuration interface.

[0161] Method three, the large model security assessment device receives a login request, and logs in to the test account in response to the login request; determines the default assessment parameters that match the test account, and determines the test indicator set and attack instruction set based on the default assessment parameters and the application field of the large model to be tested.

[0162] Of course, the large model security assessment device can also determine the test indicator set and attack instruction set in other ways, which are not limited here.

[0163] like Figure 4 As shown, it is a schematic diagram of an embodiment scenario of model optimization in the embodiment of the present application. In the process of model optimization, data collection is performed first, then data construction is performed, and then data demonstration is performed to construct triple data. Based on the constructed triple data, the training set and the test set are divided, and the scoring model (which can be a Reward Mod scoring model) is trained to improve the scoring model's ability to distinguish good and bad answers, and use the correct and accurate scoring model capabilities to build a framework for the next round. After several rounds of scoring model training, the accuracy rate on the test set reaches 100%, that is, the scores for good answers are all greater than the scores for bad answers.

[0164] like Figure 5 As shown, it is a schematic diagram of another embodiment scenario of model optimization in the embodiment of the present application, wherein the preset adversarial model for generating the problem set is equivalent to Figure 4The teacher who sets the test paper in the middle is the red team model, the student in the middle is equivalent to the large student model, the teacher who grades the test paper on the right is equivalent to the scoring model, and the tutor below is equivalent to the preset teacher model.

[0165] In the embodiment of the present application, the student large model can also be evaluated. The evaluation configuration interface may include a variety of configuration controls and a variety of parameter selection areas. The large model security evaluation device may receive the parameters of "automatically generate evaluation" or "fixed data set evaluation" selected by the user in the parameter selection area of ​​"select evaluation method", and set at least one corresponding risk category and at least one attack instruction for each evaluation method, such as receiving multi-dimensional parameters such as risk categories and attack instructions selected by the user in the parameter selection area of ​​"select evaluation dimension". And configure the number of prompt injections for adversarial problems corresponding to each risk category for each evaluation method, that is, "prompt data for each type of problem". Of course, the parameters that need to be configured may also include other parameters, which are not limited here, such as Figure 5a In the configuration interface, you can click the "Next" switch control to switch to the next page to configure other parameters. After selecting the parameters of each dimension, the large model security assessment device can complete the configuration of the parameters in response to the configuration control (such as the confirmation control or the completion control, etc.). This enables users to flexibly and conveniently configure the parameters such as the assessment method, risk category, attack instructions, and the number of prompt injections for adversarial problems of the large model to be tested according to actual assessment needs through the configuration interface.

[0166] In the actual configuration scenario, the configuration interface is shown as follows: Figure 5a As shown, in this interface, you can set the preset evaluation method, automatic generation evaluation or fixed data set evaluation, set at least one corresponding risk category and at least one attack instruction for the selected evaluation method, and configure prompt data for each type of question for the selected evaluation method.

[0167] After completing the configuration of the parameters, the large model security assessment device can accurately determine the matching test indicator set and attack instruction set based on the parameters and the application field of the large model to be tested, so as to accurately evaluate the large model to be tested.

[0168] The display format and specific content of the evaluation results of the large model can be flexibly set according to actual needs and are not limited here. The evaluation results can display the test values ​​of the large model under test for multiple different risk categories under different attack instructions, and can also display the evaluation comparison between the large model under test and similar large models, and can also display the evaluation indicators under each risk category used for the security evaluation of the large model under test and the proportion of each attack instruction, etc.

[0169] For example, Figure 5bAs shown, the evaluation results can include the proportion of evaluation indicators under each risk category, such as malicious use, harmful language, violation of national security, incorrect knowledge and information, prejudice and discrimination, information leakage, etc. It can also include the proportion of each attack instruction, such as target hijacking, unsafe inquiry, injection attack, developer mode, negative inducement, role playing, prompt leakage, etc., as well as the test values ​​corresponding to various risk categories under different attack instructions for the large model to be tested. Among them, Figure 5b A to K corresponding to the evaluation dimension represent the evaluation indicators under A to K different risk categories, such as the major categories or subcategories of risk categories. In the actual evaluation application scenario, the effect diagram showing the evaluation results is as follows: Figure 5b shown.

[0170] For example, Figure 5c As shown, the evaluation results can include the proportion of evaluation indicators under each risk category, such as harmful language, violation of national security, prejudice and discrimination, and the proportion of each attack instruction, such as unsafe inquiry, injection attack, role-playing, target hijacking, negative inducement, etc., as well as the test values ​​corresponding to various risk categories under different attack instructions for the large model to be tested. Figure 5c A to K corresponding to the evaluation dimension represent the evaluation indicators under A to K different risk categories, such as the major categories or subcategories of risk categories. In the actual evaluation application scenario, the effect diagram showing the evaluation results is as follows: Figure 5c shown.

[0171] like Figure 5d As shown, the horizontal comparison results may include horizontal comparison results between a variety of different large models, and the specific model types and quantities of similar large models are not limited here. Taking model A in 5d as the large model to be tested, and model B and model C as similar large models as an example, the displayed horizontal comparison results include the evaluation results of the large model to be tested for at least one risk category and the evaluation results of similar large models for at least one risk category. The horizontal comparison results show the risk categories of vulnerability risks between the large model to be tested and similar large models, wherein the higher the safety value of the large model, the smaller the probability of vulnerability risks, and conversely, the lower the safety value of the large model, the greater the probability of vulnerability risks. The horizontal comparison results can also be used to indicate the advantages and disadvantages between the large model to be tested and similar large models, as well as the comparison of advantages and disadvantages between the large model to be tested and similar large models, and other related information.

[0172] In the actual evaluation application scenario, the effect diagram showing the horizontal comparison results is as follows Figure 5d shown.

[0173] Among them, the horizontal comparison results show the first risk category of vulnerability risks between the large model to be tested and similar large models. The horizontal comparison results indicate the advantages and disadvantages between the large model to be tested and similar large models, as well as the comparison of advantages and disadvantages between the large model to be tested and similar large models.

[0174] Specifically, Figure 5e As shown, the large model safety assessment device can receive a first input from a user for a model comparison icon in a visual display interface, such as the shape, color, display size, and position of the model comparison icon. The first input can be a click selection or a keystroke, etc., which are not limited here. Then, the large model safety assessment device can respond to the first input, select at least one target risk category (such as risk A, risk D, and risk E), and select at least one similar large model to be compared (such as model A, model B, and model E), wherein the different target risk categories can correspond to the assessment indicators corresponding to different risk categories.

[0175] It can be understood that the selection of the large model and the selection of the risk category can be displayed on the same page of the visual display interface, or on different pages. When the selection of the large model and the selection of the risk category are displayed on different pages of the visual display interface, the large model and the risk category can be selected by switching pages.

[0176] At this time, the large model safety assessment device can display the horizontal comparison results of the large model to be tested and at least one large model of the same type to be compared for at least one risk category according to the target risk category. The target risk category and the large model of the same type are determined by user selection so that the required horizontal comparison results can be displayed, which improves the convenience and flexibility of displaying the horizontal comparison results.

[0177] In a nutshell, in the embodiments of the present application, the security of the large language model is improved by the collaboration of the training framework based on the prophet evaluation model, the adversarial model, and the coaching model. Among them, the adversarial model generates multiple rounds of adversarial questions and inputs them into the large model under test for multiple rounds of question-answer iterative training, and scores the question-answer results based on the trained prophet evaluation model, and then feeds back the scoring results to the large model under test, so that it can continuously strengthen its learning of the key points and differences between good and bad answers, until the question-answering ability is gradually iterated to the optimal. The coaching model conducts multiple rounds of question-answer training on the large model under test, and scores each round of question-answering results output by the large model under test with a pre-trained scoring model, and then feeds back the scoring results to the large model, so that it can continuously strengthen its learning of the key points and differences between good and bad answers, until the question-answering ability is gradually iterated to the optimal.

[0178] In addition to a customized training framework, the ideal model effect of the coaching model also requires a solid data foundation, that is, by building a massive, objective, comprehensive data set that has been verified by dozens of experts in the field of values ​​to ensure that the input data is correct, high-quality, and diverse in fields, and supports continuous updates and supplements in the future.

[0179] Therefore, on the one hand, after optimizing the large model based on this solution, the large model can reduce the output of false information, the generation of inappropriate content, the output of biased and discriminatory results, and avoid the occurrence of inappropriate content generated by attackers bypassing the security protection mechanism of the large model through prompt injection or adversarial attacks, thereby improving the security of the large model. On the other hand, since the Reward Model is an open source large language model based on reinforcement learning trained on a good value data set, it can achieve the effect of enhancing values ​​through continuous feedback.

[0180] On the other hand, this solution provides a full-process intelligent solution for automated evaluation and online optimization of large language models for all-round evaluation, which is suitable for scenarios with diversified application fields and complex corpus of large language models. Specifically, after the algorithm vulnerabilities of the large language model under certain risk dimensions are detected based on the continuous attack of the adversarial model, the large language model will be optimized based on the teacher model, especially when the large language model is a primary model or a student model with poor training effect. The effect is better. There is no need for the owner of the large model to optimize the model offline based on the test results given by the platform, which significantly shortens the overall optimization cycle. After each round of optimization, the model security evaluation of the next process can be directly entered, and the cycle continues until the student model converges. It can be seen that the whole process is intelligent and no human participation is required, so as to efficiently obtain an excellent large language model, which is especially suitable for massive industrial-level needs and also reduces the technical threshold for training large language models.

[0181] In order to facilitate better implementation of the large model optimization method provided in the embodiment of the present application, the embodiment of the present application also provides a device based on the large model optimization method. The meanings of the terms are the same as those in the large model optimization method, and the specific implementation details can refer to the description in the embodiment of the large model optimization method.

[0182] The large model optimization device in the embodiment of the present application has the function of implementing the large model optimization method provided in the above embodiment. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions, and the modules can be software and / or hardware.

[0183] See also Figure 6 , Figure 6The schematic diagram of the structure of the large model optimization device provided in the embodiment of the present application, the large model optimization device can be applied to a computing device in a scenario where model optimization is required. Specifically, the large model optimization device 600 may include a determination module 601 and a generation module 602, an input module 603, a scoring module 604, a coaching module 605 and a training module 606, wherein:

[0184] A determination module 601 is used to determine the application field of the student macro model to be optimized;

[0185] A generation module 602 is used to call a preset adversarial model to generate a question set for adversarial training of the student model according to the application field, wherein the question set includes at least one preset question generated based on text adversarial technology, and each question in the question set is a triple data, and the triple data is a mapping relationship between a question, an excellent answer, and a bad answer;

[0186] An input module 603 is used to input the problem set into the student model through a preset adversarial model to obtain an output result;

[0187] Scoring model 604, used to output the output result to a preset scoring model for scoring to obtain a scoring value;

[0188] The coaching module 605 is used to call a preset teacher model to train the student model according to a preset high-quality data set to obtain training results and optimization solutions;

[0189] The training module 606 is used to feed back the scoring score, the training results and the optimization plan to the student big model to be optimized, so that it can continuously strengthen the learning of the difference between the excellent answers and the bad answers corresponding to the question set, and continuously update the question set to train and score the student big model, until the question-answering ability of the student big model is gradually iterated to the optimal, and the trained student big model is obtained.

[0190] In some embodiments of the present application, the large model optimization device includes:

[0191] A model training module is used to obtain a training set for the multiple application fields, the training set includes multiple preset triple samples, each preset triple sample includes a mapping relationship between questions, excellent answers and bad answers; based on the training set, a preset adversarial model is trained so that the preset adversarial model has the function of generating a preset number of triple samples with preset multiple application fields and preset attack effects.

[0192] In some embodiments of the present application, the model training module is specifically used for:

[0193] Acquire a plurality of problems in the plurality of application fields;

[0194] Obtaining a set of answers to the multiple questions, where each question in the answer set corresponds to multiple answers;

[0195] Based on the preset prior knowledge, determine the best answer and the bad answer among the multiple answers corresponding to each question;

[0196] Based on each question, the best answer and the bad answer corresponding to each question, a triplet sample is constructed to obtain the training set.

[0197] In some embodiments of the present application, the model training module is specifically used for:

[0198] Acquire a plurality of problems in the plurality of application fields;

[0199] Obtaining a set of answers to the multiple questions, where each question in the answer set corresponds to multiple answers;

[0200] Obtain the scores of the preset experts for multiple answers corresponding to each question;

[0201] Determine the best answer and the bad answer corresponding to each question according to the score;

[0202] Based on each question, the best answer and the bad answer corresponding to each question, a triplet sample is constructed to obtain the training set.

[0203] In some embodiments of the present application, the model training module is also used to:

[0204] Get the value data of the preset data volume;

[0205] Get the preset amount of Anthropic data;

[0206] Selecting a first preset proportion of value data and a first preset proportion of Anthropic data to train an initial scoring model to obtain the scoring model;

[0207] Selecting a second preset proportion of value data and a second preset proportion of anthropic data to train an initial teacher model to obtain the teacher model;

[0208] The sum of the first preset ratio and the second preset ratio is 1.

[0209] In some embodiments of the present application, the model training module is specifically used for:

[0210] Get the preset amount of initial Anthropic data;

[0211] The initial Anthropic data having value errors are optimized to obtain correct Anthropic data.

[0212] Compared with the prior art, in the embodiment of the present application, an adversarial model is used to generate a question set for adversarial training of the student model, and the question set is input into the student model to be optimized for multiple rounds of question-answer iterative training, and the question-answering results are scored with the trained scoring model, and the scoring results are fed back to the student model to be optimized, so that it can continuously strengthen the learning of the key points and differences between good and bad answers until the question-answering ability is gradually iterated, while the teacher model performs multiple rounds of question-answering training on the student model to be optimized, and scores each round of question-answering results output by the student model to be optimized with a pre-trained scoring model, and then feeds the scoring results back to the student model. The large model is generated so that it can continuously strengthen the learning of the key points and differences between good and bad answers, realize the automation of the large model optimization process, and significantly shorten the overall optimization cycle of the large model. It is suitable for large language model application fields with diversified and complex corpus scenarios. In scenarios where there are algorithm loopholes in certain risk dimensions based on training based on a single adversarial model, by combining the gradual iteration of the scoring model and the teacher model from different angles, the loopholes in some risk dimensions are made up, so that the student large model question and answer ability is gradually iterated to the optimal, which improves the optimization efficiency of the large model. The scoring model is better able to learn excellent answers and bad answers through the question setting of triple data, further improving the performance of the optimized large model.

[0213] The large model optimization device in the embodiment of the present application is described above from the perspective of modular functional entities. The large model optimization device in the embodiment of the present application is described below from the perspective of hardware processing.

[0214] When the computing device in the embodiment of the present application is a terminal device, the embodiment of the present application also provides a terminal device, such as Figure 7 For the sake of convenience, only the parts related to the embodiments of the present application are shown. For specific technical details not disclosed, please refer to the method part of the embodiments of the present application. The terminal device can be any terminal device including a mobile phone, a tablet computer, a personal digital assistant (PDA), a point of sales (POS), a car computer, etc., taking the mobile phone as an example:

[0215] Figure 7 FIG. 1 is a block diagram showing a partial structure of a mobile phone related to a terminal device provided in an embodiment of the present application. Figure 7The mobile phone includes: a radio frequency (RF) circuit 1010, a memory 1020, an input unit 1030, a display unit 1040, a sensor 1050, an audio circuit 1060, a wireless fidelity (WiFi) module 1070, a processor 1080, and a power supply 1090. Those skilled in the art will understand that Figure 7 The mobile phone structure shown in the figure does not constitute a limitation on the mobile phone, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0216] Combine the following Figure 7 A detailed introduction to the various components of the mobile phone:

[0217] The RF circuit 1010 can be used for receiving and sending signals during information transmission or calls. In particular, after receiving the downlink information of the base station, it is sent to the processor 1080 for processing; in addition, the designed uplink data is sent to the base station. Generally, the RF circuit 1010 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (Low Noise Amplifier, LNA), a duplexer, etc. In addition, the RF circuit 1010 can also communicate with the network and other devices through wireless communication. The above wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile communication (Global System of Mobile communication, GSM), General Packet Radio Service (General Packet Radio Service, GPRS), Code Division Multiple Access (Code Division Multiple Access, CDMA), Wideband Code Division Multiple Access (Wideband Code Division Multiple Access, WCDMA), Long Term Evolution (Long Term Evolution, LTE), email, Short Messaging Service (SMS), etc.

[0218] The memory 1020 can be used to store software programs and modules. The processor 1080 executes various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 1020. The memory 1020 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 1020 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0219] The input unit 1030 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the mobile phone. Specifically, the input unit 1030 may include a touch panel 1031 and other input devices 1032. The touch panel 1031, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on the touch panel 1031 or near the touch panel 1031 using any suitable object or accessory such as a finger, stylus, etc.), and drive the corresponding connection device according to a pre-set program. Optionally, the touch panel 1031 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it to the processor 1080, and can receive and execute commands sent by the processor 1080. In addition, the touch panel 1031 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch panel 1031, the input unit 1030 may also include other input devices 1032. Specifically, the other input devices 1032 may include but are not limited to one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, etc.

[0220] The display unit 1040 can be used to display information input by the user or information provided to the user and various menus of the mobile phone. The display unit 1040 may include a display panel 1041. Optionally, the display panel 1041 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. Further, the touch panel 1031 may cover the display panel 1041. When the touch panel 1031 detects a touch operation on or near it, it is transmitted to the processor 1080 to determine the type of touch event. Subsequently, the processor 1080 provides a corresponding visual output on the display panel 1041 according to the type of touch event. Although in Figure 7 In the embodiment, the touch panel 1031 and the display panel 1041 are used as two independent components to realize the input and output functions of the mobile phone, but in some embodiments, the touch panel 1031 and the display panel 1041 can be integrated to realize the input and output functions of the mobile phone.

[0221] The mobile phone may also include at least one sensor 1050, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 1041 according to the brightness of the ambient light, and the proximity sensor may turn off the display panel 1041 and / or the backlight when the mobile phone is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that can be configured in the mobile phone, such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be repeated here.

[0222] The audio circuit 1060, the speaker 1061, and the microphone 1062 can provide an audio interface between the user and the mobile phone. The audio circuit 1060 can transmit the received audio data to the speaker 1061 after converting the received audio data into an electrical signal, which is converted into a sound signal for output; on the other hand, the microphone 1062 converts the collected sound signal into an electrical signal, which is received by the audio circuit 1060 and converted into audio data, and then the audio data is output to the processor 1080 for processing, and then sent to another mobile phone through the RF circuit 1010, or the audio data is output to the memory 1020 for further processing.

[0223] Wi-Fi is a short-range wireless transmission technology. The mobile phone can help users send and receive emails, browse web pages and access streaming media through the Wi-Fi module 1070. It provides users with wireless broadband Internet access. Figure 7 A Wi-Fi module 1070 is shown, but it is understandable that it is not an essential component of the mobile phone and can be omitted as needed without changing the essence of the invention.

[0224] The processor 1080 is the control center of the mobile phone. It uses various interfaces and lines to connect various parts of the entire mobile phone. It executes various functions of the mobile phone and processes data by running or executing software programs and / or modules stored in the memory 1020, and calling data stored in the memory 1020, so as to monitor the mobile phone as a whole. Optionally, the processor 1080 may include one or more processing units; optionally, the processor 1080 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and application programs, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 1080.

[0225] The mobile phone also includes a power supply 1090 (such as a battery) for supplying power to various components. Optionally, the power supply can be logically connected to the processor 1080 through a power management system, thereby managing functions such as charging, discharging, and power consumption management through the power management system.

[0226] Although not shown, the mobile phone may also include a camera, a Bluetooth module, etc., which will not be described in detail here.

[0227] In the embodiment of the present application, the processor 1080 included in the mobile phone also has the function of controlling the execution of the above-mentioned large model optimization method process performed by the large model optimization device.

[0228] The present application also provides a server. Figure 8 , Figure 81 is a schematic diagram of a server structure provided in an embodiment of the present application. The server 1100 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (full name in English: central processing units, abbreviated in English: CPU) 1122 (for example, one or more processors) and memory 1132, one or more storage media 1130 (for example, one or more mass storage devices) storing application programs 1142 or data 1144. Among them, the memory 1132 and the storage medium 1130 can be short-term storage or permanent storage. The program stored in the storage medium 1130 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Furthermore, the central processing unit 1122 can be configured to communicate with the storage medium 1130 to execute a series of instruction operations in the storage medium 1130 on the server 1100.

[0229] The server 1100 may also include one or more power supplies 1126, one or more wired or wireless network interfaces 1150, one or more input and output interfaces 1158, and / or one or more operating systems 1141, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc.

[0230] The steps in the large model optimization method in the above embodiment can be based on the Figure 8 The structure of the server 1100 is shown.

[0231] For example, the CPU 1122 performs the following operations by calling instructions in the memory 1132:

[0232] Determine the application area of ​​the student model to be optimized;

[0233] Calling a preset adversarial model to generate a question set for adversarial training of the student model according to the application field, wherein the question set includes at least one preset question generated based on text adversarial technology, and each question in the question set is a triple data, and the triple data is a mapping relationship between a question, an excellent answer, and a bad answer;

[0234] Input the problem set into the student big model through a preset adversarial model to obtain an output result;

[0235] Outputting the output result to a preset scoring model for scoring to obtain a scoring value;

[0236] Calling a preset teacher model to train the student model according to a preset high-quality data set to obtain training results and optimization solutions;

[0237] The scoring score, the training result and the optimization plan are respectively fed back to the student big model to be optimized, so that it can continuously reinforce the learning of the difference between the excellent answers and the bad answers corresponding to the question set, and continuously update the question set to train and score the student big model, until the question-answering ability of the student big model is gradually iterated to the optimal, and the trained student big model is obtained.

[0238] In one embodiment, the method further comprises:

[0239] Acquire training sets for the multiple application fields, the training sets comprising multiple preset triple samples, each preset triple sample comprising a mapping relationship between a question, an excellent answer, and a bad answer;

[0240] A preset adversarial model is trained based on the training set, so that the preset adversarial model conforms to a model that can generate triple samples of a preset number, a preset field, and a preset attack effect.

[0241] In one embodiment, obtaining the training sets of the multiple application fields includes:

[0242] Acquire a plurality of problems in the plurality of application fields;

[0243] Obtaining a set of answers to the multiple questions, where each question in the answer set corresponds to multiple answers;

[0244] Based on the preset prior knowledge, determine the best answer and the bad answer among the multiple answers corresponding to each question;

[0245] Based on each question, the best answer and the bad answer corresponding to each question, a triplet sample is constructed to obtain the training set.

[0246] In one embodiment, obtaining the training sets of the multiple application fields includes:

[0247] Acquire a plurality of problems in the plurality of application fields;

[0248] Obtaining a set of answers to the multiple questions, where each question in the answer set corresponds to multiple answers;

[0249] Obtain the scores of the preset experts for multiple answers corresponding to each question;

[0250] Determine the best answer and the bad answer corresponding to each question according to the score;

[0251] Based on each question, the best answer and the bad answer corresponding to each question, a triplet sample is constructed to obtain the training set.

[0252] In one embodiment, the method further comprises:

[0253] Get the value data of the preset data volume;

[0254] Get the preset amount of Anthropic data;

[0255] Selecting a first preset proportion of value data and a first preset proportion of Anthropic data to train an initial scoring model to obtain the scoring model;

[0256] Selecting a second preset proportion of value data and a second preset proportion of anthropic data to train an initial teacher model to obtain the teacher model;

[0257] The sum of the first preset ratio and the second preset ratio is 1.

[0258] In one embodiment, obtaining a preset amount of Anthropic data includes:

[0259] Get the preset amount of initial Anthropic data;

[0260] The initial Anthropic data having value errors are optimized to obtain correct Anthropic data.

[0261] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0262] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and modules described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0263] In the several embodiments provided in the embodiments of the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0264] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0265] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0266] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0267] The computer program product includes one or more computer instructions. When the computer program is loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from a computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, a computer, a server, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or a data center that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0268] The technical solutions provided in the embodiments of the present application are introduced in detail above. The principles and implementation methods of the embodiments of the present application are explained by using specific examples in the embodiments of the present application. The description of the above embodiments is only used to help understand the methods and core ideas of the embodiments of the present application. At the same time, for those skilled in the art, according to the ideas of the embodiments of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the embodiments of the present application.

Claims

1. A large model optimization method, characterized in that: The method comprises: Determine the application area of ​​the student model to be optimized; Calling a preset adversarial model to generate a question set for adversarial training of the student model according to the application field, wherein the question set includes at least one preset question generated based on text adversarial technology, and each question in the question set is a triple data, and the triple data is a mapping relationship between a question, an excellent answer, and a bad answer; Input the problem set into the student big model through a preset adversarial model to obtain an output result; Outputting the output result to a preset scoring model for scoring to obtain a scoring value; Calling a preset teacher model to train the student model according to a preset high-quality data set to obtain training results and optimization solutions, wherein the high-quality question set is a question set that matches questions with preset excellent answers; The scoring score, the training result and the optimization plan are respectively fed back to the student big model to be optimized, so that it can continuously reinforce the learning of the difference between the excellent answers and the bad answers corresponding to the question set, and continuously update the question set to train and score the student big model, until the question-answering ability of the student big model is gradually iterated to the optimal, and the trained student big model is obtained.

2. The large model optimization method according to claim 1, characterized in that: The method further comprises: Acquire training sets for multiple application fields, wherein the training sets include multiple preset triplet samples, each preset triplet sample includes a mapping relationship between a question, an excellent answer, and a bad answer; A preset adversarial model is trained based on the training set, so that the preset adversarial model has the function of generating a preset number of triple samples with preset multiple application fields and preset attack effects.

3. The large model optimization method according to claim 2, characterized in that: The acquiring of the training sets of the multiple application fields comprises: Acquire a plurality of problems in the plurality of application fields; Obtaining a set of answers to the multiple questions, where each question in the answer set corresponds to multiple answers; Based on the preset prior knowledge, determine the best answer and the bad answer among the multiple answers corresponding to each question; Based on each question, the best answer and the bad answer corresponding to each question, a triplet sample is constructed to obtain the training set.

4. The large model optimization method according to claim 2, characterized in that: The acquiring of the training sets of the multiple application fields comprises: Acquire a plurality of problems in the plurality of application fields; Obtaining a set of answers to the multiple questions, where each question in the answer set corresponds to multiple answers; Obtain the scores of the preset experts for multiple answers corresponding to each question; Determine the best answer and the bad answer corresponding to each question according to the score; Based on each question, the best answer and the bad answer corresponding to each question, a triplet sample is constructed to obtain the training set.

5. The large model optimization method according to claim 2, characterized in that: The obtaining of the training sets of the multiple application fields comprises: Get multiple questions in the three areas of politics, laws and regulations, and mental health; Setting corresponding optimal answers and bad answers for the multiple questions; Based on each question, the best answer and the bad answer corresponding to each question, a triplet sample is constructed to obtain a sensitive training set, which includes the sensitive training set. The sensitive training set is used to train the student model to output answers that meet value requirements with respect to sensitive controversial issues.

6. The large model optimization method according to claim 1, characterized in that: The method further comprises: Get the value data of the preset data volume; Get the preset amount of Anthropic data; Selecting a first preset proportion of value data and a first preset proportion of Anthropic data to train an initial scoring model to obtain the scoring model; Selecting a second preset proportion of value data and a second preset proportion of anthropic data to train an initial teacher model to obtain the teacher model; The sum of the first preset ratio and the second preset ratio is 1.

7. The large model optimization method according to claim 6, characterized in that: The step of obtaining a preset amount of Anthropic data includes: Get the preset amount of initial Anthropic data; The initial Anthropic data having value errors are optimized to obtain correct Anthropic data.

8. The large model optimization method according to claim 1, characterized in that: The step of obtaining the value data of a preset amount of data includes: Access to public government documents; Through the design of the prompt, controversial questions are generated based on the government documents, as well as excellent answers that meet the preset values, and poor answers that do not meet the preset values.

9. A large model training method, characterized in that: The method comprises: Determine the application area of ​​the student model to be optimized; According to the application field, a question set for adversarial training of the student model is generated, wherein the question set includes at least one preset question generated based on text adversarial technology, and each question in the question set is a triple data, and the triple data is a mapping relationship between a question, an excellent answer, and a bad answer; The question set is input into the student big model to obtain the output result of the student big model.

10. A large model optimization method, characterized in that: The method comprises: According to a preset high-quality data set, the student model is trained to obtain training results and optimization solutions, wherein the high-quality question set is a question set that matches the preset excellent answers; The training results and optimization scheme are fed back to the student big model to be optimized respectively, so that it can continuously reinforce the learning of the difference between excellent answers and bad answers corresponding to the question set, until the question-answering ability of the student big model is gradually iterated to the optimal level, and the trained student big model is obtained.

11. A large model optimization device, characterized in that: The device includes: A determination module, used to determine the application field of the student large model to be optimized; A generation module, used for calling a preset adversarial model to generate a question set for adversarial training of the student model according to the application field, wherein the question set includes at least one preset question generated based on text adversarial technology, and each question in the question set is a triple data, and the triple data is a mapping relationship between a question, an excellent answer, and a bad answer; An input module, used for inputting the problem set into the student big model through a preset adversarial model to obtain an output result; A scoring model is used to output the output result to a preset scoring model for scoring to obtain a scoring value; A tutoring module is used to call a preset teacher model to train the student model according to a preset high-quality data set to obtain training results and optimization solutions; The training module is used to feed back the scoring score, the training result and the optimization plan to the student big model to be optimized, so that it can continuously strengthen the learning of the difference between the excellent answers and the bad answers corresponding to the question set, and continuously update the question set to train and score the student big model, until the question-answering ability of the student big model is gradually iterated to the optimal, and the trained student big model is obtained.

12. A computing device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 10 when executing the computer program.

13. A computer-readable storage medium, characterized in that: It comprises instructions which, when executed on a computer, cause the computer to perform the method as claimed in any one of claims 1 to 10.

14. A computer program product comprising instructions, the computer program product comprising program instructions, which, when executed on a computer or a processor, cause the computer or the processor to perform the method according to any one of claims 1 to 10.

15. A chip system, characterized in that: The chip system includes: A communication interface for inputting and / or outputting information; A processor, configured to execute a computer executable program so that a device equipped with the chip system executes a method as claimed in any one of claims 1 to 10.

Images