Data processing method and related equipment

Abstract

The embodiment of the invention provides a data processing method and related equipment, the method can be used in the field of large language models in the field of artificial intelligence, and the method comprises the following steps: obtaining a first question input by a user, and generating a first prompt corresponding to the first question through a trained first machine learning model; the first question and the first prompt are input into a large language model LLM to obtain a first answer, or the first question and the first prompt are sent, and the first question and the first prompt are used for being input into the LLM to obtain the first answer. The security of the first question is improved through the first prompt, and the controllability of the answer output by the LLM is improved; the first prompt is generated through the first machine learning model, different prompts can be given for different problems, the first prompt better matched with the first problem is obtained, the first machine learning model and the LLM are mutually independent and can be separated at any time, and the flexibility of the first machine learning model in the deployment stage can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a data processing method and related equipment. Background Art

[0002] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a branch of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0003] Large language model (LLM) is a machine learning model used for text processing in artificial intelligence technology. LLM has a huge impact on the industry, but it also causes security issues. For example, in the process of talking to LLM, the answer output by LLM may carry harmful or inappropriate content; if the content output by LLM does not meet security requirements, LLM will not be widely deployed and may even be removed from the shelves. Therefore, a solution to improve the controllability of the answers output by LLM is urgently needed. Summary of the invention

[0004] The embodiment of the present application provides a data processing method and related equipment, which can improve the security of the first question through a first prompt. Since the security of the first information input into the LLM is higher, the security of the answer output by the LLM will be higher accordingly, thereby improving the controllability of the answer output by the LLM.

[0005] To solve the above technical problems, the present application provides the following technical solutions:

[0006] In a first aspect, an embodiment of the present application provides a data processing method that can be used in the field of large language models in the field of artificial intelligence, the method comprising: after obtaining a first question input by a user, the execution device can generate a first prompt corresponding to the aforementioned first question (question) through a trained first machine learning model; then the execution device inputs the first question and the first prompt into a locally deployed large language model LLM, and obtains a first answer corresponding to the first question generated by the LLM; or, the execution device sends the aforementioned first question and the first prompt to a device that deploys the LLM, and the first question and the first prompt are used to be input into the LLM to obtain the first answer generated by the LLM.

[0007] For example, the first question is Question A, and the first prompt is “Please use a civilized manner when answering…”, and the first information can be “Please use a civilized manner when answering Question A”; for another example, the first question is Question A, and the first prompt is “Please answer in a manner that is consistent with mainstream values”, and the first information can be “Please answer Question A in a manner that is consistent with mainstream values”, etc.

[0008] In this implementation, a first prompt corresponding to the first question can be generated by the trained first machine learning model, and then the first prompt and the first question are input into the LLM. The security of the first question can be improved by the first prompt. Since the security of the first information input into the LLM is higher, the security of the answer output by the LLM will be higher, thereby improving the controllability of the answer output by the LLM. In addition, the first prompt is generated by the trained first machine learning model, which is conducive to giving different prompts for different questions, that is, it is conducive to obtaining a first prompt that better matches the first question, and the trained first machine learning model and LLM are independent of each other and can be separated at any time, which is conducive to improving the flexibility of the first machine learning model in the deployment stage.

[0009] In a possible implementation, the first prompt is used to enhance the security of the first question; wherein, the first prompt is used to enhance the security of the first question, or it can also be understood as "the first prompt is used to prompt the first LLM to use a safer expression to answer the question", or it can also be understood as "the security of the first information is higher than the security of the first question", etc. Exemplarily, "security" in this application can be understood as any one or more of the following: complying with laws and regulations, not causing harm to the user's mental health, being unbiased, peaceful and friendly, complying with ethics or meeting other requirements, etc.

[0010] In a possible implementation, the execution device may pre-store a black library and a white library. For example, the black library includes one or more questions, for example, all the questions in the black library are questions that carry harmful information. The white library includes one or more questions and answers to each of the one or more questions, for example, all the questions in the white library are questions that carry sensitive information and require reasonable responses.

[0011] The execution device obtains the first question input by the user, which may include: the execution device obtains the question input by the user; when the question input by the user does not belong to the black library or the white library, the question input by the user is determined to be the first question. When the question input by the user belongs to the black library, it can be proved that the question input by the user carries harmful information, and the execution device can determine to refuse to answer the aforementioned question carrying harmful information; when the question input by the user belongs to the question in the white library, it can be proved that the question input by the user carries sensitive information but requires a reasonable response, and the execution device can obtain the answer to the aforementioned question input by the user from the white library, and the answer obtained from the white library will be displayed to the user.

[0012] In this implementation, a black library and a white library may be pre-stored. When the question input by the user belongs to the question in the black library, the user's question may be directly refused to be answered; when the question input by the user belongs to the question in the white library, the answer to the question may be directly obtained from the white library. Only when the question input by the user belongs to neither the black library nor the white library, a prompt corresponding to the question will be generated through the trained first machine learning model, which is conducive to improving the efficiency of the process of generating answers corresponding to the questions input by the user.

[0013] In one possible implementation, the method further includes: the execution device connects the first question and the first prompt to obtain the first information. Exemplarily, the first prompt may be located in front of the first question, or the first prompt may be located behind the first question, or the first question may be placed in the middle of the first prompt, etc. Then the execution device inputs the first question and the first prompt into the large language model LLM, including: inputting the first information into the LLM; or, sending the first question and the first prompt, the first question and the first prompt are used to be input into the LLM, including: sending the first information, the first information is used to be input into the LLM. In this implementation, a specific implementation method for obtaining the first information based on the first question and the first prompt is provided, which reduces the difficulty of implementing this solution.

[0014] In one possible implementation, the method is applied to an execution device, in which third information is deployed, and the third information includes prompts corresponding to each category of questions in at least one category. The execution device generates a first prompt corresponding to the first question through a trained first machine learning model, including: the execution device inputs the first question into the trained first machine learning model, obtains a predicted category of the first question output by the trained first machine learning model, and the predicted category is included in at least one category; and obtains the first prompt corresponding to the predicted category from the third information.

[0015] In this implementation, the third information is pre-deployed. After obtaining the first question, the trained first machine learning model can be used to generate a predicted category for the first question. Then, based on the predicted category of the first question, the first prompt corresponding to the first question can be quickly obtained from the third information, which is conducive to improving the efficiency of the process of "obtaining the first prompt corresponding to the first question". In addition, another implementation method for obtaining the first prompt is provided, which improves the implementation flexibility of this solution.

[0016] In one possible implementation, the method is applied to an execution device, in which a plurality of combination information is deployed, each combination information including a fourth question and a prompt corresponding to the fourth question. The execution device generates a first prompt corresponding to the first question through a trained first machine learning model, including: the execution device determines the similarity between the first question and the fourth question in the plurality of combination information through the trained first machine learning model; according to the similarity between the first question and the fourth question, obtains at least one first combination information corresponding to the first question from the plurality of combination information; and further determines a first prompt corresponding to the first question according to the first combination information.

[0017] Exemplarily, the execution device can obtain K target questions that are most similar to the first question from multiple fourth questions included in multiple combination information, where K is an integer greater than or equal to 1; the execution device can obtain K combination information to which the aforementioned K target questions belong from multiple combination information, that is, obtain K first combination information.

[0018] In this implementation, multiple question-prompt pairs are pre-deployed. After obtaining the first question, the first prompt corresponding to the first question is determined based on the similarity between the first question and the questions in the multiple question-prompt pairs, thereby providing another implementation method for obtaining the first prompt, further improving the implementation flexibility of this solution.

[0019] In one possible implementation, the execution device generates a first prompt corresponding to the first question through the trained first machine learning model, including: the execution device inputs the first question into the trained first machine learning model to obtain the first prompt output by the trained first machine learning model. The training phase of the first machine learning model includes a first training operation of the first machine learning model, in which the second question, the expected prompt corresponding to the second question, and the loss function are used in the first training operation of the first machine learning model, the second question is a malicious question, the expected prompt corresponding to the second question is pre-stored, the loss function indicates the similarity between the predicted prompt corresponding to the second question and the expected prompt, and the predicted prompt is obtained by the trained first machine learning model based on the second question in the aforementioned first training operation.

[0020] For example, malicious questions may be "Are the people in XX region of low quality?", "Are the people in XX region of low morals?" or other questions, etc. The above-mentioned "expected prompts" and "predicted prompts" are both a kind of prompts, that is, "expected prompts" can also be combined with "second questions" and input into LLM, and "predicted prompts" can also be combined with "second questions" and input into LLM; it should be noted that the above description is only for the convenience of understanding the role of "expected prompts" and "predicted prompts", and does not mean that in actual situations, it is necessary to combine "expected prompts" and "second questions" and input them into LLM, nor does it mean that in actual situations, it is necessary to combine "predicted prompts" and "second questions" and input them into LLM.

[0021] In this implementation, the first question is input into the trained first machine learning model, and the trained first machine learning model can automatically generate a first prompt that is adapted to the first question, thereby being able to cover a wide range of different questions, that is, being able to generate corresponding prompts for a variety of different questions; and being able to generate corresponding prompts for each question, thereby improving the level of refinement in the process of generating the first prompt. When the first prompt corresponding to the first question can improve the security of the first question, the security of the dialogue process between the user and the LLM can be greatly improved.

[0022] In one possible implementation, the second question is obtained based on the third question, the third question corresponds to the third answer, and both the third question and the third answer are safe; "the third question and the third answer are safe" can be understood as the third question and the third answer can be obtained based on a text that is safe. For example, when the first LLM is a large language model for children, "text that is safe" can include the script of the host of the children's channel; for another example, when the first LLM is a large language model for postgraduate students, "text that is safe" can include teaching materials for postgraduate entrance examinations, etc.

[0023] The second question is a malicious question corresponding to the third question, and the expected prompt corresponding to the second question is obtained from at least one alternative prompt corresponding to the second question based on the similarity information. The similarity information indicates the similarity between each second answer and the third answer in at least one second answer, and each second answer is obtained based on a second information, and a second information includes the third question and an alternative prompt, that is, each second answer is generated by the LLM after a second information is input into the LLM. Since at least one second answer corresponds to at least one alternative prompt one by one, optionally, a second answer that is most similar to the third answer can be obtained from at least one second answer based on the similarity information, and the above-mentioned expected prompt can correspond to the aforementioned most similar second answer.

[0024] In this implementation, since the expected prompt corresponding to the second question is obtained based on the third answer, and the third answers all meet security requirements, it is helpful to ensure that the "expected prompt corresponding to the second question" can improve the security of the second question, that is, it is helpful to ensure that the trained first machine learning model can learn how to generate security prompts for questions during the training phase, which in turn helps to improve the security of users interacting with LLM.

[0025] In one possible implementation, the training phase of the first machine learning model also includes a second training operation of the first machine learning model, the second training operation of the first machine learning model is based on the idea of ​​reinforcement learning and utilizes the trained second machine learning model, the training phase of the second machine learning model adopts a first training sample and a second loss function, the first training sample includes a first malicious question and a prompt corresponding to the first malicious question, and the second loss function indicates the similarity between the predicted score corresponding to the first training sample and the expected score.

[0026] In this implementation, not only the first training operation is performed on the first machine learning model, but the second training operation is also performed on the trained first machine learning model using the trained second machine learning model, which is beneficial to further improve the effectiveness of the prompt information output by the trained first machine learning model; and the second machine learning model is used to score "malicious questions and prompts corresponding to malicious questions", which is beneficial to urge the first machine learning model to learn how to improve the security of malicious questions, and thus help improve the security of the user's interaction process with LLM.

[0027] In one possible implementation, the second training operation of the trained first machine learning model is based on the idea of ​​reinforcement learning and utilizes the trained second machine learning model. The training process of the second machine learning model adopts a second training sample and a second loss function. The second training sample includes a second question and alternative prompts, that is, a "malicious question" can specifically adopt a "second question" and a "prompt corresponding to a malicious question" can specifically adopt an "alternative prompt corresponding to the second question". The second loss function indicates the similarity between the predicted score corresponding to the second training sample and the expected score. The expected score corresponding to the second training sample is obtained based on the similarity between the second answer and the third answer.

[0028] In this implementation, there is an overlap between the "process of generating training data used when training the second machine learning model" and the "process of generating training data used when training the trained first machine learning model", which reduces the difficulty of the process of generating training data for the second machine learning model and helps to save computer resources used in the process of generating training data for the trained first machine learning model and the second machine learning model.

[0029] On the second aspect, the embodiment of the present application provides a data processing method that can be used in the field of large language models in the field of artificial intelligence, and the method includes: a training device obtains a second question and an expected prompt corresponding to the second question; the second question is input into a first machine learning model to obtain a prediction prompt corresponding to the second question output by the first machine learning model; a first training operation is performed on the first machine learning model according to a first loss function to obtain a trained first machine learning model. Among them, the first loss function indicates the similarity between the prediction prompt corresponding to the second question and the expected prompt, and the expected prompt and the second question can be input into the large language model LLM. It should be noted that the above description is only for the convenience of understanding the role of the "expected prompt", and does not mean that in actual situations, the "expected prompt" and the "second question" need to be combined and input into the LLM.

[0030] In one possible implementation, the method also includes: a training device obtains a third question and a third answer corresponding to the third question, both of which comply with security requirements; generates a second question, which is a malicious question corresponding to the third question; obtains at least one alternative prompt corresponding to the second question; obtains at least one second information corresponding one-to-one to the at least one alternative prompt, each second information including the second question and an alternative prompt; inputs each second information in the at least one second information into a large language model LLM to obtain a second answer corresponding to each second information; and obtains an expected prompt corresponding to the second question from multiple alternative prompts based on the similarity between each second answer in the at least one second answer corresponding one-to-one to the at least one second information and the third answer.

[0031] In one possible implementation, the method further includes: the training device acquires a trained second machine learning model, wherein the training process of the second machine learning model uses a first training sample and a second loss function, the first training sample includes a first malicious question and a prompt corresponding to the first malicious question, the output of the second machine learning model during the training process includes a predicted score corresponding to the first training sample, and the second loss function indicates the similarity between the predicted score corresponding to the first training sample and the expected score. Based on the idea of ​​reinforcement learning, the training device uses the trained second machine learning model to perform a second training operation on the first machine learning model that has performed the first training operation.

[0032] In one possible implementation, the method further includes: the training device obtains a trained second machine learning model, wherein the training process of the second machine learning model uses a second training sample and a second loss function, the second training sample includes a second question and an alternative prompt corresponding to the second question, the output of the second machine learning model during the training process includes a predicted score corresponding to the second training sample, the second loss function indicates the similarity between the predicted score corresponding to the second training sample and the expected score, and the expected score corresponding to the second training sample is obtained based on the similarity between the second answer and the third answer. Based on the idea of ​​reinforcement learning, the training device uses the trained second machine learning model to perform a second training operation on the first machine learning model that has performed the first training operation.

[0033] The specific implementation methods of the steps in the second aspect, the meanings of the terms and the beneficial effects brought about can all be referred to the first aspect and will not be repeated here.

[0034] In a third aspect, an embodiment of the present application provides a data processing device that can be used in the field of large language models in the field of artificial intelligence, the device comprising: an acquisition module for acquiring a first question input by a user; a generation module for generating a first prompt corresponding to the first question through a trained first machine learning model; a processing module for inputting the first question and the first prompt into the large language model LLM to obtain a first answer generated by the LLM, or sending the first question and the first prompt, and the first question and the first prompt are used to be input into the LLM to obtain a first answer generated by the LLM.

[0035] In the third aspect of the present application, the data processing device is also used to execute the steps performed by the execution device in the first aspect and various possible implementation methods of the first aspect. The specific implementation methods of the steps in the third aspect, the meanings of the terms and the beneficial effects brought about can all be referred to the first aspect and will not be repeated here.

[0036] In a fourth aspect, an embodiment of the present application provides a data processing device that can be used in the field of large language models in the field of artificial intelligence, the device comprising: an acquisition module, used to acquire a second question and an expected prompt corresponding to the second question; an input module, used to input the second question into a first machine learning model, and obtain a prediction prompt corresponding to the second question output by the first machine learning model; a training module, used to perform a first training operation on the first machine learning model according to a first loss function, and obtain a trained first machine learning model, the first loss function indicating the similarity between the prediction prompt corresponding to the second question and the expected prompt, wherein the expected prompt and the second question can be input into the large language model LLM.

[0037] In the fourth aspect of the present application, the data processing device is also used to execute the steps performed by the training device in the second aspect and various possible implementation methods of the second aspect. The specific implementation methods of the steps in the fourth aspect, the meanings of the terms and the beneficial effects brought about can all be referred to the second aspect and will not be repeated here.

[0038] In a fifth aspect, an embodiment of the present application provides an execution device, including a processor and a memory, wherein the processor is coupled to the memory, the memory is used to store programs; the processor is used to execute the programs in the memory, so that the execution device executes the data processing method described in the first aspect above.

[0039] In a sixth aspect, an embodiment of the present application provides a training device, including a processor and a memory, wherein the processor is coupled to the memory, the memory is used to store programs; the processor is used to execute the programs in the memory, so that the training device executes the data processing method described in the second aspect above.

[0040] In a seventh aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer executes the method described in the first aspect or the second aspect above.

[0041] In an eighth aspect, an embodiment of the present application provides a computer program product, which includes a program. When the program runs on a computer, the computer executes the method described in the first or second aspect above.

[0042] In a ninth aspect, the present application provides a chip system, which includes a processor for supporting the implementation of the functions involved in the above aspects, for example, sending or processing the data and / or information involved in the above methods. In one possible design, the chip system also includes a memory, which is used to store program instructions and data necessary for a terminal device or a communication device. The chip system can be composed of a chip, or it can include a chip and other discrete devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A schematic diagram of the structure of the artificial intelligence main framework provided in the embodiment of the present application;

[0044] Figure 2 A system architecture diagram of a data processing system provided in an embodiment of the present application;

[0045] Figure 3 A schematic diagram of a flow chart of a data processing method provided in an embodiment of the present application;

[0046] Figure 4 A schematic diagram for obtaining the first question provided by the present application;

[0047] Figure 5 A schematic diagram of a process for obtaining a first prompt corresponding to a first question provided by the present application;

[0048] Figure 6 Another schematic diagram of a process for obtaining a first prompt corresponding to a first question provided by the present application;

[0049] Figure 7 Another flowchart of the data processing method provided in the embodiment of the present application;

[0050] Figure 8 Another flowchart of the data processing method provided in the embodiment of the present application;

[0051] Fig. 9 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;

[0052] Fig.10 Another structural schematic diagram of a data processing device provided in an embodiment of the present application;

[0053] Fig.11 A schematic diagram of the structure of an execution device provided in an embodiment of the present application;

[0054] Fig.12 A schematic diagram of the structure of a training device provided in an embodiment of the present application;

[0055] Fig.13 A schematic diagram of the structure of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0057] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0058] First, the overall workflow of the artificial intelligence system is described. Figure 1 , Figure 1 The figure shows a structural diagram of the main framework of artificial intelligence. The following is an explanation of the above artificial intelligence theme framework from the two dimensions of "intelligent information chain" (horizontal axis) and "IT value chain" (vertical axis). Among them, the "intelligent information chain" reflects a series of processes from data acquisition to processing. For example, it can be a general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, intelligent execution and output. In this process, the data has undergone a condensation process of "data-information-knowledge-wisdom". The "IT value chain" reflects the value that artificial intelligence brings to the information technology industry from the underlying infrastructure of human intelligence, information (providing and processing technology implementation) to the industrial ecology process of the system.

[0059] (1) Infrastructure

[0060] The infrastructure provides computing power support for the artificial intelligence system, enables communication with the outside world, and is supported by the basic platform. It communicates with the outside world through sensors; computing power is provided by smart chips, which can specifically use hardware acceleration chips such as central processing units (CPU), embedded neural network processing units (NPU), graphics processing units (GPU), application specific integrated circuits (ASIC) or field programmable gate arrays (FPGA); the basic platform includes distributed computing frameworks and networks and other related platform guarantees and support, which can include cloud storage and computing, interconnected networks, etc. For example, sensors communicate with the outside world to obtain data, and these data are provided to the smart chips in the distributed computing system provided by the basic platform for calculation.

[0061] (2) Data

[0062] The data on the upper layer of the infrastructure is used to represent the data sources in the field of artificial intelligence. The data involves graphics, images, voice, text, and IoT data of traditional devices, including business data of existing systems and perception data such as force, displacement, liquid level, temperature, and humidity.

[0063] (3) Data processing

[0064] Data processing usually includes data training, machine learning, deep learning, search, reasoning, decision-making and other methods.

[0065] Among them, machine learning and deep learning can symbolize and formalize data for intelligent information modeling, extraction, preprocessing, and training.

[0066] Reasoning refers to the process of simulating human intelligent reasoning in computers or intelligent systems, using formalized information to perform machine thinking and solve problems based on reasoning control strategies. Typical functions are search and matching.

[0067] Decision-making refers to the process of making decisions after intelligent information is reasoned, usually providing functions such as classification, sorting, and prediction.

[0068] (4) General ability

[0069] After the data has undergone the data processing mentioned above, some general capabilities can be further formed based on the results of the data processing, such as an algorithm or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.

[0070] (5) Smart products and industry applications

[0071] Smart products and industry applications refer to the products and applications of artificial intelligence systems in various fields. They are the encapsulation of the overall artificial intelligence solution, which productizes intelligent information decision-making and realizes practical applications. Its application areas mainly include: smart terminals, smart manufacturing, smart transportation, smart homes, smart medical care, smart security, autonomous driving, smart cities, etc.

[0072] The method provided in this application can be applied to various application fields of artificial intelligence technology, and can be applied to various application scenarios using large language models (LLMs). The following examples are given of application scenarios in multiple application fields of this application.

[0073] Application field 1: Intelligent terminal field

[0074] For example, a language interaction function can be provided to users on mobile phones, tablets, computers or other types of smart terminals. Exemplarily, an LLM interface can be deployed on the smart terminal, and the user can input the questions they want to send to the LLM through the smart terminal. The smart terminal sends the questions input by the user to the device where the LLM is deployed through the aforementioned interface. The device where the LLM is deployed can input the user's questions into the LLM, obtain the answers corresponding to the user's questions output by the LLM, and send the aforementioned answers to the smart terminal; the smart terminal then displays the aforementioned answers to the user, thereby realizing the aforementioned language interaction function. Exemplarily, the device where the LLM is deployed can be specifically manifested as a server, a server cluster or other product forms, etc., which are not limited in the embodiments of the present application.

[0075] Application field 2: Smart home or smart manufacturing

[0076] For example, there may be intelligent robots in the field of smart home or smart manufacturing, and the intelligent robots may provide language interaction functions to users. For example, an LLM may be deployed in the intelligent robot, and after the intelligent robot receives a question input by the user, the question may be input into the LLM to obtain an answer output by the LLM; the intelligent robot may output the aforementioned answer to the user.

[0077] It should be noted that the method provided in the present application can also be applied to other application scenarios in which LLM will be used. The above examples of various application scenarios of the present application are only for the convenience of understanding the present solution and are not used to limit the present solution.

[0078] LLM is required in all of the above-mentioned application scenarios. In order to improve the controllability of the answer output by LLM, the method provided in the present application uses a first machine learning model to generate a first prompt corresponding to the first question, and obtains the first information input into the LLM based on the first prompt and the first question. Since the first information is not only based on the first question but also introduces the first prompt, the security of the first question can be improved through the first prompt. The higher the security of the first information input into the LLM, the higher the security of the answer output by the LLM, thereby improving the controllability of the answer output by the LLM. In addition, the first prompt is generated by the first machine learning model, which is conducive to giving different prompts for different questions, that is, it is conducive to obtaining a first prompt that better matches the first question, and the first machine learning model and LLM are independent of each other and can be separated at any time, which is conducive to improving the flexibility of the first machine learning model in the deployment stage.

[0079] Before describing the method provided in this application in detail, please refer to Figure 2 , Figure 2 A system architecture diagram of a data processing system provided in an embodiment of the present application, in Figure 2 In the embodiment, the data processing system 200 includes a training device 210 , a database 220 , an execution device 230 , a data storage system 240 and a client device 250 , and the execution device 230 includes a computing module 231 .

[0080] Among them, a training data set is stored in the database 220. In the training stage of the first machine learning model 201, the training device 210 generates a first machine learning model 201 that has not yet performed a training operation, and uses the training data set to iteratively train the first machine learning model 201 to obtain the first machine learning model 201 that has performed a training operation. The first machine learning model 201 can be specifically expressed as a neural network, or it can be expressed as a non-neural network model. In the embodiments of the present application, only the first machine learning model 201 expressed as a neural network is used as an example for explanation. Furthermore, when the first machine learning model 201 is expressed as a neural network, the first machine learning model 201 can be a large language model, or it can also be other types of neural networks. For example, the first machine learning model 201 can also be expressed as a recurrent neural network, a neural network based on an attention mechanism, etc., which are not limited in the embodiments of the present application.

[0081] The first machine learning model 201 that has been trained and obtained by the training device 210 (that is, the first machine learning model 201 after training) can be deployed in the computing module 231 of the execution device 230. The execution device 230 can call the data, code, etc. in the data storage system 240, or store the data, instructions, etc. in the data storage system 240. The data storage system 240 can be placed in the execution device 230, or the data storage system 240 can be an external memory relative to the execution device 230. It should be noted that when the first machine learning model 201 is a large language model, the training device 210 and the execution device 230 can be the same device or different devices. In the application stage of the first machine learning model 201, the first machine learning model 201 will be used to automatically model the operations optimization problem.

[0082] In some embodiments of this application, please refer to Figure 2 The execution device 230 and the client device 250 are independent devices. The execution device 230 is configured with an input / output (I / O) interface to exchange data with the client device 250. After obtaining the first description information corresponding to at least one problem, the client device 250 can obtain the first information based on the first description information, and then send the first information to the execution device 230 through the I / O interface. The first description information includes the description information of the first operations optimization problem; after receiving the first information, the execution device 230 can generate a first prompt corresponding to the first information through the first machine learning model 201 in the computing module 231, and then send the aforementioned first prompt to the client device 250 through the I / O interface.

[0083] For example, the client device 250 can be a terminal device or an edge device, and the execution device 230 can be a cloud device, such as a server or a server cluster, etc. The product form of the client device 250 and the execution device 230 is not limited in the embodiments of the present application.

[0084] It is worth noting that Figure 2 It is only an architectural diagram of two data processing systems provided by an embodiment of the present invention, and the positional relationship between the devices, components, modules, etc. shown in the figure does not constitute any limitation. For example, in other embodiments of the present application, the execution device 230 and the client device 250 may be integrated in the same device, and the user can interact directly with the execution device 230. Exemplarily, when the client device 250 is a mobile phone or a tablet, the execution device 230 may be a module in the host processor (Host CPU) of the mobile phone or tablet that uses the first machine learning model to perform data processing, and the execution device 230 may also be a neural network processor (NPU) in the mobile phone or tablet. The NPU is mounted on the host processor as a coprocessor, and the tasks are assigned by the host processor.

[0085] In combination with the above description, the following describes the specific implementation process of the reasoning phase and the training phase of the first machine learning model provided in the embodiment of the present application.

[0086] 1. Application Phase

[0087] In the embodiment of the present application, the application stage describes the process in which the execution device 230 generates a first prompt corresponding to the first question using the trained first machine learning model. For details, please refer to Figure 3 , Figure 3 A flow chart of a data processing method provided in an embodiment of the present application. The data processing method provided in an embodiment of the present application may include:

[0088] 301. The execution device obtains a first question input by a user.

[0089] Exemplarily, in one case, the first question can be any question input by the user, that is, a corresponding prompt can be generated for each question input by the user. In another case, the execution device may be pre-configured with a black library and a white library, and the black library and the white library may be data sets in the memory of the execution device; Exemplarily, the black library includes one or more questions, for example, the questions in the black library are all questions that carry harmful information; the white library includes one or more questions and the answers to each of the one or more questions, for example, the questions in the white library are all questions that carry sensitive information and require reasonable responses. Optionally, the first question is a question after the execution device uses the black library and the white library to screen the questions input by the user, and the first question does not belong to the questions in the black library or the white library. Exemplarily, if the question input by the user belongs to the question in the black library, it can be proved that the question input by the user carries harmful information, and the execution device can determine to refuse to answer the question carrying harmful information. If the question input by the user belongs to the question in the white library, it can be proved that the question input by the user carries sensitive information but requires a reasonable response, and the execution device can obtain the answer to the question input by the user from the white library, and the answer obtained from the white library will be displayed to the user.

[0090] For a more intuitive understanding of this solution, please refer to Figure 4 , Figure 4 A schematic diagram of obtaining the first question provided by this application. Figure 4 As shown, the execution device obtains the question input by the user, matches the question input by the user with the question in the black library, and if the question in the black library is matched, refuses to reply the answer to the user; matches the question input by the user with the question in the black library, and if the question in the white library is matched, reply the user with the standard answer in the white library; if the question input by the user does not match the question in the black library or the question in the white library, inputs the question input by the user (that is, the first question) into the trained first machine learning model, and obtains the first prompt corresponding to the first question generated by the trained first machine learning model. It should be understood that Figure 4 The examples are only for facilitating the understanding of this solution and are not intended to limit this solution.

[0091] In an embodiment of the present application, a black library and a white library may be pre-stored. When the question input by the user belongs to a question in the black library, the user's question may be directly refused to be answered; when the question input by the user belongs to a question in the white library, the answer to the question may be directly obtained from the white library. Only when the question input by the user belongs to neither the black library nor the white library, a prompt corresponding to the question will be generated through the trained first machine learning model, which is conducive to improving the efficiency of the process of generating an answer corresponding to the question input by the user.

[0092] 302. The execution device generates a first prompt corresponding to the first question through the trained first machine learning model.

[0093] In an embodiment of the present application, after obtaining the first question, the execution device can generate a first prompt corresponding to the first question through the trained first machine learning model, and the input of the LLM includes the aforementioned first question and the first prompt; illustratively, the first prompt is used to be combined with the first question and then input into the LLM; for example, "the combination of the first prompt and the first question" can be understood as connecting the first prompt and the first question, or can also be understood as splicing the first prompt and the first question.

[0094] Exemplarily, in one implementation, step 302 may include: the training device inputs the first question into the trained first machine learning model to obtain a first prompt output by the trained first machine learning model; it should be noted that the detailed implementation process of the training phase of the trained first machine learning model will be described in subsequent embodiments and will not be repeated here.

[0095] Among them, the training stage of the first machine learning model includes the first training operation of the first machine learning model, the second question, the expected prompt corresponding to the second question and the first loss function are used in the aforementioned first training operation, the second question is a malicious question, the expected prompt corresponding to the second question is pre-stored, and the first loss function indicates the similarity between the predicted prompt corresponding to the second question and the expected prompt corresponding to the second question; the aforementioned "expected prompt" and "prediction prompt" are both a kind of prompt, that is, the "expected prompt" can also be combined with the "second question" and input into the LLM, and the "prediction prompt" can also be combined with the "second question" and input into the LLM. It should be noted that the aforementioned description is only for the convenience of understanding the role of "expected prompt" and "prediction prompt", and does not mean that in actual situations, the "expected prompt" and "second question" need to be combined and input into the LLM, nor does it mean that in actual situations, the "expected prompt" and "second question" need to be combined and input into the LLM.

[0096] The above-mentioned prediction prompt corresponding to the second question is obtained by the first machine learning model based on the second question during the training process, that is, in the training stage of the first machine learning model, the input of the first machine learning model includes the second question, and the output of the first machine learning model includes the prediction prompt corresponding to the second question.

[0097] In an embodiment of the present application, the first question is input into the trained first machine learning model, and the trained first machine learning model can automatically generate a first prompt adapted to the first question, thereby being able to cover a wide variety of different questions, that is, being able to generate corresponding prompts for a variety of different questions; and being able to generate a corresponding prompt for each question, thereby improving the level of refinement in the process of generating the first prompt. When the first prompt corresponding to the first question can improve the security of the first question, the security of the dialogue process between the user and the LLM can be greatly improved.

[0098] In another implementation, the execution device is deployed with third information, and the third information includes prompts corresponding to each category of questions in at least one category. Exemplarily, the at least one category may include any one or more of the following categories: physical health, mental health, property privacy, prejudice, ethics or other categories of inappropriate speech, etc. It should be noted that the example of "at least one category" here is for the convenience of understanding this solution. The specific categories of "at least one category" can be determined in combination with the actual application scenario, and are not limited here.

[0099] Optionally, in the construction stage of the third information, the prompts included in the third information can be obtained from texts that meet the security requirements. For example, the sources of prompts corresponding to simple psychological questions can include texts produced in mental health lectures, texts in mental health textbooks, mental health papers or other texts, etc. The examples here are only for the convenience of understanding this solution and are not used to limit this solution.

[0100] It should be noted that the steps of the “third information construction phase” can be executed by the execution device or by a first device other than the execution device. Before executing step 302, the execution device can also obtain the third information sent by the first device.

[0101] Step 302 may include: the execution device inputs the first question into the trained first machine learning model to obtain the predicted category of the first question output by the trained first machine learning model; wherein the predicted category is included in at least one category, that is, the predicted category is a category in at least one category; for example, the first machine learning model in this implementation may use a classifier. Then the execution device obtains a first prompt corresponding to the predicted category from the third information.

[0102] For a more intuitive understanding of this solution, please refer to Figure 5 , Figure 5A flowchart for obtaining a first prompt corresponding to a first question provided for the present application. In the construction stage, the third information can be constructed based on a security-compliant database, and then the third information can be deployed on an execution device. In the application stage, after obtaining the first question, the execution device can input the first question into a classifier (that is, an example of a trained first machine learning model) to obtain a predicted category of the first question output by the classifier; the execution device matches the predicted category with at least one category in the third information to determine a first prompt corresponding to the predicted category, that is, to obtain a first prompt corresponding to the first question. It should be understood that Figure 5 The examples are only for facilitating the understanding of this solution and are not intended to limit this solution.

[0103] In an embodiment of the present application, third information is pre-deployed. After obtaining the first question, the trained first machine learning model can be used to generate a predicted category for the first question, and then according to the predicted category of the first question, the first prompt corresponding to the first question can be quickly obtained from the third information, which is conducive to improving the efficiency of the process of "obtaining the first prompt corresponding to the first question". In addition, another implementation method for obtaining the first prompt is provided, which improves the implementation flexibility of the present solution.

[0104] In another implementation, a plurality of combination information is deployed in the execution device, and each combination information may include a fourth question and a prompt corresponding to the fourth question; illustratively, each combination information may also be understood as a question-safe prompt (Q-Sp) pair. Optionally, in the construction stage of the plurality of combination information, each combination information may be derived from a text that complies with security. For an example of a "text that complies with security", please refer to the above description, which will not be repeated here.

[0105] Step 302 may include: the execution device determines the similarity between the first question and each fourth question included in the plurality of combined information through the trained first machine learning model; that is, the execution device calculates the similarity between the first question and each fourth question through the trained first machine learning model. Exemplarily, the trained first machine learning model may adopt a Bert model or other neural network, etc., which is not limited here; "the similarity between the first question and each fourth question" may be understood as "the semantic similarity between the first question and each fourth question".

[0106] The execution device obtains at least one first combination information corresponding to the first question from the multiple combination information according to the similarity between the first question and each fourth question in the multiple combination information; and determines a first prompt corresponding to the first question according to the at least one first combination information.

[0107] Exemplarily, after obtaining the similarity between the first question and each fourth question in the plurality of combined information, the execution device can obtain K target questions most similar to the first question from the plurality of fourth questions included in the plurality of combined information, where K is an integer greater than or equal to 1; the execution device can obtain K combined information to which the aforementioned K target questions belong from the plurality of combined information, that is, obtain K first combined information. For example, the value of K can be 1, 2, 3, 5 or other values, etc., which is not limited in the embodiments of the present application.

[0108] For example, in one implementation, after obtaining K first combination information, the execution device can fuse the K first combination information, input the fused K first combination information into the second LLM, and obtain the first prompt corresponding to the first question output by the second LLM; it should be noted that the concept of "first LLM" will be used in subsequent steps and will not be introduced here. Exemplarily, the aforementioned fusion method can adopt splicing or other fusion methods, which are not limited in the embodiments of the present application.

[0109] In another implementation, the value of K is 1, and after obtaining one first combination information, the execution device may also directly determine the first prompt included in the aforementioned one first combination information as the first prompt corresponding to the first question.

[0110] In another implementation, after obtaining K first combination information, the execution device may also splice the multiple prompts included in the K first combination information to obtain a first prompt corresponding to the first question, etc. The execution device may also use other methods to implement "determining the first prompt based on the K first combination information". The example here is only for the convenience of understanding this solution and is not used to limit this solution.

[0111] For a more intuitive understanding of this solution, please refer to Figure 6 , Figure 6 Another flowchart for obtaining a first prompt corresponding to a first question provided in the present application. In the construction phase, multiple question-safety prompt (Q-sP) pairs can be obtained from security-compliant text, and each Q-sP pair includes a fourth question and a safety prompt corresponding to the fourth question. In the application phase, after the execution device obtains the first question, the semantic similarity between the first question and the fourth question in each Q-sP pair in the multiple Q-sP pairs can be obtained, and based on the semantic similarity between the first question and each fourth question, the three target questions (i.e., Q1, Q2, and Q3) that are most similar to the first question are obtained from the multiple fourth questions included in the multiple Q-sP pairs. The execution device obtains the three Q-sP pairs (i.e., Q1, Q2, and Q3) to which Q1, Q2, and Q3 belong. Figure 6Q1-sP1, Q2-sP2 and Q3-sP3 in the above formula are merged and input into the second LLM to obtain the first prompt corresponding to the first question output by the second LLM. It should be understood that Figure 6 The examples are only for facilitating the understanding of this solution and are not intended to limit this solution.

[0112] In an embodiment of the present application, multiple question-prompt pairs are pre-deployed. After obtaining the first question, a first prompt corresponding to the first question is determined based on the similarity between the first question and the questions in the multiple question-prompt pairs, thereby providing another implementation method for obtaining the first prompt, further improving the implementation flexibility of the present solution.

[0113] 303. The execution device inputs the first question and the first prompt into the large language model LLM to obtain a first answer, or sends the first question and the first prompt, and the first question and the first prompt are used to be input into the LLM to obtain the first answer.

[0114] In the embodiment of the present application, for the convenience of distinction, the LLM used in the above step 303 is referred to as the "first LLM". Optionally, the first prompt is used to enhance the security of the first question, or it can also be understood as "the first prompt is used to prompt the first LLM to use a safer expression to answer the question", or it can also be understood as "the security of the first information is higher than the security of the first question", etc. Exemplarily, the "security" in the present application can be understood as any one or more of the following: complying with laws and regulations, not causing harm to the user's mental health, unbiased, peaceful and friendly, complying with ethics or meeting other requirements, etc., which are not emphasized in the embodiments of the present application.

[0115] For example, the first prompt may be “please answer politely”, “please answer in a manner that complies with the law”, “please answer in a manner that conforms to mainstream values”, “please follow public order and good morals when answering”, “please answer in a manner that children can accept”, or the first prompt may be specifically expressed as other information, etc., which are not exhaustive here.

[0116] Optionally, the execution device may also obtain the first information according to the first question and the first prompt before executing step 303. Step 303 may include: the execution device inputs the first information into the first LLM to obtain a first answer corresponding to the first question generated by the first LLM; or the execution device sends the first information to the device that deploys the first LLM, and the first information is used for the device that deploys the first LLM to input the first information into the first LLM to obtain a first answer corresponding to the first question generated by the first LLM.

[0117] Exemplarily, the execution device obtains the first information according to the first question and the first prompt, which may include: the execution device connects the first question and the first prompt to obtain the first information. Exemplarily, the first prompt may be located in front of the first question, or the first prompt may be located behind the first question, or the first question may be placed in the middle of the first prompt, etc. The relative position relationship between the specific first question and the first prompt may be determined in combination with the actual application scenario. In an embodiment of the present application, a specific implementation method for obtaining the first information based on the first question and the first prompt is provided, which reduces the difficulty of implementing the present solution.

[0118] For example, the first question is question A, and the first prompt is “Please use a civilized manner when answering...”, and the first message can be “Please use a civilized manner when answering question A”; for another example, the first question is question A, and the first prompt is “Please answer in a manner that is consistent with mainstream values”, and the first message can be “Please answer question A in a manner that is consistent with mainstream values”, and so on. It should be understood that the examples given here are only for the convenience of understanding this solution and are not used to limit this solution.

[0119] It should be noted that other methods may be used to implement “obtaining the first information based on the first question and the first prompt”, as long as the first information includes the first question and the first information, and this is not limited in the embodiments of the present application.

[0120] After the execution device obtains the first information, the first information needs to be input into the first LLM to obtain a first answer corresponding to the first information output by the first LLM. For example, in one case, the first LLM is deployed on the execution device, and the execution device can directly input the first information into the first LLM to obtain a first answer (Answer) corresponding to the first information output by the first LLM.

[0121] In another case, the device deploying the first LLM and the execution device are different devices. The execution device can send the first information to the device deployed with the first LLM, and then the device deploying the first LLM inputs the first information into the first LLM to obtain a first answer corresponding to the first information output by the first LLM.

[0122] It should be noted that in the subsequent embodiments of the present application, "inputting certain information into the LLM" can be understood as inputting the information into the locally deployed LLM, and can also be understood as sending the information to the device deployed with the LLM.

[0123] Among them, the "first LLM" and the "second LLM" can be the same large language model or different large language models. The specific details can be flexibly determined in combination with the actual application scenario and are not limited in the embodiments of the present application.

[0124] For a more intuitive understanding of this solution, please refer to Figure 7 , Figure 7 Another flowchart of the data processing method provided in the embodiment of the present application. Figure 7 Can be combined with the above Figure 4 To understand, Figure 7 In the example, after obtaining the question input by the user, the execution device can obtain the first question from the question input by the user, and the first question is a question that does not hit the black library or the white library; the execution device obtains the first prompt corresponding to the first question. The first prompt and the first question are then input into the large language model LLM to obtain the first answer corresponding to the first question output by the LLM. It should be noted that the device that generates the first prompt corresponding to the first question (that is, the execution device) and the device that deploys the LLM can be the same device or different devices; if the execution device and the device that deploys the LLM are the same device, then after generating the first prompt, the execution device can input the first prompt and the first question into the locally deployed LLM; if the execution device and the device that deploys the LLM are different devices, then after generating the first prompt, the execution device can send the first prompt and the first question to the device that deploys the LLM, and the device that deploys the LLM inputs the first prompt and the first question into the LLM. It should be understood that Figure 7 The examples are only for facilitating the understanding of this solution and are not intended to limit this solution.

[0125] 2. Training Phase

[0126] In the embodiment of the present application, the training phase describes the process in which the training device 210 trains the first machine learning model 201 using the training data set in the database 220. For details, please refer to Figure 8 , Figure 8 Another flow chart of the data processing method provided in the embodiment of the present application is as follows. The data processing method provided in the embodiment of the present application may include:

[0127] 801. Obtain a third question and a third answer corresponding to the third question, where both the third question and the third answer meet security requirements.

[0128] In the embodiment of the present application, step 801 is an optional step. Exemplarily, the second combined information may include a third question and a third answer, and “the third question and the third answer are both safety-compliant” can be understood as the third question and the third answer can be obtained based on a text that complies with safety. For example, when the first LLM is a large language model for children, “text that complies with safety” may include the script of the host of the children’s channel; for another example, when the first LLM is a large language model for postgraduate students, “text that complies with safety” may include teaching materials for postgraduate entrance examinations, etc. Examples of “text that complies with safety” can also be found above. Figure 3Corresponding to the description in the embodiments, the examples in the embodiments of the present application are only for facilitating the understanding of the present solution and are not used to limit the present solution.

[0129] In one implementation, multiple first texts can be obtained from security-compliant texts, and each first text can be input into a third machine learning model to obtain a question corresponding to each first text output by the third machine learning model. Then, each first text can be determined as a third answer, and the question corresponding to each first text can be determined as a third question.

[0130] In another implementation, the third question can be generated by a large language model. For example, "Please give a question corresponding to the first text" is input into the third LLM to obtain the question corresponding to each first text output by the third LLM, and then the first text is determined as the third answer, and the question corresponding to the first text is determined as the third question. The third LLM, the second LLM and the first LLM can use the same large language model or different large language models, which can be flexibly determined in combination with the actual application scenario.

[0131] The training device repeats the above steps for multiple times, and can obtain multiple pieces of second combination information.

[0132] 802. Generate a second question, where the second question is a malicious question corresponding to the third question.

[0133] In the embodiment of the present application, step 802 is an optional step. For any second combination of information among the plurality of second combination information, after obtaining a third question and a third answer corresponding to the third question, the training device can generate n second questions corresponding to the third question, each second question is a malicious question corresponding to the third question, and n is an integer greater than or equal to 1.

[0134] Exemplarily, the training device can generate a second question corresponding to the third question through a large language model; for example, "Please generate a question that is semantically similar to the third question but has malicious tendencies" can be input into the third LLM to obtain the second question output by the third LLM. It should be noted that other information can also be input into the third LLM to obtain the second question, such as "Please generate a question that is similar in meaning to the third question but has malicious tendencies", etc. The examples given here are only for the convenience of understanding this solution and are not used to limit this solution.

[0135] 803. Obtain at least one alternative prompt corresponding to the second question.

[0136] In the embodiment of the present application, step 803 is an optional step. After obtaining the second question corresponding to a third question, the training device may obtain n alternative prompts corresponding to the second question, where n is an integer greater than or equal to 1.

[0137] Exemplarily, the training device can generate a second question corresponding to the third question through a large language model; for example, "please generate multiple security questions corresponding to the second question" can be input into the third LLM to obtain n alternative prompts output by the third LLM; for another example, "please generate multiple prompts to improve the security of the second question" can be input into the third LLM, and so on. Other information can also be input into the third LLM to obtain n alternative prompts. The examples given here are only for the convenience of understanding this solution and are not used to limit this solution.

[0138] For example, the second question is "Can artificial intelligence develop to a level far beyond that of humans?" The alternative prompt may be "Please consider that humans are creatures with advanced intelligence and answer the question from a perspective that is consistent with human values." It should be understood that the examples here are only given to facilitate understanding of this solution and are not used to limit this solution.

[0139] 804. Obtain at least one second information corresponding to at least one alternative prompt, each second information including a second question and an alternative prompt.

[0140] In the embodiment of the present application, step 804 is an optional step. The specific implementation method of "obtaining the second information based on the second question and the alternative prompt" can be found in Figure 3 The description of "obtaining first information based on the first question and the first prompt" in the corresponding embodiment is different in that the first question is replaced by the second question, the first prompt is replaced by the alternative prompt, and the first information is replaced by the second information. The detailed implementation method will not be repeated here.

[0141] Since one second question corresponds to n alternative prompts, in step 804, the training device can obtain n second information corresponding one-to-one to the n alternative prompts.

[0142] 805. Input each second information of the at least one second information into the first LLM to obtain a second answer corresponding to each second information.

[0143] In the embodiment of the present application, step 805 is an optional step. After obtaining n second information, the training device can input each second information into the first LLM to obtain a second answer corresponding to each second information, and the second answer corresponding to each second information is generated by the first LLM.

[0144] 806. Obtain a similarity between each second answer and the third answer in at least one second answer corresponding one-to-one to the at least one second information.

[0145] In the embodiment of the present application, step 806 is an optional step. The "similarity between each second answer and the third answer" can be understood as the semantic similarity between each second answer and the third answer. Exemplarily, the "similarity between each second answer and the third answer" can be obtained based on the cosine similarity, Euclidean distance, L1 distance, L2 distance or other information between each second answer and the third answer, or the "similarity between each second answer and the third answer" can be obtained by neural network calculation, etc. The specific implementation methods of step 806 are not exhaustively listed here, and the examples given here are not used to limit this solution.

[0146] 807. Obtain a desired prompt corresponding to the second question from a plurality of alternative prompts based on a similarity between each second answer in at least one second answer corresponding one-to-one to at least one second information and the third answer.

[0147] In the embodiment of the present application, step 807 is an optional step. Exemplarily, after obtaining the similarity between each second answer and the third answer among the n second answers corresponding one-to-one to the n second information, the training device can obtain a target answer that is most similar to the third answer from the n second answers. Since the n second information corresponds one-to-one to the n alternative prompts, that is, the n second answers correspond one-to-one to the n alternative prompts, an alternative prompt corresponding to the target answer can be obtained from the n alternative prompts, and the "alternative prompt corresponding to the target answer" is confirmed as the expected prompt corresponding to the second question.

[0148] The training device repeatedly executes steps 802 to 807 for multiple times, and can obtain multiple second questions and the expected prompt corresponding to each second question.

[0149] 808. Obtain a second question and an expected prompt corresponding to the second question.

[0150] In the embodiment of the present application, steps 801 to 807 are all optional steps. If steps 801 to 807 are executed, the second question is obtained based on the third question, the third question corresponds to the third answer, the third question and the third answer both meet security requirements, the second question is a malicious question corresponding to the third question, and the expected prompt corresponding to the second question is obtained from at least one alternative prompt corresponding to the second question based on the similarity information; wherein the similarity information indicates the similarity between each second answer and the third answer in at least one second answer, each second answer is obtained based on a second information, and a second information includes a third question and an alternative prompt. In the embodiment of the present application, since the expected prompt corresponding to the second question is obtained based on the third answer, and the third answers all meet security requirements, it is beneficial to ensure that the "expected prompt corresponding to the second question" can improve the security of the second question, that is, it is beneficial to ensure that the first machine learning model can learn how to generate security prompts for questions during the training phase, and thus it is beneficial to improve the security of users interacting with LLM.

[0151] If steps 801 to 807 are not performed, multiple second questions and the expected prompt corresponding to each second question may also be manually determined, and the training device obtains multiple second questions and the expected prompt corresponding to each second question sent by other devices.

[0152] 809. Input the second question into the first machine learning model to obtain a prediction prompt corresponding to the second question output by the first machine learning model.

[0153] 810. Perform a first training operation on the first machine learning model according to a first loss function, the first loss function indicates the similarity between the predicted prompt corresponding to the second question and the expected prompt, and in the inference stage of the first machine learning model, the first machine learning model is used to generate a first prompt corresponding to the first question, the first prompt and the first question are used to obtain first information, and the first information is used to be input into the first LLM.

[0154] In an embodiment of the present application, exemplarily, the purpose of training the first machine learning model using the first loss function includes improving the similarity between the predicted prompt and the expected prompt corresponding to the second question.

[0155] Exemplarily, in the first training operation of the first machine learning model, the training device can generate a function value of the first loss function according to the prediction prompt and the expected prompt corresponding to the second question; perform gradient differentiation on the function value of the first loss function, and use the back propagation algorithm to update the parameters of the first machine learning model to implement one training of the first machine learning model. The training device can use multiple second questions and the expected prompt corresponding to each second question to repeatedly perform steps 808 to 810 to implement iterative training of the first machine learning model until the convergence condition is met, so as to complete the first training operation of the first machine learning model.

[0156] For example, the aforementioned convergence condition may be to satisfy the convergence condition of the first loss function, or the number of iterative trainings of the first machine learning model using the first loss function reaches a preset number.

[0157] Optionally, before performing the first training operation on the first machine learning model, a third training operation may also be performed on the first machine learning model. Exemplarily, the third training operation may be performed on the first machine learning model using the multiple third questions generated in step 801 and the third answer corresponding to each third question. In the third training operation, the training device may input the third question into the first machine learning model to obtain the prediction information corresponding to the third question output by the first machine learning model; the first machine learning model may be trained according to the prediction information corresponding to the third question, the third answer, and the third loss function; the third loss function indicates the similarity between the prediction information corresponding to the third question and the third answer, and the purpose of training using the third loss function includes improving the similarity between the prediction information corresponding to the third question and the third answer.

[0158] Optionally, the first machine learning model obtained when performing the third training operation is a machine learning model that has performed a pre-training operation, and the third training operation is to perform supervised fine-tuning (SFT) training on the pre-trained first machine learning model. Exemplarily, the method used for SFT training of the pre-trained first machine learning model can be quantized low-rank adaptation (QLoRA), low-rank adaptation (LoRA) or other fine-tuning training methods, etc., which are not limited in the embodiments of the present application.

[0159] 811. Obtain a trained second machine learning model, wherein the training process of the second machine learning model adopts a first training sample and a second loss function, the first training sample includes a first malicious question and a prompt corresponding to the first malicious question, the output of the second machine learning model during the training process includes a predicted score corresponding to the first training sample, and the second loss function indicates the similarity between the predicted score corresponding to the first training sample and the expected score.

[0160] In the embodiment of the present application, step 811 is an optional step. The purpose of training with the second loss function includes improving the similarity between the predicted score corresponding to the first training sample and the expected score; the higher the predicted score corresponding to the first training sample, the higher the degree of fit between the first malicious question included in the first training sample and the prompt corresponding to the first malicious question.

[0161] Exemplarily, during the training phase of the second machine learning model, the first training sample can be input into the second machine learning model to obtain the predicted score corresponding to the first training sample output by the second machine learning model; the function value of the second loss function is generated according to the predicted score and expected score corresponding to the first training sample; the function value of the second loss function is gradient derived, and the parameters of the second machine learning model are updated using the back propagation algorithm to complete one training of the second machine learning model. Repeat the aforementioned steps to implement multiple training of the second machine learning model until the convergence condition is met; the aforementioned convergence condition can be the convergence condition of the second loss function, or the number of times the second machine learning model is trained reaches a preset number of times, etc.

[0162] Exemplarily, the embodiment of the present application also provides a source of the first training sample of the second machine learning model. In one case, the first training sample of the second machine learning model can be obtained by manual annotation.

[0163] In another case, the training device can determine the first training sample of the second machine learning model and the expected score corresponding to the first training sample based on the second question obtained in the aforementioned steps, the n alternative prompts corresponding to the second question, and the similarity between the n second answers and the third answer. For the convenience of distinction, the first training sample from this source is referred to as the second training sample. It should be noted that the first training samples used in training the second machine learning model can all be from this source, that is, the first training samples and the second training samples have the same meaning; or, the first training samples used in training the second machine learning model can also have other sources.

[0164] Among them, a second training sample includes a second question and an alternative prompt (for the convenience of description, hereinafter referred to as the "target alternative prompt"), that is, the "first malicious question" can specifically adopt the above-mentioned "second question", and the "prompt corresponding to the first malicious question" can specifically adopt the above-mentioned "alternative prompt corresponding to the second question", and the expected score corresponding to the second training sample is obtained based on the similarity between the second answer corresponding to the aforementioned target alternative prompt and the third answer.

[0165] Optionally, the higher the similarity between the second answer and the third answer corresponding to the target alternative prompt, the higher the expected score corresponding to the second training sample. Exemplarily, the expected score corresponding to the second training sample can adopt the similarity between the second answer and the third answer corresponding to the aforementioned target alternative prompt.

[0166] To further understand this solution, for example, there are 3 alternative prompts corresponding to the second question A, the second question A is a malicious question corresponding to the third question A, and the third question A and the third answer A both meet security requirements. The 3 alternative prompts are alternative prompt 1, alternative prompt 2, and alternative prompt 3. Based on the second question A and alternative prompt 1, the second information 1 is obtained, and based on the second information 1, the second answer 1 is obtained; based on the second question A and alternative prompt 2, the second information 2 is obtained, and based on the second information 2, the second answer 2 is obtained; based on the second question A and alternative prompt 3, the second information 3 is obtained, and based on the second information 3, the second answer 3 is obtained. The similarity between the second answer 1 and the third answer A is 0.3, and the expected score corresponding to the second training sample 1 composed of the second question A and the alternative prompt 1 can be 0.3; the similarity between the second answer 2 and the third answer A is 0.8, and the expected score corresponding to the second training sample 2 composed of the second question A and the alternative prompt 2 can be 0.8; the similarity between the second answer 3 and the third answer A is 0.9, and the expected score corresponding to the second training sample 3 composed of the second question A and the alternative prompt 3 can be 0.9. It should be understood that the examples here are only for the convenience of understanding the meaning of "the expected score corresponding to the second training sample" and are not used to limit this solution.

[0167] In an embodiment of the present application, there is an overlap between the "process of generating training data used when training the second machine learning model" and the "process of generating training data used when training the first machine learning model", which reduces the difficulty of the process of generating training data for the second machine learning model and is conducive to saving computer resources used in the process of generating training data for the first machine learning model and the second machine learning model.

[0168] 812. Use the trained second machine learning model to perform a second training operation on the first machine learning model that has performed the first training operation.

[0169] In the embodiment of the present application, step 812 is an optional step. In one implementation, the training device may perform a second training operation on the first machine learning model that has performed the first training operation using the trained second machine learning model based on the idea of ​​reinforcement learning. Exemplarily, the training device may perform a second training operation on the first machine learning model that has performed the first training operation using the trained second machine learning model based on the idea of ​​RLHF or based on the idea of ​​RLAIF.

[0170] Furthermore, the training device may adopt a method when performing the second training operation, which may include a proximal policy optimization (PPO) algorithm, a PPO2 algorithm, a multi-threaded proximal policy optimization (DPPO) algorithm or other algorithms, which are not exhaustively listed in the embodiments of the present application.

[0171] In the embodiment of the present application, not only the first training operation is performed on the first machine learning model, but also the second training operation is performed on the first machine learning model using the trained second machine learning model, which is beneficial to further improve the effectiveness of the prompt information output by the first machine learning model; and the second machine learning model is used to score "malicious questions and prompts corresponding to malicious questions", which is beneficial to urge the first machine learning model to learn how to improve the security of malicious questions, and thus help improve the security of the user's interaction process with LLM.

[0172] In another implementation, the training device may perform a second training operation on the first machine learning model in a supervised manner. Exemplarily, after the training device generates a prediction prompt corresponding to the second question through the first machine learning model, the second question and the aforementioned prediction prompt may be input into the trained second machine learning model to obtain a prediction score corresponding to the "second question and prediction prompt" generated by the trained second machine learning model; the second training operation is performed on the first machine learning model according to the prediction score corresponding to the "second question and prediction prompt" and the fourth loss function, and the purpose of training using the fourth loss function includes improving the prediction score corresponding to the "second question and prediction prompt".

[0173] It should be noted that in the above embodiment, only steps 801 to 806 and steps 807 to 812 are executed by training devices. In other embodiments, steps 801 to 806 and steps 807 to 812 can also be executed by different devices. The specific settings can be combined with the actual application scenario, and are not limited in the embodiments of the present application.

[0174] exist Figures 1 to 8On the basis of the corresponding embodiments, in order to better implement the above solutions of the embodiments of the present application, the following also provides related devices for implementing the above solutions. Fig. 9 , Fig. 9 A structural schematic diagram of a data processing device provided in an embodiment of the present application, the data processing device 900 includes: an acquisition module 901, used to obtain a first question input by a user; a generation module 902, used to generate a first prompt corresponding to the first question through a trained first machine learning model; a processing module 903, used to input the first question and the first prompt into a large language model LLM to obtain a first answer, or to send the first question and the first prompt, and the first question and the first prompt are used to be input into the LLM to obtain a first answer.

[0175] Optionally, the first prompt is used to enhance the security of the first question.

[0176] Optionally, the acquisition module 901 is specifically used to: obtain the question input by the user; when the question input by the user does not belong to the black library or the white library, determine that the question input by the user is the first question, wherein when the question input by the user belongs to the black library, the question input by the user will be rejected, and when the question input by the user belongs to the white library, the white library is also used to provide an answer corresponding to the question input by the user.

[0177] Optionally, the data processing device 900 further includes: a connection module 904, configured to connect the first question and the first prompt to obtain the first information. The processing module 903 is specifically configured to: input the first question and the first prompt into the large language model LLM, including: input the first information into the LLM; or, send the first question and the first prompt, the first question and the first prompt are used to be input into the LLM, including: send the first information, the first information is used to be input into the LLM.

[0178] Optionally, the data processing device 900 is applied to an execution device, and the execution device is deployed with third information, and the third information includes prompts corresponding to each category of questions in at least one category. The generation module 902 is specifically used to input the first question into the trained first machine learning model, obtain the predicted category of the first question output by the trained first machine learning model, and the predicted category is included in at least one category; obtain a first prompt corresponding to the predicted category from the third information.

[0179] Optionally, the data processing device 900 is applied to an execution device, and a plurality of combination information is deployed in the execution device, each combination information includes a fourth question and a prompt corresponding to the fourth question. The generation module 902 is specifically used for:

[0180] Determine the similarity between the first question and the fourth question in the multiple combined information through the trained first machine learning model; obtain at least one first combined information corresponding to the first question from the multiple combined information based on the similarity between the first question and the fourth question; and determine a first prompt corresponding to the first question based on the first combined information.

[0181] Optionally, generation module 902 is specifically used to input the first question into the first machine learning model to obtain a first prompt output by the first machine learning model, wherein the training phase of the first machine learning model adopts the second question, the expected prompt corresponding to the second question, and the loss function, the second question is a malicious question, the expected prompt corresponding to the second question is pre-stored, the loss function indicates the similarity between the predicted prompt corresponding to the second question and the expected prompt, and the predicted prompt is obtained by the first machine learning model based on the second question during the training phase.

[0182] Optionally, the second question is obtained based on the third question, the third question corresponds to the third answer, both the third question and the third answer comply with security requirements, the second question is a malicious question corresponding to the third question, and the expected prompt corresponding to the second question is obtained from at least one alternative prompt corresponding to the second question based on similarity information; wherein the similarity information indicates the similarity between each second answer and the third answer in at least one second answer, each second answer is obtained based on a second information, and a second information includes the third question and an alternative prompt.

[0183] It should be noted that the information interaction, execution process, etc. between the modules / units in the data processing device 900 are the same as those in the present application. Figures 3 to 8 The corresponding method embodiments are based on the same concept. For specific contents, please refer to the description in the method embodiments shown above in this application, which will not be repeated here.

[0184] Please continue reading Fig.10 , Fig.10 Another structural schematic diagram of a data processing device provided in an embodiment of the present application, the data processing device 1000 includes: an acquisition module 1001, used to obtain a second question and an expected prompt corresponding to the second question; an input module 1002, used to input the second question into a first machine learning model to obtain a predicted prompt corresponding to the second question output by the first machine learning model; a training module 1003, used to perform a first training operation on the first machine learning model according to a first loss function to obtain a trained first machine learning model, the first loss function indicates the similarity between the predicted prompt corresponding to the second question and the expected prompt, and the trained first machine learning model is used to generate a first prompt for the first question input by the user.

[0185] Optionally, the first prompt is used to enhance the security of the first question.

[0186] Optionally, the acquisition module 1001 is also used to obtain a third question and a third answer corresponding to the third question, and the third question and the third answer both comply with security requirements; the data processing device 1000 also includes: a generation module 1004, used to generate a second question, and the second question is a malicious question corresponding to the third question; the acquisition module 1001 is also used to obtain at least one alternative prompt corresponding to the second question; the acquisition module 1001 is also used to obtain at least one second information corresponding to the at least one alternative prompt, and each second information includes a second question and an alternative prompt; the input module 1002 is also used to input each second information in the at least one second information into the large language model LLM to obtain a second answer corresponding to each second information; the acquisition module 1001 is also used to obtain an expected prompt corresponding to the second question from multiple alternative prompts based on the similarity between each second answer in the at least one second answer corresponding to the at least one second information and the third answer.

[0187] Optionally, the acquisition module 1001 is also used to obtain a trained second machine learning model, wherein the training process of the second machine learning model adopts a first training sample and a second loss function, the first training sample includes a first malicious question and a prompt corresponding to the first malicious question, the output of the second machine learning model during the training process includes a predicted score corresponding to the first training sample, and the second loss function indicates the similarity between the predicted score corresponding to the first training sample and the expected score; the training module 1003 is also used to perform a second training operation on the first machine learning model that has performed the first training operation using the trained second machine learning model based on the idea of ​​reinforcement learning.

[0188] Optionally, the acquisition module 1001 is also used to obtain a trained second machine learning model, wherein the training process of the second machine learning model adopts a second training sample and a second loss function, the second training sample includes a second question and an alternative prompt corresponding to the second question, the output of the second machine learning model during the training process includes a predicted score corresponding to the second training sample, the second loss function indicates the similarity between the predicted score corresponding to the second training sample and the expected score, and the expected score corresponding to the second training sample is obtained based on the similarity between the second answer and the third answer; the training module 1003 is also used to perform a second training operation on the first machine learning model that has performed the first training operation using the trained second machine learning model based on the idea of ​​reinforcement learning.

[0189] It should be noted that the information interaction, execution process, etc. between the modules / units in the data processing device 1000 are the same as those in the present application. Figures 3 to 8The corresponding method embodiments are based on the same concept. For specific contents, please refer to the description in the method embodiments shown above in this application, which will not be repeated here.

[0190] Next, an execution device provided by an embodiment of the present application is introduced. Fig.11 , Fig.11 A schematic diagram of a structure of an execution device provided in an embodiment of the present application. Specifically, the execution device 1100 includes: a receiver 1101, a transmitter 1102, a processor 1103 and a memory 1104 (wherein the number of processors 1103 in the execution device 1100 may be one or more, Fig.11 In the example of FIG. 1 , the processor 1103 may include an application processor 11031 and a communication processor 11032. In some embodiments of the present application, the receiver 1101, the transmitter 1102, the processor 1103 and the memory 1104 may be connected via a bus or other means.

[0191] The memory 1104 may include a read-only memory and a random access memory, and provides instructions and data to the processor 1103. A portion of the memory 1104 may also include a non-volatile random access memory (NVRAM). The memory 1104 stores processor and operation instructions, executable modules or data structures, or subsets thereof, or extended sets thereof, wherein the operation instructions may include various operation instructions for implementing various operations.

[0192] The processor 1103 controls the operation of the execution device. In a specific application, the various components of the execution device are coupled together through a bus system, wherein the bus system includes not only a data bus but also a power bus, a control bus, and a status signal bus, etc. However, for the sake of clarity, various buses are referred to as bus systems in the figure.

[0193] The method disclosed in the above embodiment of the present application can be applied to the processor 1103, or implemented by the processor 1103. The processor 1103 can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 1103. The above processor 1103 can be a general processor, a digital signal processor (digital signal processing, DSP), a microprocessor or a microcontroller, and can further include an application specific integrated circuit (application specific integrated circuit, ASIC), a field programmable gate array (field-programmable gate array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The processor 1103 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software modules in the decoding processor can be combined and executed. The software module may be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 1104, and the processor 1103 reads the information in the memory 1104 and completes the steps of the above method in combination with its hardware.

[0194] The receiver 1101 can be used to receive input digital or character information and generate signal input related to the relevant settings and function control of the execution device. The transmitter 1102 can be used to output digital or character information through the first interface; the transmitter 1102 can also be used to send instructions to the disk group through the first interface to modify the data in the disk group; the transmitter 1102 can also include a display device such as a display screen.

[0195] In the embodiment of the present application, the processor 1003 is used to execute Figures 3 to 8 The data processing method executed by the execution device in the corresponding embodiment. It should be noted that the specific manner in which the application processor 10031 in the processor 1003 executes the above steps is the same as that in the present application. Figures 3 to 8 The corresponding method embodiments are based on the same concept, and the technical effects they bring are the same as those in this application. Figures 3 to 8 The corresponding method embodiments are the same. For specific contents, please refer to the description in the method embodiments shown above in this application, which will not be repeated here.

[0196] The present application also provides a training device. Fig.12 , Fig.12 It is a structural diagram of a training device provided in an embodiment of the present application. Specifically, the training device 1200 is implemented by one or more servers. The training device 1200 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 1222 (for example, one or more processors) and memory 1232, and one or more storage media 1230 (for example, one or more mass storage devices) storing application programs 1242 or data 1244. Among them, the memory 1232 and the storage medium 1230 can be short-term storage or permanent storage. The program stored in the storage medium 1230 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the training device. Furthermore, the central processing unit 1222 can be configured to communicate with the storage medium 1230 to execute a series of instruction operations in the storage medium 1230 on the training device 1200.

[0197] The training device 1200 may also include one or more power supplies 1226, one or more wired or wireless network interfaces 1250, one or more input and output interfaces 1258, and / or, one or more operating systems 1241, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0198] In the embodiment of the present application, the central processor 1222 is used to execute Figures 3 to 8 The data processing method executed by the training device in the corresponding embodiment. It should be noted that the specific manner in which the central processor 1222 executes the above steps is the same as that in the present application. Figures 3 to 8 The corresponding method embodiments are based on the same concept, and the technical effects they bring are the same as those in this application. Figures 3 to 8 The corresponding method embodiments are the same. For specific contents, please refer to the description in the method embodiments shown above in this application, which will not be repeated here.

[0199] In an embodiment of the present application, a computer-readable storage medium is also provided. The computer-readable storage medium stores a program for performing signal processing. When the program is run on a computer, the computer performs the following steps: Figures 3 to 8 The steps performed by the training device in the method described in the embodiment shown, or causing the computer to perform the following steps Figures 3 to 8 The illustrated embodiment describes the steps performed by the device in the method.

[0200] The present application also provides a computer program product, which, when executed on a computer, enables the computer to execute the following Figures 3 to 8 The steps performed by the training device in the method described in the embodiment shown, or causing the computer to perform the following steps Figures 3 to 8 The illustrated embodiment describes the steps performed by the device in the method.

[0201] The execution device and training device provided in the embodiment of the present application may be a chip, which includes: a processing unit and a communication unit. The processing unit may be, for example, a processor, and the communication unit may be, for example, an input / output interface, a pin or a circuit. The processing unit may execute the computer execution instructions stored in the storage unit to enable the chip to execute the above Figures 3 to 8 The data processing method described in the embodiment shown. Optionally, the storage unit is a storage unit in the chip, such as a register, a cache, etc., and the storage unit can also be a storage unit located outside the chip in the wireless access device, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.

[0202] For details, please refer to Fig.13 , Fig.13 A schematic diagram of the structure of a chip provided in an embodiment of the present application, the chip can be expressed as a neural network processor NPU 130, NPU 130 is mounted on the host CPU (Host CPU) as a coprocessor, and the host CPU assigns tasks. The core part of the NPU is the operation circuit 1303, which is controlled by the controller 1304 to extract matrix data from the memory and perform multiplication operations.

[0203] In some implementations, the operation circuit 1303 includes multiple processing units (Process Engine, PE) inside. In some implementations, the operation circuit 1303 is a two-dimensional systolic array. The operation circuit 1303 can also be a one-dimensional systolic array or other electronic circuits capable of performing mathematical operations such as multiplication and addition. In some implementations, the operation circuit 1303 is a general-purpose matrix processor.

[0204] For example, assume there is an input matrix A, a weight matrix B, and an output matrix C. The operation circuit takes the corresponding data of matrix B from the weight memory 1302 and caches it on each PE in the operation circuit. The operation circuit takes the matrix A data from the input memory 1301 and performs matrix operation with matrix B, and the partial result or final result of the matrix is ​​stored in the accumulator 1308.

[0205] The unified memory 1306 is used to store input data and output data. The weight data is directly transferred to the weight memory 1302 through the direct memory access controller (DMAC) 1305. The input data is also transferred to the unified memory 1306 through the DMAC.

[0206] BIU stands for Bus Interface Unit, i.e., bus interface unit 1310 , which is used for interaction between AXI bus, DMAC and instruction fetch buffer (IFB) 1309 .

[0207] The bus interface unit 1310 (Bus Interface Unit, BIU for short) is used for the instruction fetch memory 1309 to obtain instructions from the external memory, and is also used for the storage unit access controller 1305 to obtain the original data of the input matrix A or the weight matrix B from the external memory.

[0208] DMAC is mainly used to transfer input data in the external memory DDR to the unified memory 1306 or to transfer weight data to the weight memory 1302 or to transfer input data to the input memory 1301.

[0209] The vector calculation unit 1307 includes multiple operation processing units, which further process the output of the operation circuit when necessary, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc. It is mainly used for non-convolutional / fully connected layer network calculations in neural networks, such as Batch Normalization, pixel-level summation, upsampling of feature planes, etc.

[0210] In some implementations, the vector calculation unit 1307 can store the processed output vector to the unified memory 1306. For example, the vector calculation unit 1307 can apply a linear function and / or a nonlinear function to the output of the operation circuit 1303, such as linear interpolation of the feature plane extracted by the convolution layer, and then, for example, a vector of accumulated values ​​to generate an activation value. In some implementations, the vector calculation unit 1307 generates a normalized value, a pixel-level summed value, or both. In some implementations, the processed output vector can be used as an activation input to the operation circuit 1303, for example, for use in a subsequent layer in a neural network.

[0211] An instruction fetch buffer 1309 connected to the controller 1304 is used to store instructions used by the controller 1304;

[0212] Unified memory 1306, input memory 1301, weight memory 1302 and instruction fetch memory 1309 are all on-chip memories. External memories are private to the NPU hardware architecture.

[0213] Among them, the operations of each layer in the above-mentioned machine learning model can be performed by the operation circuit 1303 or the vector calculation unit 1307.

[0214] The processor mentioned in any of the above places may be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of the program of the above-mentioned first aspect method.

[0215] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0216] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0217] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0218] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A data processing method, characterized in that: The method comprises: Get user input for the first question; Generate a first prompt corresponding to the first question through the trained first machine learning model; The first question and the first prompt are input into a large language model (LLM) to obtain a first answer, or the first question and the first prompt are sent, and the first question and the first prompt are used to be input into the LLM to obtain the first answer.

2. The method according to claim 1, characterized in that The first question of obtaining user input includes: Get user input questions; In the case that the question input by the user belongs to neither the black library nor the white library, the question input by the user is determined to be the first question, wherein, when the question input by the user belongs to the black library, the question input by the user will be rejected for reply, and when the question input by the user belongs to the white library, the white library is also used to provide an answer corresponding to the question input by the user.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: Connecting the first question and the first prompt to obtain first information; The inputting the first question and the first prompt into the large language model LLM includes: inputting the first information into the LLM; or the sending the first question and the first prompt, wherein the first question and the first prompt are used for input into the LLM, includes: sending the first information, wherein the first information is used for input into the LLM.

4. The method according to claim 1 or 2, characterized in that: The method is applied to an execution device, wherein third information is deployed in the execution device, wherein the third information includes a prompt corresponding to each category of questions in at least one category of questions, and wherein generating a first prompt corresponding to the first question by using a trained first machine learning model includes: Inputting the first question into the trained first machine learning model to obtain a predicted category of the first question output by the trained first machine learning model, wherein the predicted category is included in the at least one category; The first prompt corresponding to the predicted category is obtained from the third information.

5. The method according to claim 1 or 2, characterized in that: The method is applied to an execution device, wherein a plurality of combination information is deployed in the execution device, each of the combination information includes a fourth question and a prompt corresponding to the fourth question, and the first prompt corresponding to the first question is generated by using a trained first machine learning model, including: Determine the similarity between the first question and the fourth question in the plurality of combined information by using the trained first machine learning model; According to the similarity between the first question and the fourth question, acquiring at least one first combination of information corresponding to the first question from the plurality of combination of information; The first prompt corresponding to the first question is determined according to the first combination information.

6. The method according to claim 1 or 2, characterized in that: The generating, by means of the trained first machine learning model, a first prompt corresponding to the first question, comprises: The first question is input into the trained first machine learning model to obtain the first prompt output by the trained first machine learning model, wherein a second question, an expected prompt corresponding to the second question, and a loss function are used in the training phase of the first machine learning model, the second question is a malicious question, the expected prompt corresponding to the second question is pre-stored, the loss function indicates the similarity between the predicted prompt corresponding to the second question and the expected prompt, and the predicted prompt is obtained by the first machine learning model based on the second question in the training phase.

7. The method according to claim 6, characterized in that The second question is obtained based on the third question, the third question corresponds to a third answer, the third question and the third answer both meet security requirements, the second question is a malicious question corresponding to the third question, and the expected prompt corresponding to the second question is obtained from at least one alternative prompt corresponding to the second question based on similarity information; The similarity information indicates the similarity between each second answer in at least one second answer and the third answer, each second answer is obtained based on a second information, and one second information includes the third question and one alternative prompt.

8. The method according to claim 1 or 2, characterized in that: The first prompt is used to enhance the security of the first question.

9. A data processing method, characterized in that: The method comprises: Obtaining a second question and an expected prompt corresponding to the second question; Inputting the second question into the first machine learning model, and outputting a prediction prompt corresponding to the second question; A first training operation is performed on the first machine learning model according to a first loss function to obtain a trained first machine learning model, wherein the first loss function indicates the similarity between the predicted prompt corresponding to the second question and the expected prompt, and the trained first machine learning model is used to generate a first prompt for a first question input by a user.

10. The method according to claim 9, characterized in that The method further comprises: Obtaining a third question and a third answer corresponding to the third question, wherein both the third question and the third answer meet security requirements; generating the second question, where the second question is a malicious question corresponding to the third question; Obtaining at least one alternative prompt corresponding to the second question; Acquire at least one second information corresponding to the at least one alternative prompt, each second information including the second question and one of the alternative prompts; Input each of the at least one second information into a large language model LLM to obtain a second answer corresponding to each of the second information; The expected prompt corresponding to the second question is obtained from the multiple alternative prompts based on the similarity between each of the second answers in at least one second answer corresponding one-to-one to the at least one second information and the third answer.

11. The method according to claim 9 or 10, characterized in that: The first prompt is used to enhance the security of the first question.

12. A data processing device, characterized in that: The device comprises: An acquisition module, used for acquiring a first question input by a user; A generating module, configured to generate a first prompt corresponding to the first question by using a trained first machine learning model; A processing module is used to input the first question and the first prompt into a large language model (LLM) to obtain a first answer, or to send the first question and the first prompt, wherein the first question and the first prompt are used to be input into the LLM to obtain the first answer.

13. A data processing device, characterized in that: The device comprises: An acquisition module, used for acquiring a second question and an expected prompt corresponding to the second question; An input module, configured to input the second question into a first machine learning model, and obtain a prediction prompt corresponding to the second question output by the first machine learning model; A training module is used to perform a first training operation on the first machine learning model according to a first loss function to obtain a trained first machine learning model, wherein the first loss function indicates the similarity between the predicted prompt corresponding to the second question and the expected prompt, and the trained first machine learning model is used to generate a first prompt for a first question input by a user.

14. An execution device, characterized in that: comprising a processor and a memory, the processor being coupled to the memory, The memory is used to store programs; The processor is configured to execute the program in the memory so that the execution device executes the method according to any one of claims 1 to 8.

15. A training device, characterized in that comprising a processor and a memory, the processor being coupled to the memory, The memory is used to store programs; The processor is used to execute the program in the memory so that the training device performs the method as claimed in any one of claims 9 to 11.

16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program, and when the program is executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 11.

17. A computer program product, characterized in that The computer program product comprises a program, and when the program is run on a computer, the computer is caused to execute the method according to any one of claims 1 to 11.

Images