Model instruction detection, model task execution method, device and computer equipment

Abstract

The present application relates to a method, device and computer equipment for model instruction detection and model task execution. The method includes: obtaining candidate instructions of target task information of a target task model; extracting semantic features corresponding to the candidate instructions and target task information respectively, obtaining candidate instruction features and task information features; calculating the distance between the candidate instruction features and the task information features, obtaining the semantic accuracy corresponding to the candidate instructions; obtaining a test sample of the target task model, performing at least two task tests on the test sample and the candidate instruction through the target task model, obtaining at least two task test result information corresponding to the candidate instruction; calculating the discrete degree of at least two task test result information, and determining the test accuracy corresponding to the candidate instruction based on the discrete degree; determining the detection result of the candidate instruction based on the semantic accuracy and the test accuracy. The use of this method can improve the accuracy of model instruction detection.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a model instruction detection, model task execution method, device, computer equipment, storage medium and computer program product. Background Art

[0002] With the development of artificial intelligence technology, large artificial intelligence models have emerged, which are characterized by a large number of parameters and complex structures. Large models are usually able to capture complex patterns and relationships in the data, thereby achieving excellent performance on various tasks. And in order to help large models better understand the purpose of the task and generate more accurate answers, the input instructions (Prompt) of the large model are usually provided through instruction engineering to guide the model to perform tasks and generate corresponding outputs. At present, in order to improve the quality of input instructions for large models, it is usually necessary to use pre-set evaluation rules to evaluate the quality of the instructions and then filter out the input instructions. However, there is a problem of low detection accuracy when detecting the quality of instructions through evaluation rules. It is easy to use poor quality as model input instructions, which can easily lead to errors in the execution of large model tasks and waste execution resources. Summary of the invention

[0003] Based on this, it is necessary to provide a model instruction detection, model task execution method, device, computer equipment, computer-readable storage medium and computer program product that can improve the accuracy of instruction detection, thereby improve the accuracy of model task execution and save execution resources in response to the above-mentioned technical problems.

[0004] In one aspect, the present application provides a model instruction detection method. The method comprises:

[0005] Obtaining target task information of the target task model and candidate instructions of the target task model;

[0006] Semantic features corresponding to candidate instructions and target task information are extracted respectively to obtain candidate instruction features and task information features;

[0007] Calculate the distance between the candidate instruction features and the task information features to obtain the semantic accuracy corresponding to the candidate instruction;

[0008] Obtain a test sample of the target task model, perform at least two task tests on the test sample and the candidate instruction through the target task model, and obtain at least two task test result information corresponding to the candidate instruction;

[0009] Calculating the discreteness of the test result information of at least two tasks, and determining the test accuracy corresponding to the candidate instruction based on the discreteness;

[0010] Based on the semantic accuracy and the test accuracy, the candidate instructions are determined as the detection results of the input instructions of the target task model.

[0011] On the other hand, the present application also provides a model instruction detection device. The device comprises:

[0012] An instruction acquisition module, used to acquire target task information of the target task model and candidate instructions of the target task model;

[0013] A feature extraction module is used to extract semantic features corresponding to candidate instructions and target task information, respectively, to obtain candidate instruction features and task information features;

[0014] A distance calculation module is used to calculate the distance between the candidate instruction features and the task information features to obtain the semantic accuracy corresponding to the candidate instruction;

[0015] A testing module is used to obtain a test sample of a target task model, perform at least two task tests on the test sample and the candidate instruction through the target task model, and obtain at least two task test result information corresponding to the candidate instruction;

[0016] A discrete calculation module, used to calculate the discreteness of the test result information of at least two tasks, and determine the test accuracy corresponding to the candidate instruction based on the discreteness;

[0017] The result determination module is used to determine the candidate instruction as the detection result of the input instruction of the target task model based on the semantic accuracy and the test accuracy.

[0018] On the other hand, the present application also provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0019] Obtaining target task information of the target task model and candidate instructions of the target task model;

[0020] Semantic features corresponding to candidate instructions and target task information are extracted respectively to obtain candidate instruction features and task information features;

[0021] Calculate the distance between the candidate instruction features and the task information features to obtain the semantic accuracy corresponding to the candidate instruction;

[0022] Obtain a test sample of the target task model, perform at least two task tests on the test sample and the candidate instruction through the target task model, and obtain at least two task test result information corresponding to the candidate instruction;

[0023] Calculating the discreteness of the test result information of at least two tasks, and determining the test accuracy corresponding to the candidate instruction based on the discreteness;

[0024] Based on the semantic accuracy and the test accuracy, the candidate instructions are determined as the detection results of the input instructions of the target task model.

[0025] On the other hand, the present application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0026] Obtaining target task information of the target task model and candidate instructions of the target task model;

[0027] Semantic features corresponding to candidate instructions and target task information are extracted respectively to obtain candidate instruction features and task information features;

[0028] Calculate the distance between the candidate instruction features and the task information features to obtain the semantic accuracy corresponding to the candidate instruction;

[0029] Obtain a test sample of the target task model, perform at least two task tests on the test sample and the candidate instruction through the target task model, and obtain at least two task test result information corresponding to the candidate instruction;

[0030] Calculating the discreteness of the test result information of at least two tasks, and determining the test accuracy corresponding to the candidate instruction based on the discreteness;

[0031] Based on the semantic accuracy and the test accuracy, the candidate instructions are determined as the detection results of the input instructions of the target task model.

[0032] On the other hand, the present application also provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0033] Obtaining target task information of the target task model and candidate instructions of the target task model;

[0034] Semantic features corresponding to candidate instructions and target task information are extracted respectively to obtain candidate instruction features and task information features;

[0035] Calculate the distance between the candidate instruction features and the task information features to obtain the semantic accuracy corresponding to the candidate instruction;

[0036] Obtain a test sample of the target task model, perform at least two task tests on the test sample and the candidate instruction through the target task model, and obtain at least two task test result information corresponding to the candidate instruction;

[0037] Calculating the discreteness of the test result information of at least two tasks, and determining the test accuracy corresponding to the candidate instruction based on the discreteness;

[0038] Based on the semantic accuracy and the test accuracy, the candidate instructions are determined as the detection results of the input instructions of the target task model.

[0039] The above-mentioned model instruction detection method, device, computer equipment, storage medium and computer program product obtain the target task information of the target task model and the candidate instructions of the target task model. Then extract the semantic features corresponding to the candidate instructions and the target task information respectively, obtain the candidate instruction features and the task information features, and then calculate the distance between the candidate instruction features and the task information features to obtain the semantic accuracy corresponding to the candidate instructions. Then obtain the test sample of the target task model, perform at least two task tests on the test sample and the candidate instruction through the target task model, obtain at least two task test result information corresponding to the candidate instruction, then calculate the discrete degree of at least two task test result information, and determine the test accuracy corresponding to the candidate instruction based on the discrete degree. Finally, determine the detection result of the candidate instruction as the input instruction of the target task model based on the semantic accuracy and the test accuracy. That is, the accuracy of the candidate instruction can be measured from different angles, and then the detection result of the candidate instruction is determined, which improves the accuracy of the model instruction detection, and then the input instruction of the target task model can be determined according to the detection result, that is, the accuracy of the input instruction of the target task model is improved, and then the accuracy of the model task execution can be improved, saving execution resources.

[0040] In one aspect, the present application provides a model task execution method. The method comprises:

[0041] Obtain model input data and input instructions;

[0042] Input the model input data and input instructions into the target task model, guide the model input data to perform the task through the input instructions, and obtain the task execution result information;

[0043] Among them, the input instruction is determined based on the detection result of the candidate instruction as the input instruction of the target task model; the detection result is determined based on the semantic accuracy and the test accuracy; the semantic accuracy is obtained by calculating the distance between the candidate instruction features of the candidate instruction and the task information features of the target task information; the test accuracy is obtained by performing at least two task tests on the test samples and the candidate instructions through the target task model, obtaining at least two task test result information corresponding to the candidate instructions, and calculating the discrete degree of at least two task test result information; the candidate instructions and the target task information are corresponding to the target task model.

[0044] On the other hand, the present application also provides a model task execution device. The device comprises:

[0045] A data acquisition module, used to obtain model input data and input instructions;

[0046] A task execution module is used to input model input data and input instructions into a target task model, guide the model input data to perform task execution through the input instructions, and obtain task execution result information; wherein, the input instruction is determined based on the detection result of the candidate instruction as the input instruction of the target task model; the detection result is determined based on semantic accuracy and test accuracy; the semantic accuracy is obtained by calculating the distance between the candidate instruction feature of the candidate instruction and the task information feature of the target task information; the test accuracy is obtained by performing at least two task tests on the test sample and the candidate instruction through the target task model, obtaining at least two task test result information corresponding to the candidate instruction, and calculating the discrete degree of at least two task test result information; the candidate instruction and the target task information are corresponding to the target task model.

[0047] On the other hand, the present application also provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0048] Obtain model input data and input instructions;

[0049] Input the model input data and input instructions into the target task model, guide the model input data to perform the task through the input instructions, and obtain the task execution result information;

[0050] Among them, the input instruction is determined based on the detection result of the candidate instruction as the input instruction of the target task model; the detection result is determined based on the semantic accuracy and the test accuracy; the semantic accuracy is obtained by calculating the distance between the candidate instruction features of the candidate instruction and the task information features of the target task information; the test accuracy is obtained by performing at least two task tests on the test samples and the candidate instructions through the target task model, obtaining at least two task test result information corresponding to the candidate instructions, and calculating the discrete degree of at least two task test result information; the candidate instructions and the target task information are corresponding to the target task model.

[0051] On the other hand, the present application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0052] Obtain model input data and input instructions;

[0053] Input the model input data and input instructions into the target task model, guide the model input data to perform the task through the input instructions, and obtain the task execution result information;

[0054] Among them, the input instruction is determined based on the detection result of the candidate instruction as the input instruction of the target task model; the detection result is determined based on the semantic accuracy and the test accuracy; the semantic accuracy is obtained by calculating the distance between the candidate instruction features of the candidate instruction and the task information features of the target task information; the test accuracy is obtained by performing at least two task tests on the test samples and the candidate instructions through the target task model, obtaining at least two task test result information corresponding to the candidate instructions, and calculating the discrete degree of at least two task test result information; the candidate instructions and the target task information are corresponding to the target task model.

[0055] On the other hand, the present application also provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0056] Obtain model input data and input instructions;

[0057] Input the model input data and input instructions into the target task model, guide the model input data to perform the task through the input instructions, and obtain the task execution result information;

[0058] Among them, the input instruction is determined based on the detection result of the candidate instruction as the input instruction of the target task model; the detection result is determined based on the semantic accuracy and the test accuracy; the semantic accuracy is obtained by calculating the distance between the candidate instruction features of the candidate instruction and the task information features of the target task information; the test accuracy is obtained by performing at least two task tests on the test samples and the candidate instructions through the target task model, obtaining at least two task test result information corresponding to the candidate instructions, and calculating the discrete degree of at least two task test result information; the candidate instructions and the target task information are corresponding to the target task model.

[0059] The above-mentioned model task execution method, device, computer equipment, storage medium and computer program product obtain model input data and input instructions, then input the model input data and input instructions into the target task model, guide the model input data to perform task execution through the input instructions, and obtain task execution result information. The input instruction is determined based on the detection result of the candidate instruction as the input instruction of the target task model. The detection result is determined based on the semantic accuracy and test accuracy. The semantic accuracy is obtained by calculating the distance between the candidate instruction feature of the candidate instruction and the task information feature of the target task information; the test accuracy is obtained by performing at least two task tests on the test sample and the candidate instruction through the target task model, obtaining at least two task test result information corresponding to the candidate instruction, and calculating the discrete degree of at least two task test result information. That is, the detection result of the candidate instruction can be determined by measuring the accuracy of the candidate instruction from different angles, and then the input instruction of the target task model can be determined according to the detection result, thereby improving the accuracy of the input instruction obtained. Then, the input instruction and input data are used to perform task execution through the target task model, thereby improving the accuracy of the model task execution, thereby reducing the situation of model task execution errors and saving execution resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 A diagram of an application environment of a model instruction detection method in an embodiment;

[0061] Figure 2 is a schematic flow chart of a model instruction detection method in one embodiment;

[0062] Figure 3 It is a schematic diagram of a framework of feature extraction in a specific embodiment;

[0063] Figure 4 A schematic diagram of the architecture of a coding neural network in a specific embodiment;

[0064] Figure 5 A schematic diagram of a flow chart for obtaining the test accuracy in one embodiment;

[0065] Figure 6 A schematic diagram of a process for obtaining a detection result in one embodiment;

[0066] Figure 7 A schematic diagram of an architecture for obtaining the accuracy of test semantics in a specific embodiment;

[0067] Figure 8 A schematic diagram of a flow chart of a model task execution method in one embodiment;

[0068] Fig. 9 A schematic diagram of a framework for executing a model task in a specific embodiment;

[0069] Fig.10 It is a schematic diagram of the principle of model instruction detection in a specific embodiment;

[0070] Fig.11 is a flow chart of a model instruction detection method in a specific embodiment;

[0071] Fig.12 is a structural block diagram of a model instruction detection device in an embodiment;

[0072] Fig.13 is a structural block diagram of a model task execution device in an embodiment;

[0073] Fig.14 is an internal structure diagram of a computer device in one embodiment;

[0074] Fig.15 FIG. 4 is a diagram showing the internal structure of a computer device in another embodiment. DETAILED DESCRIPTION

[0075] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0076] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0077] The model instruction detection method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other servers. The server 104 can obtain the target task information of the target task model and the candidate instructions of the target task model from the terminal 102. Then the server 104 extracts the semantic features corresponding to the candidate instructions and the target task information respectively, and obtains the candidate instruction features and the task information features. The server 104 calculates the distance between the candidate instruction features and the task information features to obtain the semantic accuracy corresponding to the candidate instructions; the server 104 obtains the test sample of the target task model, and performs at least two task tests on the test sample and the candidate instructions through the target task model to obtain at least two task test result information corresponding to the candidate instructions. The server 104 calculates the discrete degree of at least two task test result information, and determines the test accuracy corresponding to the candidate instruction based on the discrete degree. The server 104 determines the detection result of the candidate instruction as the input instruction of the target task model based on the semantic accuracy and the test accuracy. Finally, the server 104 can return the candidate instruction as the detection result of the input instruction of the target task model to the terminal 102. Among them, the terminal 102 can be, but is not limited to, various desktop computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services. The terminal and the server can be directly or indirectly connected via wired or wireless communication, and this application does not limit this.

[0078] In one embodiment, Figure 2 As shown, a model instruction detection method is provided, which is applied to Figure 1 The server in the example is used for explanation. It can be understood that the method can also be applied to a terminal, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0079] S202, obtaining target task information of the target task model and candidate instructions of the target task model.

[0080] Among them, the target task model refers to a large artificial intelligence model that needs to perform the target task, for example, it can be a model for performing image tasks, it can be a model for performing language tasks, it can be a model for performing audio tasks, it can be a model for performing video tasks, and so on. Preferably, the target task model can be a language model, for example, a model for performing text tasks. The target task model can be a trained model. When deploying and applying the target task model, the input instructions of the target task model can be obtained by detecting and deploying it, thereby further improving the accuracy of model execution. The target task model can also be a pre-trained model. The input instructions of the target task model can be obtained by detecting and further training the model, thereby improving the accuracy of model training and thus improving the accuracy of model execution.

[0081] The target task information refers to the specific information of the target task to be performed by the target task model, and the target task information may include the task element information and task background information of the target task, etc. The task element information refers to the key features and components that must be possessed to constitute a complete task, and the task element information includes but is not limited to at least one of the model task name, model task goal, model task result, model task execution method, and the number of model task results. The task background information may be information such as the reason for task execution and the basis for task execution. Among them, the target task refers to the task to be performed by the model, which may be a task for processing images, a task for processing natural language text, a task for processing audio, a task for processing videos, etc. The processing may be recognition, detection, classification, and generation, etc. Preferably, the target task may be a task for processing natural language text.

[0082] Candidate instructions are instructions that need to be tested to see if they can be used as input instructions for the target task model. Instructions are information used to instruct the artificial intelligence model to perform the corresponding task. Instructions can be text instructions, image instructions, voice instructions, etc. Preferably, instructions can be text instructions, such as a question, a sentence, a paragraph of text, or a group of instructions, etc. Instructions can help the model better understand the intent of the input and respond accordingly, so as to better understand and complete the task.

[0083] Specifically, the server can obtain the target task information of the target task model uploaded by the terminal and the candidate instructions of the target task model. The server can also directly obtain the target task information of the target task model and the candidate instructions of the target task model from the database. The server can also obtain the target task information of the target task model and generate the candidate instructions of the target task model.

[0084] In some embodiments, the server may first obtain the target task information of the target task model, and then return the target task information to the terminal, the terminal displays the target task information, and obtains the candidate instructions input according to the target task information, and then uploads the candidate instructions to the server. At this point, the server obtains the target task information of the target task model and the candidate instructions of the target task model.

[0085] In some embodiments, the server may obtain the target task information of the target task model and the candidate instructions of the target task model sent by the service provider providing the target task service, and finally send the detection result of the candidate instructions to the service provider of the target task service.

[0086] S204, extracting semantic features corresponding to the candidate instructions and the target task information respectively, to obtain candidate instruction features and task information features.

[0087] Among them, the candidate instruction feature is a feature used to characterize the semantic information contained in the candidate instruction, which can be represented by a feature vector. The task information feature is a feature used to characterize the semantic information contained in the target task information, which can be represented by a feature vector. Semantic features are usually features used to characterize the basic concepts or meanings expressed by the candidate instruction or target task information. The feature vector is a mathematical representation used to convert information into a set of numerical values ​​that can be processed by a computer. The information can be natural language text, images, speech, video, etc. For example, in natural language processing (NLP), the feature vector captures the key information of the text, such as the importance of words, context, grammatical structure and semantic content, so that the text can be analyzed and understood through the feature vector, and tasks such as classification, search, and sentiment analysis can be performed.

[0088] Specifically, the server can extract the semantic features corresponding to the candidate instructions through a language feature extraction algorithm to obtain the candidate instruction features, and at the same time extract the semantic features corresponding to the target task information through a semantic feature extraction algorithm to obtain the task information features. Among them, the semantic feature extraction algorithm may include a bag-of-words model, TF-IDF (term frequency-inverse document frequency) and a word embedding algorithm, etc. The word embedding algorithm may be a Word2Vec (converting words into vectors) algorithm or a BERT (a language representation model) feature extraction algorithm, etc.

[0089] In some embodiments, the server may input the candidate instructions and the target task information into a neural network model for semantic feature extraction to extract semantic features. The neural network model for semantic feature extraction may be pre-trained, for example, the neural network model for semantic feature extraction may be a pre-trained neural network model for encoding.

[0090] S206, calculating the distance between the candidate instruction feature and the task information feature to obtain the semantic accuracy corresponding to the candidate instruction.

[0091] Among them, the semantic accuracy is used to characterize the semantic distance between the candidate instruction and the target task information, and the semantic distance is positively correlated with the semantic accuracy, that is, the closer the semantic distance, the higher the semantic accuracy, and the farther the semantic distance, the lower the semantic accuracy. When the semantic accuracy is smaller, it indicates that the candidate instruction to be evaluated is semantically farther from the information to be expressed by the target task information, that is, it is used to characterize the deviation caused by reasons such as whether the information expression of the candidate instruction is accurate, the information elements are complete, or the logic is correct. When the semantic accuracy is larger, it indicates that the candidate instruction to be evaluated is semantically closer to the information to be expressed by the target task information, which means that the candidate instruction is more accurate, the quality is higher, and the possibility of the candidate instruction as the input instruction of the target task model is higher. That is, the greater the semantic accuracy, the more accurate the information expression of the candidate instruction, the more complete the information elements, and the higher the logical correctness.

[0092] Specifically, the server can use a semantic distance calculation algorithm between feature vectors to calculate the distance between the candidate instruction feature and the task information feature, wherein the semantic distance can be determined by calculating the ratio between the number of identical feature values ​​in the candidate instruction feature and the task information feature and the number of total feature values, and the semantic distance can also be determined by calculating the positional relationship between the candidate instruction feature and the task information feature in the dictionary. Then, the semantic accuracy corresponding to the candidate instruction is determined based on the distance between the candidate instruction feature and the task information feature. For example, the distance between the candidate instruction feature and the task information feature can be directly used as the semantic accuracy corresponding to the candidate instruction, or the distance between the candidate instruction feature and the task information feature can be normalized to obtain the semantic accuracy corresponding to the candidate instruction.

[0093] In some embodiments, the server may also use a similarity algorithm to calculate the similarity between the candidate instruction features and the task information features, and then determine the semantic accuracy corresponding to the candidate instruction based on the similarity. The similarity may be positively correlated with the semantic accuracy, that is, the higher the similarity, the higher the semantic accuracy, and the lower the similarity, the lower the semantic accuracy.

[0094] S208, obtaining a test sample of the target task model, performing at least two task tests on the test sample and the candidate instructions through the target task model, and obtaining at least two task test result information corresponding to the candidate instructions.

[0095] Among them, the test sample refers to the input data of the target task model used in the test. The input data is the object that the target task model needs to perform the task. The input data is different for different tasks. For example, the input data of the text task can be text, the input data of the image task can be image, and the input data of the audio task can be audio. The task test result information refers to the information output by the target task model after the task is performed. For example, the image recognition result information output by the image recognition task model after the image recognition task is performed, and the text category result information output by the text classification task model after the text classification task is performed. When the accuracy of the candidate instruction is low, that is, the semantic accuracy to be expressed by the candidate instruction is low or the information elements are incomplete or there are problems with the logic, the target task model will be unstable when performing the task. That is, when the candidate instruction and the test sample perform the task in the target task model, due to the low accuracy of the candidate instruction, there will be a situation where the model cannot be accurately guided to perform the task on the test sample, which will lead to different results of the model output. When the accuracy of the candidate instruction is higher, when the target task model is performing the task, the candidate instruction can accurately guide the model to perform the task on the test sample, so that the results output by the model can be more concentrated in the semantic space, that is, there will be more cases where the results are the same. That is, by using the candidate instruction to perform multiple tests on the same test sample, the task test result information of all tests is obtained, and then all the task test result information is used to measure the accuracy of the candidate instruction, the accuracy of the candidate instruction can be further improved.

[0096] Specifically, the server can obtain the test sample of the target task model from the database, the server can also obtain the test sample uploaded by the terminal, and the server can also obtain the test sample from the service provider providing data services. At this time, the server performs at least two task tests on the test sample and the candidate instruction through the target task model, and obtains at least two task test result information corresponding to the candidate instruction. That is, the server inputs the test sample and the candidate instruction into the target task model, and the target task model performs the task execution on the test sample according to the candidate instruction, and obtains the task test result information corresponding to the candidate instruction. At this time, the pre-set number of task tests is obtained, and then the test sample and the candidate instruction are continuously input into the target task model according to the number of task tests and continuously repeated. That is, the server uses the test sample and the candidate instruction as the model input and performs the same number of task executions according to the number of task tests, and obtains the same number of task test result information as the number of task tests.

[0097] In some embodiments, the server may obtain a test sample set of the target task model, and then select a test sample from the test sample set. The server may also perform a task test on each test sample in the test sample set together with the candidate instruction through the target task model to obtain task test result information.

[0098] S210, calculating the discreteness of the test result information of at least two tasks, and determining the test accuracy corresponding to the candidate instruction based on the discreteness.

[0099] Among them, the degree of discreteness is used to characterize the degree of difference between the test result information of each task. When the degree of discreteness of the task test result information of the same test sample is smaller, it means that the difference between all the task test result information is smaller, that is, the possibility that all the task test result information is the same is higher. At this time, the possibility that the candidate instruction is used as the target task model input instruction is higher. When the degree of discreteness of the task test result information of the same test sample is larger, it means that the difference between all the task test result information is larger, that is, the possibility that all the task test result information is the same is smaller. At this time, the possibility that the candidate instruction is used as the target task model input instruction is lower. The test accuracy is used to characterize the semantic accuracy of the candidate instruction obtained by testing the test sample using the target task model. The higher the test accuracy, the smaller the degree of difference in the task test result information obtained by testing the same test sample through the candidate instruction, that is, the higher the accuracy of the candidate instruction. The test accuracy has an inverse correlation with the degree of discreteness.

[0100] Specifically, the server calculates the degree of difference between all task test result information of the test sample to obtain the degree of discreteness, wherein the average difference between all task test result information can be calculated to obtain the degree of discreteness, the standard deviation between all task test result information can be calculated to obtain the degree of discreteness, the range between all task test result information can be calculated to obtain the degree of discreteness, etc., and the coefficient of variation between all task test result information can also be calculated to obtain the degree of discreteness. Then the server determines the test accuracy corresponding to the candidate instruction according to the degree of discreteness of all task test result information. For example, the test accuracy corresponding to the candidate instruction can be directly determined according to the degree of discreteness, or the test accuracy corresponding to the candidate instruction can be determined after the degree of discreteness is normalized. The degree of discreteness can also be combined with other degrees of discreteness obtained by testing the candidate instruction using other test samples to determine the test accuracy corresponding to the candidate instruction, etc.

[0101] S212, determining the candidate instruction as the detection result of the target task model input instruction based on the semantic accuracy and the test accuracy.

[0102] The detection result refers to the result of detecting whether the candidate instruction is the input instruction of the target task model, which may include the result that the candidate instruction is the input instruction of the target task model and the result that the candidate instruction is not the input instruction of the target task model. The detection result is used to characterize the possibility of the candidate instruction as the input instruction of the target task model. The higher the possibility, the more likely the corresponding candidate instruction is to be the input instruction of the target task model.

[0103] Specifically, the server can determine the integrated accuracy of the candidate instruction based on the semantic accuracy and the test accuracy, and then determine whether the candidate instruction can be used as the input instruction of the target task model based on the integrated accuracy. The server determines that when the integrated accuracy meets the pre-set input instruction condition, a detection result is obtained that the candidate instruction is the input instruction of the target task model. When the integrated accuracy does not meet the pre-set input instruction condition, a detection result is obtained that the candidate instruction is not the input instruction of the target task model and needs to be eliminated. The input instruction condition can be a condition for the candidate instruction to be used as an input instruction, which can include but is not limited to the integrated accuracy exceeding a pre-set threshold or the integrated accuracy being the maximum accuracy among all candidate instructions. The input instruction condition can be obtained and set through testing.

[0104] The above-mentioned model instruction detection method, device, computer equipment, storage medium and computer program product obtain the target task information of the target task model and the candidate instructions of the target task model. Then, the semantic features corresponding to the candidate instructions and the target task information are extracted respectively to obtain the candidate instruction features and the task information features, and then the distance between the candidate instruction features and the task information features is calculated to obtain the semantic accuracy corresponding to the candidate instructions. Then, the test sample of the target task model is obtained, and at least two task tests are performed on the test sample and the candidate instruction through the target task model to obtain at least two task test result information corresponding to the candidate instruction, and then the discrete degree of the at least two task test result information is calculated, and the test accuracy corresponding to the candidate instruction is determined based on the discrete degree. Finally, the candidate instruction is determined as the detection result of the input instruction of the target task model based on the semantic accuracy and the test accuracy. That is, the detection result of the candidate instruction can be determined by measuring the accuracy obtained by the candidate instruction from different angles, which improves the accuracy of the model instruction detection, and then the input instruction of the target task model can be determined according to the detection result, that is, the accuracy of the input instruction of the target task model is improved, and then the accuracy of the model task execution can be improved, saving execution resources.

[0105] In one embodiment, S204, respectively extracting semantic features corresponding to candidate instructions and target task information to obtain candidate instruction features and task information features includes:

[0106] The candidate instructions and target task information are vectorized respectively to obtain the initial features of the candidate instructions and the initial features of the task information; the initial features of the candidate instructions are self-attention encoded to obtain the candidate instruction features, and the initial features of the task information are self-attention encoded to obtain the task information features.

[0107] In this embodiment, when extracting semantic features, it is usually necessary to vectorize the candidate instructions and target task information to be feature extracted, that is, the candidate instructions and target task information can be vectorized using a vectorization algorithm to obtain the initial features of the candidate instructions and the initial features of the task information. The initial features of the candidate instructions are feature vectors obtained after the candidate instructions are vectorized, and the initial features of the task information are feature vectors obtained after the target task information is vectorized. The vectorization algorithm refers to an algorithm that converts non-numerical data into numerical data. For example, the vectorization algorithm may include a hot encoding algorithm, a dictionary algorithm, and a neural network algorithm, etc. For example, text information can be vectorized by a hot encoding algorithm, and image information can be vectorized by a neural network algorithm. Then the server performs self-attention encoding on the initial features of the candidate instructions according to the self-attention mechanism to obtain the candidate instruction features, and at the same time, performs self-attention encoding on the initial features of the task information to obtain the task information features. The self-attention mechanism can extract the dependency information between different elements in the input data. The self-attention mechanism calculates the correlation between the input vectors and weights the input vectors according to the correlation, so that more accurate semantic information can be extracted.

[0108] In some embodiments, after the server performs self-attention encoding on the initial features of the candidate instructions, the self-attention features of the candidate instructions are obtained, and then the self-attention features of the candidate instructions can be nonlinearly transformed to obtain the candidate instruction features. Similarly, after the server performs self-attention encoding on the initial features of the task information, the self-attention features of the target task information can be obtained, and then the self-attention features of the target task information can be nonlinearly transformed to obtain the task information features.

[0109] In a specific embodiment, Figure 3 As shown, it is a schematic diagram of the feature extraction framework, wherein the server can input the vector representation of the candidate instruction into the encoding neural network for self-attention encoding to obtain the output semantic vector. The encoding neural network can be a neural network established by self-attention encoding. The semantic vector refers to the candidate instruction feature. Similarly, the vector representation of the target task information can also be input into the encoding neural network for self-attention encoding to obtain the output semantic vector, i.e., the task information feature.

[0110] In some embodiments, the encoding neural network may include multiple neural networks connected in sequence, and then the semantic vector output by the encoding neural network may be input into the next encoding neural network for further encoding. After encoding by all encoding neural networks, the final semantic vector is output, that is, the accuracy of semantic feature extraction can be improved through multiple encodings.

[0111] In the above embodiment, the candidate instructions and the target task information are vectorized respectively to obtain the initial features of the candidate instructions and the initial features of the task information. Then, the initial features of the candidate instructions are self-attention encoded to obtain the candidate instruction features, and the initial features of the task information are self-attention encoded to obtain the task information features. That is, semantic features can be extracted by self-attention encoding, thereby improving the accuracy of semantic feature extraction.

[0112] In one embodiment, the initial features of the candidate instructions are self-attention encoded to obtain the candidate instruction features, including:

[0113] Extracting self-attention candidate features of the candidate instruction initial features, and fusing the self-attention candidate features with the candidate instruction initial features to obtain candidate fused features;

[0114] Normalizing the candidate fusion features to obtain normalized candidate features, and performing nonlinear mapping on the normalized candidate features to obtain nonlinear candidate features;

[0115] The nonlinear candidate features are fused with the candidate fusion features to obtain the target candidate features, and the target candidate features are normalized to obtain the candidate instruction features.

[0116] In this embodiment, the self-attention candidate feature refers to the feature corresponding to the candidate instruction extracted by the self-attention mechanism. That is, the server can use the self-attention neural network obtained by training the self-attention mechanism to extract the self-attention feature of the initial feature of the candidate instruction to obtain the self-attention candidate feature. Then the self-attention candidate feature can be fused with the initial feature of the candidate instruction, that is, the feature sum between the self-attention candidate feature and the initial feature of the candidate instruction can be calculated, or the weighted sum between the features can be calculated, or the product between the features can be calculated, so as to obtain the candidate fusion feature. The candidate fusion feature is the feature obtained by fusing the self-attention candidate feature with the initial feature of the candidate instruction. Then the candidate fusion feature can be normalized by a normalization algorithm, and the feature can be adjusted to a specific range or a standardized form by normalization to avoid the problem of reduced feature extraction accuracy due to excessive differences between the features. For example, it can be normalized using a normalization function Norm. Then the server nonlinearly maps the normalized candidate feature through a nonlinear mapping network to obtain a nonlinear candidate feature. The nonlinear mapping network can be a feedforward neural network that is pre-trained. At this time, the nonlinear candidate features are fused with the candidate fusion features, that is, the sum of the nonlinear candidate features and the candidate fusion features can be calculated, the weighted sum between the features can be calculated, the product between the features can be calculated, etc., so as to obtain the target candidate features. Finally, the target candidate features are normalized using the normalization algorithm to obtain the candidate instruction features.

[0117] In a specific embodiment, a neural network can be encoded for self-attention encoding. Figure 4 , which is a schematic diagram of the architecture of the encoding neural network. Specifically, the initial features of the candidate instructions are used as the input of the encoding neural network, and then the self-attention features are extracted through the self-attention neural network, wherein the self-attention features can be extracted using the formula (1) shown below.

[0118] Formula (1)

[0119] Where Q refers to the initial features of the candidate instructions. In the self-attention mechanism, K=Q, V=Q. d represents the feature dimension of the initial features of the candidate instructions. T represents the transpose of the feature matrix. The function is normalized. Then the output of the self-attention neural network is added to the initial features of the candidate instructions, and the normalization operation is performed, that is, the normalized result x1 is calculated by Norm(Q+Attention(Q, K, V)). At this time, the normalized result x1 is input into the feedforward neural network for calculation, and the output result is Feed_forward(x1), where Feed_forward is the calculation unit of the feedforward neural network. Then the calculation results of the feedforward neural network are added and normalized, that is, calculated by Norm(x1+ Feed_forward(x1)) to obtain the final output candidate instruction features.

[0120] In some embodiments, the encoding neural network may be a transformer encoder, and directly using the transformer encoder for self-attention encoding may improve the efficiency of semantic feature extraction.

[0121] In some embodiments, the encoding neural network may include multiple sequentially connected candidate instruction features, and then the candidate instruction features output by the encoding neural network may be input into the next encoding neural network for further encoding. After encoding by all encoding neural networks, the final output candidate instruction features are obtained, that is, the accuracy of the obtained candidate instruction features is improved through multiple encodings.

[0122] In the above embodiment, by extracting the self-attention candidate features of the candidate instruction initial features and fusing the self-attention candidate features with the candidate instruction initial features, the information of the candidate instruction initial features in the self-attention candidate features can be enhanced. Then the candidate fusion features are normalized to obtain normalized candidate features, and the normalized candidate features are nonlinearly mapped to obtain nonlinear candidate features. That is, through further calculation by nonlinear mapping, the information can be fully fused, thereby improving the accuracy of the obtained nonlinear candidate features. Then the nonlinear candidate features are fused with the candidate fusion features to obtain the target candidate features, so as to enhance the information of the candidate fusion features in the nonlinear candidate features, and finally the target candidate features are normalized to obtain the candidate instruction features, thereby improving the accuracy of the obtained candidate instruction features.

[0123] In one embodiment, the initial features of the task information are self-attention encoded to obtain the task information features, including:

[0124] Extracting the self-attention task features of the initial features of the task information, and fusing the self-attention task features with the initial features of the task information to obtain the task fusion features;

[0125] Normalizing the task fusion features to obtain normalized task features, and performing nonlinear mapping on the normalized task features to obtain nonlinear task features;

[0126] The nonlinear task features are fused with the task fusion features to obtain the target task features, and the target task features are normalized to obtain the task information features.

[0127] In this embodiment, the self-attention task feature refers to the feature corresponding to the target task information extracted by the self-attention mechanism. That is, the server can use the self-attention neural network obtained by training the self-attention mechanism to extract the self-attention feature of the initial feature of the task information to obtain the self-attention task feature. Then the self-attention task feature can be fused with the initial feature of the task information, that is, the feature sum between the self-attention task feature and the initial feature of the task information can be calculated, or the weighted sum between the features can be calculated, or the product between the features can be calculated, so as to obtain the task fusion feature. The task fusion feature is the feature obtained by fusing the self-attention task feature with the initial feature of the task information. Then the task fusion feature can be normalized by a normalization algorithm, and the feature can be adjusted to a specific range or a standardized form by normalization to avoid the problem of reduced feature extraction accuracy due to excessive differences between the features. For example, it can be normalized using a normalization function Norm. Then the server nonlinearly maps the normalized task feature through a nonlinear mapping network to obtain a nonlinear task feature. The nonlinear mapping network can be a feedforward neural network that is pre-trained. At this time, the nonlinear task features are fused with the task fusion features, that is, the sum of the nonlinear task features and the task fusion features can be calculated, the weighted sum between the features can be calculated, and the product between the features can be calculated, so as to obtain the target task features. Finally, the target task features are normalized using the normalization algorithm to obtain the task information features.

[0128] In a specific embodiment, the server may input the initial characteristics of the task information into Figure 4 In the encoding neural network shown, encoding is performed through the self-attention neural network and the feedforward neural network to obtain the output task information features.

[0129] In some embodiments, the encoding neural network may include multiple task information features connected in sequence, and then the task information features output by the encoding neural network may be input into the next encoding neural network for further encoding. After encoding by all encoding neural networks, the final output task information features are obtained, that is, the accuracy of the obtained task information features is improved through multiple encodings.

[0130] In the above embodiment, by extracting the self-attention task feature of the initial feature of the task information and fusing the self-attention task feature with the initial feature of the task information, the information of the initial feature of the candidate instruction in the self-attention candidate feature can be enhanced. Then the task fusion feature is normalized to obtain the normalized task feature, and the normalized task feature is nonlinearly mapped to obtain the nonlinear task feature. That is, by further calculating through nonlinear mapping, the information can be fully fused, thereby improving the accuracy of the obtained nonlinear task feature. Then the nonlinear task feature is fused with the task fusion feature to obtain the target task feature, so as to enhance the information of the candidate fusion feature in the nonlinear candidate feature, and finally the target task feature is normalized to obtain the task information feature, thereby improving the accuracy of the obtained task information feature.

[0131] In some embodiments, S206, calculating the distance between the candidate instruction feature and the task information feature to obtain the semantic accuracy corresponding to the candidate instruction includes:

[0132] The distance similarity between the candidate instruction features and the task information features is calculated, and the distance similarity is used as the semantic accuracy corresponding to the candidate instruction.

[0133] In this embodiment, the distance similarity is calculated by a distance similarity algorithm, and the distance similarity is used to characterize the semantic distance between the candidate instruction feature and the task information feature. The distance similarity algorithm may include but is not limited to Euclidean distance similarity and cosine distance similarity.

[0134] In a specific embodiment, the distance similarity can be calculated using the formula (2) shown below, and then the distance similarity is directly used as the semantic accuracy.

[0135] Formula (2)

[0136] Among them, e refers to the task information feature. refers to the candidate instruction features of the i-th candidate instruction. It refers to the multiplication of eigenvectors. and are the vector lengths, The semantic distance between features refers to the semantic accuracy of the candidate instructions, and its value range is [-1,1]. If the value is negative, it means that the semantic direction of the candidate instruction is opposite to that of the target task information. The closer it is to -1, the more serious the opposite is. This may be caused by the logical inconsistency of the candidate instruction or the serious deviation in the information expression of the candidate instruction. The value is a positive number. The closer it is to 1, the closer the semantic accuracy of the candidate instruction is to the target task information, that is, the candidate instruction has no logical contradictions and the clearer the element information is, the closer it is to the requirements of the target task information.

[0137] In the above embodiment, by calculating the distance similarity between the candidate instruction features and the task information features, the distance similarity is used as the semantic precision corresponding to the candidate instruction, that is, the semantic precision of the candidate instruction is calculated by the distance similarity, which can ensure the accuracy of the obtained semantic precision.

[0138] In one embodiment, Figure 5 As shown, calculating the discreteness of at least two task test result information, and determining the test accuracy corresponding to the candidate instruction based on the discreteness, includes:

[0139] S502, respectively extracting semantic features corresponding to at least two task test result information to obtain at least two result information features.

[0140] In this embodiment, the result information feature refers to the feature of the semantic information used to characterize the task test result information. The server encodes each task test result information separately to extract the corresponding semantic feature vector, thereby obtaining the corresponding result information feature. For example, Figure 4 The encoding neural network shown extracts semantic features from each task test result information, that is, extracts semantic features through the self-attention neural network and the feedforward neural network to obtain output result information features. Among them, each task test result information can be input into the semantic feature extraction in parallel at the same time, or it can be input into the encoding neural network in sequence in a serial manner for semantic feature extraction.

[0141] S504, calculating the central feature corresponding to the at least two result information features, and calculating the distances between the central feature and the at least two result information features, to obtain at least two feature distances.

[0142] S506, calculating the average distance of at least two feature distances to obtain a discrete degree, and normalizing the discrete degree to obtain a test accuracy corresponding to the candidate instruction.

[0143] In this embodiment, all the result information features obtained by testing the same test sample are taken as the same cluster. At this time, the server can calculate the center of the same cluster, that is, calculate the central feature corresponding to all the result information features. For example, the server can calculate the average of all the result information features to obtain the central feature. Then the server can use the distance similarity algorithm to calculate the distance between the central feature and each result information feature, and obtain the feature distance between each result information feature and the central feature. For example, the server can use formula (2) to calculate the feature distance between the result information feature and the central feature. At this time, the server calculates the average distance of all feature distances, that is, calculates the sum of all feature distances to obtain the sum of feature distances, and then counts the number of all result information features, and finally calculates the ratio of the sum of feature distances to the number of features to obtain the average distance. At this time, the server can use the average distance as the degree of discreteness, and then normalize the degree of discreteness through a normalization algorithm. For example, the maximum distance among all feature distances can be obtained, and then the ratio of the maximum distance to the degree of discreteness can be calculated, and the test accuracy corresponding to the candidate instruction can be determined according to the ratio.

[0144] In a specific embodiment, the variance between all result information features can be calculated and the variance can be used as the degree of dispersion. That is, the degree of dispersion can be calculated using the following formula (3).

[0145] Formula (3)

[0146] in, It refers to the degree of discreteness. Refers to the i-th result information feature. refers to the central feature. n refers to the number of all resulting informative features.

[0147] In the above embodiment, the result information feature is obtained by extracting the semantic feature corresponding to the task test result information. Then the average distance between all the result information features and the central feature is calculated to obtain the degree of discreteness, and the degree of discreteness is normalized to obtain the test accuracy corresponding to the candidate instruction, that is, the test accuracy is determined by the average distance between the result information feature and the central feature, which improves the accuracy of the test accuracy and thus improves the accuracy of the detection result. That is, when the average distance is larger, the semantic feature corresponding to the task test result information is more dispersed. At this time, the test accuracy corresponding to the candidate instruction is worse, that is, the ambiguity caused by using the worse candidate instruction as the input instruction is greater. When the average distance is smaller, the semantic feature corresponding to the task test result information is closer. When the average distance is larger, the semantic feature corresponding to the task test result information is more dispersed. At this time, the test accuracy corresponding to the candidate instruction is better, that is, the ambiguity caused by using the worse candidate instruction as the input instruction is smaller.

[0148] In some embodiments, S504, calculating the central feature corresponding to at least two result information features includes:

[0149] The at least two result information features are subjected to feature fusion to obtain a result fusion feature; the result fusion feature is subjected to center transformation according to the feature quantity of the at least two result information features to obtain a center feature.

[0150] In this embodiment, feature fusion may be to perform vector operations on vectors of all result information features, to calculate the sum of all result information feature vectors, to calculate the product of all result information feature vectors, or to combine or splice all result information feature vectors to obtain result fusion features. Finally, the server performs a center transformation on the result fusion features according to the feature quantity of all result information features, for example, the ratio between the result fusion features and the feature quantity may be calculated to obtain the center feature.

[0151] In some specific embodiments, the central feature can be calculated using the formula (4) shown below.

[0152] Formula (4)

[0153] Among them, the central feature is obtained by calculating the eigenvector sum of n result information features and then calculating the ratio between the eigenvector sum and the number of features. .

[0154] In the above embodiment, at least two result information features are subjected to feature fusion to obtain a result fusion feature. Finally, the result fusion feature is subjected to center transformation according to the feature quantity of the at least two result information features to obtain a center feature, that is, by calculating the center feature after feature fusion, the accuracy of the obtained center feature can be guaranteed.

[0155] In some embodiments, S506, normalizing the discrete degree to obtain the test accuracy corresponding to the candidate instruction includes:

[0156] A target feature distance is obtained from at least two feature distances; the discrete degree is normalized based on the target feature distance to obtain a test accuracy corresponding to the candidate instruction.

[0157] In this embodiment, the target feature distance refers to the maximum feature distance of all feature distances. That is, the server compares all feature distances, selects the largest feature distance, and obtains the target feature matrix. The server can also sort all feature distances in order from large to small, and then use the first-ranked feature distance as the target feature distance value. At this time, the server can calculate the discrete degree, that is, the ratio between the average distance and the target feature distance, so as to obtain the normalized discrete degree, and then obtain the test accuracy corresponding to the candidate instruction according to the normalized discrete degree. For example, the normalized discrete degree can be in the range of 0 to 1. At this time, the difference between the value 1 and the normalized discrete degree can be calculated to obtain the test accuracy.

[0158] In some specific embodiments, the target feature distance may be the maximum distance among the feature distances, that is, the maximum distance between the feature and the cluster center is used as the target feature distance, that is, the target feature distance Then the normalized discrete degree can be calculated using formula (5) as shown below.

[0159] Formula (5)

[0160] The test accuracy is obtained by calculating the ratio between the average distance, that is, the degree of dispersion, and the maximum distance within the cluster. That is, the discrete degree value of the cluster after normalization, that is, the semantic accuracy value obtained by the test. The closer it is to 0, the closer the distances between the n result information features are, which means that the semantic accuracy of the candidate instruction in this test is better. The closer it is to 1, the more dispersed the distances between the n result information features are, which means that the semantic accuracy of the candidate instructions in this test is worse.

[0161] In the above embodiment, the target feature distance is obtained from at least two feature distances, and then the discrete degree is normalized using the target feature distance to obtain the test accuracy corresponding to the candidate instruction, that is, through normalization, it can be ensured that the obtained test accuracy and the obtained semantic accuracy are in the same range, which is convenient for subsequent use, that is, the accuracy of the test accuracy is guaranteed.

[0162] It is understandable that only one test sample is used for testing, and due to the insufficient number, it is not possible to fully and accurately evaluate the test accuracy of the candidate instruction. At this time, in some embodiments, the test sample includes at least two; Figure 6 As shown, the model instruction detection method also includes:

[0163] S602, determining a current test sample from at least two test samples;

[0164] S604, performing at least two task tests on the current test sample and the candidate instruction through the target task model to obtain at least two task test result information of the current test sample for the candidate instruction;

[0165] S606, calculating the discreteness of the test result information of at least two tasks, and determining the test accuracy of the current test sample for the candidate instruction based on the discreteness;

[0166] S608, traversing at least two test samples to obtain the test accuracy of the at least two test samples for the candidate instructions respectively;

[0167] S610, performing statistical calculations based on the test accuracy of each test to obtain the target test accuracy corresponding to the candidate instruction, and determining the candidate instruction as the detection result of the input instruction of the target task model based on the semantic accuracy and the target test accuracy.

[0168] In this embodiment, the server can obtain multiple test samples, and test the semantic accuracy of the candidate instructions through multiple test samples. That is, the server can test each test sample in sequence in a serial manner. At this time, the server can first determine the current test sample to be tested from multiple test samples. Then the current test sample and the candidate instruction are input into the target task model to execute the target task at least twice, and multiple task test result information obtained by testing the current test sample is obtained. At this time, the multiple task test result information of the current test sample can be used as a cluster cluster, and then the discrete degree of the cluster cluster is calculated, that is, the average distance between each task test result information and the cluster center is calculated, so as to obtain the test accuracy corresponding to the candidate instruction obtained by testing using the current test sample. Similarly, the server selects the next test sample as the current test sample for testing until all test samples are traversed to obtain the test accuracy corresponding to each test sample. Finally, the server can calculate the average of each test accuracy, or can also calculate the mode of each test accuracy, or can also calculate the median of each test accuracy, so as to obtain the target test accuracy corresponding to the candidate instruction. At this time, the server uses semantic accuracy and target test accuracy to detect whether the candidate instruction is the input instruction of the target task model to obtain the detection result.

[0169] In some embodiments, the server can also test multiple test samples simultaneously in a parallel manner, that is, the server can simultaneously input each test sample and the corresponding candidate instruction into the target task model for task execution, and then repeat multiple times to obtain the output of multiple task test result information corresponding to each test sample. Then, the multiple task test result information corresponding to each test sample is used as a cluster respectively, and the cluster center of each cluster is calculated, and then the discrete degree of each cluster is calculated, and then the accuracy of each test is obtained according to the discrete degree, thereby improving the efficiency of the obtained test accuracy.

[0170] In a specific embodiment, the target test accuracy corresponding to the candidate instruction can be calculated using the formula (6) shown below.

[0171] Formula (6)

[0172] Where D represents the number of test samples. It means calculating the sum of the test accuracy corresponding to all test samples. Finally, the ratio of the sum of the test accuracy and the number is calculated to get the target test accuracy. That is, the average test accuracy is calculated, and the average test accuracy is used as the target test accuracy corresponding to the candidate instructions. The closer it is to 1, the better the semantic accuracy of the candidate instruction in the production process. The closer it is to 0, the worse the semantic accuracy of the candidate instruction in the production process.

[0173] In the above embodiment, the target test accuracy corresponding to the candidate instructions is obtained by testing multiple test samples, that is, by expanding the number of test samples, the breadth of the test is improved, thereby ensuring the accuracy of the target test accuracy, and further improving the accuracy of the obtained detection results.

[0174] In a specific embodiment, Figure 7As shown, it is a schematic diagram of the architecture for testing the semantic accuracy of candidate instructions using test samples to obtain the semantic accuracy of the test, wherein the server can obtain a pre-trained semantic encoder, which can be established using a neural network. Then the trained target task model is obtained. At this time, it is necessary to obtain a test sample set as the input of the target task model. When testing, a test sample is obtained from the test sample set, and then the candidate instruction is tested through the target task model using the test sample, and the test sample is used and executed n (a positive integer, pre-set according to the requirements) times to obtain n outputs of the target task model, that is, to obtain n test task result information. At this time, the server extracts the semantic feature vectors of the n test task result information through the semantic encoder to obtain the corresponding n result information feature vectors. Then the discrete degree of the n result information feature vectors is calculated, that is, the discrete degree of the current test sample is obtained. The server continues to iterate, traverses each test sample in the test sample set, and obtains the discrete degree of each test sample. The server calculates the average of the discreteness of all test samples to obtain the average discreteness, and then determines the semantic accuracy score of the candidate instruction obtained through the test based on the average discreteness. That is, the semantic accuracy score of the candidate instruction is obtained by testing multiple test samples, which can ensure the accuracy of the obtained semantic accuracy score.

[0175] In order to improve the accuracy of the detection result, the semantic accuracy calculated from different angles may be fused to detect whether the candidate instruction is an input instruction. That is, in some embodiments, S212, based on the semantic accuracy and the test accuracy, determines that the candidate instruction is a detection result of the input instruction of the target task model, including the steps of:

[0176] The semantic accuracy and test accuracy are integrated to obtain the target accuracy corresponding to the candidate instructions; when the target accuracy meets the preset accuracy conditions, the candidate instructions are used as input instructions of the target task model.

[0177] In this embodiment, the preset accuracy condition refers to the preset condition for using the candidate instruction as the input instruction of the target task model, which can be a preset accuracy threshold, or the maximum accuracy among the target accuracy corresponding to each candidate instruction. The server can also fuse the semantic accuracy and the test accuracy to obtain the target accuracy corresponding to the candidate instruction. For example, the sum of the semantic accuracy and the test accuracy can be calculated, the average of the semantic accuracy and the test accuracy can be calculated, and the product of the semantic accuracy and the test accuracy can be calculated to obtain the target accuracy, which is used to characterize the accuracy of the candidate instruction when it is used as the input instruction of the target task model. The higher the target accuracy, the higher the accuracy of the target task model when the corresponding candidate instruction is used as the input instruction. Then the server compares the target accuracy with the accuracy threshold, or determines whether the target accuracy is the maximum accuracy among the target accuracy corresponding to all candidate instructions. When the target accuracy meets the preset accuracy condition, it means that using the candidate instruction as the input instruction can improve the accuracy of the target task model in executing the task. At this time, the candidate instruction is used as the input instruction of the target task model. When the target accuracy meets the preset accuracy condition, it means that using the candidate instruction as the input instruction can improve the accuracy of the target task model in executing the task. At this time, the candidate instruction is used as the input instruction of the target task model.

[0178] In a specific embodiment, the average of the semantic accuracy and the test accuracy can be calculated to obtain the target accuracy corresponding to the candidate instruction. That is, the comprehensive index Score of the semantic accuracy of the candidate instruction can be calculated using the following formula (7).

[0179] Score( ) = Formula (7)

[0180] Among them, Score( ) refers to the target accuracy of the ith candidate instruction, which is obtained by calculating the average of the semantic accuracy and the test accuracy.

[0181] In the above embodiment, the target accuracy corresponding to the candidate instruction is obtained by fusing the semantic accuracy and the test accuracy, and then when the target accuracy meets the preset accuracy condition, the candidate instruction is used as the input instruction of the target task model. That is, the candidate instruction whose accuracy meets the preset accuracy condition is used as the input instruction of the target task model, thereby improving the accuracy of the input instruction, thereby reducing the situation of task execution failure, and improving the accuracy of the target task model in performing task execution.

[0182] It is understandable that when there are multiple candidate instructions, all candidate instructions can be sorted according to the accuracy, and then the candidate instruction as the target task model input instruction is selected according to the sorting result. That is, in some embodiments, the candidate instructions include at least two. Then S212, based on the semantic accuracy and the test accuracy, determines the candidate instruction as the detection result of the target task model input instruction, including the steps of:

[0183] Obtain semantic accuracy and test accuracy corresponding to at least two candidate instructions respectively; fuse the semantic accuracy and test accuracy corresponding to at least two candidate instructions respectively to obtain target accuracy corresponding to at least two candidate instructions respectively; sort at least two candidate instructions based on the target accuracy to obtain a candidate instruction sequence, and determine the input instructions of the target task model from the candidate instruction sequence.

[0184] In this embodiment, when multiple candidate instructions need to be detected, the server can calculate the semantic accuracy and test accuracy corresponding to each candidate instruction, thereby obtaining the target accuracy of each candidate instruction. At this time, the server can sort all candidate instructions from large to small according to the corresponding target accuracy to obtain a candidate instruction sequence. Then, a corresponding number of candidate instructions can be selected from the candidate instruction sequence in accordance with the number of pre-set top columns, and then all the selected candidate instructions are used as input instructions of the target task model. That is, when there are multiple candidate instructions, the server can also select the preset number of candidate instructions in the top columns of the sorting order from large to small according to the target accuracy as the input instructions of the target task model. Then, when the user uses the target task model, he can select the input instruction of the task to be executed from multiple input instructions, and then the target task model can perform the corresponding task according to the selected input instruction. For example, the candidate instruction gives the corresponding accuracy score, and then the accuracy ranking of the instruction can be obtained. When the candidate instruction is greater than the candidate instruction 2, the candidate instruction 2 is greater than the candidate instruction 1. If a candidate instruction is to be selected as the input value, at this time, the candidate instruction 3 will be selected and applied to the actual production scenario of the target task model.

[0185] In the above embodiment, the semantic accuracy and test accuracy corresponding to at least two candidate instructions are merged to obtain the target accuracy corresponding to at least two candidate instructions. Then, at least two candidate instructions are sorted based on the target accuracy to obtain a candidate instruction sequence, and the input instruction of the target task model is determined from the candidate instruction sequence, that is, the input instruction of the target task model can be detected from multiple candidate instructions, which can improve the accuracy of the input instruction obtained.

[0186] In one embodiment, Figure 8As shown, a model task execution method is provided, which is applied to Figure 1 The server in the example is used for explanation. It can be understood that the method can also be applied to a terminal, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0187] S802, obtaining model input data and input instructions;

[0188] S804, input the model input data and input instructions into the target task model, guide the model input data to perform the task through the input instructions, and obtain the task execution result information; wherein, the input instruction is determined based on the detection result of the candidate instruction as the input instruction of the target task model; the detection result is determined based on the semantic accuracy and the test accuracy; the semantic accuracy is obtained by calculating the distance between the candidate instruction feature of the candidate instruction and the task information feature of the target task information; the test accuracy is obtained by performing at least two task tests on the test samples and the candidate instructions through the target task model, obtaining at least two task test result information corresponding to the candidate instructions, and calculating the discrete degree of at least two task test result information; the candidate instructions and the target task information are corresponding to the target task model.

[0189] In this embodiment, the server has detected the input instruction. At this time, the input instruction can be applied to the specific use process, that is, the server can detect the input instruction from the candidate instructions through the steps in any embodiment of the above-mentioned model instruction detection method. That is, the server can obtain the model input data and input instructions from the database, and the server can also obtain the model input data and input instructions uploaded by the terminal. Then the model input data and input instructions are input into the target task model, and the model input data is guided to perform the task through the input instructions to obtain the task execution result information. Different target task execution results can have different task execution result information. For example, the task execution result information obtained by the classification task model can be the specific category of the input data. The task execution result information obtained by the recognition task model can be the specific recognition result of the input data.

[0190] In some embodiments, when there are multiple input instructions, the server can obtain the input instruction to be executed from the multiple input instructions, for example, the input instruction to be executed can be obtained according to the user's selection, or the input instruction to be executed can be determined according to the execution order of the input instructions set in advance, or the input instruction to be executed can be randomly obtained from multiple input instructions. Then, the input instruction to be executed and the input data are used together to perform the task through the target task model.

[0191] The above-mentioned model task execution method obtains model input data and input instructions, and then inputs the model input data and input instructions into the target task model, guides the model input data to perform task execution through the input instructions, and obtains task execution result information. The input instruction is determined based on the detection result of the candidate instruction as the input instruction of the target task model. The detection result is determined based on the semantic accuracy and test accuracy. The semantic accuracy is obtained by calculating the distance between the candidate instruction feature of the candidate instruction and the task information feature of the target task information; the test accuracy is obtained by performing at least two task tests on the test sample and the candidate instruction by the target task model, obtaining at least two task test result information corresponding to the candidate instruction, and calculating the discrete degree of at least two task test result information. That is, the detection result of the candidate instruction can be determined by measuring the accuracy of the candidate instruction from different angles, and then the input instruction of the target task model can be determined according to the detection result, thereby improving the accuracy of the input instruction obtained. Then, the input instruction and input data are used to perform task execution through the target task model, thereby improving the accuracy of the model task execution, thereby reducing the situation of model task execution errors and saving execution resources.

[0192] In a specific embodiment, Fig. 9 As shown, a schematic diagram of the framework of model task execution is provided. Specifically, the server can mine candidate instructions to obtain each candidate instruction. Then the quality of each candidate instruction is evaluated, that is, the accuracy of the candidate instruction is calculated, that is, the candidate instruction can be detected by any step in any embodiment of the model instruction detection method to obtain the detection result of the candidate instruction. For example, Fig.10The above is a schematic diagram of the principle of model instruction detection, wherein the semantic accuracy of the candidate instruction is calculated and the test accuracy of the candidate instruction is obtained through the target task model test, and then the detection result of the candidate instruction is determined according to the semantic accuracy and the test accuracy. Then the candidate instruction is screened according to the detection result of the candidate instruction to obtain the input instruction of the model, that is, the candidate instruction screened as the input instruction is stored, so as to obtain a set of model input instructions. At this time, when the model needs to be used for task execution, the server can obtain the input instruction from the set of model input instructions, and then use the input instruction and input data to input into the model for task execution. That is, by accurately evaluating the semantics of the candidate instruction to obtain the input instruction of the model, it can be ensured that the target task model understands the user's intention and needs more accurately, so that the model generates more relevant and high-quality responses. Not only improves the efficiency of model task execution, but also reduces the risk of erroneous or irrelevant output. And by outputting more accurate results through the model, users can obtain faster and more accurate responses, thereby improving the overall user experience. At the same time, improving the response quality of the model through more accurate input instructions can significantly reduce the execution of repeated tasks and the waste of time caused by misunderstanding the input instructions, thereby reducing resource consumption and saving resources. In some embodiments, the instruction set application domain can be applied to the fields of health / finance / scientific research / law / education using a large model. The types of target tasks processed by the target task model may include, but are not limited to, sentiment classification task types, professional examination task types, programming language task types, software development and problem finding task types, image understanding task types, mathematical and scientific problem solving task types, data comprehensive analysis task types, and game and entertainment task types.

[0193] In a specific embodiment, Fig.11 As shown, a flow chart of a model instruction detection method is provided, which is executed by a computer device, which may be a server or a terminal, preferably a server, and specifically includes the following steps:

[0194] S1102, obtaining target task information of the target task model and candidate instructions of the target task model, vectorizing the candidate instructions and the target task information respectively, and obtaining initial features of the candidate instructions and initial features of the task information.

[0195] S1104, extract the self-attention candidate features of the candidate instruction initial features, and fuse the self-attention candidate features with the candidate instruction initial features to obtain candidate fused features. Normalize the candidate fused features to obtain normalized candidate features, and perform nonlinear mapping on the normalized candidate features to obtain nonlinear candidate features. Fuse the nonlinear candidate features with the candidate fused features to obtain target candidate features, and normalize the target candidate features to obtain candidate instruction features.

[0196] S1106, extracting the self-attention task feature of the initial feature of the task information, and fusing the self-attention task feature with the initial feature of the task information to obtain a task fusion feature. Normalizing the task fusion feature to obtain a normalized task feature, and performing nonlinear mapping on the normalized task feature to obtain a nonlinear task feature. Fusing the nonlinear task feature with the task fusion feature to obtain a target task feature, and normalizing the target task feature to obtain a task information feature.

[0197] S1108, calculating the distance similarity between the candidate instruction feature and the task information feature, and using the distance similarity as the semantic accuracy corresponding to the candidate instruction.

[0198] S1110, obtaining at least two test samples of the target task model, determining a current test sample from the at least two test samples, performing at least two task tests on the current test sample and the candidate instructions through the target task model, and obtaining at least two task test result information corresponding to the candidate instructions.

[0199] S1112, respectively extracting semantic features corresponding to at least two task test result information to obtain at least two result information features, performing feature fusion on the at least two result information features to obtain a result fusion feature, performing a center transformation on the result fusion feature according to the number of features of the at least two result information features to obtain a center feature.

[0200] S1114, calculate the distances between the central feature and at least two result information features respectively to obtain at least two feature distances. Calculate the average distance of the at least two feature distances to obtain the degree of dispersion, and obtain the target feature distance from the at least two feature distances, normalize the degree of dispersion based on the target feature distance, and obtain the test accuracy corresponding to the candidate instruction.

[0201] S1116, traverse at least two test samples to obtain the test accuracy of the at least two test samples for the candidate instructions respectively, perform statistical calculations based on each test accuracy, and obtain the target test accuracy corresponding to the candidate instruction.

[0202] S1118, the semantic accuracy and the target test accuracy are integrated to obtain the target accuracy corresponding to the candidate instruction. When the target accuracy meets the preset accuracy condition, the candidate instruction is used as the input instruction of the target task model.

[0203] In the above embodiment, the detection result of the candidate instruction is determined by measuring the accuracy of the candidate instruction from different angles, including the semantic accuracy determined by the distance similarity and the target test accuracy obtained by the test, thereby improving the accuracy of the model instruction detection. Then, the input instruction of the target task model can be determined according to the detection result, that is, the accuracy of the input instruction of the target task model is improved, thereby improving the accuracy of the model task execution and saving execution resources. And by detecting and optimizing the semantic accuracy of the instruction, the model can be better adapted to different application scenarios and requirements, and the flexibility and applicability of the model can be enhanced.

[0204] In a specific embodiment, the model instruction detection method can be applied to the scenario of classifying the sentiment of text. Specifically, at present, the semantic accuracy index of candidate instructions can be evaluated from two perspectives. Perspective a includes evaluating the accuracy of information expression or the completeness of information elements of candidate instructions, that is, evaluating whether the candidate instructions include all the complete instruction information required to instruct the model to complete the task or evaluating whether the information elements to be conveyed by the candidate instructions are missing. Perspective b includes evaluating the logical correctness or consistency of candidate instructions, that is, evaluating whether the logic of candidate instructions is consistent and has no conflicts, and can clearly instruct the model to complete the task.

[0205] At this time, the server obtains the candidate instructions and classification task information, that is, all the task element information to be expressed is represented by a vector. At the same time, the vector semantic vector representation of the candidate instructions to be evaluated is obtained. Then the distance between the semantic vectors is calculated. If the candidate instruction is farther away from the semantic distance to be expressed by the classification task information, it means that the candidate instruction to be evaluated is semantically farther from the information to be expressed by the classification task information, that is, the candidate instruction may have deviations caused by reasons such as inaccurate information expression or incomplete information elements or incorrect logic. For example, the candidate instruction is "Please classify the sentiment of this text, including positive, negative and neutral sentiments. Just tell me whether this text is positive." The classification task information can be "Classify the sentiment of the text, including positive, negative and neutral sentiments." At this time, the candidate instruction has information "Just tell me whether this text is positive" that is completely deviated from the classification task information, which will cause the candidate instruction to be semantically farther from the information to be expressed by the classification task information, that is, the semantic accuracy of the candidate instruction is lower, that is, it is inconsistent with the task goal of the model.

[0206] Then, if there are defects in the accuracy of information expression, or the integrity of information elements, or the logical correctness or consistency of the candidate instructions, the classification task model will be unstable when executing the task, that is, for the same input, if the results are generated multiple times, the results will be more dispersed in the semantic space. If the semantic accuracy of the candidate instructions is higher, the defects in angle a or b will be fewer. If the classification task model generates results multiple times, the results will be more concentrated in the semantic space. That is, the test accuracy of the candidate instructions can be further calculated. For example, the candidate instruction "Please classify the sentiment of this text, including positive, negative and neutral sentiments. Just tell me whether this text is positive." is tested multiple times through the classification task model. Since the classification task model cannot accurately determine whether to execute the segment "Please classify the sentiment of this text, including positive, negative and neutral sentiments" or the segment "Just tell me whether this text is positive". These two segments are two contradictory instructions. At this time, the results generated by the model will be very random. If the model is executed multiple times, the results generated will be unstable.

[0207] If the input data is "Today is sunny and refreshing". Repeat the input data and the candidate instruction for 5 times and input them into the classification task model. At this time, the 5 results include output 1 is positive, output 2 is yes, output 3 is positive, output 4 is yes, and output 5 is yes and positive. It can be clearly seen that at this time, the five output results have a large distance in the language space, indicating that the candidate instruction will cause great instability, that is, the semantic accuracy of the candidate instruction is low. If the candidate instruction is "Please classify the sentiment of this text, including positive, negative and neutral sentiments", at this time, repeat the candidate instruction and the input data for 5 times and input them into the classification task model. At this time, the 5 results include output 1 is positive, output 2 is positive, output 3 is positive, output 4 is positive, and output 5 is positive. It can be clearly seen that the five output results have a small distance in the language space, indicating that the candidate instruction will not cause great instability, and the semantic accuracy of the candidate instruction is very high. Therefore, the candidate instruction with high accuracy can be used as the input instruction of the text sentiment classification task model. Then, when performing sentiment classification on the text, the text to be classified is obtained and input together with the input instruction into the text sentiment classification task model for sentiment classification, and the output text sentiment category is obtained, thereby improving the accuracy of text sentiment classification, reducing the need for repeated execution when classification errors occur, and saving execution resources.

[0208] In a specific embodiment, the model instruction detection method can be applied to the scenario of text generation. Specifically, the server can obtain each candidate instruction, which is used to describe the rules of text generation and guide the text generation model to generate specific text. Among them, the text generation model can be a comment text generation model. At this time, the server obtains each candidate instruction of the comment text generation model. Then the target task information of the comment text generation model is obtained, and the target task information includes the purpose of generating the comment task, the style information of the comment text, and the length of the comment text to be generated, etc. Then the server extracts the semantic features of each candidate instruction and the target task information, and then calculates the distance between the semantic features to obtain the semantic accuracy corresponding to each candidate instruction. Then the data to be commented is obtained, and the data to be commented and the candidate instructions are repeatedly input into the comment text generation model for multiple times to generate comment texts, and each comment text corresponding to the data to be commented is obtained, and then the discrete degree of each comment text is calculated, and the test semantic accuracy of each candidate instruction is determined according to the discrete degree. At this time, the server averages the semantic accuracy and the test semantic accuracy to obtain the target accuracy of each candidate instruction. Then, according to the target accuracy, the candidate instruction with the maximum target accuracy can be selected from each candidate instruction as the input instruction of the comment text generation model, that is, the input instruction is obtained by screening the candidate instructions through the semantic accuracy and the test semantic accuracy, thereby improving the accuracy of the output instruction. Then, when the comment text generation model is used in the generation environment, the input instruction can be used to generate the comment text, thereby improving the accuracy of the comment text generation, reducing the need for repeated execution when errors occur, and saving execution resources.

[0209] In a specific embodiment, the model instruction detection method can be applied to the scene of image classification. Specifically, the server can obtain each candidate instruction, which is used to describe the rules of image generation and guide the image generation model to generate a specific image. Among them, the image generation model can be an animation image generation model. At this time, the server obtains each candidate instruction of the animation image generation model. Then the target task information of the animation image generation model is obtained, and the target task information includes the task purpose of generating animation images, the style information of animation images, and the size of animation images to be generated. Then the server extracts the semantic features of each candidate instruction and the target task information, and then calculates the distance between the semantic features to obtain the semantic accuracy corresponding to each candidate instruction. Then the input image is obtained, and the input image and the candidate instructions are repeatedly input into the animation image generation model for animation image generation to obtain each animation image corresponding to the input image, and then the discrete degree of each animation image is calculated, and the test semantic accuracy of each candidate instruction is determined according to the discrete degree. At this time, the server averages the semantic accuracy and the test semantic accuracy to obtain the target accuracy of each candidate instruction. Then, according to the target accuracy, the candidate instruction with the maximum target accuracy can be selected from each candidate instruction as the input instruction of the comment text generation model, that is, the input instruction is obtained by screening the candidate instructions through the semantic accuracy and the test semantic accuracy, thereby improving the accuracy of the output instruction. Then, when the animation image generation model is used in the generation environment, the input instruction can be used to generate animation images, thereby improving the accuracy of animation image generation, reducing the need for repeated execution when errors occur, and saving execution resources.

[0210] In a specific embodiment, the model instruction detection method can be applied to the scene of video generation, specifically: the server can obtain each candidate instruction, which is used to describe the rules of video generation and guide the text generation model to generate a specific video. Among them, the video generation model can be a news video generation model. At this time, the server obtains each candidate instruction of the news video generation model. Then the target task information of the news video generation model is obtained, and the target task information includes the purpose of generating news video tasks, the style information of news videos, and the length of news videos to be generated, etc. Then the server extracts the semantic features of each candidate instruction and the target task information, and then calculates the distance between the semantic features to obtain the semantic accuracy corresponding to each candidate instruction. Then the news data is obtained, and the news data and the candidate instructions are repeatedly input into the news video generation model for news video generation to obtain each news video corresponding to the news data, and then the discrete degree of each news video is calculated, and the test semantic accuracy of each candidate instruction is determined according to the discrete degree. At this time, the server averages the semantic accuracy and the test semantic accuracy to obtain the target accuracy of each candidate instruction. Then, according to the target accuracy, the candidate instruction with the maximum target accuracy can be selected from each candidate instruction as the input instruction of the news video generation model, that is, the input instruction is obtained by screening the candidate instructions through the semantic accuracy and the test semantic accuracy, thereby improving the accuracy of the output instruction. Then, when the news video generation model is used in the generation environment, the input instruction can be used to generate the news video, thereby improving the accuracy of the news video generation, reducing the need for repeated execution when errors occur, and saving execution resources.

[0211] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0212] Based on the same inventive concept, the embodiment of the present application also provides a model instruction detection device for implementing the model instruction detection method involved above and a model task execution device for the model task execution method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiments of one or more model instruction detection devices and model task execution devices provided below can refer to the limitations of the model instruction detection method and the model task execution method above, and will not be repeated here.

[0213] In one embodiment, Fig.12 As shown, a model instruction detection device 1200 is provided, comprising: an instruction acquisition module 1202, a feature extraction module 1204, a distance calculation module 1206, a test module 1208, a discrete calculation module 1210 and a result determination module 1212, wherein:

[0214] An instruction acquisition module 1202 is used to acquire target task information of the target task model and candidate instructions of the target task model;

[0215] A feature extraction module 1204 is used to extract semantic features corresponding to candidate instructions and target task information, respectively, to obtain candidate instruction features and task information features;

[0216] The distance calculation module 1206 is used to calculate the distance between the candidate instruction feature and the task information feature to obtain the semantic accuracy corresponding to the candidate instruction;

[0217] A testing module 1208 is used to obtain a test sample of a target task model, perform at least two task tests on the test sample and the candidate instruction through the target task model, and obtain at least two task test result information corresponding to the candidate instruction;

[0218] A discrete calculation module 1210, used to calculate the discreteness of at least two task test result information, and determine the test accuracy corresponding to the candidate instruction based on the discreteness;

[0219] The result determination module 1212 is used to determine the candidate instruction as the detection result of the target task model input instruction based on the semantic accuracy and the test accuracy.

[0220] In one embodiment, the feature extraction module 1204 is also used to vectorize the candidate instructions and the target task information respectively to obtain the initial features of the candidate instructions and the initial features of the task information; perform self-attention encoding on the initial features of the candidate instructions to obtain the candidate instruction features, and perform self-attention encoding on the initial features of the task information to obtain the task information features.

[0221] In one embodiment, the feature extraction module 1204 is also used to extract self-attention candidate features of the candidate instruction initial features, and fuse the self-attention candidate features with the candidate instruction initial features to obtain candidate fused features; normalize the candidate fused features to obtain normalized candidate features, and perform nonlinear mapping on the normalized candidate features to obtain nonlinear candidate features; fuse the nonlinear candidate features with the candidate fused features to obtain target candidate features, and normalize the target candidate features to obtain candidate instruction features.

[0222] In one embodiment, the feature extraction module 1204 is also used to extract the self-attention task features of the initial features of the task information, and fuse the self-attention task features with the initial features of the task information to obtain task fusion features; normalize the task fusion features to obtain normalized task features, and perform nonlinear mapping on the normalized task features to obtain nonlinear task features; fuse the nonlinear task features with the task fusion features to obtain target task features, and normalize the target task features to obtain task information features.

[0223] In one embodiment, the distance calculation module 1206 is further used to calculate the distance similarity between the candidate instruction features and the task information features, and use the distance similarity as the semantic accuracy corresponding to the candidate instruction.

[0224] In one embodiment, the discrete calculation module 1210 is also used to extract semantic features corresponding to at least two task test result information respectively, to obtain at least two result information features; calculate the central features corresponding to the at least two result information features, and calculate the distances between the central features and the at least two result information features respectively, to obtain at least two feature distances; calculate the average distance of at least two feature distances, to obtain the degree of discreteness, and normalize the degree of discreteness to obtain the test accuracy corresponding to the candidate instruction.

[0225] In one embodiment, the discrete calculation module 1210 is further used to perform feature fusion on at least two result information features to obtain a result fusion feature; and perform center transformation on the result fusion feature according to the number of features of the at least two result information features to obtain a center feature.

[0226] In one embodiment, the discrete calculation module 1210 is further used to obtain a target feature distance from at least two feature distances; normalize the discrete degree based on the target feature distance to obtain a test accuracy corresponding to the candidate instruction.

[0227] In one embodiment, the test sample includes at least two; the model instruction detection device 1200 further includes:

[0228] A target test module is used to determine a current test sample from at least two test samples; perform at least two task tests on the current test sample and candidate instructions through a target task model to obtain at least two task test result information of the current test sample for the candidate instructions; calculate the discreteness of the at least two task test result information, and determine the test accuracy of the current test sample for the candidate instructions based on the discreteness; traverse at least two test samples to obtain the test accuracy of at least two test samples for the candidate instructions respectively; perform statistical calculations based on each test accuracy to obtain the target test accuracy corresponding to the candidate instruction, and determine the detection result of the candidate instruction as the input instruction of the target task model based on the semantic accuracy and the target test accuracy.

[0229] In one embodiment, the result determination module 1212 is also used to merge the semantic accuracy and the test accuracy to obtain the target accuracy corresponding to the candidate instruction; when the target accuracy meets the preset accuracy condition, the candidate instruction is used as the input instruction of the target task model.

[0230] In one embodiment, the candidate instructions include at least two; the result determination module 1212 is also used to obtain the semantic accuracy and test accuracy corresponding to the at least two candidate instructions respectively; the semantic accuracy and test accuracy corresponding to the at least two candidate instructions are merged to obtain the target accuracy corresponding to the at least two candidate instructions respectively; the at least two candidate instructions are sorted based on the target accuracy to obtain a candidate instruction sequence, and the input instructions of the target task model are determined from the candidate instruction sequence.

[0231] In one embodiment, Fig.13 As shown, a model task execution device 1300 is provided, including: a data acquisition module 1302 and a task execution module 1304, wherein:

[0232] Data acquisition module 1302, used to acquire model input data and input instructions;

[0233] The task execution module 1304 is used to input the model input data and input instructions into the target task model, guide the model input data to perform task execution through the input instructions, and obtain task execution result information; wherein, the input instruction is determined based on the detection result of the candidate instruction as the input instruction of the target task model; the detection result is determined based on the semantic accuracy and the test accuracy; the semantic accuracy is obtained by calculating the distance between the candidate instruction feature of the candidate instruction and the task information feature of the target task information; the test accuracy is obtained by performing at least two task tests on the test samples and the candidate instructions through the target task model, obtaining at least two task test result information corresponding to the candidate instructions, and calculating the discrete degree of at least two task test result information; the candidate instructions and the target task information are corresponding to the target task model.

[0234] Each module in the above-mentioned model instruction detection device and model task execution device can be implemented in whole or in part by software, hardware and their combination. Each module can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.

[0235] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig.14 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as test samples, target task information, candidate instructions, target task models, etc. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a model instruction detection method or a model task execution method is implemented.

[0236] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.15As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a model instruction detection method or a model task execution method is implemented. The display unit of the computer device is used to form a visually visible image, and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse, etc.

[0237] Those skilled in the art will understand that Fig.14 or Fig.15 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0238] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0239] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0240] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0241] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0242] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0243] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0244] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A model instruction detection method, characterized in that: The method comprises: Acquire target task information of a target task model and candidate instructions of the target task model; Extracting semantic features corresponding to the candidate instructions and the target task information respectively to obtain candidate instruction features and task information features; Calculating the distance between the candidate instruction feature and the task information feature to obtain the semantic accuracy corresponding to the candidate instruction; Acquire a test sample of the target task model, perform at least two task tests on the test sample and the candidate instruction through the target task model, and obtain at least two task test result information corresponding to the candidate instruction; Calculating the discreteness of the test result information of the at least two tasks, and determining the test accuracy corresponding to the candidate instruction based on the discreteness; The candidate instruction is determined as a detection result of the target task model input instruction based on the semantic accuracy and the test accuracy.

2. The method according to claim 1, characterized in that The extracting semantic features corresponding to the candidate instructions and the target task information respectively to obtain candidate instruction features and task information features includes: Vectorizing the candidate instructions and the target task information respectively to obtain initial features of the candidate instructions and initial features of the task information; The initial features of the candidate instructions are self-attention encoded to obtain the candidate instruction features, and the initial features of the task information are self-attention encoded to obtain the task information features.

3. The method according to claim 2, characterized in that The self-attention encoding of the initial features of the candidate instructions to obtain the features of the candidate instructions includes: Extracting self-attention candidate features of the candidate instruction initial features, and fusing the self-attention candidate features with the candidate instruction initial features to obtain candidate fused features; Normalizing the candidate fusion features to obtain normalized candidate features, and performing nonlinear mapping on the normalized candidate features to obtain nonlinear candidate features; The nonlinear candidate feature is fused with the candidate fusion feature to obtain a target candidate feature, and the target candidate feature is normalized to obtain the candidate instruction feature.

4. The method according to claim 2, characterized in that: The performing self-attention encoding on the initial feature of the task information to obtain the feature of the task information includes: Extracting a self-attention task feature of the initial feature of the task information, and fusing the self-attention task feature with the initial feature of the task information to obtain a task fusion feature; Normalizing the task fusion feature to obtain a normalized task feature, and performing nonlinear mapping on the normalized task feature to obtain a nonlinear task feature; The nonlinear task feature is fused with the task fusion feature to obtain a target task feature, and the target task feature is normalized to obtain the task information feature.

5. The method according to claim 1, characterized in that The calculating the distance between the candidate instruction feature and the task information feature to obtain the semantic accuracy corresponding to the candidate instruction includes: The distance similarity between the candidate instruction feature and the task information feature is calculated, and the distance similarity is used as the semantic accuracy corresponding to the candidate instruction.

6. The method according to claim 1, characterized in that The calculating the discreteness of the at least two task test result information, and determining the test accuracy corresponding to the candidate instruction based on the discreteness, includes: Extracting semantic features corresponding to the at least two task test result information respectively to obtain at least two result information features; Calculating the central feature corresponding to the at least two result information features, and calculating the distances between the central feature and the at least two result information features, respectively, to obtain at least two feature distances; The average distance of the at least two characteristic distances is calculated to obtain the discrete degree, and the discrete degree is normalized to obtain the test accuracy corresponding to the candidate instruction.

7. The method according to claim 6, characterized in that The calculating the central feature corresponding to the at least two result information features includes: Performing feature fusion on the at least two result information features to obtain a result fusion feature; The result fusion feature is subjected to a center transformation according to the feature quantity of the at least two result information features to obtain the center feature.

8. The method according to claim 6, characterized in that The step of normalizing the discrete degree to obtain the test accuracy corresponding to the candidate instruction includes: Acquire a target characteristic distance from the at least two characteristic distances; The discrete degree is normalized based on the target feature distance to obtain the test accuracy corresponding to the candidate instruction.

9. The method according to claim 1, characterized in that: The test samples include at least two; the method further includes: determining a current test sample from the at least two test samples; Perform at least two task tests on the current test sample and the candidate instruction through the target task model to obtain at least two task test result information of the current test sample for the candidate instruction; Calculating the discreteness of the test result information of the at least two tasks, and determining the test accuracy of the current test sample for the candidate instruction based on the discreteness; Traversing the at least two test samples to obtain the test accuracy of the at least two test samples for the candidate instructions respectively; Statistical calculations are performed based on the various test accuracies to obtain the target test accuracy corresponding to the candidate instruction, and the candidate instruction is determined as a detection result of the input instruction of the target task model based on the semantic accuracy and the target test accuracy.

10. The method according to claim 1, characterized in that The step of determining the candidate instruction as the detection result of the target task model input instruction based on the semantic accuracy and the test accuracy includes: The semantic accuracy and the test accuracy are combined to obtain a target accuracy corresponding to the candidate instruction; When the target accuracy meets the preset accuracy condition, the candidate instruction is used as the input instruction of the target task model.

11. The method according to claim 1, characterized in that: The candidate instructions include at least two; the step of determining the candidate instructions as the detection result of the target task model input instruction based on the semantic accuracy and the test accuracy includes: Obtaining semantic accuracy and test accuracy corresponding to at least two candidate instructions respectively; The semantic accuracy and the test accuracy respectively corresponding to the at least two candidate instructions are integrated to obtain the target accuracy respectively corresponding to the at least two candidate instructions; The at least two candidate instructions are sorted based on the target accuracy to obtain a candidate instruction sequence, and the input instructions of the target task model are determined from the candidate instruction sequence.

12. A model task execution method, characterized in that: The method comprises: Obtain model input data and input instructions; Inputting the model input data and the input instruction into the target task model, guiding the model input data to perform the task through the input instruction, and obtaining task execution result information; Among them, the input instruction is determined based on the detection result of the candidate instruction as the input instruction of the target task model; the detection result is determined based on semantic accuracy and test accuracy; the semantic accuracy is obtained by calculating the distance between the candidate instruction feature of the candidate instruction and the task information feature of the target task information; the test accuracy is obtained by performing at least two task tests on the test sample and the candidate instruction by the target task model, obtaining at least two task test result information corresponding to the candidate instruction, and calculating the discrete degree of the at least two task test result information; the candidate instruction and the target task information correspond to the target task model.

13. A model instruction detection device, characterized in that: The device comprises: An instruction acquisition module, used to acquire target task information of a target task model and candidate instructions of the target task model; A feature extraction module, used to extract semantic features corresponding to the candidate instructions and the target task information, respectively, to obtain candidate instruction features and task information features; A distance calculation module, used to calculate the distance between the candidate instruction feature and the task information feature, and obtain the semantic accuracy corresponding to the candidate instruction; A testing module, used to obtain a test sample of the target task model, perform at least two task tests on the test sample and the candidate instruction through the target task model, and obtain at least two task test result information corresponding to the candidate instruction; A discrete calculation module, used to calculate the discrete degree of the test result information of the at least two tasks, and determine the test accuracy corresponding to the candidate instruction based on the discrete degree; A result determination module is used to determine the candidate instruction as a detection result of the target task model input instruction based on the semantic accuracy and the test accuracy.

14. The device according to claim 13, characterized in that The feature extraction module is also used to vectorize the candidate instructions and the target task information respectively to obtain candidate instruction initial features and task information initial features; perform self-attention encoding on the candidate instruction initial features to obtain candidate instruction features, and perform self-attention encoding on the task information initial features to obtain the task information features.

15. The device according to claim 14, characterized in that The feature extraction module is also used to extract self-attention candidate features of the candidate instruction initial features, and fuse the self-attention candidate features with the candidate instruction initial features to obtain candidate fused features; normalize the candidate fused features to obtain normalized candidate features, and perform nonlinear mapping on the normalized candidate features to obtain nonlinear candidate features; The nonlinear candidate feature is fused with the candidate fusion feature to obtain a target candidate feature, and the target candidate feature is normalized to obtain the candidate instruction feature.

16. The device according to claim 14, characterized in that The feature extraction module is also used to extract the self-attention task feature of the initial feature of the task information, and fuse the self-attention task feature with the initial feature of the task information to obtain a task fusion feature; Normalizing the task fusion feature to obtain a normalized task feature, and performing nonlinear mapping on the normalized task feature to obtain a nonlinear task feature; The nonlinear task feature is fused with the task fusion feature to obtain a target task feature, and the target task feature is normalized to obtain the task information feature.

17. The device according to claim 13, characterized in that The distance calculation module is further used to calculate the distance similarity between the candidate instruction feature and the task information feature, and use the distance similarity as the semantic accuracy corresponding to the candidate instruction.

18. The device according to claim 13, characterized in that The discrete calculation module is also used to extract the semantic features corresponding to the at least two task test result information respectively to obtain at least two result information features; Calculating the central feature corresponding to the at least two result information features, and calculating the distances between the central feature and the at least two result information features, respectively, to obtain at least two feature distances; The average distance of the at least two characteristic distances is calculated to obtain the discrete degree, and the discrete degree is normalized to obtain the test accuracy corresponding to the candidate instruction.

19. The device according to claim 18, characterized in that The discrete calculation module is also used to perform feature fusion on the at least two result information features to obtain a result fusion feature; and perform center transformation on the result fusion feature according to the number of features of the at least two result information features to obtain the center feature.

20. The device according to claim 18, characterized in that The discrete calculation module is further used to obtain a target feature distance from the at least two feature distances; normalize the discrete degree based on the target feature distance to obtain a test accuracy corresponding to the candidate instruction.

21. The device according to claim 13, characterized in that The test samples include at least two; the device also includes: A target test module is used to determine a current test sample from the at least two test samples; perform at least two task tests on the current test sample and the candidate instruction through the target task model to obtain at least two task test result information of the current test sample for the candidate instruction; calculate the discreteness of the at least two task test result information, and determine the test accuracy of the current test sample for the candidate instruction based on the discreteness; traverse the at least two test samples to obtain the test accuracy of the at least two test samples for the candidate instruction respectively; perform statistical calculations based on each test accuracy to obtain the target test accuracy corresponding to the candidate instruction, and determine the candidate instruction as the detection result of the input instruction of the target task model based on the semantic accuracy and the target test accuracy.

22. The device according to claim 13, characterized in that The result determination module is also used to merge the semantic accuracy and the test accuracy to obtain the target accuracy corresponding to the candidate instruction; when the target accuracy meets the preset accuracy condition, the candidate instruction is used as the input instruction of the target task model.

23. The device according to claim 13, characterized in that The candidate instructions include at least two; the result determination module is also used to obtain the semantic accuracy and test accuracy corresponding to the at least two candidate instructions respectively; the semantic accuracy and test accuracy corresponding to the at least two candidate instructions are merged to obtain the target accuracy corresponding to the at least two candidate instructions respectively; the at least two candidate instructions are sorted based on the target accuracy to obtain a candidate instruction sequence, and the input instructions of the target task model are determined from the candidate instruction sequence.

24. A model task execution device, characterized in that: The device comprises: A data acquisition module, used to obtain model input data and input instructions; A task execution module, used for inputting the model input data and the input instruction into the target task model, guiding the model input data to perform task execution through the input instruction, and obtaining task execution result information; wherein, the input instruction is determined based on the detection result of the candidate instruction as the input instruction of the target task model; the detection result is determined based on semantic accuracy and test accuracy; the semantic accuracy is obtained by calculating the distance between the candidate instruction feature of the candidate instruction and the task information feature of the target task information; the test accuracy is obtained by performing at least two task tests on the test sample and the candidate instruction through the target task model, obtaining at least two task test result information corresponding to the candidate instruction, and calculating the discrete degree of the at least two task test result information; the candidate instruction and the target task information are corresponding to the target task model.

25. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 12 are implemented.

26. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

27. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

Images