Model testing method and device, equipment and medium
By acquiring a test set and judging input anomalies according to a preset program, test results of a large model are generated, which solves the problem of low testing efficiency in existing technologies and achieves more efficient model testing.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHINA UNITED NETWORK COMM GRP CO LTD
- Filing Date
- 2025-12-31
- Publication Date
- 2026-04-28
AI Technical Summary
The problem of low testing efficiency for large models in existing technologies.
By acquiring test samples from the test set, sending them to the model under test according to the preset test program, identifying input anomalies, and generating test results, including test accuracy and average test duration.
This improves the efficiency and effectiveness of model testing, ensuring that test samples can be successfully input and generate accurate test results.
Smart Images
Figure CN121935120A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of mathematical modeling technology, and in particular to a method, apparatus, device and medium for testing a model. Background Technology
[0002] Testing large models can refer to evaluating their performance by testing them with test samples after training them to obtain a trained large model.
[0003] In existing technologies, large models are tested by manually inputting test samples, which results in low testing efficiency. Summary of the Invention
[0004] This application provides a testing method, apparatus, equipment, and medium for a model, in order to solve the problem of low testing efficiency in the prior art.
[0005] Firstly, this application provides a method for testing a model, including:
[0006] In response to the test command, the test set of the model to be tested is obtained. The test set includes multiple test sample pairs, and each test sample pair includes a test sample and the label corresponding to the test sample.
[0007] According to the preset test program, the test samples in the test set are sent to the model under test in sequence to obtain the running results of the model under test. The running results include the predicted content and the corresponding input content. The number of all input content is less than or equal to the number of all test samples.
[0008] Based on the running results, anomalies in the input of the model to be tested are judged, and the judgment results are obtained. The judgment results indicate whether all test samples in the test set can be successfully input.
[0009] Based on the judgment results, the test results of the model to be tested are obtained.
[0010] In this application, the test results of the model to be tested are obtained based on the judgment results, including:
[0011] If the judgment result is that all test samples in the test set can be successfully input, then for each input content, the test sample, the label corresponding to the test sample, and the prediction content corresponding to the test sample are stored in the first directory;
[0012] Based on the first directory, the test results of the model to be tested are obtained.
[0013] In this application, the test results of the model to be tested are obtained according to the first catalog, including:
[0014] Based on each test sample in the first directory, determine whether the label and predicted content corresponding to the test sample are the same;
[0015] If the label corresponding to the test sample is the same as the predicted content, then the correct label is generated;
[0016] The test results for the model under test are obtained by the ratio of the number of all correctly labeled items to the number of all test samples in the first directory.
[0017] In this application, the test results of the model under test are obtained based on the ratio of the number of all correctly labeled samples to the number of all test samples in the first directory, including:
[0018] The test accuracy is obtained by the ratio of the number of all correctly labeled items to the number of all test samples in the first directory;
[0019] Determine the test duration based on the start and end times in the results.
[0020] The average test duration is obtained based on the test duration and the number of predicted contents in the test results;
[0021] The test results of the model under test are obtained based on the test accuracy and average test duration.
[0022] In this application, the test results of the model to be tested are obtained based on the judgment results, including:
[0023] If the result indicates that not all test samples in the test set can be successfully input, then all abnormal test samples in the test set are stored in the abnormal directory. The abnormal test samples are different from each input content.
[0024] Based on the anomaly directory, the test results of the model to be tested are obtained.
[0025] In this application, according to a preset test procedure, test samples from the test set are sequentially sent to the model under test to obtain the running results of the model under test, including:
[0026] Based on the test set, a second directory is generated, which includes all test samples in the test set.
[0027] The second directory is used as the preset reading directory of the test program, and the model to be tested is used as the request path of the test program.
[0028] The test program sequentially sends the test samples in the second directory to the model under test, and obtains the running results of the model under test.
[0029] In this application, based on the running results, an input anomaly of the model to be tested is judged, and the judgment result is obtained, including:
[0030] The execution results are analyzed to obtain the predicted content in the execution results and the corresponding input content;
[0031] Based on the total number of all inputs and the total number of all test samples, determine whether all test samples in the test set can be successfully input.
[0032] If the total number of all inputs is the same as the total number of test samples, then all test samples in the test set can be successfully input.
[0033] If the total number of all inputs is different from the total number of all test samples, then not all test samples in the test set can be successfully entered.
[0034] Secondly, this application provides a testing apparatus for a model, comprising:
[0035] The test set acquisition module is used to acquire the test set of the model to be tested in response to the test command. The test set includes multiple test sample pairs, and each test sample pair includes a test sample and the label corresponding to the test sample.
[0036] The testing module is used to send the test samples in the test set to the model under test in sequence according to the preset test program, and obtain the running results of the model under test. The running results include the predicted content and the corresponding input content. The number of all input content is less than or equal to the number of all test samples.
[0037] The judgment module is used to judge the input anomalies of the model to be tested based on the running results and obtain the judgment result. The judgment result indicates whether all test samples in the test set can be successfully input.
[0038] The test result acquisition module is used to obtain the test results of the model under test based on the judgment results.
[0039] Thirdly, this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;
[0040] The memory stores instructions that the computer executes;
[0041] The processor executes computer-executable instructions stored in memory to implement the method of the first aspect.
[0042] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method of the first aspect.
[0043] The testing method, apparatus, equipment, and medium provided in this application acquire a test set of the model to be tested in response to user test commands. Then, according to a preset test program, test samples from the test set are sequentially sent to the model to be tested to obtain its running results. Based on these results, input anomalies are assessed, and a judgment result is obtained. The judgment result determines whether there are anomalies in the content input of the model to be tested. Finally, different methods are used to obtain the test results of the model to be tested based on different judgment results. This adds testing for content input anomalies in the predictive model, improving the model testing effect. Attached Figure Description
[0044] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0045] Figure 1 A schematic diagram of a testing scenario for a model provided in an embodiment of this application;
[0046] Figure 2 A flowchart illustrating a testing method for a model provided in an embodiment of this application;
[0047] Figure 3 A flowchart illustrating a testing method for another model provided in an embodiment of this application;
[0048] Figure 4 This is a schematic diagram of the structure of a testing device for a model provided in an embodiment of this application;
[0049] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.
[0050] The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to specific embodiments. Detailed Implementation
[0051] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.
[0052] To clearly understand the technical solution of this application, the solutions of the prior art will be described in detail first.
[0053] Testing large models can refer to evaluating their performance by testing them with test samples after training them to obtain a trained large model.
[0054] In existing technologies, large models are tested by manually inputting test samples, which results in low testing efficiency.
[0055] To address the aforementioned issue of low testing efficiency, the inventors discovered in their research that the test results can be used to determine whether all test samples can be predicted. This allows them to ascertain whether the model under test exhibits anomalies in its content input, and different test results are obtained based on different judgment outcomes. This increases the testing for anomalies in the content input of the model under test, thereby improving the model's testing effectiveness.
[0056] The following describes the application scenarios of the testing method for the model provided in the embodiments of this application.
[0057] Figure 1 This is a schematic diagram of a testing scenario for a model provided in an embodiment of this application, such as... Figure 1 As shown, the scenario includes a model to be tested and a server. The server responds to test commands by obtaining a test set for the model to be tested. The test set includes multiple test sample pairs, each including a test sample and its corresponding label. According to a pre-defined test program, the test samples in the test set are sequentially sent to the model to be tested to obtain the model's running results. The running results include predicted content and the corresponding input content, where the total number of input content is less than or equal to the total number of test samples. Based on the running results, input anomalies in the model to be tested are judged, and a judgment result is obtained. The judgment result indicates whether all test samples in the test set can be successfully input. Based on the judgment result, the test result of the model to be tested is obtained.
[0058] Figure 2 A flowchart illustrating a model testing method provided in this application embodiment is shown below. Figure 2 As shown, the method includes:
[0059] S201. In response to the test command, obtain the test set of the model to be tested. The test set includes multiple test sample pairs, and each test sample pair includes a test sample and the label corresponding to the test sample.
[0060] The test command can refer to a user-sent instruction that directs the server to initiate the model's test process. The test command can be a clickable item on the interactive interface or a string entered by the user; this application does not impose any restrictions on this.
[0061] A test set can refer to a collection of sample pairs used to test the model under test. For example, assuming the model under test is a question-answering model, the test sample pairs in the test set are question-answer pairs, the test samples are questions, and the labels corresponding to the test samples are answers. The test set can be a manually pre-set collection.
[0062] S202. According to the preset test program, the test samples in the test set are sent to the model to be tested in sequence to obtain the running results of the model to be tested. The running results include the predicted content and the input content corresponding to the predicted content. The number of all input contents is less than or equal to the number of all test samples.
[0063] In this context, a test program can refer to a program written in programming software used to automatically test the model under test. For example, a test program can be written in a scripting language. Alternatively, a test program can be a program pre-set by a human.
[0064] The input content can refer to the test samples that have been successfully input into the model to be tested for prediction.
[0065] Specifically, the test program sends a request to the model under test via the curl command. The model under test returns the output prediction content and the corresponding input content in JSON format, thus obtaining the running content.
[0066] In some implementations, according to a preset test procedure, test samples from the test set are sequentially sent to the model under test to obtain the running results of the model under test, which may include:
[0067] Based on the test set, a second directory is generated, which includes all test samples in the test set.
[0068] The second directory is used as the preset reading directory of the test program, and the model to be tested is used as the request path of the test program.
[0069] The test program sequentially sends the test samples in the second directory to the model under test, and obtains the running results of the model under test.
[0070] The second directory can refer to files that the program can read. For example, the second directory can be in the form of a table, with each row representing a test sample. The test program reads the test samples row by row and sends them to the model under test in sequence until all data is read.
[0071] S203. Based on the running results, judge the input anomalies of the model to be tested and obtain the judgment result. The judgment result indicates whether all test samples in the test set can be successfully input.
[0072] Input anomalies can refer to situations where the content input to the model under test cannot be predicted.
[0073] In this application, since the input content of the model under test may be unpredictable, affecting the user experience, in order to address and resolve this situation, the input anomalies of the model under test are judged to determine whether all test samples in the test set can be successfully input, thereby improving the testing effect of the model under test and avoiding situations where the model under test does not respond to input content.
[0074] In some implementations, based on the running results, an anomaly in the input of the model under test is determined, and a determination result is obtained, including:
[0075] The execution results are analyzed to obtain the predicted content in the execution results and the corresponding input content;
[0076] Based on the total number of all inputs and the total number of all test samples, determine whether all test samples in the test set can be successfully input.
[0077] If the total number of all inputs is the same as the total number of test samples, then all test samples in the test set can be successfully input.
[0078] If the total number of all inputs is different from the total number of all test samples, then not all test samples in the test set can be successfully entered.
[0079] In this application, the running results can be parsed using the jq command to obtain the input content and predicted content in the running results. Then, the number of input content and the number of test samples are compared to obtain the judgment result.
[0080] For example, suppose the test program returns a JSON value and the model to be tested is a question-and-answer model. Use the jq command to parse the JSON data, remove unnecessary interference, and extract the values from the question and answer fields in the return value. Create a loop and output the questions and answers returned by the model to the answer directory in turn. Compare the number of data entries in the second directory with the number of data entries in the answer directory. If the number of rows is the same, it means that all test samples in the test set can be successfully input. If the number of rows is different, it means that not all test samples in the test set can be successfully input.
[0081] S204. Based on the judgment result, obtain the test result of the model to be tested.
[0082] In this application, after obtaining the judgment result, it can be determined whether all test samples in the test set can be successfully input into the model to be tested. If all test samples in the test set can be successfully input into the model to be tested, the test result of the model to be tested can be further obtained based on the prediction result. If not all test samples in the test set can be successfully input into the model to be tested, the test result of the model to be tested can be obtained based on the test samples that could not be input.
[0083] In one implementation, if the determination result is that all test samples in the test set can be successfully input, then for each input content, the test sample, the label corresponding to the test sample, and the prediction content corresponding to the test sample can be stored in the first directory.
[0084] Based on the first directory, the test results of the model to be tested are obtained.
[0085] In this application, if the judgment result is that all test samples in the test set can be successfully input, it is necessary to compare the labels of the test samples and the predicted content of the test samples. Therefore, in order to facilitate the comparison by the server, the test samples, the labels corresponding to the test samples, and the predicted content corresponding to the test samples can be stored in the first directory. The first directory can be a data table. For example, each line of the first directory is a piece of test data. Each piece of test data includes the test sample, the label corresponding to the test sample, and the predicted result corresponding to the test sample. The server can read the test data line by line.
[0086] Specifically, based on the first directory, the test results of the model to be tested can include:
[0087] Based on each test sample in the first directory, determine whether the label and predicted content corresponding to the test sample are the same;
[0088] If the label corresponding to the test sample is the same as the predicted content, then the correct label is generated;
[0089] The test results for the model under test are obtained by the ratio of the number of all correctly labeled items to the number of all test samples in the first directory.
[0090] In this application, the server can read each line in the first directory and compare the label corresponding to the test sample with the predicted content. If the label and predicted content are the same, a correct label can be generated in the first directory. For example, a column can be added to the first directory, and the correct label can be written into the added column. Then, the number of correct labels in the first directory and the number of test samples are counted, and the ratio of the number of correct labels to the number of test samples is taken as the test result. For example, if there are A test data points in the first directory, and B correct labels, the test result is B / A.
[0091] Specifically, the test results for the model under test are obtained based on the ratio of the number of all correctly labeled samples to the number of all test samples in the first directory. These results may include:
[0092] The test accuracy is obtained by the ratio of the number of all correctly labeled items to the number of all test samples in the first directory;
[0093] Determine the test duration based on the start and end times in the results.
[0094] The average test duration is obtained based on the test duration and the number of predicted contents in the test results;
[0095] The test results of the model under test are obtained based on the test accuracy and average test duration.
[0096] In this application, based on the test accuracy, the test duration can be obtained by the difference between the start and end times in the running results. The average test duration can be obtained by the ratio of the test duration to the number of test data. Finally, the test accuracy and the average test duration are used as the test results of the model to be tested.
[0097] In another implementation, if the result is that not all test samples in the test set can be successfully input, then all abnormal test samples in the test set can be stored in the abnormal directory. The abnormal test samples are different from each input content.
[0098] Based on the anomaly directory, the test results of the model to be tested are obtained.
[0099] In this application, if the judgment result is that not all test samples in the test set can be successfully input, it indicates that the model under test has an anomaly in terms of input content. In order to further understand the specific anomaly, the abnormal test samples in the test samples can be counted and stored in the anomaly directory, so as to facilitate obtaining the test results according to the anomaly directory and facilitate users to debug the model under test according to the anomaly directory. Furthermore, the server can also read and count the characters in the abnormal test samples and summarize the commonalities in each abnormal test sample, such as the presence of special characters or typos in the abnormal test samples.
[0100] This application provides a model testing method that, in response to a user's test command, obtains a test set for the model to be tested. Then, according to a preset test program, it sequentially sends test samples from the test set to the model to be tested, obtaining the model's running results. Based on these results, it judges whether there are any input anomalies in the model, obtaining a judgment result. Based on the judgment result, it determines whether there are any anomalies in the model's content input. Finally, based on different judgment results, different methods are used to obtain the test results of the model. This method adds testing for content input anomalies in the model to be predicted, improving the model testing effect.
[0101] Figure 3 A flowchart illustrating another model testing method provided in this application embodiment is shown below. Figure 3 As shown, the method includes:
[0102] S301. Prepare the model environment for testing.
[0103] First, in order not to affect the operation of the production environment, the target model to be tested needs to be set up and the ports need to be modified in the test environment to achieve separation between the test environment and the production environment.
[0104] S302. Extract the test set and place it in the specified directory.
[0105] S303. Clean the test set, remove interfering terms, and perform data organization.
[0106] The test set is filtered and deduplicated. The test set data is all regular data. According to the format, non-standard formatted rows are deleted. The questions are extracted and saved to two directories (Directory A: Questions, Directory B: Questions plus Answers).
[0107] S304. Pour the test set into the model, run the test program, and obtain the return value.
[0108] First, adjust the test program's read directory to point to the problem directory (directory A). The test program's request path is the large model. Read the problem line by line (directory A), use the curl command to send requests to the large model, create a loop, and output the return value of the large model to the result directory (directory C).
[0109] S305. Parse the return value to obtain the test results.
[0110] Once the program finishes running, meaning all questions have been asked and answers have been returned, and since the request return value is in JSON format, the jQuery command is used to parse the JSON data, remove redundant interference, and extract the values from the question and answer fields in the return value. This is then looped through, and the questions and answers returned by the large model are output to the filtered answer directory (directory D). The number of rows in directory A (num_A) and directory D (num_D) are compared. If the number of rows is the same, proceed to the next step; if the number of rows is different, the difference rows are identified and recorded as questions.
[0111] Since the problem section of the data in directory D consists of problems extracted from the test set, a loop is used to sequentially read the problem field in directory B and the problem field in directory D and compare them. If the values are the same, the row in directory B and the target row in directory D are taken out and combined into a new row and saved to directory E. The structure of each row in directory E is: problem in the test set, answer in the test set, problem in the test result, and answer in the test result. The contents of the two problem columns should be the same. If they are different, the alignment is checked and redone.
[0112] Finally, analyze the test results and compare the values of the first and second answers in each line of directory E to see if they are the same. In other words, compare the expected results of the test set with the output results. If they are the same, add a column and mark it as true; if they are different, mark it as false. Output the data with the added column to directory F.
[0113] In directory C, find the timestamp 'a' of one data entry and the timestamp 'b' of the last data entry. Subtract timestamp 'a' from timestamp 'b' to get the program's runtime 't'. Divide the runtime by the number of data entries in the test set to get the processing time per data entry (t / num_A). Count the number of true results in directory F (num_true) and divide it by the number of data entries in the test set (num_A) to get the success rate of the test results (num_true / num_A).
[0114] Finally, the results were compiled into a report, which yielded the test results.
[0115] This application provides another model testing method, which involves preparing a model environment for testing, extracting a test set, and placing it in a designated directory. The test set is then cleaned by removing interfering terms and performing data processing. The model is then fed into the test set, and the test program is run to obtain return values. Finally, the return values are parsed to obtain the test results. By judging the return values to obtain different test results for the model, different input anomalies to the model under prediction are detected, thus improving the model testing effect.
[0116] Figure 4 This is a schematic diagram of the structure of a testing device for a model provided in an embodiment of this application, as shown below. Figure 4 As shown, the device 40 includes:
[0117] The test set acquisition module 401 is used to acquire the test set of the model to be tested in response to the test command. The test set includes multiple test sample pairs, and each test sample pair includes a test sample and a label corresponding to the test sample.
[0118] The test module 402 is used to send the test samples in the test set to the model under test in sequence according to the preset test program, and obtain the running results of the model under test. The running results include the predicted content and the input content corresponding to the predicted content. The number of all input contents is less than or equal to the number of all test samples.
[0119] The judgment module 403 is used to judge the input anomalies of the model under test based on the running results, and obtain the judgment result. The judgment result indicates whether all test samples in the test set can be successfully input.
[0120] The test result acquisition module 404 is used to obtain the test results of the model under test based on the judgment result.
[0121] In some implementations, the test result obtaining module 404 is further used for:
[0122] If the judgment result is that all test samples in the test set can be successfully input, then for each input content, the test sample, the label corresponding to the test sample, and the prediction content corresponding to the test sample are stored in the first directory;
[0123] Based on the first directory, the test results of the model to be tested are obtained.
[0124] In some implementations, the test result obtaining module 404 is further used for:
[0125] Based on each test sample in the first directory, determine whether the label and predicted content corresponding to the test sample are the same;
[0126] If the label corresponding to the test sample is the same as the predicted content, then the correct label is generated;
[0127] The test results for the model under test are obtained by the ratio of the number of all correctly labeled items to the number of all test samples in the first directory.
[0128] In some implementations, the test result obtaining module 404 is further used for:
[0129] The test accuracy is obtained by the ratio of the number of all correctly labeled items to the number of all test samples in the first directory;
[0130] Determine the test duration based on the start and end times in the results.
[0131] The average test duration is obtained based on the test duration and the number of predicted contents in the test results;
[0132] The test results of the model under test are obtained based on the test accuracy and average test duration.
[0133] In some implementations, the test result obtaining module 404 is further used for:
[0134] If the result indicates that not all test samples in the test set can be successfully input, then all abnormal test samples in the test set are stored in the abnormal directory. The abnormal test samples are different from each input content.
[0135] Based on the anomaly directory, the test results of the model to be tested are obtained.
[0136] In some implementations, the test module 402 is further used for:
[0137] Based on the test set, a second directory is generated, which includes all test samples in the test set.
[0138] The second directory is used as the preset reading directory of the test program, and the model to be tested is used as the request path of the test program.
[0139] The test program sequentially sends the test samples in the second directory to the model under test, and obtains the running results of the model under test.
[0140] In some embodiments, the determination module 403 is further used for:
[0141] The execution results are analyzed to obtain the predicted content in the execution results and the corresponding input content;
[0142] Based on the total number of all inputs and the total number of all test samples, determine whether all test samples in the test set can be successfully input.
[0143] If the total number of all inputs is the same as the total number of test samples, then all test samples in the test set can be successfully input.
[0144] If the total number of all inputs is different from the total number of all test samples, then not all test samples in the test set can be successfully entered.
[0145] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 5 As shown, the electronic device 50 includes:
[0146] The electronic device 50 may include a processor 501 with one or more processing cores, a memory 502 with one or more computer-readable storage media, a communication component 503, and other components. The processor 501, memory 502, and communication component 503 are connected via a bus 504.
[0147] In the specific implementation process, at least one processor 501 executes computer execution instructions stored in memory 502, causing at least one processor 501 to perform the above method.
[0148] The specific implementation process of processor 501 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.
[0149] In the above Figure 5In the illustrated embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.
[0150] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.
[0151] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.
[0152] In some embodiments, a computer program product is also provided, comprising a computer program or instructions that, when executed by a processor, implement the steps in any of the methods described above.
[0153] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.
[0154] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.
[0155] Therefore, embodiments of this application provide a computer-readable storage medium storing a plurality of instructions that can be loaded by a processor to execute steps in any of the methods provided in embodiments of this application.
[0156] The storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.
[0157] According to one aspect of this application, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium.
[0158] Since the instructions stored in the storage medium can execute the steps of any of the methods provided in the embodiments of this application, the beneficial effects that any of the methods provided in the embodiments of this application can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.
[0159] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.
[0160] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.
Claims
1. A method for testing a model, characterized in that, include: In response to a test command, a test set for the model to be tested is obtained. The test set includes multiple test sample pairs, and each test sample pair includes a test sample and a label corresponding to the test sample. According to the preset test program, the test samples in the test set are sent to the model under test in sequence to obtain the running result of the model under test. The running result includes the predicted content and the input content corresponding to the predicted content. The number of all input contents is less than or equal to the number of all test samples. Based on the running results, the input anomalies of the model under test are judged, and the judgment result is obtained. The judgment result indicates whether all test samples in the test set can be successfully input. Based on the judgment results, the test results of the model to be tested are obtained.
2. The method according to claim 1, characterized in that, The step of obtaining the test result of the model to be tested based on the judgment result includes: If the judgment result is that all test samples in the test set can be successfully input, then for each input content, the test sample, the label corresponding to the test sample, and the prediction content corresponding to the test sample are stored in the first directory; Based on the first directory, the test results of the model to be tested are obtained.
3. The method according to claim 2, characterized in that, The step of obtaining the test results of the model to be tested based on the first directory includes: Based on each test sample in the first directory, determine whether the label and predicted content corresponding to the test sample are the same; If the label corresponding to the test sample is the same as the predicted content, then the correct label is generated; The test results of the model under test are obtained based on the ratio of the number of all correctly labeled items to the number of all test samples in the first directory.
4. The method according to claim 3, characterized in that, The step of obtaining the test results of the model under test based on the ratio of the number of all correctly labeled items to the number of all test samples in the first directory includes: The test accuracy is obtained by the ratio of the number of all correctly labeled items to the number of all test samples in the first directory; The test duration is determined based on the start and end times in the results. The average test duration is obtained based on the test duration and the number of predicted contents in the test results; The test results of the model under test are obtained based on the test accuracy and the average test duration.
5. The method according to claim 1, characterized in that, The step of obtaining the test result of the model to be tested based on the judgment result includes: If the judgment result is that not all test samples in the test set can be successfully input, then the abnormal test samples in all test samples are stored in the abnormal directory, and the abnormal test samples are different from each input content; Based on the anomaly directory, the test results of the model to be tested are obtained.
6. The method according to any one of claims 1-5, characterized in that, The step of sequentially sending test samples from the test set to the model under test according to a preset test program to obtain the running results of the model under test includes: Based on the test set, a second directory is generated, which includes all test samples in the test set; The second directory is used as the reading directory of the preset test program, and the model to be tested is used as the request path of the test program. According to the test program, the test samples in the second directory are sent to the model under test in sequence to obtain the running results of the model under test.
7. The method according to any one of claims 1-5, characterized in that, The step of judging the input anomalies of the model under test based on the running results and obtaining the judgment result includes: The execution results are parsed to obtain the predicted content in the execution results and the corresponding input content; Based on the total number of all input contents and the total number of all test samples, determine whether all test samples in the test set can be successfully input; If the total number of all input contents is the same as the total number of all test samples, then all test samples in the test set can be successfully input. If the number of all input contents is different from the number of all test samples, then not all test samples in the test set can be successfully input.
8. A testing device for a model, characterized in that, include: The test set acquisition module is used to acquire the test set of the model to be tested in response to the test command. The test set includes multiple test sample pairs, and each test sample pair includes a test sample and a label corresponding to the test sample. The testing module is used to send the test samples in the test set to the model under test in sequence according to the preset testing program, and obtain the running result of the model under test. The running result includes the predicted content and the input content corresponding to the predicted content. The number of all input contents is less than or equal to the number of all test samples. The judgment module is used to judge the input anomalies of the model under test based on the running results, and obtain the judgment result, which indicates whether all test samples in the test set can be successfully input. The test result acquisition module is used to obtain the test results of the model to be tested based on the judgment result.
9. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-7.