Model testing method, related device, storage medium and program product

Abstract

The application discloses a model testing method, related equipment, a storage medium and a program product, which can be applied to cloud technology, artificial intelligence, intelligent transportation and the like. The method comprises the following steps: acquiring a set of to-be-tested models and model complexities of the to-be-tested models in the set of to-be-tested models, wherein the set of to-be-tested models comprises m model slots; determining slot reuse values of the corresponding model slots according to the model complexities of the to-be-tested models corresponding to the model slots and slot values of the model slots; determining a reuse slot and input data corresponding to the reuse slot from the m model slots according to the slot reuse values of the model slots; taking model output data obtained after the input data is input into the reuse slot as reuse data; and constructing one or more test tasks according to the reuse data and associated slots of the reuse slot, and obtaining test results of the to-be-tested models based on execution results of the test tasks, so that the efficiency of model testing can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a model testing method, related equipment, storage medium, and program product. Background Technology

[0002] With the rapid development of artificial intelligence technology, computer devices can now be used to complete a wide variety of tasks. In practice, computer devices can collaboratively complete a task by calling one or more data processing models. Therefore, to obtain one or more models that meet the desired processing results (e.g., highest efficiency, minimum resource consumption), it is usually necessary to test each candidate model in a set of alternative models. Traditional model testing methods, when testing each candidate model in the set, cannot avoid the repeated processing of the same data. This increases the computational load on the relevant devices during model testing, thereby reducing the efficiency of model testing. Summary of the Invention

[0003] This application provides a model testing method, related equipment, storage medium, and program product, which can improve the efficiency of model testing.

[0004] On the one hand, embodiments of this application provide a model testing method, including:

[0005] Obtain the set of models to be tested, and the model complexity of each model to be tested in the set of models to be tested. The set of models to be tested contains m model slots. Each model slot corresponds to one or more models to be tested under the same model category. Different model slots correspond to different model categories. m is an integer greater than 1.

[0006] Based on the model complexity of the model to be tested corresponding to each model slot and the slot value of each model slot, the slot reuse value of the corresponding model slot is determined; the slot reuse value is used to indicate the importance of the output result of the model to be tested corresponding to the corresponding model slot.

[0007] Based on the reuse value of each model slot, a reuse slot is determined from the m model slots, along with the input data corresponding to the reuse slot. The model output data obtained after inputting the input data into the reuse slot is then used as the reuse data.

[0008] Based on the reused data and the associated slots of the reused slots, one or more test tasks are constructed, and based on the execution results of the test tasks, test results are obtained for each model to be tested in the set of models to be tested.

[0009] Furthermore, embodiments of this application provide a model testing apparatus, including:

[0010] The acquisition unit is used to acquire the set of models to be tested and the model complexity of each model to be tested in the set of models to be tested. The set of models to be tested contains m model slots. Each model slot corresponds to one or more models to be tested under the same model category. Different model slots correspond to different model categories. m is an integer greater than 1.

[0011] The determining unit is used to determine the slot reuse value of the corresponding model slot based on the model complexity of the model to be tested corresponding to each model slot and the slot value of each model slot; the slot reuse value is used to indicate the importance of the output result of the corresponding model slot corresponding to the model to be tested.

[0012] The determining unit is further configured to determine a reuse slot from the m model slots based on the slot reuse value of each model slot, as well as the input data corresponding to the reuse slot, and use the model output data obtained after inputting the input data into the reuse slot as reuse data.

[0013] The testing unit is used to construct one or more test tasks based on the reused data and the associated slots of the reused slots, and to obtain test results for each model to be tested in the set of models to be tested based on the execution results of the test tasks.

[0014] Furthermore, embodiments of this application also provide a computer device, including:

[0015] A processor for implementing one or more computer programs;

[0016] A computer storage medium storing one or more computer programs adapted to be loaded and executed by the processor:

[0017] Obtain a set of models to be tested, and the model complexity of each model in the set. The set contains m model slots, each slot corresponding to one or more models of the same model category, with different slots corresponding to different model categories, where m is an integer greater than 1. Determine the slot reuse value of each model slot based on its model complexity and slot value. The slot reuse value indicates the importance of the output of the corresponding model. Based on the slot reuse value, determine reuse slots from the m model slots, along with the input data corresponding to each reuse slot. Use the model output data obtained after inputting the input data into the reuse slot as reused data. Construct one or more test tasks based on the reused data and the associated slots of the reused slots. Based on the execution results of the test tasks, obtain the test results for each model in the set.

[0018] In another aspect, embodiments of this application also provide a computer storage medium storing one or more computer programs, the one or more computer programs being adapted to be loaded and executed by the processor;

[0019] Obtain a set of models to be tested, and the model complexity of each model in the set. The set contains m model slots, each slot corresponding to one or more models of the same model category, with different slots corresponding to different model categories, where m is an integer greater than 1. Determine the slot reuse value of each model slot based on its model complexity and slot value. The slot reuse value indicates the importance of the output of the corresponding model. Based on the slot reuse value, determine reuse slots from the m model slots, along with the input data corresponding to each reuse slot. Use the model output data obtained after inputting the input data into the reuse slot as reused data. Construct one or more test tasks based on the reused data and the associated slots of the reused slots. Based on the execution results of the test tasks, obtain the test results for each model in the set.

[0020] In another aspect, embodiments of this application provide a computer product, the computer product including a computer program adapted to be loaded and executed by a processor:

[0021] Obtain a set of models to be tested, and the model complexity of each model in the set. The set contains m model slots, each slot corresponding to one or more models of the same model category, with different slots corresponding to different model categories, where m is an integer greater than 1. Determine the slot reuse value of each model slot based on its model complexity and slot value. The slot reuse value indicates the importance of the output of the corresponding model. Based on the slot reuse value, determine reuse slots from the m model slots, along with the input data corresponding to each reuse slot. Use the model output data obtained after inputting the input data into the reuse slot as reused data. Construct one or more test tasks based on the reused data and the associated slots of the reused slots. Based on the execution results of the test tasks, obtain the test results for each model in the set.

[0022] In this embodiment, the computer device can determine the model slots (i.e., reuse slots) that require reused model output data based on the model slots corresponding to each model under test. Further, the computer device determines reused data through the reuse slots, and then, based on the reused data and reuse slots, divides the testing process for each model under test into one or more test tasks. The computer device can then execute these one or more test tasks to test each model under test. This reduces the amount of data the computer device needs to process at once during model testing, alleviating its processing burden. Simultaneously, because the computer device uses reused data (i.e., reuses the model output data generated by the model under test corresponding to the reuse slot), it avoids redundant calculations of certain data, thereby reducing the computational load during the entire model testing process and improving the efficiency of model testing. Attached Figure Description

[0023] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1a This is a schematic diagram illustrating the principle of a model testing scheme provided in an embodiment of this application;

[0025] Figure 1b This is a schematic diagram illustrating the principle of another model testing scheme provided in the embodiments of this application;

[0026] Figure 2 This is a schematic flowchart of a model testing method provided in an embodiment of this application;

[0027] Figure 3a This is a schematic diagram of a test task initiation method provided in an embodiment of this application;

[0028] Figure 3b This is a schematic diagram of the execution flow of a test task provided in an embodiment of this application;

[0029] Figure 4 This is a flowchart illustrating another model testing method provided in an embodiment of this application;

[0030] Figure 5 This is a schematic diagram of the structure of a model testing device provided in an embodiment of this application;

[0031] Figure 6 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0032] To enable those skilled in the art to better understand the methods provided in the embodiments of this application, the technical methods in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be noted that the specific embodiments described in this application are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the various embodiments of this application without creative effort are within the scope of protection of this application.

[0033] The model testing method provided in this application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, and assisted driving. Specifically, in the above scenarios, this application is mainly used to perform model testing on multiple models to be tested, so as to determine at least two target models from the multiple models to be tested based on the model test results, in order to collaboratively complete a certain business in the corresponding scenario with high quality. It should be noted that the functions implemented by these at least two target models can be different, such as: target model A is used to implement data filtering, and target model B is used to implement data feature extraction, etc. Of course, in the actual application scenario of this application, there may be multiple target models among these at least two target models used to implement the same function, and this application does not limit this. Therefore, based on the above description, it can be understood that this application can mainly be used to determine multiple target models that meet business requirements and are used to collaboratively execute target business from multiple models to be tested, or in other words, this application can mainly be used to determine a group of models to be tested from multiple models to be tested, which meets the corresponding business requirements when executing the target business.

[0034] In practical applications of the embodiments of this application, computer devices can be used to execute the model testing methods proposed in the embodiments of this application. The computer devices mentioned herein can be terminal devices, servers, or computing systems composed of terminal devices and servers. In the embodiments of this application, terminal devices may include, but are not limited to: smartphones, tablets, laptops, desktop computers, in-vehicle terminals, intelligent voice interaction devices, smart home appliances, aircraft, etc. In specific embodiments, various clients (applications, APPs) can also run within the terminal device, such as: multimedia playback clients, social clients, browser clients, information flow clients, educational clients, and image processing clients, etc. The servers mentioned above may include, but are not limited to: independent physical servers, server clusters or distributed systems composed of multiple physical servers, and cloud servers providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The embodiments of this application do not impose specific limitations on the computer devices used; they can be flexibly combined and applied according to actual application scenarios.

[0035] In proposing the specific implementation of the model testing method in this application embodiment, this application embodiment proposes two model testing schemes in a scenario where a computer device tests multiple groups of models to be tested. It is understood that these two model testing schemes can be used to test each model in multiple groups of models to be tested, and either model testing scheme can be executed by the aforementioned computer device. Of course, it is also understood that these two schemes can be applied to other applicable model testing scenarios and are not limited to testing groups of models to be tested. To facilitate a clearer understanding of the model testing method provided by this application embodiment, the principles of the two model testing schemes proposed in this application embodiment are briefly explained below.

[0036] In the first model testing scheme, the computer device can first acquire multiple test model groups, each of which can include at least two test models in an ordered sequence. Then, when testing each test model group (i.e., when testing each test model within a test model group), the computer device can use test data to test each test model group individually. For example, the principle of the computer device performing model testing using the first model testing scheme can be found in [link to relevant documentation]. Figure 1a .exist Figure 1a In this test, the same test data is used for each group of test models, but the test results may differ between different groups. When testing any group of test models, the computer can input the test data into the first test model in the group, so that the output data processed by the first test model can be input into the second test model, which in turn processes the output data of the first test model, and so on, until the last test model in the group produces the corresponding data, thus completing one test of the group of test models.

[0037] The test model group can be obtained by the computer device based on the test models in the test model set. The test model set can include m (m is greater than 1 and m is an integer) test models of model categories. One model category corresponds to multiple test models (e.g., model category A is a data filtering model, which can correspond to data filtering model 1 and data filtering model 2). Optionally, the computer device can determine the test model group based on the test model set by selecting one test model from each model category and then combining the selected test models according to the data processing order when the computer device performs the target business (e.g., data collection → data filtering → data feature extraction). In this case, each test model group includes m test models of different model categories.

[0038] Optionally, the computer device can determine the test model group from the test model set by selecting one test model from each of at least two model categories out of m model categories (i.e., selecting one test model from one model category), and combining the selected test models according to the data processing order when implementing the target business to obtain the test model group. It should be noted that this application does not impose specific limitations on the method by which the computer device determines the test model group. However, in the following embodiments related to this application, unless otherwise specified, it is assumed that a test model group includes m test models, and these m test models have different model categories.

[0039] In the practical application of the first model testing scheme, this application embodiment found that when a computer device performs model testing on a model based on testing multiple groups of models to be tested, the computer device needs to perform a test according to the above principle for each group of models to be tested. This may result in the computer device repeatedly executing the same data processing and generating the same model output data. For example, suppose the computer device needs to use test data X to test two groups of models to be tested (group a and group b). Group a consists of model A and model B1 arranged in the order of model A→model B1, and group b consists of model A and model B2 arranged in the order of model A→model B2. Therefore, when the computer device performs model testing on each model in model group a, it needs to use model A to process the test data X and use model B1 to process the output data of model A to complete the model testing of each model in model group a. Similarly, when the computer device performs testing on each model in model group b, it needs to use model A to process the test data X and use model B2 to process the output data of model A to complete the model testing of each model in model group b. Based on the above description, it is clear that the process of the computer device testing each model in model group a and each model in model group b both include the step of using model A to process the test data X. This causes the computer device to repeatedly execute the same step during model processing, resulting in a longer testing time and thus less than ideal model testing efficiency.

[0040] Therefore, to improve the efficiency of computer equipment in the model testing process, this application proposes a second model testing scheme with reference to the first model testing scheme. The following is a general explanation of the principle of the second model testing scheme, referring to some implementation principles of the first model testing scheme. Specifically, the second model testing scheme states that: the computer equipment can first acquire a set of models to be tested. Then, the computer equipment can determine the model category (hereinafter referred to as the target model category) from the m model categories included in the set of models to be tested, which requires the reuse of the model output data generated by the relevant model to be tested. For example, if the model output data generated by the model to be tested under model category A needs to be reused, then model category A is a target model category. Further, after the computer equipment determines the target model category, the computer equipment can determine the input data of the model to be tested under the target model category (i.e., the data that the corresponding model to be tested needs to process). Furthermore, the computer device can reuse the model output data generated by the test model under the target model category based on the input data. This allows the computer device to directly obtain the relevant data when performing a test task, without having to use the same test model (or sequence of test models) for related data processing again. The principle of the computer device performing model testing in this case can be found in [link to relevant documentation]. Figure 1b Based on the above description of the second model testing scheme and... Figure 1b It is easy to see that by using the second model testing scheme for model testing, computer equipment can effectively avoid the problem of repeated execution of certain data processing steps in certain scenarios, thereby improving the efficiency of computer equipment when performing model testing.

[0041] In the second model testing scheme, the reused data can be understood as the model output data that the computer equipment can reuse. Specifically, as in the example of the first model testing scheme above, the model output data generated by model A is used. Since the input data used by models B1 and B2 in model groups a and b are both model output data generated by model A based on test data X, the model output data generated by model A can be used as reused data. A testing task can be understood as the process by which the computer equipment, when performing model testing, uses some or all of the models to be tested to process the corresponding data. For example, when the computer equipment performs model testing on each model in model group a, the process of the computer equipment using model A to process the test data can be understood as a testing task; similarly, the process by which the computer equipment uses a model sequence composed of models B1 and C to process the model output data of model A can also be understood as a testing task.

[0042] Based on the above description, it is easy to see that the practical application value of the second model testing scheme provided in this application embodiment is higher than that of the first model testing scheme. Therefore, this application embodiment will focus on providing a detailed description of the second model testing scheme. That is to say, unless otherwise specified, the model testing methods mentioned below are all assumed to be model testing methods based on the principles of the second model testing scheme.

[0043] Please see Figure 2 , Figure 2 This is a flowchart illustrating a model testing method proposed in an embodiment of this application. The model testing method can be executed by a computer device and may include steps S201-S204:

[0044] S201, obtain the set of models to be tested, and the model complexity of each model in the set.

[0045] In this embodiment, the set of models to be tested may contain m model slots (m is an integer greater than 1). Each model slot may correspond to one or more models to be tested under the same model category, and the model categories corresponding to different model slots may be different. Specifically, a model slot can be understood as a space for placing models to be tested. One model slot corresponding to one or more models to be tested means that one model slot can hold one or more models to be tested, and the model categories of the models to be tested in the same model slot are all the same. The model category can be determined by the computer device based on the function of the corresponding model to be tested. For example, if all the models to be tested under model slot A are used for image preprocessing, then the model category corresponding to model slot A can be the image preprocessing category. Similarly, if all the models to be tested under model slot B are used for image recognition, then the model category corresponding to model slot B can be the image recognition category.

[0046] Furthermore, in specific embodiments, the model under test has model complexity, which can intuitively reflect the time consumed by the computer device to process relevant data using the model under test. It should be noted that when applying the embodiments of this application to a specific model testing scenario, the model complexity can be measured from multiple aspects of the model under test based on the actual application scenario. Therefore, the embodiments of this application do not limit the method by which the computer device obtains the model complexity of the model under test.

[0047] It should also be noted that when applying the principles of this application's embodiments to other model testing application scenarios, the model categories corresponding to different model slots among the aforementioned m model slots can also be the same. For example, there can be two model slots among the m model slots that correspond to model category A. In specific applications, a computer device can construct a test model group based on a test model under each model slot, and can use the constructed test model group to implement a certain target business. Therefore, it can be understood that if there are model slots of the same model category among the m model slots, the accuracy of certain data processing performed by the computer device can be improved when using the test model group constructed under this condition to implement the target business. This is because the computer device can use a second test model of the same model category to reprocess the model output of the first test model to ensure the accuracy of this type of data processing. This application's embodiments do not elaborate on the relevant model testing methods here.

[0048] S202. Based on the model complexity of the model to be tested corresponding to each model slot and the slot value of each model slot, determine the slot reuse value of the corresponding model slot.

[0049] In this embodiment, the slot reuse value of a model slot can be used to indicate the importance of the output results of the model under test corresponding to that slot. Specifically, the higher the slot reuse value of a model slot, the higher the probability that the model output data (i.e., output results) generated by the corresponding model under test will be reused. Furthermore, when the computer device reuses this model output data, it contributes more to improving the efficiency of model testing. For example, reusing the output results generated by the model under test in a model slot with a higher reuse value can reduce the computational load on the model device during model testing, thereby maximizing the efficiency of model testing.

[0050] S203. Based on the reuse value of each model slot, determine the reuse slot from the m model slots, as well as the input data corresponding to the reuse slot, and use the model output data obtained after inputting the input data into the reuse slot as the reuse data.

[0051] In this context, a reused slot can be understood as a model slot where the output data generated by the corresponding model under test needs to be stored for reuse. The input data corresponding to a reused slot refers to the data that the model under test corresponding to the reused slot needs to process. In this embodiment, there may be one or more reused slots among the m model slots, but the number of reused slots is less than m. Reused data refers to the model output data generated by the model under test in the reused slot. Specifically, reused data can be the model output data generated by some of the models under test in the reused slot. For example, if reused slot A corresponds to two models under test, but during the model testing process of each model under test in the set of models under test, the model output data of only one model under test in reused slot A needs to be used by multiple other models under test, then the reused data can be the model output data of that one model under test.

[0052] Of course, in other application scenarios of this application embodiment, the reused data can also be the model output data generated by all the models to be tested in the reuse slot. For example, when the model output data generated by the two models to be tested corresponding to reuse slot A are used by multiple other models to be tested, the reused data can be the model output data of that one model to be tested. In addition, the reused data can also be part of the model output data generated by some (or all) of the models to be tested under the reuse slot. This application embodiment does not impose specific limitations on the composition of the reused data.

[0053] S204. Based on the reused data and the associated slots of the reused slots, construct one or more test tasks, and based on the execution results of the test tasks, obtain the test results for each model to be tested in the set of models to be tested.

[0054] In this embodiment, a reused slot can be associated with one or more slots. Therefore, a test task in this application embodiment can be executed by a computer device calling a test model under one model slot, or it can be executed collaboratively by a computer device calling test models under multiple model slots. In this application embodiment, each test task can be understood as a process in which the computer device uses the corresponding test model to perform corresponding data processing on the reused data. After the test task is executed, the computer device can obtain the execution result of the test task. Furthermore, since the test task is related to one or more test models, the execution result of the test task in this application embodiment can be used to reflect the test result of the corresponding test model.

[0055] In a specific implementation of this application embodiment, a computer device can initiate an incomplete test task (including a test task that was interrupted after being partially executed), and can also interrupt a test task that is currently being executed. The initiation and interruption behaviors of the computer device regarding test tasks are related to the resource utilization rate of the computer device. Resource utilization rate refers to the overall utilization rate of the computer device's computer resources, which indicates the ratio of the computer resources used by the computer device to the total computing resources of the computer device. It can be understood that the computing resources used by the computer device include the computing resources occupied when the computer device executes a test task, and the computing resources occupied when the computer device executes other business operations besides the test task. In a specific embodiment, before initiating the execution of any test task, the computer device can first confirm whether the current resource utilization rate of the computer device is less than a first utilization rate threshold (exemplarily, the first utilization rate threshold is 90%, and the first utilization rate can be represented by θ). For example, in this application embodiment, the computer device can initiate the execution of a test task when the resource utilization rate is less than 90%. When a computer device detects an interrupt instruction for one (or more) ongoing test tasks, or when the computer device detects that the current resource utilization rate is greater than a second utilization threshold (executively, the second utilization rate can be 95%, and the second utilization rate can be represented by θ'), the corresponding ongoing test task is interrupted to release some computer resources. It is understandable that after the computer device interrupts the test task, the idle computing resources of the computer device will increase. Therefore, it is easy to understand that by releasing some computing resources to increase idle computing resources when the resource utilization rate of the computer device exceeds the second utilization threshold, the computer device can always reserve some computing resources for performing other business operations, thus ensuring the stability and scalability of the computer device to a certain extent.

[0056] Optionally, when resource utilization is between a first utilization threshold of 90% and a second utilization threshold of 95%, the computer equipment can maintain the same number of currently executing test tasks. In other words, under these conditions, the computer equipment can either not initiate new test tasks or interrupt existing ones. This approach allows the computer equipment to maintain high resource utilization during model testing, ensuring that its computing resources are fully utilized and potentially improving the speed of model testing to some extent.

[0057] Furthermore, in this embodiment, the computer device can construct one or more test tasks during model testing and perform model testing on each model to be tested by executing these test tasks. To ensure the efficiency of the computer device's model testing, after obtaining one or more test tasks, the computer device can first determine the execution order of these test tasks (hereinafter referred to as: execution order). This allows the computer device to sequentially start the test tasks according to the execution order when performing model testing based on the constructed test tasks. Additionally, when the computer device needs to interrupt a test task to release computing resources, it can interrupt the corresponding test tasks sequentially in the reverse order of execution, so that the resource utilization of the computer device is lower than a second utilization threshold. For example, if the execution order of the test tasks is: Test Task 1 → Test Task 2 → Test Task 3, then the reverse order of execution is: Test Task 3 → Test Task 2 → Test Task 1.

[0058] Based on the above description of the test tasks, in one embodiment, an exemplary process for a computer device to execute multiple test tasks having an execution order can be found in [reference needed]. Figure 3a .like Figure 3a As shown, after determining the test task, the computer device can start the task scheduler, which then initiates and interrupts the test task. After the task scheduler starts, the computer device can allocate corresponding test tasks to computing containers for execution. In this embodiment, a computing container can be understood as a resource used to execute test tasks. In practical applications, the computer device can call the task scheduler to allocate corresponding test tasks to multiple computing containers to quickly execute multiple test tasks. Therefore, it can be understood that multiple computing containers can exist in this embodiment, forming a computing cluster. Further, the computer device can view the cluster resources. When the resource utilization rate of the cluster resources is less than 90%, the computer device calls the task scheduler to start the test tasks sequentially using the corresponding computing containers according to the execution order; or, when the resource utilization rate is greater than 95%, the computer device calls the task scheduler to interrupt the test tasks currently being executed in the corresponding computing containers in reverse order of execution.

[0059] In this design, when a computer device interrupts a test task, it can save the task's execution progress. This allows the computer to resume the test task once resource utilization meets requirements and the task is restarted, based on the previous progress, thus improving the speed of model testing. Because the test task's execution progress can be preserved, the computer device can interrupt the test task at any time to execute other higher-priority tasks, significantly enhancing its flexibility.

[0060] In addition, while the computer device is viewing cluster resources, it can also wait to receive the task completion signal for any test task, and send a task confirmation signal after receiving the task completion signal to end a test task. Once the computer device has received the task completion signals for all test tasks, it can determine that all test tasks are complete, and then summarize the execution results of each test task to obtain the test results for each model under test. In specific applications, viewing cluster resources can be understood as the computer device periodically acquiring its resource utilization rate. For example, the computer device can acquire the resource utilization rate every 30 seconds. In this case, the current resource utilization rate is the resource utilization rate acquired within the current 30-second period. For instance, assuming the computer device acquires a resource utilization rate of 92% at the 30th second, then the resource utilization rate will remain at 92% from the 30th to the 60th second, and will not be acquired again; only at the 60th second will the computer device acquire a new resource utilization rate. The purpose of periodically acquiring resource utilization data for computer equipment is to avoid the impact of instantaneous fluctuations in resource utilization on the execution of test tasks, thereby ensuring the stability of the computer equipment in executing model tests.

[0061] In one embodiment, the process of a computer device starting and executing a test task can be described as follows: Figure 3b .like Figure 3bAs shown, when a computer device starts executing a test task, it can attach the test models used in the test task and the input data of the relevant test models in the test task. In this embodiment, attaching can be understood as establishing an association. For example, assuming the test task is to process input data 1 using a model sequence [model A, model B], then attaching the corresponding test models to the computer device means that the computer device establishes a relationship between the test task and the test models to indicate which test models the computer device needs to use when executing the test task. Similarly, attaching the input data of each test model to the computer device can be understood as the computer device establishing an association between input data 1 and model A, and establishing an association between the model output interface of model A and the input interface of model B, to indicate that model A needs to process input data 1, and to indicate that model B needs to process the output data generated by model A.

[0062] See also Figure 3b As can be seen, after the computer device loads the required test model and input data, it can query the data that needs to be processed during the execution of the test task from the input data. This allows the computer device to start and execute the test task based on the data that needs to be processed. Specifically, the data that needs to be processed refers to the input data that has not yet been processed. It is understandable that if the test task is one that has been interrupted and then restarted, the data that has not yet been processed can be part of the input data; if the test task is a completely new test task, the data that has not yet been processed can be all of the input data.

[0063] Therefore, in this embodiment, the computer device can retain the execution progress when the test task is interrupted in the following ways: the computer device saves the data that has been processed (or not) from the input data; or the computer device marks the data that has been processed from the input data; or the computer device marks the data that has not been processed from the input data. It can be understood that when the computer device saves the execution progress, it is only necessary to enable the computer device to distinguish between processed data and unprocessed data. This embodiment does not impose specific restrictions on the relevant saving methods.

[0064] Based on the above regarding Figure 2 The description of each step in the process should be noted. It should be explained that the embodiments of this application can be used to test a group of models to be tested, consisting of multiple models to be tested, in order to obtain the performance of the corresponding group of models to be tested. Therefore, it can be understood that the embodiments of this application can be used in table-driven model testing scenarios. The definition of table-driven model testing is explained below with specific examples.

[0065] Table-driven testing can be understood as a table containing multiple model combinations. Within each model combination, the test models in the same column belong to the same model category (or, test models of the same category share the same header). The table can contain multiple columns and rows, with each row representing a test model group. Each test model group in the table can be obtained by the computer device selecting one test model from each of the m model categories in the test model set and combining them. Therefore, the model combinations included in the table can encompass all possible combinations obtained by the computer device based on the test models from the m model categories, with each combination corresponding to a test model group.

[0066] For example, suppose there are m model slots, specifically slot a, slot b, and slot c, and slot a corresponds to one model to be tested (e.g., model A), slot b corresponds to two models to be tested (e.g., model B1 and model B2), and slot c corresponds to two models to be tested (model C1 and model C2). Then, the table used by the computer device for table-driven model testing can be as shown in Table 1. In Table 1, three models to be tested in the same horizontal row can form a group of models to be tested. For example, [model A, model B1, model C1] can represent a group of models to be tested; [model A, model B2, model C1] can also represent a group of models to be tested. Models to be tested in the same column are models of the same model category. For example, the model category of the models to be tested in the first column is the model category corresponding to slot a; the model category of the models to be tested in the second column is the model category corresponding to slot b. Alternatively, the test models in the first column have the same header – model slot a; the test models in the second column have the same header – model slot b.

[0067] Table 1

[0068] Model A to be tested Model B1 to be tested Model C1 to be tested Model A to be tested Model B2 to be tested Model C1 to be tested Model A to be tested Model B1 to be tested Model C2 to be tested Model A to be tested Model B2 to be tested Model C2 to be tested

[0069] It should be noted that in other embodiments, the computer device may also test only a portion of the test model groups included in Table 1. That is, the table used by the computer device when performing table-driven model testing may be composed of a portion of the test model groups in Table 1. Of course, in other implementations, the number of test models included in different test model groups may also be different, and their implementation principles are similar to those in the embodiments of this application, which will not be described in detail here.

[0070] In this embodiment, when a computer device performs table-driven model testing on a model under test based on m model slots, it can determine reuse slots during the model testing process. The model output data generated by the model under test in the reuse slot based on its corresponding input data is used as reused data. This allows the computer device to construct and execute one or more test tasks based on the reused data and the associated slots of the reused slots during model testing. The model test results of each model under test are determined by obtaining the execution results of the corresponding test tasks. Therefore, in this embodiment, the model testing process is divided into multiple test tasks, reducing the amount of computation that the computer device needs to process at once during model testing, thus ensuring the performance stability of the computer device. Furthermore, the computer device determines the reused data (data that needs to be used in different test tasks), allowing the computer device to construct multiple test tasks based on the reused data without first calculating the data required for each test task using the corresponding model under test. This avoids the repeated execution of certain data processing steps by the computer device, thereby improving the efficiency of model testing.

[0071] Based on the above regarding model testing schemes and Figure 2 As can be seen from the description of the model testing method shown, the embodiments of this application can be used for table-driven model testing. Therefore, to improve the efficiency of computer devices in testing table-driven models, the embodiments of this application propose another model testing method, the flowchart of which can be found here. Figure 4 .like Figure 4 As shown, the model testing method may include steps S401-S406:

[0072] S401, obtain the set of models to be tested, and the model complexity of each model in the set.

[0073] As mentioned above, the set of models to be tested can contain m model slots. In this embodiment, the m model slots included in the set of models to be tested can have an arrangement order. For example, the arrangement order of the m model slots can be the order in which the models in any group of models to be tested are used for data processing when the computer device performs table-driven model testing. It should be noted that in this embodiment, during the table-driven model testing process, the computer device uses the same model categories (e.g., m model categories) in the tables used to form each group of models to be tested, and each group of models to be tested can be used to implement the same function. In this case, the arrangement order of the models to be tested in the group of models to be tested can be the data execution order of the group of models to be tested when it is used to implement a specific function.

[0074] Specifically, the way the computer device determines the arrangement order of model slots can be understood with reference to the following example. This example assumes there are four model slots, each corresponding to a model category, as follows: Model slot A corresponds to the image preprocessing category, model slot B to the image recognition category, model slot C to the image classification category, and model slot D to the image acquisition category. Furthermore, it is assumed that when the computer device is performing table-driven model testing, each group of models under test is used to classify the acquired images, and the process of classifying the acquired images is as follows: the computer device first acquires the images, then preprocesses the acquired images, further recognizes the preprocessed images, and finally classifies the recognized images. In this case, the arrangement order of these four model slots can be: Model slot D → Model slot A → Model slot B → Model slot C.

[0075] For ease of description, in this embodiment, the model slot located at the i-th position (i is a positive integer, and i is less than or equal to m) among the m model slots is represented as S. i 'i' can be used as the slot number of the model slot. For example, the model slot that is first in the arrangement (i.e., slot number 1) can be represented as S. 1 The model slot that is second in the order (i.e., the model slot with slot number 2) can be represented as S. 2 In the embodiments of this application, S i Specifically, this can be understood as the slot label of the i-th model slot (or: the model slot with slot number i). Therefore, the number of models to be tested corresponding to the i-th model slot can be represented by |S|. i | indicates. For example, if the first model slot corresponds to 2 models to be tested, then |S 1 |=2.

[0076] Any model in the set of models to be tested can possess model complexity. The model complexity of a model reflects the average processing time required for that model to process any data that meets the input requirements. In this embodiment, the model complexity of any model to be tested can be determined by a computer device based on any one or more of the following model attribute information: the model size of the model to be tested (in MB, e.g., 100MB), the number of model parameters of the model to be tested, and the number of network layers of the model to be tested (e.g., the number of layers in a neural network). That is, the computer device can obtain the model complexity of any model to be tested by acquiring the model attribute information of the model to be tested and then determining the model complexity based on the acquired one or more model attribute information. For ease of description and clearer understanding of the relevant principles of this embodiment, the computer device in this embodiment directly uses the model size of the model to be tested as the model complexity. That is, in this embodiment, if the set of models to be tested includes... and Given that the model sizes of the four models to be tested are 100MB, 100MB, 1000MB, and 100MB respectively, the model complexities of these four models can be respectively:

[0077] S402, determine the slot complexity of any model slot based on the model complexity of each model to be tested in any model slot.

[0078] As described above, in this embodiment of the application, any one of the m model slots can be represented as the i-th model slot, and the i-th model slot can be represented by S. i Therefore, based on this, assuming that the i-th model slot corresponds to n models to be tested, then any model to be tested corresponding to the i-th model slot can be represented as: j is a positive integer less than or equal to n. Therefore, in this embodiment, the slot complexity of the i-th model slot can be determined by the computer device based on the various models to be tested corresponding to that model slot (i.e.: The model complexity is determined by the model complexity of the i-th model slot. For example, the computer device can use the average model complexity of each model to be tested corresponding to the i-th model slot as the slot complexity of the i-th model slot. Then, the computer device can determine the slot complexity of the i-th model slot based on the model complexity of each model to be tested in the i-th model slot as shown in Equation 1:

[0079]

[0080] in, represents the slot complexity of the i-th model slot, n represents the number of models to be tested corresponding to the i-th model slot, and j represents the j-th among the n models to be tested. represents the model complexity of the j-th model to be tested in the i-th model slot.

[0081] S403. According to the slot number corresponding to any model slot, obtain the relative reuse degree and order weight of the slot of any model slot, and determine the slot value of any model slot according to the relative reuse degree and order weight of the slot.

[0082] In the embodiment of the present application, the slot value can be determined by the computer device based on the relative reuse degree and order weight of the model slot. Among them, the relative reuse degree of the model slot can be used to intuitively reflect the importance of the processing result generated by the model to be tested in the corresponding model slot. Exemplarily, the relative reuse degree of the slot can be used to indicate the number of times the processing result generated by the model to be tested under this model slot is reused (or: how many models to be tested in other model slots use it). The order weight of the model slot can be used to indicate the value of the computer device reusing the processing result generated by the model to be tested under this model slot. Exemplarily, the higher the order weight, the higher the value of the computer device reusing the processing result generated by the model to be tested under the corresponding model slot, and the higher the efficiency improvement brought to the computer device. For the convenience of clearly understanding the relevant calculation methods, the following assumes that the i-th model slot is any one of the m model slots (i is a positive integer less than or equal to m), and the calculation method of the relative reuse degree of the model slot and the calculation method of the order weight of the model slot are described in detail with specific examples respectively.

[0083] (1) The method for the computer device to calculate the relative reuse degree of the i-th model slot:

[0084] If the i-th model slot is the last model slot among the m model slots (i.e., i = m), then the relative reuse degree of the i-th model slot is 0; if the i-th model slot is any model slot other than the last model slot among the m model slots (i.e., i < m), then the relative reuse degree of the i-th model slot can be based on the number of models to be tested corresponding to the i + 1-th model slot. For example, the relative reuse degree of the i-th and i + 1-th model slots is: the number of models to be tested corresponding to the i + 1-th model slot minus one. Based on this, the specific calculation method of the relative reuse degree of the i-th model slot can be exemplarily referred to Equation 2:

[0085]

[0086] Among them, |S represents the relative reusability of the i-th model slot. i+1 | represents the number of models to be tested corresponding to the (i+1)th model slot; m is the total number of model slots. That is, assuming there are four model slots, each model slot and the number of models to be tested it contains are as follows: Model Slot S 1 S 2 and S 3 Each includes two models to be tested (i.e., |S 1 |=S 2 |=|S 3 |=2), Model slot S 4 Includes one model to be tested (i.e., |S 4 |=1). Therefore, it can be understood that the relative reuse of the first model slot is 1 (i.e.: The relative reuse of the second model slot is 1 (i.e.: The relative reuse of the third model slot and the relative reuse of the fourth model slot are both 0 (i.e.: ).

[0087] (2) The method by which the computer device calculates the order weight of the i-th model slot:

[0088] The computer device can determine the order weight of the i-th model slot by using an order weight algorithm based on the slot number of the i-th model slot and the total number m of model slots. In this embodiment, the slot number can be used to indicate the arrangement order of the corresponding model slots. For example, the model slot with slot number i has the i-th position in the arrangement order of m model slots. That is, the model slot with slot number i is the i-th model slot. For example, the order weight algorithm can be as shown in Equation 3:

[0089]

[0090] Where m represents the total number of model slots, and i represents the position or slot number of the i-th model slot. It's easy to see that the earlier the model slot is ranked (i.e., the smaller the slot number), the higher its order weight. For example, assuming there are a total of 4 model slots, the order weight of the first model slot can be (4-1). 2 ,Right now: The order weight of the second model slot can be (4-2). 2 ,Right now: The order weight of the third model slot can be (4-3). 2 ,Right now: The order weight of the 4th model slot can be (4-4). 2 ,Right now: It can be seen that the first model slot has the highest order weight, while the fourth model slot has the lowest order weight.

[0091] S404: Determine the slot reuse value of any model slot based on its slot complexity and slot value.

[0092] In this embodiment of the application, after the computer device obtains the slot complexity and slot value of the i-th model slot, the computer device can, exemplarily, calculate the slot reuse value of the i-th model slot in the manner shown in Equation 4:

[0093]

[0094] in, This represents the slot reuse value of the i-th model slot. This represents the slot complexity of the i-th model slot; This represents the relative reusability of the i-th model slot. Let γ represent the order weight of the i-th model slot, γ represent the size of the test data used by the computer device for model testing (in MB or in terms of the number of data points), and 10 be a calculation parameter set based on empirical values. In practical applications, it can be other values, and this application embodiment does not impose specific limitations. Based on Equation 4, assuming there are a total of 4 model slots and γ = 1, in this case, if for the first model slot, we have... For the second model slot, there is For the third model slot, we have: For the 4th model slot, we have: Therefore, based on Equation 4 above, the computer device can calculate the slot reuse value of the first model slot. Similarly, the slot reuse value of the second model slot can be calculated. The slot reuse value of the 3rd and 4th model slots

[0095] S405. Based on the reuse value of each model slot, determine the reuse slot from the m model slots, as well as the input data corresponding to the reuse slot, and use the model output data obtained after inputting the input data into the reuse slot as the reuse data.

[0096] In a specific embodiment, when determining reuse slots, the computer device can identify model slots whose reuse value is greater than a value threshold (e.g., 40) as reuse slots. For example, if there are 4 model slots, and the reuse value of the first model slot is... The slot reuse value of the second model slot The slot reuse value of the 3rd and 4th model slots Therefore, it can be understood that the computer device can use the first and second model slots as reused slots. Furthermore, it should be noted that, based on the above descriptions of Equations 1 and 3, during the process of determining reused slots, if the relative reuse degree of the i-th model slot is 0, then the i-th model slot does not need to be used as a reused slot. Consequently, the computer device does not need to save the model output data generated by the model under test in that model slot as reused data (or intermediate results) during model testing. Specifically, when the i-th model slot in this embodiment is the last model slot, or when the number of models under test in the (i+1)-th model slot is 1, the i-th model slot will not be determined as a reused slot.

[0097] In this embodiment, since the m model slots can have an arrangement order, the input data for the reused slot can be test data used for model testing, or it can be the model output data generated by the corresponding model to be tested under the model slot located in the preceding position of the reused slot. That is, the input data of the (i+1)th model slot can be the model output data generated by the model to be tested under the ith model slot. The following describes one implementation method for a computer device to determine the input data of the reused slot with specific examples. In this example, it is assumed that the m model slots are specifically model slot a (reuse slot), model slot b (reuse slot), and model slot c, and model slot a corresponds to one model to be tested (e.g., model A), model slot b corresponds to two models to be tested (e.g., model B1, model B2), and model slot c corresponds to two models to be tested (model C1, model C2). Therefore, if the computer device uses a table like Table 1 above when performing table-driven model testing, it is easy to see from Table 1 that the model under test in model slot a in this example needs to process the test data. Therefore, the input data when model slot a is used as a reused slot is the test data. Furthermore, it is easy to see that the model under test in model slot b needs to process the model output data generated by model A based on the test data. Therefore, similarly, the input data when model slot b is used as a reused slot can be the model output data generated by model A under test.

[0098] In this embodiment, after determining the input data for a multiplexing slot, the computer device can determine the multiplexing data corresponding to that multiplexing slot based on the input data. Specifically, the computer device can input the input data into any model to be tested in the multiplexing slot to obtain the model output data of any model to be tested in the multiplexing slot in response to the input data; further, the computer device can use the obtained model output data as multiplexing sub-data to compose multiplexing data based on the multiplexing sub-data. The following is an illustration with specific examples. In this example, it is assumed that there are four model slots in this embodiment: model slot a, model slot b, model slot c, and model slot d, and model slots a, b, and c each correspond to two models to be tested, while model slot d corresponds to one model to be tested. Then, for example, the table used by the computer device when performing table-driven model testing can be as shown in Table 2.

[0099] Table 2

[0100] A1 B1 C1 D A1 B1 C2 D A1 B2 C1 D A2 B1 C1 D

[0101] Assuming the computer determines that model slots a and b are reused slots, then for the reused data in model slot a, since the output data of both model A1 and model A2 in model slot a need to be used as input data for the model in model slot b, the computer can use either the output data of model A1 or model A2 in model slot a as reused sub-data. Therefore, the reused data in model slot a can consist of reused sub-data generated by all models in model slot a (i.e., models A1 and A2). In this case, the reused data in model slot a can be understood as: all model output data generated by all models in that model slot. Similarly, for the reused data in model slot b, since the model B1 under test in model slot b will generate data based on the model output data of model A1 under test, and also based on the model output data of model A2 under test, but only the data generated based on the model output data of model A1 under test needs to be reused in model slot c, the reused data in model slot b can be: the reused sub-data obtained by processing the model output data of model B1 under test based on the model output data of model A1 under test. In this case, it can be understood that the reused data in model slot b is: a portion of the model output data of the models under test in this model slot.

[0102] It should be noted that the above-mentioned method for determining reused data is merely an exemplary description. In the specific application of this application, the method by which the computer device determines reused data can be determined according to the actual application scenario, and this application does not impose any specific restrictions on this.

[0103] In one embodiment, after obtaining the reused sub-data that constitutes the reused data, the computer device can store the reused sub-data so that when the input data of the model to be tested in other model slots is the reused sub-data, the corresponding model to be tested can query the required reused sub-data for data processing, thereby completing the corresponding model testing process. Specifically, when storing the reused sub-data, the computer device can first obtain the storage tag of the reused sub-data, and then the computer device can use any one or more of the following storage systems to store the reused sub-data: relational database, message queue, blockchain, and in-memory database, etc. In this embodiment, to save storage space, the computer device can compress the reused sub-data before storing it. In addition, to ensure that the computer device can find the corresponding reused sub-data according to the storage tag later, in a specific implementation, the computer device can store the reused sub-data in the form of key-value pairs.

[0104] For example, the computer device can concatenate the identifier of the test task and the storage label of the multiplexed sub-data as input to SHA2-256 (Secure Hash Algorithm 2-256). Further, the computer device can Base32 encode the output bits of this function to obtain a reference string. Finally, the computer device can use the string after the placeholder (such as an equal sign) at the end of the reference string as the key of the multiplexed sub-data. For instance, assuming we choose Redis as the storage system, and assuming the identifier of this test task is xyz, and the storage label of the multiplexed sub-data is (a, b, c, d, e), then the computer device can concatenate the identifier of the test task and the storage label with a colon as the separator to obtain the string xyz: a, b, c, d, e, which is then used as input to SHA2-256. Assuming the result is PCHXXXAKUA====, the computer device can use PCHXXXAKUA as the key of the multiplexed sub-data.

[0105] The specific method by which the computer device determines the storage label of the reused sub-data is as follows: The computer device obtains the data label of the input data used to generate the reused sub-data, as well as the model label of any model to be tested in the reuse slot; then, the computer device can generate a storage label for the corresponding reused sub-data according to the data label and the model label, and store the corresponding reused sub-data according to the storage label.

[0106] The model label can be used to indicate the placement position of the model to be tested in the model slot. For example, if the model to be tested is the j-th model in the i-th model slot... i If there are j, then the model label of the model to be tested can be j. i Data labels can be used to indicate which test models processed which test data to obtain the input data. The test data was processed by which test models, represented by a sequence of model labels. For example, the model label sequence (j1, j2) indicates that the test data was processed by the j1st test model in the first model slot and the j2nd test model in the second model slot. Which test data was processed can be represented by its label. For example, the label of the test data can be any character or string used to uniquely identify the test data (e.g., a number, a character encoding, etc.). Therefore, if the data label of the input data is (δ, (j1, j2)), it means that the input data was obtained by the computer device using the j1st test model in the first model slot and the j2nd test model in the second model slot to perform a series of corresponding data processing on the test data labeled δ.

[0107] Therefore, based on this, after acquiring the data tag and model tag, the computer device can add the model tag to the sequence of model tags included in the data tag to obtain the storage tag for the reused sub-data. For example, assuming the tag used for test data is δ, the computer device determines the i1, i2, ... i1-th model tag from m model slots. k (i1 <i2<i k ( ) model slots are reused slots, and the models to be tested in the model group to be tested can be used separately. This is represented by the following assumption: Based on this, it is further assumed that the group of models to be tested can be represented by a sequence of model labels. This indicates that, and, in the group of models to be tested, the i1, i2, ..., i... k The model output data generated by the model under test in each model slot can be stored as reusable sub-data. Therefore, in this example, the model under test in the model under test group... The input data can be the model to be tested. Based on the model output data generated from the test data, and considering the above description of data labels, it is easy to understand that the data label for this input data can be (δ, ), then, for the model to be tested The generated multiplexed sub-data can be used by computer devices. The data labels added to the corresponding input data include the model label sequence. In order to obtain the storage tag (δ, of the reused sub-data) In this embodiment of the application, the reused sub-data can be used... To express.

[0108] Similarly, for the model to be tested The generated reused sub-data can be considered as a sequence of model labels used by computer devices. The corresponding models to be tested The generated reused sub-data The result is obtained after a series of data processing steps; or, the model to be tested. The generated reused sub-data can also be understood as computer devices using model label sequences. The model to be tested is obtained after performing a series of data processing steps on the test data δ. The generated multiplexed sub-data can be represented as Its storage tag can be (δ, Similarly, we can obtain the reused slots i3, i4, ..., i k The storage label for the reusable sub-data generated by the corresponding model to be tested.

[0109] In other application scenarios of this application embodiment, each model output data can have a corresponding data label. Specifically, if the test data used by the computer device includes test data labeled δ1 and test data labeled δ2, then the computer device uses the first model to be tested in the first model slot (i.e., model...). After processing all the test data, the model output data can be obtained. as well as Its data labels can be (δ1, 1) and (δ2, 1) respectively. The computer device uses the second model to be tested in the first model slot (i.e., model). After processing the test data, the model output data can be obtained. as well as Their data labels are (δ1, 2) and (δ2, 2), respectively. Furthermore, if the computer equipment uses the sequence of the model to be tested... and right After data processing, the data labels for the model output data can be obtained as follows (one data label corresponds to one model output data): (δ1, (1, 1, 1, 1)), (δ1, (1, 2, 1, 1)), (δ1, (1, 1, 1, 2)) and (δ1, (1, 2, 2, 1)). Since each data point generated during model testing has a data label, when determining the input data for a reuse slot, the computer can also determine the corresponding input data's data label. This allows the computer to query the input data based on its data label when needed.

[0110] S406: Based on the reused data and the associated slots of the reused slots, construct one or more test tasks, and based on the execution results of the test tasks, obtain the test results for each model to be tested in the set of models to be tested.

[0111] The methods for determining the associated slots of a reused slot may include, but are not limited to, the following two:

[0112] (1) If the number of reused slots in the m model slots is 1, then the associated slots of the reused slot can be: all model slots in the m model slots whose slot number is greater than the slot number of the reused slot. For example, if m is 4 and the slot number of the reused slot is 2, then the associated slots of the reused slot are: the model slots with slot numbers 3 and 4. That is, all model slots in the m model slots whose order is after the reused slot are associated slots of the reused slot.

[0113] (2) If at least two of the m model slots are reused, then the associated slot of any reused slot (e.g., any reused slot other than the one with the largest slot number among the at least two reused slots) can be composed of the following two types of model slots: ① model slots located between the slot number corresponding to the reused slot and the slot number of the next reused slot of the reused slot; ② the next reused slot. Therefore, the associated slot of the reused slot with the largest slot number among the at least two reused slots can be any model slot with a slot number greater than the slot number of that reused slot. For example, if m is 4, there are 2 reused slots among the m model slots, with slot numbers 1 and 3 respectively; then, for the reused slot with slot number 1, its associated slots are: model slot 2 and model slot 3; for the model slot with slot number 3 (i.e., the reused slot with the largest slot number), its associated slot can be: model slot 4.

[0114] After the computer device determines the associated slots and reused data of the reused slots, it can construct one or more test tasks based on these associated slots and reused data to complete table-driven model testing by executing these test tasks. The following details how the computer device constructs test tasks. First, the computer device determines the associated slots and reused data of the reused slots. After determining the associated slots, the computer device can determine the model to be tested corresponding to each associated slot. Then, the computer device can randomly combine these models to obtain one or more test model groups, thereby constructing the table used by the computer device when performing table-driven model testing. Finally, the computer device can construct a test task based on the corresponding reused sub-data in the reused data and a test model group. In other words, the computer device can treat the process of a test model group processing the reused sub-data that needs to be processed in the reused data as a test task.

[0115] Test tasks can have task tags, which can be specifically represented as follows: Where R represents the corresponding reused data, s1, s2, ..., s n This represents the individual models to be tested within a test model group. The computer device obtains the test model group by selecting one model from each associated slot and combining them. In this case, assume there are m model slots in total, denoted as S. 1 S 2 S m The test models for each model slot are n1, n2, ..., n m And assume the i-th, i-th, ..., i-th k Each model slot is a reused slot. Therefore, the computer device can randomly combine the models to be tested from the (i1+1)th to the i2th model slots to obtain... A test model group can then be obtained. Each test task. Furthermore, in the specific implementation, the computer device can also randomly combine the models to be tested from the 1st to the i1th model slots to obtain... A test model group can then be obtained. Each test task. Based on this, the computer device can obtain all the test tasks required for table-driven model testing.

[0116] In addition, it should be noted that in the embodiments of this application, the test model group and the test model group are not the same concept. The test model group includes m test models, while the test model group only includes a portion of the test models in the m test model groups. The test model group can be composed of the test model group.

[0117] In this embodiment of the application, based on the foregoing regarding Figure 2 As described above, after dividing the test tasks into multiple parts, the computer device can sort the startup order of each test task to enable the computer device to complete model testing more efficiently. Therefore, to better understand the embodiments of this application, the following detailed explanation of how the computer device sorts the startup order of test tasks is provided with specific examples. Specifically, after the computer device divides the test tasks, it can determine the startup order of the corresponding test tasks based on the task tags of each test task. Optionally, for any two test tasks, the computer device can determine the startup order of these two test tasks sequentially based on the arrangement order of the model slots, the arrangement order of the models to be tested in the model slots, and the order of the data set.

[0118] For example, suppose there is (referred to as: Test Task 1) and (Referred to as Test Task 2) Given two test tasks, the computer can first determine the order of the smallest model slots (i.e., the model slot with the smallest slot number) in Test Task 1 and Test Task 2. Since the smallest model slot in Test Task 1 is S... 1 Its arrangement order is located in the corresponding smallest model slot S in test task 2. 3 Therefore, test task 1 is executed before test task 2. Let's assume there is... (referred to as: Test Task 1) and (Referring to Test Task 3) Given two test tasks, since the minimum model slots in Test Task 1 and Test Task 3 are identical, the computer can determine the order in which the models to be tested correspond to the minimum model slots in both test tasks. Therefore, because... This indicates the first model to be tested in the first model slot. This indicates the second model to be tested in the first model slot, that is, the model to be tested corresponding to the smallest model slot in test task 1. The arrangement order is the model to be tested corresponding to the smallest model slot in test task 2. Therefore, test task 1 will be started before test task 3. Let's assume there is... (referred to as: Test Task a) and (Referring to test task b) Two test tasks exist, where R1 = {δ1} and R2 = {δ2} are both subsets of the dataset R{δ1, δ2}, and δ1 is arranged before δ2 in the dataset R. The order of the datasets can be understood as the order of R1 and R2. Since the minimum model slots are the same in both test tasks a and b, and the minimum model slots correspond to the same slots of the models to be tested, the computer can determine the start order of the test tasks based on the arrangement of the datasets corresponding to each test task. Specifically, since the dataset containing the input data of test task a is arranged before the dataset containing the input data of test task b, the start order of test task a is before that of test task b.

[0119] Furthermore, in specific applications of this application, to avoid excessive time consumption of a single test task, the computer device can divide a test task into multiple test subtasks. Specifically, the computer device first sets a processing threshold (or: processing threshold of the corresponding test model group) for each test task, which indicates the maximum amount of data that each test task (or: corresponding test model group) can process, such as 5000 records. Based on this, when dividing test subtasks, the computer device can obtain the amount of reused data that the test model group needs to process in the test task and the processing threshold of the test model group; when the amount of reused data input to the test model group is greater than the processing threshold, the computer device can split the reused data based on the processing threshold of the test model group to obtain multiple data groups of reused data; further, the computer device can input each data group in the multiple data groups into the test model group respectively to summarize and obtain the execution result of a test task. Of course, these multiple test subtasks also have a startup order, and the method for determining the startup order can be referred to the method for determining test tasks, which will not be elaborated in this application embodiment.

[0120] In this embodiment, the computer device can determine the reuse slots and their associated slots during model testing, and use the model output data generated by the model under test in the reuse slot based on its corresponding input data as reused data. This allows the computer device to perform table-driven model testing by constructing one or more test tasks. Specifically, for cases where different groups of models under test require the same data processing, the computer device constructs only one corresponding test task, enabling data processing based solely on any one group of models under test. Furthermore, the computer device can use the processed data as reused data, allowing it to directly retrieve this data when needed during testing other groups of models under test, significantly reducing the computational load. In addition, because this embodiment divides the entire model testing process into multiple test tasks, and the progress of each test task can be retained even when interrupted, model testing can be interrupted and resumed at any time, improving the processing stability and model testing efficiency of the computer device to a certain extent.

[0121] Based on the above regarding Figure 2 and Figure 4 The embodiments of the model testing method shown in this application also disclose a model testing apparatus, which can be a computer program (including program code) running on the aforementioned computer device. In specific embodiments, the model testing apparatus can be used to perform, for example... Figure 2 or Figure 4 The model testing method is shown. Please refer to [link / reference]. Figure 5 , Figure 5 This is a schematic diagram of the structure of a model testing device provided in an embodiment of this application, as shown below. Figure 5 As shown, the model testing device may include: an acquisition unit 501, a determination unit 502, and a testing unit 503.

[0122] The acquisition unit 501 is used to acquire the set of models to be tested and the model complexity of each model to be tested in the set of models to be tested. The set of models to be tested contains m model slots. Each model slot corresponds to one or more models to be tested under the same model category. Different model slots correspond to different model categories. m is an integer greater than 1.

[0123] The determining unit 502 is used to determine the slot reuse value of the corresponding model slot based on the model complexity of the model to be tested corresponding to each model slot and the slot value of each model slot; the slot reuse value is used to indicate the importance of the output result of the model to be tested corresponding to the corresponding model slot.

[0124] The determining unit 502 is further configured to determine a reuse slot from the m model slots based on the slot reuse value of each model slot, as well as the input data corresponding to the reuse slot, and use the model output data obtained after inputting the input data into the reuse slot as reuse data.

[0125] Test unit 503 is used to construct one or more test tasks based on the reused data and the associated slots of the reused slots, and to obtain test results for each model to be tested in the set of models to be tested based on the execution results of the test tasks.

[0126] In one implementation, the determining unit 502, when determining the slot reuse value of any model slot, may specifically perform the following:

[0127] The slot complexity of any model slot is determined based on the model complexity of each model to be tested in any model slot.

[0128] Based on the slot number corresponding to any model slot, obtain the relative reuse degree and order weight of any model slot, and determine the slot value of any model slot based on the relative reuse degree and order weight.

[0129] Based on the slot complexity and slot value of any model slot, the slot reuse value of any model slot is determined.

[0130] In another embodiment, the any model slot is the i-th model slot, where i is a positive integer less than or equal to m; when the determining unit 502 obtains the relative reuse degree and order weight of the any model slot according to the slot number corresponding to the any model slot, it can be specifically used to perform:

[0131] Obtain the slot number i of the i-th model slot. If i equals m, then determine that the relative reuse of the i-th model slot is 0. If i is less than m, then determine the relative reuse of the i-th model slot based on the number of models to be tested contained in the (i+1)-th model slot.

[0132] Based on the slot number i of the i-th model slot and the total number of model slots m, the sequential weight algorithm is used to calculate the sequential weight of the i-th model slot.

[0133] In another embodiment, when the determining unit 502 obtains multiplexed data after inputting the input data into the multiplexing slot, it can specifically perform the following:

[0134] The input data is input into any test model of the reuse slot to obtain the model output data of any test model of the reuse slot in response to the input data.

[0135] The model output data of any test model in the multiplexing slot in response to the input data is used as multiplexed sub-data, and the multiplexed data is composed based on the multiplexed sub-data.

[0136] In another embodiment, the model testing apparatus further includes a data storage unit 504, which can be used to perform:

[0137] Obtain the data labels of the input data used to generate the multiplexed sub-data, and the model labels of any test model in the multiplexed slot;

[0138] Based on the data tag and the model tag, a storage tag for the reused sub-data is generated, and the reused sub-data is stored according to the storage tag.

[0139] In another embodiment, when the test unit 503 constructs a test task based on the reused data and the associated slots of the reused slots, it can specifically be used to execute:

[0140] Obtain the associated slots of the reused slot and the model to be tested corresponding to the associated slots; wherein, if the number of reused slots is one, the associated slot is a model slot whose corresponding slot number is greater than the reused slot number; or, if the number of reused slots is at least two, the associated slots of any reused slot include: model slots whose slot number is between the slot number corresponding to any reused slot and the slot number of the next reused slot of any reused slot;

[0141] The test models corresponding to the associated slots are randomly combined to obtain one or more test model groups;

[0142] A test task is constructed based on the corresponding reused sub-data in the reused data and a test model group.

[0143] In another embodiment, the test model group has a processing threshold, which indicates a threshold amount of data that the test model group can process; the test unit 503 can also be specifically used to perform:

[0144] Obtain the data volume of the reused sub-data and the processing threshold of any test model group;

[0145] When the amount of data of the reused sub-data of any test model group is greater than the processing threshold of any test model group, the reused sub-data is split based on the processing threshold of any test model group to obtain multiple data groups of the reused sub-data.

[0146] Each of the multiple data groups is input into any of the test model groups to obtain the execution result of a test task.

[0147] According to a feasible embodiment of the present application, Figure 2 and Figure 4 Each step of the model testing method involved can be derived from... Figure 5 The test is performed by each unit in the model testing apparatus shown. For example: Figure 2 Step S201 in the model testing method shown can be performed by... Figure 5 The acquisition unit 501 in the model testing device shown is used to execute the steps; steps S202 to S203 can be performed by... Figure 5 The determination unit 502 in the model testing apparatus shown is responsible for executing step S204; step S204 can be performed by... Figure 5 The test unit 503 in the model test apparatus shown is used to perform the test. For example, Figure 4 Step S401 in the model testing method shown can be performed by... Figure 5 The acquisition unit 501 in the model testing device shown is used to execute the steps; steps S402 to S405 can all be performed by... Figure 5 The determination unit 502 in the model testing apparatus shown is responsible for executing step S405; step S405 can be performed by... Figure 5 The test unit 503 in the model test apparatus shown is used to perform the test.

[0148] According to one feasible embodiment of the present application, Figure 5 The units in the model testing apparatus shown can be divided based on logical functions. These units can also be individually or entirely merged into one or more other units. Alternatively, some units can be further divided into multiple functionally smaller units. This achieves the same operation without affecting the technical effects of the embodiments of this application. In other embodiments of this application, the model testing apparatus may also include other units. In practical applications, these functions can be implemented with the assistance of other units, and multiple units can assist in their implementation.

[0149] According to another embodiment of this application, a general-purpose computing device, such as a domain name management device, which includes processing elements and storage elements such as a central processing unit (CPU), random access storage medium (RAM), and read-only storage medium (ROM), can be used to run an application capable of performing tasks such as... Figure 2 and Figure 4 The computer program (including program code) involved in each step of the method shown is used to construct, for example... Figure 5 The model testing apparatus shown is illustrated, as well as the model testing method for implementing the embodiments of this application. The computer program may be recorded on, for example, a computer storage medium, loaded onto the aforementioned computing device via the computer storage medium, and run therein.

[0150] In this embodiment, the model testing device can determine the model slots (i.e., reuse slots) that require reused model output data based on the model slots corresponding to each model under test. Further, the model testing device determines reused data through the reuse slots, and then, based on the reused data and reuse slots, divides the testing process for each model under test into one or more test tasks. The model testing device can then perform tests on each model under test by executing these test tasks. This reduces the amount of data that the model testing device needs to process at once during model testing, alleviating its processing burden. Simultaneously, because the model testing device uses reused data (i.e., reuses the model output data generated by the model under test corresponding to the reuse slot), it avoids repeated calculations of certain data, reducing the computational load of the model testing device throughout the entire model testing process, thereby improving the efficiency of model testing.

[0151] Based on the descriptions of the above method and device embodiments, this application also provides a computer device. Please refer to [link to relevant documentation]. Figure 6The computer device includes at least a processor 601 and a computer storage medium 602, and the processor 601 and the computer storage medium 602 can be connected via a bus or other means. The aforementioned computer storage medium 602 is a memory device in the computer device used to store programs and data. It is understood that the computer storage medium 602 here can include the built-in storage medium of the computer device, or it can include extended storage media supported by the computer device. The computer storage medium 602 provides storage space, which stores the operating system of the computer device. Furthermore, this storage space also stores one or more computer programs suitable for loading and execution by the processor 601; these computer programs can be one or more program codes. It should be noted that the computer storage medium here can be high-speed RAM memory, or non-volatile memory, such as at least one disk storage device; optionally, it can also be at least one storage medium located remotely from the aforementioned processor. The processor 601 (or CPU (Central Processing Unit)) is the computing and control core of a computer device. It is suitable for implementing one or more computer programs, specifically for loading and executing one or more computer programs to achieve corresponding methods or functions.

[0152] In one embodiment, processor 601 may load and execute one or more computer programs stored in computer storage medium 602 to achieve the aforementioned related functions. Figure 2 as well as Figure 4 The corresponding method steps in the illustrated method embodiment. In a specific implementation, one or more computer programs in the computer storage medium 602 can be loaded and executed by the processor 601 as follows:

[0153] Obtain the set of models to be tested, and the model complexity of each model to be tested in the set of models to be tested. The set of models to be tested contains m model slots. Each model slot corresponds to one or more models to be tested under the same model category. Different model slots correspond to different model categories. m is an integer greater than 1.

[0154] Based on the model complexity of the model to be tested corresponding to each model slot and the slot value of each model slot, the slot reuse value of the corresponding model slot is determined; the slot reuse value is used to indicate the importance of the output result of the model to be tested corresponding to the corresponding model slot.

[0155] Based on the reuse value of each model slot, a reuse slot is determined from the m model slots, along with the input data corresponding to the reuse slot. The model output data obtained after inputting the input data into the reuse slot is then used as the reuse data.

[0156] Based on the reused data and the associated slots of the reused slots, one or more test tasks are constructed, and based on the execution results of the test tasks, test results are obtained for each model to be tested in the set of models to be tested.

[0157] In one implementation, when determining the slot reuse value of any model slot, the processor 601 can specifically be used to load and execute:

[0158] The slot complexity of any model slot is determined based on the model complexity of each model to be tested in any model slot.

[0159] Based on the slot number corresponding to any model slot, obtain the relative reuse degree and order weight of any model slot, and determine the slot value of any model slot based on the relative reuse degree and order weight.

[0160] Based on the slot complexity and slot value of any model slot, the slot reuse value of any model slot is determined.

[0161] In another embodiment, the any model slot is the i-th model slot, where i is a positive integer less than or equal to m; when the processor 601 obtains the relative reuse degree and order weight of the any model slot according to the slot number corresponding to the any model slot, it can be specifically used for loading and execution:

[0162] Obtain the slot number i of the i-th model slot. If i equals m, then determine that the relative reuse of the i-th model slot is 0. If i is less than m, then determine the relative reuse of the i-th model slot based on the number of models to be tested contained in the (i+1)-th model slot.

[0163] Based on the slot number i of the i-th model slot and the total number of model slots m, the sequential weight algorithm is used to calculate the sequential weight of the i-th model slot.

[0164] In another embodiment, when the processor 601 obtains multiplexed data after inputting the input data into the multiplexing slot, it can specifically be used to load and execute:

[0165] The input data is input into any test model of the reuse slot to obtain the model output data of any test model of the reuse slot in response to the input data.

[0166] The model output data of any test model in the multiplexing slot in response to the input data is used as multiplexed sub-data, and the multiplexed data is composed based on the multiplexed sub-data.

[0167] In yet another embodiment, the processor 601 may specifically be used to load and execute:

[0168] Obtain the data labels of the input data used to generate the multiplexed sub-data, and the model labels of any test model in the multiplexed slot;

[0169] Based on the data tag and the model tag, a storage tag for the reused sub-data is generated, and the reused sub-data is stored according to the storage tag.

[0170] In another embodiment, when the processor 601 loads and executes a test task constructed based on the multiplexed data and the associated slots of the multiplexed slots, it can specifically be used to load and execute:

[0171] Obtain the associated slots of the reused slot and the model to be tested corresponding to the associated slots; wherein, if the number of reused slots is one, the associated slot is a model slot whose corresponding slot number is greater than the reused slot number; or, if the number of reused slots is at least two, the associated slots of any reused slot include: model slots whose slot number is between the slot number corresponding to any reused slot and the slot number of the next reused slot of any reused slot;

[0172] The test models corresponding to the associated slots are randomly combined to obtain one or more test model groups;

[0173] A test task is constructed based on the corresponding reused sub-data in the reused data and a test model group.

[0174] In another embodiment, the test model group has a processing threshold, which indicates a threshold amount of data that the test model group can process; the processor 601 may also be specifically used to load and execute:

[0175] Obtain the data volume of the reused sub-data and the processing threshold of any test model group;

[0176] When the amount of data of the reused sub-data of any test model group is greater than the processing threshold of any test model group, the reused sub-data is split based on the processing threshold of any test model group to obtain multiple data groups of the reused sub-data.

[0177] Each of the multiple data groups is input into any of the test model groups to obtain the execution result of a test task.

[0178] In this embodiment, the computer device can determine the model slots (i.e., reuse slots) that require reused model output data based on the model slots corresponding to each model under test. Further, the computer device determines reused data through the reuse slots, and then, based on the reused data and reuse slots, divides the testing process for each model under test into one or more test tasks. The computer device can then perform tests on each model under test by executing these test tasks. This reduces the amount of data the computer device needs to process at once during model testing, alleviating its processing burden. Simultaneously, because the computer device uses reused data (i.e., reuses the model output data generated by the model under test corresponding to the reuse slot), it avoids redundant calculations of certain data, reducing the computational load throughout the model testing process and thus improving the efficiency of model testing.

[0179] This application also provides a computer storage medium storing one or more computer programs corresponding to the above-described model testing method. When one or more processors load and execute the one or more computer programs, the method described in this application can be implemented. Figure 2 or Figure 4 The model testing method described herein can be found in the foregoing related embodiments for its specific implementation, and will not be repeated here. Furthermore, the beneficial effects of using the same method described in the embodiments of this application will also not be repeated here. It is also understood that the related computer programs in the embodiments of this application can be deployed and executed on one or more devices capable of communicating with each other.

[0180] It should be noted that, according to another aspect of the embodiments of this application, this application also provides a computer program product or computer program, which includes a computer program stored in a computer storage medium. A processor in a computer device reads the computer program from the computer storage medium and then executes the computer program, thereby enabling the computer device to perform the aforementioned... Figure 2 as well as Figure 4 The methods provided are from various alternative embodiments of the model testing method shown.

[0181] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. Furthermore, the computer program can be stored in a computer storage medium, and when executed, it can include the processes of the embodiments of the model testing method described above. The computer storage medium may include, but is not limited to, one or more of the following: magnetic disks, optical disks, read-only memory (ROM), or random access memory (RAM), etc.

[0182] It is understood that the above-disclosed embodiments are merely partial examples of the model testing methods involved in this application, and should not be construed as limiting the scope of the claims of this application. Furthermore, those skilled in the art will understand that implementing all or part of the processes of the above embodiments, and making equivalent changes with reference to the claims of this application, still falls within the scope of the embodiments of this application.

Claims

1. A model testing method, characterized in that, include: Obtain the set of models to be tested, and the model complexity of each model to be tested in the set of models to be tested. The set of models to be tested contains m model slots. Each model slot corresponds to one or more models to be tested under the same model category. Different model slots correspond to different model categories. m is an integer greater than 1. Based on the model complexity of the model to be tested corresponding to each model slot and the slot value of each model slot, the slot reuse value of the corresponding model slot is determined; the slot reuse value is used to indicate the importance of the output result of the model to be tested corresponding to the corresponding model slot. Based on the reuse value of each model slot, a reuse slot is determined from the m model slots, along with the input data corresponding to the reuse slot. The model output data obtained after inputting the input data into the reuse slot is then used as the reuse data. Based on the reused data and the associated slots of the reused slots, one or more test tasks are constructed, and based on the execution results of the test tasks, test results are obtained for each model to be tested in the set of models to be tested.

2. The method according to claim 1, characterized in that, Methods for determining the reuse value of any model slot include: The slot complexity of any model slot is determined based on the model complexity of each model to be tested in any model slot. Based on the slot number corresponding to any model slot, obtain the relative reuse degree and order weight of any model slot, and determine the slot value of any model slot based on the relative reuse degree and order weight. Based on the slot complexity and slot value of any model slot, the slot reuse value of any model slot is determined.

3. The method according to claim 2, characterized in that, The aforementioned model slot is the i-th model slot, where i is a positive integer less than or equal to m; the step of obtaining the relative reuse degree and order weight of the aforementioned model slot based on the slot number corresponding to the aforementioned model slot includes: Obtain the slot number i of the i-th model slot. If i equals m, then determine that the relative reuse of the i-th model slot is 0. If i is less than m, then determine the relative reuse of the i-th model slot based on the number of models to be tested contained in the (i+1)-th model slot. Based on the slot number i of the i-th model slot and the total number of model slots m, the sequential weight algorithm is used to calculate the sequential weight of the i-th model slot.

4. The method according to claim 1, characterized in that, The methods for obtaining reused data after inputting the input data into the reuse slot include: The input data is input into any test model of the reuse slot to obtain the model output data of any test model of the reuse slot in response to the input data. The model output data of any test model in the multiplexing slot in response to the input data is used as multiplexed sub-data, and the multiplexed data is composed based on the multiplexed sub-data.

5. The method according to claim 4, characterized in that, The method further includes: Obtain the data labels of the input data used to generate the multiplexed sub-data, and the model labels of any test model in the multiplexed slot; Based on the data tag and the model tag, a storage tag for the reused sub-data is generated, and the reused sub-data is stored according to the storage tag.

6. The method according to claim 1 or 4, characterized in that, The methods for constructing a test task based on the reused data and the associated slots of the reused slots include: Obtain the associated slots of the reused slot and the model to be tested corresponding to the associated slots; wherein, if the number of reused slots is one, the associated slot is a model slot whose corresponding slot number is greater than the reused slot number; or, if the number of reused slots is at least two, the associated slots of any reused slot include: model slots whose slot number is between the slot number corresponding to any reused slot and the slot number of the next reused slot of any reused slot; The test models corresponding to the associated slots are randomly combined to obtain one or more test model groups; A test task is constructed based on the corresponding reused sub-data in the reused data and a test model group.

7. The method according to claim 6, characterized in that, The test model group has a processing threshold, which indicates the threshold amount of data that the test model group can process; the method further includes: Obtain the data volume of the reused sub-data and the processing threshold of any test model group; When the amount of data of the reused sub-data of any test model group is greater than the processing threshold of any test model group, the reused sub-data is split based on the processing threshold of any test model group to obtain multiple data groups of the reused sub-data. Each of the multiple data groups is input into any of the test model groups to obtain the execution result of a test task.

8. A model testing device, characterized in that, include: The acquisition unit is used to acquire the set of models to be tested and the model complexity of each model to be tested in the set of models to be tested. The set of models to be tested contains m model slots. Each model slot corresponds to one or more models to be tested under the same model category. Different model slots correspond to different model categories. m is an integer greater than 1. The determining unit is used to determine the slot reuse value of the corresponding model slot based on the model complexity of the model to be tested corresponding to each model slot and the slot value of each model slot; the slot reuse value is used to indicate the importance of the output result of the corresponding model slot corresponding to the model to be tested. The determining unit is further configured to determine a reuse slot from the m model slots based on the slot reuse value of each model slot, as well as the input data corresponding to the reuse slot, and use the model output data obtained after inputting the input data into the reuse slot as reuse data. The testing unit is used to construct one or more test tasks based on the reused data and the associated slots of the reused slots, and to obtain test results for each model to be tested in the set of models to be tested based on the execution results of the test tasks.

9. A computer device, characterized in that, include: A processor for implementing one or more computer programs; A computer storage medium storing one or more computer programs, said one or more computer programs being adapted to be loaded by the processor and executed as described in any one of claims 1-7.

10. A computer storage medium, characterized in that, The computer storage medium stores one or more computer programs, which are adapted to be loaded by a processor and executed by the model testing method as described in any one of claims 1-7.

11. A computer product, characterized in that, The computer product includes a computer program adapted to be loaded by a processor and executed as described in any one of claims 1-7.

Images