Evaluation Method and Device of Algorithm Model, Storage Medium, Electronic Device

Abstract

An embodiment of the present invention provides a method and apparatus for evaluating an algorithm model, a storage medium, and an electronic device. The method includes: using a first algorithm model to output test values of K test samples to obtain K test values, where K is a natural number greater than 1; determining a first test sample corresponding to a first test value and a second test sample corresponding to a second test value among the K test values, where the first test value is greater than the second test value; searching for a third test value and a fourth test value in the determined test matrix; and evaluating the first algorithm model and the second algorithm model based on the third test value and the fourth test value. Through the present invention, the problem of time-consuming and laborious evaluation of algorithm models in the related art is solved, and the effect of efficiently and accurately evaluating algorithm models is achieved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of computers, and in particular, to a method and device for evaluating an algorithm model, a storage medium, and an electronic device. Background Art

[0002] With the continuous development of machine learning and deep learning, computer vision algorithms have been applied to various fields. With the rapid progress of the image and video processing capabilities of computers and embedded systems, prediction algorithms for image or video sequences emerge in an endless stream. For the current massive algorithms, researchers usually classify the algorithms into major categories such as detection, classification, and regression according to the final tasks of the algorithms. The regression task means that the algorithm performs a series of processes by inputting an image or video sequence, and finally outputs one or more floating-point scores, and relies on these scores to measure the attributes of the input data. For example, in a quality assessment algorithm, the algorithm evaluates the quality of image or video visual imaging through the output objective quality score; in a person re-identification algorithm, the score of the target is used to determine whether the test image and the query image are the same target. In these currently popular algorithm tasks, researchers have proposed a very large number of algorithms with different calculation methods respectively. These algorithms accept the same input and output the same type. In the process of evaluating the advantages and disadvantages of these algorithms, the accuracy and precision of the algorithms are often measured by objective evaluation indicators. However, in fact, these objective evaluation indicators all have certain defects more or less, and the human eye, as the final carrier, has a decisive advantage in evaluating the algorithm effect. However, in the face of massive data, subjective evaluation is time-consuming and laborious.

[0003] In view of the above technical problems, no effective solution has been proposed in the related art. Summary of the Invention

[0004] Embodiments of the present invention provide a method and device for evaluating an algorithm model, a storage medium, and an electronic device, so as to at least solve the problem of time-consuming and laborious evaluation of the algorithm model in the related art.

[0005] According to an embodiment of the present invention, there is provided a method for evaluating an algorithm model, including: using a first algorithm model to output test values of K test samples, obtaining K test values, where K is a natural number greater than 1; determining a first test sample corresponding to a first test value and a second test sample corresponding to a second test value among the K test values, where the first test value is greater than the second test value; searching for a third test value and a fourth test value in a determined test matrix, where the third test value is obtained by processing the first test sample based on a second algorithm model, and the fourth test value is obtained by processing the second test sample based on the second algorithm model; evaluating the first algorithm model and the second algorithm model based on the third test value and the fourth test value.

[0006] According to another embodiment of the present invention, there is provided an apparatus for evaluating an algorithm model, including: a first output module, configured to use a first algorithm model to output test values of K test samples, obtaining K test values, where K is a natural number greater than 1; a first determination module, configured to determine a first test sample corresponding to a first test value and a second test sample corresponding to a second test value among the K test values, where the first test value is greater than the second test value; a first search module, configured to search for a third test value and a fourth test value in a determined test matrix, where the third test value is obtained by processing the first test sample based on a second algorithm model, and the fourth test value is obtained by processing the second test sample based on the second algorithm model; a first evaluation module, configured to evaluate the first algorithm model and the second algorithm model based on the third test value and the fourth test value.

[0007] In an exemplary embodiment, the apparatus further includes: a second determination module, configured to determine the test matrix before using the first algorithm model to output test values of K test samples, obtaining K test values, where the test matrix includes I algorithm models and J test samples, both I and J are natural numbers greater than 1, and the elements in the test matrix are used to represent the test values of any test sample among the M test samples output by the N algorithm models.

[0008] In an exemplary embodiment, the apparatus further includes: a third determination module, configured to, after determining the test matrix, determine the i-th algorithm model among the I algorithm models as the first algorithm model; a fourth determination module, configured to determine the other algorithm models except the i-th algorithm model among the I algorithm models as the second algorithm model.

[0009] In an exemplary embodiment, the above device further includes: a first partitioning module, configured to, after determining the above test matrix, partition the above J test samples to obtain P sample blocks, where each of the above P sample blocks includes K test samples, and the above P is a natural number greater than 1.

[0010] In an exemplary embodiment, the above first searching module includes: a first searching unit, configured to search for the third test value corresponding to the above first test sample and the fourth test value corresponding to the above second test sample output by the above second algorithm model in the above test matrix based on the column index in the above test matrix.

[0011] In an exemplary embodiment, the above first evaluation module includes: a first determining unit, configured to determine that the numerical results output by the above first algorithm model and the above second algorithm model are consistent if it is determined that the third test value is less than or equal to the above fourth test value.

[0012] In an exemplary embodiment, the above first evaluation module includes: a second determining unit, configured to determine the similarity between the above first test sample and the above second test sample when the above third test value is greater than the above fourth test value; a third determining unit, configured to determine that the accuracy of the result output by the first algorithm model is higher than the accuracy of the result output by the second algorithm model when the similarity between the above first test sample and the above second test sample is greater than or equal to a preset threshold; a fourth determining unit, configured to determine that the accuracy of the result output by the second algorithm model is higher than the accuracy of the result output by the first algorithm model when the similarity between the above first test sample and the above second test sample is less than the preset threshold.

[0013] In an exemplary embodiment, the above device further includes: a first statistics module, configured to count the number of times that the accuracy of the result output by the first algorithm model is higher than the accuracy of the result output by the second algorithm model among the P sample blocks in the test matrix to obtain a first value, where each of the P sample blocks includes K test samples; a second statistics module, configured to count the number of times that the accuracy of the result output by the second algorithm model is higher than the accuracy of the result output by the first algorithm model among the P sample blocks in the test matrix to obtain a second value; a second evaluation module, configured to evaluate the above first algorithm model and the above second algorithm model based on the above first value and the above second value.

[0014] According to another embodiment of the present invention, there is also provided a computer-readable storage medium, in which a computer program is stored, and the computer program is configured to execute the steps in any one of the above method embodiments when running.

[0015] According to another embodiment of the present invention, an electronic device is further provided, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0016] Through the present invention, by using a first algorithm model to output test values of K test samples, K test values are obtained, where K is a natural number greater than 1; determining a first test sample corresponding to a first test value and a second test sample corresponding to a second test value among the K test values, where the first test value is greater than the second test value; searching for a third test value and a fourth test value in the determined test matrix, where the third test value is obtained by processing the first test sample based on a second algorithm model, and the fourth test value is obtained by processing the second test sample based on the second algorithm model; evaluating the first algorithm model and the second algorithm model based on the third test value and the fourth test value. The purpose of amplifying the difference in the output results of different algorithm models is achieved. Therefore, the problem of time-consuming and laborious evaluation of algorithm models in the related art can be solved, and the effect of efficiently and accurately evaluating algorithm models can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a hardware structure block diagram of a mobile terminal for an algorithm model evaluation method according to an embodiment of the present invention;

[0018] Figure 2 is a flowchart of an algorithm model evaluation method according to an embodiment of the present invention;

[0019] Figure 3 is a flowchart according to a specific embodiment of the present invention;

[0020] Figure 4 is a schematic diagram of element division according to an embodiment of the present invention;

[0021] Figure 5 is a structure block diagram of an algorithm model evaluation device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0022] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings and in conjunction with the embodiments.

[0023] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence.

[0024] The method embodiments provided in the embodiments of the present application can be executed on a mobile terminal, a computer terminal, or a similar computing device. Taking running on a mobile terminal as an example, Figure 1It is a block diagram of the hardware of a mobile terminal for an evaluation method of an algorithm model according to an embodiment of the present invention. As Figure 1 shown, the mobile terminal may include one or more ( Figure 1 only one is shown in the figure) processors 102 (the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Among them, the above mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those of ordinary skill in the art can understand that Figure 1 the structure shown is only schematic and does not limit the structure of the above mobile terminal. For example, the mobile terminal may further include more or fewer components than Figure 1 shown in the figure, or have a different configuration from Figure 1 shown in the figure.

[0025] The memory 104 can be used to store computer programs. For example, software programs and modules of application software, such as the computer program corresponding to the evaluation method of the algorithm model in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implements the above method. The memory 104 may include a high-speed random access memory, and may further include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory 104 may further include a memory remotely provided relative to the processor 102, and these remote memories may be connected to the mobile terminal through a network. Examples of the above network include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and their combinations.

[0026] The transmission device 106 is used to receive or send data via a network. Specific examples of the above network may include a wireless network provided by a communication provider of the mobile terminal. In one instance, the transmission device 106 includes a network adapter (Network Interface Controller, abbreviated as NIC), which can be connected to other network devices through a base station and thus can communicate with the Internet. In one instance, the transmission device 106 may be a radio frequency (Radio Frequency, abbreviated as RF) module, which is used to communicate with the Internet wirelessly.

[0027] In this embodiment, an evaluation method of an algorithm model is provided. Figure 2 It is a flowchart of the evaluation method of the algorithm model according to an embodiment of the present invention. As Figure 2 shown, the process includes the following steps:

[0028] Step S202: Use the first algorithm model to output the test values of K test samples, obtaining K test values, where K is a natural number greater than 1;

[0029] Step S204: Determine the first test sample corresponding to the first test value and the second test sample corresponding to the second test value among the K test values, where the first test value is greater than the second test value;

[0030] Step S206: Search for the third test value and the fourth test value in the determined test matrix, where the third test value is obtained by processing the first test sample based on the second algorithm model, and the fourth test value is obtained by processing the second test sample based on the second algorithm model;

[0031] Step S208: Evaluate the first algorithm model and the second algorithm model based on the third test value and the fourth test value.

[0032] Among them, the execution subject of the above steps can be a terminal, etc., but is not limited thereto.

[0033] This embodiment includes but is not limited to being applied to scenarios for evaluating the accuracy of the output results of algorithm models. For example, in a quality assessment algorithm, the algorithm evaluates the pros and cons of image or video visual imaging through the objective quality scores output; in a person re-identification algorithm, it determines whether a test image and a query image are of the same target by scoring the target.

[0034] In this embodiment, the first algorithm model can be used as a defensive algorithm, and the second algorithm model can be used as an offensive algorithm. The two algorithm models can magnify the differences in different output results.

[0035] In this embodiment, the first test value can be the maximum value, and the second test value can be the minimum value.

[0036] Through the above steps, by using the first algorithm model to output the test values of K test samples, obtaining K test values, where K is a natural number greater than 1; determining the first test sample corresponding to the first test value and the second test sample corresponding to the second test value among the K test values, where the first test value is greater than the second test value; searching for the third test value and the fourth test value in the determined test matrix, where the third test value is obtained by processing the first test sample based on the second algorithm model, and the fourth test value is obtained by processing the second test sample based on the second algorithm model; evaluating the first algorithm model and the second algorithm model based on the third test value and the fourth test value. The purpose of magnifying the differences in the output results of different algorithm models is achieved. Therefore, the problem of time-consuming and laborious evaluation of algorithm models in related technologies can be solved, and the effect of efficiently and accurately evaluating algorithm models can be achieved.

[0037] In an exemplary embodiment, before obtaining K test values by using a first algorithm model to output the test values of K test samples, the method further includes:

[0038] S1. Determine a test matrix, where the test matrix includes I algorithm models and J test samples, both I and J are natural numbers greater than 1, and the elements in the test matrix are used to represent the test values of any one of the M test samples output by N algorithm models.

[0039] In this embodiment, the test samples include but are not limited to pictures or videos. The element q in the i-th row and j-th column of the test matrix i,j is used to identify the output score of the j-th sample under the i-th algorithm test.

[0040] In an exemplary embodiment, after determining the test matrix, the method further includes:

[0041] S1. Determine the i-th algorithm model among the I algorithm models as the first algorithm model;

[0042] S2. Determine the other algorithm models except the i-th algorithm model among the I algorithm models as the second algorithm model.

[0043] In this embodiment, for example, select the i-th algorithm as the defender D i (initialize i = 1), and the remaining I - 1 algorithms are the attackers.

[0044] In an exemplary embodiment, after determining the test matrix, the method further includes:

[0045] S1. Divide the J test samples to obtain P sample blocks, where each of the P sample blocks includes test samples, and P is a natural number greater than 1.

[0046] In this embodiment, divide the J samples equidistantly to obtain P sample blocks, each sample block has K test samples, and the P * K test samples are the J samples.

[0047] In an exemplary embodiment, finding the third test value corresponding to the first test sample and the fourth test value corresponding to the second test sample output by using a second algorithm model in the determined test matrix includes:

[0048] S1. Based on the column index in the test matrix, find the third test value corresponding to the first test sample and the fourth test value corresponding to the second test sample output by the second algorithm model in the test matrix.

[0049] In this embodiment, the test matrix includes the third test value corresponding to the first test sample and the fourth test value corresponding to the second test sample that are pre-stored.

[0050] In an exemplary embodiment, evaluating the first algorithm model and the second algorithm model based on the third test value and the fourth test value includes:

[0051] S1. If it is determined that the third test value is less than or equal to the fourth test value, it is determined that the numerical results output by the first algorithm model and the second algorithm model are consistent.

[0052] In this embodiment, for example, the scores of two images in the first algorithm model are found in the test matrix according to the column index. If it indicates that the predictions of the two algorithm models are consistent.

[0053] In an exemplary embodiment, evaluating the first algorithm model and the second algorithm model based on the third test value and the fourth test value includes:

[0054] S1. In the case where the third test value is greater than the fourth test value, determine the similarity between the first test sample and the second test sample;

[0055] S2. In the case where the similarity between the first test sample and the second test sample is greater than or equal to a preset threshold, determine that the accuracy of the result output by the first algorithm model is higher than the accuracy of the result output by the second algorithm model;

[0056] S3. In the case where the similarity between the first test sample and the second test sample is less than the preset threshold, determine that the accuracy of the result output by the second algorithm model is higher than the accuracy of the result output by the first algorithm model.

[0057] In an exemplary embodiment, the method further includes:

[0058] S1. Among the P sample blocks in the test matrix, count the number of times that the accuracy of the result output by the first algorithm model is higher than the accuracy of the result output by the second algorithm model to obtain a first value, where each of the P sample blocks includes K test samples;

[0059] S2. Among the P sample blocks in the test matrix, count the number of times that the accuracy of the result output by the second algorithm model is higher than the accuracy of the result output by the first algorithm model to obtain a second value;

[0060] S3. Evaluate the first algorithm model and the second algorithm model based on the first value and the second value.

[0061] The present invention will be described below with specific embodiments:

[0062] In this embodiment, each algorithm model is divided into an attacking party (equivalent to the first algorithm model) and a defending party (equivalent to the second algorithm model). Through the mutual game between the attacking party and the defending party, if a contradiction occurs during the game, human subjective judgment can be intervened, and finally the evaluation score of the algorithm is obtained.

[0063] As Figure 3 shown, this embodiment includes the following steps:

[0064] S301. Assume there are I existing algorithm models and J test samples (pictures or videos), then matrix Q I×J is obtained. The element q i,j in the i-th row and j-th column of the matrix represents the output score of the j-th sample under the i-th algorithm test;

[0065] S302. Select the i-th algorithm model as the defending party D i (initialize i = 1), and the remaining I - 1 algorithm models are the attacking party;

[0066] S303. Sort the entire matrix column by column according to the values in the i-th row;

[0067] S304. Divide the J samples into P sample blocks at equal intervals, and each block has K elements, as specifically Figure 4 described;

[0068] S305. Select the attacking algorithm model where i * ∈I, and i * ≠i;

[0069] S306. In the P-th sample block, the minimum value among the K image scores obtained by testing the defending algorithm model D i is q i,k*K-K , and the maximum value is q i,k*K ;

[0070] According to the column index, find two images in the attacking algorithm model If then it means that the attacking algorithm model and the defending algorithm model have the same prediction; otherwise, it means that the two algorithm models are inconsistent, and at this time, human eye intervention is required for judgment; if these two images are indeed visually then it represents successful defense; otherwise, it represents successful attack. Count the number of successful defenses as n times and the number of failed defenses as m times. For the algorithm model D i the score on this block is Score = n - m;

[0071] S307, loop S306 until all blocks are tested, and statistically obtain the defense algorithm model D i , at this time, for the algorithm model D i the score is Loop S302 until all algorithm models are tested; obtain the evaluation scores of all algorithm models through the above method.

[0072] In summary, this embodiment adopts the mutual game between algorithm models, sets up an offensive algorithm model and a defensive algorithm model, amplifies the difference in the output results between different algorithm models through the mutual game between the offensive side and the defensive side, and uses the subjective evaluation method to quantify the evaluation score, realizing high-precision, high-robustness, and high-efficiency evaluation results.

[0073] Through the description of the above implementation manners, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation manner. Based on such an understanding, the technical solution of the present invention, in essence, or the part that makes a contribution to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disc), and includes several instructions for causing a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0074] In this embodiment, an evaluation device for an algorithm model is further provided. This device is used to implement the above embodiments and preferred implementation manners, and those that have been described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that can achieve a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0075] Figure 5 is a structural block diagram of an evaluation device for an algorithm model according to an embodiment of the present invention. As Figure 5 shown, the device includes:

[0076] A first output module 52, configured to use a first algorithm model to output test values of K test samples, and obtain K test values, where the above K is a natural number greater than 1;

[0077] A first determination module 54, configured to determine a first test sample corresponding to a first test value and a second test sample corresponding to a second test value among the above K test values, where the above first test value is greater than the above second test value;

[0078] The first search module 56 is configured to search for a third test value and a fourth test value in a determined test matrix, where the third test value is obtained by processing a first test sample based on a second algorithm model, and the fourth test value is obtained by processing a second test sample based on the second algorithm model;

[0079] The first evaluation module 58 is configured to evaluate the first algorithm model and the second algorithm model based on the third test value and the fourth test value.

[0080] In an exemplary embodiment, the apparatus further includes:

[0081] The second determination module is configured to determine the test matrix before obtaining K test values of K test samples by using the first algorithm model, where the test matrix includes I algorithm models and J test samples, both I and J are natural numbers greater than 1, and the elements in the test matrix are used to represent the test values of any one of the M test samples output by the N algorithm models.

[0082] In an exemplary embodiment, the apparatus further includes:

[0083] The third determination module is configured to, after determining the test matrix, determine the i-th algorithm model among the I algorithm models as the first algorithm model;

[0084] The fourth determination module is configured to determine the other algorithm models among the I algorithm models except the i-th algorithm model as the second algorithm model.

[0085] In an exemplary embodiment, the apparatus further includes:

[0086] The first partitioning module is configured to, after determining the test matrix, partition the J test samples to obtain P sample blocks, where each sample block in the P sample blocks includes K test samples, and P is a natural number greater than 1.

[0087] In an exemplary embodiment, the first search module includes:

[0088] The first search unit is configured to search for the third test value corresponding to the first test sample and the fourth test value corresponding to the second test sample output by the second algorithm model in the test matrix based on the column index in the test matrix;

[0089] In an exemplary embodiment, the first evaluation module includes:

[0090] A first determination unit, configured to determine that the numerical results output by the first algorithm model and the second algorithm model are consistent if it is determined that the third test value is less than or equal to the fourth test value.

[0091] In an exemplary embodiment, the first evaluation module includes:

[0092] A second determination unit, configured to determine the similarity between the first test sample and the second test sample when the third test value is greater than the fourth test value;

[0093] A third determination unit, configured to determine that the accuracy of the result output by the first algorithm model is higher than the accuracy of the result output by the second algorithm model when the similarity between the first test sample and the second test sample is greater than or equal to a preset threshold;

[0094] A fourth determination unit, configured to determine that the accuracy of the result output by the second algorithm model is higher than the accuracy of the result output by the first algorithm model when the similarity between the first test sample and the second test sample is less than the preset threshold.

[0095] In an exemplary embodiment, the apparatus further includes:

[0096] A first statistics module, configured to count the number of times that the accuracy of the result output by the first algorithm model is higher than the accuracy of the result output by the second algorithm model among P sample blocks in the test matrix to obtain a first value, where each of the P sample blocks includes K test samples;

[0097] A second statistics module, configured to count the number of times that the accuracy of the result output by the second algorithm model is higher than the accuracy of the result output by the first algorithm model among P sample blocks in the test matrix to obtain a second value;

[0098] A second evaluation module, configured to evaluate the first algorithm model and the second algorithm model based on the first value and the second value.

[0099] It should be noted that the above-mentioned modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: all the above-mentioned modules are located in the same processor; or, the above-mentioned modules are respectively located in different processors in any combination form.

[0100] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, and the computer program is configured to execute the steps in any one of the above method embodiments when running.

[0101] In this embodiment, the above computer-readable storage medium may be configured to store a computer program for executing the above steps.

[0102] In an exemplary embodiment, the above computer-readable storage medium may include, but is not limited to: various media such as USB flash drives, read-only memory (ROM for short), random access memory (RAM for short), mobile hard disks, magnetic disks, or optical discs that can store computer programs.

[0103] An embodiment of the present invention also provides an electronic device, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0104] In an exemplary embodiment, the above electronic device may further include a transmission device and an input / output device. Among them, the transmission device is connected to the above processor, and the input / output device is connected to the above processor.

[0105] In an exemplary embodiment, the above processor may be configured to execute the above steps through a computer program.

[0106] Specific examples in this embodiment may refer to the examples described in the above embodiments and exemplary embodiments, and will not be repeated here.

[0107] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general-purpose computing device. They can be concentrated on a single computing device or distributed on a network composed of multiple computing devices. They can be implemented by program code executable by the computing device, so that they can be stored in the storage device and executed by the computing device. And in some cases, the steps shown or described may be executed in a different order than here, or they may be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them may be fabricated into a single integrated circuit module to implement. In this way, the present invention is not limited to any specific combination of hardware and software.

[0108] The above is only the preferred embodiment of the present invention and is not used to limit the present invention. For those skilled in the art, the present invention may have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An evaluation method for an algorithm model, characterized in that Including: Outputting test values of K test samples by using a first algorithm model to obtain K test values, where K is a natural number greater than 1, and the test samples are video samples; Determining a first test sample corresponding to a first test value and a second test sample corresponding to a second test value among the K test values, where the first test value is greater than the second test value; Searching for a third test value and a fourth test value in a determined test matrix, where the third test value is obtained by processing the first test sample based on a second algorithm model, and the fourth test value is obtained by processing the second test sample based on the second algorithm model; Evaluating the first algorithm model and the second algorithm model based on the third test value and the fourth test value; Evaluating the first algorithm model and the second algorithm model based on the third test value and the fourth test value includes: if it is determined that the third test value is less than or equal to the fourth test value, determining that the numerical results output by the first algorithm model and the second algorithm model are consistent.

2. The method according to claim 1, wherein Before outputting test values of K test samples by using a first algorithm model to obtain K test values, the method further includes: Determining the test matrix, where the test matrix includes I algorithm models and J test samples, both I and J are natural numbers greater than 1, and the elements in the test matrix are used to represent the test values of any test sample among the M test samples output by N algorithm models.

3. The method according to claim 2, wherein After determining the test matrix, the method further includes: Determining the i-th algorithm model among the I algorithm models as the first algorithm model; Determining the other algorithm models among the I algorithm models except the i-th algorithm model as the second algorithm model.

4. The method according to claim 2, wherein After determining the test matrix, the method further includes: Dividing the J test samples to obtain P sample blocks, where each of the P sample blocks includes K test samples, and P is a natural number greater than 1.

5. The method according to claim 1, characterized in that, Searching for the third test value corresponding to the first test sample and the fourth test value corresponding to the second test sample output by using the second algorithm model in the determined test matrix includes: Searching for the third test value corresponding to the first test sample and the fourth test value corresponding to the second test sample output by the second algorithm model in the test matrix based on the column index in the test matrix.

6. The method according to claim 1, characterized in that Evaluating the first algorithm model and the second algorithm model based on the third test value and the fourth test value includes: Determining the similarity between the first test sample and the second test sample when the third test value is greater than the fourth test value; Determining that the accuracy of the result output by the first algorithm model is higher than the accuracy of the result output by the second algorithm model when the similarity between the first test sample and the second test sample is greater than or equal to a preset threshold; When the similarity between the first test sample and the second test sample is less than the preset threshold, it is determined that the accuracy of the result output by the second algorithm model is higher than the accuracy of the result output by the first algorithm model.

7. The method according to claim 6, wherein The method further includes: In the P sample blocks in the test matrix, count the number of times that the accuracy of the result output by the first algorithm model is higher than the accuracy of the result output by the second algorithm model to obtain a first value, where each of the P sample blocks includes K test samples; In the P sample blocks in the test matrix, count the number of times that the accuracy of the result output by the second algorithm model is higher than the accuracy of the result output by the first algorithm model to obtain a second value; Evaluate the first algorithm model and the second algorithm model based on the first value and the second value.

8. An evaluation device for an algorithm model, characterized in that, It includes: A first output module, configured to use a first algorithm model to output test values of K test samples to obtain K test values, where K is a natural number greater than 1, and the test samples are video samples; A first determination module, configured to determine a first test sample corresponding to a first test value and a second test sample corresponding to a second test value among the K test values, where the first test value is greater than the second test value; A first search module, configured to search for a third test value and a fourth test value in the determined test matrix, where the third test value is obtained by processing the first test sample based on a second algorithm model, and the fourth test value is obtained by processing the second test sample based on the second algorithm model; A first evaluation module, configured to evaluate the first algorithm model and the second algorithm model based on the third test value and the fourth test value; The first evaluation module is further configured to evaluate the first algorithm model and the second algorithm model based on the third test value and the fourth test value in the following manner: If it is determined that the third test value is less than or equal to the fourth test value, it is determined that the numerical results output by the first algorithm model and the second algorithm model are consistent.

9. A computer-readable storage medium, characterized in that, A computer program is stored in the computer-readable storage medium, where the computer program, when executed by a processor, implements the method described in any one of claims 1 to 7.

10. An electronic device, comprising a memory and a processor, characterized in that, A computer program is stored in the memory, and the processor is configured to run the computer program to execute the method described in any one of claims 1 to 7.

Images