Algorithm testing method and device

Abstract

The present application is applicable to the field of product testing technology, and provides an algorithm testing method and device, including: calling each algorithm module of a second hardware platform in sequence, and using the called algorithm modules and other algorithm modules of the first hardware platform excluding the called algorithm modules to test test data, and obtain multiple cross-test results; using all algorithm modules of the first hardware platform to test the test data, and obtain full-process test results; determining a target algorithm according to the cross-test results, the full-process test results, and a benchmark value, and by calling different algorithm modules of different hardware platforms for testing, respectively, obtaining errors of different algorithm modules on different hardware platforms and mutual influence of different algorithm modules in the product, and being able to determine the algorithm that needs to be iterated according to the errors of different algorithm modules on different hardware platforms, so as to quickly iterate the algorithm and reduce the error caused by algorithm transplantation.

Description

Technical Field

[0001] The present application belongs to the field of testing technology, and in particular, relates to an algorithm testing method and device. Background Art

[0002] An algorithm is a series of clear instructions for solving a problem, representing a strategy mechanism for solving a problem in a systematic way. Usually, algorithms are developed on computers, and when they are applied, they need to be ported to other hardware platforms. However, during the porting process of algorithms to different hardware platforms, due to the differences in the underlying instruction sets and algorithm acceleration processing mechanisms of different hardware platforms, there will be more or less differences in the algorithm accuracy on the hardware platform before porting and the algorithm accuracy on the hardware platform after porting. These differences may lead to poor practical performance of the product to which the algorithm has been ported.

[0003] In order to determine the differences in the algorithm during the transplantation process on different hardware platforms, the algorithm can be improved to ensure the practical performance of the transplanted product. It is usually necessary to conduct overall testing on different hardware platforms. However, since there is more than one algorithm in the product, the overall test can only test the overall differences of multiple algorithms on different hardware platforms in the product, and cannot determine the differences of a single algorithm module on different hardware platforms. If only a single algorithm module is tested and then the differences between different algorithm modules on different hardware platforms are compared, it is impossible to test the mutual influence of different algorithm modules in the product. Summary of the invention

[0004] In view of this, the present application provides an algorithm testing method and device to solve the problem that the existing algorithm testing method cannot target the differences between different algorithms in the entire product and the mutual influence of different algorithm modules in the product.

[0005] A first aspect of an embodiment of the present application provides an algorithm testing method, which is applied to a first hardware platform, wherein the first hardware platform includes at least two algorithm modules, and the algorithm testing method includes:

[0006] Calling each algorithm module of the second hardware platform in sequence, and using the called algorithm module and other algorithm modules of the first hardware platform excluding the called algorithm module to test the test data, to obtain multiple cross-test results; wherein the second hardware platform is communicatively connected with the first hardware platform, and the algorithm module included in the second hardware platform corresponds to the algorithm module of the first hardware platform; the second hardware platform includes at least two algorithm modules;

[0007] Using all algorithm modules of the first hardware platform to test the test data, and obtaining full-process test results;

[0008] A target algorithm is determined based on the cross-test results, the full-process test results and the benchmark value.

[0009] Further, the first hardware platform includes a first algorithm module, a second algorithm module and a third algorithm module, and the second hardware platform includes a fourth algorithm module corresponding to the first algorithm module, a fifth algorithm module corresponding to the second algorithm module and a sixth algorithm module corresponding to the third algorithm module;

[0010] Correspondingly, the algorithm modules of the second hardware platform are called in sequence, and the called algorithm modules and other algorithm modules of the first hardware platform except the called algorithm modules are used to test the test data to obtain multiple cross-test results, including:

[0011] calling the fourth algorithm module of the second hardware platform, and using the fourth algorithm module, the second algorithm module and the third algorithm module to test the test data to obtain a first test result;

[0012] calling the fifth algorithm module of the second hardware platform, and using the first algorithm module, the fifth algorithm module and the third algorithm module to test the test data to obtain a second test result;

[0013] The sixth algorithm module of the second hardware platform is called, and the test data is tested using the first algorithm module, the second algorithm module and the sixth algorithm module to obtain a third test result.

[0014] Further, determining the target algorithm according to the cross-test result, the full-process test result and the benchmark value includes:

[0015] A target algorithm is determined according to the first test result, the second test result, the third test result, the full-process test result and a benchmark value.

[0016] Further, determining a target algorithm according to the first test result, the second test result, the second test result, the full-process test result and the benchmark value includes:

[0017] Determine a first algorithm accuracy according to the first test result and the reference value;

[0018] Determine a second algorithm accuracy according to the second test result and the reference value;

[0019] Determine a third algorithm accuracy according to the third test result and the reference value;

[0020] Determine the accuracy of the fourth algorithm according to the full-process test result and the reference value;

[0021] The target algorithm is determined according to the first algorithm accuracy, the second algorithm accuracy, the third algorithm accuracy, and the fourth algorithm accuracy.

[0022] Further, determining the target algorithm module according to the first algorithm accuracy, the second algorithm accuracy, the third algorithm accuracy, and the fourth algorithm accuracy includes:

[0023] Obtaining a first error degree between the first algorithm accuracy and the fourth algorithm accuracy;

[0024] Obtaining a second error degree between the second algorithm accuracy and the fourth algorithm accuracy;

[0025] Obtaining a third error degree between the third algorithm accuracy and the fourth algorithm accuracy;

[0026] If the first error degree is greater than a preset error threshold, determining the algorithm in the first algorithm module as a target algorithm;

[0027] If the second error degree is greater than a preset error threshold, determining the algorithm in the second algorithm module as a target algorithm;

[0028] If the third error degree is greater than a preset error threshold, the algorithm in the third algorithm module is determined to be the target algorithm.

[0029] Furthermore, the method further comprises: obtaining verification test results; wherein the verification test results include test results obtained by the second hardware platform calling each algorithm module of the first hardware platform for testing respectively and test results obtained by the second hardware platform based on each algorithm module of the second hardware platform for testing;

[0030] The accuracy influencing error is determined according to the verification test result, the cross test result, the full-process test result and the benchmark value.

[0031] Further, the determining the accuracy impact error according to the verification test result, the cross test result, the full process test result and the reference value includes:

[0032] determining a degree of verification error based on the verification test result and the reference value;

[0033] Determine a target error degree according to the cross test result, the full process test result and the benchmark value;

[0034] The accuracy influencing error is determined according to the verification error degree and the target error degree.

[0035] A second aspect of an embodiment of the present application provides an algorithm testing device, including:

[0036] A cross-test unit, used to call each algorithm module of the second hardware platform in sequence, and use the called algorithm modules and other algorithm modules of the first hardware platform excluding the called algorithm modules to test the test data, so as to obtain multiple cross-test results; wherein the second hardware platform is communicatively connected with the first hardware platform, and the algorithm modules included in the second hardware platform correspond to the algorithm modules of the first hardware platform; the second hardware platform includes at least two algorithm modules;

[0037] A self-test unit, used to test the test data using all algorithm modules of the first hardware platform to obtain a full-process test result;

[0038] A determination unit is used to determine a target algorithm according to the cross-test result, the full-process test result and a benchmark value.

[0039] A third aspect of an embodiment of the present application provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the terminal device, wherein the processor implements the steps of the algorithm testing method provided in the first aspect when executing the computer program.

[0040] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the algorithm testing method provided in the first aspect are implemented.

[0041] A fifth aspect of an embodiment of the present application provides a computer program product. When the computer program product runs on a terminal device, the terminal device executes each step of the algorithm testing method described in any one of the first aspects above.

[0042] The algorithm testing method and device provided by the embodiments of the present application have the following beneficial effects:

[0043] By calling different algorithm modules of different hardware platforms for testing, the errors of different algorithm modules on different hardware platforms and the mutual influence of different algorithm modules in the product are obtained, and the algorithm that needs to be iterated can be determined according to the errors of different algorithm modules on different hardware platforms, so as to quickly iterate the algorithm and reduce the error caused by algorithm transplantation. This effectively solves the problem that the existing algorithm testing method cannot target the differences between different algorithms in the entire product and the mutual influence of different algorithm modules in the product. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0045] Figure 1 It is a schematic diagram of an architecture of an application scenario to which an algorithm testing method provided in an embodiment of the present application is applicable;

[0046] Figure 2 It is a flowchart of an implementation of an algorithm testing method provided in one embodiment of the present application;

[0047] Figure 3 is a schematic diagram of a process for acquiring test data in an algorithm testing method provided in another embodiment of the present application;

[0048] Figure 4 This is a schematic diagram of an application example of an algorithm testing method provided by an embodiment of the present application;

[0049] Figure 5 is a structural schematic diagram of an algorithm testing device provided by another embodiment of the present application;

[0050] Figure 6 It is a structural diagram of a terminal device provided in another embodiment of the present application. DETAILED DESCRIPTION

[0051] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0052] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0053] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0054] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0055] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0056] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0057] The present application proposes an algorithm testing method, which calls different algorithm modules of different hardware platforms for testing respectively, obtains the errors of different algorithm modules on different hardware platforms and the mutual influence of different algorithm modules in the product, and can determine the algorithm that needs to be iterated according to the errors of different algorithm modules on different hardware platforms, so as to quickly iterate the algorithm and reduce the error caused by algorithm transplantation.

[0058] See also Figure 1 , Figure 1 Schematic diagram of an algorithm testing method provided in an embodiment of the present application. Figure 1 As shown, the first hardware platform 10 may include at least two algorithm modules. The second hardware platform 20 includes an algorithm module corresponding to the first hardware platform 10. It should be noted that the algorithm in the algorithm module of the second hardware platform 20 is transplanted from the first hardware platform 10, so the algorithm module included in the second hardware platform 20 corresponds to the algorithm module of the first hardware platform 10. Of course, the second hardware platform 20 also includes at least two algorithm modules. Exemplarily, the algorithm in the fourth algorithm module of the second hardware platform 20 is transplanted from the algorithm in the first algorithm module of the first hardware platform 10, then the fourth algorithm module corresponds to the first algorithm module.

[0059] In the embodiment of the present application, the first hardware platform 10 may be a personal computer (PC) (hereinafter referred to as PC) used for algorithm development. Of course, the first hardware platform 10 may also be other electronic devices, which is not limited in the present application.

[0060] The second hardware platform 20 may be an actual electronic device that applies the algorithm developed by the first hardware platform 10, such as a 3D camera, a surveillance camera, or other electronic device. Specifically, the second hardware platform 20 may also be a processing chip in the electronic device, such as a processing chip in the 3D camera.

[0061] In the embodiment of the present application, the first hardware platform 10 may include a first algorithm module 11, a second algorithm module 12 and a third algorithm module 13. The second hardware platform 20 may include a fourth algorithm module 21, a fifth algorithm module 22 and a sixth algorithm module 23. Among them, the first algorithm module 11 corresponds to the fourth algorithm module 21, the second algorithm module 12 corresponds to the fifth algorithm module 22, and the third algorithm module 13 corresponds to the sixth algorithm module 23. That is to say, the algorithm in the first algorithm module 11 is the same as the algorithm in the fourth algorithm module 21, for example, both are head detection algorithms. The algorithm in the second algorithm module 12 is the same as the algorithm in the fifth algorithm module 22, for example, both are head tracking algorithms. The algorithm in the third algorithm module 13 is the same as the algorithm in the sixth algorithm module 23, both are human key point detection algorithms.

[0062] The first hardware platform 10 can call the algorithm module of the second hardware platform 20 through the communication protocol. For example, the first hardware platform 10 can call the fourth algorithm module 21 of the second hardware platform 20. Of course, the second hardware platform 20 can also call the algorithm module of the first hardware platform 10.

[0063] By respectively calling the algorithm modules of the second hardware platform 20 through the first hardware platform 10, and then using the called algorithm modules and its own other algorithm modules (i.e., other algorithm modules after excluding the called algorithm modules) to test the test data, the errors of different algorithm modules on different hardware platforms and the mutual influence of different algorithm modules in the product can be obtained, and the algorithm that needs to be iterated can be determined according to the errors of different algorithm modules on different hardware platforms, so as to quickly iterate the algorithm and reduce the error caused by algorithm transplantation.

[0064] Specifically, the first hardware platform 10 can use the fourth algorithm module 21, the second algorithm module 12 and the third algorithm module 13 to test the test data simultaneously when calling the fourth algorithm module 21, use the first algorithm module 11, the fifth algorithm module 22 and the third algorithm module 13 to test the test data simultaneously when calling the fifth algorithm module 22, and use the first algorithm module 11, the second algorithm module 12 and the sixth algorithm module 23 to test the test data simultaneously when calling the sixth algorithm module 23.

[0065] In an embodiment of the present application, the hypertext transfer protocol (http protocol) can be used to implement calls between the first hardware platform 10 and the second hardware platform 20, that is, the first hardware platform 10 uses the http protocol to respectively call the fourth algorithm module 21, the fifth algorithm module 22 and the sixth algorithm module 23 of the second hardware platform.

[0066] It should be noted that the above-mentioned first hardware platform 10 and the above-mentioned second hardware platform 20 can be interchangeable, that is, the above-mentioned first hardware platform 10 can also be an electronic device for applying algorithms, such as 3D cameras, surveillance cameras and other electronic devices, and the above-mentioned second hardware platform 20 can also be an electronic device for developing algorithms. The above-mentioned second hardware platform 20 can also call various algorithm modules of the first hardware platform 10, and this application does not impose any restrictions on this.

[0067] In order to explain in detail an algorithm testing method provided in an embodiment of the present application, the following will be combined with Figure 1 and Figure 2 For detailed explanation:

[0068] See also Figure 2 , Figure 2 The implementation flow diagram of the algorithm testing method provided in the embodiment of the present application is shown. The execution subject of the algorithm testing method provided in the embodiment of the present application may be Figure 1 The first hardware platform 10 in.

[0069] like Figure 2 As shown, the above algorithm testing method includes the following steps:

[0070] S11: calling each algorithm module of the second hardware platform in sequence, and using the called algorithm modules and other algorithm modules of the first hardware platform excluding the called algorithm modules to test the test data, to obtain a plurality of cross-test results.

[0071] In the embodiment of the present application, the second hardware platform is communicatively connected to the first hardware platform, and the algorithm module included in the second hardware platform corresponds to the algorithm module of the first hardware platform.

[0072] In an embodiment of the present application, the first hardware platform may include at least two algorithm modules, and the algorithms in each algorithm module are different. For example, in the application scenario of counting the number of people entering and leaving a shopping mall, the first hardware platform may include three algorithm modules: a head detection algorithm module, a head tracking algorithm module, and a human key point detection algorithm module. Correspondingly, the second hardware platform also includes at least two algorithm modules, that is, the second hardware platform will also include three algorithm modules: a head detection algorithm module, a head tracking algorithm module, and a human key point detection algorithm module.

[0073] In an embodiment of the present application, the first hardware platform sequentially calls each algorithm module in the second hardware platform through a communication protocol with the second hardware platform, and then tests the test data in combination with other algorithm modules except the called algorithm module. That is, the first hardware platform calls an algorithm module of the second hardware platform, and then uses the called algorithm module and other algorithm modules except the called algorithm module to test the test data once, and a cross-test result can be obtained. The first hardware platform will sequentially call each algorithm module of the second hardware platform, and then each time the test data is tested using the called algorithm module and other algorithm modules except the called algorithm module, and a cross-test result can be obtained in one test.

[0074] In an embodiment of the present application, the first hardware platform includes a first algorithm module, a second algorithm module and a third algorithm module, and the second hardware platform includes a fourth algorithm module corresponding to the first algorithm module, a fifth algorithm module corresponding to the second algorithm module and a sixth algorithm module corresponding to the third algorithm module;

[0075] Accordingly, S11 includes the following steps:

[0076] calling the fourth algorithm module of the second hardware platform, and using the fourth algorithm module, the second algorithm module and the third algorithm module to test the test data to obtain a first test result;

[0077] calling the fifth algorithm module of the second hardware platform, and using the first algorithm module, the fifth algorithm module and the third algorithm module to test the test data to obtain a second test result;

[0078] The sixth algorithm module of the second hardware platform is called, and the first algorithm module, the second algorithm module and the third algorithm module are used to test the test data to obtain a third test result.

[0079] Similarly, in the application scenario of counting the number of people entering and leaving a shopping mall, the first hardware platform may include three algorithm modules: a head detection algorithm module, a head tracking algorithm module, and a human key point detection algorithm module. Correspondingly, the second hardware platform may also include three algorithm modules: a head detection algorithm module, a head tracking algorithm module, and a human key point detection algorithm module.

[0080] First, the first hardware platform calls the head detection algorithm module of the second hardware platform, and then uses the called head detection algorithm module, its own head tracking algorithm module, and its own human key point detection algorithm module to test the test data to obtain the number of people entering the shopping mall, leaving the shopping mall, and passing through the shopping mall after processing by the above three algorithm modules. This is the first test result.

[0081] Then the first hardware platform tests the test data through its own head detection algorithm module, the called head tracking algorithm module of the second hardware platform, and its own human key point detection algorithm module, and also obtains the number of people entering the shopping mall, leaving the shopping mall, and passing through the shopping mall after processing by the above three algorithm modules. This is the second test result.

[0082] Next, the first hardware platform tests the test data through its own head detection algorithm module, its own head tracking algorithm module and the human feature point algorithm module of the called second platform, and also obtains the number of people entering the shopping mall, leaving the shopping mall and passing through the shopping mall after processing by the above three algorithm modules. This is the third test result.

[0083] S12: Use all algorithm modules of the first hardware platform to test the test data to obtain full-process test results.

[0084] In the embodiment of the present application, in order to determine the error in algorithm accuracy on different platforms, it is also necessary to test the same test data through all the algorithm modules of the above-mentioned first hardware platform itself, and then obtain the corresponding full-process test results.

[0085] Exemplarily, taking the above-mentioned application scenario of counting the number of people entering and leaving a shopping mall or store as an example, the above-mentioned first hardware platform may include three algorithm modules: head detection algorithm module, head tracking algorithm module, and human key point detection algorithm module. The first hardware platform does not call the algorithm module of the second hardware platform, but tests the test data through its own head detection algorithm module, head tracking algorithm module, and human key point detection algorithm module to obtain the number of people entering the shopping mall or store, the number of people leaving the shopping mall or store, and the number of people passing through the shopping mall or store. This is the full process test result.

[0086] S13: Determine a target algorithm according to the cross-test result, the full-process test result and a benchmark value.

[0087] In the embodiment of the present application, the above-mentioned reference value is an accurate result that can be obtained based on the test data. For example, in the application scenario of counting the number of people entering and leaving a store in a shopping mall, the actual number of people entering and leaving the mall is the reference value.

[0088] In the embodiment of the present application, by comparing the cross-test results and the full-process test results with the benchmark values ​​respectively, the corresponding algorithm accuracy can be determined, and then it can be concluded which algorithm will affect the performance of the product, thereby determining the target algorithm.

[0089] In the embodiment of the present application, the above-mentioned target algorithm is the algorithm that needs to be improved. After the target algorithm is determined, the developer can be notified to improve the algorithm so as to quickly iterate the algorithm and reduce the error caused by algorithm transplantation.

[0090] In an embodiment of the present application, the above-mentioned multiple cross-test results may include a first test result, a second test result, and a third test result.

[0091] In one embodiment of the present application, the above S13 includes the following steps:

[0092] Determine a first algorithm accuracy according to the first test result and the reference value;

[0093] Determine a second algorithm accuracy according to the second test result and the reference value;

[0094] Determine a third algorithm accuracy according to the third test result and the reference value;

[0095] Determine the accuracy of the fourth algorithm according to the full-process test result and the reference value;

[0096] The target algorithm is determined according to the first algorithm accuracy, the second algorithm accuracy, the third algorithm accuracy, and the fourth algorithm accuracy.

[0097] In the embodiment of the present application, the application scenario of the above-mentioned statistics of the number of people entering and leaving the mall and the store is also taken as an example. The first hardware platform is tested by calling the head detection algorithm module of the second hardware platform, and then the head detection algorithm module of the second hardware platform called, the head tracking algorithm module of the self, and the key point detection algorithm module of the human body are used to test the test data, and the number of people entering the mall and the store, the number of people leaving the mall and the store, and the number of people passing through the mall and the store (the first test result) after the above-mentioned three algorithm modules are processed are compared with the number of people entering the mall and the store, the number of people leaving the mall and the store, and the number of people passing through the mall and the store (the reference value) manually marked based on the video data, and then the first algorithm accuracy is determined. For example, based on the indicator of the number of people entering the mall and the store, it is assumed that the number of people entering the mall and the store in the above-mentioned first test result is 75 people, and the number of people entering the mall and the store based on the video data is 80 people, then the algorithm accuracy is 93.75%. Assuming that the number of people leaving the mall and the store is analyzed, it is assumed that the number of people entering the mall and the store in the above-mentioned first test result is 55 people, and the number of people entering the mall and the store based on the video data is 55, then the algorithm accuracy is 100%. Of course, it is also possible to analyze based on the indicator of the number of people passing through the mall and store. Assuming that the number of people entering the mall and store in the above-mentioned first test result is 180 people, and the number of people passing through the mall and store based on the video data artificially marked is 200 people, then the algorithm accuracy is 90%. In order to reflect the comprehensive performance of the algorithm in different dimensions (i.e., different indicator parameters), the above-mentioned first algorithm accuracy can also be comprehensively determined based on the number of people entering the mall and store, the number of people leaving the mall and store, and the number of people passing through the mall and store. The value obtained by adding up the number of people entering the mall and store (75 people), the number of people leaving the mall and store (55 people), and the number of people passing through the mall and store (180 people) in the first test result is calculated with the value obtained by adding up the number of people entering the mall and store (80 people), the number of people leaving the mall and store (55 people), and the number of people passing through the mall and store (200 people) artificially marked, and the above-mentioned first algorithm accuracy is 93%.

[0098] Similarly, the first hardware platform is tested on the test data through its own head detection algorithm module, calling the head tracking algorithm module of the second hardware platform and its own human key point detection algorithm module, and the number of people entering the mall store, the number of people leaving the mall store and the number of people passing through the mall store (the second test result) after being processed by the above three algorithm modules are also obtained, and compared with the number of people entering the mall store, the number of people leaving the mall store and the number of people passing through the mall store (the benchmark value) manually marked based on the video data, and then the second algorithm accuracy is determined. For example, based on the indicator of the number of people entering the mall store, it is assumed that the number of people entering the mall store in the above second test result is 71 people, and the number of people entering the mall store based on the manual marking of the video data is 80 people, then the algorithm accuracy is 88.75%. Assuming that the number of people leaving the mall store is analyzed, it is assumed that the number of people entering the mall store in the above second test result is 52 people, and the number of people entering the mall store based on the manual marking of the video data is 55, then the algorithm accuracy is 95%. Of course, it is also possible to analyze based on the indicator of the number of people passing through the mall and store. Assuming that the number of people entering the mall and store in the above second test result is 178, and the number of people passing through the mall and store based on the video data manually marked is 200, then the algorithm accuracy is 89%. In order to reflect the comprehensive performance of the algorithm in different dimensions (i.e., different indicator parameters), the above second algorithm accuracy can also be comprehensively determined based on the above number of people entering the mall and store, the number of people leaving the mall and store, and the number of people passing through the mall and store. The value obtained by adding up the number of people entering the mall and store (71 people), the number of people leaving the mall and store (52 people), and the number of people passing through the mall and store (178 people) in the second test result and the value obtained by adding up the number of people entering the mall and store (80 people), the number of people leaving the mall and store (55 people), and the number of people passing through the mall and store (200 people) manually marked is calculated, and the above second algorithm accuracy is 90%.

[0099] The first hardware platform is tested on the test data through its own head detection algorithm module, head tracking algorithm module and calling the human feature point algorithm module of the second platform, and the number of people entering the mall store, the number of people leaving the mall store and the number of people passing through the mall store (the third test result) obtained after the above three algorithm modules are processed are compared with the number of people entering the mall store, the number of people leaving the mall store and the number of people passing through the mall store (the benchmark value) manually marked based on the video data, and then the third algorithm accuracy is determined. For example, based on the indicator of the number of people entering the mall store, it is assumed that the number of people entering the mall store in the above third test result is 78 people, and the number of people entering the mall store based on the manual marking of the video data is 80 people, then the algorithm accuracy is 97.5%. Assuming that the number of people leaving the mall store is analyzed, it is assumed that the number of people entering the mall store in the above third test result is 54 people, and the number of people entering the mall store based on the manual marking of the video data is 55, then the algorithm accuracy is 98.2%. Of course, it is also possible to analyze based on the indicator of the number of people passing through the mall store. Assuming that the number of people entering the mall store in the third test result is 197, and the number of people passing through the mall store based on the video data artificial marking is 200, then the algorithm accuracy is 98.5%. In order to reflect the comprehensive performance of the algorithm in different dimensions (i.e., different indicator parameters), the third algorithm accuracy can also be comprehensively determined based on the number of people entering the mall store, the number of people leaving the mall store, and the number of people passing through the mall store. The value obtained by adding up the number of people entering the mall store (78 people), the number of people leaving the mall store (54 people), and the number of people passing through the mall store (197 people) in the second test result and the value obtained by adding up the number of people entering the mall store (80 people), the number of people leaving the mall store (55 people), and the number of people passing through the mall store (200 people) is calculated, and the third algorithm accuracy can be obtained to be 98.2%.

[0100] The first hardware platform is tested on the test data through its own head detection algorithm module, head tracking algorithm module and human key point detection algorithm module, and the number of people entering the mall, leaving the mall and passing through the mall (full process test results) is compared with the number of people entering the mall, leaving the mall and passing through the mall (benchmark value) based on the manual marking of the video data, and then the fourth algorithm accuracy is determined. For example, based on the indicator of the number of people entering the mall, the analysis is performed. Assuming that the number of people entering the mall in the above-mentioned full process test results is 78 people, and the number of people entering the mall based on the manual marking of the video data is 80 people, then the algorithm accuracy is 97.5%. Assuming that the analysis is performed based on the indicator of the number of people leaving the mall, assuming that the number of people entering the mall in the above-mentioned full process test results is 55 people, and the number of people entering the mall based on the manual marking of the video data is 55, then the algorithm accuracy is 100%. Of course, it is also possible to analyze based on the indicator of the number of people passing through the mall and store. Assuming that the number of people entering the mall and store in the above-mentioned full-process test results is 198, and the number of people passing through the mall and store based on the video data artificially marked is 200, then the algorithm accuracy is 99%. In order to reflect the comprehensive performance of the algorithm in different dimensions (i.e., different indicator parameters), the above-mentioned fourth algorithm accuracy can also be comprehensively determined based on the above-mentioned number of people entering the mall and store, the number of people leaving the mall and store, and the number of people passing through the mall and store. The value obtained by adding up the number of people entering the mall and store (78 people), the number of people leaving the mall and store (55 people), and the number of people passing through the mall and store (198 people) of the full-process test results and the value obtained by adding up the number of people entering the mall and store (80 people), the number of people leaving the mall and store (55 people), and the number of people passing through the mall and store (200 people) is calculated, and the above-mentioned fourth algorithm accuracy is 98.8%.

[0101] In an embodiment of the present application, the determining the target algorithm according to the first algorithm accuracy, the second algorithm accuracy, the third algorithm accuracy, and the fourth algorithm accuracy includes:

[0102] Obtaining a first error degree between the first algorithm accuracy and the fourth algorithm accuracy;

[0103] Obtaining a second error degree between the second algorithm accuracy and the fourth algorithm accuracy;

[0104] Obtaining a third error degree between the third algorithm accuracy and the fourth algorithm accuracy;

[0105] If the first error degree is greater than a preset error threshold, determining the algorithm in the first algorithm module as a target algorithm;

[0106] If the second error degree is greater than a preset error threshold, determining the algorithm in the second algorithm module as a target algorithm;

[0107] If the third error degree is greater than a preset error threshold, the algorithm in the third algorithm module is determined to be the target algorithm.

[0108] In an embodiment of the present application, the first algorithm accuracy is compared with the fourth algorithm accuracy, and the first error degree obtained can represent the error value between the first algorithm module on the first hardware platform and the first algorithm module on the second hardware platform. The second algorithm accuracy is compared with the fourth algorithm accuracy, and the second error degree obtained can represent the error value between the second algorithm module on the first hardware platform and the second algorithm module on the second hardware platform. The third algorithm accuracy is compared with the fourth algorithm accuracy, and the third error degree obtained can represent the error value between the second algorithm module on the first hardware platform and the third algorithm module on the second hardware platform.

[0109] After determining the error degree of different algorithm modules on different platforms, it is possible to determine whether the algorithm in each algorithm module needs to be improved based on the preset error threshold. If improvement is required, the algorithm in the corresponding algorithm module is determined as the target algorithm. It should be noted that as long as the error degree is greater than the preset error threshold, the algorithm in the corresponding algorithm module is determined to need to be improved. That is, when the first error degree (indicated by the error degree between the first algorithm module on the first hardware platform and the first algorithm module on the second hardware platform) is greater than the preset error threshold, it means that the algorithm in the first algorithm module needs to be improved, so the algorithm in the first algorithm module is determined as the target algorithm; when the second error degree (indicated by the error between the second algorithm module on the first hardware platform and the second algorithm module on the second hardware platform) is greater than the preset error threshold, it means that the algorithm in the second algorithm module needs to be improved, so the algorithm in the second algorithm module is determined as the target algorithm; when the third error degree (indicated by the error between the third algorithm module on the first hardware platform and the third algorithm module on the second hardware platform) is greater than the preset threshold, it means that the algorithm in the third algorithm module needs to be improved, so the algorithm in the third algorithm module is determined as the target algorithm. It is understandable that the above-mentioned target algorithm can be one or more.

[0110] It should be noted that the above-mentioned preset error threshold can be set according to the actual scenario, and this application does not impose any restrictions on this.

[0111] In one embodiment of the present application, the algorithm testing method further includes the following steps:

[0112] Get the test data.

[0113] In the embodiment of the present application, the above test data refers to data used to verify the algorithm accuracy of each algorithm module. In specific applications, the test data can be processed in advance so that the format of the processed data (including data format and data size) can meet the requirements of each algorithm module and different hardware platforms.

[0114] Taking the above application scenario of counting the number of people entering and leaving a store in a shopping mall as an example, the test data is the video data captured by a camera installed in the store in the shopping mall. Specifically, the camera can be a 3D camera.

[0115] In a specific application, after the video data is obtained, the video data can be decoded by the video data decoding module, and then each video frame image is saved as a BMP (Bitmap) picture required by different algorithm modules. It should be noted that the BMP picture is only an implementation method of the embodiment of the present application, and other formats can be selected to save the picture as needed, and the present application is not limited to this. It should be noted that the BMP picture refers to a picture saved in a bitmap image format. The BMP picture is a picture in a lossless format, which can ensure that the data of the picture on different hardware platforms can be consistent, avoiding errors in the test results due to inconsistency of the picture data. The BMP picture obtained after decoding is preprocessed so that the size of the BMP picture can meet the input requirements of each algorithm module.

[0116] It should also be noted that the decoding in the embodiment of the present application refers to decoding the JPEG image in the video data into YUV raw data.

[0117] In specific applications, it is also necessary to save the images in the video into the size required by each algorithm module. For example, the size of the image input to the head detection algorithm module is required to be 416 pixels * 416 pixels. The size of the image input to the head tracking algorithm module is required to be 1280 pixels * 720 pixels. The size of the image input to the human key point detection algorithm module is required to be 1920 pixels * 1080 pixels.

[0118] Specifically, the acquired video data can be decoded by a decoding module, and then preprocessed by a video preprocessing module to obtain YUV image data, and then the YUV image data is saved as a 1920 pixel * 1080 pixel (1080P) BMP picture for use by a human key point detection algorithm module; then the 1920 pixel * 1080 pixel (1080P) BMP picture is geometrically scaled to a 1280 pixel * 720 pixel BMP picture for use by a head tracking algorithm module; and then the 1920 pixel * 1080 pixel (1080P) BMP picture is geometrically scaled to a 416 pixel * 234 pixel BMP image, and then the pixel points with RGB values ​​of [128, 128, 128] are filled to obtain a 416 pixel * 416 pixel BMP image for use by a head detection algorithm. It should be noted that the above-mentioned decoding module can be a decoding module of a processing chip in a 3D camera, and the above-mentioned video preprocessing module can also be a video preprocessing module in a 3D camera.

[0119] It should be noted that YUV image data is image data obtained by color encoding image data based on the color encoding format. Among them, "Y" represents brightness (Luminance or Luma), that is, the grayscale value, and "U" and "V" represent chrominance (Chrominance or Chroma), which is used to describe the color and saturation of the image and the color used to specify the pixel.

[0120] For example, Figure 3 As shown, by inputting the acquired test data into the video decoding module for processing, and then inputting the decoded picture into the preprocessing module for processing, three required pictures of different sizes can be obtained, namely, the first test picture (1920 pixels * 1080 pixels), the second test picture (1280 pixels * 720 pixels) and the third test picture (416 pixels * 416 pixels).

[0121] In one embodiment of the present application, the algorithm testing method further includes the following steps:

[0122] Obtain verification test results;

[0123] The accuracy influencing error is determined according to the verification test result, the cross test result, the full-process test result and the benchmark value.

[0124] In an embodiment of the present application, the verification test results include test results obtained by the second hardware platform calling each algorithm module of the first hardware platform for testing respectively and test results obtained by the second hardware platform based on its own algorithm modules.

[0125] That is to say, the second hardware platform can also call various algorithm modules of the first hardware platform for testing. Specifically, the second hardware platform calls various algorithm modules of the first hardware platform for testing respectively, and then the second hardware platform can also perform testing based on its own algorithm modules.

[0126] Specifically, the corresponding accuracy influencing error can be obtained by comprehensively combining the test results obtained by the first hardware platform calling the various algorithm modules of the second hardware platform and testing them separately in combination with its own algorithm modules (i.e., the cross-test results in the embodiments of the present application), the test results obtained by the first hardware platform based on its own algorithm modules for testing (i.e., the full-process test results in the embodiments of the present application), the test results obtained by the second hardware platform calling the various algorithm modules of the first hardware platform for testing separately, and the test results obtained by the second hardware platform based on its own algorithm modules for testing (verification test results).

[0127] In the embodiment of the present application, the above-mentioned accuracy impact error refers to the mutual influence relationship between different algorithm modules on products on different hardware platforms.

[0128] In an embodiment of the present application, the above-mentioned accuracy influencing error can be obtained by comparing the verification error degree obtained by comparing the verification test results with the benchmark value and the error degree obtained by comparing the cross-test results, the full-process test results and the benchmark value (that is, the above-mentioned first error degree, the second error degree and the third error degree).

[0129] Specifically, determining the accuracy impact error according to the verification test result, the cross test result, the full-process test result, and the reference value may include:

[0130] determining a degree of verification error based on the verification test result and the reference value;

[0131] Determine a target error degree according to the cross test result, the full process test result and the benchmark value;

[0132] The accuracy influencing error is determined according to the verification error degree and the target error degree.

[0133] Specifically, the algorithm accuracy obtained by the second hardware platform calling each algorithm module of the first hardware platform and combining it with the remaining algorithm modules for testing can be determined according to the verification test results and the benchmark value. Exemplarily, assuming that the second hardware platform uses its own head detection algorithm module, head tracking algorithm module and human key point detection algorithm module for testing, the algorithm accuracy obtained is 82.80%; the second hardware platform uses the called head detection algorithm module of the first hardware platform, its own head tracking algorithm module and human key point detection algorithm module for testing, and the algorithm accuracy obtained is 88.00%; the second hardware platform uses the called head tracking algorithm module of the first hardware platform, its own head detection algorithm module and human key point detection algorithm module for testing, and the algorithm accuracy obtained is 93.00%; the second hardware platform uses the called human key point detection algorithm module of the first hardware platform, its own head detection algorithm module and head tracking algorithm module for testing, and the algorithm accuracy obtained is 84.00%.

[0134] Then, based on the algorithm accuracy obtained by calling the various algorithm modules of the first hardware platform and combining it with its own remaining algorithm modules for testing, the verification error degrees of testing on the second hardware platform and then replacing the algorithm modules of the first hardware platform are: 5.2% (88.00%-82.80%), 10.20% (93.00%-82.80%) and 1.2% (84.00%-82.80%) respectively.

[0135] Then, the accuracy influencing error can be determined based on the error degree of the algorithm module that is tested on the first hardware platform and then replaces the second hardware platform (ie, the first error degree, the second error degree, and the third error degree in S13).

[0136] Exemplarily, assuming that the first error degree is 5.8%, the second error degree is 8.8%, and the third error degree is 0.6%, then the accuracy impact error between the first hardware platform calling the head detection algorithm module of the second hardware platform and the second hardware platform calling the head detection algorithm module of the first hardware platform is 0.6% (5.8%-5.2%). The accuracy impact error between the first hardware platform calling the head tracking algorithm module of the second hardware platform and the second hardware platform calling the head tracking algorithm module of the first hardware platform is 1.4% (10.2%-8.8%). The accuracy impact error between the first hardware platform calling the human key point detection algorithm module of the second hardware platform and the second hardware platform calling the human key point detection module of the first hardware platform is 0.6% (1.2%-0.6%).

[0137] To further illustrate the embodiment of the present application, the following is an example in which the first hardware platform is a PC and the second hardware platform is a processing chip of a 3D camera:

[0138] First, if Figure 4 As shown, the PC terminal includes a head detection algorithm module, a head tracking algorithm module and a human key point detection algorithm module. Similarly, the processing chip of the 3D camera includes a head detection algorithm module, a head tracking algorithm module and a human key point detection algorithm module.

[0139] The head detection algorithm in the processing chip of the 3D camera is called by the PC, and then each frame of the video data is recognized using the head tracking algorithm module and the human key point detection algorithm module in the PC, and finally the first test result is output, and then the first algorithm accuracy is obtained based on the comparison between the first test result and the reference value. The above first test result, reference value and first algorithm accuracy are shown in Table 1:

[0140] Table 1:

[0141]

[0142] The head tracking algorithm in the processing chip of the 3D camera is called by the PC, and then each frame of the video data is recognized using the head detection algorithm module and the human key point detection algorithm module in the PC, and finally the second test result is output, and then the second algorithm accuracy is obtained based on the comparison between the second test result and the benchmark value. The above second test result, benchmark value and second algorithm accuracy are shown in Table 2:

[0143] Table 2:

[0144]

[0145] The human key point detection algorithm module in the processing chip of the 3D camera is called by the PC, and then the head detection algorithm module and the head tracking algorithm module in the PC are used to identify each frame of the video data, and finally the third test result is output, and then the third algorithm accuracy is obtained based on the comparison between the third test result and the benchmark value. The above third test result, benchmark value and third algorithm accuracy are shown in Table 3:

[0146] Table 3:

[0147]

[0148] Each frame of the video data is identified through the head detection algorithm module, head tracking algorithm module and human key point detection algorithm module in the PC end, and the full process test results are finally output, and then the fourth algorithm accuracy is obtained based on the full process test results and the benchmark value. The above full process test results, benchmark values ​​and fourth algorithm accuracy are shown in Table 4:

[0149] Table 4:

[0150]

[0151] The first error degree, the second error degree and the third error degree may be determined according to the first algorithm accuracy, the second algorithm accuracy, the third algorithm accuracy and the fourth algorithm accuracy.

[0152] Specifically, the first error degree = the fourth algorithm accuracy - the first algorithm accuracy, that is, the first error degree is 5.8%. The second error degree = the fourth algorithm accuracy - the second algorithm accuracy, that is, the second error degree is 8.8%. The third error degree = the fourth algorithm accuracy - the third algorithm accuracy, that is, the third error degree is 0.6%.

[0153] Assuming that the preset error threshold is 1%, it can be determined that the head detection algorithm in the head detection algorithm module and the head tracking algorithm in the head tracking algorithm module are target algorithms, which need to be improved so that the algorithms can have better performance and higher detection performance on the product.

[0154] Additionally, validation test results can be obtained to determine the degree of accuracy impact.

[0155] Specifically, the verification test results obtained by calling the algorithm module on the PC to detect the test data through the 3D camera are shown in Table 5:

[0156] Table 5:

[0157]

[0158] According to the algorithm accuracy in Table 5, the error degrees of testing in the processing chip of the 3D camera and then replacing the algorithm module on the PC side are: 5.2% (88.00%-82.80%), 10.20% (93.00%-82.80%) and 1.2% (84.00%-82.80%).

[0159] From this, we can see that there will be certain errors when calling different algorithm modules for processing on different platforms, that is, different short-hair modules have certain mutual influences on different hardware platforms.

[0160] The accuracy impact error between the head detection algorithm module of the processing chip called by the PC and the head detection algorithm module called by the processing chip on the PC is 0.6% (5.8%-5.2%). The accuracy impact error between the head tracking algorithm module of the processing chip called by the PC and the head tracking algorithm module of the processing chip called by the processing chip on the PC is 1.4% (10.2%-8.8%). The accuracy impact error between the human key point detection algorithm module of the processing chip called by the PC and the human key point detection module of the processing chip called by the processing chip on the PC is 0.6% (1.2%-0.6%).

[0161] From the above, it can be seen that an algorithm testing method provided in this embodiment, by respectively calling different algorithm modules of different hardware platforms for testing, obtains the errors of different algorithm modules on different hardware platforms and the mutual influence of different algorithm modules in the product, and can determine the algorithm that needs to be iterated according to the errors of different algorithm modules on different hardware platforms, so as to quickly iterate the algorithm and reduce the error caused by algorithm transplantation.

[0162] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0163] See also Figure 5 , Figure 5 is a structural block diagram of an algorithm testing device provided in an embodiment of the present application. In this embodiment, the terminal device includes various units for executing Figure 2 For details, please refer to the steps in the corresponding embodiment. Figure 2 as well as Figure 2 For the convenience of explanation, only the parts related to this embodiment are shown. Figure 5 The algorithm testing device 50 includes: a cross-testing unit 51, a self-testing unit 52 and a determining unit 53. Among them:

[0164] The cross-test unit 51 is used to call each algorithm module of the second hardware platform in sequence, and use the called algorithm module and other algorithm modules of the first hardware platform excluding the called algorithm module to test the test data to obtain multiple cross-test results; wherein, the second hardware platform is communicatively connected with the first hardware platform, and the algorithm module included in the second hardware platform corresponds to the algorithm module of the first hardware platform; the second hardware platform includes at least two algorithm modules;

[0165] The self-test unit 52 is used to test the test data using all the algorithm modules of the first hardware platform to obtain the full-process test results;

[0166] The determination unit 53 is used to determine a target algorithm according to the cross-test result, the full-process test result and a benchmark value, wherein the benchmark value is an accurate result determined based on the test data.

[0167] As an embodiment of the present application, the first hardware platform includes a first algorithm module, a second algorithm module and a third algorithm module, the second hardware platform includes a fourth algorithm module corresponding to the first algorithm module, a fifth algorithm module corresponding to the second algorithm module and a sixth algorithm module corresponding to the third algorithm module, and the cross-test unit 51 is specifically used to: call the fourth algorithm module of the second hardware platform, and use the fourth algorithm module, the second algorithm module and the third algorithm module to test the test data to obtain a first test result; call the fifth algorithm module of the second hardware platform, and use the first algorithm module, the fifth algorithm module and the third algorithm module to test the test data to obtain a second test result; call the sixth algorithm module of the second hardware platform, and use the first algorithm module, the second algorithm module and the sixth algorithm module to test the test data to obtain a third test result.

[0168] As an embodiment of the present application, the above-mentioned determination unit 53 is specifically used to determine the target algorithm according to the first test result, the second test result, the third test result, the full-process test result and the benchmark value.

[0169] As an embodiment of the present application, the above-mentioned determination unit 53 is specifically used to: determine the first algorithm accuracy according to the first test result and the benchmark value; determine the second algorithm accuracy according to the second test result and the benchmark value; determine the third algorithm accuracy according to the third test result and the benchmark value; determine the fourth algorithm accuracy according to the full-process test result and the benchmark value; determine the target algorithm according to the first algorithm accuracy, the second algorithm accuracy, the third algorithm accuracy and the fourth algorithm accuracy. Specifically, obtain the first error degree between the first algorithm accuracy and the fourth algorithm accuracy; obtain the second error degree between the second algorithm accuracy and the fourth algorithm accuracy; obtain the third error degree between the third algorithm accuracy and the fourth algorithm accuracy; if the first error degree is greater than the preset error threshold, determine the algorithm in the first algorithm module as the target algorithm; if the second error degree is greater than the preset error threshold, determine the algorithm in the second algorithm module as the target algorithm; if the third error degree is greater than the preset error threshold, determine the algorithm in the third algorithm module as the target algorithm.

[0170] As an embodiment of the present application, the algorithm testing device 50 further includes an acquisition unit and a verification unit.

[0171] The acquisition unit is used to acquire verification test results; wherein, the verification test results include test results obtained by the second hardware platform calling each algorithm module of the first hardware platform for testing respectively and test results obtained by the second hardware platform based on its own algorithm modules.

[0172] The verification unit is used to determine the accuracy influencing error according to the verification test result, the cross test result, the full-process test result and the benchmark value.

[0173] The verification unit is specifically used to determine the degree of verification error according to the verification test result and the reference value;

[0174] The target error degree is determined according to the cross-test result, the full-process test result and the benchmark value; the accuracy influencing error is determined according to the verification error degree and the target error degree.

[0175] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0176] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0177] Figure 6 is a structural block diagram of a terminal device provided by another embodiment of the present application. Figure 6 As shown, the terminal device 6 of this embodiment includes: a processor 60, a memory 61, and a computer program 62 stored in the memory 61 and executable on the processor 60, such as a program of the algorithm testing method. When the processor 60 executes the computer program 62, the steps in each embodiment of the above-mentioned algorithm testing method are implemented, such as Figure 2Alternatively, the processor 60 implements the above when executing the computer program 62 Figure 5 The functions of each unit in the corresponding embodiment, or as Figure 5 The functions of each unit in the corresponding embodiment are, for example, Figure 5 For details on the functions of the units 41 to 43 shown, please refer to Figure 4 The relevant descriptions in the corresponding embodiments are not repeated here.

[0178] Exemplarily, the computer program 62 may be divided into one or more units, which are stored in the memory 61 and executed by the processor 60 to complete the present application. The one or more units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 62 in the terminal device 6.

[0179] The terminal device may include, but is not limited to, a processor 60 and a memory 61. Those skilled in the art will appreciate that Figure 6 It is only an example of the terminal device 6 and does not constitute a limitation on the terminal device 6. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal device may also include input and output devices, network access devices, buses, etc.

[0180] The processor 60 may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0181] The memory 61 may be an internal storage unit of the terminal device 6, such as a hard disk or memory of the terminal device 6. The memory 61 may also be an external storage device of the terminal device 6, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 6. Further, the memory 61 may also include both an internal storage unit and an external storage device of the terminal device 6. The memory 61 is used to store the computer program and other programs and data required by the terminal device. The memory 61 may also be used to temporarily store data that has been output or is to be output.

[0182] An embodiment of the present application provides a computer program product. When the computer program product runs on a mobile terminal, the mobile terminal can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0183] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0184] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0185] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0186] In the embodiments provided in the present application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0187] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0188] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. An algorithm testing method, characterized in that: Applied to a first hardware platform, the first hardware platform includes at least two algorithm modules, and the algorithm testing method includes: Calling each algorithm module of the second hardware platform in sequence, and using the called algorithm module and other algorithm modules of the first hardware platform excluding the called algorithm module to test the test data, to obtain multiple cross-test results; wherein the second hardware platform is communicatively connected with the first hardware platform, and the algorithm module included in the second hardware platform corresponds to the algorithm module of the first hardware platform; the second hardware platform includes at least two algorithm modules; Using all algorithm modules of the first hardware platform to test the test data, and obtaining full-process test results; Determine a target algorithm according to the cross-test result, the full-process test result and a benchmark value, wherein the benchmark value is an accurate result determined based on the test data; Determining a target algorithm according to the cross-test result, the full-process test result and the benchmark value includes: Calculating the algorithm accuracy of the corresponding cross-called algorithm module according to the cross-test result and the benchmark value; Calculating the algorithm accuracy of the algorithm module of the first hardware platform according to the full-process test result and the benchmark value; Calculating an error degree according to the algorithm accuracy of the cross-call algorithm module and the algorithm accuracy of the algorithm module of the first hardware platform, wherein the error degree is used to characterize an error value between the algorithm module of the first hardware platform and the corresponding algorithm module in the second hardware platform; If the error degree is greater than a preset error threshold, the algorithm corresponding to the algorithm module is determined to be the target algorithm.

2. The algorithm testing method according to claim 1, characterized in that: The first hardware platform includes a first algorithm module, a second algorithm module and a third algorithm module, and the second hardware platform includes a fourth algorithm module corresponding to the first algorithm module, a fifth algorithm module corresponding to the second algorithm module and a sixth algorithm module corresponding to the third algorithm module; Accordingly, the algorithm modules of the second hardware platform are called in sequence, and the test data are tested using the called algorithm modules and other algorithm modules of the first hardware platform excluding the called algorithm modules, to obtain multiple cross-test results, including: calling the fourth algorithm module of the second hardware platform, and using the fourth algorithm module, the second algorithm module and the third algorithm module to test the test data to obtain a first test result; calling the fifth algorithm module of the second hardware platform, and using the first algorithm module, the fifth algorithm module and the third algorithm module to test the test data to obtain a second test result; The sixth algorithm module of the second hardware platform is called, and the test data is tested using the first algorithm module, the second algorithm module and the sixth algorithm module to obtain a third test result.

3. The algorithm testing method according to claim 2, characterized in that: Determining a target algorithm according to the cross-test result, the full-process test result and the benchmark value includes: A target algorithm is determined according to the first test result, the second test result, the third test result, the full-process test result and a benchmark value.

4. The algorithm testing method according to claim 3, characterized in that: Determining a target algorithm according to the first test result, the second test result, the second test result, the full-process test result, and a benchmark value includes: Determine a first algorithm accuracy according to the first test result and the reference value; Determine a second algorithm accuracy according to the second test result and the reference value; Determine a third algorithm accuracy according to the third test result and the reference value; Determine the accuracy of the fourth algorithm according to the full-process test result and the reference value; The target algorithm is determined according to the first algorithm accuracy, the second algorithm accuracy, the third algorithm accuracy, and the fourth algorithm accuracy.

5. The algorithm testing method according to claim 4, characterized in that: The determining the target algorithm module according to the first algorithm accuracy, the second algorithm accuracy, the third algorithm accuracy, and the fourth algorithm accuracy includes: Obtaining a first error degree between the first algorithm accuracy and the fourth algorithm accuracy; Obtaining a second error degree between the second algorithm accuracy and the fourth algorithm accuracy; Obtaining a third error degree between the third algorithm accuracy and the fourth algorithm accuracy; If the first error degree is greater than a preset error threshold, determining the algorithm in the first algorithm module as a target algorithm; If the second error degree is greater than a preset error threshold, determining the algorithm in the second algorithm module as a target algorithm; If the third error degree is greater than a preset error threshold, the algorithm in the third algorithm module is determined to be the target algorithm.

6. The algorithm testing method according to any one of claims 1 to 5, characterized in that: Also includes: Obtain verification test results; wherein the verification test results include test results obtained by the second hardware platform calling each algorithm module of the first hardware platform for testing respectively and test results obtained by the second hardware platform based on its own algorithm modules; The accuracy influencing error is determined according to the verification test result, the cross test result, the full-process test result and the benchmark value.

7. The algorithm testing method according to claim 6, characterized in that: The determining the accuracy impact error according to the verification test result, the cross test result, the full process test result and the reference value includes: determining a degree of verification error based on the verification test result and the reference value; Determine a target error degree according to the cross test result, the full process test result and the benchmark value; The accuracy influencing error is determined according to the verification error degree and the target error degree.

8. An algorithm testing device, characterized in that: include: A cross-test unit, used to call each algorithm module of the second hardware platform in sequence, and use the called algorithm modules and other algorithm modules of the first hardware platform excluding the called algorithm modules to test the test data, so as to obtain multiple cross-test results; wherein the second hardware platform is communicatively connected with the first hardware platform, and the algorithm modules included in the second hardware platform correspond to the algorithm modules of the first hardware platform; A self-test unit, used to test the test data using all algorithm modules of the first hardware platform to obtain a full-process test result; A determination unit, configured to determine a target algorithm according to the cross test result, the full-process test result and a benchmark value; The determining unit is specifically used to calculate the algorithm accuracy of the corresponding cross-calling algorithm module according to the cross-test result and the reference value; Calculating the algorithm accuracy of the algorithm module of the first hardware platform according to the full-process test result and the benchmark value; Calculating an error degree according to the algorithm accuracy of the cross-call algorithm module and the algorithm accuracy of the algorithm module of the first hardware platform, wherein the error degree is used to characterize an error value between the algorithm module of the first hardware platform and the corresponding algorithm module in the second hardware platform; If the error degree is greater than a preset error threshold, the algorithm corresponding to the algorithm module is determined to be the target algorithm.

9. A terminal device, characterized in that: The terminal device comprises a memory, a processor, and a computer program stored in the memory and executable on the terminal device, and the processor implements the steps of the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images