Method and system for evaluating image recognition algorithm indexes based on hardware acceleration component

By using the multi-parameter set loop test of hardware acceleration components and automatic export of abnormal materials in image recognition algorithm testing, the problems of low testing efficiency and poor multi-platform adaptability in the existing technology are solved, and efficient and accurate image recognition algorithm performance is achieved Evaluate.

CN119065906APending Publication Date: 2024-12-03GUANGXI PUBLIC INFORMATION IND CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411184654.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-27
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The existing image recognition algorithm testing methods have the problems of single supply methods of testing materials, inability to automatically collect abnormal materials, only single evaluation, and unification of multi-platform testing, resulting in low testing efficiency and inaccurate results.

Method used

Using a method based on hardware acceleration components, the test end and the operation end are coordinated to conduct cyclic testing, and the multi-parameter set evaluation and automatic export of abnormal materials are realized, and unified testing is supported in a multi-platform environment. The hardware acceleration component is used to automatically evaluate the performance indicators of the image recognition algorithm. .

Benefits of technology

It improves testing efficiency and accuracy, realizes unified testing on multiple platforms, and automatically handles abnormal materials, improving the utilization rate of test materials and overall testing ease of use.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119065906A_ABST
    Figure CN119065906A_ABST
Patent Text Reader

Abstract

The invention discloses a method and a system for evaluating an image recognition algorithm index based on a hardware acceleration component. The method is mainly completed by a test end and an operation end. The method includes the following steps that firstly, before testing is started, materials are classified and collected into a test data set at an operation end; 2, configuring an algorithm parameter set sequence at an operation end before the test is started; 3, before the test is started, the hardware acceleration component is confirmed after the test end runs and is started; step 4, the test end and the operation end cooperate to carry out loop testing; and 5, exporting the abnormal test material at the operation end. According to the method and the system, the problems existing in the existing algorithm index test method and system are solved, the performance indexes of the image recognition algorithm under the current hardware condition can be automatically tested and evaluated based on the hardware acceleration component in various cross-platform environments, the test method is unified under different hardware conditions, and the test efficiency is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of IT and software development technology, and specifically relates to a method and system for evaluating image recognition algorithm indicators based on hardware acceleration components. Background Art

[0002] Image recognition refers to the technology of using computers to process, analyze and understand images to identify various targets and objects of different patterns. It is an important field of artificial intelligence. In order to compile computer programs that simulate human image recognition activities, people have proposed different image recognition algorithm models. In order to ensure the performance of the algorithm in practical applications, discover potential problems, optimize the algorithm, and improve the overall performance of the system, the algorithm needs to be tested and evaluated. Test evaluation is a key step to ensure that the image recognition algorithm performs as expected in practical applications. Through testing, the accuracy, robustness and performance of the algorithm in different scenarios can be verified. In addition, testing can also help discover potential problems and optimize the algorithm, thereby improving the overall performance of the system. Therefore, people have conducted relevant research and proposed some solutions for testing the performance indicators of image recognition algorithms.

[0003] For example, Chinese patent application No. 202010014300.0 discloses an algorithm model testing method based on SKU image recognition, which is used to test the performance of the image recognition algorithm model of a smart container, including the following steps: Step S1, collect multiple test cases associated with a specific product SKU through an online sales system and store them in a test database, each test case including a test order and a first test image collected before the smart container opens the door and a second test image collected after the smart container closes the door, and all test product SKUs in the test order are composed of the specific product SKU and the common product SKU; Step S2, collect test cases according to the needs The SKU list template to be tested, the target test cases are screened from the test database to form a test set, and the SKU list template contains all the test product SKUs in the test set; step S3, the first test image and the second test image in each test case in the test set are compared by a pre-trained algorithm model to obtain the order recognition result for each test case, and then the order recognition result is compared with the expected order result of the test order in the corresponding test case to determine whether the order recognition result meets the expected requirements, and a test report is formed and output. In the step S1, according to the movement quantity of the ordinary product SKU in each test case, each test case is classified and stored in the test database by category. In the step S2, according to the number of the test product SKU in the SKU list template to be tested, the test cases whose movement quantity of the ordinary product SKU is not less than the number of the test product SKU in the SKU list template are screened from the test database to form the test set. In step S1, each of the test cases also includes first weight data and second weight data, wherein the first weight data is the weight data of the smart container collected before the smart container opens its door, and the second weight data is the weight data of the smart container collected after the smart container closes its door, and the algorithm model combines the weight difference between the first weight data and the second weight data, performs image recognition comparison on the first test image and the second test image, and obtains the order recognition result for the test case. This patent invention solves the technical problems of low efficiency of existing algorithm model testing and inability to reuse test cases.

[0004] For another example, Chinese patent application No. 202011247047.X discloses a testing method based on a video image recognition algorithm, comprising: receiving a video image to be tested sent by a video image acquisition device; cutting the video image to be tested into video frame images, and eliminating useless video frame images; performing grayscale processing, rotation and scaling processing, and image enhancement processing on the video frame images in sequence; calling a preset recognition algorithm to test the processed video frame images, and generating test result information; displaying the test result information through a display screen; wherein the display screen is a touch-screen display, and different electronic buttons are displayed on the display screen, and the electronic buttons correspond to different pre-recognition algorithms; wherein the preset recognition algorithms include multiple types, and calling the preset recognition algorithm to test the video image data to obtain target data includes: receiving a user through the The number corresponding to the preset recognition algorithm inputted on the display screen; calling the corresponding preset recognition algorithm according to the number; testing the video frame image according to the preset recognition algorithm; wherein, after calling the preset recognition algorithm to test the video frame image, it also includes: calculating the test accuracy of each preset algorithm, and obtaining the corresponding evaluation curve graph according to the test accuracy; wherein, after calculating the test accuracy of each preset algorithm, and obtaining the corresponding evaluation curve graph according to the test accuracy, it also includes: saving the test result information, the test accuracy and the evaluation curve graph; after calculating the test accuracy of each preset algorithm, and obtaining the corresponding evaluation curve graph according to the test accuracy, it also includes: displaying the test result information, the test accuracy and the evaluation curve graph through the display screen. The image enhancement processing of the video frame image includes: denoising the video frame image according to different scenes when the video image to be tested is collected; converting the denoised video frame image from RGB mode to HSV mode; adjusting at least one of the blur, resolution and brightness of the video frame image using python code according to the test requirements. After the video frame image is processed by grayscale, rotation and scaling, and image enhancement, the video frame image is saved. Each of the preset recognition algorithms corresponds to a recognition type, and the recognition type includes at least two of face recognition, area recognition, quantity recognition, and clothing recognition. This technical solution realizes the automation of algorithm testing, saves labor costs, and improves the testing efficiency of the algorithm.

[0005] With the popularization of artificial intelligence, machine learning and the widespread application of edge intelligent terminal devices. The way of performing image recognition based on real-time video frames has changed from connecting multiple video streams to the back-end central recognition server for unified analysis to completing the analysis immediately after each front-end edge intelligent terminal collects real-time video. Due to the difference in computing power between the computing power server and the edge intelligent terminal of the image recognition algorithm model, it is impossible to correctly evaluate the recognition index and recognition efficiency of a certain algorithm model on the edge intelligent terminal. That is, the existing algorithm index testing methods and systems have the following problems: 1. The existing method has a single way of supplying test materials, and there are limitations in the collection of test materials.

[0006] 2. The existing methods cannot automatically collect test materials with abnormal output, and cannot effectively improve the utilization rate of test materials.

[0007] 3. Only a single evaluation test can be performed. Multiple evaluation tests are cumbersome and prone to the risk of human error, which will affect the test results.

[0008] 4. The existing testing methods cannot be tested in a unified manner and can only be fixed on a certain dedicated platform, and cannot make a smooth transition to multiple platforms. Summary of the invention

[0009] The purpose of the present invention is to address the deficiencies of the prior art and propose a method and system for evaluating image recognition algorithm indicators based on hardware acceleration components. The present invention can be used in a variety of cross-platform environments to automatically test and evaluate the performance indicators of image recognition algorithms under current hardware conditions based on hardware acceleration components, so as to unify the test methods under different hardware conditions, improve the test efficiency, and evaluate the algorithm indicators (the algorithm indicators refer to how to maximize the recognition accuracy of the algorithm under the same algorithm model and test materials).

[0010] In order to achieve the above object, the present invention adopts the following technical solutions: A method for evaluating image recognition algorithm indicators based on hardware acceleration components is mainly completed by a test end and an operation end; and includes the following steps: Step 1: Before the test begins, classify the materials into test data sets on the operation end; Step 2: Before the test starts, configure the algorithm parameter set sequence on the operation end; Step 3: Before the test starts, confirm the hardware acceleration component after the test end is started; Step 4: The test end and the operation end cooperate to perform a cyclic test; Step 5: Export the abnormal test materials on the operation side.

[0011] The method of the present invention further illustrates that, in step one, the material includes video test material and picture test material.

[0012] The method of the present invention is further described, wherein the step 1 is specifically: If the material is a video test material, the recognition results that the recognition algorithm should output within a certain video time interval are manually marked in sequence, and then cut into multiple video sequences according to the marked time interval, and classified and stored according to the output recognition results; If the material is a picture test material, mark the recognition result that each picture recognition algorithm should output, and classify and collect and store it according to the output recognition results; The classified test materials are called sub-test data sets, which are used for subsequent tests; After starting the test, the operator scans the storage directory and generates an MD5 digest based on the image name or video sequence name to establish a mapping relationship between the marking result and the image file and video file.

[0013] The method of the present invention further illustrates that in step 2, the algorithm parameter set sequence is composed of multiple parameter sets; the parameter set is composed of multiple key-value pairs. The key-value pair is a storage representation of the current parameter, the key name refers to the name definition of the parameter, and the value corresponding to the key name is the value definition of the parameter. For example: "name": "panminjie", this key-value pair means: the parameter name is "name", and the parameter value is "panminjie".

[0014] The method of the present invention further illustrates that the specific step 2 is: multiple parameter configurations are formed into parameter sets, and then the defined parameters are fine-tuned to form a new parameter set, and then the multiple parameter sets are used to form a test algorithm parameter set sequence for subsequent test configuration. The parameter fine-tuning refers to the size of the changed parameter value, such as changing the "size interval of the recognition threshold", "recognition area range", etc.; that is, adjusting the upper or lower limit of a certain parameter.

[0015] The method of the present invention is further described, and step three is specifically: the test end dynamically loads the hardware acceleration component operating environment according to the type of hardware acceleration component solidified in the test program. If a certain hardware acceleration component is successfully loaded, the use of the hardware acceleration component is confirmed, otherwise the CPU is used by default to complete the decoding operation of the test data in the subsequent test process.

[0016] The hardware acceleration component operating environment, that is, the hardware acceleration component and operating environment that supports detection include: Windows desktop operating system supporting D3D11 or CUDA, Linux desktop operating system supporting VAAPI or CUDA, edge smart devices supporting CUDA or Soc chip built-in GPU, and IPC smart cameras with Soc chip built-in GPU.

[0017] The method of the present invention is further described, wherein the step 4 comprises: The operation end traverses the test materials of all sub-test data sets from the test data set to generate a test data list. If the test data is a video sequence, a unique resource request index is generated according to the video sequence; if it is image data, a resource download link is generated according to the image name; The operator extracts the parameter set required for this test from the algorithm parameter set sequence, and sends the test data list and parameter set to the test end via the HTTP protocol to complete the test configuration; The test end completes the algorithm model parameter configuration according to the received parameter set; determines the content of the received test data list; if the test item is a video sequence, uses the MD5 digest of the resource to request the resource from the operation end through the HTTP protocol; the operation end generates the streaming media protocol address supported by the test end on demand according to the test configuration; the test end uses the streaming media protocol to pull the stream according to the streaming media protocol address responded by the operation end; if the test item is image data, the test data is pulled by downloading the image stream; After collecting the test data, the test end uses the confirmed hardware acceleration component to complete the decoding operation of the test data, converts the decoded YUV data into grayscale, and sends the grayscale data to the algorithm model to complete this data processing operation; The algorithm model performs algorithm analysis based on the configured algorithm model parameters and outputs the recognition results; After the test end completes the identification of all data in the test data list, it notifies the operation end to perform the next round of testing. The operation end extracts the next set of parameter sets and test data list, and sends them to the test end to complete a new round of test configuration until all parameter sets have completed the configuration test.

[0018] The parameter configuration on the operation side is to complete the construction of the overall test environment. Each test will extract a different "parameter set" from the "algorithm parameter set sequence" and provide it to the test side, rather than the entire "algorithm parameter set sequence"; and the parameter configuration on the test side is to configure the algorithm model according to the different "parameter sets" each time.

[0019] The resource mentioned in this article refers to the material (picture or video) provided to the test end for testing. Here (step 4), if the resource (test material) is a video file, the operation end generates the MD5 digest of the resource and uses the digest to bind it to the resource (test material) for access by the test end; if the resource (test material) is a picture, the operation end generates a download link for the picture for access by the test end.

[0020] The method described in the present invention is further described, and the specific step five is: the operating end determines whether the recognition result is abnormal based on the output result of the test end, and if the result is abnormal, exports the test material to the abnormal data directory; wherein, the output result of the test end includes: algorithm recognition output result, algorithm model configuration parameters, algorithm recognition time, test resource name and other attributes, and uses the attributes of the test resource name in the output result to find the corresponding test material by traversing and comparing in the mapping table established when classifying and grouping the test data set, and completes the export of the corresponding material.

[0021] The present invention also provides a system for evaluating image recognition algorithm indicators based on hardware acceleration components, which is used to implement the above method, including: The test data set collection module is used to classify and collect materials on the operation side; The test data set storage module is used to store the classified test data sets on the operation side; The algorithm parameter set sequence configuration module is used to configure multiple parameters into a parameter set at the operation end, and then fine-tune the defined parameters to form a new parameter set, and then form a test algorithm parameter set sequence from multiple parameter sets; The hardware acceleration component confirmation module is used to confirm the hardware acceleration component after the test end is started; A cyclic test module, including a data extraction unit, a test configuration unit, a data decoding and conversion unit, and an algorithm analysis unit; The data extraction unit is used to implement the operation end to traverse the test materials of all sub-test data sets from the test data set and generate a test data list and extract the parameter set required to be configured for this test, and send the test data list and the parameter set to the test end through the HTTP protocol; The test configuration unit is used to complete the algorithm model parameter configuration at the test end according to the received parameter set; determine the content of the received test data list, and if the test item is a video sequence, use the MD5 digest of the resource to request the resource from the operation end through the HTTP protocol, and the operation end generates the streaming media protocol address supported by the test end on demand according to the test configuration, and the test end uses the streaming media protocol to pull the stream according to the streaming media protocol address responded by the operation end; if the test item is picture data, the test data is pulled in the form of picture stream download; The data decoding and conversion unit is used to realize that after the test end collects the test data, it uses the confirmed hardware acceleration component to complete the decoding operation of the test data, converts the decoded YUV data into grayscale, and sends the grayscale data to the algorithm model; The algorithm analysis unit is used to implement the algorithm model according to the configured algorithm model parameters, and complete the recognition result output after performing algorithm analysis; The abnormal test material export module is used to realize the export of abnormal test materials on the operation side.

[0022] In order to better understand the technical content of the present invention, some technical terms are explained below: HTTP protocol: HTTP protocol is the abbreviation of Hyper Text Transfer Protocol, which is a transmission protocol used to transfer hypertext from the World Wide Web (WWW: World Wide Web) server to the local browser. HTTP is a TCP / IP-based communication protocol to transfer data (HTML files, image files, query results, etc.).

[0023] JSON format: JSON (JavaScript Object Notation) is a lightweight data exchange format. It is based on a subset of ECMAScript (the js specification developed by the European Computer Manufacturers Association) and uses a text format that is completely independent of the programming language to store and represent data. The concise and clear hierarchical structure makes JSON an ideal data exchange language. It is easy for people to read and write, and it is also easy for machines to parse and generate, and it effectively improves network transmission efficiency.

[0024] YUV data: YUV is a type of image format used in applications such as video, pictures, and cameras. It is actually the name of the color space common to all "YUV" pixel formats. Unlike the RGB format (red-green-blue), YUV is represented by a "brightness" component called Y (equivalent to grayscale) and two "chrominance" components, called U (blue projection) and V (red projection), hence the name.

[0025] GPU: Codec acceleration component.

[0026] MD5: Message Digest Algorithm, a widely used cryptographic hash function that can generate a 128-bit hash value to ensure the integrity and consistency of information transmission.

[0027] Compared with the existing testing methods, the present invention has the following advantages: 1. In terms of technical application: multiple parameter sets are cycled tested to facilitate testing and evaluation of optimal parameters.

[0028] 2. In terms of scenario application: unified testing methods to facilitate transition testing from local to target hardware.

[0029] 3. In terms of convenience: it supports automatic export and collection of test materials that output abnormal results, thereby improving material utilization.

[0030] 4. In terms of practicality: a variety of test material supply methods to improve the ease of use of the overall test. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 Schematic diagram of the overall process of the method described in one embodiment of the present invention.

[0032] Figure 2 Schematic diagram of a sequence flow of the method described in one embodiment of the present invention.

[0033] Figure 3 The figure is a schematic diagram of the material classification and collection process of the method described in one embodiment of the present invention.

[0034] Figure 4 The figure is a schematic diagram of the hardware acceleration component confirmation process of the method described in one embodiment of the present invention.

[0035] Figure 5 The figure is a schematic diagram of a cyclic test process of the method described in one embodiment of the present invention.

[0036] Figure 6 The figure is a schematic diagram of the test material data processing flow of the method described in one embodiment of the present invention. DETAILED DESCRIPTION

[0037] The present invention is further described below in conjunction with the accompanying drawings. Example

[0038] A method for evaluating image recognition algorithm indicators based on hardware acceleration components, mainly completed by the test end and the operation end; Figure 1 and Figure 2 As shown, the following steps are included: Step 1: Before the test begins, classify the materials into test data sets on the operation end (such as Figure 3 shown); The materials include video test materials and picture test materials; If the material is a video test material, the recognition results that the recognition algorithm should output within a certain video time interval are manually marked in sequence, and then cut into multiple video sequences according to the marked time interval, and classified and stored according to the output recognition results; If the material is a picture test material, mark the recognition result that each picture recognition algorithm should output, and classify and collect and store it according to the output recognition results; The classified test materials are called sub-test data sets, which are used for subsequent tests; After starting the test, the operator scans the storage directory and generates an MD5 digest based on the image name or video sequence name to establish a mapping relationship between the marking result and the image file and video file.

[0039] Step 2: Before the test starts, configure the algorithm parameter set sequence on the operation end; The algorithm parameter set sequence is composed of multiple parameter sets; the parameter set is composed of multiple key-value pairs. Depending on the algorithm model, there are many different key-value configurations: configure the recognition threshold parameter, and when the credibility of the recognition result is higher than this threshold, this result is output; configure the ROI parameter, focus on analyzing the ROI area, and ignore the influence of image data in other areas. Here, multiple parameter configurations are combined into a parameter set, and then the defined parameters are fine-tuned to form a new parameter set, and then the algorithm parameter set sequence composed of multiple parameter sets is used for subsequent test configuration.

[0040] Step 3: Before the test starts, confirm the hardware acceleration components after the test end is started; Figure 4 As shown, specifically: The test end dynamically loads the hardware acceleration component operating environment according to the type of hardware acceleration component solidified in the test program. If a certain hardware acceleration component is successfully loaded, it is confirmed to be used. Otherwise, the CPU is used by default to complete the decoding operation of the test data in the subsequent test process.

[0041] The hardware acceleration component operating environment, that is, the hardware acceleration component and operating environment that supports detection include: Windows desktop operating system supporting D3D11 or CUDA; Linux desktop operating system supporting VAAPI or CUDA; edge smart devices supporting CUDA or Soc chip built-in GPU; IPC smart camera with Soc chip built-in GPU.

[0042] Step 4: The test end and the operation end cooperate to perform a cyclic test; Figure 5 and Figure 6 As shown, including: The operation end traverses the test materials of all sub-test data sets from the test data set to generate a test data list. If the test data is a video sequence, a unique resource request index is generated according to the video sequence; if it is image data, a resource download link is generated according to the image name; The operator extracts the parameter set required for this test from the algorithm parameter set sequence, and sends the test data list and parameter set to the test end via the HTTP protocol to complete the test configuration; The test end completes the algorithm model parameter configuration according to the received parameter set; determines the content of the received test data list; if the test item is a video sequence, uses the MD5 digest of the resource to request the resource from the operation end through the HTTP protocol; the operation end generates the streaming media protocol address supported by the test end on demand according to the test configuration; the test end uses the streaming media protocol to pull the stream according to the streaming media protocol address responded by the operation end; if the test item is image data, the test data is pulled by downloading the image stream; After collecting the test data, the test end uses the confirmed hardware acceleration component to complete the decoding operation of the test data, converts the decoded YUV data into grayscale, and sends the grayscale data to the algorithm model to complete this data processing operation; The algorithm model performs algorithm analysis based on the configured algorithm model parameters and outputs the recognition results; The operation end determines whether the output result is correct according to the output result of the test end, and calculates the test time cycle. If the recognition result is abnormal, the abnormal result is imported into the material export queue. After the subsequent test is completed, the operation end exports the material according to the resource file marked in the result; After the test end completes the identification of all data in the test data list, it notifies the operation end to perform the next round of testing. The operation end extracts the next set of parameter sets and test data list, and sends them to the test end to complete a new round of test configuration until all parameter sets have completed the configuration test.

[0043] Step 5: Export abnormal test materials at the operation end; specifically: the operation end determines whether the recognition result is abnormal based on the output result of the test end. If the result is abnormal, the test material is exported to the abnormal data directory; the output result of the test end includes: algorithm recognition output result, algorithm model configuration parameters, algorithm recognition time, test resource name and other attributes. Use the attributes of the test resource name in the output result to find the corresponding test material by traversing and comparing in the mapping table established when classifying and grouping the test data set, and complete the export of the corresponding material. Specific examples include others: algorithm model output, test data grouping mapping, etc.

[0044] This embodiment also provides a system for evaluating image recognition algorithm indicators based on hardware acceleration components, which is used to implement the above method, including: The test data set collection module is used to classify and collect materials on the operation side; The test data set storage module is used to store the classified test data sets on the operation side; The algorithm parameter set sequence configuration module is used to configure multiple parameters into a parameter set at the operation end, and then fine-tune the defined parameters to form a new parameter set, and then form a test algorithm parameter set sequence from multiple parameter sets; The hardware acceleration component confirmation module is used to confirm the hardware acceleration component after the test end is started; A cyclic test module, including a data extraction unit, a test configuration unit, a data decoding and conversion unit, and an algorithm analysis unit; The data extraction unit is used to implement the operation end to traverse the test materials of all sub-test data sets from the test data set and generate a test data list and extract the parameter set required to be configured for this test, and send the test data list and the parameter set to the test end through the HTTP protocol; The test configuration unit is used to complete the algorithm model parameter configuration at the test end according to the received parameter set; determine the content of the received test data list, and if the test item is a video sequence, use the MD5 digest of the resource to request the resource from the operation end through the HTTP protocol, and the operation end generates the streaming media protocol address supported by the test end on demand according to the test configuration, and the test end uses the streaming media protocol to pull the stream according to the streaming media protocol address responded by the operation end; if the test item is picture data, the test data is pulled in the form of picture stream download; The data decoding and conversion unit is used to realize that after the test end collects the test data, it uses the confirmed hardware acceleration component to complete the decoding operation of the test data, converts the decoded YUV data into grayscale, and sends the grayscale data to the algorithm model; The algorithm analysis unit is used to implement the algorithm model according to the configured algorithm model parameters, and complete the recognition result output after performing algorithm analysis; The abnormal test material export module is used to realize the export of abnormal test materials on the operation side.

[0045] Obviously, the above embodiments are only examples for clearly explaining the present invention, and are not intended to limit the implementation of the present invention. For ordinary technicians in the relevant technical field, other different forms of changes or modifications can be made based on the above description; it is not necessary and impossible to list all the implementation methods here; and the obvious changes or modifications derived from them are still within the protection scope of the present invention.

Claims

1. A method for evaluating image recognition algorithm indicators based on hardware acceleration components, mainly completed by a test end and an operation end; characterized in that The following steps are involved: Step 1: Before the test begins, classify the materials into test data sets on the operation end; Step 2: Before the test starts, configure the algorithm parameter set sequence on the operation end; Step 3: Before the test starts, confirm the hardware acceleration component after the test end is started; Step 4: The test end and the operation end cooperate to perform a cyclic test; Step 5: Export the abnormal test materials on the operation side.

2. The method for evaluating image recognition algorithm indicators based on hardware acceleration components according to claim 1, characterized in that: In step one, the material includes video test material and picture test material.

3. The method for evaluating image recognition algorithm indicators based on hardware acceleration components according to claim 2, characterized in that: The step 1 is specifically as follows: If the material is a video test material, the recognition results that the recognition algorithm should output within a certain video time interval are manually marked in sequence, and then cut into multiple video sequences according to the marked time interval, and classified and stored according to the output recognition results; If the material is a picture test material, mark the recognition result that each picture recognition algorithm should output, and classify and collect and store it according to the output recognition results; The classified test materials are called sub-test data sets, which are used for subsequent tests; After starting the test, the operator scans the storage directory and generates an MD5 digest based on the image name or video sequence name to establish a mapping relationship between the marking result and the image file and video file.

4. The method for evaluating image recognition algorithm indicators based on hardware acceleration components according to claim 1, characterized in that: In step 2, the algorithm parameter set sequence consists of multiple parameter sets; and the parameter set consists of multiple key-value pairs.

5. The method for evaluating image recognition algorithm indicators based on hardware acceleration components according to claim 1, characterized in that: The step 2 is specifically as follows: multiple parameter configurations are formed into a parameter set, and then the defined parameters are fine-tuned to form a new parameter set, and then the multiple parameter sets are used to form a test algorithm parameter set sequence for subsequent test configuration.

6. The method for evaluating image recognition algorithm indicators based on hardware acceleration components according to claim 1, characterized in that: The step three is specifically as follows: the test end dynamically loads the hardware acceleration component operating environment according to the type of hardware acceleration component solidified in the test program. If a certain hardware acceleration component is successfully loaded, the use of the hardware acceleration component is confirmed; otherwise, the CPU is used by default to complete the decoding operation of the test data in the subsequent test process.

7. The method for evaluating image recognition algorithm indicators based on hardware acceleration components according to claim 6, characterized in that: The hardware acceleration component operating environment, that is, the hardware acceleration component and operating environment that supports detection include: Windows desktop operating system supporting D3D11 or CUDA, Linux desktop operating system supporting VAAPI or CUDA, edge smart devices supporting CUDA or Soc chip built-in GPU, and IPC smart cameras with Soc chip built-in GPU.

8. The method for evaluating image recognition algorithm indicators based on hardware acceleration components according to claim 1, characterized in that: The fourth step comprises: The operation end traverses the test materials of all sub-test data sets from the test data set to generate a test data list. If the test data is a video sequence, a unique resource request index is generated according to the video sequence; if it is image data, a resource download link is generated according to the image name; The operator extracts the parameter set required for this test from the algorithm parameter set sequence, and sends the test data list and parameter set to the test end via the HTTP protocol to complete the test configuration; The test end completes the algorithm model parameter configuration according to the received parameter set; determines the content of the received test data list; if the test item is a video sequence, uses the MD5 digest of the resource to request the resource from the operation end through the HTTP protocol; the operation end generates the streaming media protocol address supported by the test end on demand according to the test configuration; the test end uses the streaming media protocol to pull the stream according to the streaming media protocol address responded by the operation end; if the test item is image data, the test data is pulled by downloading the image stream; After collecting the test data, the test end uses the confirmed hardware acceleration component to complete the decoding operation of the test data, converts the decoded YUV data into grayscale, and sends the grayscale data to the algorithm model to complete this data processing operation; The algorithm model performs algorithm analysis based on the configured algorithm model parameters and outputs the recognition results; After the test end completes the identification of all data in the test data list, it notifies the operation end to perform the next round of testing. The operation end extracts the next set of parameter sets and test data list, and sends them to the test end to complete a new round of test configuration until all parameter sets have completed the configuration test.

9. The method for evaluating image recognition algorithm indicators based on hardware acceleration components according to claim 1, characterized in that: The specific steps of step five are as follows: the operation end determines whether the recognition result is abnormal based on the output result of the test end. If the result is abnormal, the test material is exported to the abnormal data directory; wherein the output result of the test end includes: the algorithm recognition output result, the algorithm model configuration parameters, the algorithm recognition time, and the attributes of the test resource name; the operation end uses the attributes of the test resource name in the output result, traverses and compares in the mapping table established when classifying and grouping the test data set, finds the corresponding test material, and completes the export of the corresponding material.

10. A system for evaluating image recognition algorithm indicators based on hardware acceleration components, used to implement the method as claimed in claim 1, characterized in that include: The test data set collection module is used to classify and collect materials on the operation side; The test data set storage module is used to store the classified test data sets on the operation side; The algorithm parameter set sequence configuration module is used to configure multiple parameters into a parameter set at the operation end, and then fine-tune the defined parameters to form a new parameter set, and then form a test algorithm parameter set sequence from multiple parameter sets; The hardware acceleration component confirmation module is used to confirm the hardware acceleration component after the test end is started; A cyclic test module, including a data extraction unit, a test configuration unit, a data decoding and conversion unit, and an algorithm analysis unit; The data extraction unit is used to implement the operation end to traverse the test materials of all sub-test data sets from the test data set and generate a test data list and extract the parameter set required to be configured for this test, and send the test data list and the parameter set to the test end through the HTTP protocol; The test configuration unit is used to complete the algorithm model parameter configuration at the test end according to the received parameter set; determine the content of the received test data list, and if the test item is a video sequence, use the MD5 digest of the resource to request the resource from the operation end through the HTTP protocol, and the operation end generates the streaming media protocol address supported by the test end on demand according to the test configuration, and the test end uses the streaming media protocol to pull the stream according to the streaming media protocol address responded by the operation end; if the test item is picture data, the test data is pulled in the form of picture stream download; The data decoding and conversion unit is used to realize that after the test end collects the test data, it uses the confirmed hardware acceleration component to complete the decoding operation of the test data, converts the decoded YUV data into grayscale, and sends the grayscale data to the algorithm model; The algorithm analysis unit is used to implement the algorithm model according to the configured algorithm model parameters, and complete the recognition result output after performing algorithm analysis; The abnormal test material export module is used to realize the export of abnormal test materials on the operation side.

Citation Information

Patent Citations

  • Algorithm Model Testing Method and System Based on SKU Image Recognition

    CN111258883B

  • Test method and device based on video image recognition algorithm and test terminal

    CN112364761A