An image target recognition performance evaluation device and evaluation method

Abstract

The application discloses an image target recognition performance evaluation device and an evaluation method, the device is based on a full programmable array on-chip digital processing chip as a framework, comprising a gigabit network data transmission module, a video interface conversion module, a VDMA module, an AXI4-stream to Video out module, a communication module, an OSD module, a video stream timing generation module, a dynamic clock configuration module, a full programmable array on-chip digital processing module and a performance evaluation module; the evaluation device and the evaluation method can effectively reduce the field test guarantee requirement, can comprehensively and objectively evaluate the adaptability to the scene according to different scene requirements, can fully utilize different data set resources or increase interference measures, and are suitable for the needs of evaluating the target recognition performance of various embedded image processors with different video interfaces.

Description

TECHNICAL FIELD

[0001] The present application relates to the target recognition technology in the engineering application field, and in particular to an image target recognition performance evaluation device and an evaluation method. BACKGROUND

[0002] Deep learning has been widely applied in the target recognition field. In order to evaluate the advantages and disadvantages between different models and methods, some classic data sets are generated. The average accuracy and processing speed are obtained by training and testing using these data sets to evaluate the indicators, and with the average accuracy and other indicators constantly refreshing records, the model is also accompanied by the complication, the network is constantly deepened, and the hardware requirement is improved. However, in engineering applications, those top-ranking algorithms have not been widely popularized, but those simple and efficient methods have been widely applied, which also reflects that the current target recognition algorithm evaluation method is not completely suitable for engineering applications.

[0003] The classic data sets generated provide a source of test and evaluation data for target recognition algorithms, but it is difficult to directly evaluate the performance of target recognition algorithms on embedded platforms through these data sets. Often, it is necessary to rely on tests in different environments and increase interference in scenes, which is time-consuming, has high security requirements, and has poor repeatability in different scenes. Therefore, it is necessary to use a device and method to repeatedly inject image data in different environments and increase interference in scenes, as well as classic data sets, into different embedded image processors to evaluate the target recognition performance. SUMMARY

[0004] In view of the above problems, the present application provides an image target recognition performance evaluation device and an evaluation method in engineering applications, which provides a reference for evaluating the target recognition performance of different embedded image processors.

[0005] The technical scheme adopted by the present application is: an image target recognition performance evaluation device, the device is based on a fully programmable array on-chip digital processing chip as a framework, including a gigabit network data transmission module, a video interface conversion module, a VDMA module, an AXI4-stream to Video out module, a communication module, an OSD (on-screen display) module, a video stream timing generation module, a dynamic clock configuration module, a fully programmable array on-chip digital processing module and a performance evaluation module.

[0006] The performance evaluation module transmits the prepared scene video data to the full programmable array on-chip digital processing module through the gigabit network data transmission module. The VDMA module reads the prepared image and tells the embedded image processor to start recognition through the communication module. At the same time, the VDMA module injects the video stream data in the memory into the embedded image processor through the video interface conversion module under the timing generated by the video stream timing generation module according to the requirements of the embedded image processor input video. The embedded image processor starts target recognition after receiving the injected image data. The recognition result is transmitted to the full programmable array on-chip digital processing module through the communication module. The full programmable array on-chip digital processing module transmits the recognition result data to the performance evaluation module through the gigabit network data transmission module. The performance evaluation module performs performance evaluation in different dimensions according to the issued and received result data.

[0007] Further, the gigabit network data transmission module enables the performance evaluation module to repeatedly inject scene image data in different environments and with increased interference and classic data sets into different embedded image processors, and has a communication function.

[0008] Further, the VDMA module has a high bandwidth to realize the reading and storage functions of the video stream in the memory.

[0009] Further, the video stream timing generation module generates timing that matches the video input interface timing of different embedded image processors.

[0010] Further, the video interface conversion module realizes video injection of different embedded image processors under the cooperation of the video stream timing generation module.

[0011] Further, the full programmable array on-chip digital processing module is the core data processing module of the entire device, which is the center of data stream control, workflow control and data interaction.

[0012] Further, the performance evaluation module collects embedded image processor working process data after injecting scene image data in different environments and with increased interference and classic data sets, and evaluates the target recognition performance of the embedded image processor in different dimensions to form comprehensive performance chart data.

[0013] A target recognition evaluation method based on the above image target recognition performance evaluation device, the specific implementation steps are as follows: step one, the performance evaluation module transmits the prepared scene video data to the full programmable array on-chip digital processing module through the gigabit network data transmission module. The full programmable array on-chip digital processing module receives video frames F1, F2, F3…F nThe video data after the preparation of the image F1 is generated, and the time reference t0 is established by using the full programmable array on-chip digital processing module, the frame count 1 is generated, and the OSD module is superimposed on the F1 image frame. At the same time, the VDMA module reads the image data required by the embedded image processor to generate the embedded image processor under the multi-channel buffer mechanism of the VDMA module. Switch to the V1 channel to store the image.

[0014] Step two, the VDMA module reads the prepared image to generate an image display interrupt, and tells the embedded image processor that it can start identification through the communication module. The time is recorded as t1, and the identification result of the embedded image processor is received through the communication module. The time is recorded as t 11 At the same time, the V1 channel image in step one is switched to read through the VDMA multi-channel buffer mechanism. The target real position data of the target recognition received is and and is transmitted to the performance evaluation module for record keeping through the gigabit network data transmission module; repeat step one, which can realize the processing of F2 frame image, and so on, that is, the processing of the prepared scene video F n .

[0015] Step three, the performance evaluation module processes the prepared scene video data F1, F2, F3…F n , the real position data of the target in the scene is C 11 C 12 C 13 …C 1i , C 21 C 22 C 23 …C 2j , and C n1 C n2 C n3 …C nk , the total number of targets R total is The time of receiving the identification result of the embedded image processor is t 11 t 21 t 31 …t n1 , and the video data F1', F2', F3'…F n ' processed by the embedded image processor is received.

[0016] At the same time, multiple scene data are prepared, and steps one, two and three are repeated to obtain different application scene analysis data.

[0017] F1, F2, F3…F n , the processing time is T1T2T3…T n , and the processing time of F1 frame is T1=t11 -t1, F n Frame processing time is T n = t n1 -t n ;

[0018] In the identification result, if the Euclidean distance between the position coordinates received by the nth frame and the real position coordinates is less than a certain value th (such as 3 pixels), it can be considered correct, and is recorded as R n , otherwise it is considered to be incorrect;

[0019]

[0020] The number of correct identifications is:

[0021]

[0022] The recognition accuracy rec is the most important indicator, defined as the ratio of the number of correct samples to the total number of samples, recorded as:

[0023]

[0024] The recognition speed V rec is the speed of realizing the target recognition function, and is recorded as:

[0025]

[0026] The generalization ability f is the ability to correctly identify the target when inputting non-training sample data, recorded as:

[0027]

[0028] In formula (5), R1' is the number of correctly identified non-training sample targets in the first frame, R1'' is the total number of non-training sample targets in the first frame, m is the number of identification frames, and jj m is the total number of real targets of the mth frame of non-training samples.

[0029] Step four, according to the above indicators, the comprehensive evaluation target recognition performance comprehensive index is obtained by weighting each indicator, recorded as:

[0030] E = w1·rec + w2·f - w3·V vec (6)

[0031] In formula (6), w1, w2 and w3 are the weight values of each indicator, which can be allocated according to the actual engineering application scene.

[0032] Compared with the prior art, the advantages of the present application are:

[0033] 1) It can effectively reduce the requirements for field testing and can comprehensively and objectively evaluate the adaptability of a scenario based on different scenario needs, and can make full use of different dataset resources or add interference measures. It has the characteristics of high repeatability, strong scalability, high comprehensiveness and low assurance requirements, and is suitable for evaluating the target recognition performance of embedded image processors with various video interfaces.

[0034] 2) The proposed target recognition evaluation method can solve the performance evaluation problem of target recognition algorithms in engineering applications when there is a lack of public datasets, and provide a reference for the selection of the best target recognition algorithm;

[0035] 3) The proposed target recognition evaluation method is based on the principles of comprehensiveness, quantifiability, intuitive results, and public availability. It combines the weighted ranking method and the comprehensive evaluation method, and uses three dimensions—recognition accuracy, generalization ability, and recognition speed—to evaluate the target recognition performance. Attached Figure Description

[0036] Figure 1 This is a schematic diagram of the evaluation device of the present invention;

[0037] Figure 2 This is a flowchart illustrating the operational information flow of the evaluation device of the present invention. Detailed Implementation

[0038] The technical solutions of the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0039] Example 1:

[0040] like Figure 1 As shown, an image target recognition performance evaluation device for engineering applications is based on a fully programmable array on-chip digital processing chip and includes a gigabit network data transmission module, a video interface conversion module, a VDMA module, an AXI4-stream to Video out module, a communication module, an OSD (on-screen display) module, a video stream timing generation module, a dynamic clock configuration module, a fully programmable array on-chip digital processing module, and a performance evaluation module.

[0041] The gigabit network data transmission module realizes that the performance evaluation module repeatedly injects the scene image data and the classic data set with different environments and increased interference into different embedded image processors, and has a communication function. The VDMA module realizes the reading and storage functions of the video stream in the memory. The video stream timing generation module generates a timing sequence that is adapted to the video input interface timing sequence of the different embedded image processors. The video interface conversion module realizes the video injection of different embedded image processors, including PAL, CML, cameralink, and LVDS, under the cooperation of the video stream timing generation module. The all-programmable array on-chip digital processing module is the core data processing module of the entire device, and is the center of data flow control, workflow control, and data interaction. The performance evaluation module collects the working process data of the embedded image processor after injecting the scene image data and the classic data set with different environments and increased interference, and evaluates the target recognition performance of the embedded image processor in different dimensions to form a comprehensive performance chart data. The AXI4-stream to Videoout module is a stream video conversion to line field video format module. The OSD (on-screen display) module is a character superposition module. The dynamic clock configuration module is a clock configuration module for configuring the required video output format.

[0042] The performance evaluation module transmits the prepared scene video data to the all-programmable array on-chip digital processing module through the gigabit network data transmission module. After the transmission of one frame is completed, an image interrupt is generated. The VDMA module reads the prepared image and informs the embedded image processor to start recognition through the communication module. At the same time, the VDMA module injects the video stream data in the memory into the embedded image processor through the video interface conversion module under the timing sequence generated by the video stream timing generation module according to the requirements of the embedded image processor input video. The embedded image processor starts target recognition after receiving the injected image data, and transmits the recognition result to the all-programmable array on-chip digital processing module through the communication module. The all-programmable array on-chip digital processing module transmits the recognition result data to the performance evaluation module through the gigabit network data transmission module. The performance evaluation module performs performance evaluation in different dimensions according to the transmitted and received result data.

[0043] Embodiment two

[0044] A target recognition evaluation method based on an image target recognition performance evaluation device, the working information flow of which is shown in Figure 2 The specific implementation steps are as follows:

[0045] Step one, the performance evaluation module transmits the prepared scene video data through the gigabit network data transmission module. The all-programmable array on-chip digital processing module receives the video data. The video frame count is F1, F2, F3…F nUpon receiving the F1 image preparation interrupt, the on-chip digital processing module of the fully programmable array establishes a time base t0, generates a frame count of 1, and overlays it onto the F1 image frame through the OSD module. At the same time, the prepared image of the V0 channel is read through the VDMA module to generate the image data required by the timing of the embedded image processor, and the image is stored in the V1 channel under the multi-channel caching mechanism of the VMDA module.

[0046] Step 2: After the VDMA module finishes reading the prepared image, it generates an image display interrupt and notifies the embedded image processor via the RS422 communication module that recognition can begin. This time is recorded as t1. The time when the recognition result is received from the embedded image processor via the communication module is recorded as t. 11 Simultaneously, through the VDMA module's multi-channel caching mechanism, the image from channel V1 in step one is switched for reading, and the received target identification data shows the true location of the target. and The data is then transmitted to the performance evaluation module via the gigabit network data transmission module for storage and recording. Repeating step one allows for the processing of F2 frame images, and so on, to process the prepared scene video. n The processing.

[0047] Step 3: Performance Evaluation Module Data Processing - Prepared scene video data F1, F2, F3…F n The actual location data of the target in the scene is C. 11 C 12 C 13 …C 1i C 21 C 22 C 23 …C 2j and C n1 C n2 C n3 …C nk The total number of targets R total for The time to receive the recognition result from the embedded image processor is t. 11 t 21 t 31 …t n1 The received video data F1′, F2′, F3′…F after processing by the embedded image processor n ′.

[0048] Meanwhile, by preparing data from multiple scenarios and repeating steps one, two, and three, we can obtain analytical data for different application scenarios.

[0049] F1, F2, F3…F n Processing time is T1T2T3…T n, F1 frame processing time is T1=t 11 -t1, F n frame processing time is T n =t n1 -t n .

[0050] In the recognition result, if the n-th frame receives a position coordinate and the real position coordinate Euclidean distance is less than a certain value th (such as 3 pixels), it can be considered correct, recorded as R n , otherwise it is considered to be incorrect.

[0051]

[0052] The number of correct recognitions is:

[0053]

[0054] The recognition accuracy rec is the most important indicator, defined as the ratio of the number of correct samples to the total number of samples, recorded as:

[0055]

[0056] The recognition speed V rec is the speed of realizing the target recognition function, which is the average of the target recognition processing time of each frame image, recorded as:

[0057]

[0058] The generalization ability f is the ability to correctly recognize the target when inputting non-training sample data, recorded as:

[0059]

[0060] In formula (5), R1' is the number of correctly recognized non-training sample targets in the first frame, R1" is the total number of non-training sample targets in the first frame, m is the number of recognition frames, and jj m is the total number of real targets of the m-th frame of non-training samples.

[0061] Step four, according to the above indicators, the comprehensive evaluation target recognition performance comprehensive index is obtained by weighting each indicator, recorded as:

[0062] E=w1·rec+w2·f-w3·V vec (6)

[0063] In formula (6), w1, w2 and w3 are the weight values of each indicator, which can be allocated according to the actual engineering application scene.

[0064] Obviously, the above embodiments are only a part of the embodiments of the present application, and are not all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the protection scope of the present application.

Claims

1. An image target recognition performance evaluation apparatus characterized by comprising: The device is based on a full programmable array on-chip digital processing chip as a framework, comprising a gigabit network data transmission module, a video interface conversion module, a VDMA module, an AXI4-stream to Video out module, a communication module, an OSD module, a video stream timing generation module, a dynamic clock configuration module, a full programmable array on-chip digital processing module and a performance evaluation module. The performance evaluation module transmits the prepared scene video data to the full programmable array on-chip digital processing module through the gigabit network data transmission module, and the VDMA module reads the prepared image and tells the embedded image processor to start recognition through the communication module. At the same time, the VDMA module injects the video stream data in the memory into the embedded image processor through the video interface conversion module under the timing generated by the video stream timing generation module according to the requirements of the embedded image processor input video. The embedded image processor receives the injected image data to start target recognition, and the recognition result is transmitted to the full programmable array on-chip digital processing module through the communication module. The full programmable array on-chip digital processing module transmits the recognition result data to the performance evaluation module through the gigabit network data transmission module. The performance evaluation module performs performance evaluation in different dimensions according to the result data issued and received. The gigabit network data transmission module realizes that the performance evaluation module repeatedly injects scene image data in different environments and with increased interference and classic data sets into different embedded image processors, and also has communication function. The video stream timing generation module generates timing suitable for different embedded image processor video input interface timing according to the video stream in the memory. The video interface conversion module realizes video injection of different embedded image processors under the cooperation of the video stream timing generation module. The performance evaluation module collects embedded image processor working process data after injecting scene image data in different environments and with increased interference and classic data set data, and evaluates the target recognition performance of the embedded image processor in different dimensions to form comprehensive performance chart data.

2. The image target recognition performance evaluation apparatus according to claim 1, wherein The VDMA module realizes the reading and storage function of the video stream in the memory with high bandwidth.

3. The image target recognition performance evaluation apparatus according to claim 2, wherein The full programmable array on-chip digital processing module is the core data processing module of the entire device, which is the center of data flow control, workflow control and data interaction.

Images