A general performance evaluation system for visual collision detection algorithms

Abstract

This invention relates to the field of collision detection technology, and in particular to a general performance evaluation system for visual collision detection algorithms. The system includes a video data module for calibrating optical variables and storing motion modal videos of approaching collision objects to obtain a standard video dataset; a data transmission module for exporting data from the standard video dataset to a test algorithm model to obtain test results, and also for importing the test results into an analysis module; and an analysis module for analyzing the test algorithm model using performance analysis and algorithm theory analysis to obtain analysis results. This invention can comprehensively evaluate collision detection algorithms, helping to analyze the specific advantages and disadvantages of visual collision detection algorithms, promoting the overall development and performance improvement of such algorithms, and increasing the practical applicability of visual collision detection algorithms.

Description

Technical Field

[0001] This invention relates to the field of collision object detection technology, and in particular to a general performance evaluation system for a visual collision detection algorithm. Background Technology

[0002] Collision detection involves detecting objects within the machine's vision range that may collide with it in advance, giving the machine sufficient time to avoid collisions. Current collision detection algorithms include those based on collision time. τ is used for detecting and warning of colliding objects; partly based on the peak release rate of neurons and The linear relationship between l (object size) and v (object approach velocity) is used to calculate the threshold angle θ for collision detection; collision detection is performed by recognizing objects based on neural networks and estimating depth information using datasets and regression algorithms; collision risk is assessed by detecting the motion information v of objects based on optical flow; collision detection is performed by recognizing objects based on feature points and determining their expansion rate (Er); obstacle avoidance detection is performed by extracting the motion velocity information of objects in videos and calculating membrane potential values ​​based on the locust visual neuron mechanism.

[0003] With the increasing number of collision detection algorithms, each algorithm has its own advantages and disadvantages in different scenarios. How to comprehensively evaluate the quality of the algorithm is an urgent problem. Solving this problem is of great significance to promoting the development of such algorithms. Some current evaluation methods include: (1) Constructing a real and simulated scenario (setting obstacle points at different locations and adding factors that may cause the algorithm to make detection errors), mounting the algorithm on an intelligent mobile robot, and then placing the intelligent robot at any location in the scenario and starting it up. Let it move continuously in the scenario and calculate the probability of successfully detecting approaching objects and correctly avoiding collisions; (2) Placing the same algorithm in different scenarios and comparing the detection effect of the algorithm in different scenarios to judge the algorithm's performance in detecting approaching objects. For example, the algorithm can be used to detect approaching, retreating, and translating objects in bright and dark environments to illustrate the movement mode detected by the algorithm. (3) By constructing a dataset, labeling each video as a collision video, evaluating the test effect of similar algorithms on the dataset, and comparing the effects to find the approach detection algorithm with better performance than other algorithms, mainly by comparing the detection success rate; (4) By improving the algorithm, comparing the test effects of the algorithm before and after the improvement in the same scene to illustrate the advantages of the improved algorithm, such as: compared with LGMD, DLGMD can eliminate the false detection of looming caused by the drone's pitch and acceleration; (5) Comparing the difference between the true value and the detected value to illustrate the accuracy of the approach detection algorithm's detection quantity.

[0004] These evaluation methods for collision detection algorithms each have their own advantages, but they often only evaluate the algorithm based on a specific application performance, resulting in a relatively singular evaluation perspective that fails to reveal the essence of the collision detection algorithm. This leads to many current collision detection algorithms exhibiting only a single advantage in practical applications, making them unsuitable for complex real-world scenarios. If a comprehensive statistical analysis of the algorithm's affinity to various collision-related visual cues could be conducted, it would be possible to clarify which aspect of the approaching object the algorithm is detecting, thus revealing a deeper understanding of the collision detection algorithm's essence and guiding the direction of algorithm optimization. Summary of the Invention

[0005] To address the aforementioned issues, this invention provides a general performance evaluation system for visual collision detection algorithms. This system comprehensively considers optical, visual, and physical spatial variables related to collision detection, fully reflects the properties of collision detection algorithms, and provides analytical evaluation results. This helps analyze the specific advantages and disadvantages of visual collision detection algorithms, promotes the overall development and performance improvement of such algorithms, and increases the practical applicability of visual collision detection algorithms.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A general performance evaluation system for a visual collision detection algorithm includes a video data module, a data transmission module, an analysis module, and the evaluation module.

[0008] The video data module is used for optical variable calibration of the motion modal video of the approaching collision object and storage of the motion modal video of the approaching collision object, so as to obtain a standard video dataset.

[0009] The data transmission module is used to export data from the standard video dataset to the test algorithm model to obtain the test results of the test algorithm model, and the data transmission module is also used to import the test results;

[0010] The analysis module is used to acquire the standard video dataset through the data transmission module, and the analysis module analyzes the test algorithm model through application performance analysis and algorithm theory analysis to obtain analysis results;

[0011] The evaluation module is used to acquire data from the analysis module and the data transmission module, so as to obtain an evaluation report based on the test results and the analysis results.

[0012] Furthermore, the optical variable calibration in the video data module includes collision time, collision distance, angular acceleration, angle, angular velocity, and the object's expansion rate.

[0013] Furthermore, the data transmission module is provided with a data export interface to export the standard video dataset of the video data module to the test algorithm model and the analysis module; the data transmission module is provided with a data import interface to import the test results of the test algorithm model into the data transmission module.

[0014] Furthermore, in the application performance analysis, noise is added to the videos in the standard video dataset, and the analysis module imports the noise-added videos into the test algorithm model.

[0015] Furthermore, the analysis module calibrates the difference map of the approaching object in the noisy video to distinguish the difference regions of the background and the colliding object, and the analysis module tests the calibrated difference map through the test algorithm model to obtain application performance analysis results.

[0016] Furthermore, in the algorithm theoretical analysis, each optical variable of the video in the standard video dataset is extracted, and the analysis module calculates the Spearman correlation coefficient of the optical variables and the test results to obtain the algorithm theoretical analysis results.

[0017] Furthermore, a correlation coefficient diagram is obtained based on the theoretical analysis results of the algorithm to represent the relationship between the test results and the optical variables.

[0018] Furthermore, the evaluation module is used to obtain an analysis report based on the application performance analysis results and the algorithm theoretical analysis results.

[0019] The beneficial effects of this invention are:

[0020] The video data module captures various motion modes of approaching objects to form a video dataset. Simultaneously, the optical variables of the approaching objects are labeled using motion capture systems and image processing methods, and noise is added using MATLAB to expand the standard video dataset. The standard dataset module contains video data covering most of the motion modes of objects tested by current model algorithms for collisions, and the labeled optical variables cover those studied by existing collision algorithm models, making the analysis results more comprehensive and the evaluation scope broader. Videos from the selected standard video dataset are used to test the algorithm model to obtain test results. The analysis module processes the videos from the standard video dataset to perform application performance analysis and algorithm theory analysis on the test algorithm model, obtaining analysis results. The test results are compared with the analysis results to determine the performance of the test algorithm model.

[0021] By studying the collision process from the observer's perspective, it can be seen that the probability of a collision gradually increases as the time of the impending collision approaches. From this perspective, all known visual collision detection algorithms can be classified as abstracting certain optical features X in an image and then transforming them into an estimate of the time of the impending collision or a prediction of the probability of the collision through some kind of mapping. Therefore, this invention constructs a labeled dataset to conduct a specific analysis of the algorithm in terms of both theory and practical application (a comprehensive analysis of both characteristics and performance). This allows us to obtain the algorithm's response to specific optical feature variables, thereby more accurately evaluating the performance of the algorithm model in various scenarios. Attached Figure Description

[0022] Figure 1 This is a block diagram of a general performance evaluation system for a visual collision detection algorithm according to a preferred embodiment of the present invention.

[0023] Figure 2 This is a schematic diagram of the optical variable change process of a general performance evaluation system for a visual collision detection algorithm according to a preferred embodiment of the present invention.

[0024] Figure 3(a) is a schematic diagram of the polygonal box annotation of a general performance evaluation system for a visual collision detection algorithm according to a preferred embodiment of the present invention.

[0025] Figure 3(b) is a segmentation image of a general performance evaluation system for a visual collision detection algorithm according to a preferred embodiment of the present invention.

[0026] Figure 4(a) is a schematic diagram of the first method of angle annotation of a general performance evaluation system for a visual collision detection algorithm according to a preferred embodiment of the present invention.

[0027] Figure 4(b) is a schematic diagram of the second method of angle annotation for a general performance evaluation system of the visual collision detection algorithm of a preferred embodiment of the present invention.

[0028] Figure 5(a) is a schematic diagram of a general performance evaluation system for a visual collision detection algorithm according to a preferred embodiment of the present invention with added salt and pepper noise.

[0029] Figure 5(b) is a schematic diagram of translational motion interference in a general performance evaluation system for a visual collision detection algorithm according to a preferred embodiment of the present invention.

[0030] Figure 6(a) is a differential diagram of a general performance evaluation system for a visual collision detection algorithm according to a preferred embodiment of the present invention.

[0031] Figure 6(b) is a mask image of a general performance evaluation system for a visual collision detection algorithm according to a preferred embodiment of the present invention.

[0032] Figure 6(c) is a labeled difference graph of a general performance evaluation system for a visual collision detection algorithm according to a preferred embodiment of the present invention.

[0033] Figure 6 is a line graph showing the algorithm noise reduction analysis of a general performance evaluation system for the visual collision detection algorithm according to a preferred embodiment of the present invention.

[0034] Figure 7 This is a flowchart illustrating the algorithmic theoretical analysis of a general performance evaluation system for a visual collision detection algorithm according to a preferred embodiment of the present invention.

[0035] Figure 8 This is a schematic diagram of the correlation coefficients between optical variables and test results in a general performance evaluation system for a visual collision detection algorithm according to a preferred embodiment of the present invention.

[0036] Figure 9 A diagram of the data transmission module interface of a general performance evaluation system for a visual collision detection algorithm according to a preferred embodiment of the present invention. Detailed Implementation

[0037] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0038] It should be noted that when a component is described as "fixed to" another component, it can be directly on the other component or may have a component in between. When a component is considered "connected to" another component, it can be directly connected to the other component or may have a component in between. When a component is considered "set on" another component, it can be directly set on the other component or may have a component in between. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this document are for illustrative purposes only.

[0039] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0040] like Figures 1 to 8 As shown, a general performance evaluation system for a visual collision detection algorithm according to a preferred embodiment of the present invention includes a video data module 1, a data transmission module 2, an analysis module 3, and an evaluation module 4.

[0041] Video data module 1 is used for optical variable calibration of motion modal videos of approaching collision objects and storage of motion modal videos of approaching collision objects to obtain a standard video dataset.

[0042] In this embodiment, the optical variable calibration in the video data module 1 includes collision time, collision distance, angular acceleration, angle, angular velocity, and object expansion rate.

[0043] The process of optical variable change is as follows Figure 2 As shown, L is the maximum imaging size of the approaching object in the field of view R. L moves from time T3 to time T1, and the object's several basic optical variables (T, D, θ, ...) change. The process of change is shown in the figure. The expansion rate E is actually the change in the area of ​​the image formed by the object in the human eye.

[0044] Image annotation method: We accurately annotated the approaching objects in the video dataset with polygonal bounding boxes using the Darwin V7 website, as shown in Figure 3(a). Through annotation, we can accurately obtain the instance segmentation image of the pixel region where the moving object is located, as shown in Figure 3(b), and the pixel coordinates of each vertex of the polygon.

[0045] The Time-to-Collision (TTC) notation method is as follows: the collision time is set to zero, and the timestamps of the video frames preceding the collision are sequentially negative. The notation formula is:

[0046]

[0047] Where framenumber is the ordinal number of the labeled frame, totalframenumbers is the total number of frames in the video, and frame_rate is the frame rate of the video.

[0048] The method for labeling the dilation rate is as follows: The instance segmentation images of the approaching object in the video stream I(x,y,t) are binarized with the background, i.e., the labeled region of the approaching object is set to 1, and the background region is set to zero, resulting in a binarized background image sequence I. Binary_Background (x,y,t) and the binarized approach target image sequence I Binary_Looming (x, y, t). Label as follows:

[0049]

[0050] The area of ​​an object is represented by the sum of the pixel values ​​of the binarized images of the segmented images of the approaching object instances in the two frames.

[0051] Angle labeling:

[0052] The first annotation method: Utilizing labeled datasets and the Python programming language for computation. Firstly, as... Figure 1 We extract the pixel coordinates of the vertices of the annotation box obtained by the annotation software, and calculate the pixel coordinates of the two farthest points as A(x) and A(x). A ,y A ), B(x) B ,y B According to the cosine formula, the field of view angle corresponding to the largest dimension AB of the object is:

[0053]

[0054] As shown in Figure 4(a), with the top left corner of the image as the origin (0, 0), C(x) center ,y center ) represents the center point of the pixel plane. H represents the camera focus. θ fov This is the horizontal field of view of the camera.

[0055] The second annotation method: θ is calculated using the Vicon Tracker motion capture system, as shown in Figure 4(b). The test object is an approximate cube, where E((x) = 0.05. E ,y E , z E ), F(x) F ,y F , z F ), G(x) G ,y H , z G In a real-world scenario, V(x) represents the cursor sphere located at the three vertices of one face of a cube. V ,y V , z V ( ) represents the location of the camera. In the actual scene, E and G are diagonal vertices, i.e., the longest sides of the object. Calculate the corresponding field of view angle:

[0056]

[0057] Method for specifying angular velocity:

[0058]

[0059] Where, θ previous_frame θ next_frame These are the angles corresponding to the approaching object in the two frames before and after.

[0060] Method for labeling angular acceleration:

[0061]

[0062] in, The angular velocity of the approaching object in the two consecutive frames.

[0063] Distance annotation methods:

[0064] As shown in the motion capture scene in Figure 4(b), the distance D is calibrated by calculating the distance from V to surface EFG. First, the equation of surface EFG is calculated using the coordinates of points E, F, and G: ax + by + cz + d = 0. Then:

[0065]

[0066] The values ​​calculated by the motion capture system are then labeled in the corresponding video frames.

[0067] This embodiment uses video datasets of various motion modes of colliding objects (e.g., a vehicle camera moving towards an object; an object moving towards a camera from various angles; objects translating and moving backward; video data of controlling the speed and direction of colliding objects of various common shapes in different scenarios in Unreal Engine simulations; and manually annotated difference map datasets). The optical variables of the colliding objects in each video are labeled, and the corresponding labels are presented in a table format as shown in Table 1. Furthermore, MATLAB programming is used to intentionally add noise to the dataset, selecting some typical video noise to expand it into a new dataset.

[0068]

[0069]

[0070]

[0071] Table 1. Dataset of Basic Optical Variables of Approaching Object

[0072] Using collision time, collision distance, angular acceleration, angle, angular velocity, and object expansion rate, the annotation data of each frame of the video is obtained through OpenCV programming and motion capture system, and stored in the database in the form of Table 1.

[0073] The data transmission module 2 is used to export data from the standard video dataset to the test algorithm model in order to obtain the test results of the test algorithm model. The data transmission module 2 is also used to import the test results.

[0074] In this embodiment, the data transmission module 2 is provided with a data export interface to export the standard video dataset from the video data module 1 to the test algorithm model and analysis module 3; the data transmission module 2 is also provided with a data import interface to import the test results of the test algorithm model into the data transmission module 2. For example... Figure 9As shown, in the software interface created using C# forms, the data transmission module 2 in this embodiment selects the video dataset option, selects the video dataset used to evaluate the model algorithm, and clicks download to obtain the required video dataset.

[0075] Analysis module 3 is used to acquire standard video datasets through data transmission module 2, and analysis module 3 analyzes the test algorithm model through application performance analysis and algorithm theory analysis to obtain analysis results.

[0076] In the application performance analysis, noise is added to the videos in the standard video dataset, and the analysis module 3 imports the noise-added videos into the test algorithm model.

[0077] Analysis module 3 calibrates the difference map of the approaching object in the noisy video to distinguish the difference regions of the background and the colliding object. Analysis module 3 then tests the calibrated difference map using a test algorithm model to obtain application performance analysis results. In this embodiment, the application performance analysis focuses on the noise resistance of the DLGMD algorithm. Noisy video of a car with a camera traveling towards a colliding object at 2.5 m / s is extracted and imported into the test algorithm model to obtain application performance analysis results. Similarly, standard video dataset of the same scenario is extracted and imported into the test algorithm model to obtain test results.

[0078] By comparing the application performance analysis results with the test results, the video with added salt and pepper noise and translational motion interference was selected for analysis. Figure 5(a) is a schematic diagram of added salt and pepper noise, and Figure 5(b) is a schematic diagram of translational motion interference.

[0079] Simultaneously, a difference map was used to verify the algorithm's suppression of the original background noise. The difference map annotation method is as follows: Figure 6(a) , 6(b) Figure 6(c) shows the difference image, where Figure 6(a) is the mask image, and Figure 6(c) is the labeled difference image obtained by binarizing the mask image and multiplying it by Figure 6(b). The labeled difference image is then tested in the algorithm to obtain the application performance analysis results of the labeled difference image. The test results and application performance analysis results of this embodiment are combined to generate a line graph of the algorithm's noise reduction analysis, as shown in Figure 6(d).

[0080] In the application performance analysis, the DLGMD algorithm of the test algorithm model was tested for noise resistance. The test results of the test algorithm model, along with the labeled difference map data, image data with added salt and pepper noise, and image data with added translation noise, are presented in Table 2. The application performance analysis comparison data in this embodiment can be imported in .txt file format.

[0081]

[0082]

[0083]

[0084]

[0085] Table 2 Comparison of Application Performance Analysis

[0086] In the algorithm theory analysis, each optical variable is extracted from the standard video dataset, and analysis module 3 calculates the Spearman correlation coefficient between the optical variables and the test results to obtain the algorithm theory analysis results. A correlation coefficient diagram is obtained based on the algorithm theory analysis results to represent the relationship between the test results and the optical variables.

[0087] The formula for the Spearman correlation coefficient is shown below:

[0088]

[0089] Where Li is the ranking coefficient corresponding to the optical variable in the standard video dataset, and Gi is the ranking coefficient corresponding to the test result. The two correspond one-to-one with video frames, where i represents the frame number. A P is calculated in this way for each optical variable. LG The results are plotted as a radar chart to represent the relationship between the algorithm and each optical variable. The flowchart of the algorithm's theoretical analysis is as follows: Figure 7 As shown.

[0090] In the algorithm theory analysis example, a video of a car with a camera traveling at 2.5 m / s towards a collision object is used to test the DLGMD algorithm. Spearman correlation coefficients are calculated for each optical variable and the test result combination extracted from the standard video dataset. The resulting correlation coefficients between the optical variables and the test results are illustrated in the diagram below. Figure 8 As shown.

[0091] Evaluation module 4 is used to analyze the data acquired by module 3 and data transmission module 2, and to obtain an evaluation report based on the test results and analysis results. In this embodiment, a report is generated based on the line graph and relational number system diagram of the algorithm noise suppression analysis, and provided to the user.

[0092] This embodiment uses video data module 1 to capture various motion modes of approaching objects to form a video dataset. Simultaneously, the optical variables of the approaching objects are labeled with their true values, and noise is added using MATLAB to expand the standard video dataset. The standard dataset module contains video data covering most of the motion modes of collision objects tested by current model algorithms, and the labeled optical variables cover those studied by existing collision algorithm models, thus broadening the evaluation scope of this invention. Videos from the selected standard video dataset are tested against the test algorithm model to obtain test results. Analysis module 3 processes the videos from the standard video dataset to perform application performance analysis and algorithm theory analysis on the test algorithm model, obtaining analysis results. The test results are compared with the analysis results to determine the performance of the test algorithm model.

[0093] By studying the collision process from an observer's perspective, it can be seen that the probability of a collision gradually increases as the imminent collision time approaches. From this perspective, all known visual collision detection algorithms can be categorized as abstracting and transforming certain optical features X in an image into estimates of the imminent collision time or predictions of the collision probability. Therefore, this invention, by constructing a labeled dataset, conducts a detailed analysis of the algorithm's theoretical and practical applications, enabling the testing of the algorithm model's performance in various scenarios and the identification of specific optical feature variables. This embodiment helps to understand the essence of the algorithm from a theoretical perspective, making it easier to improve the algorithm as a whole and increasing its practical applicability.

Claims

1. A general performance evaluation system of visual collision detection algorithms, characterized in that, It includes a video data module (1), a data transmission module (2), an analysis module (3), and an evaluation module (4); The video data module (1) is used for optical variable calibration of the motion modal video of the approaching collision object and storage of the motion modal video of the approaching collision object, so as to obtain a standard video dataset. The data transmission module (2) is used to export the data of the standard video dataset to the test algorithm model to obtain the test results of the test algorithm model, and the data transmission module (2) is also used to import the test results; The analysis module (3) is used to obtain the standard video dataset through the data transmission module (2), and the analysis module (3) analyzes the test algorithm model through application performance analysis and algorithm theory analysis to obtain analysis results; In the application performance analysis, the videos in the standard video dataset are denoised, and the analysis module (3) imports the denoised videos into the test algorithm model; The analysis module (3) calibrates the difference map of the approaching object in the noisy video to distinguish the difference regions of the background and the colliding object. The analysis module (3) tests the calibrated difference map through the test algorithm model to obtain the application performance analysis results. In the algorithm theory analysis, each optical variable of the video in the standard video dataset is extracted. The analysis module (3) calculates the Spearman correlation coefficient of the optical variables and the test results to obtain the algorithm theory analysis results. The evaluation module (4) is used to acquire data from the analysis module (3) and the data transmission module (2) to obtain an evaluation report based on the test results and the analysis results.

2. The general performance evaluation system of visual collision detection algorithm according to claim 1, characterized in that: The optical variable calibration in the video data module (1) includes collision time, collision distance, angular acceleration, angle, angular velocity and the expansion rate of the object.

3. The general performance evaluation system of visual collision detection algorithm according to claim 1, characterized in that: The data transmission module (2) is provided with a data export interface to export the standard video dataset of the video data module (1) to the test algorithm model and the analysis module (3); the data transmission module (2) is provided with a data import interface to import the test results of the test algorithm model into the data transmission module (2).

4. The general performance evaluation system for a visual collision detection algorithm according to claim 1, characterized in that: Based on the theoretical analysis results of the algorithm, a correlation coefficient diagram is obtained to represent the relationship between the test results and the optical variables.

5. The general performance evaluation system of visual collision detection algorithm according to claim 1, characterized in that: The evaluation module (4) is used to obtain an analysis report based on the application performance analysis results and the algorithm theoretical analysis results.

Images