Method, device, equipment and storage medium for visualizing target detection effect

Abstract

The present disclosure provides a method, apparatus, device and storage medium for visualizing target detection effects, which relate to the field of image processing, and in particular to the field of target detection. The specific implementation scheme is as follows: the method draws a true value box and a detection box on a 2D canvas respectively according to the annotation results and detection results of the point cloud data containing 3D information, and matches the annotation results and the detection results according to the 3D information. If they do not match, the real target type is added to the true value box, and the predicted target type and prediction score are added to the detection box. In this way, the annotation results and detection results of the point cloud data can be converted into a 2D visualization interface, and the difference between the true value box and the detection box can be intuitively obtained. Since the matching of the annotation results and the detection results is based on 3D information, the error in height between the detection results and the annotation results can still be reflected.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to the field of target detection technology. Background Art

[0002] The roadside perception system can provide autonomous vehicles with a wider full-range field of view and can send blind spot information to autonomous vehicles. LiDAR, as the core sensor of the roadside perception system, plays a vital role in the perception ability of autonomous vehicles, especially in special scenarios such as nighttime.

[0003] However, visualizing LiDAR detection results has always been a challenge in the industry. Compared to traditional image recognition, point clouds are difficult to represent. Existing solutions typically use 3D engines for processing, but loading each frame takes a long time. This makes it difficult to quickly locate the root cause when problems arise in the LiDAR detection algorithm, which in turn affects the iteration speed of the detection algorithm. Summary of the Invention

[0004] The present disclosure provides a method, apparatus, device, and storage medium for visualizing target detection effects.

[0005] According to one aspect of the present disclosure, a method for visualizing target detection effects is provided, including: drawing a 2D rectangular box of a first style on a 2D canvas according to the annotation results of point cloud data to obtain a true value box, the annotation result including first 3D information of the detected target and the true target type; drawing a 2D rectangular box of a second style on the 2D canvas according to the detection results of the point cloud data to obtain a detection box, the detection result including second 3D information of the detected target, a predicted target type and a predicted score; comparing the first 3D information and the second 3D information to determine whether the detection result matches the annotation result, and if not, adding the true target type to the true value box, and adding the predicted target type and the predicted score to the detection box.

[0006] According to another aspect of the present disclosure, a device for visualizing target detection effects is provided, including: a true value box drawing module, used to draw a 2D rectangular box of a first style on a 2D canvas according to the annotation results of point cloud data to obtain a true value box, and the annotation results include first 3D information of the detection target and the true target type; a detection box drawing module, used to draw a 2D rectangular box of a second style on the 2D canvas according to the detection results of the point cloud data to obtain a detection box, and the detection results include second 3D information of the detection target, a predicted target type and a predicted score; an error information adding module, used to compare the first 3D information and the second 3D information to determine whether the detection result matches the annotation result, and if not, add the true target type to the true value box, and add the predicted target type and the predicted score to the detection box.

[0007] The present disclosure provides a method, apparatus, device and storage medium for visualizing target detection effects. The method draws a true value box and a detection box on a 2D canvas respectively based on the annotation results and detection results of the point cloud data containing 3D information, and matches the annotation results and the detection results according to the 3D information in the point cloud data. If they do not match, the real target type is added to the true value box, and the predicted target type and prediction score are added to the detection box. In this way, the annotation results and detection results of the point cloud data can be converted into a 2D visualization interface, and the user can intuitively obtain the difference between the true value box and the detection box from the interface. Although the interface is 2D, since the matching of the annotation results and the detection results is based on 3D information, when the real target type on the true value box is the same as the predicted target type on the detection box, it can be inferred that the height of the detected target is different from the height of the real target, so that the height error in the 3D information can still be reflected through the 2D interface.

[0008] Compared with using 3D space to display the difference between the annotation results and the detection results of point cloud data, the interface displayed by the target detection effect visualization method disclosed in the present invention is more intuitive, easy to implement, and has a faster processing speed. It is very helpful for quickly locating the problem type and cause of the target detection problem, and can indirectly promote the rapid convergence of the target detection algorithm and improve the precision and accuracy of the target detection algorithm.

[0009] Moreover, the target detection effect visualization method disclosed in the present invention has low requirements for computing resources, can significantly save system computing power, and is easier to promote.

[0010] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0012] Figure 1 1 is a flow chart of a method for visualizing target detection effects according to the first embodiment of the present disclosure;

[0013] Figure 2 is a schematic diagram of a truth box and a detection box of the first embodiment of the present disclosure;

[0014] Figure 3 is a schematic diagram of an example of visualization of the detection effect of the first embodiment of the present disclosure;

[0015] Figure 4This is a schematic diagram of the effect of drawing the background and detection target based on point cloud data in the second embodiment of the present disclosure;

[0016] Figure 5 is a schematic diagram of a canvas drawn based on point cloud data acquired by a laser radar according to the third embodiment of the present disclosure;

[0017] Figure 6 3 is a flow chart of a method for visualizing target detection effects according to the third embodiment of the present disclosure;

[0018] Figure 7 A schematic diagram of the structure of a device for visualizing target detection effects according to an embodiment of the present disclosure;

[0019] Figure 8 4 is a block diagram of an electronic device used to implement the method for visualizing target detection effects according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0020] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0021] Figure 1 The main process of the method for realizing the visualization of target detection effect according to an embodiment of the present disclosure is shown. Figure 1 The method includes: operation S110, drawing a 2D rectangular box of a first style on a 2D canvas according to the annotation result of the point cloud data to obtain a true value box, and the annotation result includes the first 3D information of the detection target and the true target type; operation S120, drawing a 2D rectangular box of a second style on the 2D canvas according to the detection result of the point cloud data to obtain a detection box, and the detection result includes the second 3D information of the detection target, the predicted target type and the prediction score; operation S130, comparing the first 3D information and the second 3D information to determine whether the detection result matches the annotation result. If they do not match, adding the true target type to the true value box and adding the predicted target type and the prediction score to the detection box.

[0022] In operation S110 , point cloud data refers to a set of vectors in a three-dimensional coordinate system, recorded in the form of points, each of which contains three-dimensional coordinates, and some may contain color information (RGB) or reflection intensity information (Intensity).

[0023] The annotation results of point cloud data refer to the true value annotation of the detection target in the point cloud data. For example, the true target position of the detection target (usually a cubic area) and the true target type of the detection target are marked in the point cloud data for comparison of the detection results.

[0024] A 2D canvas is a two-dimensional surface used for drawing, on which images are drawn, and is usually composed of points with two-dimensional coordinates.

[0025] Style refers to the attributes of a graphic, such as color, grayscale, and fill pattern, that can give it different display effects. It is mainly used to distinguish different objects in addition to shape and size.

[0026] A 2D rectangular frame refers to a rectangular or square frame formed by four sides. It is often used in target detection to frame the image portion where the target is located.

[0027] 3D information generally includes three-dimensional coordinates representing length, width, and height.

[0028] Drawing a line or shape on the canvas means changing the points corresponding to the line or shape to a set style and displaying it again to achieve an effect similar to drawing with a pen on paper in the real world.

[0029] The true value box is a 2D rectangular box drawn after performing a two-dimensional transformation on the real space where the detection target is located in the annotation result of the point cloud data, that is, the space described by the first 3D information, and projecting it onto the 2D canvas to obtain the two-dimensional coordinates.

[0030] When converting the first 3D information of the detection target into two-dimensional coordinates, it can be obtained by simply removing the height information; it can also be obtained by projecting according to a certain angle as needed, that is, first converting the three-dimensional coordinates according to a certain angle and then removing the height information.

[0031] In operation S130, the detection result refers to the output result obtained by inputting the point cloud data into the target detection system for target detection, which generally includes the second 3D information of the detected target, the predicted target type, and the prediction score. The predicted target type is the target type obtained by the target detection system after detecting the target. The predicted target type is the predicted value given by the target detection system, which generally provides a probability corresponding to the value, and the prediction score is the probability value of the predicted target type.

[0032] Similar to the true value box, the detection box is a 2D rectangular box drawn after the space where the detection target in the detection result is located, that is, the space described by the first 3D information, is two-dimensionally transformed and projected onto the 2D canvas to obtain two-dimensional coordinates.

[0033] To facilitate the distinction and comparison between the ground truth box and the detection box, the detection box is presented in a second style that differs from the first. For example, different styles such as different colors or different line types (for example, solid line, dashed line, dotted line, etc.) are used. The higher the contrast between the first and second styles, the easier it is to distinguish and compare the ground truth box and the detection box, and the easier it is to detect the difference between the ground truth box and the detection box.

[0034] After completing operations S110 and S120, the detection effects such as the gap between the position of the detection target in the detection result and the actual position of the detection target in the annotation result, as well as the degree of overlap, can be basically observed intuitively through the truth box and the detection box. However, it is not possible to know whether the target type of the detection target in the detection result is correct.

[0035] Therefore, in operation S130, it is necessary to add the true target type to the ground truth box and the predicted target type and prediction score to the detection box. In this way, it is possible to visually observe whether the position of the detected target in the detection result is correct and whether the target type in the detection result is correct.

[0036] However, since both the ground truth box and the detection box are projected results, they cannot reflect the difference in height. To this end, the inventors of the present disclosure have creatively come up with a method of comparing the first 3D information in the annotation result with the second 3D information in the detection result. Only when the first 3D information does not match the second 3D information or the predicted target type is different from the true target type (i.e., the detection result does not match the annotation result), the true target type is added to the ground truth box and the predicted target type is added to the predicted box.

[0037] When the first 3D information and the second 3D information are compared, the three-dimensional coordinates of the space where the detection target is located are compared, which includes not only length and width but also height information.

[0038] In this way, when the ground truth box and the detection box overlap and the predicted object type is the same as the true object type, the true object type is still added to the ground truth box, and the predicted object type and prediction score are added to the detection box. This indicates that the height information of the detected object in the detection result is different from the height information in the annotation result. This cleverly achieves the purpose of reflecting height differences on the 2D canvas.

[0039] Figure 2A visualization example of the target detection effect is shown, where the detection box 2012 represents the detection box of the detection target 201 and is drawn with a solid line; the ground truth box 2011 represents the ground truth box of the detection target 201 and is drawn with a dotted line; if, after comparing the first 3D information and the second 3D information, it is determined that the detection result does not match the annotation result, the predicted target type 20121 and the predicted score 20122 are added to the detection box 2012, and the real target type 20111 is added to the ground truth box.

[0040] By using the above rules, different detection effects can be displayed differently on the canvas, so they can all be observed intuitively.

[0041] Figure 3 It shows how several different detection effects are reflected in the 2D canvas.

[0042] Among them, the detection box (solid box) and the true value box (dashed box) of the detection target A do not overlap, but the predicted target type added to the detection box is consistent with the real target type added to the true value box, both of which are "vehicles". This means that in the target detection result of the detection target A, the position is inaccurate but the type is correct.

[0043] The detection frame (solid frame) and the true value frame (dashed frame) of detection target A overlap, but the predicted target type added to the detection frame is "vehicle", which is different from the real target type "traffic sign screen" added to the true value frame. This means that the target detection type for detection target B is incorrect.

[0044] The detection target C only has a detection box (solid box) but no true value box, which means that the background is mistakenly detected as the detection target.

[0045] The detection target D only has a true value box (dashed box) but no detection box, which means that the detection target D has not been detected and has been missed.

[0046] The detection box (solid box) and the true value box (dashed box) of the detection target E overlap, and the predicted target type added to the detection box and the real target type added to the true value box are both "vehicle", which means that the height information of the detection target E in the detection result is inconsistent with the height information in the annotation result.

[0047] It can be seen that by using the method of visualizing the target detection effect provided by the embodiment of the present disclosure, the difference between the detection results and the true value annotation results can be discovered more intuitively and quickly, and the specific errors in the detection results can be specifically located, for example, whether the position is accurate or the type is incorrect; whether it is a missed detection or a false detection; whether the plane detection is inaccurate or the height detection is inaccurate, etc.

[0048] This can help R&D personnel quickly locate problems in target detection algorithms and the root causes of these problems, thereby achieving rapid iteration and making the target detection algorithm more robust.

[0049] In addition, since the method for visualizing the target detection effect provided in this embodiment is implemented through a 2D canvas, it is more intuitive than a 3D space display, and has lower algorithm complexity and low requirements for computing resources. It can greatly save system computing power and is easier to promote.

[0050] In another embodiment of the present disclosure, the background and detection target are drawn on the 2D canvas according to the point cloud data. Figure 4 shown.

[0051] The light gray part is the ground, and the dark gray part that looks like a top view of a vehicle is the detection target: the vehicle.

[0052] This allows for a more intuitive visual inspection of the accuracy of the detection frame and the predicted target type. In particular, if errors in the annotation data are present, the background and detected targets drawn from the point cloud data can be displayed on the canvas to correct misjudgments.

[0053] Secondly, drawing the background and detection target on a 2D canvas based on the point cloud data can also help with preliminary analysis to determine the cause of detection errors. For example, observing the similarities between the drawn background and the detection target can determine the cause of the background misdetection or the missed detection of the detection target.

[0054] Furthermore, drawing the background and detection targets on the 2D canvas can be set as an option, allowing the user to decide whether to display them or not. This allows users to choose not to draw the background and detection targets during initial analysis, making the ground truth and detection frames clearer and easier to observe and identify any issues in the detection results. Once issues have been identified, the background and detection targets can be drawn for further confirmation and analysis.

[0055] In another embodiment of the present disclosure, when, after comparing the first 3D information and the second 3D information, it is determined that the detection result matches the annotation result, that is, the first 3D information and the second 3D information almost overlap and the predicted target type is consistent with the actual target type, the predicted target type and prediction score will still be added to the detection frame.

[0056] In this way, even if the detection results match the annotation results, the predicted scores of the predicted probabilities can be further observed, allowing for more fine-tuning of the model accuracy.

[0057] In another embodiment of the present disclosure, a 2D canvas is recreated each time the method for visualizing target detection results is executed to ensure that each target detection result visualization result is based on a clean canvas and is not affected by previous visualization results.

[0058] In another embodiment of the present disclosure, the completed 2D canvas is also stored as an image. In this way, the image generated by the object detection result visualization method can be retained for subsequent analysis or as historical data to evaluate the improvement trend and effect comparison of the object detection algorithm.

[0059] It should be noted that in the above-mentioned embodiments of the present disclosure, the point cloud data can be 3D point cloud data collected in any scenario, for example, 3D point cloud data constructed by 3D modeling based on images captured by cameras at different locations. The above-mentioned method for visualizing target detection effects can be applied to any scenario where target detection is performed using 3D point cloud data, such as a compliance detection system used to monitor safety and compliance operations on a production line.

[0060] Figure 5 and Figure 6 Another embodiment of the present disclosure is shown. In this embodiment, the detection target is an obstacle on the road. The point cloud data used is acquired by a roadside lidar. Accordingly, the annotation results also include first radar coordinate information, and the detection results also include second radar coordinate information. The radar coordinate information includes the X, Y, and Z axis coordinates and the Z-axis heading angle in the radar coordinate system.

[0061] In order to more clearly display the position of the detected target in the radar coordinate system, the horizontal field of view angle boundary is also drawn on the 2D canvas according to the horizontal field of view angle range of the laser radar.

[0062] If the horizontal field of view angle range is 70 degrees, the horizontal field of view angle boundary is two line segments drawn with the radar as the center. The two line segments are symmetrical up and down, and the angle between them is 70 degrees. Figure 5 The horizontal field of view angle boundary 501 is shown in FIG.

[0063] In addition, in order to quickly find the distance between the problem obstacle and the laser radar, you can also use the laser radar as the center of the circle and draw at least two concentric sectors with scales within the two horizontal field of view angle edges. For example, concentric sectors are drawn at 60 meters, 90 meters, 120 meters, 150 meters, and 180 meters, as shown in the figure. Figure 5 Concentric sectors 502 are drawn with dashed lines within the middle horizontal field of view boundary 501 .

[0064] Furthermore, in order to clearly show the range perceived by the laser radar, an effective perception range rectangle can be drawn on the 2D canvas according to the perception range of the laser radar, such as Figure 5 The rectangular box 503 shown in FIG.

[0065] In addition, since the point cloud data obtained by the LiDAR is frame by frame, each frame has a unique label to facilitate fast data positioning. Therefore, labels can also be added to the effective sensing range rectangle, such as Figure 5 The label 504 shown in the figure is used to label constants such as the distance from the rectangular box and the number of frames, so as to facilitate identification of the distance from the rectangular box and the current visualization frame distance and / or frame number.

[0066] Specifically, Figure 6 The main process of the method for visualizing the target detection effect in the embodiment of the present disclosure is shown. In the embodiment of the present disclosure, the method for visualizing the target detection effect is performed by implementing a visualization tool, such as Figure 6 As shown, it mainly includes:

[0067] Perform the following steps to obtain the annotation results of the point cloud data:

[0068] Step S6110: The annotator processes the point cloud file and the corresponding image file obtained by the roadside lidar using a true value annotation system to obtain an annotation result text;

[0069] Step S6120: The visualization tool reads the length, width, and height of the 3D information in the annotation result, the X, Y, and Z coordinates in the radar coordinate system, the Z-axis heading angle, and the obstacle label (detection target type);

[0070] Step S6130 , determining a total set of three-dimensional bounding boxes (3DBBoxes) of the true obstacles based on the data read in step S6120 ;

[0071] In another process parallel to the above process, the following steps are performed to obtain the detection results of the point cloud data:

[0072] Step S6210: Input the point cloud file acquired by the roadside LiDAR into the roadside LiDAR perception system to obtain a detection result;

[0073] Step S6220: The visualization tool reads the 3D information (length, width, and height of the 3DBBox, X, Y, and Z coordinates in the radar coordinate system, and the Z-axis heading angle) from the annotation result, the label (obstacle type), and the algorithm's predicted score for the obstacle type.

[0074] Step S6230, determining the total set of obstacle information detected in the current frame based on the data read in step S6220;

[0075] In another process parallel to the above process, the following steps are performed to obtain a 2D point cloud canvas:

[0076] Step S6310: The visualization tool creates a blank canvas;

[0077] Step S6320: The visualization tool initializes the blank canvas according to the hardware properties of the laser radar to obtain an initial canvas;

[0078] Specifically, it includes:

[0079] 1) Draw the horizontal field of view angle boundary; 2) Draw concentric and equidistant sectors; 3) Draw the effective perception range rectangle on a blank canvas based on the lidar's perception range capability; 4) Label constants such as the distance rectangle and the number of frames.

[0080] Step S6330: The visualization tool reads the point cloud file and draws the ground and obstacle points on the initial canvas to obtain a point cloud canvas.

[0081] Afterwards, the visualization tool verifies the detection effect based on the total set of 3DBBoxes of the true obstacle obtained in step S6130 and the total set of 3DBBoxes of the detected obstacle obtained in step S6230, specifically including:

[0082] Step S6240: Calculate the 3D bounding box of the ground-truth obstacle and the detected 3D bounding box based on the 3D bounding boxes in the total set of 3DBBoxes of the detected obstacles and the 3D bounding boxes in the total set of 3DBBoxes of the true obstacles.

[0083] Specifically, the following algorithm is used:

[0084] Suppose there are two cubes M and N divided into the following three cases:

[0085] M and N have no intersection, IoU = 0;

[0086] M and N have an intersection, IoU = (M∩N) / (M∪N), M∩N represents the volume of the intersection of M and N, and M∪N represents the volume of the union of M and N.

[0087] The threshold value is set as 0.5 for motor vehicles and 0.25 for other small obstacles.

[0088] When the IoU value is greater than the threshold setting, it is considered that the spatial position of the detected obstacle and the true obstacle are matched successfully, otherwise the matching fails.

[0089] After the spatial position is successfully matched, the obstacle label can be further matched. If they are consistent, the detected obstacle and the true obstacle are successfully matched. Otherwise, the obstacle and the true obstacle are not matched, and the problem in the detection algorithm needs further analysis.

[0090] Subsequently, the visualization tool draws information on the point cloud canvas based on the total set of 3DBBoxes of the true obstacles obtained in step S6130, the total set of 3DBBoxes of the detected obstacles obtained in step S6230, and the matching results between the detected obstacles and the true obstacles, to obtain the obstacle point cloud canvas. It should be noted that the point cloud canvas here refers to a 2D canvas that can display all point cloud information, not a 3D display space. Specifically, it includes:

[0091] In step S6340, if the detected obstacle matches the true obstacle successfully, the obstacle prediction type and prediction score are marked; if the match fails, 1) the true category of the obstacle is marked on the true value box, and 2) the predicted category and prediction score of the obstacle are marked on the detection box.

[0092] Step S6350: Save the obstacle point cloud canvas as an image file for subsequent processing.

[0093] In the disclosed embodiments, the aforementioned method for visualizing target detection results facilitates rapid iteration of the target detection algorithm of a roadside LiDAR and quickly identifies the LiDAR detection algorithm's detection capabilities for the current dataset. By examining the overlap between the detection frame and the ground truth frame frame by frame, it is possible to quickly determine whether the current algorithm is experiencing large-scale missed detections, false detections, or continuous missed detections in the dataset. Furthermore, by examining obstacles individually, problems can be identified and their causes located more quickly.

[0094] Furthermore, due to the above-mentioned method of visualizing target detection effects, the detection accuracy of roadside lidar has been rapidly improved, while also indirectly improving the perception accuracy of autonomous vehicles and vehicle-road coordination systems, which is conducive to the popularization and faster development of intelligent transportation.

[0095] It should be noted that the above embodiments of the present disclosure are merely illustrative descriptions and are not intended to limit the implementation methods and application scenarios of the method for visualizing target detection effects of the present disclosure.

[0096] In the technical solutions disclosed herein, the acquisition, storage, and application of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0097] According to an embodiment of the present disclosure, the present disclosure also provides a device for visualizing target detection effects, such as Figure 7As shown, the device 70 includes: a true value box drawing module 701, which is used to draw a 2D rectangular box on a 2D canvas according to the annotation result of the point cloud data to obtain a true value box, and the annotation result includes the first 3D information of the detection target and the true target type; a detection box drawing module 702, which is used to draw a 2D rectangular box on the 2D canvas according to the detection result of the point cloud data to obtain a detection box, and the detection result includes the second 3D information of the detection target, the predicted target type and the predicted score; an error information adding module 703, which is used to compare the first 3D information and the second 3D information to determine whether the detection result matches the annotation result. If they do not match, the true target type is added to the true value box, and the predicted target type and the predicted score are added to the detection box.

[0098] According to another embodiment of the present disclosure, the apparatus 70 further includes: a point cloud data drawing module, configured to draw a background and a detection target on the 2D canvas according to the point cloud data.

[0099] According to another embodiment of the present disclosure, the error information adding module is further used to: compare the first 3D information and the second 3D information, determine whether the detection result matches the annotation result, and if so, add the predicted target type and prediction score to the detection box.

[0100] According to another embodiment of the present disclosure, the apparatus 70 further includes: a canvas storage module, configured to store the completed 2D canvas as an image.

[0101] According to another embodiment of the present disclosure, the apparatus 70 further includes: a canvas creation module, configured to create a 2D canvas.

[0102] According to another embodiment of the present disclosure, the point cloud data includes point cloud data acquired by a laser radar, and accordingly, the annotation result also includes first radar coordinate information, and the detection result also includes second radar coordinate information. The device 70 also includes: a horizontal field of view angle boundary drawing module, which is used to draw a horizontal field of view angle boundary on the 2D canvas according to the horizontal field of view angle range of the laser radar.

[0103] According to another embodiment of the present disclosure, the device 70 further includes: a concentric sector drawing module, configured to draw at least two concentric sectors with scales within the horizontal field of view angle boundary with the laser radar as the center of the circle.

[0104] According to another embodiment of the present disclosure, the device 70 further includes: a perception range drawing module, configured to draw an effective perception range rectangular frame on the 2D canvas according to the perception range of the laser radar.

[0105] According to another embodiment of the present disclosure, the device 70 further includes: a label drawing module, configured to add a label on the effective perception range rectangular frame, wherein the label includes a distance and / or a frame number.

[0106] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0107] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0108] like Figure 8 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. Various programs and data required for the operation of the device 800 can also be stored in the RAM 803. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0109] Various components in device 800 are connected to I / O interface 805, including an input unit 806, such as a keyboard, mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, optical disk, etc.; and a communication unit 809, such as a network card, modem, wireless communication transceiver, etc. The communication unit 809 allows device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0110] The computing unit 801 can be a variety of general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the method for visualizing target detection effects of the present disclosure. For example, in some embodiments, the method for visualizing target detection effects of the present disclosure can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the method for visualizing target detection effects of the present disclosure described above can be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to execute the target detection effect visualization method of the present disclosure in any other appropriate manner (for example, by means of firmware).

[0111] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0112] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0113] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0114] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0115] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0116] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0117] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0118] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A method for visualizing target detection effects, comprising: Drawing a 2D rectangular box of a first style on a 2D canvas according to the annotation results of the point cloud data to obtain a ground truth box, wherein the annotation results include the first 3D information of the detected target and the true target type; Drawing a 2D rectangular box of a second style on the 2D canvas according to a detection result of the point cloud data to obtain a detection box, wherein the detection result includes second 3D information of the detected target, a predicted target type, and a predicted score; Comparing the first 3D information with the second 3D information, and comparing the true target type with the predicted target type to determine whether the detection result matches the annotation result; if not, adding the true target type to the ground truth box and adding the predicted target type and prediction score to the detection box; if they match, adding the predicted target type and prediction score to the detection box; If the detection frame and the true value frame do not overlap, and the predicted target type is consistent with the true target type, it indicates that the detection position of the detection target is inaccurate but the detection type is correct; If the detection frame and the true value frame overlap, and the predicted target type is inconsistent with the true target type, it indicates that the detection position of the detection target is accurate but the detection type is inaccurate; If the detection frame and the true value frame overlap, the predicted target type is consistent with the true target type, but the true target type exists on the true value frame, then the height information in the detection result is inconsistent with the height information in the annotation result; If only the detection frame exists but the true value frame does not exist, it indicates that there is a false detection; If only the true value frame exists but the detection frame does not exist, it indicates that there is a missed detection.

2. The method according to claim 1, further comprising: Based on the point cloud data, a background and a detection target are drawn on the 2D canvas.

3. The method according to claim 1, further comprising: The 2D canvas after drawing is stored as an image.

4. The method according to claim 1, further comprising: Create a 2D canvas.

5. The method according to claim 1, wherein the point cloud data comprises point cloud data acquired by a laser radar, and accordingly, The annotation result also includes the first radar coordinate information, The detection result also includes the second radar coordinate information, The method further comprises: Draw a horizontal field of view angle boundary on the 2D canvas according to the horizontal field of view angle range of the laser radar.

6. The method according to claim 5, further comprising: With the laser radar as the center of the circle, at least two concentric sectors with scales are drawn within the boundary of the horizontal field of view angle.

7. The method according to claim 5, further comprising: According to the perception range of the laser radar, an effective perception range rectangular frame is drawn on the 2D canvas.

8. The method according to claim 7, further comprising: A label is added to the effective perception range rectangular frame, where the label includes distance and / or frame number.

9. A device for visualizing target detection effects, comprising: A true value box drawing module is used to draw a 2D rectangular box on a 2D canvas according to the annotation results of the point cloud data to obtain a true value box, wherein the annotation results include the first 3D information of the detected target and the true target type; a detection box drawing module, configured to draw a 2D rectangular box on the 2D canvas to obtain a detection box based on a detection result of the point cloud data, wherein the detection result includes the second 3D information of the detected target, the predicted target type, and the predicted score; an error information adding module, configured to compare the first 3D information and the second 3D information, and to compare the true target type and the predicted target type to determine whether the detection result matches the annotation result; if not, adding the true target type to the ground truth box and adding the predicted target type and prediction score to the detection box; if they do match, adding the predicted target type and prediction score to the detection box; If the detection frame and the true value frame do not overlap, and the predicted target type is consistent with the true target type, it indicates that the detection position of the detection target is inaccurate but the detection type is correct; If the detection frame and the true value frame overlap, and the predicted target type is inconsistent with the true target type, it indicates that the detection position of the detection target is accurate but the detection type is inaccurate; If the detection frame and the true value frame overlap, the predicted target type is consistent with the true target type, but the true target type exists on the true value frame, then the height information in the detection result is inconsistent with the height information in the annotation result; If only the detection frame exists but the true value frame does not exist, it indicates that there is a false detection; If only the true value frame exists but the detection frame does not exist, it indicates that there is a missed detection.

10. The apparatus according to claim 9, further comprising: The point cloud data drawing module is used to draw the background and the detection target on the 2D canvas according to the point cloud data.

11. The apparatus according to claim 9, further comprising: The canvas storage module is used to store the completed 2D canvas as an image.

12. The apparatus according to claim 9, further comprising: Canvas creation module, used to create 2D canvas.

13. The apparatus according to claim 9, wherein the point cloud data comprises point cloud data acquired by a laser radar, and accordingly, the annotation result further comprises first radar coordinate information, and the detection result further comprises second radar coordinate information, and the apparatus further comprises: The horizontal field of view angle boundary drawing module is used to draw the horizontal field of view angle boundary on the 2D canvas according to the horizontal field of view angle range of the laser radar.

14. The apparatus according to claim 13, further comprising: The concentric sector drawing module is used to draw at least two concentric sectors with scales within the horizontal field of view angle boundary with the laser radar as the center of the circle.

15. The apparatus according to claim 13, further comprising: The perception range drawing module is used to draw an effective perception range rectangular frame on the 2D canvas according to the perception range of the laser radar.

16. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

17. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-8.

18. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 8.

Images