Obstacle truncation attribute labeling method and device, equipment and medium

Abstract

The invention relates to an obstacle truncation attribute marking method and device, equipment and a medium, and the method comprises the steps: collecting obstacle information through a radar sensor and an image sensor, and obtaining a detection frame of an obstacle in an image; for a target obstacle in which a detection frame of the obstacle in the image and an image boundary have a truncation relationship, converting the detection frame of the target obstacle to a preset view angle; acquiring a first geometric parameter corresponding to a part, outside an image sensor FOV, of a detection frame of the target obstacle and a second geometric parameter corresponding to the detection frame of the target obstacle at the preset view angle; and generating a truncation attribute of the target obstacle through a ratio between the first geometric parameter and the second geometric parameter, so as to mark the image by using the truncation attribute of the target obstacle. According to the technical scheme of the invention, automatic marking of the obstacle truncation attribute can be realized.

Description

Technical Field

[0001] This disclosure relates to the field of data annotation technology, and in particular to a method, apparatus, device and medium for annotating the truncation attribute of an obstacle. Background Technology

[0002] Camera obstacle truncation refers to the situation where, due to the limited field of view of a camera, obstacles at the edges of the view cannot be fully captured in their original form. This is called truncation. Obstacle truncation attributes are common in publicly available autonomous driving datasets, such as the percentage of an obstacle that is truncated. This attribute is crucial for subsequent autonomous driving decisions and other computational tasks.

[0003] Currently, the main method for obtaining the truncation attributes of obstacles in images is to acquire images by camera and then manually annotate them to set truncation attributes. Annotators estimate the size of the obstacle outside the camera's field of view and manually judge the truncation attributes. This method is inefficient, and the accuracy of manual judgments varies, which directly affects the subsequent model training process and even the model's output accuracy. Therefore, how to efficiently and accurately obtain the truncation attributes of obstacles in images has become an urgent problem to be solved. Summary of the Invention

[0004] To address the aforementioned technical problems, this disclosure provides a method, apparatus, device, and medium for labeling obstacle cutoff attributes.

[0005] In a first aspect, embodiments of this disclosure provide a method for labeling obstacle cutoff attributes, including:

[0006] Obstacle information is collected using radar sensors and image sensors to obtain the detection bounding boxes of obstacles in the image;

[0007] For target obstacles whose detection boxes in the image are truncated from the image boundary, the detection boxes of the target obstacles are switched to a preset viewpoint;

[0008] Under the preset viewpoint, the first geometric parameters corresponding to the portion of the target obstacle's detection box outside the image sensor's field of view and the second geometric parameters corresponding to the target obstacle's detection box are obtained;

[0009] The cutoff attribute of the target obstacle is generated by the ratio between the first geometric parameter and the second geometric parameter.

[0010] Secondly, embodiments of this disclosure provide a device for labeling obstacle cutoff attributes, comprising:

[0011] The detection module is used to collect obstacle information through radar sensors and image sensors to obtain the detection box of the obstacle in the image;

[0012] The processing module is used to convert the detection box of the target obstacle in the image to a preset viewpoint for the target obstacle whose detection box is truncated with the image boundary;

[0013] The acquisition module is used to acquire, under the preset view, the first geometric parameters corresponding to the portion of the detection box of the target obstacle outside the FOV of the image sensor and the second geometric parameters corresponding to the detection box of the target obstacle;

[0014] The annotation module is used to generate the truncation attribute of the target obstacle by using the ratio between the first geometric parameter and the second geometric parameter.

[0015] Thirdly, embodiments of this disclosure provide an electronic device, including: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory and execute the instructions to implement the obstacle cutoff attribute annotation method described in the first aspect above.

[0016] Fourthly, embodiments of this disclosure provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the obstacle cutoff attribute annotation method described in the first aspect.

[0017] Fifthly, embodiments of this disclosure provide a vehicle including an obstacle cutoff attribute marking device as described in the second aspect above, or an electronic device as described in the third aspect above.

[0018] Compared with the prior art, the technical solution provided in this disclosure has the following advantages: obstacle information is collected by radar sensor and image sensor to obtain the detection box of obstacle in the image. For target obstacles whose detection box in the image has a truncation relationship with the image boundary, the first geometric parameter corresponding to the part of the target obstacle detection box outside the FOV of the image sensor and the second geometric parameter corresponding to the target obstacle detection box are obtained under a preset view. The truncation attribute of the target obstacle is generated by the ratio between the first geometric parameter and the second geometric parameter. Thus, the image can be labeled using the truncation attribute of the target obstacle, realizing the automatic labeling of obstacle truncation attribute, improving labeling efficiency, and ensuring the accuracy of obstacle truncation attribute labeling. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0020] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A flowchart illustrating a method for labeling obstacle cutoff attributes provided in an embodiment of this disclosure;

[0022] Figure 2 A schematic diagram of an obstacle cutoff provided in an embodiment of this disclosure;

[0023] Figure 3 This is a schematic diagram of the minimum closure frame of a detection box provided in an embodiment of this disclosure;

[0024] Figure 4 A schematic diagram of a BEV perspective provided in an embodiment of this disclosure;

[0025] Figure 5 This is a schematic diagram of a detection frame provided in an embodiment of the present disclosure;

[0026] Figure 6 A flowchart illustrating another method for labeling obstacle cutoff attributes provided in this embodiment of the present disclosure;

[0027] Figure 7 A schematic diagram of a planar graphic provided in an embodiment of this disclosure;

[0028] Figure 8 A schematic diagram of a line segment provided in an embodiment of this disclosure;

[0029] Figure 9 This is a schematic diagram of the structure of an obstacle interception attribute labeling device provided in an embodiment of this disclosure. Detailed Implementation

[0030] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0031] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0032] Figure 1This is a flowchart illustrating a method for labeling obstacle cutoff attributes according to an embodiment of the present disclosure. The method provided in this embodiment can be executed by an obstacle cutoff attribute labeling device, which can be implemented in software and / or hardware and can be integrated into any electronic device with computing capabilities, such as an in-vehicle terminal.

[0033] like Figure 1 As shown, the method for labeling obstacle cutoff attributes provided in this embodiment of the disclosure may include:

[0034] Step 101: Obstacle information is collected using radar sensors and image sensors to obtain the detection boxes of obstacles in the image.

[0035] Step 102: For target obstacles whose detection boxes in the image are truncated from the image boundary, the detection boxes of the target obstacles are switched to a preset viewpoint.

[0036] In this embodiment, a radar sensor and an image sensor can be installed on the vehicle. The radar sensor is used to detect surrounding obstacles, obtain obstacle information, and generate a 3D detection box for the obstacle. The image sensor can be a camera used to acquire images, which are then used as images to be labeled. The extrinsic and intrinsic parameters of the radar sensor and camera can be pre-calibrated to achieve conversion between the radar coordinate system, camera coordinate system, and image coordinate system. The obstacle information includes position information, attitude information, and size information, for example, using a 7-dimensional representation with the center point on the X-axis, Y-axis, and Z-axis, as well as length, width, height, and yaw angle. The specific form of the detection box can be selected as needed; for example, an eight-point box (i.e., a cuboid with eight vertices) can be used, without specific limitations.

[0037] In this embodiment, after detecting surrounding obstacles using a radar sensor, obtaining obstacle information, and generating 3D obstacle information, the 3D detection bounding box of the obstacle can be projected from the radar coordinate system to the image coordinate system using the calibration parameters of the radar sensor and the image sensor. This results in a 2D detection bounding box of the obstacle in the image coordinate system. This 2D detection bounding box indicates specific information about the obstacle in the image, such as obstacle type and position in the image coordinate system. It should be noted that since the image itself has a certain range, and the range of the image in the image coordinate system is fixed, the 2D detection bounding box of the obstacle in the image coordinate system may not be fully displayed in the image. Obstacles whose 2D detection bounding boxes cannot be fully displayed within the image boundary can be identified as target obstacles with a truncation relationship. In this embodiment, the detection bounding boxes of each obstacle can be converted to a preset viewpoint, or only for target obstacles with a truncation relationship with the image can the detection bounding box of the target obstacle be converted to a preset viewpoint to calculate the truncation attribute of the target obstacle under the preset viewpoint. Among them, the preset viewpoint includes, for example, the camera's forward viewpoint, and the truncation attribute includes the truncation value, which represents the proportion by which the obstacle is truncated by the image.

[0038] Optionally, target obstacles that are truncated from the detected obstacles can be identified for subsequent processing. A schematic diagram illustrating the truncated relationship between obstacles is shown below. Figure 2 As shown, obstacles that do not have a truncation relationship are not processed to reduce the amount of computation.

[0039] There are multiple ways to determine whether an obstacle has a truncation relationship.

[0040] As an example, for an image captured by a camera, if the detection box of an obstacle has vertices outside the camera's FOV (field of view) and vertices inside the camera's FOV, then it is determined that the obstacle has a truncation relationship with the image. In this example, when determining whether the vertices of the detection box are inside the camera's FOV, the detection box of the obstacle can be projected onto the image coordinate system of the image using calibration parameters. Based on the projected detection box, it can be determined whether the obstacle corresponding to the detection box has a truncation relationship with the image. Optionally, it can be determined whether the vertices of the projected detection box are within the image region in the image coordinate system. If the vertices are within the image region, it is determined that the detection box has vertices inside the camera's FOV; if the vertices are outside the image region, it is determined that the detection box has vertices outside the camera's FOV.

[0041] As another example, obstacles that meet the following conditions are identified as target obstacles: 1) In the image coordinate system of the image, the ratio between the intersection area of ​​the detection box and the camera FOV and the area of ​​the minimum closure box is greater than a threshold, or any vertex of the detection box is within the camera FOV; and 2) any vertex of the detection box is in front of the camera; and 3) in the BEV (Bird's Eye View) perspective, the number of vertices of the detection box within the camera FOV is greater than or equal to a preset number; 4) the detection box has vertices outside the camera FOV.

[0042] In this example, for obstacles acquired by the radar sensor, the obstacles are projected onto each viewpoint. The projection process is as follows: the input consists of the center point coordinates, length, width, height, and orientation angle. The input is converted into a detection box. A 4x4 extrinsic matrix is ​​used to transform the detection box from the radar coordinate system to the camera coordinate system. Then, camera intrinsic parameters are used to transform the detection box from the camera coordinate system to the image coordinate system, thereby performing obstacle edge detection. For condition 1), when calculating the area ratio, reference is made to... Figure 3 The dashed lines in the diagram represent the minimum closure of the detection box. The threshold can be set as needed, for example, a threshold of 0.1. For condition 2), to determine whether a vertex is in front of the camera, we can determine whether the vertex's coordinates satisfy the camera coordinate system z>0. For condition 3), refer to... Figure 4 , Figure 4 The diagram illustrates the camera's field of view (FOV) from a BEV perspective. The solid line represents the camera's FOV, which is approximately 120°. The preset number can be set according to the actual scene; for example, for an eight-point bounding box, the preset number can be 2. Therefore, by using the above conditions, target obstacles with a truncation relationship to the image can be identified. For edge cases where the obstacle has only a small overlap with the image and the truncation attribute is not important, these conditions can be used to filter them out, thus accurately determining whether an obstacle has a truncation relationship, reducing the impact of misjudgments on the dataset, and reducing processing workload.

[0043] Step 103: Under a preset viewing angle, obtain the first geometric parameters corresponding to the portion of the target obstacle's detection box outside the image sensor's FOV, and the second geometric parameters corresponding to the target obstacle's detection box.

[0044] The geometric parameters include length, area, and volume.

[0045] As an example, obtaining the first geometric parameters corresponding to the portion of the target obstacle's detection box outside the camera's field of view (FOV) and the second geometric parameters corresponding to the target obstacle's detection box includes: obtaining the first volume of the portion of the target obstacle's detection box outside the camera's FOV and the second volume of the target obstacle's detection box. In this example, taking the camera's forward view as an example, the three-dimensional detection box of the target obstacle is transformed from the radar coordinate system to the camera coordinate system. In the camera coordinate system, the positional relationship between the detection box and the camera's FOV can be determined. Based on this positional relationship, the first volume of the target obstacle's detection box outside the camera's FOV can be determined, and the truncation attribute can be calculated using the first volume and the second volume.

[0046] Reference Figure 5 Based on such Figure 2 The truncation situation shown, Figure 5 A schematic diagram is shown illustrating the acquisition of the first and second volumes of the detection box under this truncation condition.

[0047] Step 104: Generate the cutoff attribute of the target obstacle by using the ratio between the first geometric parameter and the second geometric parameter.

[0048] In this embodiment, after determining the truncation attributes of each target obstacle, the truncation attributes of each target obstacle are labeled in the image accordingly. Optionally, the ratio is positively correlated with the truncation value of the truncation attribute; for example, the ratio can be used as the truncation value of the truncation attribute. This achieves automatic and accurate labeling of obstacle truncation attributes.

[0049] As an example, after generating the truncation attributes of the target obstacle, the target obstacle or its detection box in the image is labeled using these attributes. Optionally, when labeling the truncation attributes, a truncation value can be labeled, or different truncation levels can be labeled as needed. For example, when the ratio = 0, the truncation level is none; when the ratio > 70%, the truncation level is high; when the ratio is between 50% and 70%, the truncation level is medium; when the ratio is between 30% and 50%, the truncation level is low; when the ratio is between 10% and 30%, the truncation level is very low, and so on. The above is just one example, and specific settings can be made according to needs. No restrictions are imposed here.

[0050] According to the technical solution of this disclosure, obstacle information is collected by a radar sensor and an image sensor to obtain the detection box of the obstacle in the image. For target obstacles whose detection box in the image has a truncation relationship with the image boundary, under a preset viewing angle, the first geometric parameter corresponding to the part of the target obstacle's detection box outside the camera's FOV and the second geometric parameter corresponding to the target obstacle's detection box are obtained. The truncation attribute of the target obstacle is generated by the ratio between the first geometric parameter and the second geometric parameter. The image is then labeled using the truncation attribute of the target obstacle. Thus, automatic labeling of obstacle truncation attributes is realized, improving labeling efficiency while ensuring the accuracy of obstacle truncation attribute labeling.

[0051] Based on the above embodiments, Figure 6 This is a flowchart illustrating another method for labeling obstacle cutoff attributes provided in an embodiment of this disclosure, as shown below. Figure 6 As shown, the method includes:

[0052] Step 601: Obstacle information is collected using radar sensors and image sensors to obtain the detection boxes of obstacles in the image.

[0053] Step 602: For target obstacles whose detection boxes in the image are truncated from the image boundary, determine the truncation type of the target obstacle so as to convert the detection boxes of the target obstacle to the viewpoint corresponding to the truncation type.

[0054] In this embodiment, the truncation type of the target obstacle can be further distinguished. For example, the truncation type includes left-right truncation and top-bottom truncation. Left-right truncation indicates that the obstacle has a truncation relationship with the left and right sides of the image, while top-bottom truncation indicates that the obstacle has a truncation relationship with the top and bottom sides of the image. Optionally, the detection box of the target obstacle is projected from the radar coordinate system to the image coordinate system. Based on the positional relationship between each vertex of the target obstacle detection box after projection and the area where the image is located, the truncation type of the target obstacle is determined.

[0055] Optionally, determining the truncation type includes: by projecting the detection box from the radar coordinate system to the image coordinate system of the image, the first vertex of the detection box outside the image area after projection and the position information of the first vertex in the image coordinate system can be obtained; by dividing the non-image area into multiple preset areas, each preset area corresponds to a truncation type, which indicates that if there is a vertex falling in this area, then the corresponding truncation type is determined to exist.

[0056] As an example, if the image region is the area with horizontal coordinates of 0-1800 and vertical coordinates of 0-600 in the image coordinate system, and there are two vertices in the area with horizontal coordinates less than 0 and vertical coordinates of 0-600, and one vertex in the area with vertical coordinates less than 0 and horizontal coordinates of 0-1800, then the truncation type corresponding to the target obstacle is left-right truncation and top-bottom truncation.

[0057] It should be noted that the above specific method for determining the truncation type is only an example. The appropriate method can be selected to determine the truncation type as needed. For example, the truncation type can also be determined by detecting whether the edge of the bounding box intersects with the edge of the image in the image coordinate system. No specific restrictions are imposed here.

[0058] In this embodiment, the preset viewing angles include the BEV view and the camera's forward view. After determining the truncation type of the target obstacle, if the truncation type includes left and right truncation, the target obstacle's detection box is converted to the BEV view; if the truncation type includes top and bottom truncation, the target obstacle's detection box is converted to the camera's forward view. Specifically, if the truncation type is left and right truncation, the target obstacle's 3D detection box is converted to the BEV view; if the truncation type is top and bottom truncation, the target obstacle's 3D detection box is converted to the camera's forward view; if the truncation type includes both left and right truncation and top and bottom truncation, the target obstacle's 3D detection box is converted to both the BEV view and the camera's forward view. In this case, the truncation attributes of the left and right truncations are calculated based on the BEV view, and the truncation attributes of the top and bottom truncations are calculated based on the camera's forward view.

[0059] Step 603: Under the corresponding viewpoint, obtain the first geometric parameters of the part of the target obstacle detection box outside the FOV of the image sensor and the second geometric parameters of the target obstacle detection box.

[0060] In some embodiments, based on the camera's forward view, truncation attributes for left and right truncations can be calculated, as well as truncation attributes for top and bottom truncations. Optionally, for top and bottom truncations, under the camera's forward view, the first geometric parameters corresponding to the portion of the target obstacle's detection box outside the camera's field of view and the second geometric parameters corresponding to the target obstacle's detection box are obtained.

[0061] In some embodiments, based on the BEV perspective, truncation attributes for left and right truncations can be calculated. Optionally, for left and right truncations, in the BEV perspective, the first geometric parameters corresponding to the portion of the target obstacle's detection box outside the camera's FOV and the second geometric parameters corresponding to the target obstacle's detection box are obtained.

[0062] Since the camera's field of view (FOV) is a predicted value, from the BEV perspective, the projection points of the left and right edges of the image can be obtained and projected onto the BEV image. Then, the projection points are connected to the camera's location in the BEV image to obtain the corrected camera FOV, thus obtaining a more accurate camera FOV. Furthermore, when calculating the first geometric parameters, the first geometric parameters corresponding to the portion of the target obstacle's detection box outside the corrected camera FOV are obtained, thus more accurately obtaining the first geometric parameters of the portion of the detection box outside the camera FOV. Therefore, for cases of left and right truncation, the truncation attribute can be calculated more accurately, improving the accuracy of truncation attribute labeling.

[0063] The geometric parameters include length, area, and volume.

[0064] The following explanation combines different geometric parameters and different perspectives.

[0065] In one embodiment of this disclosure, obtaining the first geometric parameters corresponding to the portion of the target obstacle's detection frame outside the camera's field of view (FOV) and the second geometric parameters corresponding to the target obstacle's detection frame includes: projecting the target obstacle's detection frame onto a preset plane to convert it into a planar shape on the preset plane; and obtaining the first area of ​​the planar shape outside the camera's FOV and the second area of ​​the planar shape. For example, see... Figure 7 , Figure 7 The planar figures in the text can be such as Figure 5 The method is obtained by projecting the frame shown. Therefore, compared to calculating the truncation attribute using the first and second volumes, this method is easier to calculate, reduces the computational load, and ensures the accuracy of the truncation attribute.

[0066] As an example, the preset plane corresponds to the viewing angle used. For instance, if the camera acquires images using a forward viewing angle, for left and right truncation, in the BEV view, the detection box of the target obstacle is projected onto the plane where the ground is located, and converted into a planar figure on that plane. Then, the first area of ​​the planar figure outside the corrected camera FOV and the second area of ​​the planar figure are obtained, and the truncation value is calculated using the ratio of the first area and the second area. For up and down truncation, in the forward viewing angle of the camera, the detection box of the target obstacle is projected onto the camera imaging plane, such as the plane where Z=0 in the camera coordinate system, and converted into a planar figure on that plane. Then, the first area of ​​the planar figure outside the camera FOV and the second area of ​​the planar figure are obtained, and the truncation value is calculated using the ratio of the first area and the second area.

[0067] In one embodiment of this disclosure, obtaining the first geometric parameters corresponding to the portion of the target obstacle's detection frame outside the camera's field of view (FOV) and the second geometric parameters corresponding to the target obstacle's detection frame includes: projecting the target obstacle's detection frame onto a preset plane to convert it into a planar shape on the preset plane; converting the planar shape into a line segment on the preset plane along a specified direction; and obtaining the first length and second length of the line segment outside the camera's FOV. Specifically, when converting the planar shape into a line segment on the preset plane along the specified direction, the planar shape can be compressed along the direction of the edge that does not intersect with the camera's FOV to convert it into a line segment on the preset plane, for example, referring to... Figure 8 , Figure 8 The line segments in the text can be like... Figure 7 The image shown is obtained by projecting the planar graphic. This method further reduces the computational load.

[0068] As an example, the preset plane corresponds to the viewing angle used. For instance, if the camera acquires images using a forward viewing angle, for left-right truncation, under the BEV view, the detection box of the target obstacle is projected onto the plane where the ground is located, and converted into a planar graphic on that plane. Then, the planar graphic is converted into a line segment on that plane along a specified direction. The first length of the line segment outside the corrected camera FOV and the second length of the line segment are obtained, and the truncation value is calculated using the ratio of the first length and the second length. For up-down truncation, under the camera's forward viewing angle, the detection box of the target obstacle is projected onto the camera's imaging plane, and converted into a planar graphic on that plane. Then, the planar graphic is converted into a line segment on that plane along a specified direction. The first length of the line segment outside the camera FOV and the second length of the line segment are obtained, and the truncation value is calculated using the ratio of the first length and the second length.

[0069] The method described above for calculating truncation properties using geometric parameters can be selected as needed.

[0070] Step 604: Generate the truncation attribute of the target obstacle by using the ratio between the first geometric parameter and the second geometric parameter, so as to annotate the image using the truncation attribute of the target obstacle.

[0071] In this embodiment, the truncation attribute can be a truncation value, or it can include a truncation value, truncation type, etc. The truncation value can be a floating-point number from 0 (non-truncation) to 1 (truncation) to represent the degree of truncation. After determining the truncation attribute of each target obstacle, the truncation attribute of each target obstacle is labeled in the image accordingly, so as to achieve automatic and accurate labeling of obstacle truncation attributes.

[0072] In this embodiment of the disclosure, by determining the truncation type, for left and right truncations, the truncation attributes are calculated based on the BEV viewpoint. The accuracy of the calculation results is improved by using the corrected camera FOV accurate value. Furthermore, by using the truncation ratio calculation method based on area or length, the calculation process is simplified, the amount of calculation is reduced, and the annotation efficiency is further improved.

[0073] This disclosure also proposes a device for labeling obstacle cutoff attributes.

[0074] Figure 9 This is a schematic diagram of the structure of a device for labeling obstacle cutoff attributes provided in an embodiment of this disclosure, as shown below. Figure 9 As shown, the obstacle truncation attribute labeling device includes: a detection module 91, a processing module 92, an acquisition module 93, and a labeling module 94.

[0075] Among them, the detection module 91 is used to collect obstacle information through radar sensor and image sensor to obtain the detection box of obstacle in image;

[0076] Processing module 92 is used to convert the detection box of the target obstacle in the image to a preset viewpoint for the target obstacle whose detection box is truncated to the image boundary;

[0077] The acquisition module 93 is used to acquire, under the preset view, the first geometric parameters corresponding to the part of the detection frame of the target obstacle outside the camera FOV and the second geometric parameters corresponding to the detection frame of the target obstacle;

[0078] The annotation module 94 is used to generate the truncation attribute of the target obstacle by using the ratio between the first geometric parameter and the second geometric parameter.

[0079] In one embodiment of this disclosure, the processing module 92 is specifically used for:

[0080] The detection bounding box of the target obstacle is projected from the radar coordinate system onto the image coordinate system of the image;

[0081] Based on the positional relationship between each vertex of the detection box of the target obstacle after projection and the region where the image is located, the truncation type of the target obstacle is determined; the truncation type includes left-right truncation and top-bottom truncation;

[0082] If the cutoff type of the target obstacle includes left and right cutoffs, then the detection box of the target obstacle is switched to the BEV view.

[0083] If the truncation type of the target obstacle includes top and bottom truncation, then the detection frame of the target obstacle is switched to the frontal view of the camera.

[0084] In one embodiment of this disclosure, the acquisition module 93 is specifically used for:

[0085] Obtain the projection points of the left and right edges of the image, and project the projection points onto the BEV image;

[0086] In the BEV image, the projection point is connected to the location of the camera to obtain the corrected camera FOV;

[0087] Obtain the first geometric parameters corresponding to the portion of the detection frame of the target obstacle outside the corrected camera FOV.

[0088] In one embodiment of this disclosure, the target obstacle that is truncated in relation to the image captured by the camera is determined by the following steps:

[0089] An obstacle that meets the following conditions is identified as the target obstacle:

[0090] In the image coordinate system of the image, the ratio between the intersection area of ​​the minimum closure box of the detection box and the camera FOV and the area of ​​the minimum closure box is greater than a threshold, or any vertex of the detection box is within the camera FOV.

[0091] Furthermore, any vertex of the detection frame is located in front of the camera;

[0092] Furthermore, from the BEV perspective, the number of vertices within the camera's FOV of the detection box is greater than or equal to a preset number;

[0093] Furthermore, the detection frame has vertices located outside the camera's field of view (FOV).

[0094] In one embodiment of this disclosure, the acquisition module 93 is specifically used for:

[0095] Obtain the first volume of the detection frame of the target obstacle outside the camera's field of view, and the second volume of the detection frame of the target obstacle.

[0096] In one embodiment of this disclosure, the acquisition module 93 is specifically used for:

[0097] By projecting the detection frame of the target obstacle onto a preset plane, the detection frame of the target obstacle is converted into a planar graphic on the preset plane;

[0098] Obtain the first area of ​​the portion of the planar graphic outside the camera's field of view (FOV) and the second area of ​​the planar graphic.

[0099] In one embodiment of this disclosure, the acquisition module 93 is specifically used for:

[0100] By projecting the detection frame of the target obstacle onto a preset plane, the detection frame of the target obstacle is converted into a planar graphic on the preset plane;

[0101] The planar graphic is converted into a line segment on the preset plane along a specified direction, and the first length of the line segment outside the camera's field of view and the second length of the line segment are obtained.

[0102] The obstacle truncation attribute annotation device provided in this disclosure can execute any obstacle truncation attribute annotation method provided in this disclosure, and has the corresponding functional modules and beneficial effects of the method execution. Content not described in detail in the device embodiments of this disclosure can be referred to the description in any method embodiment of this disclosure.

[0103] This disclosure also provides an electronic device, which includes one or more processors and a memory.

[0104] A processor can be a central processing unit (CPU) or other form of processing unit with data processing and / or instruction execution capabilities, and can control other components in an electronic device to perform desired functions.

[0105] The memory may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and a processor may execute the program instructions to implement the methods of the embodiments of this disclosure described above and / or other desired functions. Various contents such as input signals, signal components, and noise components may also be stored in the computer-readable storage medium.

[0106] In one example, the electronic device may further include input and output devices, which are interconnected via a bus system and / or other forms of connection mechanisms. Furthermore, the input device may include, for example, a keyboard, a mouse, etc. The output device can output various information to the outside, including determined distance information, direction information, etc. The output device may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0107] Of course, for simplicity, only some of the components in the electronic device that are relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device may include any other suitable components depending on the specific application.

[0108] In addition to the methods and apparatus described above, embodiments of this disclosure may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform any of the methods provided in the embodiments of this disclosure.

[0109] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of embodiments of this disclosure. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0110] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions that, when executed by a processor, cause the processor to perform any of the methods provided in the embodiments of this disclosure.

[0111] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0112] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0113] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for labeling obstacle truncation attributes, characterized in that, include: Obstacle information is collected using radar sensors and image sensors to obtain the detection bounding boxes of obstacles in the image; For target obstacles whose detection boxes in the image are truncated from the image boundary, the detection boxes of the target obstacles are switched to a preset viewpoint; Under the preset viewpoint, the first geometric parameters corresponding to the portion of the target obstacle's detection box outside the image sensor's field of view and the second geometric parameters corresponding to the target obstacle's detection box are obtained; The cutoff attribute of the target obstacle is generated by the ratio between the first geometric parameter and the second geometric parameter.

2. The method as described in claim 1, characterized in that, The step of switching the detection frame of the target obstacle to a preset viewpoint includes: The detection bounding box of the target obstacle is projected from the radar coordinate system onto the image coordinate system of the image; Based on the positional relationship between each vertex of the detection box of the target obstacle after projection and the region where the image is located, the truncation type of the target obstacle is determined; the truncation type includes left-right truncation and top-bottom truncation; If the cutoff type of the target obstacle includes left and right cutoffs, then the detection box of the target obstacle is switched to the BEV view. If the truncation type of the target obstacle includes top and bottom truncation, then the detection box of the target obstacle is switched to the forward view of the image sensor.

3. The method as described in claim 2, characterized in that, After converting the detection bounding box of the target obstacle to the BEV view, obtaining the first geometric parameters corresponding to the portion of the detection bounding box of the target obstacle outside the FOV of the image sensor includes: Obtain the projection points of the left and right edges of the image, and project the projection points onto the BEV image; In the BEV image, the projection point is connected to the location of the image sensor to obtain the corrected image sensor FOV; Obtain the first geometric parameters of the portion of the detection box of the target obstacle outside the FOV of the corrected image sensor.

4. The method as described in claim 1, characterized in that, The target obstacle with the truncation relationship is identified through the following steps: An obstacle that meets the following conditions is identified as the target obstacle: In the image coordinate system of the image, the ratio between the intersection area of ​​the minimum closure box of the detection box and the FOV of the image sensor and the area of ​​the minimum closure box is greater than a threshold, or any vertex of the detection box is within the FOV of the image sensor. Furthermore, any vertex of the detection frame is located in front of the image sensor; Furthermore, from the BEV perspective, the number of vertices within the FOV of the image sensor in the detection box is greater than or equal to a preset number; Furthermore, the detection frame has vertices located outside the FOV of the image sensor.

5. The method as described in claim 1, characterized in that, The acquisition of the first geometric parameters corresponding to the portion of the detection box of the target obstacle outside the FOV of the image sensor and the second geometric parameters corresponding to the detection box of the target obstacle includes: The first volume of the detection frame of the target obstacle outside the FOV of the image sensor and the second volume of the detection frame of the target obstacle are obtained.

6. The method as described in claim 1, characterized in that, The acquisition of the first geometric parameters corresponding to the portion of the detection box of the target obstacle outside the FOV of the image sensor and the second geometric parameters corresponding to the detection box of the target obstacle includes: By projecting the detection frame of the target obstacle onto a preset plane, the detection frame of the target obstacle is converted into a planar graphic on the preset plane; Obtain a first area of ​​the portion of the planar graphic outside the FOV of the image sensor, and a second area of ​​the planar graphic.

7. The method as described in claim 1, characterized in that, The acquisition of the first geometric parameters corresponding to the portion of the detection box of the target obstacle outside the FOV of the image sensor and the second geometric parameters corresponding to the detection box of the target obstacle includes: By projecting the detection frame of the target obstacle onto a preset plane, the detection frame of the target obstacle is converted into a planar graphic on the preset plane; The planar graphic is converted into a line segment on the preset plane along a specified direction, and the first length of the line segment outside the FOV of the image sensor and the second length of the line segment are obtained.

8. A device for labeling obstacle cutoff attributes, characterized in that, include: The detection module is used to collect obstacle information through radar sensors and image sensors to obtain the detection box of the obstacle in the image; The processing module is used to convert the detection box of the target obstacle in the image to a preset viewpoint for the target obstacle whose detection box is truncated with the image boundary; The acquisition module is used to acquire, under the preset view, the first geometric parameters corresponding to the portion of the detection box of the target obstacle outside the FOV of the image sensor and the second geometric parameters corresponding to the detection box of the target obstacle; The annotation module is used to generate the truncation attribute of the target obstacle by using the ratio between the first geometric parameter and the second geometric parameter.

9. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the obstacle cutoff attribute annotation method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method for labeling obstacle cutoff attributes as described in any one of claims 1-7.

Images