An object recognition method, system, device, and storage medium

Abstract

This invention discloses an object recognition method, comprising the following steps: acquiring a 3D point cloud and a 2D image; recognizing the 3D point cloud to obtain a first category and a first location information for each first object in the 3D point cloud; recognizing the 2D image to obtain a second category and a second location information for each second object in the 2D image; determining a second object corresponding to each first object; and combining the first category, first location information, second category, and second location information of the corresponding first and second objects to obtain the final category and location information of the corresponding first and second objects. This invention also discloses a system, a computer device, and a readable storage medium. The proposed solution combines the results of 3D and 2D recognition to obtain the final category and location information of objects, fully leveraging the advantages of different sensors and improving the robustness and accuracy of the algorithm.

Description

Technical Field

[0001] This invention relates to the field of vehicle-road cooperation, and specifically to an object recognition method, system, device, and storage medium. Background Technology

[0002] When using the vehicle's 2D sensors to identify objects, orientation angle information cannot be obtained. If another detection network is used to obtain the orientation angle, on the one hand, the coupling relationship between different tasks cannot be fully utilized, and on the other hand, the algorithm's running speed cannot be guaranteed when multiple independent tasks are stacked for detection.

[0003] When using roadside 3D sensors to identify objects, false detections and missed detections are likely to occur for occluded objects, distant and small objects, and objects with unclear laser point cloud textures. Therefore, relying solely on vehicle-side or roadside sensors for object identification is unreliable. Summary of the Invention

[0004] In view of this, in order to overcome at least one aspect of the above problems, embodiments of the present invention propose an object recognition method, comprising the following steps:

[0005] Acquire 3D point clouds and 2D images;

[0006] The three-dimensional point cloud is identified to obtain the first category and first position information of each first object in the three-dimensional point cloud, and the two-dimensional image is identified to obtain the second category and second position information of each second object in the two-dimensional image;

[0007] Determine the second object corresponding to each of the first objects;

[0008] The final category and location information of the first and second objects with corresponding relationships are obtained by combining the first category, first location information, second category, and second location information of the first and second objects with corresponding relationships.

[0009] In some embodiments, identifying the three-dimensional point cloud to obtain first category and first location information for each first object in the three-dimensional point cloud further includes:

[0010] The three-dimensional point cloud is detected using a first detection algorithm to obtain multiple three-dimensional bounding boxes, wherein each three-dimensional bounding box is assigned a first category label, a confidence level of the first category label, and a cosine angle of the orientation angle by the first detection algorithm;

[0011] Based on the three-dimensional bounding box, determine the first category, the confidence level of the first category, and the cosine angle of the orientation angle for each first object.

[0012] In some embodiments, identifying the two-dimensional image to obtain second category and second location information for each second object in the two-dimensional image further includes:

[0013] The second detection algorithm is used to detect the two-dimensional image to obtain multiple two-dimensional bounding boxes, wherein each two-dimensional bounding box is assigned a second category label, a confidence level of the second category label, and an orientation by the second detection algorithm;

[0014] The second category, confidence level, and orientation of each second object are determined based on the two-dimensional bounding box.

[0015] In some embodiments, the final category and location information of the corresponding first object and second object are obtained by combining the first category, first location information, second category, and second location information of the corresponding first object and second object, further including:

[0016] Compare the confidence scores of the first category and the second category labels of the first and second objects that have a corresponding relationship;

[0017] In response to the fact that the confidence level of the first category is greater than that of the second category, the first category of the first object is taken as the final category of the first object and the second object with the corresponding relationship;

[0018] In response to the fact that the confidence level of the first category is less than that of the second category, the second category of the second object is taken as the final category of the first and second objects that have a corresponding relationship.

[0019] In some embodiments, it also includes:

[0020] By combining the cosine angle of the corresponding orientation angle and the corresponding orientation, the final position information of the first and second objects with the corresponding relationship is obtained.

[0021] In some embodiments, the second detection algorithm is used to detect multiple two-dimensional bounding boxes in the two-dimensional image, further including:

[0022] A loss function is constructed based on the regression loss of the two-dimensional bounding box, the loss of the second category, the positive and negative sample losses of the two-dimensional bounding box, and the orientation loss.

[0023] In some embodiments, a loss function is constructed based on the regression loss of the two-dimensional bounding box, the loss of the second category, the positive and negative sample losses of the two-dimensional bounding box, and the orientation loss, further including:

[0024] Loss = L box +L cls +M1*Lobj +L orien As the loss function of the second detection algorithm, L box L is the regression loss for the two-dimensional bounding box. cls For the loss of the second category, L obj L represents the positive and negative sample loss of the two-dimensional bounding box. orien For the orientation loss, where:

[0025]

[0026]

[0027]

[0028]

[0029] Where M1, M2, M3, M4, and M5 are weights; S 2 S represents a grid of S cells; B represents that each grid cell generates B two-dimensional bounding boxes. This indicates that if there is a target in the 2D bounding box at positions i and j, the value is 1; otherwise, it is 0. This indicates that if the 2D bounding box at positions i and j has no target, its value is 1; otherwise, it is 0. For GIOU loss function, This is the binary cross-entropy loss function.

[0030] In some embodiments, it also includes:

[0031] The confidence level of the second category is calculated using the Logits loss function.

[0032] In some embodiments, acquiring a 3D point cloud and a 2D image further includes:

[0033] The three-dimensional point cloud is obtained using radar scanning at the roadside;

[0034] The two-dimensional image is obtained by using a two-dimensional camera on the vehicle or a two-dimensional camera on the road.

[0035] In some embodiments, determining the second object corresponding to each of the first objects further includes:

[0036] The first object is projected onto the two-dimensional image using a transformation matrix;

[0037] The second object at the projection position of the two-dimensional image is taken as the corresponding second object.

[0038] In some embodiments, identifying the two-dimensional image to obtain second category and second location information for each second object in the two-dimensional image further includes:

[0039] The two-dimensional images are filtered by combining multiple images to find those where the positional information of the second object changes abruptly.

[0040] Based on the same inventive concept, according to another aspect of the present invention, embodiments of the present invention also provide an object recognition system, comprising:

[0041] The acquisition module is configured to acquire 3D point clouds and 2D images;

[0042] The recognition module is configured to recognize the three-dimensional point cloud to obtain the first category and first position information of each first object in the three-dimensional point cloud, and to recognize the two-dimensional image to obtain the second category and second position information of each second object in the two-dimensional image;

[0043] The projection module is configured to determine the second object corresponding to each of the first objects;

[0044] The correction module is configured to combine the first category, first location information, second category, and second location information of the first and second objects that have a corresponding relationship to obtain the final category and location information of the first and second objects that have a corresponding relationship.

[0045] Based on the same inventive concept, according to another aspect of the present invention, embodiments of the present invention also provide a computer device, comprising:

[0046] At least one processor; and

[0047] A memory storing a computer program that can run on the processor, which, when executing the program, performs the steps of any of the object recognition methods described above.

[0048] Based on the same inventive concept, according to another aspect of the present invention, embodiments of the present invention also provide a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of any of the object recognition methods described above.

[0049] The present invention has one of the following beneficial technical effects: the proposed solution obtains the final category and location information of the object by combining the results of three-dimensional recognition and two-dimensional recognition, giving full play to the advantages of different sensors and improving the robustness and accuracy of the algorithm. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings without creative effort.

[0051] Figure 1 A flowchart illustrating the object recognition method provided in an embodiment of the present invention;

[0052] Figure 2 A schematic diagram of vehicle orientation provided for an embodiment of the present invention;

[0053] Figure 3 A schematic diagram of the structure of an object recognition system provided for an embodiment of the present invention;

[0054] Figure 4 A schematic diagram of the structure of a computer device provided for an embodiment of the present invention;

[0055] Figure 5 A schematic diagram of the structure of a computer-readable storage medium provided for an embodiment of the present invention. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to specific examples and the accompanying drawings.

[0057] It should be noted that all uses of "first" and "second" in the embodiments of the present invention are for the purpose of distinguishing two entities or parameters with the same name but different names. It is clear that "first" and "second" are only for the convenience of expression and should not be construed as limiting the embodiments of the present invention. Subsequent embodiments will not explain this in detail.

[0058] According to one aspect of the present invention, embodiments of the present invention provide an object recognition method, such as... Figure 1 As shown, it may include the following steps:

[0059] S1, acquire 3D point cloud and 2D image;

[0060] S2, identify the three-dimensional point cloud to obtain the first category and first position information of each first object in the three-dimensional point cloud, and identify the two-dimensional image to obtain the second category and second position information of each second object in the two-dimensional image;

[0061] S3, determine the second object corresponding to each of the first objects;

[0062] S4, combine the first category, first location information, second category and second location information of the first object and the second object with corresponding relationship to obtain the final category and location information of the first object and the second object with corresponding relationship.

[0063] The proposed solution combines the results of 3D and 2D recognition to obtain the final category and location information of the object, giving full play to the advantages of different sensors and improving the robustness and accuracy of the algorithm.

[0064] In some embodiments, S2, identifying the three-dimensional point cloud to obtain the first category and first location information of each first object in the three-dimensional point cloud, further includes:

[0065] S211, the three-dimensional point cloud is detected using the first detection algorithm to obtain multiple three-dimensional bounding boxes, wherein each three-dimensional bounding box is assigned a first category label, a confidence level of the first category label, and a cosine angle of the orientation angle by the first detection algorithm;

[0066] S212, based on the three-dimensional bounding box, determine the first category, the confidence level of the first category, and the cosine angle of the orientation angle for each first object.

[0067] Specifically, the first detection algorithm can be a 3D point cloud detection algorithm, such as CenterPoint, PointPillars, voxelnet, SECOND, pointRCNN, SMOKE, FCOS3D, etc. After detecting the 3D point cloud using the 3D point cloud detection algorithm, a 3D bounding box can be obtained. Each 3D bounding box is assigned a first category label, a confidence level for the first category label, and orientation angle information. The orientation angle is continuous angular numerical information, represented by the cosine angle function cosθ, where θ∈[0,2π), i.e., cosθ∈[-1,1]. This determines the first category, the confidence level of the first category, and the cosine angle of the orientation angle for each first object in the 3D point cloud. In some embodiments, the first object can be a pedestrian, vehicle, or cyclist.

[0068] In some embodiments, S2, identifying the two-dimensional image to obtain second category and second location information for each second object in the two-dimensional image, further includes:

[0069] S221, the second detection algorithm is used to detect the two-dimensional image to obtain multiple two-dimensional bounding boxes, wherein each two-dimensional bounding box is assigned a second category label, a confidence level of the second category label, and an orientation by the second detection algorithm;

[0070] S222, determine the second category, confidence level and orientation of each second object based on the two-dimensional bounding box.

[0071] Specifically, the second detection algorithm can be a two-dimensional image detection algorithm, such as YOLO, fcos, RetinaNet, SSD, Faster R-CNN, DETR, etc. After detecting the two-dimensional image using the two-dimensional image detection algorithm, two-dimensional bounding boxes can be obtained. Each two-dimensional bounding box is assigned a second class label, a second class label confidence score, and orientation information, including the four cardinal directions (north, south, east, and west). This determines the second class, the confidence score, and the orientation of each second object in the two-dimensional image. In some embodiments, the second object can be a pedestrian, vehicle, or cyclist.

[0072] The three tasks (2dbox, class, and orientation) are connected in the last layer in the form of a multi-task head. The 2dbox loss uses GIOU loss, the class probability and orientation probability use binary cross-entropy, the target score confidence uses the Logits loss function, and the fl_gamma parameter is used to activate the Focal loss to calculate the loss function.

[0073] In some embodiments, the orientation of the second object can take values ​​from [0, 1, 2, 3]:

[0074] 0 represents an object facing the camera and including all angles, such as southeast and southwest.

[0075] 1 represents an object facing away from the camera and including all included angles, such as northeast and northwest.

[0076] 2 represents an object that is completely horizontal on the road and is facing due left;

[0077] 3 represents an object that is completely horizontal on the road and is facing directly to the right.

[0078] For example, such as Figure 2 The vehicle shown can take values ​​from [0, 1, 2, 3] for its front facing. 0 represents the vehicle facing the camera and including all angles, such as southeast and southwest. 1 represents the vehicle facing away from the camera and including all angles, such as northeast and northwest. 2 represents the vehicle is completely sideways on the road and facing due left. 3 represents the vehicle is completely sideways on the road and facing due right.

[0079] In some embodiments, the second detection algorithm is used to detect multiple two-dimensional bounding boxes in the two-dimensional image, further including:

[0080] A loss function is constructed based on the regression loss of the two-dimensional bounding box, the loss of the second category, the positive and negative sample losses of the two-dimensional bounding box, and the orientation loss.

[0081] In some embodiments, a loss function is constructed based on the regression loss of the two-dimensional bounding box, the loss of the second category, the positive and negative sample losses of the two-dimensional bounding box, and the orientation loss, further including:

[0082] Loss = L box +L cls +M1*L obj +L orien As the loss function of the second detection algorithm, L box L is the regression loss for the two-dimensional bounding box. cls For the loss of the second category, L obj L represents the positive and negative sample loss of the two-dimensional bounding box. orien For the orientation loss, where:

[0083]

[0084]

[0085]

[0086]

[0087] Where M1, M2, M3, M4, and M5 are weights; S 2 S represents a grid of S cells; B represents that each grid cell generates B two-dimensional bounding boxes. This indicates that if there is a target in the 2D bounding box at positions i and j, the value is 1; otherwise, it is 0. This indicates that if the 2D bounding box at positions i and j has no target, its value is 1; otherwise, it is 0. For GIOU loss function, This is the binary cross-entropy loss function.

[0088] In some embodiments, it also includes:

[0089] The confidence level of the second category is calculated using the Logits loss function.

[0090] In some embodiments, M1, M2, M3, M4, and M5 can be adjusted according to the training results of the second detection algorithm.

[0091] In some embodiments, S4, combining the first category, first location information, second category, and second location information of the corresponding first object and second object to obtain the final category and location information of the corresponding first object and second object, further includes:

[0092] S41, compare the confidence scores of the first category and the second category labels of the first and second objects that have a corresponding relationship;

[0093] S42, in response to the fact that the confidence level of the first category is greater than the confidence level of the second category, the first category of the first object is taken as the final category of the first object and the second object with the corresponding relationship;

[0094] S43, in response to the fact that the confidence level of the first category is less than the confidence level of the second category, the second category of the second object is taken as the final category of the first object and the second object with the corresponding relationship.

[0095] Specifically, the confidence levels of the categories obtained from 3D and 2D recognition can be compared, and the category with the higher confidence level can be used as the final category. For example, if the category of the first object obtained from 3D recognition is vehicle with a confidence level of A, and the category of the second object corresponding to the first object obtained from 2D recognition is pedestrian with a confidence level of B, then if A is greater than B, the final category of the first and second objects with the corresponding relationship is vehicle; if A is less than B, the final category of the first and second objects with the corresponding relationship is pedestrian.

[0096] It should be noted that the confidence level is represented using float32, which has high precision. Therefore, the confidence levels of the first object and the second object will generally not be equal.

[0097] In some embodiments, it also includes:

[0098] S44, combining the cosine angle of the corresponding orientation angle and the corresponding orientation, the final position information of the first and second objects with the corresponding relationship is obtained.

[0099] Specifically, the cosine angle of the orientation obtained by 3D recognition is prone to fluctuation in the left-right or north-south positive and negative directions (for example, the values ​​of cos90° and cos270° are the same, but the orientation cannot be determined). The orientation result of 2D recognition is four discrete orientation information, which is more stable. Therefore, the final position information can be obtained by combining the results of 3D recognition and 2D recognition.

[0100] In some embodiments, S1, acquiring a 3D point cloud and a 2D image, further includes:

[0101] S11, the three-dimensional point cloud is obtained by radar scanning at the road end;

[0102] S12, the two-dimensional image is obtained by using a two-dimensional camera at the vehicle end or a two-dimensional camera at the road end.

[0103] Specifically, the roadside sensor is equipped with a 3D LiDAR and at least one 2D camera, with the pole erected at the edge of the road. The vehicle-mounted sensor can also be equipped with at least one ordinary 2D camera, and optionally a 3D LiDAR. The 3D point cloud data is then obtained by scanning with the roadside LiDAR, and the 2D image is obtained by capturing images with the vehicle-mounted or roadside camera.

[0104] In some embodiments, S3, determining the second object corresponding to each of the first objects further includes:

[0105] S31, the first object is projected onto the two-dimensional image using a transformation matrix;

[0106] S32, the second object at the projection position of the two-dimensional image is taken as the corresponding second object.

[0107] Specifically, the first object is projected onto the two-dimensional image using the calibrated transformation matrix Mt (radar extrinsic parameters): [u, v, 1] = Mt * [x, y, z, 1]

[0108] Where u and v represent coordinates on the two-dimensional image, and x, y, and z represent coordinates in the three-dimensional point cloud.

[0109] In some embodiments, S2, identifying the two-dimensional image to obtain second category and second location information for each second object in the two-dimensional image, further includes:

[0110] S220, filter the two-dimensional images in combination with the multiple two-dimensional images to find those where the position information of the second object changes abruptly.

[0111] Specifically, by combining N images and filtering out frames with obvious abrupt changes in orientation, the jitter problem of orientation information can be effectively solved. For example, if the orientation of an object in the N+1 images changes significantly compared to the orientation of the same object in the previous N images, these frames can be filtered out.

[0112] The proposed solution utilizes roadside sensors as the primary source, leveraging the unique advantages of vehicle-to-infrastructure (V2I) communication. This saves on the equipment cost of vehicle-mounted sensors (mainly LiDAR). Furthermore, using roadside LiDAR sensors reduces instability factors such as sensor data jitter and ghosting at high speeds, improving the accuracy and stability of detection and recognition, thereby ensuring driving safety. In addition, by combining the complementary characteristics of 2D and 3D data, and integrating 3D and 2D target results with orientation information, the corresponding positions are found through a projection matrix, improving the robustness and accuracy of the algorithm and enhancing driving safety.

[0113] Based on the same inventive concept, according to another aspect of the present invention, embodiments of the present invention also provide an object recognition system 400, such as... Figure 3 As shown, it includes:

[0114] The acquisition module 401 is configured to acquire 3D point clouds and 2D images;

[0115] The recognition module 402 is configured to recognize the three-dimensional point cloud to obtain the first category and first position information of each first object in the three-dimensional point cloud, and to recognize the two-dimensional image to obtain the second category and second position information of each second object in the two-dimensional image.

[0116] Projection module 403 is configured to determine the second object corresponding to each of the first objects;

[0117] The correction module 404 is configured to combine the first category, first location information, second category and second location information of the first object and the second object that have a corresponding relationship to obtain the final category and location information of the first object and the second object that have a corresponding relationship.

[0118] In some embodiments, the identification module 402 is further configured to:

[0119] The three-dimensional point cloud is detected using a first detection algorithm to obtain multiple three-dimensional bounding boxes, wherein each three-dimensional bounding box is assigned a first category label, a confidence level of the first category label, and a cosine angle of the orientation angle by the first detection algorithm;

[0120] Based on the three-dimensional bounding box, determine the first category, the confidence level of the first category, and the cosine angle of the orientation angle for each first object.

[0121] In some embodiments, the identification module 402 is further configured to:

[0122] The second detection algorithm is used to detect the two-dimensional image to obtain multiple two-dimensional bounding boxes, wherein each two-dimensional bounding box is assigned a second category label, a confidence level of the second category label, and an orientation by the second detection algorithm;

[0123] The second category, confidence level, and orientation of each second object are determined based on the two-dimensional bounding box.

[0124] In some embodiments, the correction module 404 is further configured to:

[0125] Compare the confidence scores of the first category and the second category labels of the first and second objects that have a corresponding relationship;

[0126] In response to the fact that the confidence level of the first category is greater than that of the second category, the first category of the first object is taken as the final category of the first object and the second object with the corresponding relationship;

[0127] In response to the fact that the confidence level of the first category is less than that of the second category, the second category of the second object is taken as the final category of the first and second objects that have a corresponding relationship.

[0128] In some embodiments, the correction module 404 is further configured to:

[0129] By combining the cosine angle of the corresponding orientation angle and the corresponding orientation, the final position information of the first and second objects with the corresponding relationship is obtained.

[0130] In some embodiments, the identification module 402 is further configured to:

[0131] Loss = L box +L cls +M1*L obj +L orien As the loss function of the second detection algorithm, where:

[0132]

[0133]

[0134]

[0135]

[0136] Where M1, M2, M3, M4, and M5 are weights; S 2 S represents a grid of S cells; B represents that each grid cell generates B two-dimensional bounding boxes. This indicates that if there is a target in the 2D bounding box at positions i and j, the value is 1; otherwise, it is 0. This indicates that if the 2D bounding box at positions i and j has no target, its value is 1; otherwise, it is 0. For GIOU loss function, This is the binary cross-entropy loss function.

[0137] In some embodiments, the identification module 402 is further configured to:

[0138] The confidence level of the second category is calculated using the Logits loss function.

[0139] In some embodiments, the acquisition module 401 is further configured to:

[0140] The three-dimensional point cloud is obtained using radar scanning at the roadside;

[0141] The two-dimensional image is obtained by using a two-dimensional camera on the vehicle or a two-dimensional camera on the road.

[0142] In some embodiments, the projection module 403 is further configured to:

[0143] The first object is projected onto the two-dimensional image using a transformation matrix;

[0144] The second object at the projection position of the two-dimensional image is taken as the corresponding second object.

[0145] In some embodiments, the identification module 402 is further configured to:

[0146] The two-dimensional images are filtered by combining multiple images to find those where the positional information of the second object changes abruptly.

[0147] The proposed solution utilizes roadside sensors as the primary source, leveraging the unique advantages of vehicle-to-infrastructure (V2I) communication. This saves on the equipment cost of vehicle-mounted sensors (mainly LiDAR). Furthermore, using roadside LiDAR sensors reduces instability factors such as sensor data jitter and ghosting at high speeds, improving the accuracy and stability of detection and recognition, thereby ensuring driving safety. In addition, by combining the complementary characteristics of 2D and 3D data, and integrating 3D and 2D target results with orientation information, the corresponding positions are found through a projection matrix, improving the robustness and accuracy of the algorithm and enhancing driving safety.

[0148] Based on the same inventive concept, according to another aspect of the present invention, such as Figure 4 As shown, an embodiment of the present invention also provides a computer device 501, comprising:

[0149] At least one processor 520; and

[0150] The memory 510 stores a computer program 511 that can run on a processor. When the processor 520 executes the program, it performs the steps of any of the object recognition methods described above.

[0151] Based on the same inventive concept, according to another aspect of the present invention, such as Figure 5 As shown, embodiments of the present invention also provide a computer-readable storage medium 601, which stores a computer program 610. When the computer program 610 is executed by a processor, it performs the steps of any of the object recognition methods described above.

[0152] Finally, it should be noted that those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods.

[0153] Furthermore, it should be understood that the computer-readable storage medium (e.g., memory) described herein may be volatile memory or non-volatile memory, or may include both volatile memory and non-volatile memory.

[0154] Those skilled in the art will also understand that the various exemplary logic blocks, modules, circuits, and algorithm steps described in conjunction with the disclosure herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, the functionality of various illustrative components, blocks, modules, circuits, and steps has been generally described. Whether this functionality is implemented as software or as hardware depends on the specific application and the design constraints imposed on the system as a whole. Those skilled in the art can implement the functionality in various ways for each specific application, but such implementation decisions should not be construed as departing from the scope of the embodiments disclosed herein.

[0155] The above are exemplary embodiments disclosed in this invention. However, it should be noted that various changes and modifications can be made without departing from the scope of the embodiments of this invention as defined by the claims. The functions, steps, and / or actions of the methods according to the disclosed embodiments described herein do not need to be performed in any particular order. Furthermore, although the elements disclosed in the embodiments of this invention may be described or claimed individually, they may be understood as multiple unless explicitly limited to a singular number.

[0156] It should be understood that, as used herein, the singular form “a” is intended to include the plural form as well, unless the context clearly supports an exception. It should also be understood that, as used herein, “and / or” refers to any and all possible combinations of one or more of the associated listed items.

[0157] The embodiment numbers disclosed in the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0158] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0159] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of the invention (including the claims) is limited to these examples. Within the framework of the invention, technical features of the above embodiments or different embodiments can be combined, and many other variations of different aspects of the invention exist, which are not provided in the details for the sake of brevity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the protection scope of the invention.

Claims

1. An object recognition method, characterized in that, Includes the following steps: Acquire 3D point clouds and 2D images; The three-dimensional point cloud is identified to obtain the first category and first position information of each first object in the three-dimensional point cloud, and the two-dimensional image is identified to obtain the second category and second position information of each second object in the two-dimensional image, wherein the first position information is the cosine angle of the orientation angle, and the second position information is discrete orientation information. Determine the second object corresponding to each of the first objects; By combining the first category, first location information, second category, and second location information of the corresponding first and second objects, the final category and location information of the corresponding first and second objects are obtained. The final category and location information of the corresponding first and second objects are obtained by combining the first category, first location information, second category, and second location information of the first and second objects, further including: Compare the confidence scores of the first category and the second category labels of the first and second objects that have a corresponding relationship; In response to the fact that the confidence level of the first category is greater than that of the second category, the first category of the first object is taken as the final category of the first object and the second object with the corresponding relationship; In response to the fact that the confidence level of the first category is lower than that of the second category, the second category of the second object is taken as the final category of the first object and the second object with the corresponding relationship; By combining the cosine angle of the corresponding orientation angle and discrete orientation information, the final position information of the first and second objects with the corresponding relationship is obtained.

2. The method as described in claim 1, characterized in that, The process of identifying the 3D point cloud to obtain the first category and first location information of each first object in the 3D point cloud further includes: The three-dimensional point cloud is detected using a first detection algorithm to obtain multiple three-dimensional bounding boxes, wherein each three-dimensional bounding box is assigned a first category label, a confidence level of the first category label, and a cosine angle of the orientation angle by the first detection algorithm; Based on the three-dimensional bounding box, determine the first category, the confidence level of the first category, and the cosine angle of the orientation angle for each first object.

3. The method as described in claim 2, characterized in that, The process of identifying the two-dimensional image to obtain the second category and second location information of each second object in the two-dimensional image further includes: The second detection algorithm is used to detect the two-dimensional image to obtain multiple two-dimensional bounding boxes, wherein each two-dimensional bounding box is assigned a second category label, a confidence level of the second category label, and an orientation by the second detection algorithm; The second category, confidence level, and orientation of each second object are determined based on the two-dimensional bounding box.

4. The method as described in claim 3, characterized in that, The second detection algorithm is used to detect multiple two-dimensional bounding boxes in the two-dimensional image, further including: A loss function is constructed based on the regression loss of the two-dimensional bounding box, the loss of the second category, the positive and negative sample losses of the two-dimensional bounding box, and the orientation loss.

5. The method as described in claim 4, characterized in that, A loss function is constructed based on the regression loss of the two-dimensional bounding box, the loss of the second category, the positive and negative sample losses of the two-dimensional bounding box, and the orientation loss, further including: Will As the loss function of the second detection algorithm, L box The regression loss is the value of the two-dimensional bounding box. L cls For the second category of loss, L obj The positive and negative sample losses of the two-dimensional bounding box are... L orien For the orientation loss, where: ； ； ； ； in, It is the weight; S 2 S represents a grid of S cells; B represents that each grid cell generates B two-dimensional bounding boxes. Indicates if in i, j If the 2D bounding box at a location contains a target, its value is 1; otherwise, it is 0. Indicates if in i, j The 2D bounding box at a location has no target, so its value is 1; otherwise, it is 0. For GIOU loss function, , , This is the binary cross-entropy loss function.

6. The method as described in claim 4, characterized in that, Also includes: The confidence level of the second category is calculated using the Logits loss function.

7. The method as described in claim 1, characterized in that, Acquiring 3D point clouds and 2D images, further including: The three-dimensional point cloud is obtained using radar scanning at the roadside; The two-dimensional image is obtained by using a two-dimensional camera on the vehicle or a two-dimensional camera on the road.

8. The method as described in claim 1, characterized in that, Determining the second object corresponding to each of the first objects further includes: The first object is projected onto the two-dimensional image using a transformation matrix; The second object at the projection position of the two-dimensional image is taken as the corresponding second object.

9. The method as described in claim 1, characterized in that, The process of identifying the two-dimensional image to obtain the second category and second location information of each second object in the two-dimensional image further includes: The two-dimensional images are filtered by combining multiple images to find those where the positional information of the second object changes abruptly.

10. An object recognition system, characterized in that, include: The acquisition module is configured to acquire 3D point clouds and 2D images; The recognition module is configured to recognize the three-dimensional point cloud to obtain the first category and first position information of each first object in the three-dimensional point cloud, and to recognize the two-dimensional image to obtain the second category and second position information of each second object in the two-dimensional image, wherein the first position information is the cosine angle of the orientation angle, and the second position information is discrete orientation information. The projection module is configured to determine the second object corresponding to each of the first objects; The correction module is configured to combine the first category, first location information, second category, and second location information of the corresponding first and second objects to obtain the final category and location information of the corresponding first and second objects; The correction module is also configured as follows: Compare the confidence scores of the first category and the second category labels of the first and second objects that have a corresponding relationship; In response to the fact that the confidence level of the first category is greater than that of the second category, the first category of the first object is taken as the final category of the first object and the second object with the corresponding relationship; In response to the fact that the confidence level of the first category is lower than that of the second category, the second category of the second object is taken as the final category of the first object and the second object with the corresponding relationship; By combining the cosine angle of the corresponding orientation angle and discrete orientation information, the final position information of the first and second objects with the corresponding relationship is obtained.

11. A computer device, comprising: At least one processor; as well as A memory storing a computer program executable on the processor, characterized in that the processor executes the program by performing the steps of the method as described in any one of claims 1-9.

12. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it performs the steps of the method as described in any one of claims 1-9.

Images