Enhancement method and device for camera data detection result, computer device and storage medium

Abstract

The application relates to a camera data detection result enhancement method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring camera data of a detection area; inputting the camera data into a preset detection model to obtain an enhanced detection result of the camera data; wherein the enhanced detection result of the camera data is a result comprising three-dimensional physical information; the detection model is obtained by mapping first sample point cloud data to first sample camera data, correlatively matching the mapping result of the first sample point cloud data and the detection result of the first sample camera data according to the similarity between the mapping result of the first sample point cloud data and the detection result of the first sample camera data, and training a preset initial detection model according to the correlatively matched result; the first sample camera data and the first sample point cloud data are data of the same detection area. The method can improve the accuracy of the obtained enhanced detection result of the camera data.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target detection, in particular to a camera data detection result enhancement method and device, computer equipment and a storage medium. BACKGROUND

[0002] A camera, short for camera, is a device that forms an image using optical imaging principles and records the image using a film. The light reflected by the object to be photographed is focused through the camera lens and shutter to control the exposure, and the object forms a latent image on the photosensitive material in the dark box, which becomes a permanent image after processing.

[0003] However, due to the characteristics of camera data, when the object to be photographed is blocked, in the case of night light or weather influence, there will be problems such as camera shooting blind area and camera data distortion; or when the object to be photographed is too far from the camera, the pixel points of the object to be photographed in the camera data are relatively small, and thus the object to be photographed cannot be detected in the camera data.

[0004] Therefore, the traditional camera data detection method has the problem of low detection accuracy. SUMMARY

[0005] Therefore, it is necessary to provide a camera data detection result enhancement method, device, computer equipment and storage medium capable of improving the camera data detection accuracy in view of the above technical problems.

[0006] A camera data detection result enhancement method, the method comprising:

[0007] Obtaining camera data of a detection area;

[0008] Inputting the camera data into a preset detection model to obtain an enhanced detection result of the camera data; wherein the enhanced detection result of the camera data is a result including three-dimensional physical information; the detection model is obtained by mapping first sample point cloud data to first sample camera data, associating and matching the mapping result of the first sample point cloud data and the detection result of the first sample camera data according to the similarity between the mapping result of the first sample point cloud data and the detection result of the first sample camera data, and training a preset initial detection model according to the association and matching result; the first sample camera data and the first sample point cloud data are data of the same detection area.

[0009] In one embodiment, the training method of the detection model comprises:

[0010] According to a pre-established mapping relationship, mapping the detection result of the first sample point cloud data to the first sample camera data to obtain a mapping result of the detection result of the first sample point cloud data.

[0011] calculate a first similarity between the mapping result of the detection result of the first sample point cloud data and the detection result of the first sample camera data;

[0012] according to the first similarity, the mapping result of the detection result of the first sample point cloud data and the detection result of the first sample camera data are associated matched;

[0013] according to the associated matching result, the first sample camera data is labeled to obtain the label of the first sample camera data;

[0014] the initial detection model is trained according to the first sample camera data and the label of the first sample camera data, and the detection model is obtained.

[0015] In one of the embodiments, the training method of the detection model comprises:

[0016] according to the pre-established mapping relationship, the first sample camera data is mapped into the point cloud data coordinate system to obtain the mapping result of the first sample camera data;

[0017] calculate a second similarity between the mapping result of the first sample camera data and the detection result of the first sample point cloud data;

[0018] according to the second similarity, the detection result of the first sample point cloud data and the mapping result of the first sample camera data are associated matched;

[0019] according to the associated matching result, the first sample camera data is labeled to obtain the label of the first sample camera data;

[0020] the initial detection model is trained according to the first sample camera data and the label of the first sample camera data, and the detection model is obtained.

[0021] In one of the embodiments, the method further comprises:

[0022] obtain second sample point cloud data;

[0023] convert the second sample point cloud data to the camera coordinate system through the external conversion matrix of the laser radar to camera coordinate system to obtain the processed second sample point cloud data; the processed second sample point cloud data is the point filtered out which is not in the camera collection visual angle and the laser radar does not receive successfully;

[0024] convert the processed second sample point cloud data to the polar coordinate system to obtain the polar angle of the processed second sample point cloud data;

[0025] For two points with equal polar angles in the processed second sample point cloud data, fill points are added by equally dividing the polar length to obtain the filled second sample point cloud data.

[0026] The filled second sample point cloud data is transformed from the point cloud coordinate system to the pixel coordinate system, and a mapping relationship is established between the depth points of the filled second sample point cloud data and the pixels in the pixel coordinate system.

[0027] Based on the mapping relationship between the depth points of the filled second sample point cloud data and the pixels in the pixel coordinate system, the pixel coordinate information corresponding to the filled second sample point cloud data is fitted to obtain the calibration parameters.

[0028] The calibration parameters are used to match the point cloud data of the third sample and the camera data of the third sample in the same sample detection area;

[0029] The mapping relationship is established based on the matching results of the third sample point cloud data and the third sample camera data.

[0030] A method for enhancing camera data detection results, the method comprising:

[0031] Acquire point cloud data and camera data of the detection area;

[0032] The point cloud data is inspected to obtain the inspection results of the point cloud data;

[0033] The camera data is detected to obtain the detection result of the camera data;

[0034] Based on the pre-established mapping relationship, the detection results of the point cloud data and the detection results of the camera data are associated and matched;

[0035] Based on the association matching result, the detection result of the camera data is enhanced to obtain the enhanced detection result of the camera data; wherein, the enhanced detection result of the camera data includes three-dimensional physical information.

[0036] In one embodiment, the step of associating and matching the detection results of the point cloud data and the detection results of the camera data according to a pre-established mapping relationship includes:

[0037] Based on the mapping relationship, the detection results of the point cloud data are mapped onto the camera data to obtain the mapping results of the point cloud data detection results;

[0038] Calculate the first similarity between the mapping result of the detection result of the point cloud data and the detection result of the camera data;

[0039] According to the first similarity, the detection result of the point cloud data and the detection result of the camera data are associated matched.

[0040] In one of the embodiments, the associated matching of the detection result of the point cloud data and the detection result of the camera data according to the pre-established mapping relationship comprises:

[0041] According to the mapping relationship, the camera data is mapped into the point cloud data coordinate system to obtain a mapping result of the camera data;

[0042] A second similarity between the mapping result of the camera data and the detection result of the point cloud data is calculated;

[0043] According to the second similarity, the detection result of the point cloud data and the detection result of the camera data are associated matched.

[0044] In one of the embodiments, the enhancing of the detection result of the camera data according to the associated matching result to obtain an enhanced detection result of the camera data comprises:

[0045] If the associated matching result is that the detection result of the point cloud data and the detection result of the camera data are matched successfully, the detection result of the point cloud data is mapped into the detection result of the camera data to obtain the enhanced detection result of the camera data;

[0046] If the associated matching result is that the detection result of the point cloud data and the detection result of the camera data are not matched successfully, the detection result of the point cloud data is determined as the enhanced detection result of the camera data.

[0047] In one of the embodiments, the method further comprises:

[0048] According to the positioning frame of the detection result of the camera data, a positioning point of the detection result of the camera data in the point cloud coordinate system is determined;

[0049] According to the positioning point and a pre-established fitting positioning relationship, a position of the detection result of the camera data in the point cloud coordinate system is determined;

[0050] The detection result of the camera data is associated matched by using a video tracking algorithm and the position of the detection result of the camera data in the point cloud coordinate system to obtain a driving track of the detection result of the camera data in the point cloud coordinate system;

[0051] According to the detection result of the camera data, a driving track in the point cloud coordinate system is obtained, and feature information of the detection result of the camera data is obtained; the feature information of the detection result of the camera data includes at least one of position, speed, heading angle, and acceleration of the detection result of the camera data.

[0052] In one of the embodiments, the method further comprises:

[0053] According to the size of the bounding box of the detection result of the camera data, the size of the detection result of the camera data is obtained.

[0054] In one of the embodiments, the method further comprises:

[0055] Obtain second sample point cloud data;

[0056] Convert the second sample point cloud data to the camera coordinate system through an external conversion matrix of the laser radar to the camera coordinate system to obtain processed second sample point cloud data; the processed second sample point cloud data is filtered out of points not within the camera collection angle and points not successfully received by the laser radar;

[0057] Convert the processed second sample point cloud data to a polar coordinate system to obtain a polar angle of the processed second sample point cloud data;

[0058] Add a filling point to two points with equal polar angles of the processed second sample point cloud data by bisecting the polar length to obtain filled second sample point cloud data;

[0059] Convert the filled second sample point cloud data from the point cloud coordinate system to the pixel coordinate system to establish a mapping relationship pair between the depth points of the filled second sample point cloud data and the pixel points in the pixel coordinate system;

[0060] According to the mapping relationship pair between the depth points of the filled second sample point cloud data and the pixel points in the pixel coordinate system, pixel coordinate information corresponding to the filled second sample point cloud data is fitted to obtain calibration parameters;

[0061] Use the calibration parameters to match third sample point cloud data and third sample camera data in the same sample detection area;

[0062] According to the matching result of the third sample point cloud data and the third sample camera data, the mapping relationship is established.

[0063] A camera data detection result enhancement device, the device comprising:

[0064] An acquisition module is configured to acquire camera data of a detection area.

[0065] An enhancement module is configured to input the camera data into a preset detection model to obtain an enhanced detection result of the camera data; wherein the detection model is obtained by mapping first sample point cloud data to first sample camera data, associating and matching a mapping result of the first sample point cloud data and a detection result of the first sample camera data according to a similarity between the mapping result and the detection result, and training a preset initial detection model according to an association and matching result.

[0066] An enhancement device for a camera data detection result, the device comprising:

[0067] A first acquisition module is configured to acquire point cloud data and camera data of a detection area.

[0068] A second acquisition module is configured to detect the point cloud data to obtain a detection result of the point cloud data.

[0069] A third acquisition module is configured to detect the camera data to obtain a detection result of the camera data.

[0070] An association module is configured to associate and match the detection result of the point cloud data and the detection result of the camera data according to a preset mapping relationship.

[0071] An enhancement module is configured to enhance the detection result of the camera data according to the association and matching result to obtain an enhanced detection result of the camera data; wherein the enhanced detection result of the camera data is a result including three-dimensional physical information.

[0072] A computer device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the following steps when executing the computer program:

[0073] An acquisition module is configured to acquire camera data of a detection area.

[0074] input the camera data into a preset detection model to obtain an enhanced detection result of the camera data; wherein the enhanced detection result of the camera data is a result including three-dimensional physical information; the detection model is obtained by mapping first sample point cloud data to first sample camera data, correlatively matching a mapping result of the first sample point cloud data and a detection result of the first sample camera data according to a similarity between the mapping result of the first sample point cloud data and the detection result of the first sample camera data, and training a preset initial detection model according to a correlatively matched result; the first sample camera data and the first sample point cloud data are data of a same detection region.

[0075] A computer readable storage medium having stored thereon a computer program, the computer program being executed by a processor to implement the following steps:

[0076] obtaining camera data of a detection region;

[0077] inputting the camera data into a preset detection model to obtain an enhanced detection result of the camera data; wherein the enhanced detection result of the camera data is a result including three-dimensional physical information; the detection model is obtained by mapping first sample point cloud data to first sample camera data, correlatively matching a mapping result of the first sample point cloud data and a detection result of the first sample camera data according to a similarity between the mapping result of the first sample point cloud data and the detection result of the first sample camera data, and training a preset initial detection model according to a correlatively matched result; the first sample camera data and the first sample point cloud data are data of a same detection region.

[0078] The aforementioned method, apparatus, computer equipment, and storage medium for enhancing camera data detection results, because the detection model maps first sample point cloud data of the same detection area onto first sample camera data, and performs association matching between the mapping result of the first sample point cloud data and the detection result of the first sample camera data based on the similarity between them, and trains a preset initial detection model based on the association matching result, the resulting detection model can accurately detect the input camera data, solving problems such as blind spots and data distortion caused by the characteristics of camera data, when the target object is occluded, at night, or due to weather conditions; or when the target... When the target is too far from the camera, the pixels in the camera data are relatively small, and the algorithm cannot detect the target in the data. In addition, due to the high dimensionality of point cloud data, by mapping the first sample point cloud data onto the first sample camera data, the similarity between the mapping result of the first sample point cloud data and the detection result of the first sample camera data can be accurately associated and matched. Targets in the camera data with low feature dimensionality, insufficient completeness, or small pixels can be labeled. Thus, the preset initial detection model can be accurately trained based on the association and matching results, thereby improving the accuracy of the enhanced detection results obtained from the camera data through the detection model. Attached Figure Description

[0079] Figure 1 A schematic diagram of the internal structure of a computer device provided for one embodiment;

[0080] Figure 2 This is a flowchart illustrating a method for enhancing camera data detection results in one embodiment;

[0081] Figure 3 This is a flowchart illustrating a method for enhancing camera data detection results in another embodiment;

[0082] Figure 4 This is a flowchart illustrating a method for enhancing camera data detection results in another embodiment;

[0083] Figure 5 This is a flowchart illustrating a method for enhancing camera data detection results in another embodiment;

[0084] Figure 5a This is a schematic diagram of the second sample point cloud data in one embodiment;

[0085] Figure 5b This is a schematic diagram illustrating the filtering of point cloud data that is not within the camera's field of view in one embodiment.

[0086] Figure 5c A schematic diagram of filtering out the second sample point cloud data that is not within the camera's capture angle and is not successfully received by the laser radar in one embodiment;

[0087] Figure 5d A flowchart of obtaining the polar angle of the processed second sample point cloud data in one embodiment;

[0088] Figure 5e A flowchart of obtaining the filled second sample point cloud data in one embodiment;

[0089] Figure 6 A flowchart of the enhancement method of the camera data detection result in one embodiment;

[0090] Figure 7 A flowchart of the enhancement method of the camera data detection result in another embodiment;

[0091] Figure 8 A flowchart of the enhancement method of the camera data detection result in another embodiment;

[0092] Figure 9 A flowchart of the enhancement method of the camera data detection result in another embodiment;

[0093] Figure 10 A flowchart of the enhancement method of the camera data detection result in another embodiment;

[0094] Figure 11 A flowchart of the enhancement method of the camera data detection result in another embodiment;

[0095] Figure 12 A structural block diagram of the enhancement device of the camera data detection result in one embodiment;

[0096] Figure 13 A structural block diagram of the enhancement device of the camera data detection result in one embodiment. DETAILED DESCRIPTION

[0097] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0098] The enhancement method of the camera data detection result provided by the embodiments of the present application can be applied to, for example, Figure 1The computer device shown in the figure. The computer device includes a processor, a memory connected by a system bus, the memory stores a computer program, and the processor executes the computer program to perform the steps of the method embodiments described below. Optionally, the computer device can also include a network interface, a display screen and an input device. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with external terminals through network connection. Optionally, the computer device can be a server, a personal computer, a personal digital assistant, or other terminal devices such as tablets, mobile phones, etc. It can also be a cloud or remote server, and the specific form of the computer device is not limited in the present application.

[0099] In one embodiment, as shown in Figure 2 , a method for enhancing camera data detection results is provided. This method is applied to a computer device in Figure 1 for illustration, including the following steps:

[0100] S201, obtaining camera data of a detection area.

[0101] Among them, the camera data is the data including a plurality of pixel points obtained by shooting the target object through the camera. Specifically, the computer device obtains the camera data corresponding to the detection area. Optionally, the computer device can obtain the camera data corresponding to the detection area from the base station corresponding to the detection area. Optionally, the detection area can be an intersection, or a region such as a station. Optionally, the camera data of the detection area can be collected by any one of a gun-type camera, a hemispherical camera, and a spherical camera.

[0102] S202, inputting the camera data into a preset detection model to obtain an enhanced detection result of the camera data; wherein the enhanced detection result of the camera data is a result including three-dimensional physical information; the detection model is obtained by mapping first sample point cloud data to first sample camera data, associating and matching the mapping result of the first sample point cloud data and the detection result of the first sample camera data according to the similarity between the mapping result of the first sample point cloud data and the detection result of the first sample camera data, and training the preset initial detection model according to the association and matching result; The first sample camera data and the first sample point cloud data are data of the same detection area.

[0103] Specifically, the computer device inputs the obtained camera data into a preset detection model to obtain an enhanced detection result of the camera data. The enhanced detection result of the camera data is a result including three-dimensional physical information. The preset detection model is obtained by mapping first sample point cloud data of a same detection region to first sample camera data, associating and matching a mapping result of the first sample point cloud data and a detection result of the first sample camera data according to a similarity between the mapping result of the first sample point cloud data and the detection result of the first sample camera data, and training a preset initial detection model according to an association matching result. Optionally, the first sample point cloud data can be collected by any one of an 8-line laser radar, a 16-line laser radar, a 24-line laser radar, a 32-line laser radar, a 64-line laser radar, and a 128-line laser radar, and the first sample camera data can be collected by any one of a gun-shaped camera, a half-sphere-shaped camera, and a sphere-shaped camera.

[0104] In the enhanced method of the camera data detection result, the detection model is obtained by mapping first sample point cloud data of a same detection region to first sample camera data, associating and matching a mapping result of the first sample point cloud data and a detection result of the first sample camera data according to a similarity between the mapping result of the first sample point cloud data and the detection result of the first sample camera data, and training a preset initial detection model according to an association matching result. The detection model obtained in this way can accurately detect input camera data, and solves the problems of a blind area, data distortion, and the like of the camera data due to the characteristics of the camera data when the target is blocked, at night, or under the influence of weather, or the problem that the algorithm cannot detect the target in the data when the target is too far from the camera. In addition, because the dimensionality of the point cloud data is high, the detection result of the first sample point cloud data and the detection result of the first sample camera data can be accurately associated and matched by mapping the first sample point cloud data to the first sample camera data and according to the similarity between the mapping result of the first sample point cloud data and the detection result of the first sample camera data. The target with low feature dimensionality or insufficient completeness or small pixel points in the camera data can be labeled, so that the preset initial detection model can be accurately trained according to the association matching result, and the accuracy of the enhanced detection result of the camera data obtained by the detection model is further improved.

[0105] In the scenario of inputting the camera data into the preset detection model to obtain the enhanced detection result of the camera data, the detection model needs to be trained in advance. In one embodiment, as shown in Figure 3 the training method of the detection model includes:

[0106] S301, mapping the detection result of the first sample point cloud data to the first sample camera data according to a pre-established mapping relationship to obtain a mapping result of the detection result of the first sample point cloud data.

[0107] Specifically, the computer device maps the detection result of the first sample point cloud data to the first sample camera data according to a pre-established mapping relationship to obtain a mapping result of the detection result of the first sample point cloud data. It should be noted that the detection result of the first sample point cloud data includes point cloud data wrapped by a target box and feature information corresponding to the point cloud data wrapped by the target box. Optionally, the computer device can establish a mapping relationship of the detection result of the first sample point cloud data to the first sample camera data by using the calibration parameters of the first sample point cloud data and the first sample camera data, and map the detection result of the first sample point cloud data to the first sample camera data according to the mapping relationship to obtain the mapping result of the detection result of the first sample point cloud data.

[0108] S302, calculating a first similarity between the mapping result of the detection result of the first sample point cloud data and the detection result of the first sample camera data.

[0109] Specifically, the computer device calculates a first similarity between the mapping result of the detection result of the first sample point cloud data and the detection result of the first sample camera data. Optionally, the computer device can calculate an overlap degree between the mapping result of the detection result of the first sample point cloud data and the detection result of the first sample camera data, and obtain the first similarity between the mapping result of the detection result of the first sample point cloud data and the detection result of the first sample camera data according to the overlap degree between the mapping result of the detection result of the first sample point cloud data and the detection result of the first sample camera data.

[0110] S303, according to the first similarity, associating and matching the mapping result of the detection result of the first sample point cloud data and the detection result of the first sample camera data.

[0111] Specifically, the computer device associates and matches the mapping result of the detection result of the first sample point cloud data and the detection result of the first sample camera data according to the first similarity between the mapping result of the detection result of the first sample point cloud data and the detection result of the first sample camera data. Optionally, the computer device can compare the first similarity between the mapping result of the detection result of the first sample point cloud data and the detection result of the first sample camera data with a preset threshold (such as 0.8), and when the first similarity between the mapping result of the detection result of the first sample point cloud data and the detection result of the first sample camera data reaches the preset threshold, it is considered that the mapping result of the detection result of the first sample point cloud data and the detection result of the first sample camera data are successfully matched, and a matching result of successful matching is obtained.

[0112] S304, according to the association matching result, the first sample camera data is labeled to obtain the label of the first sample camera data.

[0113] Specifically, the computer device labels the first sample camera data according to the mapping result of the detection result of the first sample point cloud data and the association matching result of the detection result of the first sample camera data, to obtain the label of the first sample camera data. For example, if the detection result of the first sample camera data is a car, the computer device maps the detection result of the first sample point cloud data to the first sample camera data, and according to the position of the car detected from the first sample point cloud data in the camera coordinate system, the first sample camera data at the corresponding position is labeled as a car, and the label corresponding to the first sample camera data is a car.

[0114] S305, according to the first sample camera data and the label of the first sample camera data, the initial detection model is trained to obtain the detection model.

[0115] Specifically, the computer device trains the initial detection model according to the first sample camera data and the label of the first sample camera data to obtain the detection model. It can be understood that the computer device can input the first sample camera data into the initial detection model to obtain the detection result of the first sample camera data, and according to the detection result of the first sample camera data and the label of the first sample camera data, the value of the loss function of the initial detection model is obtained, and then the initial detection model is trained according to the value of the loss function of the initial detection model, until the value of the loss function of the initial detection model reaches the minimum value or reaches the stable value, to obtain the detection model.

[0116] In this embodiment, the computer device can accurately map the detection result of the first sample point cloud data to the first sample camera data according to the pre-established mapping relationship, improve the accuracy of the mapping result of the detection result of the first sample point cloud data, and accurately calculate the first similarity between the mapping result of the detection result of the first sample point cloud data and the detection result of the first sample camera data, and then accurately associate the mapping result of the detection result of the first sample point cloud data and the detection result of the first sample camera data according to the first similarity between the mapping result of the detection result of the first sample point cloud data and the detection result of the first sample camera data, to obtain the association matching result with high accuracy, so that the first sample camera data can be accurately labeled according to the obtained association matching result, and the accuracy of the label of the first sample camera data is improved, and then the initial detection model can be accurately trained according to the first sample camera data and the label of the first sample camera data, so that the accuracy of the obtained detection model is improved.

[0117] In the above scenario of inputting the camera data into the preset detection model to obtain the enhanced detection result of the camera data, the detection model needs to be trained in advance. In one embodiment, as shown in Figure 4 the training method of the detection model includes:

[0118] S401, according to a pre-established mapping relationship, mapping the first sample camera data into a point cloud data coordinate system to obtain a mapping result of the first sample camera data.

[0119] Specifically, the computer device maps the first sample camera data into the point cloud data coordinate system according to the pre-established mapping relationship to obtain the mapping result of the first sample camera data. Optionally, the computer device can establish a mapping relationship of the first sample camera data mapping into the point cloud data coordinate system by using the calibration parameters of the first sample point cloud data and the first sample camera data, and map the first sample camera data into the point cloud data coordinate system according to the mapping relationship to obtain the mapping result of the first sample camera data.

[0120] S402, calculating a second similarity between the mapping result of the first sample camera data and the detection result of the first sample point cloud data.

[0121] Specifically, the computer device calculates the second similarity between the mapping result of the first sample camera data and the detection result of the first sample point cloud data. Optionally, the computer device can calculate the overlap degree between the mapping result of the first sample camera data and the detection result of the first sample point cloud data, and obtain the second similarity between the mapping result of the first sample camera data and the detection result of the first sample point cloud data according to the overlap degree between the mapping result of the first sample camera data and the detection result of the first sample point cloud data.

[0122] S403, according to the second similarity, associating and matching the detection result of the first sample point cloud data and the mapping result of the first sample camera data.

[0123] Specifically, the computer device associates and matches the detection result of the first sample point cloud data and the mapping result of the first sample camera data according to the second similarity between the mapping result of the first sample camera data and the detection result of the first sample point cloud data obtained above. Optionally, the computer device can compare the second similarity between the mapping result of the first sample camera data and the detection result of the first sample point cloud data with a preset threshold (such as 0.6), and when the second similarity between the mapping result of the first sample camera data and the detection result of the first sample point cloud data reaches the preset threshold, it is considered that the mapping result of the first sample camera data and the detection result of the first sample point cloud data are successfully matched, and a matching result of successful matching is obtained.

[0124] S404, according to the association matching result, the first sample camera data is labeled to obtain the label of the first sample camera data.

[0125] Specifically, the computer device labels the first sample camera data according to the association matching result of the detection result of the first sample point cloud data and the mapping result of the first sample camera data, and obtains the label of the first sample camera data. Illustratively, if the detection result of the first sample point cloud data is a car, the computer device maps the first sample point cloud data into the camera coordinate system, and according to the position of the car in the camera coordinate system detected by the first sample point cloud data, the first sample camera data corresponding to the position is labeled as a car, and the label corresponding to the first sample camera data is a car.

[0126] S405, according to the first sample camera data and the label of the first sample camera data, the initial detection model is trained to obtain the detection model.

[0127] Specifically, the computer device trains the initial detection model according to the first sample camera data and the label of the first sample camera data to obtain the detection model. It can be understood that the computer device can input the first sample camera data into the initial detection model to obtain the detection result of the first sample camera data, and according to the detection result of the first sample camera data and the label of the first sample camera data, the value of the loss function of the initial detection model is obtained, and then the initial detection model is trained according to the value of the loss function of the initial detection model, until the value of the loss function of the initial detection model reaches the minimum value or reaches the stable value, and the above detection model is obtained.

[0128] In this embodiment, the computer device can accurately map the first sample camera data into the point cloud data coordinate system according to the pre-established mapping relationship, improve the accuracy of the mapping result of the obtained first sample camera data, and accurately calculate the second similarity between the mapping result of the first sample camera data and the detection result of the first sample point cloud data, and then accurately associate and match the detection result of the first sample point cloud data and the mapping result of the first sample camera data according to the second similarity between the mapping result of the first sample camera data and the detection result of the first sample point cloud data, and obtain the association matching result with high accuracy, so that the first sample camera data can be accurately labeled according to the obtained association matching result, and the accuracy of the label of the obtained first sample camera data is improved, and then the initial detection model can be accurately trained according to the first sample camera data and the label of the first sample camera data, so that the accuracy of the obtained detection model is improved.

[0129] In the above, according to the pre-established mapping relationship, the detection result of the first sample point cloud data is mapped to the first sample camera data, or according to the pre-established mapping relationship, the first sample camera data is mapped to the scene in the point cloud data coordinate system, and the mapping relationship needs to be established in advance. In an embodiment, as shown in Figure 5 The above method further includes:

[0130] S501, obtaining second sample point cloud data.

[0131] Specifically, the computer device obtains the second sample point cloud data. Optionally, the computer device can obtain the second sample point cloud data from the base station corresponding to the detection area. Optionally, the detection area can be an intersection, a station area or the like. Optionally, the second sample point cloud data can be collected by any one of an 8-line laser radar, a 16-line laser radar, a 24-line laser radar, a 32-line laser radar, a 64-line laser radar, and a 128-line laser radar. As shown in Figure 5a , Figure 5a is a schematic diagram of the obtained second sample point cloud data.

[0132] S502, converting the second sample point cloud data to a camera coordinate system through an external conversion matrix of the laser radar to the camera coordinate system, to obtain processed second sample point cloud data; the processed second sample point cloud data is filtered to remove points not in the camera collection angle and points not successfully received by the laser radar.

[0133] Specifically, the computer device converts the above second sample point cloud data to a camera coordinate system through an external conversion matrix of the laser radar to the camera coordinate system, to obtain processed second sample point cloud data. The processed second sample point cloud data is filtered to remove points not in the camera collection angle and points not successfully received by the laser radar. As shown in Figure 5b , Figure 5c is a schematic diagram of the processed second sample point cloud data filtered to remove points not in the camera collection angle, Figure 5b is a schematic diagram of the processed second sample point cloud data filtered to remove points not in the camera collection angle and points not successfully received by the laser radar. Figure 5c

[0134] S503, converting the processed second sample point cloud data to a polar coordinate system to obtain a polar angle of the processed second sample point cloud data.

[0135] Specifically, the computer device converts the above obtained processed second sample point cloud data to a polar coordinate system to obtain a polar angle of the processed second sample point cloud data. As shown in Figure 5d ​As shown, the eye point of the processed second sample point cloud data (X, Y, Z) is (X, Y, 0), the computer device converts the eye point (X, Y) to polar coordinates on the plane of Z = 0 to obtain the polar angle of the processed second sample point cloud data.

[0136] S504, two points with equal polar angles of the processed second sample point cloud data are added with filling points by equalizing the polar length to obtain the filled second sample point cloud data.

[0137] Specifically, the computer device adds filling points to two points with equal polar angles of the processed second sample point cloud data by equalizing the polar length to obtain the filled second sample point cloud data. For example, as shown, the laser radar has different beams, and the computer device adds filling points to two points with equal polar angles and adjacent beams by equalizing the polar length to obtain the filled second sample point cloud data. Figure 5e

[0138] S505, the filled second sample point cloud data is converted from the point cloud coordinate system to the pixel coordinate system to establish a mapping relationship pair between the depth points of the filled second sample point cloud data and the pixel points in the pixel coordinate system.

[0139] Specifically, the computer device converts the filled second sample point cloud data from the point cloud coordinate system to the pixel coordinate system to establish a mapping relationship pair between the depth points of the filled second sample point cloud data and the pixel points in the pixel coordinate system. Optionally, the computer device can convert the filled second sample point cloud data from the point cloud coordinate system to the pixel coordinate system through an external conversion matrix of the laser radar to the camera coordinate system and an internal conversion matrix of the laser radar to the camera coordinate system. At this time, the three-dimensional coordinates of each filled second sample point cloud data and the coordinates mapped to the pixel coordinate system are one-to-one corresponding, so that the computer device can establish a mapping relationship pair between the depth points of the filled second sample point cloud data and the pixel points in the pixel coordinate system.

[0140] S506, according to the mapping relationship pair between the depth points of the filled second sample point cloud data and the pixel points in the pixel coordinate system, the pixel coordinate information corresponding to the filled second sample point cloud data is fitted to obtain the calibration parameter.

[0141] Specifically, the computer device fits the pixel coordinate information corresponding to the filled second sample point cloud data according to the mapping relationship pair between the depth points of the filled second sample point cloud data and the pixel points in the pixel coordinate system to obtain the calibration parameter. Optionally, the computer device can fit the pixel coordinate information corresponding to the filled second sample point cloud data to obtain the calibration parameter.

[0142] S507, the third sample point cloud data and the third sample camera data of the same sample detection area are matched by using the calibration parameter.​

[0143] Specifically, the computer device matches the third sample point cloud data and the third sample camera data of the same sample detection area by using the calibration parameters obtained above. Optionally, the computer device can obtain the coordinates (x_0, y_0) of a pixel of a point in the third sample camera data, then obtain two circles closest to the point from the inside and outside according to the fitting circles of the line bundles described in S506 above, then obtain the intersection points of the straight line x = x_0 and the two circles, take the mapping points (x_0, y_1), (x_0, y_2) of the two point clouds closest to the two circles in the line bundles corresponding to the two fitting circles, then obtain the actual point cloud coordinates (X_1, Y_1, Z_1), (X_2, Y_2, Z_2) corresponding to the mapping points by using the fitting parameters of the circles corresponding to the line bundles in S506 above, and then obtain the three-dimensional coordinates corresponding to (x_0, y_0) by proportionally obtaining (X_1, Y_1, Z_1), (X_2, Y_2, Z_2) from (x_0, y_0) to (x_0, y_1) and (x_0, y_2). At this time, the matching of the third sample point cloud data and the third sample camera data of the same sample detection area is completed.

[0144] S508, according to the matching result of the third sample point cloud data and the third sample camera data, a mapping relationship is established.

[0145] Specifically, the computer device establishes a mapping relationship between the detection result of the point cloud data and the camera data, or a mapping relationship between the camera data and the point cloud data coordinate system, according to the matching result of the third sample point cloud data and the third sample camera data.

[0146] In this embodiment, the computer device can convert the obtained second sample point cloud data to the camera coordinate system by the external conversion matrix of the laser radar to the camera coordinate system, accurately obtain the processed second sample point cloud data filtered out from the camera collection angle and the laser radar unsuccessful reception, and then convert the processed second sample point cloud data to the polar coordinate system to obtain the polar angle of the processed second sample point cloud data. Furthermore, the two points with equal polar angle of the processed second sample point cloud data can be added with filling points by equal division of the polar length to obtain the filled second sample point cloud data. Further, the filled second sample point cloud data can be converted from the point cloud coordinate system to the pixel coordinate system to establish the mapping relationship pair of the depth points of the filled second sample point cloud data and the pixel points in the pixel coordinate system, so as to fit the pixel coordinate information corresponding to the filled second sample point cloud data to obtain the calibration parameter. Furthermore, the third sample point cloud data and the third sample camera data of the same sample detection region can be matched by using the calibration parameter, so that the mapping relationship between the detection result of the point cloud data and the camera data, or the mapping relationship between the camera data and the point cloud data coordinate system can be accurately established according to the matching result of the third sample point cloud data and the third sample camera data.

[0147] In one embodiment, as shown in Figure 6 , a point cloud data detection result enhancement method is provided. The method is applied to the computer device in Figure 1 for example, and includes the following steps:

[0148] S601, obtaining point cloud data and camera data of a detection region.

[0149] Specifically, the computer device obtains point cloud data and camera data of a detection region. Optionally, the computer device can obtain point cloud data and camera data corresponding to the detection region from a base station corresponding to the detection region. It should be noted that the point cloud data and camera data corresponding to the detection region are data of the same time and the same spatial dimension. Optionally, the detection region can be an intersection, or a station area.

[0150] S602, detecting the point cloud data to obtain a detection result of the point cloud data.

[0151] Specifically, the computer device detects the obtained point cloud data of the detection region to obtain a detection result of the point cloud data. Optionally, the computer device can use a preset deep learning model and a preset tracking algorithm to detect the point cloud data. Optionally, the obtained detection result of the point cloud data can include the position, size, speed and heading angle of the target object in the point cloud coordinate system.

[0152] S603, detecting the camera data to obtain a detection result of the camera data.

[0153] Specifically, the computer device detects the obtained camera data to obtain a detection result of the camera data. Optionally, the computer can use a preset deep learning model and a preset tracking algorithm to detect the camera data. Optionally, the detection result of the camera data can include a pixel rectangular frame in which a target object is located, target confidence, target category, target tracking ID and the like.

[0154] S604, according to a pre-established mapping relationship, the detection result of the point cloud data and the detection result of the camera data are associated and matched.

[0155] Specifically, the computer device associates and matches the detection result of the point cloud data and the detection result of the camera data in the detection region according to the pre-established mapping relationship. Optionally, the computer device can use the calibration parameters of the sample point cloud data and the sample camera data to establish a mapping relationship in which the sample camera data is mapped into the coordinate system of the point cloud data, and associate and match the detection result of the point cloud data and the detection result of the camera data in the detection region according to the mapping relationship.

[0156] S605, according to the associated matching result, the detection result of the camera data is enhanced to obtain an enhanced detection result of the camera data; wherein the enhanced detection result of the camera data is a result including three-dimensional physical information.

[0157] Specifically, the computer device enhances the detection result of the camera data in the detection region according to the associated matching result of the detection result of the point cloud data and the detection result of the camera data in the detection region, to obtain an enhanced detection result of the camera data in the detection region, wherein the enhanced detection result of the camera data is a result including three-dimensional physical information. Optionally, the computer device can determine the detection result of the point cloud data as the enhanced detection result of the camera data according to the associated matching result of the detection result of the point cloud data and the detection result of the camera data in the detection region.

[0158] In the enhancement method of the camera data detection result, the computer device can obtain the detection result of the point cloud data by detecting the point cloud data, and obtain the detection result of the camera data by detecting the camera data, so as to associate and match the detection result of the point cloud data and the detection result of the camera data according to the pre-established mapping relationship, and then enhance the detection result of the point cloud data of the detection region according to the association and matching result of the detection result of the point cloud data and the detection result of the camera data. The problems of blind area and data distortion of the camera data due to the characteristics of the camera data under the conditions of target occlusion, night light or weather influence are solved. Or when the target is too far from the camera, the pixel point in the camera data is relatively small, and the algorithm cannot detect the target in the data. In addition, since the dimension of the point cloud data is high, the detection result of the first sample point cloud data and the detection result of the first sample camera data can be accurately associated and matched according to the similarity between the mapping result of the first sample point cloud data and the detection result of the first sample camera data by mapping the first sample point cloud data to the first sample camera data. The target with low feature dimension or insufficient completeness or small pixel point in the camera data can be labeled, so that the initial detection model can be accurately trained according to the association and matching result, and the accuracy of the enhanced detection result of the camera data obtained by the detection model is improved.

[0159] In the above scene of associating and matching the detection result of the point cloud data and the detection result of the camera data according to the pre-established mapping relationship, in one embodiment, as shown in Figure 7 S604 includes:

[0160] S701, according to the mapping relationship, mapping the detection result of the point cloud data to the camera data to obtain the mapping result of the detection result of the point cloud data.

[0161] Specifically, the computer device maps the detection result of the point cloud data to the camera data according to the mapping relationship to obtain the mapping result of the detection result of the point cloud data. It should be noted that the detection result of the point cloud data includes the point cloud data wrapped by the target frame and the feature information corresponding to the point cloud data wrapped by the target frame. Optionally, the computer device can use the calibration parameters of the point cloud data and the camera data to establish the mapping relationship of the detection result of the point cloud data to the camera data, and map the detection result of the point cloud data to the camera data according to the mapping relationship to obtain the mapping result of the detection result of the point cloud data.

[0162] S702, calculating the first similarity between the mapping result of the detection result of the point cloud data and the detection result of the camera data.

[0163] Specifically, the computer device calculates a first similarity between the mapping result of the detection result of the point cloud data and the detection result of the camera data. Optionally, the computer device can calculate an overlap degree between the mapping result of the detection result of the point cloud data and the detection result of the camera data, and obtain the first similarity between the mapping result of the detection result of the point cloud data and the detection result of the camera data according to the overlap degree between the mapping result of the detection result of the point cloud data and the detection result of the camera data.

[0164] S703, according to the first similarity, the detection result of the point cloud data and the detection result of the camera data are associated and matched.

[0165] Specifically, the computer device associates and matches the detection result of the point cloud data and the detection result of the camera data according to the first similarity between the mapping result of the detection result of the point cloud data and the detection result of the camera data in the detection area. Optionally, the computer device can compare the first similarity between the mapping result of the detection result of the point cloud data and the detection result of the camera data in the detection area with a preset threshold (such as 0.6), and when the first similarity between the mapping result of the detection result of the point cloud data and the detection result of the camera data reaches the preset threshold, it is considered that the detection result of the point cloud data and the detection result of the camera data are successfully matched, and a matching result of successful matching is obtained.

[0166] In this embodiment, the computer device can accurately map the detection result of the point cloud data of the detection area to the camera data according to the pre-established mapping relationship, improve the accuracy of the mapping result of the obtained detection result of the point cloud data, and thus accurately calculate the first similarity between the mapping result of the detection result of the point cloud data and the detection result of the camera data, and then accurately associate and match the detection result of the point cloud data and the detection result of the camera data according to the first similarity between the mapping result of the detection result of the point cloud data and the detection result of the camera data, and obtain a high-accuracy associated matching result.

[0167] In the above scenario of associating and matching the detection result of the point cloud data and the detection result of the camera data according to the pre-established mapping relationship, in one embodiment, as shown in Figure 8 S604 includes:

[0168] S801, according to the mapping relationship, the camera data is mapped into the point cloud data coordinate system to obtain the mapping result of the camera data.

[0169] Specifically, the computer device maps the camera data of the detection region into the point cloud data coordinate system according to the pre-established mapping relationship, to obtain a mapping result of the camera data. Optionally, the computer device can establish a mapping relationship of sample camera data mapping into the point cloud data coordinate system by using calibration parameters of sample point cloud data and sample camera data, and map the camera data into the point cloud data coordinate system according to the mapping relationship, to obtain the mapping result of the camera data.

[0170] S802, calculate a second similarity between the mapping result of the camera data and the detection result of the point cloud data.

[0171] Specifically, the computer device calculates a second similarity between the mapping result of the camera data of the detection region and the detection result of the point cloud data. Optionally, the computer device can calculate an overlap degree between the mapping result of the camera data and the detection result of the point cloud data, and obtain the second similarity between the mapping result of the camera data and the detection result of the point cloud data according to the overlap degree between the mapping result of the camera data and the detection result of the point cloud data.

[0172] S803, according to the second similarity, perform associated matching on the detection result of the point cloud data and the detection result of the camera data.

[0173] Specifically, the computer device performs associated matching on the detection result of the point cloud data and the detection result of the camera data according to the second similarity between the mapping result of the camera data and the detection result of the point cloud data obtained above. Optionally, the computer device can compare the second similarity between the mapping result of the camera data and the detection result of the point cloud data with a preset threshold (such as 0.8), and when the second similarity between the mapping result of the camera data and the detection result of the point cloud data reaches the preset threshold, it is considered that the mapping result of the camera data and the detection result of the point cloud data are successfully matched, to obtain a matching result of successful matching.

[0174] In this embodiment, the computer device can accurately map the camera data of the detection region into the point cloud data coordinate system according to the pre-established mapping relationship, improve the accuracy of the obtained mapping result of the camera data, so as to accurately calculate the second similarity between the mapping result of the camera data and the detection result of the point cloud data, and then accurately perform associated matching on the detection result of the point cloud data and the detection result of the camera data according to the second similarity between the mapping result of the camera data and the detection result of the point cloud data, to obtain an associated matching result with higher accuracy.

[0175] In the association matching result of the detection result of the point cloud data and the detection result of the camera data, in the scene of enhancing the detection result of the camera data, in an embodiment, on the basis of the above-mentioned embodiment, as shown in the following S605, comprises: Figure 9

[0176] S901, if the association matching result is that the detection result of the point cloud data matches the detection result of the camera data successfully, the detection result of the point cloud data is mapped into the detection result of the camera data to obtain the enhanced detection result of the camera data.

[0177] Specifically, if the association matching result of the detection result of the point cloud data and the detection result of the camera data is that the detection result of the point cloud data matches the detection result of the camera data successfully, the computer device maps the detection result of the point cloud data into the detection result of the camera data to obtain the enhanced detection result of the camera data. For example, if the detection result of the camera data in the detection area is a car and the detection result of the point cloud data is also a car, after the computer device performs the association matching of the detection result of the point cloud data and the detection result of the camera data, the result is that the matching is successful, then the computer device maps the detection result of the point cloud data, i.e. the car, into the detection result of the camera data to obtain the enhanced detection result of the camera data.

[0178] S902, if the association matching result is that the detection result of the point cloud data does not match the detection result of the camera data successfully, the detection result of the point cloud data is determined as the enhanced detection result of the camera data.

[0179] Specifically, if the association matching result of the detection result of the point cloud data and the detection result of the camera data is that the detection result of the point cloud data does not match the detection result of the camera data successfully, the computer device determines the detection result of the point cloud data in the detection area as the enhanced detection result of the camera data in the detection area. For example, if the detection result of the point cloud data in the detection area is a car and the detection result of the camera data is that no target is detected, after the computer device performs the association matching of the detection result of the camera data and the detection result of the point cloud data, the result is that the matching is not successful, then the computer device determines the detection result of the point cloud data, i.e. the car, as the enhanced detection result of the camera data.

[0180] ​In this embodiment, if the associated matching result of the detection result of the point cloud data and the detection result of the camera data is that the detection result of the point cloud data and the detection result of the camera data match successfully, the computer device maps the detection result of the point cloud data into the detection result of the camera data to obtain an enhanced detection result of the camera data. If the associated matching result of the detection result of the point cloud data and the detection result of the camera data is that the detection result of the point cloud data and the detection result of the camera data do not match successfully, the computer device determines the detection result of the point cloud data as the enhanced detection result of the camera data. Since the dimension of the point cloud data is higher, the accuracy of the detection result of the point cloud data is higher, and thus, the detection result of the point cloud data is determined as the enhanced detection result of the camera data, which improves the accuracy of the obtained enhanced detection result of the camera data.

[0181] In some scenarios, after obtaining the detection result of the camera data, the computer device can further determine feature information of the detection result of the camera data according to the detection result of the camera data. In one embodiment, as shown in Figure 10 The above method further includes:

[0182] S1001, determining a positioning point of the detection result of the camera data in the point cloud coordinate system according to a positioning box of the detection result of the camera data.

[0183] Specifically, the computer device determines the positioning point of the detection result of the camera data in the point cloud coordinate system according to the positioning box of the detection result of the camera data. Optionally, the computer device can determine the center point of the lower edge of the positioning box of the detection result of the camera data as the positioning point of the detection result of the camera data in the point cloud coordinate system. Optionally, the computer device can also determine other points on the positioning box of the detection result of the camera data as the positioning point of the detection result of the camera data in the point cloud coordinate system, for example, the center point of the positioning box.

[0184] S1002, determining a position of the detection result of the camera data in the point cloud coordinate system according to the positioning point and a pre-established fitting positioning relationship.

[0185] Specifically, the computer device determines the position of the detection result of the camera data in the point cloud coordinate system according to the determined positioning point of the detection result of the camera data in the point cloud coordinate system and the pre-established fitting positioning relationship. The pre-established fitting positioning relationship is a fitting relationship between the positioning point of the detection result of the camera data in the point cloud coordinate system and the position of the detection result of the camera data in the point cloud coordinate system. After determining the positioning point of the detection result of the camera data in the point cloud coordinate system, the computer device can determine the position of the detection result of the camera data in the point cloud coordinate system according to the fitting positioning relationship.

[0186] S1003, associating the detection result of the camera data according to the video tracking algorithm and the position of the detection result of the camera data in the point cloud coordinate system, to obtain a driving track of the detection result of the camera data in the point cloud coordinate system.

[0187] Specifically, the computer device associates the detection result of the camera data according to the video tracking algorithm and the position of the detection result of the camera data in the point cloud coordinate system, to obtain a driving track of the detection result of the camera data in the point cloud coordinate system. Optionally, the computer device can predict the position of the detection result of the camera data in the point cloud coordinate system according to the video tracking algorithm, to obtain a driving track of the detection result of the camera data in the point cloud coordinate system, and then associate the position of the detection result of the camera data in the point cloud coordinate system with the detection result of the camera data, to obtain a driving track of the detection result of the camera data in the point cloud coordinate system.

[0188] S1004, obtaining feature information of the detection result of the camera data according to the driving track of the detection result of the camera data in the point cloud coordinate system; the feature information of the detection result of the camera data includes at least one of the position of the detection result of the camera data, the speed of the detection result of the camera data, the heading angle of the detection result of the camera data, and the acceleration of the detection result of the camera data.

[0189] Specifically, the computer device obtains feature information of the detection result of the camera data according to the driving track of the detection result of the camera data in the point cloud coordinate system. The feature information of the detection result of the camera data includes at least one of the position of the detection result of the camera data, the speed of the detection result of the camera data, the heading angle of the detection result of the camera data, and the acceleration of the detection result of the camera data. It can be understood that the computer device can calculate the position of the detection result of the camera data, the speed of the detection result of the camera data, the heading angle of the detection result of the camera data, and the acceleration of the detection result of the camera data according to the driving track of the detection result of the camera data in the point cloud coordinate system, to obtain the feature information of the detection result of the camera data.

[0190] In this embodiment, the computer device can quickly determine the positioning point of the detection result of the camera data in the point cloud coordinate system according to the positioning box of the detection result of the camera data, so as to quickly determine the position of the detection result of the camera data in the point cloud coordinate system according to the positioning point of the detection result of the camera data in the point cloud coordinate system and the pre-established fitting positioning relationship, and further, the computer device can quickly obtain the driving track of the detection result of the camera data in the point cloud coordinate system by using the video tracking algorithm and the position of the detection result of the camera data in the point cloud coordinate system, and further, the computer device can quickly obtain the feature information of the detection result of the camera data according to the driving track of the detection result of the camera data in the point cloud coordinate system, thereby improving the efficiency of obtaining the feature information of the detection result of the camera data.

[0191] In some scenarios, the computer device also needs to determine the size of the detection result of the camera data according to the detection result of the camera data. In one embodiment, the above method further includes: obtaining the size of the detection result of the camera data according to the size of the positioning box of the detection result of the camera data.

[0192] Specifically, the computer device obtains the size of the detection result of the camera data according to the size of the positioning box of the detection result of the camera data. Optionally, the computer device can determine the size of the positioning box of the detection result of the camera data as the size of the detection result of the camera data. For example, if the size of the positioning box of the detection result of the camera data is 16*16, the computer device can determine the size of the detection result of the camera data as 16*16.

[0193] In this embodiment, the computer device can quickly obtain the size of the detection result of the camera data according to the size of the positioning box of the detection result of the camera data, thereby improving the efficiency of obtaining the size of the detection result of the camera data, and further, the computer device can accurately obtain the size of the detection result of the camera data according to the size of the positioning box of the detection result of the camera data, thereby improving the accuracy of the obtained size of the detection result of the camera data.

[0194] In the above scenario of mapping the detection result of the point cloud data to the camera data according to the pre-established mapping relationship, or mapping the camera data to the point cloud data coordinate system according to the pre-established mapping relationship, the mapping relationship needs to be pre-established. In one embodiment, as shown in Figure 11 the above method further includes:

[0195] S1101, obtaining second sample point cloud data.

[0196] Specifically, the computer device obtains second sample point cloud data. Optionally, the computer device can obtain the second sample point cloud data from a base station corresponding to a detection area. Optionally, the detection area can be an intersection or a station area. Optionally, the second sample point cloud data can be collected by any one of an 8-line laser radar, a 16-line laser radar, a 24-line laser radar, a 32-line laser radar, a 64-line laser radar, and a 128-line laser radar. For example, please continue to refer to Figure 5a , Figure 5a for a schematic diagram of the obtained second sample point cloud data.

[0197] S1102, converting the second sample point cloud data to a camera coordinate system through an external conversion matrix of the laser radar to the camera coordinate system, to obtain processed second sample point cloud data; the processed second sample point cloud data is filtered to remove points that are not within the camera collection angle and are not successfully received by the laser radar.

[0198] Specifically, the computer device converts the above-mentioned second sample point cloud data to a camera coordinate system through an external conversion matrix of the laser radar to the camera coordinate system, to obtain processed second sample point cloud data. The processed second sample point cloud data is filtered to remove points that are not within the camera collection angle and are not successfully received by the laser radar. For example, please continue to refer to Figure 5b 、 Figure 5c , Figure 5b for the point cloud data filtered to remove points that are not within the camera collection angle, Figure 5c for the processed second sample point cloud data filtered to remove points that are not within the camera collection angle and are not successfully received by the laser radar.

[0199] S1103, converting the processed second sample point cloud data to a polar coordinate system to obtain a polar angle of the processed second sample point cloud data.

[0200] Specifically, the computer device converts the above-mentioned processed second sample point cloud data to a polar coordinate system to obtain a polar angle of the processed second sample point cloud data. For example, please continue to refer to Figure 5d If the overhead point (X, Y, Z) of the processed second sample point cloud data (X, Y, Z) is (X, Y, 0), the computer device converts the overhead point (X, Y) to a polar coordinate system on the plane of Z=0 to obtain a polar angle of the processed second sample point cloud data.

[0201] S1104, adding a filling point to two points with equal polar angles of the processed second sample point cloud data by bisecting the polar length, to obtain filled second sample point cloud data.

[0202] Specifically, the computer device adds a filling point to two points with equal polar angles of the processed second sample point cloud data by bisecting the polar length, to obtain the filled second sample point cloud data. For example, please continue to refer to Figure 5e The laser radar has different line beams, and the computer device adds a filling point to two points with equal polar angles and adjacent line beams by bisecting the polar length, to obtain the filled second sample point cloud data.

[0203] S1105, the computer device converts the filled second sample point cloud data from the point cloud coordinate system to the pixel coordinate system, to establish a mapping relationship pair between the depth points of the filled second sample point cloud data and the pixel points in the pixel coordinate system.

[0204] Specifically, the computer device converts the filled second sample point cloud data from the point cloud coordinate system to the pixel coordinate system, to establish a mapping relationship pair between the depth points of the filled second sample point cloud data and the pixel points in the pixel coordinate system. Optionally, the computer device can convert the filled second sample point cloud data from the point cloud coordinate system to the pixel coordinate system through an external conversion matrix of the laser radar to the camera coordinate system and an internal conversion matrix of the laser radar to the camera coordinate system. At this time, the three-dimensional coordinates of each filled second sample point cloud data and the coordinates mapped to the pixel coordinate system are in one-to-one correspondence, so that the computer device can establish a mapping relationship pair between the depth points of the filled second sample point cloud data and the pixel points in the pixel coordinate system according to the mapping relationship pair.

[0205] S1106, according to the mapping relationship pair between the depth points of the filled second sample point cloud data and the pixel points in the pixel coordinate system, the computer device fits the pixel coordinate information corresponding to the filled second sample point cloud data, to obtain the calibration parameter.

[0206] Specifically, the computer device fits the pixel coordinate information corresponding to the filled second sample point cloud data according to the mapping relationship pair between the depth points of the filled second sample point cloud data and the pixel points in the pixel coordinate system, to obtain the calibration parameter. Optionally, the computer device can fit the pixel coordinate information corresponding to the filled second sample point cloud data by using a circle, to obtain the calibration parameter.

[0207] S1107, the computer device matches the third sample point cloud data and the third sample camera data of the same sample detection region by using the calibration parameter.

[0208] Specifically, the computer device matches the third sample point cloud data and the third sample camera data of the same sample detection area by using the calibration parameters obtained above. Optionally, the computer device can obtain the coordinates (x_0, y_0) of a pixel of a point in the third sample camera data, then obtain two circles of the inner side and the outer side closest to the point according to the fitting circles of the line bundles described in S1006 above, then obtain the intersection points of the straight line x = x_0 and the two circles, take the mapping points (x_0, y_1), (x_0, y_2) of the two point clouds closest to the two circles in the line bundles corresponding to the two fitting circles, then obtain the actual point cloud coordinates (X_1, Y_1, Z_1), (X_2, Y_2, Z_2) corresponding to the mapping points by using the fitting parameters of the circles corresponding to the line bundles in S1006 above, and then obtain the three-dimensional coordinates corresponding to (x_0, y_0) by proportionally obtaining (X_1, Y_1, Z_1), (X_2, Y_2, Z_2) from (x_0, y_0) to (x_0, y_1) and (x_0, y_2). At this time, the matching of the third sample point cloud data and the third sample camera data of the same sample detection area is completed.

[0209] S1108, according to the matching result of the third sample point cloud data and the third sample camera data, a mapping relationship is established.

[0210] Specifically, the computer device establishes the mapping relationship between the detection result of the point cloud data and the camera data, or the mapping relationship between the camera data and the point cloud data coordinate system, according to the matching result of the third sample point cloud data and the third sample camera data.

[0211] In this embodiment, the computer device can convert the obtained second sample point cloud data to the camera coordinate system through the external conversion matrix of the laser radar to the camera coordinate system, accurately obtain the processed second sample point cloud data filtered out from the camera collection visual angle and the laser radar unsuccessful reception, and then convert the processed second sample point cloud data to the polar coordinate system to obtain the polar angle of the processed second sample point cloud data. Then, the two points with equal polar angle of the processed second sample point cloud data can be added with filling points by equal division of the polar length to obtain the filled second sample point cloud data. Further, the filled second sample point cloud data can be converted from the point cloud coordinate system to the pixel coordinate system to establish the mapping relationship pair of the depth points of the filled second sample point cloud data and the pixel points in the pixel coordinate system, so as to fit the pixel coordinate information corresponding to the filled second sample point cloud data to obtain the calibration parameter. Then, the third sample point cloud data and the third sample camera data of the same sample detection region can be matched by using the calibration parameter, so that the mapping relationship between the detection result of the point cloud data and the camera data, or the mapping relationship between the camera data and the point cloud data coordinate system can be accurately established according to the matching result of the third sample point cloud data and the third sample camera data.

[0212] It should be understood that, although Figures 2-11 the steps in the flowcharts are shown in sequence according to the arrows, these steps are not necessarily executed in sequence according to the arrows. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, Figures 2-11 at least part of the steps in the flowcharts can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times. The execution sequence of these steps or stages is not necessarily sequential, but can be executed in rotation or alternation with other steps or steps or stages in other steps.

[0213] In one embodiment, as shown in Figure 12 , an enhanced device of camera data detection result is provided, comprising: a first acquisition module and an enhancement module, wherein:

[0214] The first acquisition module is configured to acquire camera data of a detection region.

[0215] The enhancement module is configured to input the camera data into a preset detection model to obtain an enhanced detection result of the camera data, wherein the detection model is obtained by mapping first sample point cloud data to first sample camera data, associating and matching a mapping result of the first sample point cloud data and a detection result of the first sample camera data according to a similarity between the mapping result of the first sample point cloud data and the detection result of the first sample camera data, and training a preset initial detection model according to an association and matching result.

[0216] The camera data detection result enhancement device provided in the embodiment can execute the method embodiments, and has similar implementation principles and technical effects, which will not be described here.

[0217] On the basis of the above-mentioned embodiments, the device can further include a first mapping module, a first calculation module, a first matching module, a first labeling module and a first training module.

[0218] The first mapping module is configured to map the detection result of the first sample point cloud data to the first sample camera data according to a preset mapping relationship to obtain a mapping result of the detection result of the first sample point cloud data.

[0219] The calculation module is configured to calculate a first similarity between the mapping result of the detection result of the first sample point cloud data and the detection result of the first sample camera data.

[0220] The first matching module is configured to associate and match the mapping result of the detection result of the first sample point cloud data and the detection result of the first sample camera data according to the first similarity.

[0221] The first labeling module is configured to label the first sample camera data according to the association and matching result to obtain a label of the first sample camera data.

[0222] The first training module is configured to train the initial detection model according to the first sample camera data and the label of the first sample camera data to obtain the detection model.

[0223] The camera data detection result enhancement device provided in the embodiment can execute the method embodiments, and has similar implementation principles and technical effects, which will not be described here.

[0224] On the basis of the above-mentioned embodiments, the device can further include a second mapping module, a second calculation module, a second matching module, a second labeling module and a second training module.

[0225] The second mapping module is configured to map the first sample camera data into a point cloud data coordinate system according to a pre-established mapping relationship, to obtain a mapping result of the first sample camera data.

[0226] The second calculation module is configured to calculate a second similarity between the mapping result of the first sample camera data and the detection result of the first sample point cloud data.

[0227] The second matching module is configured to perform associated matching on the detection result of the first sample point cloud data and the mapping result of the first sample camera data according to the second similarity.

[0228] The second labeling module is configured to label the first sample camera data according to the associated matching result, to obtain a label of the first sample camera data.

[0229] The second training module is configured to train the initial detection model according to the first sample camera data and the label of the first sample camera data, to obtain the detection model.

[0230] The camera data detection result enhancement device provided in the embodiment can execute the method embodiments, and has similar implementation principles and technical effects, which will not be described here.

[0231] On the basis of the above-mentioned embodiments, the device can further include a second acquisition module, a third acquisition module, a fourth acquisition module, a fifth acquisition module, a first establishment module, a fitting module, a third matching module, and a second establishment module, wherein:

[0232] The second acquisition module is configured to acquire second sample point cloud data.

[0233] The third acquisition module is configured to convert the second sample point cloud data to a camera coordinate system through an external conversion matrix of the laser radar to the camera coordinate system, to obtain processed second sample point cloud data; the processed second sample point cloud data is second sample point cloud data from which points that are not within the camera collection angle and points that are not successfully received by the laser radar are filtered out.

[0234] The fourth acquisition module is configured to convert the processed second sample point cloud data to a polar coordinate system, to obtain a polar angle of the processed second sample point cloud data.

[0235] The fifth acquisition module is configured to add a filling point to two points with equal polar angles of the processed second sample point cloud data through equal division of the polar length, to obtain filled second sample point cloud data.

[0236] The first establishment module is configured to convert the filled second sample point cloud data from a point cloud coordinate system to a pixel coordinate system, to establish a mapping relationship pair of a depth point of the filled second sample point cloud data and a pixel point in the pixel coordinate system.

[0237] The fitting module is configured to fit pixel coordinate information corresponding to the filled second sample point cloud data according to the mapping relationship between the depth points of the filled second sample point cloud data and the pixel points in the pixel coordinate system, to obtain the calibration parameter.

[0238] The third matching module is configured to match the third sample point cloud data and the third sample camera data of the same sample detection region by using the calibration parameter.

[0239] The second establishing module is configured to establish the mapping relationship according to the matching result of the third sample point cloud data and the third sample camera data.

[0240] The camera data detection result enhancement device provided in this embodiment can execute the method embodiments, and has similar implementation principles and technical effects, which will not be described here.

[0241] The specific limitations of the camera data detection result enhancement device can be referred to the limitations of the camera data detection result enhancement method described above, which will not be described here. The modules in the camera data detection result enhancement device described above can be realized by software, hardware, and combinations thereof, in whole or in part. The modules described above can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to the modules.

[0242] In one embodiment, as shown in Figure 13 , a camera data detection result enhancement device is provided, which includes a first obtaining module, a second obtaining module, a third obtaining module, an association module, and an enhancement module,

[0243] Among them:

[0244] The first obtaining module is configured to obtain point cloud data and camera data of a detection region.

[0245] The second obtaining module is configured to detect the point cloud data to obtain a detection result of the point cloud data.

[0246] The third obtaining module is configured to detect the camera data to obtain a detection result of the camera data.

[0247] The association module is configured to associate and match the detection result of the point cloud data and the detection result of the camera data according to a pre-established mapping relationship.

[0248] The enhancement module is configured to enhance the detection result of the camera data according to the association and matching result, to obtain an enhanced detection result of the camera data; wherein the enhanced detection result of the camera data is a result including three-dimensional physical information.

[0249] The camera data detection result enhancement device provided in the embodiment can execute the method embodiments, and has similar implementation principles and technical effects, details of which are not repeated here.

[0250] On the basis of the above embodiment, optionally, the association module comprises a first acquisition unit, a first calculation unit and a first matching unit, wherein:

[0251] The first acquisition unit is configured to map the detection result of the point cloud data to the camera data according to the mapping relationship to obtain a mapping result of the detection result of the point cloud data.

[0252] The first calculation unit is configured to calculate a first similarity between the mapping result of the detection result of the point cloud data and the detection result of the camera data.

[0253] The first matching unit is configured to perform association matching on the detection result of the point cloud data and the detection result of the camera data according to the first similarity.

[0254] The camera data detection result enhancement device provided in the embodiment can execute the method embodiments, and has similar implementation principles and technical effects, details of which are not repeated here.

[0255] On the basis of the above embodiment, optionally, the association module comprises a second acquisition unit, a second calculation unit and a second matching unit, wherein:

[0256] The second acquisition unit is configured to map the camera data to a point cloud data coordinate system according to the mapping relationship to obtain a mapping result of the camera data.

[0257] The second calculation unit is configured to calculate a second similarity between the mapping result of the camera data and the detection result of the point cloud data.

[0258] The second matching unit is configured to perform association matching on the detection result of the point cloud data and the detection result of the camera data according to the second similarity.

[0259] The camera data detection result enhancement device provided in the embodiment can execute the method embodiments, and has similar implementation principles and technical effects, details of which are not repeated here.

[0260] On the basis of the above embodiment, optionally, the enhancement module comprises a first enhancement unit and a second enhancement unit, wherein:

[0261] The first enhancement unit is configured to, if the association matching result is that the detection result of the point cloud data and the detection result of the camera data are successfully matched, map the detection result of the point cloud data into the detection result of the camera data to obtain an enhanced detection result of the camera data.

[0262] The second enhancement unit is configured to determine the detection result of the point cloud data as an enhanced detection result of the camera data if the associated matching result is that the detection result of the point cloud data and the detection result of the camera data are not successfully matched.

[0263] The camera data detection result enhancement device provided in the embodiment can execute the method embodiments, and has similar implementation principles and technical effects, which will not be described here.

[0264] On the basis of the above-mentioned embodiments, the device further includes a first determination module, a second determination module, a fourth acquisition module and a fifth acquisition module, wherein:

[0265] The first determination module is configured to determine a positioning point of the detection result of the camera data in the point cloud coordinate system according to the positioning frame of the detection result of the camera data.

[0266] The second determination module is configured to determine a position of the detection result of the camera data in the point cloud coordinate system according to the positioning point and a pre-established fitting positioning relationship.

[0267] The fourth acquisition module is configured to associate and match the detection result of the camera data by using a video tracking algorithm and the position of the detection result of the camera data in the point cloud coordinate system, to obtain a driving track of the detection result of the camera data in the point cloud coordinate system.

[0268] The fifth acquisition module is configured to obtain feature information of the detection result of the camera data according to the driving track of the detection result of the camera data in the point cloud coordinate system; and the feature information of the detection result of the camera data includes at least one of position information, speed information, heading angle information and acceleration information of the detection result of the camera data.

[0269] The camera data detection result enhancement device provided in the embodiment can execute the method embodiments, and has similar implementation principles and technical effects, which will not be described here.

[0270] On the basis of the above-mentioned embodiments, the device further includes a sixth acquisition module, wherein:

[0271] The sixth acquisition module is configured to obtain a size of the detection result of the camera data according to a size of the positioning frame of the detection result of the camera data.

[0272] The camera data detection result enhancement device provided in the embodiment can execute the method embodiments, and has similar implementation principles and technical effects, which will not be described here.

[0273] On the basis of the above-mentioned embodiments, optionally, the apparatus further comprises a seventh acquisition module, an eighth acquisition module, a ninth acquisition module, a tenth acquisition module, a first establishment module, a fitting module, a matching module, and a second establishment module, wherein:

[0274] The seventh acquisition module is configured to acquire second sample point cloud data.

[0275] The eighth acquisition module is configured to convert the second sample point cloud data to a camera coordinate system by using an external conversion matrix of the laser radar to the camera coordinate system, to obtain processed second sample point cloud data; the processed second sample point cloud data is second sample point cloud data from which points that are not within the camera collection angle and points that are not successfully received by the laser radar are filtered out.

[0276] The ninth acquisition module is configured to convert the processed second sample point cloud data to a polar coordinate system, to obtain polar angles of the processed second sample point cloud data.

[0277] The tenth acquisition module is configured to add padding points to two points with equal polar angles of the processed second sample point cloud data by equally dividing the polar length, to obtain padded second sample point cloud data.

[0278] The first establishment module is configured to convert the padded second sample point cloud data from a point cloud coordinate system to a pixel coordinate system, to establish a mapping relationship pair of depth points of the padded second sample point cloud data and pixel points in the pixel coordinate system.

[0279] The fitting module is configured to fit pixel coordinate information corresponding to the padded second sample point cloud data according to the mapping relationship pair of the depth points of the padded second sample point cloud data and the pixel points in the pixel coordinate system, to obtain calibration parameters.

[0280] The matching module is configured to use the calibration parameters to match third sample point cloud data and third sample camera data of the same sample detection region.

[0281] The second establishment module is configured to establish a mapping relationship according to a matching result of the third sample point cloud data and the third sample camera data.

[0282] The camera data detection result enhancement apparatus provided in this embodiment can execute the above-mentioned method embodiments, and has similar implementation principles and technical effects, which will not be described here again.

[0283] The specific definition of the enhancement device for the camera data detection result can refer to the definition of the enhancement method for the camera data detection result in the foregoing, which will not be described here. Each module in the enhancement device for the camera data detection result can be implemented by software, hardware, or a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so that the processor calls and executes the operations corresponding to each module.

[0284] In one embodiment, a computer device is provided, comprising a memory and a processor, the memory storing a computer program, and the processor implementing the following steps when executing the computer program:

[0285] obtaining camera data of a detection region;

[0286] inputting the camera data into a preset detection model to obtain an enhanced detection result of the camera data; wherein the enhanced detection result of the camera data is a result including three-dimensional physical information; the detection model is obtained by mapping first sample point cloud data to first sample camera data, associating and matching the mapping result of the first sample point cloud data and the detection result of the first sample camera data according to the similarity between the mapping result of the first sample point cloud data and the detection result of the first sample camera data, and training a preset initial detection model according to the association and matching result; the first sample camera data and the first sample point cloud data are data of the same detection region.

[0287] The computer device provided in the above embodiment has similar implementation principles and technical effects to the above method embodiments, which will not be described here.

[0288] In one embodiment, a computer device is provided, comprising a memory and a processor, the memory storing a computer program, and the processor implementing the following steps when executing the computer program:

[0289] obtaining point cloud data and camera data of a detection region;

[0290] detecting the point cloud data to obtain a detection result of the point cloud data;

[0291] detecting the camera data to obtain a detection result of the camera data;

[0292] associating and matching the detection result of the point cloud data and the detection result of the camera data according to a pre-established mapping relationship;

[0293] enhancing the detection result of the camera data according to the association and matching result to obtain an enhanced detection result of the camera data; wherein the enhanced detection result of the camera data is a result including three-dimensional physical information.

[0294] The computer device provided in the above embodiment has similar implementation principles and technical effects to the above method embodiments, and thus detailed description is omitted here.

[0295] In one embodiment, a computer readable storage medium is provided, which stores a computer program. The computer program is executed by a processor to implement the following steps:

[0296] Obtaining camera data of a detection area;

[0297] Inputting the camera data into a preset detection model to obtain an enhanced detection result of the camera data; wherein the enhanced detection result of the camera data is a result including three-dimensional physical information; the detection model is obtained by mapping first sample point cloud data to first sample camera data, associating and matching the mapping result of the first sample point cloud data and the detection result of the first sample camera data according to the similarity between the mapping result of the first sample point cloud data and the detection result of the first sample camera data, and training a preset initial detection model according to the associating and matching result; the first sample camera data and the first sample point cloud data are data of the same detection area.

[0298] The computer readable storage medium provided in the above embodiment has similar implementation principles and technical effects to the above method embodiments, and thus detailed description is omitted here.

[0299] In one embodiment, a computer readable storage medium is provided, which stores a computer program. The computer program is executed by a processor to implement the following steps:

[0300] Obtaining point cloud data and camera data of a detection area;

[0301] Detecting the point cloud data to obtain a detection result of the point cloud data;

[0302] Detecting the camera data to obtain a detection result of the camera data;

[0303] According to a pre-established mapping relationship, associating and matching the detection result of the point cloud data and the detection result of the camera data;

[0304] According to the associating and matching result, enhancing the detection result of the camera data to obtain an enhanced detection result of the camera data; wherein the enhanced detection result of the camera data is a result including three-dimensional physical information.

[0305] The computer readable storage medium provided in the above embodiment has similar implementation principles and technical effects to the above method embodiments, and thus detailed description is omitted here.

[0306] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, storage, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0307] The technical features of the above embodiments can be combined in any way. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, but as long as the combinations of the technical features do not exist, they should be considered as the scope of the present application.

[0308] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be pointed out that for those skilled in the art, without departing from the concept of the present application, some modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of the patent of the present application should be subject to the appended claims.

Claims

1. A method for enhancing camera data detection results, characterized in that, The method includes: Acquire point cloud data and camera data of the detection area; The point cloud data is inspected to obtain the inspection results of the point cloud data; The camera data is detected to obtain the detection result of the camera data; Based on the pre-established mapping relationship, the detection results of the point cloud data and the detection results of the camera data are associated and matched; Based on the association matching results, the detection results of the camera data are enhanced to obtain the enhanced detection results of the camera data; wherein, the enhanced detection results of the camera data include three-dimensional physical information; The step of enhancing the detection results of the camera data based on the association matching results to obtain enhanced detection results of the camera data includes: If the association matching result is that the detection result of the point cloud data and the detection result of the camera data are successfully matched, then the detection result of the point cloud data is mapped to the detection result of the camera data to obtain the enhanced detection result of the camera data; If the correlation matching result shows that the detection result of the point cloud data and the detection result of the camera data do not match successfully, then the detection result of the point cloud data is determined as the enhanced detection result of the camera data.

2. The method according to claim 1, characterized in that, The step of associating and matching the detection results of the point cloud data and the detection results of the camera data according to the pre-established mapping relationship includes: Based on the mapping relationship, the detection results of the point cloud data are mapped onto the camera data to obtain the mapping results of the point cloud data detection results; Calculate the first similarity between the mapping result of the detection result of the point cloud data and the detection result of the camera data; Based on the first similarity, the detection results of the point cloud data and the detection results of the camera data are correlated and matched.

3. The method according to claim 1, characterized in that, The step of associating and matching the detection results of the point cloud data and the detection results of the camera data according to the pre-established mapping relationship includes: Based on the mapping relationship, the camera data is mapped to the point cloud data coordinate system to obtain the mapping result of the camera data; Calculate the second similarity between the mapping result of the camera data and the detection result of the point cloud data; Based on the second similarity, the detection results of the point cloud data and the detection results of the camera data are correlated and matched.

4. The method according to claim 1, characterized in that, The method further includes: Based on the location box of the detection result of the camera data, determine the location point of the detection result of the camera data in the point cloud coordinate system; Based on the positioning points and the pre-established fitting positioning relationship, the position of the detection result of the camera data in the point cloud coordinate system is determined; By using the video tracking algorithm and the position of the camera data detection results in the point cloud coordinate system, the camera data detection results are correlated and matched to obtain the driving trajectory of the camera data detection results in the point cloud coordinate system. Based on the travel trajectory of the camera data detection results in the point cloud coordinate system, feature information of the camera data detection results is obtained; the feature information of the camera data detection results includes at least one of the following: the position of the camera data detection results, the velocity of the camera data detection results, the heading angle of the camera data detection results, and the acceleration of the camera data detection results.

5. The method according to claim 4, characterized in that, The method further includes: The size of the camera data detection result is obtained based on the size of the positioning frame of the detection result.

6. The method according to claim 2 or 3, characterized in that, The method further includes: Obtain the second sample point cloud data; The second sample point cloud data is transformed to the camera coordinate system using an external transformation matrix from the lidar to the camera coordinate system, resulting in processed second sample point cloud data. The processed second sample point cloud data is obtained by filtering out points that are not within the camera's field of view or that the lidar failed to receive. The processed second sample point cloud data is converted to polar coordinates to obtain the polar angle of the processed second sample point cloud data; For two points with equal polar angles in the processed second sample point cloud data, fill points are added by equally dividing the polar length to obtain the filled second sample point cloud data. The filled second sample point cloud data is transformed from the point cloud coordinate system to the pixel coordinate system, and a mapping relationship is established between the depth points of the filled second sample point cloud data and the pixels in the pixel coordinate system. Based on the mapping relationship between the depth points of the filled second sample point cloud data and the pixels in the pixel coordinate system, the pixel coordinate information corresponding to the filled second sample point cloud data is fitted to obtain the calibration parameters. Using the calibration parameters, the point cloud data of the third sample and the camera data of the third sample in the same sample detection area are matched; The mapping relationship is established based on the matching results of the third sample point cloud data and the third sample camera data.

7. An enhancement device for camera data detection results, characterized in that, The device includes: The first acquisition module is used to acquire point cloud data and camera data of the detection area; The second acquisition module detects the point cloud data and obtains the detection result of the point cloud data; The third acquisition module detects the camera data and obtains the detection result of the camera data; The association module performs association matching between the detection results of the point cloud data and the detection results of the camera data based on the pre-established mapping relationship; The enhancement module enhances the detection results of the camera data based on the association matching results to obtain enhanced detection results of the camera data; wherein, the enhanced detection results of the camera data include three-dimensional physical information. The enhancement module is specifically used to map the detection result of the point cloud data to the detection result of the camera data if the association matching result is a successful match between the detection result of the point cloud data and the detection result of the camera data, thereby obtaining the enhanced detection result of the camera data. If the correlation matching result shows that the detection result of the point cloud data and the detection result of the camera data do not match successfully, then the detection result of the point cloud data is determined as the enhanced detection result of the camera data.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images