A target identification method, device, apparatus and storage medium

Abstract

The application relates to the technical field of roadside perception, and discloses a target identification method, device and equipment and a storage medium, historical point cloud data corresponding to a road scene is acquired, current point cloud data corresponding to the road scene at a current time and a current image are acquired; the current point cloud data is filtered according to a drivable three-dimensional space range, radar calibration parameters and the historical point cloud data, so as to obtain to-be-identified point cloud data; the current image and the to-be-identified point cloud data are analyzed according to a BEV fusion perception algorithm, so as to obtain an initial target detection result; the to-be-identified point cloud data is subjected to multi-scale clustering processing according to a plurality of distance thresholds, so as to determine a target three-dimensional inclusive frame; the target three-dimensional inclusive frame and the initial target detection result are integrated, so as to obtain a final target detection result; the target detection result contains a recognized vehicle type and a corresponding three-dimensional inclusive frame, and the accurate identification of a target in a complex roadside scene is ensured.

Description

Technical Field

[0001] This invention relates to the field of roadside sensing technology, and in particular to a target identification method, apparatus, device, and storage medium. Background Technology

[0002] With the rapid advancement of urban modernization and sustained economic development, people's travel habits are changing, and the number of private cars is increasing rapidly year by year. In order to improve road traffic efficiency and ensure safety, Intelligent Traffic Systems (ITS) have been introduced into the road traffic field, effectively applying intelligent technologies such as artificial intelligence, sensor technology, and computer technology to the fields of transportation and traffic supervision.

[0003] As a crucial component of ITS (Information Technology System), the level of intelligent management in roadside parking management systems directly impacts the quality of urban travel experiences. However, due to the open nature of parking spaces and the complexity of parking environments, the current level of intelligence in roadside parking management systems is not high. Early methods often employed multi-camera combined fields of view to monitor multiple consecutive parking spaces. Later, methods based on deep learning were developed to identify and estimate the positions of a vehicle's four wheels in images, replacing the method of describing vehicles using two-dimensional (2D) rectangular bounding boxes. In principle, this can improve the accuracy of determining whether a vehicle is within a parking space. However, the accuracy of vehicle tire detection, especially for estimating the position of obscured tires, is low, thus limiting the practicality of such methods.

[0004] In recent years, with the gradual addition of sensors such as LiDAR and millimeter-wave radar, multi-sensor fusion perception technology has developed rapidly. The algorithm dimension has expanded from two-dimensional to three-dimensional (3D) point clouds. However, these algorithms are highly dependent on data, and data annotation is extremely difficult and costly. Furthermore, the algorithm performance deteriorates significantly under occlusion, making it difficult to meet practical requirements. Although current pure vision solutions based on multi-camera combinations and multi-source sensor fusion perception solutions both use deep learning models, these models can only recognize fixed types of vehicles and cannot identify common non-standard vehicles in road scenes, such as engineering vehicles. Vehicle type, vehicle trajectory, and license plate information can all be considered as targets to be identified. Current implementations suffer from inaccurate target identification due to factors such as vehicle occlusion and the model's limitation to recognizing only fixed types of vehicles.

[0005] It is evident that how to accurately identify targets in complex roadside scenarios is a problem that needs to be solved by those skilled in the art. Summary of the Invention

[0006] The purpose of this invention is to provide a target recognition method, apparatus, device, and storage medium that can solve the problem of low accuracy in target recognition in complex roadside scenarios.

[0007] To address the aforementioned technical problems, embodiments of the present invention provide a target recognition method, comprising:

[0008] Acquire historical point cloud data corresponding to the road scene, acquire current point cloud data and current image corresponding to the road scene at the current time;

[0009] Based on the pre-set drivable three-dimensional space range, radar calibration parameters, and historical point cloud data, the current point cloud data is filtered to obtain the point cloud data to be identified.

[0010] The current image and the point cloud data to be identified are analyzed according to the BEV fusion perception algorithm to obtain the initial target detection result;

[0011] The point cloud data to be identified is subjected to multi-scale clustering processing according to multiple set distance thresholds to determine the target three-dimensional bounding box;

[0012] The target 3D bounding box and the initial target detection results are integrated to obtain the final target detection result; wherein, the target detection result includes the identified vehicle type and its corresponding 3D bounding box.

[0013] On the one hand, before filtering the current point cloud data according to the pre-set drivable three-dimensional space range, radar calibration parameters, and historical point cloud data to obtain the point cloud data to be identified, the process also includes:

[0014] The historical point cloud data and the current point cloud data are matched using a point cloud matching algorithm to determine the first transformation parameters; wherein, the first transformation parameters include a first rotation matrix and a first translation vector;

[0015] The deviation of the current point cloud data is determined based on the first rotation matrix and the first translation vector;

[0016] Based on the set deviation requirements, the deviation, the first rotation matrix, the first translation vector, and the current point cloud data are processed to obtain the final current point cloud data.

[0017] On one hand, the process of processing the deviation, the first rotation matrix, the first translation vector, and the current point cloud data based on the set deviation requirement to obtain the final current point cloud data includes:

[0018] Determine whether the deviation is less than the deviation threshold;

[0019] If the deviation is less than the deviation threshold, the current point cloud data is used as the final current point cloud data.

[0020] If the deviation is greater than or equal to the deviation threshold, the current point cloud data is transformed using the first rotation matrix and the first translation vector to obtain pose-calibrated point cloud data, and the pose-calibrated point cloud data is used as the final current point cloud data.

[0021] On the one hand, the acquisition of historical point cloud data corresponding to the road scene includes:

[0022] Acquire the initial point cloud data corresponding to different times within a set time period for the road scene;

[0023] The point cloud matching algorithm is used to sequentially match the initial point cloud data at adjacent time points to determine the initial transformation parameters corresponding to each adjacent time point; wherein, the initial transformation parameters include an initial rotation matrix and an initial translation vector;

[0024] The initial deviation is determined based on the initial rotation matrix and the initial translation vector corresponding to each of the adjacent time moments.

[0025] Based on the aforementioned deviation requirements, the initial deviations, initial rotation matrices, initial translation vectors, and initial point cloud data are processed to obtain the historical point cloud data.

[0026] On one hand, based on the deviation requirement, the initial deviation, the initial rotation matrix, the initial translation vector, and the initial point cloud data are processed to obtain the historical point cloud data, including:

[0027] Determine whether the initial deviation corresponding to each of the adjacent time points is less than the deviation threshold;

[0028] If the initial deviation at each of the adjacent time points is less than the deviation threshold, the initial point cloud data are superimposed to form the historical point cloud data.

[0029] If the initial deviation at each of the adjacent time points is not less than the deviation threshold, the initial point cloud data corresponding to each of the initial rotation matrices and the initial translation vectors are used to transform the initial point cloud data to obtain the pose-calibrated initial point cloud data, and the pose-calibrated initial point cloud data are superimposed to obtain the historical point cloud data.

[0030] On the one hand, before analyzing the current image and the point cloud data to be identified according to the BEV fusion perception algorithm to obtain the initial target detection result, the method further includes:

[0031] According to the set unified coordinate system, the current image and all the point cloud data to be identified are subjected to coordinate transformation to obtain the coordinate transformed image and the coordinate transformed point cloud data to be identified.

[0032] The step of analyzing the current image and the point cloud data to be identified according to the BEV fusion perception algorithm to obtain the initial target detection result includes:

[0033] The BEV fusion sensing algorithm is used to analyze the coordinate-transformed image and the coordinate-transformed point cloud data to be identified in order to obtain the initial target detection results.

[0034] On the one hand, before performing coordinate transformation on the current image and all the point cloud data to be identified according to the set unified coordinate system to obtain the coordinate-transformed image and the coordinate-transformed point cloud data to be identified, the method further includes:

[0035] Based on the camera's intrinsic parameters, distortion correction is performed on the initial image captured by the camera to obtain a corrected image;

[0036] The current point cloud data and the pose-calibrated point cloud data are projected onto the corrected image to obtain a first image with the same name and a second image with the same name.

[0037] Based on the matching relationship of corresponding points between the first and second identical images, the image calibration affine transformation parameters are determined.

[0038] The initial image is transformed using the image calibration affine transformation parameters to obtain a pose-calibrated image, and the pose-calibrated image is used as the current image.

[0039] On the one hand, the analysis of the current image and the point cloud data to be identified based on the BEV fusion perception algorithm to obtain the initial target detection result includes:

[0040] The BEV fusion sensing algorithm is used to extract features from the coordinate-transformed image to obtain three-dimensional image spatial features;

[0041] The BEV fusion sensing algorithm is used to extract features from the coordinate-transformed point cloud data to obtain three-dimensional point cloud spatial features.

[0042] The aligned 3D image spatial features and the 3D point cloud spatial features are fused to obtain fused features;

[0043] The fused features are analyzed using the target detection head included in the BEV fusion perception algorithm to output the initial target detection result; wherein, the initial target detection result includes the identified targets and their pose information.

[0044] On the one hand, the filtering process performed on the current point cloud data based on the pre-set drivable three-dimensional space range, radar calibration parameters, and historical point cloud data to obtain the point cloud data to be identified includes:

[0045] Using a pre-defined drivable three-dimensional space range, extract drivable area point cloud data from the current point cloud data;

[0046] Based on the radar calibration parameters and the historical point cloud data, the drivable area point cloud data is filtered to obtain the point cloud data to be identified.

[0047] On one hand, the step of filtering the feasible area point cloud data according to the radar calibration parameters and the historical point cloud data to obtain the point cloud data to be identified includes:

[0048] Ground point cloud data is filtered out from the drivable area point cloud data according to the radar calibration parameters to obtain initial filtered point cloud data;

[0049] The historical point cloud data is matched with the initial filtered point cloud data to filter out background point cloud data from the initial filtered point cloud data.

[0050] The initial filtered point cloud data after removing the background point cloud data is used as the point cloud data to be identified.

[0051] On the one hand, the step of performing multi-scale clustering processing on the point cloud data to be identified according to multiple set distance thresholds to determine the target three-dimensional bounding box includes:

[0052] Spatial clustering processing is performed on the point cloud data to be identified according to each of the distance thresholds to obtain the identification result corresponding to each distance threshold.

[0053] Based on the set combination method and the target spatial location and target size contained in each of the recognition results, the recognition results are merged to obtain multiple merged results;

[0054] Construct a grid according to the set yaw angle interval and spatial position interval;

[0055] The merged results are projected onto the grid to determine the optimal merged result, and the optimal merged result is used as the target 3D bounding box; wherein, the target 3D bounding box includes the optimal target center position coordinates, the optimal target size, and the optimal target yaw angle.

[0056] On the one hand, based on the set combination method and the target spatial location and target size contained in each of the recognition results, the recognition results are merged to obtain multiple merged results, including:

[0057] By combining them in pairs, the overlap relationship between the target spatial location and target size contained in each of the identification results under each combination is identified; wherein, the target spatial location includes the center position coordinates of the target, and the target size includes the length, width and height of the target;

[0058] Based on the overlap relationship corresponding to each combination, the recognition results under each combination are merged to obtain the merged result corresponding to each combination.

[0059] On one hand, projecting each of the merged results onto the grid to determine the optimal merged result includes:

[0060] The first target contained in each of the merging results is projected onto the grid; wherein, the first target is any identical target contained in each of the merging results;

[0061] The yaw angle corresponding to the grid cell that projects the most point cloud data of the first target in the grid is taken as the optimal target yaw angle.

[0062] On the one hand, the integration of the target's three-dimensional bounding box and the initial target detection results to obtain the final target detection result includes:

[0063] Based on the target 3D bounding box and the image of the parking space corresponding to the parking space number closest to the target 3D bounding box, target filtering is performed on the target 3D bounding box to obtain the suspected vehicle target recognition result;

[0064] Based on the consistency judgment rules, the vehicle 3D bounding boxes contained in the suspected vehicle target recognition results and the initial target detection results are analyzed to determine the consistency.

[0065] Based on the merging method matched by the consistency, the vehicle 3D bounding boxes contained in the suspected vehicle target recognition result and the initial target detection result are fused to obtain the final target detection result.

[0066] On one hand, the step of filtering the target three-dimensional bounding box based on the target three-dimensional bounding box and the image of the parking space corresponding to the parking space number closest to the target three-dimensional bounding box to obtain the suspected vehicle target recognition result includes:

[0067] Obtain the image of the berth number that is closest to the target 3D bounding box;

[0068] The point cloud data corresponding to the target 3D bounding box is back-projected onto the berth image to determine the first target rectangular region;

[0069] The target detection algorithm is used to identify the target in the berth image to determine the second target rectangular region;

[0070] Based on the first target rectangular region and the second target rectangular region, the target three-dimensional bounding box is filtered to obtain the suspected vehicle target recognition result.

[0071] On the one hand, the step of filtering the target's three-dimensional bounding box based on the first target rectangular region and the second target rectangular region to obtain the suspected vehicle target recognition result includes:

[0072] Calculate the intersection-union ratio of the first target rectangular region and the second target rectangular region;

[0073] If the intersection-union ratio is greater than or equal to the upper limit of the intersection-union ratio, the point cloud data corresponding to the first target rectangular region is filtered out from the point cloud data corresponding to the target three-dimensional bounding box.

[0074] On the one hand, based on the consistency judgment rules, the three-dimensional bounding boxes of the vehicles included in the suspected vehicle target recognition results and the initial target detection results are analyzed to determine the consistency, which includes:

[0075] The consistency score is obtained by applying the consistency calculation formula to process the vehicle 3D bounding boxes contained in the suspected vehicle target recognition result and the initial target detection result, respectively; the consistency calculation formula is as follows:

[0076]

[0077] Where J represents the degree of consistency. This indicates the three-dimensional bounding box of the vehicle included in the suspected vehicle target recognition result. This indicates the vehicle's 3D bounding box included in the initial target detection result. This represents the intersection-union ratio (IoU) of the vehicle 3D bounding boxes included in the suspected vehicle target recognition result and the vehicle 3D bounding boxes included in the initial target detection result. This indicates the yaw angle included in the suspected vehicle target identification result. This indicates the yaw angle included in the initial target detection result.

[0078] On the one hand, the merging method based on the consistency degree, which fuses the vehicle 3D bounding boxes contained in the suspected vehicle target recognition result and the initial target detection result respectively, to obtain the final target detection result includes:

[0079] If the consistency is greater than the first threshold, the initial target detection result is used as the final target detection result;

[0080] When the consistency is less than or equal to the first threshold and greater than the second threshold, the vehicle three-dimensional bounding boxes contained in the suspected vehicle target recognition result and the initial target detection result are respectively back-projected onto the camera image to obtain the corresponding suspected vehicle rectangular region and the initial vehicle rectangular region.

[0081] Based on the image features contained in the suspected vehicle rectangular region and the initial vehicle rectangular region, the vehicle 3D bounding boxes contained in the suspected vehicle target recognition result and the initial target detection result are merged to obtain the final target detection result.

[0082] On the one hand, after integrating the target's three-dimensional bounding box and the initial target detection results to obtain the final target detection result, the method further includes:

[0083] Obtain the kinematic parameters of the target within a preset time period before the current moment; wherein, the kinematic parameters include the horizontal velocity and the angular velocity of the yaw angle change;

[0084] The kinematic parameters are analyzed using a target tracking algorithm to determine a prediction result that includes the estimated pose information of the target at the current moment;

[0085] Based on the consistency judgment rules, the three-dimensional bounding box contained in the final target detection result and the three-dimensional bounding box corresponding to the prediction result are analyzed to determine a new consistency.

[0086] Based on the merging method matched by the new consistency, the 3D bounding box contained in the final target detection result and the 3D bounding box corresponding to the prediction result are fused to obtain the integrated result.

[0087] On the one hand, after fusing the 3D bounding box contained in the final target detection result and the 3D bounding box corresponding to the prediction result based on the merging method matched by the new consistency, to obtain the integrated result, the method further includes:

[0088] Based on the integration results and the three-dimensional spatial relationship of the berths, the usage status of the berths is determined.

[0089] On the one hand, after fusing the 3D bounding box contained in the final target detection result and the 3D bounding box corresponding to the prediction result based on the merging method matched by the new consistency, to obtain the integrated result, the method further includes:

[0090] Feature association analysis is performed on the initial target detection results and the integrated results to obtain the vehicle's three-dimensional trajectory information.

[0091] On the one hand, after performing feature association analysis on the initial target detection results and the integrated results to obtain the vehicle's three-dimensional trajectory information, the method further includes:

[0092] The vehicle's three-dimensional trajectory information is verified in chronological order, and the three-dimensional trajectory segments belonging to the same vehicle are selected.

[0093] The three-dimensional bounding box corresponding to each vehicle in each of the three-dimensional trajectory segments is back-projected onto the camera image corresponding to the nearest parking space in the three-dimensional space to obtain the rectangular box of each vehicle.

[0094] Using the feature information contained in each of the vehicle bounding boxes, each of the three-dimensional trajectory segments is fitted to obtain complete three-dimensional trajectory information.

[0095] On the one hand, the process of fitting each of the three-dimensional trajectory segments using the feature information contained in each of the vehicle bounding boxes to obtain complete three-dimensional trajectory information includes:

[0096] Extract the license plate information contained in the rectangular frames of each vehicle;

[0097] If the license plate information contained in each of the vehicle rectangles is consistent, the three-dimensional trajectory segments are merged to obtain complete three-dimensional trajectory information.

[0098] If the license plate information contained in each of the vehicle rectangles is inconsistent, extract the vehicle texture information contained in each of the vehicle rectangles.

[0099] If the vehicle texture information contained in each of the vehicle rectangles matches, the three-dimensional trajectory segments are merged to obtain complete three-dimensional trajectory information.

[0100] In the case where the vehicle texture information contained in each of the vehicle rectangles does not match, the smoothing point on each of the three-dimensional trajectory segments is determined according to the angle of the curve tangent vector of each point on each of the three-dimensional trajectory segments.

[0101] The line segments formed by the smooth points on each of the three-dimensional trajectory segments and the endpoints on each of the three-dimensional trajectory segments are fitted to construct the predicted trajectory line;

[0102] The three-dimensional trajectory segments are merged with the predicted trajectory line to obtain complete three-dimensional trajectory information.

[0103] On one hand, fitting the line segments formed by the smoothing points on each of the three-dimensional trajectory segments and the endpoints on each of the three-dimensional trajectory segments to construct the predicted trajectory line includes:

[0104] The first trajectory segment is constructed based on the smoothing point on the first three-dimensional trajectory segment and the end point on the first three-dimensional trajectory segment;

[0105] A second trajectory segment is constructed based on the smoothing point on the second three-dimensional trajectory segment and the starting endpoint on the second three-dimensional trajectory segment; wherein, the first three-dimensional trajectory segment and the second three-dimensional trajectory segment are two three-dimensional trajectory segments that are adjacent in time on each of the three-dimensional trajectory segments;

[0106] The first trajectory segment is fitted to obtain the first trajectory prediction line;

[0107] The second trajectory segment is fitted to obtain the second trajectory prediction line;

[0108] If the vertical distance between the first trajectory prediction line and the second trajectory prediction line is less than a set distance threshold, the first trajectory prediction line and the second trajectory prediction line are fitted to determine the predicted trajectory line between the end endpoint of the first three-dimensional trajectory segment and the start endpoint of the second three-dimensional trajectory segment.

[0109] On the one hand, after fitting each of the three-dimensional trajectory segments using the feature information contained in each of the vehicle bounding boxes to obtain complete three-dimensional trajectory information, the method further includes:

[0110] Based on the complete three-dimensional trajectory information of each vehicle and its three-dimensional positional relationship with the parking space, the status and time information of each vehicle entering and leaving the parking space are determined.

[0111] On the one hand, after fitting each of the three-dimensional trajectory segments using the feature information contained in each of the vehicle bounding boxes to obtain complete three-dimensional trajectory information, the method further includes:

[0112] Obtain the original camera images corresponding to the complete 3D trajectory information of each vehicle;

[0113] Based on the boundary clarity of the license plate area, the license plate envelope tilt, and / or the number of license plate character segments in each of the original camera images, license plate area images are selected from each of the original camera images;

[0114] License plate information is extracted from the license plate area image.

[0115] This invention also provides a target recognition device, including an acquisition unit, a filtering unit, an analysis unit, a clustering unit, and an integration unit;

[0116] The acquisition unit is used to acquire historical point cloud data corresponding to the road scene, acquire current point cloud data and current image corresponding to the road scene at the current time;

[0117] The filtering unit is used to filter the current point cloud data according to the preset drivable three-dimensional space range, radar calibration parameters and historical point cloud data to obtain the point cloud data to be identified.

[0118] The analysis unit is used to analyze the current image and the point cloud data to be identified according to the BEV fusion perception algorithm to obtain the initial target detection result;

[0119] The clustering unit is used to perform multi-scale clustering processing on the point cloud data to be identified according to multiple set distance thresholds in order to determine the target three-dimensional bounding box.

[0120] The integration unit is used to integrate the target 3D bounding box and the initial target detection result to obtain the final target detection result; wherein, the target detection result includes the identified vehicle type and its corresponding 3D bounding box.

[0121] On the one hand, it also includes a first matching unit, a first deviation determination unit, and a first processing unit;

[0122] The first matching unit is used to match the historical point cloud data and the current point cloud data using a point cloud matching algorithm to determine the first transformation parameters; wherein, the first transformation parameters include a first rotation matrix and a first translation vector;

[0123] The first deviation determination unit is used to determine the deviation of the current point cloud data based on the first rotation matrix and the first translation vector;

[0124] The first processing unit is used to process the deviation, the first rotation matrix, the first translation vector, and the current point cloud data based on a set deviation requirement to obtain the final current point cloud data.

[0125] On one hand, the processing unit is used to determine whether the deviation is less than the deviation threshold; if the deviation is less than the deviation threshold, the current point cloud data is used as the final current point cloud data; if the deviation is greater than or equal to the deviation threshold, the current point cloud data is transformed using the first rotation matrix and the first translation vector to obtain pose-calibrated point cloud data, and the pose-calibrated point cloud data is used as the final current point cloud data.

[0126] On the one hand, regarding the acquisition of the historical point cloud data, the device includes an acquisition unit, a second matching unit, a second deviation determination unit, and a second processing unit;

[0127] The acquisition unit is used to acquire the initial point cloud data corresponding to different times within a set time period in the road scene;

[0128] The second matching unit is used to sequentially match the initial point cloud data at adjacent time points using the point cloud matching algorithm to determine the initial transformation parameters corresponding to each adjacent time point; wherein, the initial transformation parameters include an initial rotation matrix and an initial translation vector;

[0129] The second deviation determination unit is used to determine the initial deviation corresponding to each of the adjacent time moments based on the initial rotation matrix and the initial translation vector corresponding to each of the adjacent time moments.

[0130] The second processing unit is used to process each of the initial deviations, each of the initial rotation matrices, each of the initial translation vectors, and the initial point cloud data based on the deviation requirements to obtain the historical point cloud data.

[0131] On one hand, the second processing unit is used to determine whether the initial deviation corresponding to each of the adjacent time moments is less than the deviation threshold; if the initial deviation corresponding to each of the adjacent time moments is less than the deviation threshold, the initial point cloud data is superimposed as the historical point cloud data; if the initial deviation corresponding to each of the adjacent time moments is not less than the deviation threshold, the initial point cloud data corresponding to each of the initial rotation matrices and the initial translation vectors are used to transform the initial point cloud data corresponding to each of the two to obtain the pose-calibrated initial point cloud data, and the pose-calibrated initial point cloud data is superimposed as the historical point cloud data.

[0132] On the one hand, it also includes coordinate transformation units;

[0133] The coordinate transformation unit is used to perform coordinate transformation on the current image and all the point cloud data to be identified according to a set unified coordinate system, so as to obtain the coordinate transformed image and the coordinate transformed point cloud data to be identified.

[0134] The analysis unit is used to analyze the coordinate-transformed image and the coordinate-transformed point cloud data to be identified according to the BEV fusion perception algorithm to obtain the initial target detection result.

[0135] On the one hand, it also includes a correction unit, a projection unit, a parameter determination unit, and a calibration unit;

[0136] The correction unit is used to perform distortion correction on the initial image acquired by the camera based on the camera's intrinsic parameters to obtain a corrected image.

[0137] The projection unit is used to project the current point cloud data and the pose-calibrated point cloud data onto the corrected image, respectively, to obtain a first image with the same name and a second image with the same name.

[0138] The parameter determination unit is used to determine the image calibration affine transformation parameters based on the matching relationship of corresponding points between the first image with the same name and the second image with the same name.

[0139] The calibration unit is used to transform the initial image using the image calibration affine transformation parameters to obtain a pose-calibrated image, and to use the pose-calibrated image as the current image.

[0140] On one hand, the analysis unit is used to extract features from the coordinate-transformed image using the BEV fusion sensing algorithm to obtain three-dimensional image spatial features; to extract features from the coordinate-transformed point cloud data to be identified using the BEV fusion sensing algorithm to obtain three-dimensional point cloud spatial features; to fuse the aligned three-dimensional image spatial features and the three-dimensional point cloud spatial features to obtain fused features; and to analyze the fused features using the target detection head included in the BEV fusion sensing algorithm to output the initial target detection result; wherein the initial target detection result includes the identified targets and their pose information.

[0141] On one hand, the filtering unit is used to extract drivable area point cloud data from the current point cloud data using a pre-set drivable three-dimensional space range; and to perform filtering processing on the drivable area point cloud data according to the radar calibration parameters and the historical point cloud data to obtain the point cloud data to be identified.

[0142] On one hand, the filtering unit is used to filter out ground point cloud data from the drivable area point cloud data according to the radar calibration parameters to obtain initial filtered point cloud data; match the historical point cloud data with the initial filtered point cloud data to filter out background point cloud data from the initial filtered point cloud data; and use the initial filtered point cloud data after filtering out the background point cloud data as the point cloud data to be identified.

[0143] On the one hand, the clustering unit includes a processing subunit, a merging subunit, a construction subunit, and a projection subunit;

[0144] The processing subunit is used to perform spatial clustering processing on the point cloud data to be identified according to each of the distance thresholds, so as to obtain the identification result corresponding to each of the distance thresholds.

[0145] The merging subunit is used to merge the recognition results based on the set combination method and the target spatial location and target size contained in each recognition result, so as to obtain multiple merged results;

[0146] The construction sub-unit is used to construct a grid according to the set yaw angle interval and spatial position interval;

[0147] The projection subunit is used to project each of the merging results onto the grid to determine the optimal merging result, and use the optimal merging result as the target three-dimensional bounding box; wherein, the target three-dimensional bounding box includes the optimal target center position coordinates, the optimal target size, and the optimal target yaw angle.

[0148] On the one hand, the merging subunit is used to identify the overlap relationship between the target spatial position and target size contained in each of the identification results under each combination in a pairwise combination manner; wherein, the target spatial position includes the center position coordinates of the target, and the target size includes the length, width and height of the target; according to the overlap relationship corresponding to each combination, the identification results under each combination are merged to obtain the merging result corresponding to each combination.

[0149] On one hand, the projection subunit is used to project the first target contained in each of the merging results onto the grid; wherein, the first target is any identical target contained in each of the merging results; and the yaw angle corresponding to the grid cell in the grid that projects the most point cloud data of the first target is taken as the optimal target yaw angle.

[0150] On the one hand, the integration unit includes a filtering subunit, an analysis subunit, and a fusion subunit;

[0151] The filtering subunit is used to filter the target three-dimensional bounding box based on the target three-dimensional bounding box and the image of the parking space corresponding to the parking space number closest to the target three-dimensional bounding box, so as to obtain the suspected vehicle target recognition result.

[0152] The analysis subunit is used to analyze the vehicle 3D bounding boxes contained in the suspected vehicle target recognition result and the initial target detection result respectively according to the consistency judgment rule, so as to determine the consistency.

[0153] The fusion subunit is used to fuse the vehicle 3D bounding boxes contained in the suspected vehicle target recognition result and the initial target detection result respectively, based on the merging method matched by the consistency degree, so as to obtain the final target detection result.

[0154] On one hand, the filtering subunit is used to acquire the berth image corresponding to the berth number closest to the target three-dimensional bounding box; back-project the point cloud data corresponding to the target three-dimensional bounding box onto the berth image to determine the first target rectangular region; use a target detection algorithm to perform target recognition on the berth image to determine the second target rectangular region; and perform target filtering on the target three-dimensional bounding box based on the first target rectangular region and the second target rectangular region to obtain the suspected vehicle target recognition result.

[0155] On one hand, the filtering subunit is used to calculate the intersection-union ratio (IUR) of the first target rectangular region and the second target rectangular region; when the IUR is greater than or equal to the upper limit of the IUR, the point cloud data corresponding to the first target rectangular region is filtered out from the point cloud data corresponding to the target three-dimensional bounding box.

[0156] On one hand, the analysis subunit is used to call the consistency calculation formula to process the vehicle 3D bounding boxes contained in the suspected vehicle target recognition result and the initial target detection result, so as to obtain the consistency score; the consistency calculation formula is:

[0157]

[0158] Where J represents the degree of consistency. This indicates the three-dimensional bounding box of the vehicle included in the suspected vehicle target recognition result. This indicates the vehicle's 3D bounding box included in the initial target detection result. This represents the intersection-union ratio (IoU) of the vehicle 3D bounding boxes included in the suspected vehicle target recognition result and the vehicle 3D bounding boxes included in the initial target detection result. This indicates the yaw angle included in the suspected vehicle target identification result. This indicates the yaw angle included in the initial target detection result.

[0159] On one hand, the fusion subunit is used to take the initial target detection result as the final target detection result when the consistency is greater than a first threshold; when the consistency is less than or equal to the first threshold and greater than a second threshold, the vehicle 3D bounding boxes contained in the suspected vehicle target recognition result and the initial target detection result are back-projected onto the camera image to obtain the corresponding suspected vehicle rectangular region and the initial vehicle rectangular region; based on the image features contained in the suspected vehicle rectangular region and the initial vehicle rectangular region, the vehicle 3D bounding boxes contained in the suspected vehicle target recognition result and the initial target detection result are merged to obtain the final target detection result.

[0160] On the one hand, it also includes a kinematic parameter acquisition unit, a parameter analysis unit, a bounding box analysis unit, and a fusion unit;

[0161] The kinematic parameter acquisition unit is used to acquire the kinematic parameters of the target within a preset time period before the current moment; wherein, the kinematic parameters include the horizontal velocity and the angular velocity of the yaw angle change;

[0162] The parameter analysis unit is used to analyze the kinematic parameters using a target tracking algorithm to determine a prediction result that includes the estimated pose information of the target at the current moment.

[0163] The bounding box analysis unit is used to analyze the three-dimensional bounding box contained in the final target detection result and the three-dimensional bounding box corresponding to the prediction result according to the consistency judgment rule, so as to determine a new consistency.

[0164] The fusion unit is used to fuse the 3D bounding box contained in the final target detection result and the 3D bounding box corresponding to the prediction result based on the merging method matched by the new consistency, so as to obtain an integrated result.

[0165] On the one hand, it also includes a berth status determination unit;

[0166] The berth status determination unit is used to determine the berth's usage status based on the integration result and the three-dimensional spatial position relationship of the berth.

[0167] On the one hand, it also includes a correlation analysis unit;

[0168] The correlation analysis unit is used to perform feature correlation analysis on the initial target detection results and the integrated results to obtain vehicle three-dimensional trajectory information.

[0169] On the one hand, it also includes verification units, backprojection units, and fitting units;

[0170] The verification unit is used to verify the vehicle's three-dimensional trajectory information in chronological order and filter out the three-dimensional trajectory segments belonging to the same vehicle.

[0171] The back projection unit is used to back project the three-dimensional bounding box corresponding to the vehicle in each of the three-dimensional trajectory segments onto the camera image corresponding to the nearest parking space in the three-dimensional space, so as to obtain the rectangular box of each vehicle.

[0172] The fitting unit is used to fit each of the three-dimensional trajectory segments using the feature information contained in each of the vehicle rectangles, so as to obtain complete three-dimensional trajectory information.

[0173] On one hand, the fitting unit includes a first extraction subunit, a first merging subunit, a second extraction subunit, a second merging subunit, a smoothing point determination subunit, a trajectory construction subunit, and a third merging subunit;

[0174] The first extraction subunit is used to extract the license plate information contained in each of the vehicle rectangles;

[0175] The first merging subunit is used to merge the three-dimensional trajectory segments to obtain complete three-dimensional trajectory information when the license plate information contained in each of the vehicle rectangles is consistent.

[0176] The second extraction subunit is used to extract vehicle texture information contained in each of the vehicle rectangles when the license plate information contained in each of the vehicle rectangles is inconsistent.

[0177] The second merging subunit is used to merge the three-dimensional trajectory segments to obtain complete three-dimensional trajectory information when the vehicle texture information contained in each of the vehicle rectangles is matched.

[0178] The smoothing point determination subunit is used to determine the smoothing point on each of the three-dimensional trajectory segments based on the angle of the curve tangent vector of each point on each of the three-dimensional trajectory segments when the vehicle texture information contained in each of the vehicle rectangles does not match.

[0179] The trajectory construction subunit is used to fit the line segments formed by the smooth points on each of the three-dimensional trajectory segments and the endpoints on each of the three-dimensional trajectory segments to construct a predicted trajectory line.

[0180] The third merging subunit is used to merge each of the three-dimensional trajectory segments with the predicted trajectory line to obtain complete three-dimensional trajectory information.

[0181] On one hand, the trajectory construction subunit is used to construct a first trajectory segment based on the smoothing point on the first three-dimensional trajectory segment and the end point on the first three-dimensional trajectory segment; and to construct a second trajectory segment based on the smoothing point on the second three-dimensional trajectory segment and the start point on the second three-dimensional trajectory segment; wherein the first three-dimensional trajectory segment and the second three-dimensional trajectory segment are two three-dimensional trajectory segments that are adjacent in time on each of the three-dimensional trajectory segments; the first trajectory segment is fitted to obtain a first trajectory prediction line; the second trajectory segment is fitted to obtain a second trajectory prediction line; and when the vertical distance between the first trajectory prediction line and the second trajectory prediction line is less than a set distance threshold, the first trajectory prediction line and the second trajectory prediction line are fitted to determine the predicted trajectory line between the end point on the first three-dimensional trajectory segment and the start point on the second three-dimensional trajectory segment.

[0182] On the one hand, it also includes a vehicle status determination unit;

[0183] The vehicle status determination unit is used to determine the status and time information of each vehicle entering and leaving the parking space based on the complete three-dimensional trajectory information of each vehicle and the three-dimensional positional relationship of the parking space.

[0184] On the one hand, it also includes an image acquisition unit, an image filtering unit, and a license plate extraction unit;

[0185] The image acquisition unit is used to acquire the original camera image corresponding to the complete three-dimensional trajectory information of each vehicle;

[0186] The image filtering unit is used to filter out license plate area images from each of the original camera images based on the boundary clarity of the license plate area, the license plate envelope tilt, and / or the number of license plate character segments in each of the original camera images.

[0187] The license plate extraction unit is used to extract license plate information from the license plate area image.

[0188] This invention also provides a target recognition device, comprising:

[0189] Memory, used to store computer programs;

[0190] A processor for executing the computer program to implement the steps of the target recognition method described above.

[0191] This invention also provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of the target recognition method described above.

[0192] As can be seen from the above technical solution, the process involves acquiring historical point cloud data corresponding to the road scene, acquiring current point cloud data corresponding to the road scene at the current time, and acquiring the current image; filtering the current point cloud data based on a pre-set drivable three-dimensional space range, radar calibration parameters, and historical point cloud data to obtain point cloud data to be identified; analyzing the current image and the point cloud data to be identified using a BEV fusion perception algorithm to obtain initial target detection results; performing multi-scale clustering processing on the point cloud data to be identified according to multiple set distance thresholds to determine the target's three-dimensional bounding box; integrating the target's three-dimensional bounding box and the initial target detection results to obtain the final target detection result; wherein, the target detection result includes the identified vehicle type and its corresponding three-dimensional bounding box. The beneficial effects of this invention are that point cloud data acquisition is simple and can be automatically updated. By filtering the point cloud data, environmental interference can be effectively eliminated, thereby reducing the computational load of the BEV fusion perception algorithm. Considering that the point cloud data to be identified may contain spatially close targets, single-scale spatial clustering can easily lead to under-clustering or over-clustering. Therefore, this invention effectively solves this problem by employing multi-scale clustering processing, ensuring the accuracy of the target's 3D bounding box. Furthermore, the target's 3D bounding box includes all targets appearing within the corresponding region of the point cloud data, meaning it includes unidentifiable non-standard vehicles. By integrating the target's 3D bounding box with the initial target detection results, the final target detection result ensures that it includes both vehicles that the BEV fusion perception algorithm can identify and non-standard vehicles, guaranteeing accurate target identification in complex roadside scenarios. Attached Figure Description

[0193] To more clearly illustrate the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0194] Figure 1 A flowchart of a target recognition method provided in an embodiment of the present invention;

[0195] Figure 2 A flowchart illustrating a method for acquiring current point cloud data provided in an embodiment of the present invention;

[0196] Figure 3 A flowchart illustrating a method for acquiring historical point cloud data provided in an embodiment of the present invention;

[0197] Figure 4A schematic diagram illustrating the unified coordinates corresponding to both the camera and the lidar, provided as an embodiment of the present invention;

[0198] Figure 5 A flowchart illustrating a method for determining an initial target detection result provided in an embodiment of the present invention;

[0199] Figure 6 An architecture diagram of a BEV fusion sensing algorithm provided in an embodiment of the present invention;

[0200] Figure 7 A schematic diagram of a drivable space provided for an embodiment of the present invention;

[0201] Figure 8 This is a flowchart of a method for multi-scale clustering of point cloud data to be identified, provided by an embodiment of the present invention;

[0202] Figure 9 A flowchart illustrating a method for result integration provided in an embodiment of the present invention;

[0203] Figure 10 This is a flowchart illustrating a method for fitting a three-dimensional trajectory segment to obtain complete three-dimensional trajectory information, provided by an embodiment of the present invention.

[0204] Figure 11 This is a schematic diagram illustrating how a vehicle's trajectory is interrupted when it enters or leaves a parking space due to obstruction by a passing vehicle, as provided in an embodiment of the present invention.

[0205] Figure 12 This is a schematic diagram of the structure of a target recognition device provided in an embodiment of the present invention;

[0206] Figure 13 This is a structural diagram of a target recognition device provided in an embodiment of the present invention. Detailed Implementation

[0207] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the present invention.

[0208] The terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may include steps or units not listed.

[0209] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0210] Next, a target recognition method provided by an embodiment of the present invention will be described in detail. Figure 1 A flowchart of a target recognition method provided in an embodiment of the present invention, the method comprising:

[0211] S101: Obtain historical point cloud data corresponding to the road scene, obtain current point cloud data and current image corresponding to the road scene at the current time.

[0212] In this embodiment of the invention, in order to accurately identify targets in a road scene, multi-source sensors can be used for fusion sensing. Various types of sensors are available, such as visible light cameras (single or multiple), infrared cameras, lidar (single or multiple), and millimeter-wave radar.

[0213] In practical applications, all sensors can be encapsulated within a protective enclosure, ensuring a secure connection between them and maintaining their positional relationship. The protective enclosure can be installed on supports at different heights on either side of the road, depending on the specific requirements of the application scenario.

[0214] The target recognition method provided in this invention can be applied to a wide variety of targets. Taking vehicles as an example, target recognition can include identifying the type of vehicle, identifying the vehicle's trajectory, and identifying the vehicle's license plate information. For ease of explanation, the following description will use vehicles as an example.

[0215] S102: Based on the pre-set drivable three-dimensional space range, radar calibration parameters, and historical point cloud data, the current point cloud data is filtered to obtain the point cloud data to be identified.

[0216] Based on vehicle type, vehicles can be divided into conventional vehicles and non-conventional vehicles. Conventional vehicles refer to vehicles whose types can be identified through algorithmic analysis of collected data. In other words, conventional vehicles are those whose vehicle types are identifiable. Non-conventional vehicles refer to vehicles whose types cannot be identified through algorithmic analysis of collected data. Conventional vehicles include cars, trucks, buses, and vans, while non-conventional vehicles include construction vehicles.

[0217] S103: Analyze the current image and the point cloud data to be identified according to the BEV fusion perception algorithm to obtain the initial target detection results.

[0218] The Bird's Eye View (BEV) fusion perception algorithm is also known as the BEV fusion perception algorithm.

[0219] This invention proposes a method for analyzing the current image and the point cloud data to be identified using the BEV fusion sensing algorithm. In practical applications, a model can be built using the BEV fusion sensing algorithm. By inputting the current image and the point cloud data to be identified into the model, the initial target detection results can be obtained.

[0220] Taking vehicles as an example, the initial target detection results can include the vehicle's three-dimensional pose information.

[0221] S104: Perform multi-scale clustering processing on the point cloud data to be identified according to multiple set distance thresholds to determine the target's three-dimensional bounding box.

[0222] Considering that the point cloud data to be identified may contain spatially close targets, if single-scale spatial clustering is used, under-clustering or over-clustering problems are likely to occur. Therefore, this invention processes the point cloud data to be identified by using multi-scale clustering, thereby accurately identifying the target three-dimensional bounding box corresponding to each target in the point cloud data to be identified.

[0223] S105: Integrate the target's 3D bounding box and the initial target detection results to obtain the final target detection result.

[0224] Taking vehicles as an example, the initial target detection result includes the identified vehicle type and its corresponding 3D bounding box. The 3D bounding box can contain information such as the center position coordinates, size, and yaw angle of each identified vehicle. The size can include the vehicle's length, width, and height.

[0225] The target 3D bounding box encompasses the 3D bounding boxes corresponding to all targets appearing within the drivable 3D space. By integrating the target 3D bounding boxes with the initial target detection results, the identification of irregularly shaped vehicles is achieved, effectively overcoming the limitation that the initial target detection results can only detect vehicles of identifiable types.

[0226] As can be seen from the above technical solution, the process involves acquiring historical point cloud data corresponding to the road scene, acquiring current point cloud data corresponding to the road scene at the current time, and acquiring the current image; filtering the current point cloud data based on a pre-set drivable three-dimensional space range, radar calibration parameters, and historical point cloud data to obtain point cloud data to be identified; analyzing the current image and the point cloud data to be identified using a BEV fusion perception algorithm to obtain initial target detection results; performing multi-scale clustering processing on the point cloud data to be identified according to multiple set distance thresholds to determine the target's three-dimensional bounding box; integrating the target's three-dimensional bounding box and the initial target detection results to obtain the final target detection result; wherein, the target detection result includes the identified vehicle type and its corresponding three-dimensional bounding box. The beneficial effects of this invention are that point cloud data acquisition is simple and can be automatically updated. By filtering the point cloud data, environmental interference can be effectively eliminated, thereby reducing the computational load of the BEV fusion perception algorithm. Considering that the point cloud data to be identified may contain spatially close targets, single-scale spatial clustering can easily lead to under-clustering or over-clustering. Therefore, this invention effectively solves this problem by employing multi-scale clustering processing, ensuring the accuracy of the target's 3D bounding box. Furthermore, the target's 3D bounding box includes all targets appearing within the corresponding region of the point cloud data, meaning it includes unidentifiable non-standard vehicles. By integrating the target's 3D bounding box with the initial target detection results, the final target detection result ensures that it includes both vehicles that the BEV fusion perception algorithm can identify and non-standard vehicles, guaranteeing accurate target identification in complex roadside scenarios.

[0227] Figure 2 A flowchart of a method for acquiring current point cloud data provided in an embodiment of the present invention, the method comprising:

[0228] S201: Use a point cloud matching algorithm to match historical point cloud data with current point cloud data to determine the first transformation parameter.

[0229] In practical applications, the sensor mounting rod may sway or vibrate due to factors such as strong winds or the passage of large or heavy vehicles. If the vibration is not eliminated in time, it will cause errors in the acquisition of the vehicle's true position and trajectory information.

[0230] In this embodiment of the invention, the collected point cloud data can be transformed by relying on transformation parameters, thereby eliminating the effects of jitter.

[0231] In practical applications, point cloud matching algorithms can be used to match historical point cloud data with current point cloud data to obtain the first transformation parameters.

[0232] The first transformation parameters include the first rotation matrix and the first translation vector. For ease of description, R can be used. now Let T represent the first rotation matrix. now This represents the first translation vector.

[0233] S202: Determine the deviation of the current point cloud data based on the first rotation matrix and the first translation vector.

[0234] The first rotation matrix is ​​generally a 3x3 matrix, i.e., R. now =R 3*3 The first translation vector is generally a 3*1 vector, i.e., T. now =T 3*1 .

[0235] In this embodiment of the invention, a 4*4 matrix M1 can be constructed based on the first rotation matrix and the first translation vector. make Calculate the deviation D of the point cloud data using the following formula. L ,

[0236]

[0237] in, This represents the element in the j-th row and i-th column of matrix M1. This represents the element in the j-th row and i-th column of matrix M2.

[0238] S203: Based on the set deviation requirements, process the deviation, the first rotation matrix, the first translation vector, and the current point cloud data to obtain the final current point cloud data.

[0239] In practical applications, a deviation threshold can be set to evaluate point cloud data. The system determines whether the deviation is less than the deviation threshold.

[0240] If the deviation is less than the deviation threshold, it means that the sensor pose has not changed or has only changed slightly. In this case, the current point cloud data can be used as the final current point cloud data.

[0241] If the deviation is greater than or equal to the deviation threshold, it indicates that the sensor pose has changed. In order to reduce the problem of inaccurate point cloud data caused by changes in sensor pose, the point cloud data can be calibrated.

[0242] In a specific implementation, the current point cloud data can be transformed using the first rotation matrix and the first translation vector to obtain the pose-calibrated point cloud data, and the pose-calibrated point cloud data can be used as the final current point cloud data.

[0243] In this embodiment of the invention, by evaluating the deviation of point cloud data, sensor jitter can be detected in a timely manner. Based on the transformation parameters, the point cloud data can be transformed to eliminate the impact of sensor jitter on the point cloud data and ensure the accuracy of the point cloud data.

[0244] Figure 3 A flowchart of a method for acquiring historical point cloud data provided in an embodiment of the present invention, the method comprising:

[0245] S301: Obtain the initial point cloud data corresponding to different times within a set time period in a road scene.

[0246] The prior information of the scene environment mainly includes the background information contained in the current scene. Considering that the images captured by the camera are very sensitive to changes in ambient lighting, in this embodiment of the invention, the point cloud data collected by the lidar is processed to obtain historical point cloud data, which can be used as the prior information of the scene environment.

[0247] To reduce interference from non-background factors, initial point cloud data for different times within a set time period can be acquired in the road scene. Point cloud data can be acquired every t seconds within the set time period.

[0248] The set time period can be a time when there are fewer vehicles on the road, such as from 11 p.m. to 5 a.m. It should be noted that the set time period is a dynamically adjustable parameter, and its specific value can be adjusted and determined according to the actual situation of the scenario.

[0249] S302: Use a point cloud matching algorithm to match the initial point cloud data at adjacent time points in sequence to determine the initial transformation parameters corresponding to each adjacent time point.

[0250] The initial transformation parameters may include the initial rotation matrix and the initial translation vector.

[0251] The point cloud matching algorithm for calculating transformation parameters is a relatively mature technology and will not be elaborated on here.

[0252] In practical applications, the initial point cloud data that are adjacent to each other can be grouped into a set of data according to the order of sampling time. The point cloud matching algorithm can be used to analyze the set of data to determine the initial transformation parameters corresponding to the set of data.

[0253] S303: Determine the initial deviation for each adjacent time step based on the initial rotation matrix and initial translation vector for each adjacent time step.

[0254] The method for determining the initial deviation for each adjacent time point is the same, and the following will be introduced using an adjacent time point as an example.

[0255] Using two adjacent times t i and t j For example, its corresponding initial rotation matrix is ​​R. ij And the initial translation vector is T ij According to R ij and T ij It is possible to calculate the time t between two adjacent times. i and t j The corresponding initial deviation.

[0256] R ij R is a 3x3 matrix ij =R' 3*3 T ij A 3*1 vector T ij =T' 3*1 In practical applications, it can be based on R. ij and T ij Construct a 4x4 matrix M'1, make Calculate the initial deviation D' corresponding to adjacent time points using the following formula. L ,

[0257]

[0258] in, This represents the element in the j-th row and i-th column of matrix M'1. This represents the element in the j-th row and i-th column of matrix M2.

[0259] S304: Based on the deviation requirements, process each initial deviation, each initial rotation matrix, each initial translation vector, and the initial point cloud data to obtain historical point cloud data.

[0260] A smaller initial deviation value indicates better stability of the acquired initial point cloud data; a larger initial deviation value indicates worse stability of the acquired initial point cloud data. In this embodiment of the invention, the stability of the initial point cloud data can be evaluated using a deviation threshold.

[0261] In practical applications, it can be determined whether the initial deviation at each adjacent time point is less than the deviation threshold.

[0262] If the initial deviation at each adjacent time point is less than the deviation threshold, it indicates that the stability of the initial point cloud data is very good, that is, the sensor pose is stable within the set time period. At this time, the initial point cloud data can be superimposed as historical point cloud data.

[0263] If the initial deviation at each adjacent time point is not less than the deviation threshold, it indicates that the stability of the initial point cloud data is poor and the sensor pose has changed within the set time period. At this time, the initial rotation matrix and the initial translation vector can be used to transform the initial point cloud data corresponding to each other to obtain the pose-calibrated initial point cloud data. The pose-calibrated initial point cloud data are then superimposed to form the historical point cloud data.

[0264] In this embodiment of the invention, by using stable point cloud data as historical point cloud data, the impact of ambient lighting on the data can be effectively reduced. The stability of the initial point cloud data is evaluated based on the initial deviation corresponding to each adjacent time point, and unstable point cloud data is transformed to ensure the reliability of historical point cloud data, providing reliable data assurance for subsequent analysis of the vehicle's true pose and trajectory information.

[0265] In practical applications, the BEV fusion sensing algorithm can be used to analyze the current image and the point cloud data to be identified. The number of sensors installed on a single pole is limited, and the spatial range they can detect is also limited. Considering that the current image and the point cloud data to be identified primarily come from sensors on different poles, and the installation angles of each sensor are difficult to guarantee are consistent, the calibration parameters of sensors on different poles will inevitably differ.

[0266] To ensure that the BEV fusion perception algorithm achieves optimal inference performance when analyzing the current image and the point cloud data to be identified, the sensor data from different poles can be transformed to a unified coordinate system before BEV perception inference is performed. This ensures that the trained BEV model has consistent recognition performance on all poles.

[0267] Therefore, before analyzing the current image and the point cloud data to be identified using the BEV fusion sensing algorithm to obtain the initial target detection result, a coordinate transformation can be performed on the current image and all the point cloud data to be identified according to a set unified coordinate system to obtain the coordinate-transformed image and the coordinate-transformed point cloud data to be identified. Then, the BEV fusion sensing algorithm is used to analyze the coordinate-transformed image and the coordinate-transformed point cloud data to obtain the initial target detection result.

[0268] Figure 4 This is a schematic diagram illustrating the unified coordinates corresponding to both the camera and the lidar, provided as an embodiment of the present invention. Considering the inconsistent coordinate definitions between mechanical and solid-state lidar, therefore... Figure 4 The unified coordinate system for mechanical radar and solid-state radar is shown in the text. Figure 4The coordinate system diagram at the top includes the unified coordinate system corresponding to the camera and the unified coordinate system corresponding to the mechanical radar. The coordinate system formed by dashed lines is the unified coordinate system corresponding to the camera, with the y-axis pointing vertically downwards, the x-axis pointing horizontally to the right, and the z-axis defined according to the right-hand rule, pointing horizontally forwards. The coordinate system formed by solid lines is the unified coordinate system corresponding to the mechanical radar, with the x-axis pointing vertically downwards, the y-axis pointing horizontally forwards, and the z-axis defined according to the right-hand rule, pointing horizontally to the right. Figure 4 The bottom is the unified coordinate system corresponding to solid-state radar. The y-axis is vertically upward, the x-axis is horizontal backward, and the z-axis is defined according to the right-hand system, with the z-axis horizontal to the right.

[0269] In this embodiment of the invention, in addition to collecting point cloud data, images captured by a camera are also collected. Cameras may experience camera shake, leading to inaccurate images. Therefore, to improve image accuracy, the images captured by the camera can be calibrated.

[0270] Before performing coordinate transformation on the current image and all point cloud data to be identified according to the set unified coordinate system to obtain the coordinate-transformed image and the coordinate-transformed point cloud data to be identified, the initial image acquired by the camera can be distorted according to the camera intrinsic parameters to obtain the distorted image.

[0271] By projecting the current point cloud data and the pose-calibrated point cloud data onto the corrected image, we can obtain many sets of images with the same name. For ease of explanation, we will take one set of images with the same name, namely the first image with the same name and the second image with the same name, as an example.

[0272] The current point cloud data and the pose-calibrated point cloud data are projected onto the rectified image to obtain a first image and a second image with the same name. Based on the matching relationship of the corresponding points between the first image and the second image with the same name, the image calibration affine transformation parameters are determined. The initial image is transformed using the image calibration affine transformation parameters to obtain the pose-calibrated image, and the pose-calibrated image is used as the current image.

[0273] In this embodiment of the invention, the transformation parameters corresponding to the camera image are called image calibration affine transformation parameters. For a group of sensors, when point cloud data undergoes pose calibration, cameras in the same group as the LiDAR that acquired the point cloud data will also experience jitter. By projecting the point cloud data before and after pose calibration onto an image to obtain images with the same name, image calibration affine transformation parameters can be constructed based on the matching relationship of the images with the same name, thereby achieving image pose calibration.

[0274] Figure 5 A flowchart of a method for determining an initial target detection result provided in an embodiment of the present invention, the method comprising:

[0275] S501: Use the BEV fusion sensing algorithm to extract features from the coordinate-transformed image to obtain three-dimensional image spatial features.

[0276] BEV fusion perception algorithm can identify conventional vehicles.

[0277] In this embodiment of the invention, the BEV fusion sensing algorithm can process both image and point cloud data simultaneously. Therefore, the model corresponding to the BEV fusion sensing algorithm can contain two branches: one branch for processing images and the other branch for processing point cloud data.

[0278] After obtaining the image after coordinate transformation, the image can be encoded first, and then the image features can be obtained using the backbone network. The image features can then be transformed into BEV space to realize the transformation of 2D features into 3D spatial features. The 3D spatial features of the image are called image BEV features.

[0279] S502: Use the BEV fusion sensing algorithm to extract features from the coordinate-transformed point cloud data to obtain the three-dimensional point cloud spatial features.

[0280] After obtaining the coordinate-transformed point cloud data to be identified, point cloud encoding can be performed first. Then, the backbone network is used to obtain point cloud features, which are then converted to BEV space. Point cloud data itself belongs to 3D spatial features. The conversion of point cloud data to BEV space can be achieved by compressing the point cloud features in the vertical axis direction. The point cloud features converted to BEV space can be called point cloud BEV features.

[0281] S503: The aligned 3D image spatial features and 3D point cloud spatial features are fused to obtain fused features.

[0282] In practical applications, image BEV features can be aligned with point cloud BEV features, and then feature fusion can be performed through stacking or fully connected layers to obtain fused features.

[0283] S504: Analyze the fused features using the target detection head included in the BEV fusion sensing algorithm to output initial target detection results; wherein, the initial target detection results include the identified targets and their pose information.

[0284] After obtaining the fused features, the target pose information can be obtained using the detection head.

[0285] Figure 6This is an architectural diagram of a BEV fusion sensing algorithm provided in an embodiment of the present invention. The BEV fusion sensing algorithm architecture includes two branches. The first branch is an image branch, which processes the image after coordinate transformation through modules such as image encoding, image feature generation, and conversion to bird's-eye view features, thereby obtaining image bird's-eye view features, i.e., image BEV features. The second branch is a point cloud branch, which processes the point cloud data after coordinate transformation through modules such as point cloud encoding, point cloud feature generation, point cloud feature compression along the vertical axis, and conversion to bird's-eye view features, thereby obtaining point cloud bird's-eye view features, i.e., point cloud BEV features. Aligning the image bird's-eye view features with the point cloud bird's-eye view features involves aligning the image BEV features with the point cloud BEV features. Processing the aligned BEV features through modules such as bird's-eye view feature encoding (BEV feature encoding), bird's-eye view feature fusion (BEV feature fusion), and target detection head, initial target detection results can be obtained.

[0286] In this embodiment of the invention, the BEV fusion sensing algorithm can be used to fuse image and point cloud data, thereby accurately identifying the target and its pose information contained in the image and point cloud data.

[0287] Taking a set of sensors as an example, the range of LiDAR is often wider than that of camera. In this embodiment of the invention, the target is identified by the fusion analysis of image and point cloud data. Therefore, it is not very meaningful to analyze point cloud data that exceeds the range.

[0288] In practical applications, the drivable three-dimensional space range can be preset. After acquiring the current point cloud data, the drivable area point cloud data can be extracted from the current point cloud data using the preset drivable three-dimensional space range. Based on the radar calibration parameters and historical point cloud data, the drivable area point cloud data is filtered to obtain the point cloud data to be identified.

[0289] Figure 7 This is a schematic diagram of a drivable space provided in an embodiment of the present invention. Figure 7 The outermost box is used to represent the coverage area of ​​the point cloud data collected by the lidar. Figure 7 The rectangular prism represents a drivable three-dimensional space. This drivable three-dimensional space may contain parking spaces for vehicles.

[0290] The prior information of the scene environment includes background point cloud data within the drivable three-dimensional space. In order to improve the accuracy of target recognition and reduce the computational load of the BEV fusion perception algorithm, the current point cloud data can be filtered to remove information that is not related to the target, such as ground point cloud data and background point cloud data.

[0291] In practice, ground point cloud data can be filtered out from the point cloud data of the drivable area based on radar calibration parameters to obtain initial filtered point cloud data; historical point cloud data is matched with the initial filtered point cloud data to filter out background point cloud data; and the initial filtered point cloud data after filtering out background point cloud data is used as the point cloud data to be identified.

[0292] In this embodiment of the invention, by setting a drivable area to constrain the drivable space of the point cloud data, irrelevant point cloud data can be eliminated. Then, by filtering out ground point cloud data and background point cloud data, the final point cloud data to be identified basically only contains moving targets (such as vehicles, pedestrians, bicycles, etc.), which is very suitable for subsequent point cloud spatial clustering to obtain the target's three-dimensional bounding box.

[0293] Figure 8 A flowchart of a method for multi-scale clustering processing of point cloud data to be identified, provided by an embodiment of the present invention, is included in the following method:

[0294] S801: Perform spatial clustering processing on the point cloud data to be identified according to each distance threshold to obtain the identification result corresponding to each distance threshold.

[0295] Taking point cloud data collected by LiDAR as an example, the closer the point cloud data is to the LiDAR, the smaller the distance threshold can be set. Considering that the angle of point cloud data is relatively dispersed, if a uniform distance threshold is used to cluster the point cloud data to be identified, over-clustering or under-clustering may easily occur. Therefore, in this embodiment of the invention, multiple distance thresholds can be set to improve the spatial clustering effect.

[0296] In practical applications, spatial clustering can be performed on the point cloud data to be identified based on each distance threshold to obtain the identification result corresponding to each distance threshold.

[0297] S802: Based on the set combination method and the target spatial location and target size contained in each recognition result, merge the recognition results to obtain multiple merged results.

[0298] Each distance threshold corresponds to a recognition result. To improve the merging effect of recognition results, the recognition results can be merged according to a set combination method.

[0299] In practical applications, the overlap between the target spatial location and target size in each pair of identification results can be identified. The target spatial location can include the target's center coordinates, and the target size includes its length, width, and height. Based on the overlap relationship corresponding to each pair, the identification results for each pair are merged to obtain the merged result for each pair.

[0300] S803: Construct a grid according to the set yaw angle interval and spatial position interval.

[0301] Considering that the yaw angles of the targets included in the merged results are prone to deviation, a grid can be constructed to obtain accurate yaw angles. The yaw angles can be calibrated by projecting the merged results onto the grid.

[0302] In practical applications, the yaw angle can be set from 0 to 90°, and the grid can be constructed with yaw angle intervals of 10° and spatial distance intervals of 1 meter.

[0303] S804: Project each merged result onto the grid to determine the optimal merged result, and use the optimal merged result as the target 3D bounding box.

[0304] The target's three-dimensional bounding box includes the optimal target center coordinates, the optimal target size, and the optimal target yaw angle.

[0305] In practical applications, the first target contained in each merged result can be projected onto the grid; where the first target is any identical target contained in each merged result; the yaw angle corresponding to the grid cell with the most projected first target point cloud data is taken as the optimal target yaw angle.

[0306] In this embodiment of the invention, by merging the recognition results and projecting the merged results onto a grid, the yaw angle can be calibrated, ensuring the accuracy of the yaw angle contained in the target's three-dimensional bounding box.

[0307] Figure 9 A flowchart of a method for result integration provided in an embodiment of the present invention, the method comprising:

[0308] S901: Based on the target's 3D bounding box and the image of the parking space corresponding to the parking space number closest to the target's 3D bounding box, target filtering is performed on the target's 3D bounding box to obtain the suspected vehicle target recognition result.

[0309] The target 3D bounding box obtained by multi-scale clustering of point cloud data can cover all targets. In this embodiment of the invention, the point cloud data corresponding to the target 3D bounding box can be back-projected into a 2D image to achieve the identification of irregularly shaped vehicles.

[0310] In practical applications, the image of the berth corresponding to the berth number closest to the target 3D bounding box can be obtained; the point cloud data corresponding to the target 3D bounding box can be back-projected onto the berth image to determine the first target rectangular region; the target detection algorithm can be used to identify the target in the berth image to determine the second target rectangular region; based on the first and second target rectangular regions, the target 3D bounding box can be filtered to obtain the suspected vehicle target identification result.

[0311] The first target rectangular region contains the rectangular regions corresponding to all targets. The target detection algorithm can detect identifiable types of targets on the berth image.

[0312] Object detection algorithms can identify targets such as pedestrians, bicycles, delivery vehicles, and motorcycles in parking space images. Object detection algorithms include, but are not limited to, commonly used 2D object detection algorithms such as the You Only Look Once (YOLO) series (which completes object detection in one step) and the lightweight deep neural network (Mobilenet) series.

[0313] In this embodiment of the invention, the intersection-union ratio of the first target rectangular region and the second target rectangular region can be calculated.

[0314] A higher Intersection over Union (IoU) ratio indicates that the first and second target rectangular regions contain the same target. Therefore, to identify irregularly shaped vehicles, if the IoU is greater than or equal to the upper limit of the IoU ratio, the point cloud data corresponding to the first target rectangular region can be filtered out from the point cloud data corresponding to the target's 3D bounding box.

[0315] By calculating the intersection-union ratio, the same target contained in the first target matrix region and the second target rectangular region can be identified. The point cloud data corresponding to the same target is filtered out from the point cloud data corresponding to the first target rectangular region. The remaining point cloud data are basically the point cloud data corresponding to the non-standard vehicle. The point cloud data corresponding to the non-standard vehicle can be used as the result of suspected vehicle target identification.

[0316] S902: Based on the consistency judgment rules, analyze the vehicle 3D bounding boxes contained in the suspected vehicle target recognition results and the initial target detection results to determine the consistency.

[0317] In practical applications, the consistency calculation formula can be used to process the 3D bounding boxes of vehicles contained in the suspected vehicle target recognition results and the initial target detection results to obtain the consistency score. The consistency calculation formula is as follows:

[0318]

[0319] Where J represents the degree of consistency. This indicates the 3D bounding box of the vehicle included in the suspected vehicle target recognition result. This represents the 3D bounding box of the vehicle included in the initial target detection results. This represents the intersection-union ratio (IoU) of the vehicle 3D bounding boxes included in the suspected vehicle target recognition result and the vehicle 3D bounding boxes included in the initial target detection result. This indicates the yaw angle included in the suspected vehicle target identification result. This indicates the yaw angle included in the initial target detection results.

[0320] S903: Based on the merging method matched by consistency, the vehicle 3D bounding boxes contained in the suspected vehicle target recognition results and the initial target detection results are fused to obtain the final target detection result.

[0321] In practical applications, a threshold can be set to evaluate consistency. Different merging methods can be set for different threshold ranges.

[0322] If the consistency is greater than the first threshold, it means that the initial target detection result and the suspected vehicle target recognition result contain the same or similar targets. In this case, the initial target detection result can be used as the final target detection result.

[0323] If the consistency is less than or equal to the first threshold and greater than the second threshold, it indicates that there are some identical targets among all targets included in the initial target detection result and the suspected vehicle target recognition result. In this case, the vehicle 3D bounding boxes included in the suspected vehicle target recognition result and the initial target detection result can be back-projected onto the camera image to obtain the corresponding suspected vehicle rectangular region and the initial vehicle rectangular region. Based on the image features included in the suspected vehicle rectangular region and the initial vehicle rectangular region, the vehicle 3D bounding boxes included in the suspected vehicle target recognition result and the initial target detection result are merged to obtain the final target detection result.

[0324] In practical applications, feature extraction algorithms can be used to extract the image features contained in the suspected vehicle rectangular region and the initial vehicle rectangular region, respectively. Feature extraction algorithms can include Scale-Invariant Feature Transform (SFIT), Oriented Fast and Rotated BRIEF (ORB), or feature extraction algorithms based on deep neural network architectures (ResNet3), etc.

[0325] In this embodiment of the invention, a first target rectangular region can be obtained by back-projecting the point cloud data corresponding to the target's three-dimensional bounding box onto the parking space image. By comparing the first target rectangular region and the second target rectangular region, irregular vehicles contained in the first target rectangular region can be identified, thus achieving the identification of irregular vehicles. Furthermore, by calculating consistency, the identification results of suspected vehicle targets and the initial target detection results are integrated to obtain the final target detection result.

[0326] Considering that BEV fusion sensing algorithms may miss target detections—for example, in two consecutive frames of data, the same target may not have moved spatially, but the detection results for that target may appear intermittently. This situation can introduce errors into trajectory recognition.

[0327] Therefore, in this embodiment of the invention, the kinematic parameters of the target within a preset time period before the current moment can be obtained; wherein, the kinematic parameters include the horizontal velocity and the yaw angle change angular velocity; the kinematic parameters are analyzed using a target tracking algorithm to determine a prediction result containing the estimated pose information of the target at the current moment. In order to integrate the prediction result with the final target detection result, the three-dimensional bounding box contained in the final target detection result and the three-dimensional bounding box corresponding to the prediction result can be analyzed according to the consistency judgment rule to determine a new consistency; based on the merging method matched by the new consistency, the three-dimensional bounding box contained in the final target detection result and the three-dimensional bounding box corresponding to the prediction result are fused to obtain the integrated result.

[0328] The method of integrating the prediction results with the final target detection results is similar to the method of integrating the suspected vehicle target recognition results with the initial target detection results. Please refer to the introduction of S902 and S903 for details, which will not be repeated here.

[0329] By analyzing the kinematic parameters of a target using a target tracking algorithm, the target's pose information can be predicted. This predicted pose information is then used as the prediction result. By combining the prediction results with the final target detection result, the accuracy of trajectory recognition can be improved.

[0330] After obtaining the integration results, the usage status of the berths can be determined based on the integration results and the three-dimensional positional relationship of the berth space.

[0331] By performing feature correlation analysis on the initial target detection results and the integrated results, the vehicle's three-dimensional trajectory information can be obtained.

[0332] Considering that in practical applications, factors such as obstacle occlusion may cause the trajectory information of a vehicle to be discontinuous, in order to obtain complete three-dimensional trajectory information for each vehicle, after obtaining the vehicle's three-dimensional trajectory information, the vehicle's three-dimensional trajectory information can be verified in chronological order to filter out the three-dimensional trajectory segments belonging to the same vehicle.

[0333] In this embodiment of the invention, the vehicle three-dimensional trajectory information often includes three-dimensional trajectory segments corresponding to multiple vehicles. In order to obtain the complete three-dimensional trajectory information corresponding to each vehicle, it is necessary to first summarize the three-dimensional trajectory segments belonging to the same vehicle from the vehicle three-dimensional trajectory information.

[0334] In practical applications, the timestamps of the start and end points of a 3D trajectory segment can be verified to filter out 3D trajectory segments belonging to the same vehicle.

[0335] For example, suppose there are two 3D trajectory segments, namely 3D trajectory segment ab and 3D trajectory segment cd. If the time interval from ta to tb overlaps with the time interval from tc to td, it means that these two 3D trajectory segments are not the same target trajectory, and the trajectories cannot be merged. If the time interval from ta to tb does not overlap with the time interval from tc to td, it means that these two 3D trajectory segments may belong to the same target trajectory.

[0336] The processing method for each vehicle's 3D trajectory segment is the same. Taking the 3D trajectory segment corresponding to a vehicle as an example, after obtaining the 3D trajectory segment corresponding to the vehicle, the 3D bounding box corresponding to the vehicle in each 3D trajectory segment can be back-projected onto the camera image corresponding to the nearest parking space in 3D space to obtain the vehicle's bounding box. Using the feature information contained in each vehicle's bounding box, each 3D trajectory segment is fitted to obtain complete 3D trajectory information.

[0337] Figure 10 A flowchart of a method for fitting a three-dimensional trajectory segment to obtain complete three-dimensional trajectory information, provided by an embodiment of the present invention, is included in the following:

[0338] S1001: Extract the license plate information contained in the rectangular frame of each vehicle.

[0339] It should be noted that, in this embodiment of the invention, the vehicle rectangle refers to the image area formed by the back projection of a three-dimensional bounding box onto the camera image. The vehicle rectangle contains specific image information and is not simply a border.

[0340] After obtaining the bounding boxes for each vehicle, license plate information can be extracted from the image information contained within each vehicle's bounding box. The method for extracting license plate information is a relatively mature technology and will not be elaborated upon here.

[0341] S1002: If the license plate information contained in the rectangular frames of each vehicle is consistent, merge the three-dimensional trajectory segments to obtain complete three-dimensional trajectory information.

[0342] After extracting the license plate information contained in the rectangles of each vehicle, it can be determined whether the license plate information contained in the rectangles of each vehicle is consistent.

[0343] If the license plate information contained in the rectangles of each vehicle is consistent, it means that these three-dimensional trajectory segments belong to the same vehicle. At this time, the three-dimensional trajectory segments can be merged to obtain complete three-dimensional trajectory information.

[0344] S1003: When the license plate information contained in the rectangular frames of each vehicle is inconsistent, extract the vehicle texture information contained in the rectangular frames of each vehicle.

[0345] If the license plate information contained in the bounding boxes of all vehicles is consistent, the vehicle texture information contained in each bounding box can be further extracted. In practical applications, a deep residual network (ResNet34) can be used to extract vehicle texture information.

[0346] S1004: When the vehicle texture information contained in each vehicle rectangle is matched, merge the three-dimensional trajectory segments to obtain complete three-dimensional trajectory information.

[0347] If the vehicle texture information contained in each vehicle rectangle matches, it means that these 3D trajectory segments belong to the same vehicle. At this point, the 3D trajectory segments can be merged to obtain complete 3D trajectory information.

[0348] S1005: When the vehicle texture information contained in each vehicle rectangle does not match, determine the smoothing point on each three-dimensional trajectory segment based on the angle of the curve tangent vector of each point on each three-dimensional trajectory segment.

[0349] In cases where license plate information and vehicle texture information cannot be recognized, or where license plate information and vehicle texture information do not match, the smooth points on each three-dimensional trajectory segment can be determined based on the angle of the tangent vector of the curve at each point on each three-dimensional trajectory segment.

[0350] S1006: Fit the line segments formed by the smooth points on each three-dimensional trajectory segment and the endpoints on each three-dimensional trajectory segment to construct the predicted trajectory line.

[0351] The three-dimensional trajectory segments can be sorted according to time sequence. Since the fitting method is the same for every two adjacent three-dimensional trajectory segments, we will use any two adjacent three-dimensional trajectory segments as an example. For ease of description, we can refer to any two adjacent three-dimensional trajectory segments as the first three-dimensional trajectory segment and the second three-dimensional trajectory segment; that is, the first three-dimensional trajectory segment and the second three-dimensional trajectory segment are two three-dimensional trajectory segments that are temporally adjacent to each other.

[0352] In this embodiment of the invention, a first trajectory segment can be constructed based on the smoothing point on the first three-dimensional trajectory segment and the end point on the first three-dimensional trajectory segment; a second trajectory segment can be constructed based on the smoothing point on the second three-dimensional trajectory segment and the start point on the second three-dimensional trajectory segment; the first trajectory segment is fitted to obtain a first trajectory prediction line; the second trajectory segment is fitted to obtain a second trajectory prediction line; if the vertical distance between the first trajectory prediction line and the second trajectory prediction line is less than a set distance threshold, the first trajectory prediction line and the second trajectory prediction line are fitted to determine the predicted trajectory line between the end point on the first three-dimensional trajectory segment and the start point on the second three-dimensional trajectory segment.

[0353] S1007: Merge each 3D trajectory segment with the predicted trajectory line to obtain complete 3D trajectory information.

[0354] The predicted trajectory line is a trajectory line fitted to each discontinuous 3D trajectory segment. By merging each 3D trajectory segment with the predicted trajectory line, complete 3D trajectory information can be obtained.

[0355] Figure 11 This is a schematic diagram illustrating how a vehicle's trajectory is interrupted when it enters or exits a parking space due to obstruction by a passing vehicle, according to an embodiment of the present invention. Figure 11 The trajectory includes two segments, trajectory segment 1 and trajectory segment 2. The two endpoints of trajectory segment 1 are endpoint a and endpoint b, which can also be called trajectory segment ab. The two endpoints of trajectory segment 2 are endpoint c and endpoint d, which can also be called trajectory segment cd.

[0356] For trajectory segment ab, with t b Using time as a baseline, a sliding query is performed to find the desired time point in descending order. The endpoint corresponding to this time point is b1. For trajectory segment cd, t c Using time as a baseline, the query can be performed by scrolling through the time points. The endpoint corresponding to this time point is c1. The query principle is... The change in the angle of the tangent vector of the curve at each point on the trajectory segment is less than the threshold T. θ In practical applications, T θ It can be set to 5°. For The paragraphs must be consistent. The change in the angle of the tangent vector of the curve at each point on the trajectory segment is less than the threshold T. θ .

[0357] In determining and After that, respectively The predicted trajectory line is obtained by fitting the trajectory points between them. right The predicted trajectory line is obtained by fitting the trajectory points between them. To make it easier to distinguish, the predicted trajectory line can be... This is called the first trajectory prediction line, and the predicted trajectory line... It is called the second trajectory prediction line.

[0358] Due to the predicted trajectory line and Since it is a three-dimensional curve or a straight line, the shortest distance between two predicted lines, i.e., perpendicularity, can be calculated. For example, d... l And as a basis for judgment. When d l <T d This allows fitting the first and second predicted trajectory lines together to obtain a predicted trajectory line. Concatenating trajectory segment ab, the predicted trajectory line, and trajectory segment cd yields complete trajectory information. Typically, T... d It is set to 0.2 meters.

[0359] After obtaining complete three-dimensional trajectory information, the status and time information of each vehicle entering and leaving the parking space can be determined based on the complete three-dimensional trajectory information of each vehicle and the three-dimensional positional relationship of the parking space.

[0360] In this embodiment of the invention, by selecting smooth points on a three-dimensional trajectory segment and fitting the smooth points to the endpoints of the trajectory segment, missing trajectories are supplemented, thereby constructing complete three-dimensional trajectory information. This solves the problem of incomplete vehicle trajectories caused by occlusion from passing vehicles. It provides reliable data support for subsequent identification of vehicle parking status and license plate information.

[0361] After obtaining complete 3D trajectory information, the original camera images corresponding to the complete 3D trajectory information of each vehicle can be acquired; based on the boundary clarity of the license plate area, the license plate envelope tilt, and / or the number of license plate character segments in each original camera image, license plate area images are selected from each original camera image; license plate information is extracted from the license plate area images.

[0362] The clearer the boundary of the area where the license plate is located, the easier it is to identify the license plate information.

[0363] In practical applications, due to the acquisition angle, the license plate area in the image captured by the camera may be distorted. The higher the tilt angle of the license plate envelope, the more severe the distortion, which will affect the accurate recognition of the license plate information. Therefore, when selecting images for extracting license plate information, images with a smaller license plate envelope tilt angle should be chosen.

[0364] The number of license plate character segments indicates the number of characters contained in the identified license plate information. In practical applications, a complete license plate typically consists of 7 characters. Therefore, if the number of license plate character segments in the original camera image is less than 7, it indicates that the license plate information is incomplete, and the frame can be discarded during license plate recognition.

[0365] In this embodiment of the invention, based on the boundary sharpness of the license plate area, the license plate envelope tilt, and the number of license plate character segments in each original camera image, the highest quality license plate area image can be selected from each original camera image, thereby extracting accurate license plate information from the license plate area image. This ensures the accuracy of the license plate information during subsequent parking fee collection, reducing the occurrence of charging errors due to license plate recognition errors.

[0366] Figure 12 A schematic diagram of a target recognition device provided in an embodiment of the present invention includes an acquisition unit 121, a filtering unit 122, an analysis unit 123, a clustering unit 124, and an integration unit 125;

[0367] The acquisition unit 121 is used to acquire historical point cloud data corresponding to the road scene, acquire current point cloud data and current image corresponding to the road scene at the current time;

[0368] The filtering unit 122 is used to filter the current point cloud data according to the preset drivable three-dimensional space range, radar calibration parameters and historical point cloud data to obtain the point cloud data to be identified.

[0369] The analysis unit 123 is used to analyze the current image and the point cloud data to be identified according to the BEV fusion perception algorithm to obtain the initial target detection result;

[0370] Clustering unit 124 is used to perform multi-scale clustering processing on the point cloud data to be identified according to multiple set distance thresholds in order to determine the target three-dimensional bounding box.

[0371] Integration unit 125 is used to integrate the target 3D bounding box and the initial target detection result to obtain the final target detection result; wherein, the target detection result includes the identified vehicle type and its corresponding 3D bounding box.

[0372] In some embodiments, it further includes a first matching unit, a first deviation determination unit, and a first processing unit;

[0373] The first matching unit is used to match historical point cloud data and current point cloud data using a point cloud matching algorithm to determine the first transformation parameters; wherein, the first transformation parameters include a first rotation matrix and a first translation vector;

[0374] The first deviation determination unit is used to determine the deviation of the current point cloud data based on the first rotation matrix and the first translation vector.

[0375] The first processing unit is used to process the deviation, the first rotation matrix, the first translation vector, and the current point cloud data based on the set deviation requirements to obtain the final current point cloud data.

[0376] In some embodiments, the processing unit is used to determine whether the deviation is less than a deviation threshold; if the deviation is less than the deviation threshold, the current point cloud data is used as the final current point cloud data; if the deviation is greater than or equal to the deviation threshold, the current point cloud data is transformed using a first rotation matrix and a first translation vector to obtain pose-calibrated point cloud data, and the pose-calibrated point cloud data is used as the final current point cloud data.

[0377] In some embodiments, for acquiring historical point cloud data, the apparatus includes an acquisition unit, a second matching unit, a second deviation determination unit, and a second processing unit;

[0378] The acquisition unit is used to acquire the initial point cloud data corresponding to different times within a set time period in a road scene;

[0379] The second matching unit is used to sequentially match the initial point cloud data at adjacent time points using a point cloud matching algorithm to determine the initial transformation parameters corresponding to each adjacent time point; wherein, the initial transformation parameters include the initial rotation matrix and the initial translation vector;

[0380] The second deviation determination unit is used to determine the initial deviation corresponding to each adjacent time step based on the initial rotation matrix and initial translation vector corresponding to each adjacent time step.

[0381] The second processing unit is used to process each initial deviation, each initial rotation matrix, each initial translation vector, and the initial point cloud data based on the deviation requirements to obtain historical point cloud data.

[0382] In some embodiments, the second processing unit is used to determine whether the initial deviation corresponding to each adjacent time step is less than the deviation threshold; if the initial deviation corresponding to each adjacent time step is less than the deviation threshold, the initial point cloud data are superimposed as historical point cloud data; if the initial deviation corresponding to each adjacent time step is not less than the deviation threshold, the initial point cloud data corresponding to each initial rotation matrix and each initial translation vector are used to transform the initial point cloud data corresponding to each initial point cloud data to obtain the pose-calibrated initial point cloud data, and the pose-calibrated initial point cloud data are superimposed as historical point cloud data.

[0383] In some embodiments, a coordinate transformation unit is also included;

[0384] The coordinate transformation unit is used to perform coordinate transformation on the current image and all point cloud data to be identified according to a set unified coordinate system, so as to obtain the coordinate transformed image and the coordinate transformed point cloud data to be identified.

[0385] The analysis unit is used to analyze the coordinate-transformed image and the coordinate-transformed point cloud data to be identified according to the BEV fusion sensing algorithm in order to obtain the initial target detection results.

[0386] In some embodiments, it further includes a correction unit, a projection unit, a parameter determination unit, and a calibration unit;

[0387] The correction unit is used to correct the distortion of the initial image acquired by the camera based on the camera's intrinsic parameters to obtain a corrected image.

[0388] The projection unit is used to project the current point cloud data and the pose-calibrated point cloud data onto the corrected image, respectively, to obtain the first image with the same name and the second image with the same name.

[0389] The parameter determination unit is used to determine the image calibration affine transformation parameters based on the matching relationship of corresponding points between the first and second identical images.

[0390] The calibration unit is used to transform the initial image using image calibration affine transformation parameters to obtain a pose-calibrated image, and then uses the pose-calibrated image as the current image.

[0391] In some embodiments, the analysis unit is used to extract features from the coordinate-transformed image using the BEV fusion sensing algorithm to obtain three-dimensional image spatial features; to extract features from the coordinate-transformed point cloud data to be identified using the BEV fusion sensing algorithm to obtain three-dimensional point cloud spatial features; to fuse the aligned three-dimensional image spatial features and the three-dimensional point cloud spatial features to obtain fused features; and to analyze the fused features using the target detection head included in the BEV fusion sensing algorithm to output initial target detection results; wherein, the initial target detection results include the identified targets and their pose information.

[0392] In some embodiments, the filtering unit is used to extract drivable area point cloud data from the current point cloud data using a pre-defined drivable three-dimensional space range; and to perform filtering processing on the drivable area point cloud data according to radar calibration parameters and historical point cloud data to obtain point cloud data to be identified.

[0393] In some embodiments, the filtering unit is used to filter out ground point cloud data from the drivable area point cloud data according to radar calibration parameters to obtain initial filtered point cloud data; match historical point cloud data with the initial filtered point cloud data to filter out background point cloud data from the initial filtered point cloud data; and use the initial filtered point cloud data after filtering out background point cloud data as the point cloud data to be identified.

[0394] In some embodiments, the clustering unit includes a processing subunit, a merging subunit, a building subunit, and a projection subunit;

[0395] The processing subunit is used to perform spatial clustering processing on the point cloud data to be identified according to each distance threshold, so as to obtain the identification result corresponding to each distance threshold.

[0396] The merging subunit is used to merge the recognition results based on the set combination method and the target spatial location and target size contained in each recognition result, so as to obtain multiple merged results;

[0397] Construct sub-cells to build a grid according to the set yaw angle interval and spatial position interval;

[0398] The projection sub-unit is used to project each merging result onto the grid to determine the optimal merging result, and use the optimal merging result as the target 3D bounding box; wherein, the target 3D bounding box includes the optimal target center position coordinates, the optimal target size, and the optimal target yaw angle.

[0399] In some embodiments, the merging subunit is used to identify the overlap relationship between the target spatial location and target size contained in each identification result under each combination in a pairwise manner; wherein, the target spatial location includes the center position coordinates of the target, and the target size includes the length, width and height of the target; according to the overlap relationship corresponding to each combination, the identification results under each combination are merged to obtain the merging result corresponding to each combination.

[0400] In some embodiments, the projection subunit is used to project the first target contained in each merging result onto the grid; wherein, the first target is any identical target contained in each merging result; and the yaw angle corresponding to the grid cell with the most point cloud data of the projected first target is taken as the optimal target yaw angle.

[0401] In some embodiments, the integration unit includes a filtering subunit, an analysis subunit, and a fusion subunit;

[0402] The filtering subunit is used to filter the target three-dimensional bounding box based on the target three-dimensional bounding box and the image of the parking space corresponding to the parking space number closest to the target three-dimensional bounding box, so as to obtain the suspected vehicle target recognition result.

[0403] The analysis subunit is used to analyze the vehicle 3D bounding boxes contained in the suspected vehicle target recognition results and the initial target detection results according to the consistency judgment rules, so as to determine the consistency.

[0404] The fusion subunit is used to fuse the vehicle 3D bounding boxes contained in the suspected vehicle target recognition results and the initial target detection results based on the merging method matched by the consistency degree, so as to obtain the final target detection result.

[0405] In some embodiments, the filtering subunit is used to acquire the berth image corresponding to the berth number closest to the target three-dimensional bounding box; back-project the point cloud data corresponding to the target three-dimensional bounding box onto the berth image to determine the first target rectangular region; use a target detection algorithm to perform target recognition on the berth image to determine the second target rectangular region; and perform target filtering on the target three-dimensional bounding box based on the first target rectangular region and the second target rectangular region to obtain the suspected vehicle target recognition result.

[0406] In some embodiments, the filtering subunit is used to calculate the intersection-union ratio (IUR) between the first target rectangular region and the second target rectangular region; if the IUR is greater than or equal to the upper limit of the IUR, the point cloud data corresponding to the first target rectangular region is filtered out from the point cloud data corresponding to the target three-dimensional bounding box.

[0407] In some embodiments, the analysis subunit is used to call the consistency calculation formula to process the vehicle 3D bounding boxes contained in the suspected vehicle target recognition result and the initial target detection result, so as to obtain the consistency score; the consistency calculation formula is:

[0408]

[0409] Where J represents the degree of consistency. This indicates the 3D bounding box of the vehicle included in the suspected vehicle target recognition result. This represents the 3D bounding box of the vehicle included in the initial target detection results. This represents the intersection-union ratio (IoU) of the vehicle 3D bounding boxes included in the suspected vehicle target recognition result and the vehicle 3D bounding boxes included in the initial target detection result. This indicates the yaw angle included in the suspected vehicle target identification result. This indicates the yaw angle included in the initial target detection results.

[0410] In some embodiments, the fusion subunit is used to take the initial target detection result as the final target detection result when the consistency is greater than a first threshold; when the consistency is less than or equal to the first threshold and greater than a second threshold, the vehicle 3D bounding boxes contained in the suspected vehicle target recognition result and the initial target detection result are back-projected onto the camera image to obtain the corresponding suspected vehicle rectangular region and the initial vehicle rectangular region; and the vehicle 3D bounding boxes contained in the suspected vehicle target recognition result and the initial target detection result are merged according to the image features contained in the suspected vehicle rectangular region and the initial vehicle rectangular region to obtain the final target detection result.

[0411] In some embodiments, it further includes a kinematic parameter acquisition unit, a parameter analysis unit, a bounding box analysis unit, and a fusion unit;

[0412] The kinematic parameter acquisition unit is used to acquire the kinematic parameters of the target within a preset time period before the current moment; wherein, the kinematic parameters include the horizontal velocity and the angular velocity of the yaw angle change;

[0413] The parameter analysis unit is used to analyze kinematic parameters using the target tracking algorithm to determine the prediction result containing the estimated pose information of the target at the current moment.

[0414] The bounding box analysis unit is used to analyze the 3D bounding boxes contained in the final target detection result and the 3D bounding boxes corresponding to the prediction result according to the consistency judgment rules, so as to determine the new consistency.

[0415] The fusion unit is used to fuse the 3D bounding boxes contained in the final target detection result and the 3D bounding boxes corresponding to the prediction result based on the merging method matched by the new consistency, so as to obtain the integrated result.

[0416] In some embodiments, the berth status determination unit is also included;

[0417] The berth status determination unit is used to determine the berth's usage status based on the integration results and the three-dimensional spatial relationship of the berth.

[0418] In some embodiments, a correlation analysis unit is also included;

[0419] The correlation analysis unit is used to perform feature correlation analysis on the initial target detection results and the integrated results to obtain the vehicle's three-dimensional trajectory information.

[0420] In some embodiments, it further includes a verification unit, a back projection unit, and a fitting unit;

[0421] The verification unit is used to verify the vehicle's three-dimensional trajectory information in chronological order and filter out the three-dimensional trajectory segments belonging to the same vehicle.

[0422] The back projection unit is used to back project the three-dimensional bounding box corresponding to the vehicle in each three-dimensional trajectory segment onto the camera image corresponding to the nearest parking space in three-dimensional space to obtain the rectangular box of each vehicle.

[0423] The fitting unit is used to fit each three-dimensional trajectory segment using the feature information contained in each vehicle's bounding box, so as to obtain complete three-dimensional trajectory information.

[0424] In some embodiments, the fitting unit includes a first extraction subunit, a first merging subunit, a second extraction subunit, a second merging subunit, a smoothing point determination subunit, a trajectory construction subunit, and a third merging subunit;

[0425] The first extraction subunit is used to extract the license plate information contained in the rectangular frame of each vehicle;

[0426] The first merging subunit is used to merge the three-dimensional trajectory segments to obtain complete three-dimensional trajectory information when the license plate information contained in the rectangular frames of each vehicle is consistent.

[0427] The second extraction subunit is used to extract the vehicle texture information contained in each vehicle rectangle when the license plate information contained in each vehicle rectangle is inconsistent.

[0428] The second merging subunit is used to merge the three-dimensional trajectory segments to obtain complete three-dimensional trajectory information, provided that the vehicle texture information contained in each vehicle rectangle is matched.

[0429] The smoothing point determination subunit is used to determine the smoothing point on each three-dimensional trajectory segment based on the angle of the tangent vector of each point on each three-dimensional trajectory segment when the vehicle texture information contained in each vehicle rectangle does not match.

[0430] The trajectory construction subunit is used to fit the line segments formed by the smooth points on each three-dimensional trajectory segment and the endpoints on each three-dimensional trajectory segment to construct the predicted trajectory line.

[0431] The third merging subunit is used to merge each three-dimensional trajectory segment with the predicted trajectory line to obtain complete three-dimensional trajectory information.

[0432] In some embodiments, the trajectory construction subunit is used to construct a first trajectory segment based on a smoothing point on a first three-dimensional trajectory segment and the end endpoint on the first three-dimensional trajectory segment; and to construct a second trajectory segment based on a smoothing point on a second three-dimensional trajectory segment and the start endpoint on the second three-dimensional trajectory segment; wherein the first three-dimensional trajectory segment and the second three-dimensional trajectory segment are two three-dimensional trajectory segments that are adjacent in time on each three-dimensional trajectory segment; the first trajectory segment is fitted to obtain a first trajectory prediction line; the second trajectory segment is fitted to obtain a second trajectory prediction line; and if the vertical distance between the first trajectory prediction line and the second trajectory prediction line is less than a set distance threshold, the first trajectory prediction line and the second trajectory prediction line are fitted to determine the predicted trajectory line between the end endpoint on the first three-dimensional trajectory segment and the start endpoint on the second three-dimensional trajectory segment.

[0433] In some embodiments, a vehicle status determination unit is also included;

[0434] The vehicle status determination unit is used to determine the status and time information of each vehicle entering and leaving the parking space based on the complete three-dimensional trajectory information of each vehicle and the three-dimensional positional relationship of the parking space.

[0435] In some embodiments, the system further includes an image acquisition unit, an image filtering unit, and a license plate extraction unit;

[0436] The image acquisition unit is used to acquire the original camera images corresponding to the complete three-dimensional trajectory information of each vehicle;

[0437] The image filtering unit is used to filter out license plate region images from each original camera image based on the boundary sharpness of the license plate region, the license plate envelope tilt and / or the number of license plate character segments in each original camera image;

[0438] The license plate extraction unit is used to extract license plate information from the license plate area image.

[0439] Figure 12 For a description of the features in the corresponding embodiments, please refer to Figure 1 and Figure 11 The relevant descriptions of the corresponding embodiments will not be repeated here.

[0440] As can be seen from the above technical solution, the process involves acquiring historical point cloud data corresponding to the road scene, acquiring current point cloud data corresponding to the road scene at the current time, and acquiring the current image; filtering the current point cloud data based on a pre-set drivable three-dimensional space range, radar calibration parameters, and historical point cloud data to obtain point cloud data to be identified; analyzing the current image and the point cloud data to be identified using a BEV fusion perception algorithm to obtain initial target detection results; performing multi-scale clustering processing on the point cloud data to be identified according to multiple set distance thresholds to determine the target's three-dimensional bounding box; integrating the target's three-dimensional bounding box and the initial target detection results to obtain the final target detection result; wherein, the target detection result includes the identified vehicle type and its corresponding three-dimensional bounding box. The beneficial effects of this invention are that point cloud data acquisition is simple and can be automatically updated. By filtering the point cloud data, environmental interference can be effectively eliminated, thereby reducing the computational load of the BEV fusion perception algorithm. Considering that the point cloud data to be identified may contain spatially close targets, single-scale spatial clustering can easily lead to under-clustering or over-clustering. Therefore, this invention effectively solves this problem by employing multi-scale clustering processing, ensuring the accuracy of the target's 3D bounding box. Furthermore, the target's 3D bounding box includes all targets appearing within the corresponding region of the point cloud data, meaning it includes unidentifiable non-standard vehicles. By integrating the target's 3D bounding box with the initial target detection results, the final target detection result ensures that it includes both vehicles that the BEV fusion perception algorithm can identify and non-standard vehicles, guaranteeing accurate target identification in complex roadside scenarios.

[0441] Figure 13 A structural diagram of a target recognition device provided in an embodiment of the present invention is shown below. Figure 13 As shown, the target recognition device includes: a memory 130 for storing computer programs;

[0442] The processor 131 is used to implement the steps of the target recognition method as described in the above embodiments when executing a computer program.

[0443] The target recognition device provided in this embodiment may include, but is not limited to, smartphones, tablets, laptops, or desktop computers.

[0444] The processor 131 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 131 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 131 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 131 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 131 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0445] The memory 130 may include one or more storage media, which may be non-transitory. The memory 130 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In this embodiment, the memory 130 is used to store at least the following computer program 1301, which, after being loaded and executed by the processor 131, is capable of implementing the relevant steps of the target recognition method disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 130 may also include an operating system 1302 and data 1303, and the storage method may be temporary storage or permanent storage. The operating system 1302 may include Windows, Unix, Linux, etc. The data 1303 may include, but is not limited to, point cloud data, images, etc.

[0446] In some embodiments, the target recognition device may further include a display screen 132, an input / output interface 133, a communication interface 134, a power supply 135, and a communication bus 136.

[0447] Those skilled in the art will understand that Figure 13 The structure shown does not constitute a limitation on the target recognition device and may include more or fewer components than illustrated.

[0448] It is understood that if the target identification method in the above embodiments is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the current technology, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and executes all or part of the steps of the methods in the various embodiments of the present invention. The aforementioned storage medium includes: USB flash drive, mobile hard drive, read-only memory (ROM), random access memory (RAM), electrically erasable programmable ROM, register, hard disk, removable disk, CD-ROM, magnetic disk, or optical disk, and other media capable of storing program code.

[0449] Based on this, embodiments of the present invention also provide a storage medium storing a computer program, which, when executed by a processor, implements the steps of the target recognition method described above.

[0450] The foregoing has provided a detailed description of a target identification method, apparatus, device, and storage medium provided by embodiments of the present invention. The various embodiments are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0451] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0452] The target identification method, apparatus, device, and storage medium provided by the present invention have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of the present invention. It should be noted that those skilled in the art can make various improvements and modifications to the present invention without departing from the principles of the invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

Claims

1. A target recognition method, characterized in that, include: Acquire historical point cloud data corresponding to the road scene, acquire current point cloud data and current image corresponding to the road scene at the current time; Based on the pre-set drivable three-dimensional space range, radar calibration parameters, and the historical point cloud data, the current point cloud data is filtered to obtain the point cloud data to be identified. The current image and the point cloud data to be identified are analyzed according to the BEV fusion perception algorithm to obtain the initial target detection result; The point cloud data to be identified is subjected to multi-scale clustering processing according to multiple set distance thresholds to determine the target three-dimensional bounding box; The target 3D bounding box and the initial target detection results are integrated to obtain the final target detection result; wherein, the target detection result includes the identified vehicle type and its corresponding 3D bounding box; The process of integrating the target's 3D bounding box and the initial target detection results to obtain the final target detection result includes: Obtain the image of the berth number that is closest to the target 3D bounding box; The point cloud data corresponding to the target 3D bounding box is back-projected onto the berth image to determine the first target rectangular region; The target detection algorithm is used to identify the target in the berth image to determine the second target rectangular region; Based on the first target rectangular region and the second target rectangular region, the target three-dimensional bounding box is filtered to obtain the suspected vehicle target recognition result; Based on the consistency judgment rules, the vehicle 3D bounding boxes contained in the suspected vehicle target recognition results and the initial target detection results are analyzed to determine the consistency. Based on the merging method matched by the consistency, the vehicle 3D bounding boxes contained in the suspected vehicle target recognition result and the initial target detection result are fused to obtain the final target detection result.

2. The target recognition method according to claim 1, characterized in that, The process of obtaining the current point cloud data corresponding to the road scene at the current time includes: The historical point cloud data and the current point cloud data are matched using a point cloud matching algorithm to determine the first transformation parameters; wherein, the first transformation parameters include a first rotation matrix and a first translation vector; The deviation of the current point cloud data is determined based on the first rotation matrix and the first translation vector; Based on the set deviation requirements, the deviation, the first rotation matrix, the first translation vector, and the current point cloud data are processed to obtain the final current point cloud data.

3. The target recognition method according to claim 2, characterized in that, The process of processing the deviation, the first rotation matrix, the first translation vector, and the current point cloud data based on the set deviation requirement to obtain the final current point cloud data includes: Determine whether the deviation is less than the deviation threshold; If the deviation is less than the deviation threshold, the current point cloud data is used as the final current point cloud data. If the deviation is greater than or equal to the deviation threshold, the current point cloud data is transformed using the first rotation matrix and the first translation vector to obtain pose-calibrated point cloud data, and the pose-calibrated point cloud data is used as the final current point cloud data.

4. The target recognition method according to claim 3, characterized in that, The acquisition of historical point cloud data corresponding to the road scene includes: Acquire the initial point cloud data corresponding to different times within a set time period for the road scene; The point cloud matching algorithm is used to sequentially match the initial point cloud data at adjacent time points to determine the initial transformation parameters corresponding to each adjacent time point; wherein, the initial transformation parameters include an initial rotation matrix and an initial translation vector; The initial deviation is determined based on the initial rotation matrix and the initial translation vector corresponding to each of the adjacent time moments. Based on the aforementioned deviation requirements, the initial deviations, initial rotation matrices, initial translation vectors, and initial point cloud data are processed to obtain the historical point cloud data.

5. The target recognition method according to claim 4, characterized in that, Based on the deviation requirement, the initial deviation, initial rotation matrix, initial translation vector, and initial point cloud data are processed to obtain the historical point cloud data, including: Determine whether the initial deviation corresponding to each of the adjacent time points is less than the deviation threshold; If the initial deviation at each of the adjacent time points is less than the deviation threshold, the initial point cloud data are superimposed to form the historical point cloud data. If the initial deviation at each of the adjacent time points is not less than the deviation threshold, the initial point cloud data corresponding to each of the initial rotation matrices and the initial translation vectors are used to transform the initial point cloud data to obtain the pose-calibrated initial point cloud data, and the pose-calibrated initial point cloud data are superimposed to obtain the historical point cloud data.

6. The target recognition method according to claim 3, characterized in that, Before analyzing the current image and the point cloud data to be identified according to the BEV fusion sensing algorithm to obtain the initial target detection result, the method further includes: According to the set unified coordinate system, the current image and all the point cloud data to be identified are subjected to coordinate transformation to obtain the coordinate transformed image and the coordinate transformed point cloud data to be identified. The step of analyzing the current image and the point cloud data to be identified according to the BEV fusion perception algorithm to obtain the initial target detection result includes: The BEV fusion sensing algorithm is used to analyze the coordinate-transformed image and the coordinate-transformed point cloud data to be identified in order to obtain the initial target detection results.

7. The target recognition method according to claim 6, characterized in that, Before performing coordinate transformation on the current image and all the point cloud data to be identified according to a set unified coordinate system to obtain the coordinate-transformed image and the coordinate-transformed point cloud data to be identified, the method further includes: Based on the camera's intrinsic parameters, distortion correction is performed on the initial image captured by the camera to obtain a corrected image; The current point cloud data and the pose-calibrated point cloud data are projected onto the corrected image to obtain a first image with the same name and a second image with the same name. Based on the matching relationship of corresponding points between the first and second identical images, the image calibration affine transformation parameters are determined. The initial image is transformed using the image calibration affine transformation parameters to obtain a pose-calibrated image, and the pose-calibrated image is used as the current image.

8. The target recognition method according to claim 6, characterized in that, The step of analyzing the current image and the point cloud data to be identified according to the BEV fusion perception algorithm to obtain the initial target detection result includes: The BEV fusion sensing algorithm is used to extract features from the coordinate-transformed image to obtain three-dimensional image spatial features; The BEV fusion sensing algorithm is used to extract features from the coordinate-transformed point cloud data to obtain three-dimensional point cloud spatial features. The aligned 3D image spatial features and the 3D point cloud spatial features are fused to obtain fused features; The fused features are analyzed using the target detection head included in the BEV fusion perception algorithm to output the initial target detection result; wherein, the initial target detection result includes the identified targets and their pose information.

9. The target recognition method according to claim 1, characterized in that, The step of filtering the current point cloud data based on the pre-set drivable three-dimensional space range, radar calibration parameters, and the historical point cloud data to obtain the point cloud data to be identified includes: Using a pre-defined drivable three-dimensional space range, extract drivable area point cloud data from the current point cloud data; Based on the radar calibration parameters and the historical point cloud data, the drivable area point cloud data is filtered to obtain the point cloud data to be identified.

10. The target recognition method according to claim 9, characterized in that, The step of filtering the drivable area point cloud data according to the radar calibration parameters and the historical point cloud data to obtain the point cloud data to be identified includes: Ground point cloud data is filtered out from the drivable area point cloud data according to the radar calibration parameters to obtain initial filtered point cloud data; The historical point cloud data is matched with the initial filtered point cloud data to filter out background point cloud data from the initial filtered point cloud data. The initial filtered point cloud data after removing the background point cloud data is used as the point cloud data to be identified.

11. The target recognition method according to claim 1, characterized in that, The step of performing multi-scale clustering processing on the point cloud data to be identified according to multiple set distance thresholds to determine the target 3D bounding box includes: Spatial clustering processing is performed on the point cloud data to be identified according to each of the distance thresholds to obtain the identification result corresponding to each distance threshold. Based on the set combination method and the target spatial location and target size contained in each of the recognition results, the recognition results are merged to obtain multiple merged results; Construct a grid according to the set yaw angle interval and spatial position interval; The merged results are projected onto the grid to determine the optimal merged result, and the optimal merged result is used as the target 3D bounding box; wherein, the target 3D bounding box includes the optimal target center position coordinates, the optimal target size, and the optimal target yaw angle.

12. The target recognition method according to claim 11, characterized in that, Based on the set combination method and the target spatial location and target size contained in each of the recognition results, the recognition results are merged to obtain multiple merged results, including: By combining them in pairs, the overlap relationship between the target spatial location and target size contained in each of the identification results under each combination is identified; wherein, the target spatial location includes the center position coordinates of the target, and the target size includes the length, width and height of the target; Based on the overlap relationship corresponding to each combination, the recognition results under each combination are merged to obtain the merged result corresponding to each combination.

13. The target recognition method according to claim 11, characterized in that, The step of projecting each of the merged results onto the grid to determine the optimal merged result includes: The first target contained in each of the merging results is projected onto the grid; wherein, the first target is any identical target contained in each of the merging results; The yaw angle corresponding to the grid cell that projects the most point cloud data of the first target in the grid is taken as the optimal target yaw angle.

14. The target recognition method according to claim 1, characterized in that, The step of filtering the target's three-dimensional bounding box based on the first target rectangular region and the second target rectangular region to obtain the suspected vehicle target recognition result includes: Calculate the intersection-union ratio of the first target rectangular region and the second target rectangular region; If the intersection-union ratio is greater than or equal to the upper limit of the intersection-union ratio, the point cloud data corresponding to the first target rectangular region is filtered out from the point cloud data corresponding to the target three-dimensional bounding box.

15. The target recognition method according to claim 1, characterized in that, The process involves analyzing the vehicle 3D bounding boxes contained in the suspected vehicle target recognition result and the initial target detection result, based on consistency judgment rules, to determine the consistency, including: The consistency score is obtained by applying the consistency calculation formula to process the vehicle 3D bounding boxes contained in the suspected vehicle target recognition result and the initial target detection result, respectively; the consistency calculation formula is as follows: ； in, J Indicates consistency. This indicates the three-dimensional bounding box of the vehicle included in the suspected vehicle target recognition result. This indicates the vehicle's 3D bounding box included in the initial target detection result. This represents the intersection-union ratio (IoU) of the vehicle 3D bounding boxes included in the suspected vehicle target recognition result and the vehicle 3D bounding boxes included in the initial target detection result. This indicates the yaw angle included in the suspected vehicle target identification result. This indicates the yaw angle included in the initial target detection result.

16. The target recognition method according to claim 15, characterized in that, The merging method based on the consistency degree, which merges the vehicle 3D bounding boxes contained in the suspected vehicle target recognition result and the initial target detection result respectively, to obtain the final target detection result includes: If the consistency is greater than the first threshold, the initial target detection result is used as the final target detection result; When the consistency is less than or equal to the first threshold and greater than the second threshold, the vehicle three-dimensional bounding boxes contained in the suspected vehicle target recognition result and the initial target detection result are respectively back-projected onto the camera image to obtain the corresponding suspected vehicle rectangular region and the initial vehicle rectangular region. Based on the image features contained in the suspected vehicle rectangular region and the initial vehicle rectangular region, the vehicle 3D bounding boxes contained in the suspected vehicle target recognition result and the initial target detection result are merged to obtain the final target detection result.

17. The target recognition method according to claim 1, characterized in that, After integrating the target's 3D bounding box and the initial target detection results to obtain the final target detection result, the method further includes: Obtain the kinematic parameters of the target within a preset time period before the current moment; wherein, the kinematic parameters include the horizontal velocity and the angular velocity of the yaw angle change; The kinematic parameters are analyzed using a target tracking algorithm to determine a prediction result that includes the estimated pose information of the target at the current moment; Based on the consistency judgment rules, the three-dimensional bounding box contained in the final target detection result and the three-dimensional bounding box corresponding to the prediction result are analyzed to determine a new consistency. Based on the merging method matched by the new consistency, the 3D bounding box contained in the final target detection result and the 3D bounding box corresponding to the prediction result are fused to obtain the integrated result.

18. The target recognition method according to claim 17, characterized in that, After fusing the 3D bounding box contained in the final target detection result and the 3D bounding box corresponding to the prediction result according to the merging method matched based on the new consistency, to obtain the integrated result, the method further includes: Based on the integration results and the three-dimensional spatial relationship of the berths, the usage status of the berths is determined.

19. The target recognition method according to claim 17, characterized in that, After fusing the 3D bounding box contained in the final target detection result and the 3D bounding box corresponding to the prediction result according to the merging method matched based on the new consistency, to obtain the integrated result, the method further includes: Feature association analysis is performed on the initial target detection results and the integrated results to obtain the vehicle's three-dimensional trajectory information.

20. The target recognition method according to claim 19, characterized in that, After performing feature association analysis on the initial target detection results and the integrated results to obtain the vehicle's three-dimensional trajectory information, the method further includes: The vehicle's three-dimensional trajectory information is verified in chronological order, and the three-dimensional trajectory segments belonging to the same vehicle are selected. The three-dimensional bounding box corresponding to each vehicle in each of the three-dimensional trajectory segments is back-projected onto the camera image corresponding to the nearest parking space in the three-dimensional space to obtain the rectangular box of each vehicle. Using the feature information contained in each of the vehicle bounding boxes, each of the three-dimensional trajectory segments is fitted to obtain complete three-dimensional trajectory information.

21. The target recognition method according to claim 20, characterized in that, The step of fitting each three-dimensional trajectory segment with the feature information contained in each of the vehicle bounding boxes to obtain complete three-dimensional trajectory information includes: Extract the license plate information contained in the rectangular frames of each vehicle; If the license plate information contained in each of the vehicle rectangles is consistent, the three-dimensional trajectory segments are merged to obtain complete three-dimensional trajectory information. If the license plate information contained in each of the vehicle rectangles is inconsistent, extract the vehicle texture information contained in each of the vehicle rectangles. If the vehicle texture information contained in each of the vehicle rectangles matches, the three-dimensional trajectory segments are merged to obtain complete three-dimensional trajectory information. In the case where the vehicle texture information contained in each of the vehicle rectangles does not match, the smoothing point on each of the three-dimensional trajectory segments is determined according to the angle of the curve tangent vector of each point on each of the three-dimensional trajectory segments. The line segments formed by the smooth points on each of the three-dimensional trajectory segments and the endpoints on each of the three-dimensional trajectory segments are fitted to construct the predicted trajectory line; The three-dimensional trajectory segments are merged with the predicted trajectory line to obtain complete three-dimensional trajectory information.

22. The target recognition method according to claim 21, characterized in that, The step of fitting the line segments formed by the smoothing points on each of the three-dimensional trajectory segments and the endpoints on each of the three-dimensional trajectory segments to construct the predicted trajectory line includes: The first trajectory segment is constructed based on the smoothing point on the first three-dimensional trajectory segment and the end point on the first three-dimensional trajectory segment; A second trajectory segment is constructed based on the smoothing point on the second three-dimensional trajectory segment and the starting endpoint on the second three-dimensional trajectory segment; wherein, the first three-dimensional trajectory segment and the second three-dimensional trajectory segment are two three-dimensional trajectory segments that are adjacent in time on each of the three-dimensional trajectory segments; The first trajectory segment is fitted to obtain the first trajectory prediction line; The second trajectory segment is fitted to obtain the second trajectory prediction line; If the vertical distance between the first trajectory prediction line and the second trajectory prediction line is less than a set distance threshold, the first trajectory prediction line and the second trajectory prediction line are fitted to determine the predicted trajectory line between the end endpoint of the first three-dimensional trajectory segment and the start endpoint of the second three-dimensional trajectory segment.

23. The target recognition method according to claim 20, characterized in that, After fitting each of the three-dimensional trajectory segments using the feature information contained in each of the vehicle bounding boxes to obtain complete three-dimensional trajectory information, the method further includes: Based on the complete three-dimensional trajectory information of each vehicle and its three-dimensional positional relationship with the parking space, the status and time information of each vehicle entering and leaving the parking space are determined.

24. The target recognition method according to claim 20, characterized in that, After fitting each of the three-dimensional trajectory segments using the feature information contained in each of the vehicle bounding boxes to obtain complete three-dimensional trajectory information, the method further includes: Obtain the original camera images corresponding to the complete 3D trajectory information of each vehicle; Based on the boundary clarity of the license plate area, the license plate envelope tilt, and / or the number of license plate character segments in each of the original camera images, license plate area images are selected from each of the original camera images; License plate information is extracted from the license plate area image.

25. A target recognition device, characterized in that, It includes an acquisition unit, a filtering unit, an analysis unit, a clustering unit, and an integration unit; The acquisition unit is used to acquire historical point cloud data corresponding to the road scene, acquire current point cloud data and current image corresponding to the road scene at the current time; The filtering unit is used to filter the current point cloud data according to the preset drivable three-dimensional space range, radar calibration parameters and the historical point cloud data to obtain the point cloud data to be identified. The analysis unit is used to analyze the current image and the point cloud data to be identified according to the BEV fusion perception algorithm to obtain the initial target detection result; The clustering unit is used to perform multi-scale clustering processing on the point cloud data to be identified according to multiple set distance thresholds in order to determine the target three-dimensional bounding box. The integration unit is used to integrate the target 3D bounding box and the initial target detection result to obtain the final target detection result; wherein, the target detection result includes the identified vehicle type and its corresponding 3D bounding box; The integration unit includes a filtering subunit, an analysis subunit, and a fusion subunit. The filtering subunit is used to acquire the berth image corresponding to the berth number closest to the target 3D bounding box; back-project the point cloud data corresponding to the target 3D bounding box onto the berth image to determine the first target rectangular region; use a target detection algorithm to perform target recognition on the berth image to determine the second target rectangular region; and filter the target 3D bounding box based on the first and second target rectangular regions to obtain the suspected vehicle target recognition result. The analysis subunit is used to analyze the vehicle 3D bounding boxes contained in the suspected vehicle target recognition result and the initial target detection result according to the consistency judgment rule to determine the consistency. The fusion subunit is used to fuse the vehicle 3D bounding boxes contained in the suspected vehicle target recognition result and the initial target detection result based on the merging method matched by the consistency to obtain the final target detection result.

26. A target recognition device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the target recognition method as described in any one of claims 1 to 24.

27. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the steps of the target recognition method as described in any one of claims 1 to 24.

Images