Target positioning precision optimization method and device based on roadside fusion perception

Abstract

The invention provides a target positioning precision optimization method and device based on road side fusion perception. The method comprises the following steps: acquiring latitude and longitude information of a centroid point of a target object through a millimeter-wave radar on a road side, and acquiring pixel position information of a key point of the target object through a camera on the road side; determining sensing positioning data of the target object based on the latitude and longitude information of the centroid point of the target object and the pixel position information of the key point of the target object; determining the target positioning precision based on the sensing positioning data and the real positioning data of the target object; and optimizing the target positioning precision by adjusting the plane position of the centroid point of the target object and / or the latitude and longitude information of the centroid point of the target object until the target positioning precision reaches a preset precision requirement. According to the target positioning precision optimization method and device based on the roadside fusion perception, the material center position of the target object is optimized according to the differential expression of the perception data and the truth value data, and the target object positioning precision of the roadside fusion perception is improved.

Description

Technical Field

[0001] The present invention relates to the field of target positioning technology, and in particular to a method and device for optimizing target positioning accuracy based on roadside fusion perception. Background Art

[0002] Transportation is an important link connecting cities and one of the foundations of modern industrial society. It has a significant impact on our lives. With the rapid development of new-generation information technologies such as artificial intelligence and 5G, the transportation system has become a complex system. The current problems in cities such as traffic congestion, frequent accidents, and environmental pollution cannot be solved well from the perspective of roads or vehicles alone. Vehicle-road collaboration aims to achieve vehicle-road information sharing based on advanced communication technologies such as 5G by deploying intelligent perception computing equipment on the road side, and to complement the limited line of sight and perception blind spots of vehicle-side perception through road-side perception, thereby greatly improving the safety of autonomous driving.

[0003] In the vehicle-road cooperative system, the most core element is the traffic participant element, so the fusion perception algorithm function and the accuracy of its data are particularly important. The roadside fusion perception algorithm mainly realizes the data fusion of multiple roadside sensors (including but not limited to cameras, millimeter wave radars, lidars, etc.) through the algorithm deployed in the edge computing unit, outputs structured data such as targets, events, and traffic, and supports applications such as networked vehicle safety, traffic control optimization, equipment operation and maintenance management, and digital twins.

[0004] The current roadside fusion perception algorithm has limited target recognition capabilities due to the technical limitations of the roadside sensors themselves. It has problems such as insensitivity to target recognition and poor recognition accuracy, resulting in poor target positioning accuracy. Summary of the invention

[0005] The present invention provides a method and device for optimizing target positioning accuracy based on roadside fusion perception, which are used to solve the technical problem of poor target positioning accuracy in the prior art.

[0006] In a first aspect, the present invention provides a method for optimizing target positioning accuracy based on roadside fusion perception, comprising the following steps.

[0007] The longitude and latitude information of the centroid of the target object is obtained through the millimeter wave radar on the road side, and the pixel position information of the key points of the target object is obtained through the camera on the road side; Determine the perception positioning data of the target object based on the latitude and longitude information of the centroid point of the target object and the pixel position information of the key points of the target object; Determine the target positioning accuracy based on the perceived positioning data and the real positioning data of the target object; the real positioning data is detected by using a detection device deployed on the target object; The target positioning accuracy is optimized by adjusting the plane position of the center of mass of the target object and / or the longitude and latitude information of the center of mass of the target object until the target positioning accuracy reaches a preset accuracy requirement.

[0008] In some embodiments, optimizing the target positioning accuracy by adjusting the plane position of the center of mass of the target object and / or the longitude and latitude information of the center of mass of the target object includes: Compensating the horizontal coordinate of the center of mass of the target object based on the real positioning data, and compensating the vertical coordinate of the center of mass of the target object based on the size of the target object; and / or

[0009] When the millimeter wave radar detects that the target object decelerates and the speed is lower than a preset first threshold, deleting the currently acquired longitude and latitude position of the centroid of the target object; and / or

[0010] The sensed positioning data is used as the longitude and latitude position of the center of mass of the current target object; the sensed positioning data is not less than a second threshold and not greater than 1.5 times the second threshold; the second threshold is 2 / 3 times the longitude and latitude position of the target object detected by the millimeter wave radar; and / or

[0011] When the millimeter-wave radar detects that the target object is accelerating and the speed exceeds a preset third threshold, the millimeter-wave radar is restarted to detect the longitude and latitude positions of the center of mass of the target object.

[0012] In some embodiments, determining the perception positioning data of the target object based on the latitude and longitude information of the centroid point of the target object and the pixel position information of the key points of the target object includes: The pixel position information of the key point of the target object is converted into the northeast celestial coordinate system to obtain the first projection of the key point of the target object under the bird's-eye view, and the longitude and latitude information of the centroid of the target object is converted into the northeast celestial coordinate system to obtain the second projection of the centroid of the target object under the bird's-eye view; Determine the overlap rate between the first projection and the second projection using an intersection-over-union (IOU) algorithm; Obtaining a confidence level corresponding to the first projection; the confidence level corresponding to the first projection refers to a confidence level of whether a target object exists in the first projection; When the overlap rate exceeds a preset overlap rate threshold, the perception positioning data of the target object is determined based on the confidence level corresponding to the first projection.

[0013] In some embodiments, determining the perceived positioning data of the target object based on the confidence level corresponding to the first projection includes: When the distance between the camera corresponding to the first projection and the first projection is within a preset distance range and the confidence level corresponding to the first projection is greater than a preset confidence threshold, the first projection is used as the target object position information; and / or

[0014] When the distance between the camera corresponding to the first projection and the first projection is not within a preset distance range, or the confidence level corresponding to the first projection is less than or equal to a preset confidence threshold, taking the second projection as the target object position information; The target object position information and the target object heading and speed information are used as the perception positioning data of the target object.

[0015] In some embodiments, the method further comprises: When the speed of the target object detected by the millimeter-wave radar is greater than a preset speed threshold, the heading and speed data of the target object detected by the millimeter-wave radar are used as the final heading and speed information of the target object; and / or

[0016] When the speed of the target object detected by the millimeter wave radar is less than or equal to the preset speed threshold, the heading and speed data of the target object detected by the camera are used as the final heading and speed information of the target object.

[0017] In some embodiments, determining the target positioning accuracy based on the perceived positioning data and the real positioning data of the target object includes: Filtering matching positioning data from the sensed positioning data; the matching positioning data refers to positioning data that matches the real positioning data of the target object; Performing timestamp alignment on the selected matching positioning data and the real positioning data to obtain multiple groups of trajectory point pairs; each trajectory point pair includes a first trajectory point in the matching positioning data and a second trajectory point in the real positioning data, and the timestamps of the first trajectory point and the second trajectory point in each trajectory point are aligned; Calculating the Euclidean distance between the first trajectory point and the second trajectory point in each trajectory point pair to obtain the Euclidean distance of the trajectory point pair; The average value of the Euclidean distances of all trajectory point pairs is taken as the target positioning accuracy.

[0018] In some embodiments, the method further comprises: The real positioning data of the target object is collected by a sensor deployed on the target object or a real-time differential positioning device RTK deployed on the target object.

[0019] In a second aspect, the present invention provides a device for optimizing target positioning accuracy based on roadside fusion perception, comprising the following modules.

[0020] An acquisition module is used to acquire the latitude and longitude information of the centroid of the target object through a roadside millimeter wave radar, and acquire the pixel position information of the key points of the target object through a roadside camera; A first determination module, configured to determine the perception positioning data of the target object based on the latitude and longitude information of the centroid point of the target object and the pixel position information of the key points of the target object; A second determination module is used to determine the target positioning accuracy based on the perceived positioning data and the real positioning data of the target object; the real positioning data is detected by using a detection device deployed on the target object; The optimization module is used to optimize the target positioning accuracy by adjusting the plane position of the center of mass of the target object and / or the longitude and latitude information of the center of mass of the target object until the target positioning accuracy reaches a preset accuracy requirement.

[0021] In some embodiments, the optimization module includes: a compensation unit, configured to compensate the horizontal coordinate of the center of mass of the target object based on the real positioning data, and to compensate the vertical coordinate of the center of mass of the target object based on the size of the target object; and / or

[0022] A first judgment unit is used to delete the currently acquired longitude and latitude position of the centroid of the target object when the millimeter wave radar detects that the target object is decelerating and the speed is lower than a preset first threshold; and / or

[0023] A first determination unit is configured to use the sensed positioning data as the longitude and latitude position of the centroid of the current target object; the sensed positioning data is not less than a second threshold and not greater than 1.5 times the second threshold; the second threshold is 2 / 3 times the longitude and latitude position of the target object detected by the millimeter wave radar; and / or

[0024] The second judgment unit is used to restart the millimeter wave radar and detect the longitude and latitude positions of the center of mass of the target object when the millimeter wave radar detects that the target object is accelerating and the speed exceeds a preset third threshold.

[0025] In some embodiments, the first determining module includes: A first acquisition unit is used to convert the pixel position information of the key point of the target object into the northeast celestial coordinate system to obtain a first projection of the key point of the target object from a bird's-eye view, and convert the longitude and latitude information of the centroid point of the target object into the northeast celestial coordinate system to obtain a second projection of the centroid point of the target object from a bird's-eye view; A second determining unit, configured to determine an overlap ratio between the first projection and the second projection by using an intersection-over-union (IOU) algorithm; a second acquisition unit, configured to acquire a confidence level corresponding to the first projection; the confidence level corresponding to the first projection refers to a confidence level of whether a target object exists in the first projection; The third determination unit is used to determine the perception positioning data of the target object based on the confidence level corresponding to the first projection when the overlap rate exceeds a preset overlap rate threshold.

[0026] In some embodiments, the third determining unit includes: a first determining subunit, configured to use the first projection as target object position information when the distance between the camera corresponding to the first projection and the first projection is within a preset distance range and the confidence level corresponding to the first projection is greater than a preset confidence threshold; and / or

[0027] A second determination subunit is configured to use the second projection as the target object position information when the distance between the camera corresponding to the first projection and the first projection is not within a preset distance range or the confidence level corresponding to the first projection is less than or equal to a preset confidence threshold; The third determination subunit is used to use the target object position information and the target object heading and speed information as the perception positioning data of the target object.

[0028] In some embodiments, it also includes: a fourth determination unit, configured to use the heading and speed data of the target object detected by the millimeter-wave radar as final heading and speed information of the target object when the speed of the target object detected by the millimeter-wave radar is greater than a preset speed threshold; and / or

[0029] The fifth determination unit is used to use the heading and speed data of the target object detected by the camera as the final heading and speed information of the target object when the speed of the target object detected by the millimeter wave radar is less than or equal to the preset speed threshold.

[0030] In some embodiments, the second determining module includes: a sixth determining unit, configured to filter out matching positioning data from the sensed positioning data; the matching positioning data refers to positioning data that matches the real positioning data of the target object; A third acquisition unit is used to align the timestamps of the screened matching positioning data with the real positioning data to obtain multiple groups of trajectory point pairs; each trajectory point pair includes a first trajectory point in the matching positioning data and a second trajectory point in the real positioning data, and the timestamps of the first trajectory point and the second trajectory point in each trajectory point are aligned; a fourth acquisition unit, configured to calculate the Euclidean distance between the first trajectory point and the second trajectory point in each trajectory point pair to obtain the Euclidean distance of the trajectory point pair; The seventh determination unit is used to take the average value of the Euclidean distances of all trajectory point pairs as the target positioning accuracy.

[0031] In some embodiments, it also includes: The acquisition module is used to acquire the real positioning data of the target object through a sensor deployed on the target object or a real-time differential positioning device RTK deployed on the target object.

[0032] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for optimizing target positioning accuracy based on roadside fusion perception as described in the first aspect above is implemented.

[0033] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the target positioning accuracy optimization method based on roadside fusion perception as described in the first aspect above.

[0034] In a fifth aspect, the present invention further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the target positioning accuracy optimization method based on roadside fusion perception as described in the first aspect above.

[0035] The target positioning accuracy optimization method and device based on roadside fusion perception provided by the present invention first obtains the longitude and latitude information of the centroid of the target object through the millimeter wave radar on the roadside, and obtains the pixel position information of the key points of the target object through the camera on the roadside, and determines the perceived positioning data of the target object based on the longitude and latitude information of the centroid of the target object and the pixel position information of the key points of the target object; then determines the target positioning accuracy based on the perceived positioning data and the real positioning data of the target object, wherein the real positioning data is detected by using the detection device deployed on the target object; finally, optimizes the target positioning accuracy by adjusting the plane position of the centroid of the target object and / or the longitude and latitude information of the centroid of the target object until the target positioning accuracy reaches the preset accuracy requirement. The centroid position of the target object is optimized according to the differentiated performance of the perceived data and the true value data, thereby improving the positioning accuracy of the target object of the roadside fusion perception. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 It is a flow chart of the target positioning accuracy optimization method based on roadside fusion perception provided by the present invention.

[0038] Figure 2 It is a schematic diagram of ROI configuration provided by the present invention.

[0039] Figure 3 These are projection images of the target object from a bird's eye view under different cameras provided by the present invention.

[0040] Figure 4 It is a projection diagram of different millimeter wave radar detection targets provided by the present invention from a bird's eye view.

[0041] Figure 5 This is the true value vehicle RTK trajectory map provided by the present invention.

[0042] Figure 6 It is a target object trajectory diagram of the radar-vision fusion algorithm provided by the present invention.

[0043] Figure 7 It is a structural schematic diagram of a target positioning accuracy optimization device based on roadside fusion perception provided by the present invention.

[0044] Figure 8 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0045] The millimeter-wave radar and vision fusion algorithm mainly involves the fusion technology of millimeter-wave radar and camera, aiming to improve the accuracy and efficiency of target detection by combining the target detection capability of millimeter-wave radar and the visual information of camera. The application scenarios of this algorithm include but are not limited to traffic incident analysis, full-area detection of intersections, holographic perception and visualization, etc., to provide support for the command decision-making of traffic control departments.

[0046] In the general solution, the millimeter wave radar itself is not sensitive to the recognition of static targets, and the camera has limitations in the detection of targets at the far end of the camera and has poor positioning accuracy. From the application level, there will be problems such as target loss, discontinuous trajectory, ID change, and positioning accuracy data quality not meeting the standards. The implementation process of the general solution millimeter wave radar and visual fusion algorithm includes the following key steps: (1) Image target detection and millimeter-wave radar target detection: The image deep learning technology and radar target detection technology are used to detect the target respectively. The target object recognized by the camera is a 2D detection frame, and the center of mass is inferred and calculated. The millimeter-wave radar detects the target object and outputs the center of mass position (essentially the center of mass position of the cross section of the target object detected by the millimeter-wave radar) through the principle of electromagnetic wave and its reflection ranging.

[0047] (2) Camera and millimeter-wave radar timing: Ensure that the camera, millimeter-wave radar and edge computing device are under the same timing source to ensure that the system clock is consistent.

[0048] (3) Camera and millimeter-wave radar calibration: Ensure that the data of the camera and millimeter-wave radar can be accurately aligned in the same coordinate system, which is the basis of the fusion algorithm.

[0049] (4) Single-pole millimeter-wave radar and camera data fusion: Existing algorithms mainly use Kalman filtering and Hungarian matching methods to perform fusion matching based on the target’s location information.

[0050] (5) Multi-bar structural data fusion: The main fusion judgment is based on the target object position information.

[0051] (6) Fusion of structured data output and effect display: By fusing the data from the millimeter-wave radar and the camera to output target structured data, the target detection results can be displayed on the same screen, thereby achieving a more comprehensive display of target information.

[0052] However, the current roadside fusion perception algorithm has the following disadvantages: (1) The main basis for determining the fusion of targets is based on the target's position information, which is prone to problems such as mis-fusion and failure to fusion.

[0053] (2) The centroid of the target object identified by the camera is not the optimal solution. The 3D size information of the target is estimated based on the target type, which has some deviations. The centroid of the target object identified by the millimeter-wave radar is not the optimal solution. In general, the manufacturer selects the center of the radar reflection surface. The deviation caused by the centroid of the same target object identified by both (especially in areas far away from the sensor position) may cause the target object to fail to fuse (when the position deviation is greater than the fusion threshold), resulting in splitting, and the accuracy of the data cannot be guaranteed.

[0054] (3) The positioning accuracy of the target objects output by the fusion perception, especially those in static vehicles and areas far from the sensor or diagonally opposite to the sensor, cannot meet the data quality requirements of vehicle-road collaboration-related applications.

[0055] Based on the above technical problems, the present invention proposes a target positioning accuracy optimization method based on roadside fusion perception, which can optimize the target mass center position according to the differentiated performance of perception data and true value data, thereby improving the target positioning accuracy of roadside fusion perception; and for two roadside sensors, consider using the projection frame of 3D data under a bird's-eye view to realize the fusion of multi-source sensors to obtain high-quality fusion data; and also design a vehicle static recognition and retention strategy to solve the problem of low recognition of low-speed vehicles and unreliable recognition data of millimeter-wave radar.

[0056] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] Figure 1 is a flow chart of a target positioning accuracy optimization method based on roadside fusion perception provided by the present invention, such as Figure 1 As shown, the present invention provides a method for optimizing target positioning accuracy based on roadside fusion perception. The method comprises the following steps: Step 101: Acquire the latitude and longitude information of the centroid of the target object through a roadside millimeter wave radar, and acquire the pixel position information of the key points of the target object through a roadside camera.

[0058] Specifically, in the embodiment of the present application, the target object refers to an object such as a vehicle to be located on the road, especially a static vehicle at an intersection in an urban scene. The position information of the target object is detected by using millimeter wave radars and cameras deployed on the road.

[0059] During the deployment phase of roadside sensors, roadside system timing and sensor calibration are required.

[0060] In order to ensure that the roadside sensors and edge computing devices are under the same timing source, the roadside system needs to be synchronized, such as using the Network Time Protocol (NTP) timing method with an accuracy of within 10ms. The timing of the camera, millimeter wave radar, and edge computing unit comes from the Global Positioning System (GPS) signal in the roadside C-V2X Road Side Unit (RSU).

[0061] In order to complete the association between the coordinates of the road-related feature points in the roadside sensor and the coordinate points in the world coordinate system (or geographic coordinate system), the roadside sensor needs to be calibrated. In the embodiment of the present application, the calibration is performed by static marking, mainly including camera calibration and millimeter wave radar calibration.

[0062] For example, one camera calibration method is to evenly select feature points on the lanes in the image captured by the camera, obtain the feature point positions in the camera pixel coordinate system, place cones in the middle area of ​​the intersection, measure the GPS positions of all feature points at the same time, and record the GPS positions of the feature points and the corresponding image pixels, so as to complete the association between the feature point positions in the camera pixel coordinate system and the feature point positions in the world coordinate system.

[0063] For another example, a millimeter-wave radar calibration method is: take 10 points, including 1 radar origin and 9 target collection points, where the target collection points need to be evenly distributed, collect the position of the target collection points in the radar coordinate system and the GPS position in the world coordinate system, and keep a record of the corresponding point coordinates.

[0064] When locating a target, the millimeter-wave radar on the roadside will report the latitude, longitude and direction information of the target's center of mass, and the camera on the roadside will report the pixel position information of the target's key points.

[0065] The centroid of the target object is the center point of the cross section of the target object detected by the millimeter wave radar. The key point of the target object may include the center point and / or other feature points of the target object in the image.

[0066] Step 102: Determine the perception positioning data of the target object based on the longitude and latitude information of the centroid point of the target object and the pixel position information of the key points of the target object.

[0067] Specifically, after obtaining the latitude and longitude information of the centroid of the target object and the pixel position information of the key points of the target object, the data of multi-source sensors are fused through the radar fusion algorithm to obtain the perception positioning data of the target object.

[0068] For example, the longitude and latitude information of the centroid of the target object and the pixel position information of the key points of the target object are first converted to the same coordinate system. The projection frame of the 3D data from a bird's-eye view is used, and the intersection over union (IOU) fusion algorithm is combined to fuse the data of multi-source sensors in the same coordinate system to obtain the perception positioning data of the target object.

[0069] For another example, a fusion matching method based on Kalman filtering and Hungarian matching is used to fuse data from multiple source sensors in the same coordinate system to obtain the perception and positioning data of the target object.

[0070] Step 103: Determine the target positioning accuracy based on the perceived positioning data and the real positioning data of the target object; the real positioning data is obtained by detection using a detection device deployed on the target object.

[0071] Specifically, it is also necessary to obtain the real positioning data of the target object, which refers to the actual / real positioning data of the target object on the lane, and can be obtained by detection using a detection device deployed on the target object.

[0072] The acquired perceived positioning data of the target object is equivalent to the estimated value of the target object positioning, and the acquired real positioning data of the target object is equivalent to the true value of the target object positioning. The target positioning accuracy is determined by the perceived positioning data and the real positioning data, that is, the target positioning accuracy is determined according to the gap / error between the perceived positioning data and the real positioning data. The larger the error between the perceived positioning data and the real positioning data, the lower the target positioning accuracy, and vice versa.

[0073] Step 104: Optimize the target positioning accuracy by adjusting the plane position of the center of mass of the target object and / or the longitude and latitude information of the center of mass of the target object until the target positioning accuracy reaches a preset accuracy requirement.

[0074] Specifically, determine whether the target positioning accuracy meets the preset accuracy requirement. If not, adjust the planar position of the center of mass of the target object and / or the longitude and latitude information of the center of mass of the target object to optimize the target positioning accuracy until the target positioning accuracy meets the preset accuracy requirement, and stop adjusting; if it is determined that the target positioning accuracy meets the preset accuracy requirement, no adjustment is required.

[0075] The preset accuracy requirements are set based on experience and actual application needs.

[0076] For example, the Euclidean distance between the perceived positioning data and the actual positioning data is used as the target positioning accuracy. Specifically, the Euclidean distance between the coordinate point in the perceived positioning data and the corresponding coordinate point in the actual positioning data is calculated, and then the average value of the Euclidean distance of all coordinate point pairs is calculated. If the calculated mean Euclidean distance is greater than the preset accuracy threshold (such as 1 meter), it means that the target positioning accuracy does not meet the preset accuracy requirements, and it is necessary to adjust the plane position of the center of mass point of the target object and / or the longitude and latitude information of the center of mass point of the target object to optimize the target positioning accuracy until the target positioning accuracy meets the preset accuracy requirements, and stop adjusting. If the calculated mean Euclidean distance is less than or equal to the preset accuracy threshold (such as 1 meter), it means that the target positioning accuracy meets the preset accuracy requirements, and no further adjustments are needed to optimize the target positioning accuracy.

[0077] The method for optimizing target positioning accuracy based on roadside fusion perception provided in the embodiment of the present application obtains the longitude and latitude information of the centroid of the target object through the millimeter-wave radar on the roadside, and obtains the pixel position information of the key points of the target object through the camera on the roadside, and determines the perception positioning data of the target object based on the longitude and latitude information of the centroid of the target object and the pixel position information of the key points of the target object; determines the target positioning accuracy based on the perception positioning data and the real positioning data of the target object, wherein the real positioning data is obtained by detection using a detection device deployed on the target object; determines whether it is necessary to optimize the centroid position of the target object based on the target positioning accuracy and the preset accuracy requirements, and specifically, the centroid position of the target object can be optimized by adjusting the planar position of the centroid of the target object and / or the longitude and latitude information of the centroid of the target object, thereby improving the positioning accuracy of the target object by roadside fusion perception.

[0078] In some embodiments, determining the perception positioning data of the target object based on the latitude and longitude information of the centroid point of the target object and the pixel position information of the key points of the target object includes: The pixel position information of the key point of the target object is converted into the northeast celestial coordinate system to obtain the first projection of the key point of the target object under the bird's-eye view, and the longitude and latitude information of the centroid of the target object is converted into the northeast celestial coordinate system to obtain the second projection of the centroid of the target object under the bird's-eye view; Determine the overlap rate between the first projection and the second projection using an intersection-over-union (IOU) algorithm; Obtaining a confidence level corresponding to the first projection; the confidence level corresponding to the first projection refers to a confidence level of whether a target object exists in the first projection; When the overlap rate exceeds a preset overlap rate threshold, the perception positioning data of the target object is determined based on the confidence level corresponding to the first projection.

[0079] Specifically, the data acquired by the camera and the millimeter wave radar are converted to the same coordinate system. The data collected by each can be converted to the world coordinate system (geographic coordinate system) according to the calibration information.

[0080] The millimeter-wave radar on the roadside is used to identify the centroid of the target object, and the position information of the centroid of the target object in the radar coordinate system is obtained. The position information in the radar coordinate system is converted to the position information in the world coordinate system according to the millimeter-wave radar calibration information, that is, the longitude and latitude information of the centroid of the target object is obtained. Similarly, the pixel position information of the key points of the target object collected by the camera is converted to the world coordinate system according to the camera calibration information, and the longitude and latitude information of the key points of the target object is obtained, thus realizing the preliminary coordinate system of the target object detected by the roadside sensor.

[0081] Since the geographic coordinate system is a coordinate system established relative to the irregular sphere of the earth, it involves altitude, longitude and latitude, which is not conducive to algorithm model construction and calculation. Therefore, the geographic coordinate system will be further uniformly converted into a local coordinate system.

[0082] For example, the data collected by the camera and millimeter-wave radar are converted to the geographic coordinate system WGS84, and then the data is converted from WGS84 to the Earth-Centered Earth-Fixed (ECEF) coordinate system, and then converted to the East-North-Up (ENU) coordinate system, so as to unify the target object position data collected by the camera and millimeter-wave radar into the ENU local coordinate system to achieve spatial consistency.

[0083] Among them, the ECEF coordinate system is a Cartesian coordinate system with the origin at the center of mass of the earth, the Z axis pointing to the North Pole, the X axis passing through the Greenwich meridian at the equator, and the Y axis perpendicular to the XZ plane to form a right-handed coordinate system.

[0084] The specific steps to convert the longitude and latitude positions in the geographic coordinate system WGS84 to the coordinate positions in the ENU coordinate system are as follows: Step 1: The conversion equation of the target point from WGS84 to ECEF coordinate system is as follows: X = (N + h) × cos(lat) × cos(lon)

[0085] Y = (N + h) × cos(lat) × sin(lon)

[0086] Z = ((1 - e^2) × N + h) × sin(lat)

[0087] Where (X, Y, Z) are the coordinates of the target point in the ECEF coordinate system; lat, lon and h are the latitude, longitude and altitude of the target point respectively; N = a / sqrt(1 - e²× sin²(lat)), a is the major semi-axis of the WGS84 ellipsoid, e is the first eccentricity, and sqrt() is the square root function.

[0088] Step 2: From ECEF to ENU coordinate system: Complete the coordinate transformation to the local coordinate system centered on the device. The transformation formula of the target point from ECEF to ENU coordinate system is as follows: E= -sin(lon) × (X - X0) + cos(lon) × (Y - Y0)

[0089] N= -sin(lat) × cos(lon) × (X - X0) - sin(lat) × sin(lon) × (Y -Y0) + cos(lat) × (Z - Z0)

[0090] U=cos(lat) × cos(lon) × (X - X0) + cos(lat) × sin(lon) × (Y - Y0)+ sin(lat) × (Z - Z0)

[0091] Among them, (E, N, U) are the coordinates of the target point in the ENU coordinate system; (X0, Y0, Z0) are the observer coordinates.

[0092] In some embodiments, a region of interest (ROI) is also configured, that is, an optimal detection range for the camera and the millimeter wave radar to detect the target object is defined.

[0093] Figure 2 : is a schematic diagram of ROI configuration provided by the present invention, such as Figure 2 As described above, according to the sensor characteristics of the camera and the millimeter-wave radar, as well as the target detection accuracy of the two, the camera ROI area and the millimeter-wave radar ROI area are determined, that is, the optimal detection range of the camera and the millimeter-wave radar is defined, and it is ensured that they have a coverage intersection, while shielding invalid areas for target detection and areas that are prone to large deviations.

[0094] After the coordinate transformation step, the pixel position information of the key points of the target object is converted to the ENU coordinate system to obtain the 3D data corresponding to the pixel position information of the key points of the target object (i.e., camera structured data); and the longitude and latitude information of the center of mass of the target object is converted to obtain the 3D data corresponding to the longitude and latitude information of the center of mass of the target object (i.e., millimeter-wave radar structured data).

[0095] Then, a projection frame of the 3D data at a bird's-eye view is obtained, including a first projection of the 3D data corresponding to the camera at a bird's-eye view and a second projection of the 3D data corresponding to the millimeter-wave radar at a bird's-eye view.

[0096] For example, Figure 3 These are projection images of the target object from a bird's eye view under different cameras provided by the present invention, such as Figure 3 As shown in FIG. 1 , cameras at different positions (such as camera 1, camera 2, and camera 3) illuminate the target object from different perspectives. By performing spatiotemporal synchronization and data slicing on the data of different cameras, the various features of the same target object in the perspectives of different cameras in the same time and space are identified, and the projection of the key points of the target object in the bird's-eye view (such as Figure 3Projections 1, 2, and 3 in , that is, the first projection.

[0097] For another example, Figure 4 It is a projection diagram of different millimeter wave radar detection targets provided by the present invention from a bird's eye view, such as Figure 4 As shown, after obtaining the longitude and latitude information of the centroid of the target object reported by multiple millimeter wave radars (millimeter wave radar 1, millimeter wave radar 2 and millimeter wave radar 3), and converting the longitude and latitude information of the centroid into 3D data / structured data, the projections of different targets from a bird's eye view are estimated according to different target size models. For example, the target size model includes: the bus size model is 11530×2500×3040mm, the truck size model is 7000×2400×2500mm, the car size model is 5000×1800×1600mm, etc. The projection of the target object from a bird's eye view is estimated by using the structured data of the centroid of the target object obtained, combined with the target size model, and the projection of the target object from a bird's eye view is expanded into a polygonal projection based on the centroid of the target object from a bird's eye view (such as Figure 4 Projections 1, 2, and 3 in , that is, the second projection.

[0098] After obtaining the first projection and the second projection, the overlap ratio between the first projection and the second projection is determined using the intersection over union (IOU) algorithm.

[0099] Based on the overlap ratio of the first projection and the second projection, it is determined whether the target objects corresponding to the two projections are the same. For example, when the overlap ratio exceeds a preset overlap ratio threshold, it is determined that the two projections are the same target object; when the overlap ratio is less than or equal to the preset overlap ratio threshold, it is determined that the target objects corresponding to the first projection and the second projection are not the same.

[0100] Get the confidence of whether there is a target in the first projection. The visual model will output type confidence, that is, the confidence of whether there is a target. The confidence value is a floating point number between 0 and 1. The larger the value, the more reliable it is. At the same time, when calibrating the camera, the calibration algorithm will provide a position confidence through statistics. When the millimeter wave radar is fused with the camera, the type of the target output by the algorithm is often the type value of the camera. If multiple cameras are fused, the type value with higher confidence is taken.

[0101] When the overlap rate between the first projection and the second projection exceeds a preset overlap rate threshold, it indicates that the first projection and the second projection correspond to the same target object. At this time, the perception positioning data of the target object is further determined according to the confidence level corresponding to the first projection.

[0102] The target positioning accuracy optimization method based on roadside fusion perception provided in the embodiment of the present application uses the projection frame of 3D data from a bird's-eye view to perform IOU calculation to determine whether they are the same target. If two projections correspond to the same target, multi-source sensor fusion is further performed in combination with the confidence level corresponding to the first projection to obtain perception positioning data, thereby improving the data quality and reliability of the fused perception positioning data and ensuring the positioning accuracy of the target object.

[0103] In some embodiments, determining the perceived positioning data of the target object based on the confidence level corresponding to the first projection includes: When the distance between the camera corresponding to the first projection and the first projection is within a preset distance range and the confidence level corresponding to the first projection is greater than a preset confidence threshold, the first projection is used as the target object position information; and / or

[0104] When the distance between the camera corresponding to the first projection and the first projection is not within a preset distance range, or the confidence level corresponding to the first projection is less than or equal to a preset confidence threshold, taking the second projection as the target object position information; The target object position information and the target object heading and speed information are used as the perception positioning data of the target object.

[0105] Specifically, when the overlap rate between the first projection and the second projection exceeds a preset overlap rate threshold, the perception positioning data of the target object is determined based on the confidence corresponding to the first projection. The specific determination method is to determine the perception positioning data of the target object based on the confidence corresponding to the first projection, and / or the distance between the camera corresponding to the first projection and the first projection.

[0106] For example, if the distance between the camera corresponding to the first projection and the first projection is within a preset distance range (such as 20 to 80 meters), and the confidence corresponding to the first projection is greater than a preset confidence threshold, the position detected by the camera (ie, the first projection) is used as the target object position information.

[0107] For another example, the distance between the camera corresponding to the first projection and the first projection is not within the preset distance range, such as the distance between the camera corresponding to the first projection and the first projection is less than 20 meters or greater than 80 meters. Regardless of whether the confidence corresponding to the first projection exceeds the preset confidence threshold, the position detected by the millimeter wave radar (i.e., the second projection) is used as the target object position information.

[0108] For another example, if the confidence level corresponding to the first projection is less than or equal to a preset confidence threshold, regardless of whether the distance between the camera corresponding to the first projection and the first projection is within a preset distance range, the position detected by the millimeter-wave radar (ie, the second projection) is used as the target position information.

[0109] In some embodiments, the method further comprises: When the speed of the target object detected by the millimeter-wave radar is greater than a preset speed threshold, the heading and speed data of the target object detected by the millimeter-wave radar are used as the final heading and speed information of the target object; and / or

[0110] When the speed of the target object detected by the millimeter wave radar is less than or equal to the preset speed threshold, the heading and speed data of the target object detected by the camera are used as the final heading and speed information of the target object.

[0111] Specifically, the speed and heading of objects detected by the millimeter-wave radar are relatively accurate when they are in motion. Therefore, when the speed of the target object detected by the millimeter-wave radar is greater than the preset speed threshold, the heading and speed data of the target object will directly use the data detected by the millimeter-wave radar; when the speed of the target object detected by the millimeter-wave radar is less than or equal to the preset speed threshold, the camera perception data will be used as the heading and speed data of the target object, thereby obtaining the final target heading and speed information.

[0112] After determining the target object's position information and the target object's heading and speed information, the target object's position information and the target object's heading and speed information are used as the perception positioning data of the target object.

[0113] The target positioning accuracy optimization method based on roadside fusion perception provided in the embodiment of the present application determines the target object position information based on the distance between the camera corresponding to the first projection and the first projection and / or the confidence corresponding to the first projection, combined with a preset distance range and a preset confidence threshold, and determines the target object heading and speed information based on the speed of the target object detected by the millimeter wave radar, combined with a preset speed threshold, so that the determined target object position is more accurate and effective, the heading angle performance is better, and the data quality of the fused perception positioning data is improved.

[0114] In some embodiments, determining the target positioning accuracy based on the perceived positioning data and the real positioning data of the target object includes: Filtering matching positioning data from the sensed positioning data; the matching positioning data refers to positioning data that matches the real positioning data of the target object; Performing timestamp alignment on the selected matching positioning data and the real positioning data to obtain multiple groups of trajectory point pairs; each trajectory point pair includes a first trajectory point in the matching positioning data and a second trajectory point in the real positioning data, and the timestamps of the first trajectory point and the second trajectory point in each trajectory point are aligned; Calculating the Euclidean distance between the first trajectory point and the second trajectory point in each trajectory point pair to obtain the Euclidean distance of the trajectory point pair; The average value of the Euclidean distances of all trajectory point pairs is taken as the target positioning accuracy.

[0115] Specifically, before determining the target positioning accuracy, the real positioning data of the target is collected by a sensor deployed on the target or a real-time differential positioning device (Real-time kinematic, RTK) deployed on the target.

[0116] For example, high-precision vehicle-mounted sensors and equipment are deployed on the target object, and the target object's own positioning data is output at a frequency of 10 Hz. At the same time, all lanes inside the intersection are traversed, and the vehicle stops for 10 seconds at a distance of 50 meters from the center point of the intersection to collect all true value point data (that is, the real positioning data of the target object).

[0117] For another example, place the high-precision RTK at the center of the roof, set the sampling frequency to 10 Hz, traverse all lanes inside the intersection, stop for 10 seconds at a distance of 50 meters from the center of the intersection, and collect data from all true value points.

[0118] After obtaining the real positioning data and perceived positioning data of the target object, the two are compared and matched in different distance segments.

[0119] For example, Figure 5 is the true value vehicle RTK trajectory map provided by the present invention, such as Figure 5 As shown, all real positioning data are imported into the software, and the software interface displays the true value vehicle trajectory. Figure 6 is the target object trajectory diagram of the radar-vision fusion algorithm provided by the present invention, such as Figure 6 As shown in the figure, after the perception positioning data is imported into the software, the true value vehicle trajectory is matched with the target trajectory of the radar-vision fusion algorithm. The specific matching method is: for the trajectory of a certain distance segment, when the average positioning accuracy distance deviation between the perception positioning data and the real positioning data is set to be ≤1m and the timestamp is ≤40ms, and the number of trajectory points that meet the requirements is ≥1000 frames, the perception positioning data is determined to be the true value vehicle trajectory, and all the perception positioning data determined to be the true value vehicle trajectory are used as matching positioning data.

[0120] The selected matching positioning data and the real positioning data are timestamped. The trajectory points with time fluctuations within 40ms can be timestamped. Each pair of timestamp-aligned matching positioning data and real positioning data is a set of trajectory point pairs. The Euclidean distance between each trajectory point pair is calculated, and the average value of the Euclidean distance of all trajectory point pairs in a certain distance segment is used as the target positioning accuracy of the distance segment.

[0121] For example, the formula for finding the Euclidean distance between two points is: Si²=Xi²+Yi²

[0122] Wherein, Si represents the Euclidean distance between two points of the i-th group of trajectory point pairs, and i is a positive integer; Xi represents the lateral position error between the two points; Yi represents the longitudinal position error between the two points.

[0123] The target positioning accuracy of a certain range can be expressed as: S=(S1+S2+…+Si+…+Sn) / n

[0124] Where S represents the target positioning accuracy of a range segment (the range from the first group of trajectory point pairs to the nth group of trajectory point pairs); n represents the number of trajectory point pairs in the range segment.

[0125] The target positioning accuracy optimization method based on roadside fusion perception provided in the embodiment of the present application screens out the true vehicle trajectory points from the multiple target object trajectory point data output by the radar-vision fusion algorithm, compares the target positioning accuracy, calculates the Euclidean distance between static points based on the trajectory points at the same or similar times, and calculates the deviation values ​​of different distance segments by segmented mean, so as to accurately obtain the target positioning accuracy values ​​of the perception positioning data of different distance segments.

[0126] In some embodiments, optimizing the target positioning accuracy by adjusting the plane position of the center of mass of the target object and / or the longitude and latitude information of the center of mass of the target object includes: Compensating the horizontal coordinate of the center of mass of the target object based on the real positioning data, and compensating the vertical coordinate of the center of mass of the target object based on the size of the target object; and / or

[0127] When the millimeter wave radar detects that the target object decelerates and the speed is lower than a preset first threshold, deleting the currently acquired longitude and latitude position of the centroid of the target object; and / or

[0128] The sensed positioning data is used as the longitude and latitude position of the center of mass of the current target object; the sensed positioning data is not less than a second threshold and not greater than 1.5 times the second threshold; the second threshold is 2 / 3 times the longitude and latitude position of the target object detected by the millimeter wave radar; and / or

[0129] When the millimeter-wave radar detects that the target object is accelerating and the speed exceeds a preset third threshold, the millimeter-wave radar is restarted to detect the longitude and latitude positions of the center of mass of the target object.

[0130] Specifically, the embodiment of the present application performs a millimeter-wave radar centroid compensation strategy and a millimeter-wave radar static holding strategy according to the deviation value to optimize the target positioning accuracy, so that the output perception positioning data meets the segmented positioning accuracy requirements.

[0131] Usually, the centroid of the detected target in the millimeter-wave radar structured data is the center point of the identified target cross section. During the test, it was found that there was a deviation from the true value data (the actual position of the centroid). Therefore, the following compensation strategy is used for the centroid position of the target: The centroid position compensation method compares the true value vehicle trajectory identified by the millimeter wave radar with the true value vehicle RTK trajectory to make the trajectory point output by the millimeter wave radar close to the true value vehicle RTK trajectory point. The compensation value may be added or subtracted. Combined with the size model of various targets such as vehicles, the calculation method of the compensation value of different types of vehicles is as follows: a=(a1-a11)+(a2-a22)+...+(an-ann) / n,

[0132] b=L / 2

[0133] Among them, a is the horizontal coordinate compensation value of the target material center; an is the horizontal coordinate of the target material center identified by the millimeter wave radar at the nth position point (n is a positive integer); ann is the horizontal coordinate of the target material center at the nth position point in the real positioning data; b is the vertical coordinate compensation value of the target material center; L is the length value of the vehicle in the target object size model.

[0134] Since the detection features of millimeter-wave radar for low-speed or stationary targets are not obvious, they cannot be effectively detected. Therefore, a static holding strategy for millimeter-wave radar structured data is required to maintain continuous output so that the deviation between its trajectory points and the true value trajectory points is within the allowable range. The millimeter-wave radar static holding strategy includes at least one or more of the following: (1) When the target object is decelerating, the speed of the target object detected by the millimeter wave radar decreases and is lower than a preset first threshold value, the radar output of the target object detection latitude and longitude position is abandoned, that is, the latitude and longitude position of the center of mass of the target object currently obtained is deleted.

[0135] (2) The longitude and latitude positions of the target object at this time are assigned as the values ​​generated by the fusion perception algorithm strategy, that is, the acquired perception positioning data is used as the longitude and latitude positions of the center of mass of the current target object, and the perception positioning data is not less than the second threshold and not greater than 1.5 times the second threshold (that is, it is maintained between the second threshold and 1.5 times the second threshold), where the second threshold is 2 / 3 times the longitude and latitude positions of the target object detected by the millimeter wave radar.

[0136] (3) When the target object accelerates or re-moves, the speed of the target object detected by the millimeter-wave radar increases and exceeds a preset third threshold value, the millimeter-wave radar is re-enabled to detect the longitude and latitude positions of the center of mass of the target object.

[0137] After the center of mass compensation and / or static holding strategy are implemented for the millimeter-wave radar, it is necessary to iterate a round of radar-vision fusion and target positioning accuracy determination process, that is, to re-fuse the IOU multi-source sensor of the projection frame of the camera and millimeter-wave radar 3D data from a bird's-eye view, re-collect the true value data, filter out the true value vehicle trajectory points based on the true value vehicle structured data identified by the fusion perception system, re-compare the positioning accuracy, and optimize the vehicle positioning accuracy until the calculated target positioning accuracy reaches the preset preset accuracy requirement, and then stop the iteration.

[0138] The target positioning accuracy optimization method based on roadside fusion perception provided in the embodiment of the present application optimizes the center of mass position of the identified target object through the millimeter-wave radar center of mass compensation strategy and / or the millimeter-wave radar static holding strategy, thereby solving the problem of low recognition of low-speed vehicles and unreliable recognition data of millimeter-wave radar, thereby improving the accuracy of perception positioning data, achieving an improvement in the positioning accuracy of target objects output by the perception system, and improving the recall rate of overall vehicle recognition of the fusion perception system and the stability and reliability of static vehicle output data, thereby supporting the demand for high-quality data for upper-layer applications of vehicle-road collaboration.

[0139] Figure 7 is a schematic diagram of the structure of the target positioning accuracy optimization device based on roadside fusion perception provided by the present invention, such as Figure 7 As shown, the present invention provides a device for optimizing target positioning accuracy based on roadside fusion perception, including an acquisition module 701, a first determination module 702, a second determination module 703 and an optimization module 704.

[0140] The acquisition module 701 is used to acquire the longitude and latitude information of the centroid of the target object through the millimeter wave radar on the road side, and acquire the pixel position information of the key points of the target object through the camera on the road side.

[0141] The first determination module 702 is used to determine the perception positioning data of the target object based on the longitude and latitude information of the centroid point of the target object and the pixel position information of the key points of the target object.

[0142] The second determination module 703 is used to determine the target positioning accuracy based on the perceived positioning data and the real positioning data of the target object; the real positioning data is detected by using a detection device deployed on the target object.

[0143] The optimization module 704 is used to optimize the target positioning accuracy by adjusting the plane position of the center of mass of the target object and / or the longitude and latitude information of the center of mass of the target object until the target positioning accuracy reaches a preset accuracy requirement.

[0144] In some embodiments, the optimization module includes: a compensation unit, configured to compensate the horizontal coordinate of the center of mass of the target object based on the real positioning data, and to compensate the vertical coordinate of the center of mass of the target object based on the size of the target object; and / or

[0145] A first judgment unit is used to delete the currently acquired longitude and latitude position of the centroid of the target object when the millimeter wave radar detects that the target object is decelerating and the speed is lower than a preset first threshold; and / or

[0146] A first determination unit is configured to use the sensed positioning data as the longitude and latitude position of the centroid of the current target object; the sensed positioning data is not less than a second threshold and not greater than 1.5 times the second threshold; the second threshold is 2 / 3 times the longitude and latitude position of the target object detected by the millimeter wave radar; and / or

[0147] The second judgment unit is used to restart the millimeter wave radar and detect the longitude and latitude positions of the center of mass of the target object when the millimeter wave radar detects that the target object is accelerating and the speed exceeds a preset third threshold.

[0148] In some embodiments, the first determining module includes: A first acquisition unit is used to convert the pixel position information of the key point of the target object into the northeast celestial coordinate system to obtain a first projection of the key point of the target object from a bird's-eye view, and convert the longitude and latitude information of the centroid point of the target object into the northeast celestial coordinate system to obtain a second projection of the centroid point of the target object from a bird's-eye view; A second determining unit, configured to determine an overlap ratio between the first projection and the second projection by using an intersection-over-union (IOU) algorithm; a second acquisition unit, configured to acquire a confidence level corresponding to the first projection; the confidence level corresponding to the first projection refers to a confidence level of whether a target object exists in the first projection; The third determination unit is used to determine the perception positioning data of the target object based on the confidence level corresponding to the first projection when the overlap rate exceeds a preset overlap rate threshold.

[0149] In some embodiments, the third determining unit includes: a first determining subunit, configured to use the first projection as target object position information when the distance between the camera corresponding to the first projection and the first projection is within a preset distance range and the confidence level corresponding to the first projection is greater than a preset confidence threshold; and / or

[0150] A second determination subunit is configured to use the second projection as the target object position information when the distance between the camera corresponding to the first projection and the first projection is not within a preset distance range or the confidence level corresponding to the first projection is less than or equal to a preset confidence threshold; The third determination subunit is used to use the target object position information and the target object heading and speed information as the perception positioning data of the target object.

[0151] In some embodiments, it also includes: a fourth determination unit, configured to use the heading and speed data of the target object detected by the millimeter-wave radar as final heading and speed information of the target object when the speed of the target object detected by the millimeter-wave radar is greater than a preset speed threshold; and / or

[0152] The fifth determination unit is used to use the heading and speed data of the target object detected by the camera as the final heading and speed information of the target object when the speed of the target object detected by the millimeter wave radar is less than or equal to the preset speed threshold.

[0153] In some embodiments, the second determining module includes: a sixth determining unit, configured to filter out matching positioning data from the sensed positioning data; the matching positioning data refers to positioning data that matches the real positioning data of the target object; A third acquisition unit is used to align the timestamps of the screened matching positioning data with the real positioning data to obtain multiple groups of trajectory point pairs; each trajectory point pair includes a first trajectory point in the matching positioning data and a second trajectory point in the real positioning data, and the timestamps of the first trajectory point and the second trajectory point in each trajectory point are aligned; a fourth acquisition unit, configured to calculate the Euclidean distance between the first trajectory point and the second trajectory point in each trajectory point pair to obtain the Euclidean distance of the trajectory point pair; The seventh determination unit is used to take the average value of the Euclidean distances of all trajectory point pairs as the target positioning accuracy.

[0154] In some embodiments, it also includes: The acquisition module is used to acquire the real positioning data of the target object through a sensor deployed on the target object or a real-time differential positioning device RTK deployed on the target object.

[0155] Specifically, the target positioning accuracy optimization device based on roadside fusion perception provided by the present invention can implement all the method steps implemented by the above-mentioned target positioning accuracy optimization method embodiment based on roadside fusion perception, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as the method embodiment will not be described in detail here.

[0156] It should be noted that the division of units / modules in the above-mentioned embodiments of the present invention is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, the functional units in the various embodiments of the present application may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware or in the form of software functional units.

[0157] Figure 8 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 8 As shown, the electronic device may include: a processor 801, a communications interface 802, a memory 803 and a communication bus 804, wherein the processor 801, the communications interface 802 and the memory 803 communicate with each other through the communication bus 804. The processor 801 may call the logic instructions in the memory 803 to execute the target positioning accuracy optimization method based on roadside fusion perception, and the method includes: The longitude and latitude information of the centroid of the target object is obtained through the millimeter wave radar on the road side, and the pixel position information of the key points of the target object is obtained through the camera on the road side; Determine the perception positioning data of the target object based on the latitude and longitude information of the centroid point of the target object and the pixel position information of the key points of the target object; Determine the target positioning accuracy based on the perceived positioning data and the real positioning data of the target object; the real positioning data is detected by using a detection device deployed on the target object; The target positioning accuracy is optimized by adjusting the plane position of the center of mass of the target object and / or the longitude and latitude information of the center of mass of the target object until the target positioning accuracy reaches a preset accuracy requirement.

[0158] Specifically, the processor 801 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or a complex programmable logic device (CPLD), and the processor may also adopt a multi-core architecture.

[0159] The logic instructions in the memory 803 can be implemented in the form of software functional units and can be stored in a processor-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0160] In some embodiments, a computer program product is further provided, the computer program product comprising a computer program, the computer program may be stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer can execute the target positioning accuracy optimization method based on roadside fusion perception provided by the above-mentioned method embodiments, the method comprising: The longitude and latitude information of the centroid of the target object is obtained through the millimeter wave radar on the road side, and the pixel position information of the key points of the target object is obtained through the camera on the road side; Determine the perception positioning data of the target object based on the latitude and longitude information of the centroid point of the target object and the pixel position information of the key points of the target object; Determine the target positioning accuracy based on the perceived positioning data and the real positioning data of the target object; the real positioning data is detected by using a detection device deployed on the target object; The target positioning accuracy is optimized by adjusting the plane position of the center of mass of the target object and / or the longitude and latitude information of the center of mass of the target object until the target positioning accuracy reaches a preset accuracy requirement.

[0161] Specifically, the above-mentioned computer program product provided in the embodiment of the present application can implement all the method steps implemented by the above-mentioned method embodiments, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as the method embodiment will not be described in detail here.

[0162] In some embodiments, a computer-readable storage medium is further provided, wherein the computer-readable storage medium stores a computer program, wherein the computer program is used to enable a computer to execute the target positioning accuracy optimization method based on roadside fusion perception provided by the above-mentioned method embodiments, and the method includes: The longitude and latitude information of the centroid of the target object is obtained through the millimeter wave radar on the road side, and the pixel position information of the key points of the target object is obtained through the camera on the road side; Determine the perception positioning data of the target object based on the latitude and longitude information of the centroid point of the target object and the pixel position information of the key points of the target object; Determine the target positioning accuracy based on the perceived positioning data and the real positioning data of the target object; the real positioning data is detected by using a detection device deployed on the target object; The target positioning accuracy is optimized by adjusting the plane position of the center of mass of the target object and / or the longitude and latitude information of the center of mass of the target object until the target positioning accuracy reaches a preset accuracy requirement.

[0163] Specifically, the above-mentioned computer-readable storage medium provided by the present invention can implement all the method steps implemented by the above-mentioned method embodiments, and can achieve the same technical effects. The parts and beneficial effects that are the same as the method embodiments in this embodiment will not be described in detail here.

[0164] It should be noted that the computer-readable storage medium can be any available medium or data storage device that can be accessed by the processor, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)), etc.

[0165] It should also be noted that the terms "first", "second", etc. in the present invention are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present invention can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first" and "second" are usually of the same type, and the number of objects is not limited. For example, the first object can be one or more.

[0166] In the present invention, the term "and / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0167] In the present invention, the term "plurality" refers to two or more than two, and other quantifiers are similar to them.

[0168] In the present invention, "determine B based on A" means that the factor A should be considered when determining B. It is not limited to "B can be determined based on A alone", but should also include: "determine B based on A and C", "determine B based on A, C and E", "determine C based on A, and further determine B based on C", etc. In addition, it can also include taking A as a condition for determining B, for example, "when A meets the first condition, use the first method to determine B"; for another example, "when A meets the second condition, determine B", etc.; for another example, "when A meets the third condition, determine B based on the first parameter", etc. Of course, it can also be a condition that takes A as a factor for determining B, for example, "when A meets the first condition, use the first method to determine C, and further determine B based on C", etc.

[0169] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program codes.

[0170] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer executable instructions. These computer executable instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0171] These processor executable instructions may also be stored in a processor readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the processor readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0172] These processor-executable instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0173] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for optimizing target positioning accuracy based on roadside fusion perception, characterized in that: include: The longitude and latitude information of the centroid of the target object is obtained through the millimeter wave radar on the road side, and the pixel position information of the key points of the target object is obtained through the camera on the road side; Determine the perception positioning data of the target object based on the latitude and longitude information of the centroid point of the target object and the pixel position information of the key points of the target object; Determine the target positioning accuracy based on the perceived positioning data and the real positioning data of the target object; the real positioning data is detected by using a detection device deployed on the target object; The target positioning accuracy is optimized by adjusting the plane position of the center of mass of the target object and / or the longitude and latitude information of the center of mass of the target object until the target positioning accuracy reaches a preset accuracy requirement.

2. The target positioning accuracy optimization method based on roadside fusion perception according to claim 1 is characterized in that: Optimizing the target positioning accuracy by adjusting the plane position of the center of mass of the target object and / or the longitude and latitude information of the center of mass of the target object includes: Compensating the horizontal coordinate of the center of mass of the target object based on the real positioning data, and compensating the vertical coordinate of the center of mass of the target object based on the size of the target object; and / or When the millimeter wave radar detects that the target object decelerates and the speed is lower than a preset first threshold, deleting the currently acquired longitude and latitude position of the centroid of the target object; and / or The sensed positioning data is used as the longitude and latitude position of the center of mass of the current target object; the sensed positioning data is not less than a second threshold and not greater than 1.5 times the second threshold; the second threshold is 2 / 3 times the longitude and latitude position of the target object detected by the millimeter wave radar; and / or When the millimeter-wave radar detects that the target object is accelerating and the speed exceeds a preset third threshold, the millimeter-wave radar is restarted to detect the longitude and latitude positions of the center of mass of the target object.

3. The method for optimizing target positioning accuracy based on roadside fusion perception according to claim 1 is characterized in that: The determining the perception positioning data of the target object based on the latitude and longitude information of the centroid point of the target object and the pixel position information of the key points of the target object includes: The pixel position information of the key point of the target object is converted into the northeast celestial coordinate system to obtain the first projection of the key point of the target object under the bird's-eye view, and the longitude and latitude information of the centroid of the target object is converted into the northeast celestial coordinate system to obtain the second projection of the centroid of the target object under the bird's-eye view; Determine the overlap rate between the first projection and the second projection using an intersection-over-union (IOU) algorithm; Obtaining a confidence level corresponding to the first projection; the confidence level corresponding to the first projection refers to a confidence level of whether a target object exists in the first projection; When the overlap rate exceeds a preset overlap rate threshold, the perception positioning data of the target object is determined based on the confidence level corresponding to the first projection.

4. The method for optimizing target positioning accuracy based on roadside fusion perception according to claim 3 is characterized in that: The determining the perception positioning data of the target object based on the confidence level corresponding to the first projection includes: When the distance between the camera corresponding to the first projection and the first projection is within a preset distance range and the confidence level corresponding to the first projection is greater than a preset confidence threshold, the first projection is used as the target object position information; and / or When the distance between the camera corresponding to the first projection and the first projection is not within a preset distance range, or the confidence level corresponding to the first projection is less than or equal to a preset confidence threshold, taking the second projection as the target object position information; The target object position information and the target object heading and speed information are used as the perception positioning data of the target object.

5. The method for optimizing target positioning accuracy based on roadside fusion perception according to claim 3 or 4, characterized in that: The method further comprises: When the speed of the target object detected by the millimeter-wave radar is greater than a preset speed threshold, the heading and speed data of the target object detected by the millimeter-wave radar are used as the final heading and speed information of the target object; and / or When the speed of the target object detected by the millimeter wave radar is less than or equal to the preset speed threshold, the heading and speed data of the target object detected by the camera are used as the final heading and speed information of the target object.

6. The method for optimizing target positioning accuracy based on roadside fusion perception according to claim 1 is characterized in that: The determining the target positioning accuracy based on the perceived positioning data and the real positioning data of the target object comprises: Filtering matching positioning data from the sensed positioning data; the matching positioning data refers to positioning data that matches the real positioning data of the target object; Performing timestamp alignment on the selected matching positioning data and the real positioning data to obtain multiple groups of trajectory point pairs; each trajectory point pair includes a first trajectory point in the matching positioning data and a second trajectory point in the real positioning data, and the timestamps of the first trajectory point and the second trajectory point in each trajectory point are aligned; Calculating the Euclidean distance between the first trajectory point and the second trajectory point in each trajectory point pair to obtain the Euclidean distance of the trajectory point pair; The average value of the Euclidean distances of all trajectory point pairs is taken as the target positioning accuracy.

7. The method for optimizing target positioning accuracy based on roadside fusion perception according to claim 1 is characterized in that: The method further comprises: The real positioning data of the target object is collected by a sensor deployed on the target object or a real-time differential positioning device RTK deployed on the target object.

8. A target positioning accuracy optimization device based on roadside fusion perception, characterized in that: include: An acquisition module is used to acquire the latitude and longitude information of the centroid of the target object through a roadside millimeter wave radar, and acquire the pixel position information of the key points of the target object through a roadside camera; A first determination module, configured to determine the perception positioning data of the target object based on the latitude and longitude information of the centroid point of the target object and the pixel position information of the key points of the target object; A second determination module is used to determine the target positioning accuracy based on the perceived positioning data and the real positioning data of the target object; The real positioning data is obtained by detection using a detection device deployed on the target object; The optimization module is used to optimize the target positioning accuracy by adjusting the plane position of the center of mass of the target object and / or the longitude and latitude information of the center of mass of the target object until the target positioning accuracy reaches a preset accuracy requirement.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, it implements the target positioning accuracy optimization method based on roadside fusion perception as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for optimizing target positioning accuracy based on roadside fusion perception as described in any one of claims 1 to 7 is implemented.

Images