Vehicle-road-cloud collaborative positioning method and device based on road real-scene three-dimensional data

Abstract

The present invention relates to the field of vehicle positioning and navigation technology in the field of autonomous driving, and in particular to a vehicle-road-cloud collaborative positioning method and device based on real-scene three-dimensional data of roads, wherein the method comprises: obtaining real-scene three-dimensional data of the driving road where the target vehicle is located, and obtaining the three-dimensional coordinates of the feature points of the image taken by the roadside camera based on the real-scene three-dimensional data of the driving road constructed in advance; calibrating the internal and external parameters of the roadside camera on the driving road; determining the positioning information of the target vehicle using the calibrated roadside camera, and sending it to the target vehicle; obtaining the positioning information of the target vehicle based on the INS system of the target vehicle; correcting the positioning information of the INS based on the positioning information of the roadside camera, and then locating the position of the target vehicle. Thus, the problem of limited accuracy of vehicle positioning and perception in the related technology, especially when in special and complex environments such as urban canyons and tunnels, the low accuracy and reliability of the signal source obtained by a single vehicle is solved.

Description

Technical Field

[0001] The present invention relates to the field of vehicle positioning and navigation technology in the field of autonomous driving, and in particular to a vehicle-road-cloud collaborative positioning method and device based on real-scene three-dimensional data of a road. Background Art

[0002] With the continuous improvement and implementation of technologies such as Beidou and 5G communications, driverless cars are gradually entering people's lives, which also puts higher requirements on the positioning and navigation accuracy of vehicles in complex urban environments.

[0003] In related technologies, multi-source sensors, such as cameras, GNSS (Global Navigation Satellite System), INS (Inertial Navigation System), etc., are usually used for combined navigation. In related multi-source sensor research, it mainly focuses on single-vehicle sensor fusion. The single-vehicle combined navigation solution can comprehensively utilize the complementary characteristics of multiple sensors, thereby improving the reliability of vehicle positioning and navigation.

[0004] However, although the single-vehicle combined navigation solution can comprehensively utilize the complementary characteristics of multiple sensors to improve the reliability of vehicle positioning and navigation, it is limited by the size and cost of the vehicle. The positioning and detection accuracy of the on-board sensors are easily limited, and the detection field of view and distance are small. Especially in special and complex environments such as urban canyons and tunnels, it is more susceptible to interference. The signal source obtained by a single vehicle is not ideal in terms of accuracy and reliability. Summary of the invention

[0005] The present invention provides a vehicle-road-cloud collaborative positioning method and device based on real-scene three-dimensional data of roads, so as to solve the problems in related technologies that the accuracy of vehicle positioning and detection is limited due to the limitations of vehicle size, cost and the perception range of on-board sensors, especially in special complex environments such as urban canyons and tunnels, which are more susceptible to interference, and the signal source obtained by a single vehicle is not ideal in terms of accuracy and reliability.

[0006] The first aspect of the present invention provides a vehicle-road-cloud collaborative positioning method based on real-scene three-dimensional data of roads, comprising the following steps: acquiring real-scene three-dimensional data of the driving road on which a target vehicle is located, and obtaining three-dimensional coordinates of image feature points based on the real-scene three-dimensional data of the driving road constructed in advance; calibrating the internal and external parameters of a roadside camera in the driving road according to the three-dimensional coordinates of the image feature points to obtain a calibrated roadside camera; determining the three-dimensional position information and relative position information of the target vehicle using the calibrated roadside camera, and sending the three-dimensional position information and the relative position information to the target vehicle; obtaining a three-dimensional position information based on the INS of the target vehicle. Obtain the vehicle three-dimensional position information and vehicle relative position information of the target vehicle; complete the spatiotemporal synchronization of cross-domain sensors, and verify the relative position information calculated by the roadside unit based on the vehicle relative position information calculated by the INS of the target vehicle using the synchronized sensor after spatiotemporal synchronization, obtain the timestamp corresponding to the relative position information calculated by the roadside unit whose relative position error range satisfies the preset error range, and correct the vehicle three-dimensional position information calculated by the INS of the target vehicle based on the three-dimensional position information calculated by the roadside camera corresponding to the timestamp, obtain a corrected vehicle three-dimensional positioning result, and locate the position of the target vehicle based on the vehicle three-dimensional positioning result.

[0007] Optionally, in one embodiment of the present invention, calibrating the internal and external parameters of the roadside camera in the driving road according to the three-dimensional coordinates of the image feature points to obtain a calibrated roadside camera includes: acquiring first image data of the driving road using the roadside camera, extracting feature points in the first image data, and obtaining the three-dimensional coordinates of the feature points in the driving road based on the real-scene three-dimensional data; calibrating the internal and external parameters of the roadside camera according to the three-dimensional coordinates to obtain a calibrated roadside camera.

[0008] Optionally, in one embodiment of the present invention, the use of the calibrated roadside camera to determine the three-dimensional position information and relative position information of the target vehicle includes: using the calibrated roadside camera to acquire second image data of the driving road; preprocessing the second image data to obtain third image data that meets preset conditions; based on the third image data, identifying the target vehicle in the third image data to obtain an identification frame of the target vehicle; and calculating the three-dimensional position information and relative position information of the target vehicle based on parameter information of the identification frame, internal and external parameters of the roadside camera and the third image data.

[0009] Optionally, in one embodiment of the present invention, the spatiotemporal synchronization of cross-domain sensors is completed, and the relative position information calculated by the roadside unit is verified using the vehicle relative position information calculated by the INS of the target vehicle according to the spatiotemporal synchronized synchronized sensors, so as to obtain a timestamp corresponding to the relative position information calculated by the roadside unit whose relative position error range satisfies a preset error range, including: performing time synchronization of multi-domain sensors based on the target vehicle and all sensors in the roadside unit to obtain a time-synchronized synchronized sensor; based on the synchronized sensor, using an IMU (Inertial Measurement Unit) to assist in completing the verification of the positioning accuracy of the roadside unit, that is, using the vehicle relative position information calculated by the INS of the target vehicle to verify the relative position information calculated by the roadside unit, so as to obtain a timestamp corresponding to the relative position information calculated by the roadside unit whose relative position error range satisfies the preset error range.

[0010] Optionally, in one embodiment of the present invention, correcting the vehicle three-dimensional position information and the vehicle relative position information based on the three-dimensional position information and the relative position information includes: determining whether a GNSS signal satisfies a preset signal condition; if the GNSS signal does not satisfy the preset signal condition, correcting the vehicle three-dimensional position information based on the three-dimensional position information and the relative position information, otherwise correcting the vehicle three-dimensional position information based on the GNSS signal.

[0011] The second aspect of the present invention provides a vehicle-road-cloud collaborative positioning device based on real-scene three-dimensional data of a road, comprising: a cloud communication module, which is used by a roadside unit to obtain real-scene three-dimensional data of a driving road on which a target vehicle is located, and obtain three-dimensional coordinates of feature points based on the pre-constructed real-scene three-dimensional data of the driving road; an automatic calibration module, which is used to calibrate the internal and external parameters of a roadside camera in the driving road according to the three-dimensional coordinates to obtain a calibrated roadside camera; a roadside calculation module, which is used to determine the three-dimensional position information and relative position information of the target vehicle using the calibrated roadside camera; a vehicle-road communication module, which is used to send the three-dimensional position information and the relative position information to the target vehicle; an on-board inertial navigation module, which is used to obtain the continuous vehicle three-dimensional position information and vehicle relative position information of the target vehicle based on the INS of the target vehicle; and an on-board GNSS module, which is used to obtain GNSS signal observation values ​​and GNSS PPS (Pulse Per Second). Second, the number of pulses per second) pulse signal; a vehicle-road collaborative positioning module, which is used to perform combined navigation based on the GNSS and the target vehicle INS when the GNSS signal meets the preset signal conditions, and complete the spatiotemporal synchronization of cross-domain sensors when the GNSS signal does not meet the preset signal conditions, and use the vehicle relative position information calculated by the INS of the target vehicle to verify the relative position information calculated by the roadside unit, and obtain the timestamp corresponding to the relative position information calculated by the roadside unit whose relative position error range meets the preset error range, and correct the vehicle three-dimensional position information calculated by the INS of the target vehicle based on the three-dimensional position information calculated by the roadside camera corresponding to the timestamp to obtain a corrected vehicle three-dimensional positioning result.

[0012] Optionally, in one embodiment of the present invention, the automatic calibration module includes: a first acquisition unit, used to use the roadside camera to acquire first image data of the driving road, extract feature points in the first image data, and obtain the three-dimensional coordinates of the feature points in the driving road based on the real-scene three-dimensional data; an automatic calibration unit, used to calibrate the internal and external parameters of the roadside camera according to the three-dimensional coordinates to obtain a calibrated roadside camera.

[0013] Optionally, in one embodiment of the present invention, the roadside calculation module includes: a second acquisition unit, used to acquire second image data of the driving road using the calibrated roadside camera; a preprocessing unit, used to preprocess the second image data to obtain third image data that meets preset conditions; an identification unit, used to identify the target vehicle in the third image data based on the third image data, and obtain an identification frame of the target vehicle; a roadside calculation unit, used to calculate the three-dimensional position information and relative position information of the target vehicle based on parameter information of the identification frame, internal and external parameters of the roadside camera and the third image data.

[0014] Optionally, in one embodiment of the present invention, the collaborative positioning module includes: a time synchronization unit, used to perform time synchronization of multi-domain sensors based on the target vehicle and all sensors in the roadside unit, and obtain synchronized sensors after time synchronization; a verification unit, used to complete the verification of the positioning accuracy of the roadside unit based on the synchronized sensor and with the assistance of IMU, that is, using the vehicle relative position information calculated by the INS of the target vehicle to verify the relative position information calculated by the roadside unit, and obtain the timestamp corresponding to the relative position information calculated by the roadside unit whose relative position error range satisfies the preset error range.

[0015] Optionally, in one embodiment of the present invention, the collaborative positioning module includes: a judgment unit, used to judge whether the GNSS signal meets the preset signal condition; a positioning unit, used to correct the vehicle's three-dimensional position information based on the three-dimensional position information and the relative position information when the GNSS signal does not meet the preset signal condition, otherwise correct the vehicle's three-dimensional position information based on the GNSS signal.

[0016] The third aspect of the present invention provides a server, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the vehicle-road-cloud collaborative positioning method based on real-scene three-dimensional data of a road as described in the above embodiment.

[0017] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned vehicle-road-cloud collaborative positioning method based on real-scene three-dimensional data of the road.

[0018] A fifth aspect of the present invention provides a computer program product, including a computer program, which, when executed, implements the above-mentioned vehicle-road-cloud collaborative positioning method based on real-scene three-dimensional data of the road.

[0019] The embodiment of the present invention can calibrate the internal and external parameters of the roadside camera based on the three-dimensional coordinates of the image feature points obtained from the pre-built real-scene three-dimensional model and the real-scene three-dimensional data, and then obtain a calibrated roadside camera, and use the calibrated roadside camera to correct the vehicle three-dimensional positioning result of the INS of the target vehicle to obtain the position of the target vehicle. The roadside camera can be used for vehicle-road collaborative positioning perception, which has low cost, high operating efficiency, good real-time performance, and automated data collection. It can significantly improve the availability, reliability and accuracy of the vehicle positioning function in long tunnels and other environments without GNSS signals for a long time, and improve the application scope and safety of the autonomous driving industry. Therefore, it solves the problems in the related technology that the accuracy of vehicle positioning and perception is limited due to the limitations of vehicle size, cost and the perception range of on-board sensors, especially when in special and complex environments such as urban canyons and tunnels, it is more susceptible to interference, and the signal source obtained by a single vehicle is not ideal in terms of accuracy and reliability.

[0020] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0022] Figure 1 A flowchart of a vehicle-road-cloud collaborative positioning method based on real-scene three-dimensional data of a road provided according to an embodiment of the present invention;

[0023] FIG2 (a) is a schematic diagram of a vehicle-road-cloud collaborative positioning method according to an embodiment of the present invention;

[0024] FIG2( b ) is a schematic diagram of a point cloud image of real-scene three-dimensional data provided according to an embodiment of the present invention;

[0025] Figure 3 A schematic block diagram of a roadside camera calculating three-dimensional position information and relative position information of a target vehicle according to an embodiment of the present invention;

[0026] Figure 4 A flowchart of the working principle of a vehicle-road-cloud collaborative positioning method based on real-scene three-dimensional data of a road according to an embodiment of the present invention;

[0027] Figure 5 A flowchart of a vehicle-road-cloud collaborative positioning method based on computer programming technology according to an embodiment of the present invention;

[0028] Figure 6A schematic block diagram of a vehicle-road-cloud collaborative positioning device based on real-scene three-dimensional data of a road according to an embodiment of the present invention;

[0029] Figure 7 A schematic diagram of the structure of a server provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0031] The following describes the vehicle-road-cloud collaborative positioning method and device based on road real-scene three-dimensional data of the embodiment of the present invention with reference to the accompanying drawings. In view of the problem mentioned in the above background technology that the accuracy of vehicle positioning and detection is limited due to the limitations of vehicle volume, cost and the perception range of on-board sensors, especially when in special complex environments such as urban canyons and tunnels, it is more susceptible to interference, and the signal source obtained by a single vehicle is not ideal in terms of accuracy and reliability, the present invention provides a vehicle-road-cloud collaborative positioning method based on road real-scene three-dimensional data. In this method, the internal and external parameters of the roadside camera can be calibrated based on the three-dimensional coordinates of the image feature points obtained based on the pre-built real-scene three-dimensional model and the real-scene three-dimensional data, and then the calibrated roadside camera is obtained, and the calibrated roadside camera is used to correct the vehicle three-dimensional positioning result of the INS of the target vehicle to obtain the position of the target vehicle. Using roadside cameras for vehicle-road collaborative positioning perception has low cost, high operating efficiency, good real-time performance, and automated data collection. It can significantly improve the availability, reliability and accuracy of vehicle positioning functions in long tunnels and other environments without GNSS signals for a long time, and improve the application scope and safety of the autonomous driving industry. This solves the problems in related technologies, such as the limited accuracy of vehicle positioning and perception due to vehicle size, cost and the perception range of on-board sensors. In particular, the vehicle is more susceptible to interference in special complex environments such as urban canyons and tunnels, and the signal source obtained by a single vehicle is not ideal in terms of accuracy and reliability.

[0032] Specifically, Figure 1 The present invention provides a flow chart of a vehicle-road-cloud collaborative positioning method based on real-scene three-dimensional data of a road according to an embodiment of the present invention.

[0033] like Figure 1 As shown, the vehicle-road-cloud collaborative positioning method based on real-scene three-dimensional data of the road includes the following steps:

[0034] In step S101, the real three-dimensional data of the driving road where the target vehicle is located is obtained, and the three-dimensional coordinates of the image feature points are obtained based on the pre-constructed real three-dimensional data of the driving road.

[0035] It can be understood that the embodiments of the present invention can collect raw point cloud data by driving a data collection vehicle equipped with Lidar, GNSS, IMU and other equipment along the road where the target vehicle is located, and then perform pre-processing processes such as data space alignment, point cloud filtering, and hole repair on the raw point cloud data to complete the construction of a real-life three-dimensional model based on the PCL (Point Cloud Library) framework.

[0036] As a possible implementation method, the embodiment of the present invention can obtain the three-dimensional coordinates of the image feature points on the driving road where the target vehicle is located based on the pre-constructed real-scene three-dimensional model and real-scene three-dimensional data.

[0037] For example, the embodiment of the present invention can use the structure shown in Figure 2 (a) to realize vehicle positioning to obtain three-dimensional coordinates. In Figure 2 (a), the right side is the real-scene three-dimensional data in the cloud, the middle is the roadside camera, and the left side is the target vehicle. In this embodiment of the present invention, the real-scene three-dimensional data in the cloud on the right is a point cloud image processed by the three-dimensional point cloud data obtained by the data acquisition vehicle, which is composed of a large number of point data sets and is not taken by an optical camera, as shown in Figure 2 (b). Figure 2 (a) is a schematic diagram of a vehicle-road-cloud collaborative positioning method provided according to an embodiment of the present invention; Figure 2 (b) is a point cloud image of real-scene three-dimensional data provided according to an embodiment of the present invention.

[0038] In step S102, the internal and external parameters of the roadside camera on the driving road are calibrated according to the three-dimensional coordinates of the image feature points to obtain a calibrated roadside camera.

[0039] As a possible implementation method, the embodiment of the present invention can use the three-dimensional coordinates of the image feature points to complete the calibration of the internal and external parameters of the roadside camera, and then obtain a calibrated roadside camera.

[0040] Optionally, in one embodiment of the present invention, the internal and external parameters of a roadside camera on the driving road are calibrated according to the three-dimensional coordinates of image feature points to obtain a calibrated roadside camera, including: acquiring first image data of the driving road using the roadside camera, extracting feature points in the first image data, and obtaining the three-dimensional coordinates of the feature points in the driving road based on real-scene three-dimensional data; calibrating the internal and external parameters of the roadside camera according to the three-dimensional coordinates to obtain a calibrated roadside camera.

[0041] In some embodiments, the embodiments of the present invention use the three-dimensional coordinates of the image feature points to perform real-time calibration on the roadside camera, thereby obtaining a calibrated roadside camera, the contents of which may include:

[0042] The embodiment of the present invention uses a data collection vehicle equipped with GNSS, INS, Lidar, cameras and other equipment to collect point cloud data on the road where the target vehicle is located and complete pre-processing processes such as spatial alignment. The Lidar point cloud data is used to complete the reconstruction of the real-scene three-dimensional model based on the PCL framework, and the three-dimensional coordinates of each point are obtained. A roadside camera is used to capture continuous images to obtain first image data. Furthermore, the embodiment of the present invention extracts corresponding feature points on the current frame and the previous frame of the first image data, and completes automatic calibration of internal and external parameters such as focal length and relative rotation based on polar constraints to obtain a calibrated roadside camera. Among them, due to the degradation of visual point features in special environments such as tunnels, the IMU integrated in the roadside camera is used to provide gravity direction constraints (relative rotation angles of roll angle and pitch angle). Among them, the expression of the polar constraint in the embodiment of the present invention can be, but is not limited to, expressed as:

[0043] ,

[0044] in, , is the pixel coordinate of the corresponding feature point, , is the camera internal parameter matrix, t is the relative displacement vector of the camera, R is the relative rotation matrix of the camera, is the antisymmetric matrix of the relative displacement vector t.

[0045] Furthermore, in an embodiment of the present invention, the camera intrinsic parameter matrix The expression can be, but is not limited to:

[0046] ,

[0047] That is to say, the embodiment of the present invention can use the roadside camera to obtain the first image data of the driving road, extract feature points, and obtain the three-dimensional coordinates of the feature points in combination with the real-scene three-dimensional data, thereby completing the calibration of the internal and external parameters of the roadside camera to obtain a calibrated roadside camera.

[0048] Specifically, an embodiment of the present invention can use a roadside camera to capture continuous images of a driving road to obtain first image data, and extract corresponding feature points in the previous and next frames of the continuous frame images in the first image data, and obtain the three-dimensional coordinates of the image feature points in combination with the real-scene three-dimensional data. At the same time, the IMU integrated in the roadside camera is used to obtain the rotation angles of the roadside camera in the roll and pitch directions, and the changes in internal and external parameters of the roadside camera such as the focal length and relative rotation matrix are calculated in real time based on the polar line constraints, thereby completing the real-time automatic calibration of the roadside camera.

[0049] In step S103, the calibrated roadside camera is used to determine the three-dimensional position information and relative position information of the target vehicle, and the three-dimensional position information and relative position information are sent to the target vehicle.

[0050] As a possible implementation method, the embodiment of the present invention can use a calibrated roadside camera to determine the three-dimensional position information and relative position information of the target vehicle, and send the three-dimensional position information and relative position information to the target vehicle.

[0051] Specifically, the embodiment of the present invention can continuously shoot the target vehicle through the roadside unit, dynamically detect and continuously track the target vehicle from the image taken by the roadside camera through algorithms such as neural networks, calculate the three-dimensional position information and relative position information of the target vehicle based on the posture, three-dimensional coordinates, height of the camera relative to the ground, image coordinates of the target vehicle and other information of the roadside camera, and use the communication module to transmit the calculation results to the target vehicle. Among them, the roadside unit of the embodiment of the present invention can include but is not limited to a GNSS receiver, a time synchronization module, an industrial camera, a roadside computing unit, a communication module, etc., and the present invention does not make specific restrictions. Among them, the roadside computing unit of the embodiment of the present invention is equipped with algorithms such as automatic calibration of camera parameters, target recognition, and target positioning.

[0052] Optionally, in one embodiment of the present invention, a calibrated roadside camera is used to determine the three-dimensional position information and relative position information of the target vehicle, including: using the calibrated roadside camera to acquire second image data of the driving road; preprocessing the second image data to obtain third image data that meets preset conditions; based on the third image data, identifying the target vehicle in the third image data to obtain an identification frame of the target vehicle; and calculating the three-dimensional position information and relative position information of the target vehicle based on parameter information of the identification frame, internal and external parameters of the roadside camera and the third image data.

[0053] It can be understood by those skilled in the art that the embodiment of the present invention determines the three-dimensional position information and relative position information of the target vehicle through the calibrated roadside camera, and the content thereof may be:

[0054] First, the embodiment of the present invention uses a GNSS receiver installed on the roadside to receive a PPS (Pulse Per Second) pulse signal, performs frequency multiplication processing on the signal through a time synchronization module and uses the signal to complete the hardware triggered exposure of the roadside camera, while compensating for hardware delays and timestamping the collected second image data.

[0055] Then, the embodiment of the present invention uses a roadside camera to shoot continuous frame images, completes the shooting of the second image data, and completes preprocessing such as image dimming, color adjustment, and spatial alignment to obtain third image data that meets certain conditions, wherein the certain conditions can be set by technicians in this field according to actual conditions, and the present invention does not make specific limitations.

[0056] Next, based on the third image data, the embodiment of the present invention builds a convolutional neural network based on the YOLO framework, identifies the target vehicle in the third image data, and obtains the identification frame of the target vehicle and the parameters of the identification frame (such as length, width, height, corner point image coordinates, etc., which are not specifically limited in the present invention).

[0057] Subsequently, the embodiment of the present invention calculates the three-dimensional position information and relative position information of the target vehicle through the internal and external parameters of the roadside camera, such as the camera's posture, three-dimensional coordinates, the camera's height relative to the ground and other information (the present invention is not specifically limited), and the image coordinates of continuous frames of the target vehicle.

[0058] Finally, the roadside unit of the embodiment of the present invention sends the calculated positioning information of the target vehicle, such as three-dimensional position information and relative position information, to the target vehicle through the fifth-generation mobile communication network module. The schematic diagram of solving the position information of the target vehicle is shown in FIG. Figure 3 shown.

[0059] In step S104, based on the INS of the target vehicle, the vehicle three-dimensional position information and the vehicle relative position information of the target vehicle are acquired.

[0060] It can be understood that the target vehicle in the embodiment of the present invention is equipped with an on-board unit, wherein the on-board unit may include but is not limited to a GNSS antenna, a GNSS receiver, an IMU and a time synchronization module, etc., and the present invention does not impose specific limitations.

[0061] As a possible implementation method, the embodiment of the present invention can calculate the vehicle three-dimensional position information and vehicle relative position information of the target vehicle based on the onboard INS of the target vehicle.

[0062] In step S105, the spatiotemporal synchronization of cross-domain sensors is completed, and the relative position information of the vehicle calculated by the INS of the target vehicle is used to verify the relative position information calculated by the roadside unit, and the timestamp corresponding to the relative position information calculated by the roadside unit whose relative position error range satisfies the preset error range is obtained. The three-dimensional position information of the vehicle calculated by the INS of the target vehicle is corrected based on the three-dimensional position information calculated by the roadside camera corresponding to the timestamp to obtain a corrected three-dimensional positioning result of the vehicle, and the position of the target vehicle is located based on the vehicle three-dimensional positioning result.

[0063] In some embodiments, the embodiments of the present invention can correct the vehicle's three-dimensional position information and the vehicle's relative position information based on the three-dimensional position information and the relative position information, thereby obtaining a corrected vehicle three-dimensional positioning result and completing the positioning of the target vehicle.

[0064] Specifically, the embodiment of the present invention uses the vehicle relative position information obtained by the on-board INS to verify the vehicle relative position information obtained by the roadside unit, and selects the vehicle three-dimensional position information corresponding to the timestamp with higher accuracy to perform online estimation of the INS, so as to correct the navigation result of the mechanical arrangement, and perform IMU error compensation, so as to obtain the collaborative positioning result and complete the positioning of the target vehicle. In the embodiment of the present invention, the mechanical arrangement is used to complete the initial alignment of the INS, and complete the update of the attitude, speed and position according to the INS mechanical arrangement algorithm.

[0065] Optionally, in one embodiment of the present invention, the spatiotemporal synchronization of cross-domain sensors is completed, and the relative position information calculated by the roadside unit is verified based on the vehicle relative position information calculated by the INS of the target vehicle using the synchronized sensors after spatiotemporal synchronization, and a timestamp corresponding to the relative position information calculated by the roadside unit whose relative position error range satisfies a preset error range is obtained, including: time synchronization of multi-domain sensors is performed based on all sensors in the target vehicle and the roadside unit to obtain a time-synchronized synchronized sensor; based on the synchronized sensor, the IMU is used to assist in verifying the positioning accuracy of the roadside unit, that is, the relative position information calculated by the roadside unit is verified using the vehicle relative position information calculated by the INS of the target vehicle, and a timestamp corresponding to the relative position information calculated by the roadside unit whose relative position error range satisfies the preset error range is obtained.

[0066] In the actual implementation process, in order to achieve the spatiotemporal synchronization of cross-domain sensors, it can be understood that the target vehicle INS and GNSS in the embodiment of the present invention are both timed by the target vehicle GNSS, and the roadside camera is hard-triggered by the PPS pulse signal of the roadside GNSS to complete the timing.

[0067] Based on synchronous sensors, using IMU to assist in verifying the positioning accuracy of the roadside unit can reduce the impact of errors during target recognition and feature extraction.

[0068] Furthermore, the embodiment of the present invention can use the relative position information of the vehicle calculated by the on-board INS to verify the relative position information obtained by the roadside unit, obtain a time stamp with a high accuracy whose relative position error range satisfies a certain error range, and use the three-dimensional position information calculated by the roadside camera corresponding to the timestamp to correct the on-board INS, complete the combined navigation filtering solution, and obtain the positioning result of the vehicle when the vehicle is in a shielded environment. Among them, the certain error range can be set by a person skilled in the art according to actual conditions, and the present invention does not make any specific restrictions.

[0069] Specifically, the process of correcting the three-dimensional positioning result of the vehicle in the embodiment of the present invention is as follows:

[0070] Firstly, the embodiment of the present invention completes the time synchronization between multi-domain sensors.

[0071] Then, the embodiment of the present invention uses the vehicle-mounted IMU to assist in verifying the positioning accuracy of the roadside unit, and obtains a high-precision timestamp whose relative position error range meets a certain error range.

[0072] Finally, the embodiment of the present invention uses the three-dimensional position information calculated by the roadside camera with the same timestamp obtained by the roadside unit to correct the positioning result of the vehicle-mounted INS, thereby completing the collaborative positioning filter solution.

[0073] In addition, in some embodiments, the embodiments of the present invention can complete the matching of the vehicle-mounted positioning results and the roadside positioning results through a time synchronization module, compare the vehicle relative posture information calculated by the vehicle-mounted INS with the relative posture information calculated by the roadside unit, select the roadside unit calculation results whose errors meet a certain range, and extract the timestamp.

[0074] In some embodiments, the embodiments of the present invention can use the three-dimensional position information with the same timestamp obtained by the roadside unit to perform online estimation of the vehicle-mounted INS navigation error and sensor error, correct the mechanically arranged navigation results and perform IMU error compensation to obtain a combined navigation result.

[0075] Optionally, in one embodiment of the present invention, the vehicle's three-dimensional position information and the vehicle's relative position information are corrected based on the three-dimensional position information and the relative position information, including: determining whether the GNSS signal meets a preset signal condition; if the GNSS signal does not meet the preset signal condition, correcting the vehicle's three-dimensional position information based on the three-dimensional position information and the relative position information, otherwise correcting the vehicle's three-dimensional position information based on the GNSS signal.

[0076] In the actual implementation process, the embodiment of the present invention can correct the vehicle's three-dimensional position information and the vehicle's relative position information based on the three-dimensional position information and the relative position information, and also needs to judge the vehicle's driving environment and GNSS signal to obtain the positioning result of the target vehicle.

[0077] Specifically, in an embodiment of the present invention, when the target vehicle is in an open environment and the GNSS signal meets certain signal conditions, the vehicle-mounted GNSS / INS based on graph optimization can be used to complete the positioning of the target vehicle, and the original vehicle-mounted GNSS observation values ​​and inertial navigation observation values ​​are solved in a tight combination manner to obtain the positioning result of the target vehicle.

[0078] When the target vehicle is in a shielded environment such as a long tunnel and the GNSS signal does not meet certain signal conditions, the vehicle-mounted INS can be used for filtering and inference, and the Kalman filter algorithm can be used to recursively obtain the carrier position. In other words, the embodiment of the present invention can use the roadside unit to locate the target vehicle, obtain the positioning information of the target vehicle by the roadside unit, and fuse the vehicle position information and speed information obtained by the vehicle-mounted INS with the positioning information of the roadside unit to obtain the positioning result of the target vehicle.

[0079] In addition, certain signal conditions can be set by those skilled in the art according to actual conditions, and the present invention does not impose any specific limitations thereto.

[0080] For example, in an embodiment of the present invention, when the target vehicle is in an environment such as a long tunnel where the GNSS signal does not meet certain signal conditions, the vehicle-mounted INS is used to complete filtering and calculation to obtain the relative position information and speed information of the vehicle, complete the time synchronization of the roadside unit and the vehicle-mounted unit, and fuse the calculation result of the vehicle-mounted INS with the positioning information of the roadside camera to obtain the vehicle positioning result, including:

[0081] The embodiment of the present invention uses the vehicle-mounted IMU to measure the relative motion information of the carrier in the inertial space, and calculates the relative position, speed and posture of the target vehicle according to the navigation differential equation; the roadside equipment uses the GNSS PPS pulse signal to complete the time synchronization with the vehicle-mounted IMU. Furthermore, the embodiment of the present invention uses the relative position information of the vehicle calculated by the IMU to verify the positioning result of the roadside camera, and selects the result with an error less than a certain threshold. Furthermore, the embodiment of the present invention can use the Kalman filter to fuse the vehicle positioning information obtained by the selected roadside unit and the positioning information obtained by the vehicle-mounted INS, estimate the navigation error and sensor error online, complete the IMU error compensation, and calculate the combined navigation result.

[0082] The embodiment of the present invention uses the positioning results of the roadside unit and the positioning results of the vehicle-mounted INS for loose combination processing. When the GNSS signal does not meet certain signal conditions, the target vehicle is located through the roadside camera, and the positioning results of the vehicle-mounted INS are corrected to obtain higher-precision vehicle position information, thereby further improving the accuracy and robustness of positioning.

[0083] The working principle of the vehicle-road-cloud collaborative positioning method based on real-scene three-dimensional data of the road proposed in an embodiment of the present invention is described in detail below in conjunction with a specific embodiment.

[0084] in, Figure 4 The present invention is a flowchart of the working principle of a vehicle-road-cloud collaborative positioning method based on real-scene three-dimensional data of a road according to an embodiment of the present invention.

[0085] Figure 5 A flowchart of a vehicle-road-cloud collaborative positioning method based on computer programming technology provided according to an embodiment of the present invention.

[0086] Step S401: constructing a real-scene three-dimensional model of the road.

[0087] 1.1 Use a data collection vehicle equipped with GNSS, INS, Lidar, cameras and other equipment to collect road point cloud data;

[0088] 1.2 Complete preprocessing processes such as data space alignment, point cloud filtering, and hole repair, and complete the construction of the real-scene 3D model based on the PCL framework.

[0089] 1.3 Obtain the three-dimensional coordinates of the image feature points on the road where the target vehicle is located.

[0090] Step S402: Determine the vehicle's driving environment and GNSS signal, and determine the vehicle's initial position

[0091] 2.1 When the target vehicle is in an open environment, the GNSS signal is good and meets certain signal conditions, the graph optimization method is used to combine the vehicle-mounted GNSS and inertial navigation raw observations to obtain the positioning result of the target vehicle.

[0092] 2.2 When the vehicle is in a complex environment such as a long tunnel, there is no GNSS signal, and certain signal conditions are not met, the vehicle-mounted INS is used for filtering and calculation, and the Kalman filter algorithm is used to recursively obtain the vehicle position.

[0093] Step S403: completing automatic calibration of the roadside camera based on the three-dimensional coordinates of the feature points provided by the real-scene three-dimensional data.

[0094] 3.1 Use a roadside camera to capture continuous images and extract corresponding feature points of the previous and next frames.

[0095] 3.2 Using the 3D coordinates of the feature points provided by the real-life 3D data, the IMU integrated in the camera is used to obtain the camera's roll and pitch rotation angles, and the camera's relative attitude and focal length are calculated based on the polar constraints to complete real-time automatic calibration.

[0096] Step S404: the vehicle-mounted unit continuously photographs the target vehicle and calculates the positioning information of the target vehicle based on the parameters of the roadside camera.

[0097] 4.1 The PPS pulse signal is obtained through the GNSS receiver installed on the roadside, and the PPS signal is processed through the frequency multiplication circuit, and the signal is used to trigger the camera to complete the image capture.

[0098] 4.2 The target vehicle is photographed by a camera installed on the roadside. After completing image preprocessing such as image dimming, color adjustment, and spatial alignment, a convolutional neural network based on the YOLO framework is built to identify the target vehicle in the image.

[0099] 4.3 Calculate the three-dimensional position information and relative position information of the target vehicle based on the posture, three-dimensional coordinates, height of the camera relative to the ground, image coordinates of the target vehicle and other information of the roadside camera.

[0100] 4.4 The roadside unit sends the calculated vehicle positioning information to the vehicle through the fifth-generation mobile communication network module.

[0101] Step S405: Vehicle, road and cloud collaborate to achieve high-precision positioning of the target vehicle.

[0102] 5.1 Complete time synchronization between multi-domain sensors.

[0103] 5.2 Use the on-board IMU to assist in verifying the positioning accuracy of the roadside unit and obtain a high-precision timestamp whose relative position error range meets a certain error range.

[0104] 5.3 The three-dimensional position information calculated by the roadside camera with the same timestamp calculated by the roadside unit is used to correct the positioning result of the vehicle-mounted INS, thereby completing the collaborative positioning filter solution.

[0105] Furthermore, the algorithm provided in the embodiment of the present invention can be implemented using computer programming technology. Figure 5 shown.

[0106] Among them, in the embodiment of the present invention, the algorithm is mainly divided into two parts: the vehicle-mounted GNSS RTK / INS tightly integrated navigation and positioning system and the roadside vision / vehicle-mounted INS / cloud-based real-scene 3D data vehicle-road-cloud collaborative positioning system. The following is an introduction to each subsystem:

[0107] The vehicle-road-cloud collaborative positioning system mainly consists of nine parts:

[0108] (1) Vehicle-mounted GNSS / INS integrated navigation: When the target vehicle is traveling in an open environment with good GNSS signals and certain signal conditions are met, the GNSS / INS tight combination method based on graph optimization is used to complete the positioning of the vehicle.

[0109] (2) Construction of real-scene 3D database: Use a data collection vehicle equipped with Lidar, GNSS, IMU and other equipment to complete the collection and preprocessing of raw point cloud data, build a real-scene 3D model based on the PCL framework, and obtain the 3D coordinates of each point in the scene.

[0110] (3) Real-time self-calibration of roadside cameras: Use a roadside camera to capture continuous images, use the 3D coordinates of feature points provided by the real-life 3D data of the target vehicle on the road, use the IMU integrated in the camera to provide the relative rotation angle of some directions of the camera, and calculate the changes in camera parameters such as focal length and relative rotation matrix in real time based on the principle of polar line constraints.

[0111] (4) Roadside unit dynamic target detection: Through algorithms such as neural networks, dynamic detection and continuous tracking of target vehicles are performed from images taken by roadside cameras.

[0112] (5) Roadside unit 3D positioning: The 3D position information and relative position information of the target vehicle are calculated based on the camera’s posture, 3D coordinates, and the camera’s height relative to the ground.

[0113] (6) On-board INS positioning: The linear motion and angular motion information of the vehicle in the inertial space are measured by the IMU, and the position, velocity and attitude information of the vehicle are calculated according to the navigation differential equation.

[0114] (7) Time synchronization between the RSU and the OMU: The OMU and the RSU use the same GNSS PPS pulse as the timing signal to complete time synchronization.

[0115] (8) IMU-assisted roadside positioning result screening: Use the relative position of the vehicle calculated by the IMU to verify the positioning results of the roadside camera, and select the results with an error less than a certain threshold.

[0116] (9) Vehicle-road collaborative positioning algorithm: The Kalman filter integrates the position and speed information of the roadside camera and the vehicle-mounted INS to perform online estimation of navigation error and sensor error, correct the mechanically arranged navigation results, and perform IMU error compensation.

[0117] According to the vehicle-road-cloud collaborative positioning method based on real-scene three-dimensional data of the road proposed in the embodiment of the present invention, the internal and external parameters of the roadside camera can be calibrated based on the three-dimensional coordinates of the image feature points obtained from the pre-built real-scene three-dimensional model and the real-scene three-dimensional data, thereby obtaining a calibrated roadside camera, and using the calibrated roadside camera to correct the vehicle three-dimensional positioning result of the INS of the target vehicle to obtain the position of the target vehicle. The roadside camera can be used for vehicle-road collaborative positioning perception, which has low cost, high operating efficiency, good real-time performance, and automated data collection. It can significantly improve the availability, reliability and accuracy of the vehicle positioning function in long tunnels and other environments without GNSS signals for a long time, and improve the application scope and safety of the autonomous driving industry. Therefore, the problems in the related technology that the accuracy of vehicle positioning and perception is limited due to the limitations of vehicle size, cost and the perception range of on-board sensors, especially in special and complex environments such as urban canyons and tunnels, are more susceptible to interference, and the signal source obtained by a single vehicle is not ideal in terms of accuracy and reliability are solved.

[0118] Next, a vehicle-road-cloud collaborative positioning device based on real-scene three-dimensional data of a road according to an embodiment of the present invention is described with reference to the accompanying drawings.

[0119] Figure 6 A block diagram of a vehicle-road-cloud collaborative positioning device based on real-scene three-dimensional data of a road provided according to an embodiment of the present invention.

[0120] like Figure 6 As shown, the vehicle-road-cloud collaborative positioning device 10 based on real-scene three-dimensional data of the road includes: a cloud communication module 100, an automatic calibration module 200, a roadside calculation module 300, a vehicle-road communication module 400, a vehicle-mounted inertial navigation module 500, a vehicle-mounted GNSS module 600 and a vehicle-road collaborative positioning module 700.

[0121] The cloud communication module 100 is used to obtain the real three-dimensional data of the driving road where the target vehicle is located, and obtain the three-dimensional coordinates of the feature points based on the pre-constructed real three-dimensional model of the driving road and the real three-dimensional data.

[0122] The automatic calibration module 200 is used to calibrate the internal and external parameters of the roadside camera on the driving road according to the three-dimensional coordinates to obtain a calibrated roadside camera.

[0123] The roadside calculation module 300 is used to determine the three-dimensional position information and relative position information of the target vehicle using the calibrated roadside camera.

[0124] The vehicle-road communication module 400 is used to send the three-dimensional position information and the relative position information to the target vehicle.

[0125] The vehicle-mounted inertial navigation module 500 is used to obtain the vehicle three-dimensional position information and vehicle relative position information of the target vehicle based on the INS of the target vehicle.

[0126] The vehicle-mounted GNSS module 600 is used to obtain GNSS observation values ​​and GNSS PPS pulse signals.

[0127] The vehicle-road collaborative positioning module 700 is used to perform combined navigation based on GNSS and INS of the target vehicle when the GNSS signal meets the preset signal conditions, and to complete the spatiotemporal synchronization of cross-domain sensors when the GNSS signal does not meet the preset signal conditions, and to use the vehicle relative position information calculated by the INS of the target vehicle to verify the relative position information calculated by the roadside unit, to obtain the timestamp corresponding to the relative position information calculated by the roadside unit whose relative position error range meets the preset error range, and to correct the vehicle three-dimensional position information calculated by the INS of the target vehicle based on the three-dimensional position information calculated by the roadside camera corresponding to the timestamp, to obtain the corrected vehicle three-dimensional positioning result.

[0128] Optionally, in one embodiment of the present invention, the automatic calibration module 200 includes:

[0129] Among them, the first acquisition unit is used to use a roadside camera to acquire first image data of the driving road, extract feature points in the first image data, and obtain the three-dimensional coordinates of the feature points in the driving road based on the real-scene three-dimensional data.

[0130] The automatic calibration unit is used to calibrate the internal and external parameters of the roadside camera according to the three-dimensional coordinates to obtain a calibrated roadside camera.

[0131] Optionally, in one embodiment of the present invention, the roadside calculation module 300 includes: a second acquisition unit, a preprocessing unit, an identification unit and a roadside calculation unit.

[0132] The second acquisition unit is used to acquire second image data of the driving road using a calibrated roadside camera.

[0133] The preprocessing unit is used to preprocess the second image data to obtain third image data that meets preset conditions.

[0134] The recognition unit is used to recognize the target vehicle in the third image data based on the third image data, and obtain the recognition frame of the target vehicle.

[0135] The roadside calculation unit is used to calculate the three-dimensional position information and relative position information of the target vehicle based on the parameter information of the recognition frame, the internal and external parameters of the roadside camera and the third image data.

[0136] Optionally, in one embodiment of the present invention, the collaborative positioning module 700 includes: a time synchronization unit and a verification unit.

[0137] The time synchronization unit is used to perform time synchronization of multi-domain sensors based on all sensors in the target vehicle and the roadside unit to obtain synchronized sensors after time synchronization.

[0138] The verification unit is used to verify the positioning accuracy of the roadside unit based on the synchronous sensor and with the assistance of IMU, that is, to verify the relative position information calculated by the roadside unit using the vehicle relative position information calculated by the INS of the target vehicle, and obtain the timestamp corresponding to the relative position information calculated by the roadside unit whose relative position error range satisfies the preset error range.

[0139] Optionally, in one embodiment of the present invention, the collaborative positioning module 700 includes: a judgment unit and a positioning unit.

[0140] The determination unit is used to determine whether the GNSS signal meets a preset signal condition.

[0141] The positioning unit is used to correct the three-dimensional position information of the vehicle based on the three-dimensional position information and the relative position information when the GNSS signal does not meet the preset signal condition, and otherwise correct the three-dimensional position information of the vehicle based on the GNSS signal.

[0142] It should be noted that the aforementioned explanation of the embodiment of the vehicle-road-cloud collaborative positioning method based on real-scene three-dimensional data of the road is also applicable to the vehicle-road-cloud collaborative positioning device based on real-scene three-dimensional data of the road in this embodiment, and will not be repeated here.

[0143] According to the vehicle-road-cloud collaborative positioning device based on real-scene three-dimensional data of the road proposed in the embodiment of the present invention, the internal and external parameters of the roadside camera can be calibrated based on the three-dimensional coordinates of the image feature points obtained by the pre-built real-scene three-dimensional model and the real-scene three-dimensional data, and then the calibrated roadside camera can be obtained, and the calibrated roadside camera can be used to correct the vehicle three-dimensional positioning result of the INS of the target vehicle to obtain the position of the target vehicle. The roadside camera can be used for vehicle-road collaborative positioning perception, which has low cost, high operating efficiency, good real-time performance, and automated data collection. It can significantly improve the availability, reliability and accuracy of the vehicle positioning function in long tunnels and other environments without GNSS signals for a long time, and improve the application scope and safety of the autonomous driving industry. Therefore, the problems in the related technology that the accuracy of vehicle positioning and perception is limited due to the limitations of vehicle size, cost and the perception range of on-board sensors, especially in special and complex environments such as urban canyons and tunnels, are more susceptible to interference, and the signal source obtained by a single vehicle is not ideal in terms of accuracy and reliability are solved.

[0144] Figure 7This is a schematic diagram of the structure of a server provided according to an embodiment of the present invention. The server may include:

[0145] A memory 701 , a processor 702 , and a computer program stored in the memory 701 and executable on the processor 702 .

[0146] When the processor 702 executes the program, the vehicle-road-cloud collaborative positioning method based on real-scene three-dimensional data of the road provided in the above embodiment is implemented.

[0147] Furthermore, the server also includes:

[0148] The communication interface 703 is used for communication between the memory 701 and the processor 702 .

[0149] The memory 701 is used to store computer programs that can be executed on the processor 702 .

[0150] The memory 701 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0151] If the memory 701, the processor 702 and the communication interface 703 are implemented independently, the communication interface 703, the memory 701 and the processor 702 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0152] Optionally, in a specific implementation, if the memory 701, the processor 702 and the communication interface 703 are integrated on a chip, the memory 701, the processor 702 and the communication interface 703 can communicate with each other through an internal interface.

[0153] The processor 702 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention.

[0154] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the vehicle-road-cloud collaborative positioning method based on real-scene three-dimensional data of the road is implemented as described above.

[0155] An embodiment of the present invention also provides a computer program product, including a computer program, which, when executed, implements the above-mentioned vehicle-road-cloud collaborative positioning method based on real-scene three-dimensional data of the road.

[0156] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0157] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "N" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0158] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.

[0159] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection with one or N wirings (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways as necessary and then storing it in a computer memory.

[0160] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented by hardware, as in another embodiment, it can be implemented by any one or a combination of multiple of the following technologies known in the art: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0161] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0162] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0163] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present invention. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A vehicle-road-cloud collaborative positioning method based on real-scene three-dimensional data of roads, characterized in that: The following steps are involved: Acquire real-scene three-dimensional data of the driving road on which the target vehicle is located, and obtain the three-dimensional coordinates of feature points in the image of the driving road taken by a roadside camera in the driving road based on the real-scene three-dimensional data pre-constructed on the driving road; Calibrate the internal and external parameters of the roadside camera according to the three-dimensional coordinates of the image feature points to obtain a calibrated roadside camera; Determine the three-dimensional position information and relative position information of the target vehicle by using the calibrated roadside camera, and send the three-dimensional position information and the relative position information to the target vehicle, wherein the three-dimensional position information is the world coordinate information of the target vehicle calculated by using the calibrated roadside camera to take the image of the target vehicle, and the relative position information is the relative position relationship of the target vehicle at the front and rear moments calculated based on the front and rear two frames of images taken by the calibrated roadside camera; Based on the inertial navigation system INS of the target vehicle, the vehicle three-dimensional position information and vehicle relative position information of the target vehicle are obtained, wherein the vehicle three-dimensional position information is the world coordinate information of the target vehicle calculated by the INS, and the vehicle relative position information is the relative position relationship of the target vehicle before and after the time calculated by the INS; Complete the spatiotemporal synchronization of cross-domain sensors, and use the vehicle relative position information to verify the relative position information, obtain a timestamp corresponding to the relative position information whose relative position error range satisfies a preset error range, and correct the vehicle three-dimensional position information based on the three-dimensional position information corresponding to the timestamp to obtain a corrected vehicle three-dimensional positioning result, and locate the position of the target vehicle based on the vehicle three-dimensional positioning result; Wherein, the correcting the three-dimensional position information of the vehicle includes: Determine whether the GNSS signal meets the preset signal conditions; If the GNSS signal does not meet the preset signal condition, the three-dimensional position information of the vehicle is corrected based on the three-dimensional position information corresponding to the timestamp; otherwise, the three-dimensional position information of the vehicle is corrected based on the GNSS signal.

2. The vehicle-road-cloud collaborative positioning method based on road real scene three-dimensional data according to claim 1 is characterized in that: The step of calibrating the internal and external parameters of the roadside camera on the driving road according to the three-dimensional coordinates of the image feature points to obtain a calibrated roadside camera includes: Acquire first image data of the driving road using the roadside camera, extract feature points in the first image data, and obtain three-dimensional coordinates of the feature points in the driving road based on the real-scene three-dimensional data; The internal and external parameters of the roadside camera are calibrated according to the three-dimensional coordinates to obtain a calibrated roadside camera.

3. The vehicle-road-cloud collaborative positioning method based on road real scene three-dimensional data according to claim 1 is characterized in that: The method of using the calibrated roadside camera to determine the three-dimensional position information and relative position information of the target vehicle includes: Acquiring second image data of the driving road using the calibrated roadside camera; Preprocessing the second image data to obtain third image data that meets a preset condition; Based on the third image data, identifying the target vehicle in the third image data, and obtaining an identification frame of the target vehicle; The three-dimensional position information and relative position information of the target vehicle are calculated based on the parameter information of the recognition frame, the internal and external parameters of the roadside camera and the third image data.

4. The vehicle-road-cloud collaborative positioning method based on road real scene three-dimensional data according to claim 1 is characterized in that: The method of completing the spatiotemporal synchronization of the cross-domain sensors, and verifying the relative position information calculated by the roadside unit according to the vehicle relative position information calculated by the INS of the target vehicle using the spatiotemporal synchronized synchronization sensor, and obtaining a timestamp corresponding to the relative position information calculated by the roadside unit whose relative position error range satisfies a preset error range, includes: Based on the target vehicle and all sensors in the roadside unit, time synchronization of multi-domain sensors is performed to obtain synchronized sensors after time synchronization; Based on the synchronous sensor, the inertial measurement unit IMU is used to assist in completing the verification of the positioning accuracy of the roadside unit, that is, the vehicle relative position information calculated by the INS of the target vehicle is used to verify the relative position information calculated by the roadside unit, and the timestamp corresponding to the relative position information calculated by the roadside unit whose relative position error range satisfies the preset error range is obtained.

5. A vehicle-road-cloud collaborative positioning device based on real-scene three-dimensional data of roads, characterized in that: include: A cloud communication module is used to obtain real-scene three-dimensional data of the driving road where the target vehicle is located, and obtain the three-dimensional coordinates of feature points in the image of the driving road taken by a roadside camera in the driving road based on the real-scene three-dimensional data pre-constructed on the driving road; An automatic calibration module, used for calibrating the internal and external parameters of the roadside camera on the driving road according to the three-dimensional coordinates to obtain a calibrated roadside camera; A roadside calculation module, used to determine the three-dimensional position information and relative position information of the target vehicle using the calibrated roadside camera, wherein the three-dimensional position information is the world coordinate information of the target vehicle calculated by taking the image of the target vehicle with the calibrated roadside camera, and the relative position information is the relative position relationship of the target vehicle at the front and rear moments calculated based on the front and rear two frames of images taken by the calibrated roadside camera; A vehicle-road communication module, used to send the three-dimensional position information and the relative position information to the target vehicle; A vehicle-mounted inertial navigation module, used for obtaining the vehicle three-dimensional position information and vehicle relative position information of the target vehicle based on the INS of the target vehicle, wherein the vehicle three-dimensional position information is the world coordinate information of the target vehicle calculated by the INS, and the vehicle relative position information is the relative position relationship of the target vehicle before and after the time calculated by the INS; The vehicle-mounted GNSS module is used to obtain GNSS signal observation values ​​and GNSS pulse per second PPS pulse signals; The vehicle-road collaborative positioning module is used to perform combined navigation based on the GNSS and the target vehicle INS when the GNSS signal meets the preset signal conditions, and to complete the spatiotemporal synchronization of cross-domain sensors when the GNSS signal does not meet the preset signal conditions, and to verify the relative position information using the vehicle relative position information to obtain the timestamp corresponding to the relative position information whose relative position error range meets the preset error range, and to correct the vehicle's three-dimensional position information based on the three-dimensional position information corresponding to the timestamp to obtain a corrected vehicle three-dimensional positioning result.

6. The vehicle-road-cloud collaborative positioning device based on real-scene three-dimensional data of road according to claim 5, characterized in that: The automatic calibration module comprises: an acquisition unit, configured to acquire first image data of the driving road using the roadside camera, extract feature points in the first image data, and obtain three-dimensional coordinates of the feature points in the driving road based on the real-scene three-dimensional data; The automatic calibration unit is used to calibrate the internal and external parameters of the roadside camera according to the three-dimensional coordinates to obtain a calibrated roadside camera.

7. A server, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the vehicle-road-cloud collaborative positioning method based on real-scene three-dimensional data of a road as described in any one of claims 1 to 4.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the vehicle-road-cloud collaborative positioning method based on real-scene three-dimensional data of a road as described in any one of claims 1-4.

9. A computer program product, characterized in that It includes a computer program which, when executed, is used to implement the vehicle-road-cloud collaborative positioning method based on real-scene three-dimensional data of a road as described in any one of claims 1 to 4.

Images