Vehicle positioning method, device and equipment in complex environment and storage medium

Abstract

The invention discloses a vehicle positioning method, device and equipment in a complex environment and a storage medium, and belongs to the technical field of vehicle control. When a vehicle is large in positioning error based on global navigation satellite system (GNSS) data, real-time relative movement (position change) information of the vehicle is determined based on an SLAM technology through collected image frames in a driving environment, and the final accurate GNSS is stored in real time through a positioning precision monitoring device to serve as initial data. The initial data and the motion change information are combined to obtain the calculated GNSS, GNSS positioning data of the vehicle in a complex environment in a short time are obtained, and the positioning requirement in the complex environment is met. Moreover, the real-time dynamic carrier phase difference technology is adopted to correct the positioning data, so that a short-time high-precision positioning capability can be provided in a region with a relatively large GNSS error, and poor experience and potential risks caused by sudden loss of high-precision positioning are avoided.

Description

Technical Field

[0001] The present application relates to the field of vehicle control technology, and in particular to a vehicle positioning method, device, equipment and storage medium in a complex environment. Background Art

[0002] With the development of navigation and positioning technology, accurate and reliable vehicle positioning technology is being used more and more widely. Among them, the application of global navigation satellite systems (GNSS) positioning technology is relatively common. However, in complex environments such as tunnels and urban canyons, GNSS signals will produce multipath effects due to obstruction and reflection, which will affect the positioning accuracy. Therefore, how to provide high-precision positioning in complex environments has become a problem to be solved. Summary of the invention

[0003] The embodiments of the present application provide a method, device, equipment and storage medium for positioning a vehicle in a complex environment, which can achieve accurate positioning even when the error of the global navigation satellite system data is large. The technical solution is as follows.

[0004] On the one hand, a vehicle positioning method in a complex environment is provided, the method comprising:

[0005] During the driving process of the vehicle, real-time global navigation satellite system data of the vehicle is collected by a positioning device on the vehicle, and image frames of the driving environment are continuously collected by a camera device on the vehicle;

[0006] In the case where the error of the global navigation satellite system data reaches the error condition, determining initial data, the initial data referring to the global navigation satellite system data whose error does not reach the error condition and whose collection time is the latest among the global navigation satellite system data collected at multiple time points;

[0007] Determine the real-time relative motion information of the vehicle based on the image frames respectively collected at multiple time points, wherein the relative motion information includes at least one of a moving distance and a rotation angle;

[0008] Determining real-time first positioning data of the vehicle based on the initial data and the relative motion information;

[0009] The first positioning data is corrected by using real-time dynamic carrier phase difference technology to obtain real-time target positioning data of the vehicle.

[0010] In some embodiments, the method further comprises:

[0011] During the driving of the vehicle, inertial data of the vehicle during the driving is collected by an inertial navigation device on the vehicle, wherein the inertial data includes at least one of real-time acceleration data and angular velocity data of the vehicle;

[0012] Based on the initial data and the inertial data, obtaining real-time second positioning data of the vehicle;

[0013] The method of using the real-time dynamic carrier phase difference technology to correct the first positioning data to obtain the real-time target positioning data of the vehicle includes:

[0014] The first positioning data is corrected by using real-time dynamic carrier phase difference technology, and the second positioning data and the corrected first positioning data are fused to obtain real-time target positioning data of the vehicle.

[0015] In some embodiments, the acquiring of the real-time second positioning data of the vehicle based on the initial data and the inertial data includes:

[0016] Using real-time dynamic carrier phase difference technology to correct the initial data, and obtaining the second positioning data based on the corrected initial data and the inertial data;

[0017] Alternatively, based on the initial data and the inertial data, a real-time third positioning data of the vehicle is obtained, and the third positioning data is corrected by using a real-time dynamic carrier phase difference technology to obtain the second positioning data.

[0018] In some embodiments, the fusing the second positioning data and the corrected first positioning data to obtain the real-time target positioning data of the vehicle includes:

[0019] Determine the average of the second positioning data and the corrected first positioning data as the target positioning data; or,

[0020] The target positioning data is obtained by weighted summing the second positioning data and the corrected first positioning data.

[0021] In some embodiments, the reaching the error condition includes at least one of the following:

[0022] The signal-to-noise ratio of the global navigation satellite system data is less than a first threshold;

[0023] The position error at the current moment determined based on the global navigation satellite system data reaches a second threshold;

[0024] The speed error at the current moment determined based on the global navigation satellite system data reaches a third threshold;

[0025] The number of satellites generating the global navigation satellite system data is less than a fourth threshold.

[0026] In some embodiments, the method further comprises:

[0027] In the first positioning mode, if the error of the global navigation satellite system data collected in real time no longer meets the error condition, the first positioning mode is switched to the second positioning mode, wherein the first positioning mode refers to a mode of acquiring the real-time target positioning data of the vehicle based on the image frame;

[0028] In the second positioning mode, the real-time dynamic carrier phase difference technology is used to correct the real-time global navigation satellite system data to obtain the real-time target positioning data of the vehicle.

[0029] In some embodiments, determining the real-time relative motion information of the vehicle based on image frames collected at multiple time points includes:

[0030] Extracting feature points in each image frame, wherein the feature points include at least one of an edge and a corner point in the image frame;

[0031] Feature matching is performed on adjacent image frames based on feature points, and real-time relative motion information of the vehicle is determined based on matching results of multiple image frames.

[0032] On the other hand, a vehicle positioning device in a complex environment is provided, the device comprising:

[0033] A collection module, used to collect real-time global navigation satellite system data of the vehicle through a positioning device on the vehicle during the driving process of the vehicle, and continuously collect image frames in the driving environment through a camera device on the vehicle;

[0034] a determination module, configured to determine initial data when an error of the global navigation satellite system data reaches an error condition, the initial data referring to the global navigation satellite system data whose error does not reach the error condition and whose collection time is the latest among the global navigation satellite system data collected at multiple time points;

[0035] The determination module is further configured to determine the real-time relative motion information of the vehicle based on the image frames respectively collected at multiple time points, wherein the relative motion information includes at least one of a moving distance and a rotation angle;

[0036] The determination module is further used to determine the real-time first positioning data of the vehicle based on the initial data and the relative motion information;

[0037] The correction module is also used to adopt real-time dynamic carrier phase difference technology to correct the first positioning data to obtain real-time target positioning data of the vehicle.

[0038] In some embodiments, the acquisition module is further used to:

[0039] During the driving of the vehicle, inertial data of the vehicle during the driving is collected by an inertial navigation device on the vehicle, wherein the inertial data includes at least one of real-time acceleration data and angular velocity data of the vehicle;

[0040] The determination module is further used to determine the real-time second positioning data of the vehicle based on the initial data and the inertial data;

[0041] The correction module is used to:

[0042] The first positioning data is corrected by using real-time dynamic carrier phase difference technology, and the second positioning data and the corrected first positioning data are fused to obtain real-time target positioning data of the vehicle.

[0043] In some embodiments, the determining module is used to:

[0044] Using real-time dynamic carrier phase difference technology to correct the initial data, and obtaining the second positioning data based on the corrected initial data and the inertial data;

[0045] Alternatively, based on the initial data and the inertial data, a real-time third positioning data of the vehicle is obtained, and the third positioning data is corrected by using a real-time dynamic carrier phase difference technology to obtain the second positioning data.

[0046] In some embodiments, the correction module is used to:

[0047] Determine the average of the second positioning data and the corrected first positioning data as the target positioning data; or,

[0048] The target positioning data is obtained by weighted summing the second positioning data and the corrected first positioning data.

[0049] In some embodiments, the reaching the error condition includes at least one of the following:

[0050] The signal-to-noise ratio of the global navigation satellite system data is less than a first threshold;

[0051] The position error at the current moment determined based on the global navigation satellite system data reaches a second threshold;

[0052] The speed error at the current moment determined based on the global navigation satellite system data reaches a third threshold;

[0053] The number of satellites generating the global navigation satellite system data is less than a fourth threshold.

[0054] In some embodiments, the apparatus further comprises a switching module, configured to:

[0055] In the first positioning mode, if the error of the global navigation satellite system data collected in real time no longer meets the error condition, the first positioning mode is switched to the second positioning mode, wherein the first positioning mode refers to a mode of acquiring the real-time target positioning data of the vehicle based on the image frame;

[0056] The correction module is also used to correct the real-time global navigation satellite system data by adopting real-time dynamic carrier phase difference technology in the second positioning mode to obtain real-time target positioning data of the vehicle.

[0057] In some embodiments, the determining module is used to:

[0058] Extracting feature points in each image frame, wherein the feature points include at least one of an edge and a corner point in the image frame;

[0059] Feature matching is performed on adjacent image frames based on feature points, and real-time relative motion information of the vehicle is determined based on matching results of multiple image frames.

[0060] On the other hand, a vehicle controller is provided, which includes a processor and a memory, wherein at least one program code is stored in the memory, and the at least one program code is loaded and executed by the processor to implement the above-mentioned vehicle positioning method in a complex environment.

[0061] On the other hand, a computer-readable storage medium is provided, in which at least one program code is stored. The at least one program code is loaded and executed by a processor to implement the above-mentioned vehicle positioning method in a complex environment.

[0062] On the other hand, a computer program product is provided, wherein the product stores at least one program code, and the at least one program code is used to be executed by a processor to implement the above-mentioned vehicle positioning method in a complex environment.

[0063] It is to be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the present disclosure.

[0064] In the embodiment of the present application, when the positioning error of the vehicle based on the global navigation satellite system data is large, the real-time relative motion information of the vehicle is determined based on the image frames collected in the driving environment, and the last accurate global navigation satellite system data is used as the initial data, and the initial data and the relative motion information are combined to obtain the real-time positioning data of the vehicle. Since the relative motion information of the vehicle depends on the environmental characteristics captured by the camera device, it is not affected by satellite signals or interference. The relative motion information determined in this way is accurate, and then combined with the initial data provided by the global navigation satellite system data, accurate positioning data can be obtained, reducing the overall positioning error, and achieving accurate positioning even when the error of the global navigation satellite system data is large. In addition, the real-time dynamic carrier phase difference technology is also used to correct the positioning data. Since the real-time dynamic carrier phase difference technology can eliminate the orbit error, clock error, and most of the ionospheric error and tropospheric error of the total satellite of the positioning data, the accuracy and precision of the vehicle's positioning data can be further improved. The GNSS positioning error is at the meter level, and the high-precision positioning error is at the centimeter level. This method can still achieve centimeter-level positioning capability when the satellite signal is not good. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 is a schematic diagram of an implementation environment of a vehicle positioning method in a complex environment shown in an exemplary embodiment of the present application;

[0066] Figure 2 is a flow chart of a vehicle positioning method in a complex environment shown in an exemplary embodiment of the present application;

[0067] Figure 3 is a flow chart of a vehicle positioning method in a complex environment shown in an exemplary embodiment of the present application;

[0068] Figure 4 is a block diagram of a vehicle positioning device in a complex environment shown in an exemplary embodiment of the present application;

[0069] Figure 5 It is a block diagram of a vehicle controller shown in an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0070] In order to make the technical solutions and advantages of the present application clearer, the implementation methods of the present application are described in further detail below.

[0071] The terms "first", "second", "third" and "fourth" etc. in the specification and claims of the present application and the drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0072] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions. For example, the positioning data, image frames, etc. involved in this application are all obtained with full authorization.

[0073] Please refer to Figure 1 , which shows a schematic diagram of the implementation environment of the vehicle positioning method in a complex environment shown in an exemplary embodiment of the present application; the implementation environment includes: a vehicle controller 10, a positioning device 20, a camera device 30 and an inertial navigation device 40. The vehicle controller 10 can communicate with the positioning device 20, the camera device 30 and the inertial navigation device 40 through a CAN (Controller Area Network) bus. The vehicle controller 10 can also communicate with a user terminal through a wireless network. The vehicle controller 10 collects real-time global navigation satellite system data of the vehicle through the positioning device 20, collects image frames on the road on which the vehicle is traveling through the camera device 30, and collects inertial data during the vehicle's driving process through the inertial navigation device 40, so as to obtain real-time positioning data of the vehicle through these data, and this process is described in detail in subsequent embodiments. The vehicle in the embodiment of the present application can be a new energy vehicle or a fuel vehicle; new energy vehicles include pure electric vehicles or hybrid vehicles.

[0074] Please refer to Figure 2 , which shows a flow chart of a vehicle positioning method in a complex environment shown in an exemplary embodiment of the present application. Figure 2 , the method comprises the following steps.

[0075] Step 201: During the driving process of the vehicle, the vehicle controller collects real-time global navigation satellite system data of the vehicle through the positioning device on the vehicle, and continuously collects image frames on the driving road through the camera equipment on the vehicle.

[0076] In the embodiment of the present application, the positioning device may be a GNSS (Global Navigation Satellite System) receiver, which is a high-precision GNSS receiver that supports multiple frequency bands and multiple systems. The receiver receives global navigation satellite system data, that is, the GNSS receiver receives signals from multiple satellites. These signals contain the location information of the satellite and the timestamp of the signal transmission. The GNSS receiver outputs positioning data based on the received data. The positioning data includes information such as latitude and longitude, altitude, and speed, which can be used for navigation and positioning services.

[0077] The camera device may be a monocular or binocular camera. Optionally, the camera device is installed in front of the vehicle, that is, the camera device is a front-view camera, which can obtain high-definition images with a higher frame rate.

[0078] Step 202: When the error of the GNSS data reaches the error condition, the vehicle controller determines initial data, where the initial data refers to the GNSS data whose error does not reach the error condition and whose collection time is the latest among the GNSS data collected at multiple time points.

[0079] In some embodiments, reaching the error condition includes at least one of the following:

[0080] (1) The signal-to-noise ratio of the global navigation satellite system data is less than a first threshold.

[0081] Among them, the signal-to-noise ratio refers to the ratio of the navigation signal power to the noise power. The signal-to-noise ratio is calculated in logarithmic form and is expressed in decibels. The signal-to-noise ratio reflects the relationship between the signal strength and the internal noise strength of the GNSS receiver. Therefore, a high signal-to-noise ratio means high signal quality and can provide more accurate positioning results, while a low signal-to-noise ratio means poor signal quality, which in turn provides inaccurate positioning results. Therefore, when the signal-to-noise ratio is small, it is convenient and efficient to determine whether the error of the global navigation satellite system data has reached the error condition.

[0082] (2) The position error at the current moment determined based on the global navigation satellite system data reaches a second threshold.

[0083] The position error at the current moment can be determined based on the position, speed and the position at the previous moment. For example, the position at the current moment is calculated based on the position and speed at the previous moment, and the difference between the calculated position and the position determined by the global navigation satellite system data is the position error at the current moment. The larger the position error, the lower the positioning accuracy.

[0084] (3) The speed error at the current moment determined based on the global navigation satellite system data reaches a third threshold.

[0085] The speed error at the current moment can be determined based on the speed, acceleration at the previous moment and the speed at the current moment. For example, the speed at the current moment is calculated based on the speed and acceleration at the previous moment, and the difference between the calculated speed and the speed determined by the global navigation satellite system data is the speed error at the current moment. The larger the speed error, the lower the positioning accuracy.

[0086] (4) The number of satellites generating global navigation satellite system data is less than a fourth threshold.

[0087] Among them, due to the change in positioning signal strength, the number of satellites generating GNSS data can change in real time. The more satellites there are, the higher the positioning accuracy.

[0088] In some embodiments, a weighted summation may be performed on the at least one error to obtain a comprehensive error value, and when the comprehensive error value reaches an error threshold, it is determined that the error of the global navigation satellite system data reaches an error condition.

[0089] Step 203: The vehicle controller determines the real-time relative motion information of the vehicle based on the image frames collected at multiple time points.

[0090] In some embodiments, the process of determining the real-time relative motion information of the vehicle based on image frames captured at multiple time points includes the following steps: extracting feature points in each image frame, the feature points including at least one of the edges and corner points in the image frame; performing feature matching on adjacent image frames based on the feature points, and determining the real-time relative motion information of the vehicle based on the matching results of multiple image frames.

[0091] In some embodiments, in the process of extracting feature points, abnormal points or data are also eliminated.

[0092] Among them, feature matching is performed on adjacent image frames based on feature points to determine the position change of feature points in adjacent image frames, that is, the matching result refers to the position change of feature points in adjacent image frames. Then, based on the matching result, the relative motion information of the camera (camera device) is obtained, and its relative motion information includes the rotation angle and the translation distance. Then, the relative motion information of the vehicle is obtained based on the relative motion information of the camera. Among them, the relative motion information of the vehicle is obtained by performing coordinate transformation on the relative motion information of the camera, that is, converting the camera coordinate system into the world coordinate system.

[0093] In some embodiments, a sparse map may also be established using real-time relative motion information of vehicles to provide positioning data through the sparse map.

[0094] Step 204: The vehicle controller determines the first real-time positioning data of the vehicle based on the initial data and the relative motion information.

[0095] The relative motion information includes at least one of the moving distance and the rotation angle, and the rotation angle is also the orientation angle of the vehicle, including the yaw angle, the pitch angle, the roll angle, etc. Then, after the rotation angle in the initial data changes by the rotation angle in the relative motion information, the real-time rotation angle in the first positioning data is obtained. Similarly, after the position information in the initial data changes by the moving distance in the relative motion information, the real-time position information in the first positioning data is obtained.

[0096] In some embodiments, there are multiple camera devices on the vehicle, and the relative motion information corresponding to each camera device is determined through the image frames respectively captured by each camera device, and the relative motion information corresponding to each camera device is fused, such as weighted summation, to obtain the target relative motion information, and the real-time first positioning data of the vehicle is obtained based on the initial data and the target relative motion information. Alternatively, based on the initial data and the relative motion information corresponding to each camera device, the first positioning data corresponding to each camera device is obtained, and the first positioning data corresponding to each camera device is fused, such as weighted summation, to obtain the real-time first positioning data of the vehicle.

[0097] In this embodiment, image frames are collected by multiple camera devices, and then the positioning data corresponding to the multiple image frames are fused, so that the first positioning data finally obtained is more accurate.

[0098] Step 205: The vehicle controller uses real-time dynamic carrier phase difference technology to correct the first positioning data to obtain real-time target positioning data of the vehicle.

[0099] In an embodiment of the present application, real-time kinematic (RTK) technology is implemented through the following steps: a base station receives GNSS data, and determines positioning error data based on the actual position of the base station and the position of the base station determined based on the GNSS data; a vehicle controller receives the positioning error data sent by the base station, and based on the positioning error data, corrects its own GNSS data to obtain accurate GNSS data, and then obtains accurate positioning data.

[0100] In this embodiment, the vehicle controller receives the positioning error data sent by the base station, which may be the positioning error data at the time of collecting the initial data. Then, based on the positioning error data, the first positioning data is corrected to obtain the target positioning data.

[0101] In the embodiment of the present application, the relative motion information of the vehicle is determined by image frames to obtain positioning data. This technology can be called Simultaneous Localization and Mapping (SLAM) technology. This technology can provide relatively high-precision local positioning information. Therefore, this technology is beneficial to improve the situation where GNSS data positioning is poor, to ensure that high-precision positioning can continue to be provided within a certain time or distance, and to further improve the accuracy and robustness of the positioning data.

[0102] In some embodiments, during the driving process of the vehicle, the vehicle controller also collects inertial data during the driving process of the vehicle through the inertial navigation device on the vehicle, and the inertial data includes at least one of the real-time acceleration data and angular velocity data of the vehicle; based on the initial data and the inertial data, the real-time second positioning data of the vehicle is determined. Then the above-mentioned use of real-time dynamic carrier phase difference technology to correct the first positioning data to obtain the real-time target positioning data of the vehicle also includes the following implementation method: using real-time dynamic carrier phase difference technology to correct the first positioning data, fusing the second positioning data and the corrected first positioning data to obtain the real-time target positioning data of the vehicle.

[0103] The inertial navigation device includes at least one of an acceleration sensor and a gyroscope, etc., which are used to detect acceleration and angular velocity, respectively. In this embodiment, after the first positioning data is corrected by using the real-time dynamic carrier phase difference technology, the second positioning data obtained based on the inertial data is fused with the corrected first positioning data, so that the target positioning data obtained is more accurate, further improving the accuracy and robustness of the positioning data.

[0104] Among them, the speed data is determined based on the inertial data, and then combined with the collection time of the inertial data and the collection time of the initial data, the moving distance of the vehicle at the collection time of the inertial data can be obtained, and the second positioning data can be obtained based on the moving distance and the initial data.

[0105] In some embodiments, the above process of determining the real-time second positioning data of the vehicle based on the initial data and the inertial data includes the following two implementation methods.

[0106] First, the vehicle controller uses real-time dynamic carrier phase difference technology to correct the initial data, and obtains the second positioning data based on the corrected initial data and inertial data.

[0107] In this implementation, the initial data is first corrected based on the real-time dynamic carrier phase difference technology to make the initial data more accurate, and then the second positioning data is obtained by combining the inertial data to make the obtained second positioning data more accurate.

[0108] The second method is that the vehicle controller obtains the real-time third positioning data of the vehicle based on the initial data and inertial data, and uses real-time dynamic carrier phase difference technology to correct the third positioning data to obtain the second positioning data.

[0109] In this implementation, after the positioning data is obtained based on the initial data and the inertial data, the real-time dynamic carrier phase difference technology is used to correct it, so that the obtained second positioning data is more accurate.

[0110] In some embodiments, the above-mentioned process of fusing the second positioning data and the corrected first positioning data includes the following implementation methods: determining the mean of the second positioning data and the corrected first positioning data as the target positioning data; or performing weighted summation on the second positioning data and the corrected first positioning data to obtain the target positioning data. In this embodiment, the mean or weighted sum of the two types of data is used as the target positioning data, so that the obtained target positioning data fuses the two types of data, thereby making the target positioning data more accurate.

[0111] The weights of the corrected first positioning data and the second positioning data can be set as needed. Optionally, a Kalman filter or a particle filter is used to fuse the second positioning data and the corrected first positioning data.

[0112] The Kalman filter or particle filter can perform a weighted average on the two positioning data. In some embodiments, in the process of fusing the two data, a greater weight is given to the corrected first positioning data and a smaller weight is given to the second positioning data, so that the first positioning data plays a greater role in determining the target positioning data.

[0113] In the embodiment of the present application, the camera device not only collects image frames, but also records the acquisition timestamp of each image frame. Similarly, the inertial navigation device not only collects instantaneous acceleration data and angular velocity data, but also records the acquisition timestamp of the acceleration data and angular velocity data, so as to facilitate the subsequent fusion of the first positioning data and the second positioning data at the same time.

[0114] See also Figure 3 , Figure 3It is a flow chart of a vehicle positioning method in a complex environment provided by an embodiment of the present application. Among them, when the GNSS data is corrected by RTK technology to obtain positioning data, the error of the GNSS data is detected in real time by the error monitoring module. If the error is normal, high-precision positioning is performed through the GNSS data. If the error is abnormal, the initial data in the GNSS data is obtained, combined with the collected continuous image frames, the first positioning data is obtained, and the first positioning data is corrected based on the RTK technology. Furthermore, the initial data is combined with the inertial data to obtain the second positioning data, and the corrected first positioning data and the second positioning data are fused to obtain high-precision positioning data.

[0115] In some embodiments, when the GNSS data is restored to its original accuracy, that is, when the error no longer reaches the error condition, the positioning is switched to the GNSS data. In the first positioning mode, if the error of the GNSS data collected in real time no longer reaches the error condition, the first positioning mode is switched to the second positioning mode, and the first positioning mode refers to a mode for obtaining the real-time target positioning data of the vehicle based on the image frame; in the second positioning mode, the real-time dynamic carrier phase difference technology is used to correct the real-time GNSS data to obtain the real-time target positioning data of the vehicle.

[0116] In this embodiment, positioning is performed based on the GNSS data after the GNSS data recovers its accuracy, thus avoiding the problem of large cumulative positioning errors caused by long-term use of image frames for positioning.

[0117] In the embodiment of the present application, after the global navigation satellite system data recovers its accuracy, positioning data obtained in a variety of ways can be integrated to obtain more accurate positioning data.

[0118] The first implementation method is that in the first positioning mode, if the error of the global navigation satellite system data collected in real time no longer meets the error condition, the first positioning mode is switched to the third positioning mode; in the third positioning mode, the real-time global navigation satellite system data is corrected by real-time dynamic carrier phase difference technology to obtain third positioning data; the third positioning data and the corrected first positioning data are integrated to obtain real-time target positioning data of the vehicle.

[0119] The second implementation method is that in the first positioning mode, if the error of the global navigation satellite system data collected in real time no longer meets the error condition, the first positioning mode is switched to the fourth positioning mode; in the fourth positioning mode, the real-time dynamic carrier phase difference technology is used to correct the real-time global navigation satellite system data to obtain the third positioning data; the third positioning data, the corrected first positioning data and the second positioning data are integrated to obtain the real-time target positioning data of the vehicle.

[0120] The third implementation method is that in the first positioning mode, if the error of the global navigation satellite system data collected in real time no longer meets the error condition, the first positioning mode is switched to the fifth positioning mode; in the fifth positioning mode, the real-time dynamic carrier phase difference technology is used to correct the real-time global navigation satellite system data to obtain the third positioning data; the third positioning data and the second positioning data are integrated to obtain the real-time target positioning data of the vehicle.

[0121] In the above-mentioned implementation methods, after the accuracy of the global navigation satellite system data is restored, the global navigation satellite system data and the positioning data obtained by other means are fused to obtain the final positioning data. In this way, the positioning data obtained by multiple means are fused to obtain the final positioning data, thereby further improving the accuracy and robustness of the positioning data.

[0122] In an embodiment of the present application, the vehicle controller can determine the target positioning data through different modules. These modules include a positioning monitoring module, a visual module, an inertial navigation module and a data fusion module. Among them, the positioning monitoring module is used to detect whether the error of the global navigation satellite system data reaches the error condition. If the error does not reach the error condition, the real-time dynamic carrier phase difference technology is used to correct the global navigation satellite system data to obtain real-time target positioning data. If the error reaches the error condition, it switches to the first positioning mode, obtains the image frame collected by the visual module, calculates the relative motion information of the vehicle through the image frame of a certain time window, and calculates the latest first positioning data with the initial data. At the same time, the inertial navigation module is run to calculate the relative motion information of the vehicle and calculate the latest second positioning data. The two positioning data obtained are fused through the fusion module to obtain the final target positioning data.

[0123] It should be noted that since long-term use of image frames and inertial data to obtain positioning data will produce a large cumulative error, the duration of the first positioning mode is limited each time. That is, if the duration of the first positioning mode exceeds the duration threshold or the distance traveled by the vehicle in the first positioning mode reaches the distance threshold, it will switch back to the second positioning mode.

[0124] In the embodiment of the present application, positioning is performed through image frames, which can provide stable short-term positioning when the global navigation satellite system signal is blocked or lost, and can be used to solve the high-precision positioning problem caused by inaccurate positioning of global navigation satellite system data in complex environments. In addition, the solution can process and fuse multiple sensor data in real time to ensure the real-time and reliability of positioning.

[0125] In the embodiment of the present application, when the positioning error of the vehicle based on the global navigation satellite system data is large, the real-time relative motion information of the vehicle is determined based on the image frames collected in the driving environment, and the last accurate global navigation satellite system data is used as the initial data, and the initial data and the relative motion information are combined to obtain the real-time positioning data of the vehicle. Since the relative motion information of the vehicle depends on the environmental characteristics captured by the camera device, it is not affected by satellite signals or interference. The relative motion information determined in this way is accurate, and then combined with the initial data provided by the global navigation satellite system data, accurate positioning data can be obtained, reducing the overall positioning error, and achieving accurate positioning even when the error of the global navigation satellite system data is large. In addition, the real-time dynamic carrier phase difference technology is also used to correct the positioning data. Since the real-time dynamic carrier phase difference technology can eliminate the orbit error, clock error and most of the ionospheric error and tropospheric error of the total satellite of the positioning data, the accuracy and precision of the vehicle's positioning data can be further improved.

[0126] Please refer to Figure 4 , which shows a block diagram of a vehicle positioning device in a complex environment shown in an exemplary embodiment of the present application. The device includes:

[0127] The acquisition module 401 is used to acquire real-time global navigation satellite system data of the vehicle through the positioning device on the vehicle during the driving process of the vehicle, and continuously acquire image frames in the driving environment through the camera device on the vehicle;

[0128] A determination module 402 is used to determine initial data when the error of the global navigation satellite system data reaches the error condition, where the initial data refers to the global navigation satellite system data whose error does not reach the error condition and whose collection time is the latest among the global navigation satellite system data collected at multiple time points;

[0129] The determination module 402 is further used to determine the real-time relative motion information of the vehicle based on the image frames respectively collected at multiple time points, where the relative motion information includes at least one of a moving distance and a rotation angle;

[0130] The determination module 402 is further used to determine the real-time first positioning data of the vehicle based on the initial data and the relative motion information;

[0131] The correction module 403 is further used to correct the first positioning data by adopting real-time dynamic carrier phase difference technology to obtain real-time target positioning data of the vehicle.

[0132] In some embodiments, the acquisition module 401 is further used to:

[0133] During the driving of the vehicle, inertial data of the vehicle during the driving of the vehicle is collected by an inertial navigation device on the vehicle, wherein the inertial data includes at least one of real-time acceleration data and angular velocity data of the vehicle;

[0134] The determination module 402 is further used to determine the real-time second positioning data of the vehicle based on the inertial data;

[0135] The correction module 403 is used to:

[0136] The real-time dynamic carrier phase difference technology is used to correct the first positioning data, and the second positioning data and the corrected first positioning data are integrated to obtain the real-time target positioning data of the vehicle.

[0137] In some embodiments, the determination module 402 is configured to:

[0138] The real-time dynamic carrier phase differential technology is used to correct the initial data, and the second positioning data is obtained based on the corrected initial data and inertial data;

[0139] Alternatively, based on the initial data and the inertial data, the real-time third positioning data of the vehicle is obtained, and the real-time dynamic carrier phase difference technology is used to correct the third positioning data to obtain the second positioning data.

[0140] In some embodiments, the correction module 403 is used to:

[0141] Determine the average of the second positioning data and the corrected first positioning data as the target positioning data; or,

[0142] The second positioning data and the corrected first positioning data are weighted and summed to obtain the target positioning data.

[0143] In some embodiments, reaching the error condition includes at least one of the following:

[0144] The signal-to-noise ratio of the global navigation satellite system data is less than a first threshold;

[0145] The position error at the current moment determined based on the global navigation satellite system data reaches a second threshold;

[0146] The speed error at the current moment determined based on the global navigation satellite system data reaches a third threshold;

[0147] The number of satellites generating global navigation satellite system data is less than a fourth threshold.

[0148] In some embodiments, the apparatus further comprises a switching module, configured to:

[0149] In a first positioning mode for acquiring real-time target positioning data of the vehicle based on image frames, if the error of the global navigation satellite system data collected in real time no longer meets the error condition, the first positioning mode is switched to a second positioning mode;

[0150] The correction module 403 is also used to correct the real-time global navigation satellite system data in the second positioning mode by using the real-time dynamic carrier phase difference technology to obtain the real-time target positioning data of the vehicle.

[0151] In some embodiments, the determination module 402 is configured to:

[0152] Extracting feature points in each image frame, where the feature points include at least one of an edge and a corner point in the image frame;

[0153] Feature matching is performed on adjacent image frames based on feature points, and the real-time relative motion information of the vehicle is determined based on the matching results of multiple image frames.

[0154] In an embodiment of the present application, when the positioning error of the vehicle based on the global navigation satellite system data is large, the real-time relative motion information of the vehicle is determined based on the collected image frames in the driving environment, and the last accurate global navigation satellite system data is used as the initial data. The initial data and the relative motion information are combined to obtain the real-time positioning data of the vehicle. Since the relative motion information of the vehicle depends on the environmental characteristics captured by the camera equipment, it is not affected by satellite signals or interference. The relative motion information determined in this way is accurate. Then, combined with the initial data provided by the global navigation satellite system data, accurate positioning data can be obtained to reduce the overall positioning error. In addition, real-time dynamic carrier phase difference technology is used to correct the positioning data. Since the real-time dynamic carrier phase difference technology can eliminate the orbit error, clock error and most of the ionospheric error and tropospheric error of the total satellite of the positioning data, the accuracy and precision of the vehicle's positioning data can be further improved.

[0155] It should be noted that the vehicle positioning device under a complex environment provided by the above embodiment only uses the division of the above functional modules as an example when controlling the vehicle. In actual applications, the above functional distribution can be completed by different functional modules as needed, that is, the internal structure of the vehicle controller is divided into different functional modules to complete all or part of the functions described above. In addition, the vehicle positioning device under a complex environment provided by the above embodiment and the vehicle positioning method embodiment under a complex environment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0156] Typically, the vehicle controller 500 includes: a main control module 501, a CAN interface 502, a hard-wire input interface 503, and a hard-wire output interface 504. The main control module 501 is connected to the CAN interface 502, the hard-wire input interface 503, and the hard-wire output interface 504, respectively.

[0157] The main control module 501 generally includes a processor and a memory. Among them, the processor may include one or more processing cores, such as a 4-core processor, a 5-core processor, etc. The processor may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the vehicle display screen. In some embodiments, the processor may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning. The memory may include one or more computer-readable storage media, which may be non-transitory. The memory may also include a high-speed random access memory and a non-volatile memory, such as one or more disk storage devices and flash storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory is used to store at least one computer program, and the at least one computer program is used to be executed by the processor to implement the vehicle positioning method in a complex environment provided by the method embodiment of the present application.

[0158] The CAN interface 502 may include a power CAN interface, a motor CAN interface, and a diagnostic CAN interface, wherein the power CAN interface is used to communicate with a power system module of the vehicle, the motor CAN interface is used to communicate with a motor controller of the vehicle, and the diagnostic CAN interface is used to communicate with a diagnostic device.

[0159] The hard-wire input interface 503 is used to receive hard-wire control signals. The hard-wire output interface 504 is used to send control instructions to the electronic control components of the vehicle so that the electronic control components of the vehicle perform corresponding actions. The electronic control components of the vehicle include a power management system, a motor controller, an on-board charger, a body control system, etc.

[0160] The main control module 501 can communicate with the vehicle's power system module, motor controller and diagnostic equipment through the CAN interface 502, and generate control instructions based on the hard-wired control signal received by the hard-wired input interface 503 to send control instructions to the vehicle's electronic control components through the hard-wired output interface 504.

[0161] Those skilled in the art will understand that Figure 5 The structure shown in the figure does not constitute a limitation on the vehicle controller 500, and may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0162] The embodiment of the present application also provides a computer-readable storage medium, in which at least one program code is stored, and the at least one program code is loaded and executed by a processor to implement the vehicle positioning method in a complex environment described in any of the above implementations. Optionally, the storage medium can be a non-temporary computer-readable storage medium, for example, the non-temporary computer-readable storage medium can be a ROM (Read-Only Memory), a RAM (Random Access Memory), a CD-ROM (Compact Disc Read-Only Memory), a magnetic tape, a floppy disk, and an optical data storage device.

[0163] An embodiment of the present application also provides a computer program product, which stores at least one program code, and the at least one program code is loaded and executed by a processor to implement the vehicle positioning method in a complex environment as shown in the above embodiments.

[0164] In some embodiments, the computer program product involved in the embodiments of the present application may be deployed and executed on a vehicle controller, or on multiple vehicle controllers located at one location, or on multiple vehicle controllers distributed at multiple locations and interconnected through a communication network. Multiple vehicle controllers distributed at multiple locations and interconnected through a communication network can form a blockchain system.

[0165] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0166] The above description is only for the purpose of facilitating those skilled in the art to understand the technical solution of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A vehicle positioning method in a complex environment, characterized in that: The method comprises: During the driving process of the vehicle, real-time global navigation satellite system data of the vehicle is collected by a positioning device on the vehicle, and image frames of the driving environment are continuously collected by a camera device on the vehicle; In the case where the error of the global navigation satellite system data reaches the error condition, determining initial data, the initial data referring to the global navigation satellite system data whose error does not reach the error condition and whose collection time is the latest among the global navigation satellite system data collected at multiple time points; Determine the real-time relative motion information of the vehicle based on the image frames respectively collected at multiple time points, wherein the relative motion information includes at least one of a moving distance and a rotation angle; Determining real-time first positioning data of the vehicle based on the initial data and the relative motion information; The first positioning data is corrected by using real-time dynamic carrier phase difference technology to obtain real-time target positioning data of the vehicle.

2. The method according to claim 1, characterized in that The method further comprises: During the driving of the vehicle, inertial data of the vehicle during the driving is collected by an inertial navigation device on the vehicle, wherein the inertial data includes at least one of real-time acceleration data and angular velocity data of the vehicle; Determining real-time second positioning data of the vehicle based on the initial data and the inertial data; The method of using the real-time dynamic carrier phase difference technology to correct the first positioning data to obtain the real-time target positioning data of the vehicle includes: The first positioning data is corrected by using real-time dynamic carrier phase difference technology, and the second positioning data and the corrected first positioning data are fused to obtain real-time target positioning data of the vehicle.

3. The method according to claim 2, characterized in that The step of determining the real-time second positioning data of the vehicle based on the initial data and the inertial data comprises: Using real-time dynamic carrier phase difference technology to correct the initial data, and obtaining the second positioning data based on the corrected initial data and the inertial data; Alternatively, based on the initial data and the inertial data, a real-time third positioning data of the vehicle is obtained, and the third positioning data is corrected by using a real-time dynamic carrier phase difference technology to obtain the second positioning data.

4. The method according to claim 2, characterized in that: The fusing of the second positioning data and the corrected first positioning data to obtain real-time target positioning data of the vehicle includes: Determine the average of the second positioning data and the corrected first positioning data as the target positioning data; or, The second positioning data and the corrected first positioning data are weighted and summed to obtain the target positioning data.

5. The method according to claim 1, characterized in that The reaching error condition includes at least one of the following: The signal-to-noise ratio of the global navigation satellite system data is less than a first threshold; The position error at the current moment determined based on the global navigation satellite system data reaches a second threshold; The speed error at the current moment determined based on the global navigation satellite system data reaches a third threshold; The number of satellites generating the global navigation satellite system data is less than a fourth threshold.

6. The method according to claim 1, characterized in that The method further comprises: In the first positioning mode, if the error of the global navigation satellite system data collected in real time no longer meets the error condition, the first positioning mode is switched to the second positioning mode, wherein the first positioning mode refers to a mode of acquiring the real-time target positioning data of the vehicle based on the image frame; In the second positioning mode, the real-time dynamic carrier phase difference technology is used to correct the real-time global navigation satellite system data to obtain the real-time target positioning data of the vehicle.

7. The method according to claim 1, characterized in that The determining of the real-time relative motion information of the vehicle based on the image frames respectively collected at multiple time points includes: Extracting feature points in each image frame, wherein the feature points include at least one of an edge and a corner point in the image frame; Feature matching is performed on adjacent image frames based on feature points, and real-time relative motion information of the vehicle is determined based on matching results of multiple image frames.

8. A vehicle positioning device in a complex environment, characterized in that: The device comprises: A collection module, used for collecting real-time global navigation satellite system data of the vehicle through a positioning device on the vehicle during the driving process of the vehicle, and continuously collecting image frames in the driving environment through a camera device on the vehicle; a determination module, configured to determine initial data when an error of the global navigation satellite system data reaches an error condition, the initial data referring to the global navigation satellite system data whose error does not reach the error condition and whose collection time is the latest among the global navigation satellite system data collected at multiple time points; The determination module is further configured to determine the real-time relative motion information of the vehicle based on the image frames respectively acquired at multiple time points, wherein the relative motion information includes at least one of a moving distance and a rotation angle; The determination module is further used to determine the real-time first positioning data of the vehicle based on the initial data and the relative motion information; The correction module is also used to adopt real-time dynamic carrier phase difference technology to correct the first positioning data to obtain real-time target positioning data of the vehicle.

9. A vehicle controller, characterized in that: The vehicle controller includes a processor and a memory, wherein the memory stores at least one program code, and the at least one program code is loaded and executed by the processor to implement the vehicle positioning method in a complex environment as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The storage medium stores at least one program code, and the at least one program code is loaded and executed by the processor to implement the vehicle positioning method in a complex environment as described in any one of claims 1 to 7.

Images