Fusion positioning method and device suitable for low-frequency and high-delay visual perception data, equipment and storage medium

Abstract

The invention discloses a fusion positioning method and device suitable for low-frequency and high-delay visual perception data, equipment and a storage medium, and relates to the technical field of visual fusion. Obtaining first perception information of the vehicle at the first moment according to the first visual acquisition data and high-precision map data, wherein the first moment is before the second moment; obtaining track data according to inertial measurement data, wheel speed data and the vehicle position information at the first moment; determining positioning data in visual perception data according to the first moment and the second moment; and performing fusion positioning according to the positioning data, the track data, the high-precision map data and the first sensing information to obtain a target fusion result, thereby solving the problems of low frequency and high delay in a visual fusion process under the condition of not changing hardware, reducing hardware cost and improving processing efficiency.

Description

Technical Field

[0001] The present application relates to the field of visual fusion technology, and in particular to a fusion positioning method, device, equipment and storage medium that are adaptable to low-frequency and high-latency visual perception data. Background Art

[0002] With the continuous improvement of the level of intelligent driving, the importance of high-precision positioning for high-level intelligent driving has become increasingly prominent. In the actual positioning algorithm development process, the method of multi-source data fusion positioning is basically adopted to obtain stable high-precision positioning. At this stage, the commonly used low-cost fusion positioning solutions are mainly the fusion positioning of visual perception data, high-precision map data, RTK data, IMU data and wheel speed data. However, since the main body is the visual fusion positioning method, and the fusion positioning results often require high-frequency output, once the visual perception data has low frequency and high latency problems, it will seriously affect the final real-time fusion positioning accuracy. When the actual vehicle is installed, due to the low output frequency of the camera, insufficient computing power of the platform, too long processing time for perception data, communication delay and other problems, the visual perception data often has a very low output frequency and obvious data delay. In order to cope with the low frequency and high latency of visual perception data, it is often necessary to switch to high-performance cameras, optimize the time consumption of perception data processing, switch to high-performance platforms, optimize communication methods, etc., but the above methods will lead to a large consumption of human resources and an increase in component costs.

[0003] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0004] The main purpose of this application is to provide a fusion positioning method, device, equipment and storage medium that are adaptable to low-frequency and high-latency visual perception data, aiming to solve the technical problems of greatly increased hardware cost and high human resource consumption of positioning data in the prior art.

[0005] To achieve the above purpose, the present application proposes a fusion positioning method adapted to low-frequency and high-latency visual perception data, the fusion positioning method adapted to low-frequency and high-latency visual perception data includes:

[0006] When receiving the first visual perception data collected at the first moment at the second moment, obtaining first perception information of the vehicle at the first moment according to the first visual acquisition data and the high-precision map data;

[0007] Obtaining track data according to the inertial measurement data, the wheel speed data and the vehicle position information at the first moment;

[0008] Determine positioning data in the visual perception data according to the first moment and the second moment;

[0009] Fused positioning is performed based on the positioning data, the track data, the high-precision map data and the first perception information to obtain a target fusion result.

[0010] In one embodiment, when the first visual perception data collected at the first moment is received at the second moment, the step of obtaining the first perception information of the vehicle at the first moment according to the first visual collection data and the high-precision map data includes:

[0011] When receiving the first visual perception data collected at the first moment at the second moment, parsing the first visual acquisition data to obtain lane line type information, lane line color information, and lane width information;

[0012] Determining a vehicle position of the vehicle at the first moment according to the first visual perception data;

[0013] According to the matching of the vehicle position at the first moment with the high-precision map data, map information corresponding to the vehicle position is obtained;

[0014] The first perception information at the first moment is obtained by matching the map information corresponding to the vehicle position with the lane line type information, the lane line color information and the lane width information.

[0015] In one embodiment, the step of obtaining the track data according to the inertial measurement data, the wheel speed data and the vehicle position information at the first moment comprises:

[0016] determining acceleration information of the vehicle based on the inertial measurement data;

[0017] Obtaining motion information of the vehicle according to the acceleration information and the wheel speed information;

[0018] Track data is obtained according to the motion information and the vehicle position information at the first moment.

[0019] In one embodiment, the step of determining the positioning data in the visual perception data according to the first moment and the second moment comprises:

[0020] Traversing the timestamps in the real-time dynamic differential positioning cache, and acquiring the real-time dynamic differential positioning cache data corresponding to the first moment and the second moment;

[0021] Performing interpolation processing on the real-time dynamic differential positioning cache data to obtain a real-time dynamic differential positioning result;

[0022] The real-time dynamic differential positioning result and the first perception information are converted into a coordinate system to obtain positioning data in the visual perception data in the world coordinate system.

[0023] In one embodiment, before the step of performing interpolation processing on the real-time dynamic differential positioning cache data to obtain the real-time dynamic differential positioning result, the step further includes:

[0024] Detecting the real-time dynamic differential positioning cache data corresponding to the second moment to obtain a detection result;

[0025] When the detection result is that the real-time dynamic differential positioning cache data corresponding to the second moment is empty, determining the time interval between the current time and the first moment;

[0026] When the time interval is less than a preset time threshold, using the corresponding real-time dynamic differential positioning cache data at the first moment as positioning data;

[0027] When the detection result is that the real-time dynamic differential positioning cache data corresponding to the second moment is not empty, the step of performing interpolation processing on the real-time dynamic differential positioning cache data to obtain the real-time dynamic differential positioning result is performed.

[0028] In one embodiment, the step of performing fusion positioning according to the positioning data, the track data, the high-precision map data and the first perception information to obtain a target fusion result includes:

[0029] Determine the timestamp information of the track data, and match the corresponding target timestamp information according to the first moment and the second moment;

[0030] Interpolate the positioning data based on the target timestamp information to obtain an interpolation positioning result;

[0031] A fusion positioning is performed according to the interpolation positioning result, the high-precision map data and the first perception information to obtain a target fusion result.

[0032] In one embodiment, the step of performing fusion positioning according to the interpolation positioning result, the high-precision map data and the first perception information to obtain a target fusion result includes:

[0033] Generate a world coordinate system map based on the high-precision map data;

[0034] Obtaining an initial prediction result according to the generated world coordinate system map, the first perception information and the interpolation positioning result;

[0035] Determine the last frame positioning result at the current moment according to the initial prediction result;

[0036] The target fusion result is obtained by performing fusion positioning according to the positioning result of the previous frame and the track information.

[0037] In addition, to achieve the above purpose, the present application also proposes a fusion positioning device adapted to low-frequency, high-latency visual perception data, the fusion positioning device adapted to low-frequency, high-latency visual perception data comprises:

[0038] a visual perception module, configured to, when receiving at a second moment first visual perception data collected at a first moment, obtain first perception information of the vehicle at the first moment according to the first visual collection data and high-precision map data, wherein the first moment is before the second moment;

[0039] A track determination module, used to obtain track data according to the inertial measurement data, the wheel speed data and the vehicle position information at the first moment;

[0040] A differential positioning module, used to determine positioning data in the visual perception data according to the first moment and the second moment;

[0041] The visual fusion module is used to perform fusion positioning according to the positioning data, the track data, the high-precision map data and the first perception information to obtain a target fusion result.

[0042] In addition, to achieve the above-mentioned objectives, the present application also proposes a fusion positioning device that adapts to low-frequency, high-latency visual perception data, the device comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the fusion positioning method that adapts to low-frequency, high-latency visual perception data as described above.

[0043] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the fusion positioning method that adapts to low-frequency and high-latency visual perception data as described above are implemented.

[0044] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the fusion positioning method adapted to low-frequency and high-latency visual perception data as described above.

[0045] One or more technical solutions proposed in the present application have at least the following technical effects: when the first visual perception data collected at the first moment is received at the second moment, the first perception information of the vehicle at the first moment is obtained according to the first visual collection data and the high-precision map data, and the first moment is before the second moment; the track data is obtained according to the inertial measurement data, the wheel speed data and the vehicle position information at the first moment; the positioning data in the visual perception data is determined according to the first moment and the second moment; the fusion positioning is performed according to the positioning data, the track data, the high-precision map data and the first perception information to obtain the target fusion result, so as to solve the low frequency and high delay problems in the visual fusion process without changing the hardware, reduce hardware costs and improve processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0047] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0048] Figure 1 A flow chart of Embodiment 1 of the fusion positioning method for low-frequency and high-latency visual perception data provided by the present application;

[0049] Figure 2 An actual vehicle driving schematic diagram provided for an embodiment of a fusion positioning method for low-frequency and high-latency visual perception data in this application;

[0050] Figure 3 A block diagram of a visual fusion solution provided for an embodiment of a fusion positioning method for low-frequency and high-latency visual perception data of the present application;

[0051] Figure 4 This is a schematic diagram of the module structure of a fusion positioning device that is adapted to low-frequency and high-latency visual perception data according to an embodiment of the present application;

[0052] Figure 5 This is a schematic diagram of the device structure of the hardware operating environment involved in the fusion positioning method that adapts to low-frequency and high-latency visual perception data in the embodiment of the present application.

[0053] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0054] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0055] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0056] The main solution of the embodiment of the present application is: when the first visual perception data collected at the first moment is received at the second moment, the first perception information of the vehicle at the first moment is obtained according to the first visual collection data and the high-precision map data; the track data is obtained according to the inertial measurement data, the wheel speed data and the vehicle position information at the first moment; the positioning data in the visual perception data is determined according to the first moment and the second moment; and the positioning is performed based on the positioning data, the track data, the high-precision map data and the first perception information to obtain a target fusion result.

[0057] In this embodiment, for ease of description, the following description is made by taking the fusion positioning device that recognizes and adapts to low-frequency and high-latency visual perception data as the execution subject.

[0058] In order to deal with the low frequency and high latency problems of visual perception data, existing technologies often need to switch to high-performance cameras, optimize the time consumption of perception data processing, switch to high-performance platforms, optimize communication methods, etc. However, all of the above methods will lead to greater human resource consumption and increased component costs.

[0059] This application provides a solution that does not rely on high-computing power platforms or high-performance and high-frame rate cameras, can effectively reduce hardware costs, and does not require forced optimization of visual perception algorithm time consumption and optimization of communication delay issues. It can effectively avoid increases in labor costs and time costs, and can effectively avoid the impact of positioning errors caused by low frequency and high delay of visual perception data, and can effectively improve positioning accuracy.

[0060] It can be seen from the above embodiments that the present application obtains the first perception information of the vehicle at the first moment according to the first visual acquisition data and the high-precision map data when the first visual perception data collected at the first moment is received at the second moment, and the first moment is before the second moment; obtains the track data according to the inertial measurement data, the wheel speed data and the vehicle position information at the first moment; determines the positioning data in the visual perception data according to the first moment and the second moment; performs fusion positioning according to the positioning data, the track data, the high-precision map data and the first perception information to obtain the target fusion result, so as to solve the low frequency and high delay problems in the visual fusion process without changing the hardware, reduce hardware costs and improve processing efficiency.

[0061] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a fusion positioning device adapted to low-frequency, high-latency visual perception data, etc. The following takes a fusion positioning device adapted to low-frequency, high-latency visual perception data as an example to illustrate this embodiment and the following embodiments.

[0062] Based on this, the embodiment of the present application provides a fusion positioning method adapted to low-frequency and high-latency visual perception data, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the fusion positioning method for low-frequency and high-latency visual perception data adapted to the present application.

[0063] In this embodiment, the fusion positioning method adapted to low-frequency and high-latency visual perception data includes steps S10 to S40:

[0064] Step S10, when the first visual perception data collected at the first moment is received at the second moment, the first perception information of the vehicle at the first moment is obtained according to the first visual collection data and the high-precision map data, and the first moment is before the second moment.

[0065] It should be noted that the first moment and the second moment are the moment information when visual perception is performed, wherein the first moment is before the second moment, and the visual perception data refers to the data obtained and processed by visual perception sensors, such as cameras, radars, etc. At the same time, it can also be understood that the first perception information is the visual perception information obtained at the first moment, and the high-precision map data (HDMap data) refers to the vehicle location information input into the map engine, receiving the HDMap data, and using the EHR (Electric Horizon Reconstructor) module to reconstruct the data to obtain the data.

[0066] In a specific implementation, the vehicle will continuously obtain the road information in front of the vehicle through the visual perception sensor during driving, and cache the visual perception information and add the corresponding timestamp for subsequent visual data fusion processing. During vehicle driving, if the visual perception data is obtained at the first moment, due to the certain data processing time, when the visual perception information obtained at the first moment is received, it is the second moment. At this time, the position of the vehicle has changed from the position at the first moment. Therefore, it can be understood that the first visual perception data collected at the first moment is received at the second moment. When the first visual perception data is obtained at the second moment, it can be roughly matched with the high-precision map data (HDMap, High Definition Map) to obtain the first perception information. Figure 2, Figure 2 The figure is a schematic diagram of actual vehicle driving. The high latency problem during vehicle driving can be understood as the lane line data recognized by the vehicle at time t0. At this time, the distance between the vehicle and the left lane line is C0. Due to the latency problem, the fusion positioning module actually receives the perception and recognition lane line data at time t1. At this time, the vehicle position has changed, and the distance from the left lane line is C0'. If the perception lane line data is directly used to correct the vehicle position, there will be lateral positioning errors. The low-frequency problem can be understood as: the vehicle has no visual perception data input during the output time period of two adjacent frames of visual perception data. At this time, the fusion positioning can only rely on RTK, IMU, wheel speed and other data for positioning. In actual situations, since RTK provides unbiased world coordinates, high-precision map data is biased coordinates and the high-precision map itself will also have data errors, resulting in the absence of perception data. The vehicle position located by RTK often does not match the vehicle position in the map, and may even be in the wrong lane. Moreover, when the network signal is poor, RTK services are often unavailable, and further tracking can only rely on IMU and wheel speed data. Once the time is too long, track calculation will diverge, resulting in inaccurate positioning.

[0067] Therefore, at time t1, when the fusion positioning module receives the perceived lane line data at time t0, it can first obtain the vehicle position at time t0, and use the vehicle position at time t0 to convert the perceived lane line data at time t0 to the world coordinate system. At this time, even if the vehicle position at time t1 has changed, since the perceived lane line data is already in the world coordinate system, the distance from the vehicle to the left line of the lane can be recalculated based on the vehicle position at time t1, thereby ensuring that the time delay of the perception data will not affect it. In the time period of two adjacent frames of perception data, if the set time threshold is not exceeded, the previous frame of perception data will continue to be used, thereby ensuring that the high-frequency positioning results continue to be fused and corrected by perception data and map data.

[0068] In a feasible implementation manner, when the first visual perception data collected at the first moment is received at the second moment, the step of obtaining the first perception information of the vehicle at the first moment according to the first visual collection data and the high-precision map data includes:

[0069] When receiving the first visual perception data collected at the first moment at the second moment, parsing the first visual acquisition data to obtain lane line type information, lane line color information, and lane width information;

[0070] Determining a vehicle position of the vehicle at the first moment according to the first visual perception data;

[0071] According to the matching of the vehicle position at the first moment with the high-precision map data, map information corresponding to the vehicle position is obtained;

[0072] The first perception information at the first moment is obtained by matching the map information corresponding to the vehicle position with the lane line type information, the lane line color information and the lane width information.

[0073] It should be noted that high-precision map data includes map positioning data, road information, etc., which can reflect road information and geographical feature information within a certain area. Map information refers to high-precision map information of a certain area corresponding to the current position of the vehicle.

[0074] In a specific implementation, when the first visual perception data collected at the first moment is received at the second moment, the first visually collected data can be parsed to obtain the lane line type information, lane line color information and lane width information. The road information of the road condition ahead can be determined by image recognition when the current vehicle is driving, such as whether the lane line is solid or dotted, single or double, white or yellow, or other colors, and how wide the lane is, which is obtained based on the vertical interval between the two lane lines of the current lane. In addition, the diamond marks or crosswalks on the lane can be identified and roughly matched with the lane information in the high-precision map information to obtain rough matching data, and the rough matching data is used as the first perception information at the first moment.

[0075] Step S20, obtaining track data according to the inertial measurement data, the wheel speed data and the vehicle position information at the first moment.

[0076] It should be noted that the track data refers to the inertial measurement data measured by the inertial measurement unit (IMU), the vehicle speed data obtained by the vehicle wheel speed sensor, and the high-frequency track data obtained by dead reckoning (DR), which is used to reflect the vehicle's motion trajectory from the first moment to the second moment.

[0077] In a specific implementation, the process of obtaining track data according to inertial measurement data, wheel speed data and the vehicle position information at the first moment can be described as: determining the acceleration information of the vehicle according to the inertial measurement data; obtaining the motion information of the vehicle according to the acceleration information and the wheel speed information; obtaining the track data according to the motion information and the vehicle position information at the first moment. Specifically, the inertial measurement unit (IMU) can measure and report the three basic linear motions (acceleration) and three basic angular motions (angular velocity) of the vehicle. The acceleration of the vehicle is measured by the accelerometer, and the acceleration is integrated to obtain the speed information. After integrating again, the position information of the vehicle can be obtained. According to the data output of the inertial measurement unit, the body change of each frame of the vehicle can be obtained. Through the angular rate signal collected by the gyroscope, the angle change of the vehicle can be determined, and the change of the heading angle can be obtained. Then, the wheel speed information provided by the ABS can be combined with the data items of the inertial measurement unit to obtain the track data of the vehicle, and the accuracy of the vehicle speed and position estimation can be improved.

[0078] Step S30: determining positioning data in the visual perception data according to the first moment and the second moment.

[0079] It should be noted that positioning data refers to the location information of the vehicle when the vehicle is performing visual perception data, which is used to reflect the location of obtaining visual perception information.

[0080] In a specific implementation, the positioning data is obtained by real-time kinematic differential positioning (RTK), wherein when the real-time kinematic differential positioning data is obtained, a cyclic data cache RTK_Buffer of a certain size can be created to cache the RTK positioning data results of a certain time length. The steps of determining the positioning data in the visual perception data according to the first moment and the second moment can be described as: traversing the timestamps in the real-time kinematic differential positioning cache, obtaining the real-time kinematic differential positioning cache data corresponding to the first moment and the second moment; interpolating the real-time kinematic differential positioning cache data to obtain the real-time kinematic differential positioning result; performing coordinate system conversion on the real-time kinematic differential positioning result and the first perception information to obtain the positioning data in the visual perception data in the world coordinate system.

[0081] In the specific implementation, the corresponding RTK positioning result is interpolated in the RTK_Buffer using the timestamp of the coarsely matched visual perception data, and the visually perceived lane linear points in the coarse matching result are converted to the world coordinate system using the RTK positioning result. The visual perception data converted to the world coordinate system is used to obtain the positioning data in the visual perception data in the world coordinate system, which can be used as the data input for subsequent fusion positioning.

[0082] In a feasible implementation manner, before the step of performing interpolation processing on the real-time dynamic differential positioning cache data to obtain the real-time dynamic differential positioning result, the step further includes:

[0083] Detecting the real-time dynamic differential positioning cache data corresponding to the second moment to obtain a detection result;

[0084] When the detection result is that the real-time dynamic differential positioning cache data corresponding to the second moment is empty, determining the time interval between the current time and the first moment;

[0085] When the time interval is less than a preset time threshold, using the corresponding real-time dynamic differential positioning cache data at the first moment as positioning data;

[0086] When the detection result is that the real-time dynamic differential positioning cache data corresponding to the second moment is not empty, the step of performing interpolation processing on the real-time dynamic differential positioning cache data to obtain the real-time dynamic differential positioning result is performed.

[0087] In a specific implementation, the real-time dynamic differential positioning cache data can be detected to obtain a detection result, which is used to determine whether there is new visual perception data input. When the detection result shows that the real-time dynamic differential positioning cache data corresponding to the second moment is empty, it means that there is no new visual perception data input at present. Therefore, the time interval between the first moment and the second moment can be determined. If the time interval is less than the preset time threshold δ, then the subsequent fusion positioning data input is continuously used for the coarse matching result position converted to the world coordinate system. If the detection result is that the real-time dynamic differential positioning cache data corresponding to the second moment is not empty, that is, there is new visual perception data input at present, then the step of interpolating the real-time dynamic differential positioning cache data can be executed to obtain the real-time dynamic differential positioning result.

[0088] Step S40: performing fusion positioning according to the positioning data, the track data, the high-precision map data and the first perception information to obtain a target fusion result.

[0089] It should be noted that the target fusion result refers to the visual fusion data obtained by fusion positioning based on the vehicle's positioning data, track data, high-precision map data and the perception information at the first moment.

[0090] In the specific implementation, refer to Figure 3 , Figure 3The block diagram of the visual fusion scheme. By determining the timestamp information of the track data, the corresponding target timestamp information is matched according to the first moment and the second moment; the positioning data is interpolated based on the target timestamp information to obtain the interpolation positioning result; a world coordinate system map is generated based on the high-precision map data; the initial prediction result is obtained based on the generated world coordinate system map, the first perception information and the interpolation positioning result; the positioning result of the previous frame at the current moment is determined according to the initial prediction result; the fusion positioning is performed according to the positioning result of the previous frame and the track information to obtain the target fusion result. RTK data, DR data, HDmap data, and processed coarse matching perception results are used as inputs, and the input frequency of DR data is used as a reference. Multi-source fusion positioning is performed using a graph optimization algorithm, and the fusion positioning result is output at a high frequency. The graph optimization algorithm uses the previous frame positioning data and DR data to obtain the prediction result of the next frame, and further corrects the prediction result by using RTK data as the longitudinal positioning constraint and visual perception data as the lateral constraint to obtain the best fusion positioning result.

[0091] This embodiment provides a fusion positioning method that adapts to low-frequency and high-latency visual perception data. When first visual perception data collected at a first moment is received at a second moment, first perception information of the vehicle at the first moment is obtained according to the first visual collection data and high-precision map data, and the first moment is before the second moment; track data is obtained according to inertial measurement data, wheel speed data and vehicle position information at the first moment; positioning data in the visual perception data is determined according to the first moment and the second moment; fusion positioning is performed according to the positioning data, the track data, the high-precision map data and the first perception information to obtain a target fusion result, so as to solve the low-frequency and high-latency problems in the visual fusion process without changing the hardware, reduce hardware costs and improve processing efficiency.

[0092] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the fusion positioning method of the present application that is adapted to low-frequency, high-latency visual perception data. More forms of simple transformations based on this technical concept are all within the scope of protection of the present application.

[0093] This application also provides a fusion positioning device that adapts to low-frequency and high-latency visual perception data. Please refer to Figure 4 , the fusion positioning device adapted to low-frequency and high-latency visual perception data includes:

[0094] The visual perception module 10 is configured to, when receiving at a second moment first visual perception data collected at a first moment, obtain first perception information of the vehicle at the first moment according to the first visual collection data and high-precision map data, wherein the first moment is before the second moment;

[0095] A track determination module 20, configured to obtain track data based on inertial measurement data, wheel speed data and vehicle position information at the first moment;

[0096] A differential positioning module 30, configured to determine positioning data in the visual perception data according to the first moment and the second moment;

[0097] The visual fusion module 40 is used to perform fusion positioning according to the positioning data, the track data, the high-precision map data and the first perception information to obtain a target fusion result.

[0098] In one embodiment, the visual perception module 10 is further used to parse the first visual perception data collected at the first moment when the first visual perception data is received at the second moment to obtain lane line type information, lane line color information and lane width information; determine the vehicle position of the vehicle at the first moment according to the first visual perception data; match the vehicle position at the first moment with the high-precision map data to obtain map information corresponding to the vehicle position; match the map information corresponding to the vehicle position with the lane line type information, lane line color information and lane width information to obtain the first perception information at the first moment.

[0099] In one embodiment, the track determination module 20 is further used to determine the acceleration information of the vehicle based on the inertial measurement data; obtain the motion information of the vehicle based on the acceleration information and the wheel speed information; and obtain the track data based on the motion information and the vehicle position information at the first moment.

[0100] In one embodiment, the differential positioning module 30 is further used to traverse the timestamps in the real-time dynamic differential positioning cache to obtain the real-time dynamic differential positioning cache data corresponding to the first moment and the second moment; perform interpolation processing on the real-time dynamic differential positioning cache data to obtain a real-time dynamic differential positioning result; perform coordinate system conversion on the real-time dynamic differential positioning result and the first perception information to obtain positioning data in the visual perception data in the world coordinate system.

[0101] In one embodiment, the differential positioning module 30 is further used to detect the real-time dynamic differential positioning cache data corresponding to the second moment to obtain a detection result; when the detection result is that the real-time dynamic differential positioning cache data corresponding to the second moment is empty, determine the time interval between the current time and the first moment; when the time interval is less than a preset time threshold, use the real-time dynamic differential positioning cache data corresponding to the first moment as positioning data; when the detection result is that the real-time dynamic differential positioning cache data corresponding to the second moment is not empty, execute the step of interpolating the real-time dynamic differential positioning cache data to obtain a real-time dynamic differential positioning result.

[0102] In one embodiment, the visual fusion module 40 is also used to determine the timestamp information of the track data, match the corresponding target timestamp information according to the first moment and the second moment; interpolate the positioning data based on the target timestamp information to obtain an interpolated positioning result; perform fusion positioning based on the interpolated positioning result, the high-precision map data and the first perception information to obtain a target fusion result.

[0103] In one embodiment, the visual fusion module 40 is also used to generate a world coordinate system map based on the high-precision map data; obtain an initial prediction result based on the generated world coordinate system map, the first perception information and the interpolation positioning result; determine the previous frame positioning result at the current moment based on the initial prediction result; perform fusion positioning based on the previous frame positioning result and the track information to obtain a target fusion result.

[0104] The fusion positioning device adapted to low-frequency and high-delay visual perception data provided by the present application adopts the fusion positioning method adapted to low-frequency and high-delay visual perception data in the above-mentioned embodiment, which can solve the technical problems of greatly increased hardware cost and high human resource consumption of positioning data in the prior art. Compared with the prior art, the beneficial effects of the fusion positioning device adapted to low-frequency and high-delay visual perception data provided by the present application are the same as the beneficial effects of the fusion positioning method adapted to low-frequency and high-delay visual perception data provided by the above-mentioned embodiment, and the other technical features of the fusion positioning device adapted to low-frequency and high-delay visual perception data are the same as the features disclosed in the above-mentioned embodiment method, which will not be repeated here.

[0105] The present application provides a fusion positioning device that adapts to low-frequency, high-latency visual perception data. The fusion positioning device that adapts to low-frequency, high-latency visual perception data includes: at least one processor; and a memory that is communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the fusion positioning method that adapts to low-frequency, high-latency visual perception data in the above-mentioned embodiment one.

[0106] Reference below Figure 5 , which shows a schematic diagram of the structure of a fusion positioning device that is suitable for implementing low-frequency, high-latency visual perception data in the embodiment of the present application. The fusion positioning device that is suitable for low-frequency, high-latency visual perception data in the embodiment of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The fusion positioning device adapted to low-frequency, high-latency visual perception data shown is merely an example and should not bring any limitation to the functions and scope of application of the embodiments of the present application.

[0107] like Figure 5As shown, the fusion positioning device adapted to low-frequency, high-latency visual perception data may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the fusion positioning device adapted to low-frequency, high-latency visual perception data are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 can allow the fusion positioning device adapted to low-frequency, high-latency visual perception data to communicate wirelessly or wired with other devices to exchange data. Although the figure shows a fusion positioning device adapted to low-frequency, high-latency visual perception data with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have instead.

[0108] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0109] The fusion positioning device adapted to low-frequency and high-delay visual perception data provided by the present application adopts the fusion positioning method adapted to low-frequency and high-delay visual perception data in the above-mentioned embodiment, which can solve the technical problems of greatly increased hardware cost and high human resource consumption of the existing technology positioning data. Compared with the prior art, the beneficial effects of the fusion positioning device adapted to low-frequency and high-delay visual perception data provided by the present application are the same as the beneficial effects of the fusion positioning method adapted to low-frequency and high-delay visual perception data provided by the above-mentioned embodiment, and the other technical features of the fusion positioning device adapted to low-frequency and high-delay visual perception data are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0110] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0111] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0112] The present application provides a computer-readable storage medium having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the fusion positioning method adapted to low-frequency, high-latency visual perception data in the above-mentioned embodiment.

[0113] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0114] The above-mentioned computer-readable storage medium may be included in a fusion positioning device that adapts to low-frequency, high-latency visual perception data; or it may exist independently without being assembled into a fusion positioning device that adapts to low-frequency, high-latency visual perception data.

[0115] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by a fusion positioning device adapted to low-frequency, high-latency visual perception data, the fusion positioning device adapted to low-frequency, high-latency visual perception data:

[0116] When receiving the first visual perception data collected at the first moment at the second moment, obtaining first perception information of the vehicle at the first moment according to the first visual acquisition data and the high-precision map data;

[0117] Obtaining track data according to the inertial measurement data, the wheel speed data and the vehicle position information at the first moment;

[0118] Determine positioning data in the visual perception data according to the first moment and the second moment;

[0119] Fused positioning is performed based on the positioning data, the track data, the high-precision map data and the first perception information to obtain a target fusion result.

[0120] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0121] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0122] The modules involved in the embodiments of the present application may be implemented by software or hardware, wherein the name of the module does not limit the unit itself in some cases.

[0123] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned fusion positioning method adapted to low-frequency and high-latency visual perception data, which can solve the technical problems of greatly increased hardware costs and high human resource consumption of the existing technology positioning data. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the fusion positioning method adapted to low-frequency and high-latency visual perception data provided by the above-mentioned embodiments, and will not be repeated here.

[0124] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the above-mentioned fusion positioning method adapted to low-frequency, high-latency visual perception data.

[0125] The computer program product provided by this application can solve the technical problems of greatly increased hardware cost and high human resource consumption of the existing technology positioning data. Compared with the existing technology, the beneficial effects of the computer program product provided by this application are the same as the beneficial effects of the fusion positioning method adapted to low-frequency and high-latency visual perception data provided by the above embodiment, and will not be repeated here.

[0126] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A fusion positioning method adapted to low-frequency and high-latency visual perception data, characterized in that: The fusion positioning method adapted to low-frequency and high-latency visual perception data includes: When receiving the first visual perception data collected at the first moment at the second moment, obtaining the first perception information of the vehicle at the first moment according to the first visual acquisition data and the high-precision map data, the first moment being before the second moment; Obtaining track data according to the inertial measurement data, the wheel speed data and the vehicle position information at the first moment; Determine positioning data in the visual perception data according to the first moment and the second moment; Fused positioning is performed based on the positioning data, the track data, the high-precision map data and the first perception information to obtain a target fusion result.

2. The method according to claim 1, characterized in that When the first visual perception data collected at the first moment is received at the second moment, the first perception information of the vehicle at the first moment is obtained according to the first visual collection data and the high-precision map data, and the steps of the first moment before the second moment include: When receiving the first visual perception data collected at the first moment at the second moment, parsing the first visual acquisition data to obtain lane line type information, lane line color information, and lane width information; Determining a vehicle position of the vehicle at the first moment according to the first visual perception data; According to the matching of the vehicle position at the first moment with the high-precision map data, map information corresponding to the vehicle position is obtained; The first perception information at the first moment is obtained by matching the map information corresponding to the vehicle position with the lane line type information, the lane line color information and the lane width information.

3. The method according to claim 1, characterized in that The step of obtaining the track data according to the inertial measurement data, the wheel speed data and the vehicle position information at the first moment comprises: determining acceleration information of the vehicle based on the inertial measurement data; Obtaining motion information of the vehicle according to the acceleration information and the wheel speed information; Track data is obtained according to the motion information and the vehicle position information at the first moment.

4. The method according to claim 1, characterized in that The step of determining the positioning data in the visual perception data according to the first moment and the second moment comprises: Traversing the timestamps in the real-time dynamic differential positioning cache, and acquiring the real-time dynamic differential positioning cache data corresponding to the first moment and the second moment; Performing interpolation processing on the real-time dynamic differential positioning cache data to obtain a real-time dynamic differential positioning result; The real-time dynamic differential positioning result and the first perception information are converted into a coordinate system to obtain positioning data in the visual perception data in the world coordinate system.

5. The method according to claim 4, characterized in that Before the step of interpolating the real-time dynamic differential positioning cache data to obtain the real-time dynamic differential positioning result, the method further includes: Detecting the real-time dynamic differential positioning cache data corresponding to the second moment to obtain a detection result; When the detection result is that the real-time dynamic differential positioning cache data corresponding to the second moment is empty, determining the time interval between the current time and the first moment; When the time interval is less than a preset time threshold, using the corresponding real-time dynamic differential positioning cache data at the first moment as positioning data; When the detection result is that the real-time dynamic differential positioning cache data corresponding to the second moment is not empty, the step of performing interpolation processing on the real-time dynamic differential positioning cache data to obtain the real-time dynamic differential positioning result is performed.

6. The method according to claim 1, characterized in that The step of performing fusion positioning according to the positioning data, the track data, the high-precision map data and the first perception information to obtain a target fusion result includes: Determine the timestamp information of the track data, and match the corresponding target timestamp information according to the first moment and the second moment; Interpolate the positioning data based on the target timestamp information to obtain an interpolation positioning result; A fusion positioning is performed according to the interpolation positioning result, the high-precision map data and the first perception information to obtain a target fusion result.

7. The method according to claim 6, characterized in that The step of performing fusion positioning according to the interpolation positioning result, the high-precision map data and the first perception information to obtain a target fusion result comprises: Generate a world coordinate system map based on the high-precision map data; Obtaining an initial prediction result according to the generated world coordinate system map, the first perception information and the interpolation positioning result; Determine the last frame positioning result at the current moment according to the initial prediction result; The target fusion result is obtained by performing fusion positioning according to the positioning result of the previous frame and the track information.

8. A fusion positioning device adapted to low-frequency and high-latency visual perception data, characterized in that: The device comprises: a visual perception module, configured to obtain first perception information of the vehicle at the first moment according to the first visually acquired data and the high-precision map data when receiving first visually acquired data at the first moment at the second moment; A track determination module, used to obtain track data according to the inertial measurement data, the wheel speed data and the vehicle position information at the first moment; A differential positioning module, used to determine positioning data in the visual perception data according to the first moment and the second moment; The visual fusion module is used to perform fusion positioning according to the positioning data, the track data, the high-precision map data and the first perception information to obtain a target fusion result.

9. A fusion positioning device adapted to low-frequency and high-latency visual perception data, characterized in that: The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of a fusion positioning method adapted to low-frequency, high-latency visual perception data as described in any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the fusion positioning method adapted to low-frequency and high-latency visual perception data are implemented as described in any one of claims 1 to 7.

Images