Positioning method, device, electronic device and storage medium

Abstract

The embodiments of the present disclosure disclose a positioning method, device, electronic device and storage medium, the method comprising: when a vehicle is in a driving state, determining at least one posture of the vehicle by at least one positioning method in a set time window, wherein the postures determined by different positioning methods are different; performing a first correction on the initial posture obtained by the wheel speed odometer based on the target posture in at least one posture to obtain a first correction result; continuing to correct the first correction result based on the road information acquired in the set time window to obtain a second correction result, wherein the road information includes road signs, three-dimensional points constituting the road signs and acquisition timestamps of the three-dimensional points; obtaining the final positioning result of the vehicle based on the second correction result. The present disclosure can still achieve high-precision positioning on sections where GPS signals are blocked, sections with fewer structural features, or sections that change frequently.

Description

Technical Field

[0001] The present disclosure relates to the field of autonomous driving technology, and in particular to a positioning method, device, electronic device, and storage medium. Background Art

[0002] Positioning technology is one of the core technologies in the field of autonomous driving, and high-precision positioning is a prerequisite for achieving autonomous driving.

[0003] Therefore, how to improve positioning accuracy is a research topic that requires continuous efforts in the field of autonomous driving. Summary of the Invention

[0004] In order to solve the above technical problems or at least partially solve the above technical problems, the embodiments of the present disclosure provide a positioning method, device, electronic device and storage medium, which can still achieve high-precision positioning on sections of road where GPS signals are blocked, sections with fewer structural features, or sections that change frequently.

[0005] In a first aspect, an embodiment of the present disclosure provides a positioning method, the method comprising:

[0006] When the vehicle is in a driving state, determining at least one posture of the vehicle by using at least one positioning method within a set time window, wherein the postures determined by different positioning methods are different;

[0007] Performing a first correction on the initial posture obtained by the wheel speed odometer based on the target posture in the at least one posture to obtain a first correction result;

[0008] Continuing to correct the first correction result based on road information acquired in the set time window to obtain a second correction result, the road information including road signs, three-dimensional points constituting the road signs, and acquisition timestamps of the three-dimensional points;

[0009] A final positioning result of the vehicle is obtained based on the second correction result.

[0010] In a second aspect, an embodiment of the present disclosure further provides a positioning device, the device comprising:

[0011] a first determining module, configured to determine, when the vehicle is in a driving state, at least one posture of the vehicle by using at least one positioning method within a set time window, wherein the postures determined by different positioning methods are different;

[0012] a first correction module, configured to perform a first correction on an initial posture obtained by the wheel speed odometer based on a target posture in the at least one posture, to obtain a first correction result;

[0013] a second correction module, configured to further correct the first correction result based on road information acquired during the set time window to obtain a second correction result, wherein the road information includes road signs, three-dimensional points constituting the road signs, and acquisition timestamps of the three-dimensional points;

[0014] A second determining module is configured to obtain a final positioning result of the vehicle based on the second correction result.

[0015] In a third aspect, an embodiment of the present disclosure further provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs; and when the one or more programs are executed by the one or more processors, the one or more processors implement the positioning method as described above.

[0016] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, which implements the positioning method described above when executed by a processor.

[0017] A positioning method provided by an embodiment of the present disclosure determines at least one posture of the vehicle by at least one positioning method within a set time window when the vehicle is in motion, wherein the postures determined by different positioning methods are different; an initial posture obtained by a wheel speed odometer is corrected for the first time based on a target posture in the at least one posture to obtain a first correction result; the first correction result is further corrected based on road information acquired in the set time window to obtain a second correction result, wherein the road information includes road signs, three-dimensional points constituting the road signs, and acquisition timestamps of the three-dimensional points; a final positioning result of the vehicle is obtained based on the second correction result, thereby achieving high-precision positioning in sections where GPS signals are blocked, sections with fewer structural features, or sections that change frequently. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale.

[0019] Figure 1 is a flowchart of a positioning method in an embodiment of the present disclosure;

[0020] Figure 2 A schematic diagram of setting a time window in an embodiment of the present disclosure;

[0021] Figure 3is a schematic diagram of continuing to correct a first correction result based on road information acquired within a set time window in an embodiment of the present disclosure;

[0022] Figure 4 This is a schematic diagram of further optimizing the second correction result using a third posture obtained by repositioning in an embodiment of the present disclosure;

[0023] Figure 5 Schematic diagram of a positioning architecture in an embodiment of the present disclosure;

[0024] Figure 6 Schematic diagram of the structure of a positioning device in an embodiment of the present disclosure;

[0025] Figure 7 Schematic diagram of the structure of an electronic device in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0026] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0027] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0028] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0029] Common positioning technologies include those based on differential GPS positioning, visual feature point positioning, LiDAR point cloud positioning, and semantic positioning. Among them, technologies based on differential GPS positioning can provide centimeter-level positioning information. However, when a vehicle passes through obstructed areas, such as under trees or buildings, positioning is prone to loss or incorrect positioning information. Technologies such as visual feature point positioning, LiDAR point cloud positioning, and semantic positioning require the construction of high-precision maps in advance. Both visual feature point maps and LiDAR point cloud maps require a large amount of storage space, and for some repetitive and similar road sections, such as highways, there will be a large amount of redundant and useless information. Furthermore, in some scenarios, such as tunnels and open areas, due to the scarcity of structured features and semantic information, technologies such as visual feature point positioning, LiDAR point cloud positioning, and semantic positioning find it difficult to provide stable positioning information.

[0030] To address the above problems, the embodiments of the present disclosure provide a positioning method that can achieve high-precision positioning even on sections of road where GPS signals are blocked, sections with fewer structural features, or sections with frequent changes.

[0031] Figure 1 This is a flow chart of a positioning method in an embodiment of the present disclosure. This method can be executed by a positioning device, which can be implemented in software and / or hardware, and can be configured in an electronic device. Figure 1 As shown, the method may specifically include the following steps:

[0032] Step 110: When the vehicle is in a driving state, determine at least one posture of the vehicle through at least one positioning method within a set time window, wherein the postures determined by different positioning methods are different.

[0033] The time window can be set according to actual needs, for example, it can be a specific time such as 3s or 5s.

[0034] Exemplarily, the at least one positioning method includes one or more of a laser radar wheel speed odometer positioning method, a visual inertial navigation odometer positioning method, and a repositioning method;

[0035] Correspondingly, determining the position and posture of the vehicle by at least one positioning method when the vehicle is in a driving state includes:

[0036] Determining the first position of the vehicle by using a laser radar wheel speed odometer;

[0037] and / or, determining a second pose of the vehicle by a visual inertial navigation odometry;

[0038] And / or, when the vehicle is traveling on a preset road section, a third posture of the vehicle is determined by repositioning; wherein the posture includes one or more of the first posture, the second posture and the third posture.

[0039] Specifically, the process of determining the first posture of the vehicle by the lidar wheel speed odometer is as follows: the on-board lidar scans the surrounding environment in a set time window and outputs a point cloud. For a single-frame point cloud, the curvature of the point cloud near the target point is used to determine whether the target point is a corner point or a surface point, and then feature points are extracted from the single-frame point cloud (feature points include corner points and surface points); then the wheel speed and front wheel deviation angle of the vehicle are obtained according to the wheel speed sensor, and the posture change of the vehicle between two adjacent frames is determined based on recursive operation and feature points; then inter-frame matching of the two-frame point cloud is performed, and the relative motion relationship between the two adjacent frames is estimated; then, real-time local map feature points are constructed based on the posture of the vehicle before the current moment and the point cloud, and the posture of the vehicle is optimized based on the real-time local map feature points to obtain the first posture of the vehicle at each moment in the set time window.

[0040] Determining the second posture of the vehicle by visual inertial navigation odometer includes: shooting the surrounding environment with an on-board camera in a set time window, outputting images at a certain frequency, then extracting feature points from the images, performing feature point matching between frames, and at the same time, the inertial measurement unit outputs the angular velocity and acceleration of the vehicle, and obtaining the vehicle posture corresponding to the continuous image frames through integral joint operation (the vehicle posture is the posture in the camera coordinate system), and then obtaining the second posture of the vehicle in the world coordinate system according to the parameters of the camera.

[0041] Step 120: Perform a first correction on the initial posture obtained by the wheel speed odometer based on the target posture in the at least one posture to obtain a first correction result.

[0042] Optionally, the at least one posture includes a first posture of the vehicle obtained by a lidar wheel speed odometer, and / or a second posture of the vehicle obtained by a visual inertial navigation odometer.

[0043] If the target posture is the first posture, performing a first correction on the initial posture obtained by the wheel speed odometer based on the target posture in the at least one posture to obtain a first correction result posture initial posture includes:

[0044] Determine the first posture and the initial posture with the closest timestamp within the set time window. Each posture corresponds to a timestamp, indicating the time corresponding to the posture. Usually, the working frequency of the wheel speed odometer is high, so the number of initial postures obtained by the wheel speed odometer within the set time window is large. For example, refer to Figure 2As shown in the schematic diagram of a set time window, it is assumed that the wheel speed odometer outputs 4 initial postures within the set time window, namely the initial posture of the vehicle at time T1, the initial posture of the vehicle at time T2, the initial posture of the vehicle at time T3 and the initial posture of the vehicle at time T4. Due to different working frequencies, the timestamps corresponding to the vehicle postures determined by different positioning methods are not exactly the same, such as Figure 2 As shown, within the set time window, two vehicle first postures are obtained by the lidar wheel speed odometer, which are the first posture of the vehicle at time T11 and the first posture of the vehicle at time T31; within the set time window, two vehicle second postures are obtained by the visual inertial navigation odometer lidar wheel speed odometer, which are the second posture of the vehicle at time T21 and the second posture of the vehicle at time T41.

[0045] Assume that T1 is the first second within the set time window, T11 is the first.2 seconds, T2 is the second second, T21 is the second.2 seconds, T3 is the third second, T31 is the third.2 seconds, T4 is the fourth second, and T41 is the fourth.2 seconds. Then, the first pose and the initial pose with the closest timestamps are the initial pose at time T1 and the first pose at time T11, and the initial pose at time T3 and the first pose at time T31, respectively. If the first pose and the initial pose with the closest timestamps are determined as a combination, then the initial pose at time T1 and the first pose at time T11 are one combination, and the initial pose at time T3 and the first pose at time T31 are another combination.

[0046] The posture constraint between the two timestamps is determined based on the first posture closest to the timestamp and the initial posture to form a first residual equation. Specifically, a first residual equation is determined based on the initial posture at time T1 and the first posture at time T11, and another first residual equation is determined based on the initial posture at time T3 and the first posture at time T31. That is, there are multiple first residual equations. By solving the nonlinear least squares problem, the posture after the initial posture is corrected can be obtained through multiple first residual equations, that is, the first correction result is determined according to the first residual equation.

[0047] Alternatively, if the target posture is the second posture, performing a first correction on the initial posture obtained by the wheel speed odometer based on the target posture in the at least one posture to obtain a first correction result posture initial posture includes:

[0048] Determine the second posture and the initial posture with the closest timestamps within the set time window; the second posture and the initial posture with the closest timestamps are the initial posture at time T2 and the second posture at time T21, and the initial posture at time T4 and the second posture at time T41.

[0049] A pose constraint between the two timestamps is determined based on the second pose closest to the timestamp and the initial pose to form a second residual equation. Specifically, a second residual equation is determined based on the initial pose at time T2 and the second pose at time T21, and another second residual equation is determined based on the initial pose at time T4 and the second pose at time T41. That is, there are multiple second residual equations. By solving a nonlinear least squares problem, a pose after correction of the initial pose can be obtained using the multiple second residual equations, that is, the first correction result is determined based on the second residual equations.

[0050] Step 130: Continue to correct the first correction result based on the road information acquired in the set time window to obtain a second correction result, wherein the road information includes road signs, three-dimensional points constituting the road signs, and acquisition timestamps of the three-dimensional points.

[0051] Optionally, the first correction result includes a fourth posture of the vehicle (i.e., a posture obtained after a first round of correction of the initial posture), and further correcting the first correction result based on the road information obtained in the set time window to obtain a second correction result includes:

[0052] Multiple combinations are determined based on the timestamp of the fourth pose and the acquisition timestamp of the 3D point, each of which includes the timestamp closest to the fourth pose and the 3D point. If the 3D point is a point cloud point, the acquisition timestamp of the 3D point is the timestamp of the point cloud frame containing the 3D point; if the 3D point is a pixel point, the acquisition timestamp of the 3D point is the timestamp of the image frame containing the 3D point.

[0053] Based on the fourth pose of the target combination, the 3D points in the target combination are converted to the world coordinate system to obtain the target point. The target combination is any one of the multiple combinations, the fourth pose is the pose of the vehicle in the world coordinate system, and the coordinates of the 3D point represent its relative position with respect to the vehicle. Therefore, by combining the pose of the vehicle in the world coordinate system, the 3D point can be converted to the world coordinate system, and the point corresponding to the 3D point in the world coordinate system is the target point. Based on the distance between the target point and a road marker in a preset map, a target map is determined that includes the target road marker closest to the target point. The road marker may be a lane line, a curb, or the like. Specifically, the target map is determined based on the distance from the target point to the straight line that forms the road marker. That is, the line closest to the target point is determined based on the distance between the point and the straight line, and the map including the line is determined as the target map. The target map is one of the preset maps, and the accuracy of the preset map is below a preset threshold. That is, the preset map is a low-precision map, which, unlike a high-precision map, only includes simple semantic information. For example, lane lines can be represented as solid lines without further demarcation between dashed and solid lines, nor does it require more detailed semantic information such as line color. This is because in most application scenarios, vehicles only need to travel along lane lines, along roads, or within lanes. Only heading and lateral accuracy are required for positioning, and longitudinal accuracy is not required. Therefore, a highly accurate map is not required. For example, continuous short-dashed lane lines can be represented as long straight line segments, without the precise location of each end point. Therefore, for regular road sections, it is only necessary to pre-store low-precision maps, thereby solving the problem of maps taking up a large storage space. For repeated and similar road sections, such as highways, the problem of a large amount of redundant and useless information is solved. In addition, a certain positioning accuracy can still be guaranteed in sections with less road features and semantic information (such as tunnels, open scenes, etc.).

[0054] The fourth posture is optimized according to the target point and the target map to obtain the second correction result.

[0055] The fourth posture is optimized according to the target point and the target map to obtain the second correction result, including: determining a third residual equation according to the distance from the target point to the target road sign and the error between the road sign composed of the target point and the target road sign; optimizing the fourth posture based on the third residual equation to obtain the second correction result. The distance from the target point to the target road sign is specifically the distance between the target point and the straight line constituting the target road sign. The target road sign refers to the road sign in the pre-constructed target map, and the road sign composed of the target point refers to the road sign detected in real time when the vehicle is driving. The third residual equation is determined according to the error or overlap between the two. There are multiple third residual equations. By solving the nonlinear least squares problem, the posture of the fourth posture after correction can be obtained through multiple third residual equations.

[0056] For example, see Figure 3 A schematic diagram is shown of continuing to correct a first correction result based on road information obtained in a set time window. It is assumed that the first correction result includes a fourth posture at time T1, a fourth posture at time T2, a fourth posture at time T3, and a fourth posture at time T4. At the same time, road information detected in real time by radar is obtained at time T2, and road information detected in real time by image is obtained at time T4. Then, the fourth posture at time T2 is corrected in combination with the road information at time T2, and the fourth posture at time T4 is corrected in combination with the road information at time T4.

[0057] Optionally, before continuing to correct the first correction result based on the road information acquired in the set time window, the method further includes:

[0058] Within the set time window, a point cloud of the road is acquired based on the vehicle-mounted laser radar, and / or an image of the road is acquired based on the vehicle-mounted camera; a road point cloud is segmented from the point cloud through a deep learning segmentation network, and each point in the road point cloud corresponds to a semantic category; and / or, road pixels are extracted from the image through a semantic segmentation network, and the road pixels are projected into a three-dimensional space through a dynamic inverse projection transformation based on the parameters of the vehicle-mounted camera to determine the three-dimensional coordinates of the road pixels, and each pixel of the road pixels corresponds to a semantic category; the road pixels in the road point cloud and / or three-dimensional space are clustered, and the road point cloud and / or road pixels in the three-dimensional space are fitted according to the semantic category to obtain road signs.

[0059] Step 140: Obtain a final positioning result of the vehicle based on the second correction result.

[0060] Exemplarily, the second correction result includes a fifth posture of the vehicle (i.e., a posture obtained after a second round of optimization of the initial posture in combination with the road information detected in real time); the posture of the vehicle includes a third posture of the vehicle determined by repositioning; and determining a final positioning result of the vehicle based on the second correction result includes:

[0061] Determine the third pose and the fifth pose with the closest timestamps; determine the pose constraint between the two timestamps based on the third pose and the fifth pose with the closest timestamps to form a fourth residual equation; optimize the fifth pose based on the fourth residual equation to obtain the final positioning result. There are multiple fourth residual equations, and by solving a nonlinear least squares problem, a corrected pose of the fifth pose can be obtained using multiple fourth residual equations. By combining the third pose of the vehicle determined by the relocalization method to obtain the final positioning result of the vehicle, the positioning accuracy on complex roads or critical intersections can be improved.

[0062] For example, see Figure 4 A schematic diagram of continuing to optimize the second correction result by a third posture obtained by repositioning is shown. It is assumed that the second correction result includes the fifth posture at time T1, the fifth posture at time T2, the fifth posture at time T3, and the fifth posture at time T4; at the same time, the third posture G2 obtained by repositioning is obtained at time T2, and the third posture G4 obtained by repositioning is obtained at time T4. The fourth posture is corrected at time T2 in combination with the fifth posture G2, and the fifth posture is corrected at time T4 in combination with the third posture G4.

[0063] Optionally, when the vehicle is traveling on a preset road section, a third posture of the vehicle is determined by relocalization, including: when the vehicle is traveling on the preset road section, extracting a first descriptor based on a point cloud of the preset road section (the point cloud is acquired in real time while the vehicle is traveling) using a first set algorithm (e.g., a scan context algorithm), and / or extracting a second descriptor based on an image of the preset road section using a second set algorithm (e.g., a bag-of-visual-words descriptor algorithm); matching the first descriptor and / or the second descriptor with the descriptor of a pre-stored initial keyframe to obtain a candidate keyframe; and determining the posture corresponding to the candidate keyframe as the current posture of the vehicle. The pre-stored initial keyframe is a point cloud frame obtained by pre-laser scanning of the preset road section, and descriptors are extracted based on the point cloud frame, and the descriptors are associated with the point cloud frame and the posture of the vehicle at the current moment and stored; or the pre-stored initial keyframe is an image frame obtained by pre-photographing the preset road section, and descriptors are extracted based on the image frame, and the descriptors are associated with the image frame and the posture of the vehicle at the current moment and stored. Afterwards, when the vehicle is traveling on a preset road section, point cloud frames or image frames are acquired in real time through the on-board laser radar or camera, and then descriptors are extracted for the point cloud frames acquired in real time, or descriptors are extracted for the image frames acquired in real time; the descriptors extracted in real time are matched with the pre-stored descriptors, and the point cloud frames or image frames associated with the descriptors whose matching degree reaches the set value are determined as candidate key frames, and then the posture stored in association with the candidate key frames is determined as the current posture of the vehicle at the current moment (the current moment refers to the moment when the point cloud frame or image frame is acquired).

[0064] Furthermore, a corrected posture and a confidence level corresponding to the corrected posture are determined based on the candidate keyframes, the current posture, and a pre-built map matching the preset road section. The accuracy of the map matching the preset road section exceeds a preset threshold, so a high-precision map must be pre-built for the preset road section. The corrected posture with a confidence level exceeding a set value is determined as the vehicle's third posture. If the confidence level corresponding to the corrected posture is lower than a set value and the candidate keyframes cannot be obtained within a preset time period, the vehicle is determined to have left the preset road section. Typically, during a vehicle cold start, the vehicle's posture needs to be determined through relocalization. That is, the cold start location of the vehicle is typically a preset road section, requiring a pre-built high-precision map. During actual driving, when the vehicle leaves a high-precision map area, the relocalization module (the functional module that determines the vehicle's posture based on the relocalization method is called the relocalization module) is deactivated, and the odometer calculates the distance to the next high-precision map area. When the distance is less than a set threshold, the relocalization module is reactivated to determine the vehicle's posture through relocalization.

[0065] This embodiment provides a positioning method that performs a first round of optimization on an initial posture obtained by a wheel speed odometer by combining the first posture of the vehicle obtained by a lidar wheel speed odometer or the second posture of the vehicle obtained by a visual inertial navigation odometer, performs a second round of optimization on the initial posture by combining it with road information detected in real time, and performs a third round of optimization on the initial posture by combining it with a third posture obtained by repositioning to obtain a final positioning result. This enables the positioning method provided by the embodiment of the present disclosure to achieve high-precision positioning even on sections where GPS signals are blocked, sections with fewer structural features, or sections that change frequently.

[0066] In summary, see Figure 5 The schematic diagram of a positioning architecture shown in the figure includes a relocalization module, an odometer module (specifically, positioning information obtained through the output of the vehicle's odometer, positioning information obtained through the laser radar odometer method, and positioning information obtained through the visual inertial navigation odometer method), a road information detection module (implemented by positioning and deviation correction technology based on road information), and a fusion positioning module. At key intersections and complex road sections (which can be summarized as preset sections), the relocalization module is called to obtain positioning information through relocalization. On other conventional sections, positioning information is obtained through the laser radar odometer method or the visual inertial navigation odometer method, and relevant road information is obtained through positioning and deviation correction technology based on road information. The fusion positioning module then fuses these multiple positioning information, specifically performing multiple rounds of optimization on the initial pose output by the odometer to obtain the final positioning result.

[0067] The positioning method provided in this embodiment can better utilize the advantages of multiple odometers, establish local maps in real time, and better cope with the uncertainty brought about by scene changes. In scenes such as open roads and tunnels, it can ensure the heading and lateral accuracy of positioning to ensure vehicle driving safety. At the same time, in repeated similar sections, it can reduce the dependence on high-precision rich semantic maps and only require simpler map information.

[0068] Figure 6 FIG. 1 is a schematic diagram of the structure of a positioning device in an embodiment of the present disclosure. Figure 6 As shown, the device includes: a first determination module 610, a first correction module 620, a second correction module 630 and a second determination module 640.

[0069] Among them, the first determination module 610 is used to determine at least one posture of the vehicle through at least one positioning method in a set time window when the vehicle is in a driving state, wherein the posture determined by different positioning methods is different; the first correction module 620 is used to perform a first correction on the initial posture obtained by the wheel speed odometer based on the target posture in the at least one posture to obtain a first correction result; the second correction module 630 is used to continue to correct the first correction result based on the road information obtained in the set time window to obtain a second correction result, wherein the road information includes road signs, three-dimensional points constituting the road signs, and the acquisition timestamp of the three-dimensional points; the second determination module 640 is used to obtain the final positioning result of the vehicle based on the second correction result.

[0070] Optionally, the at least one positioning method includes one or more of a lidar wheel speed odometer positioning method, a visual inertial navigation odometer positioning method, and a repositioning method.

[0071] The first determination module 610 is specifically configured to: determine the first position of the vehicle by using a laser radar wheel speed odometer;

[0072] and / or, determining a second pose of the vehicle by a visual inertial navigation odometry;

[0073] and / or, when the vehicle is traveling on a preset road section, determining a third posture of the vehicle by repositioning;

[0074] The posture includes one or more of the first posture, the second posture and the third posture.

[0075] Optionally, the at least one posture includes a first posture of the vehicle obtained by a lidar wheel speed odometer, and / or a second posture of the vehicle obtained by a visual inertial navigation odometer; if the target posture is the first posture, the first correction module 620 includes: a first determination unit, used to determine the first posture and the initial posture with the closest timestamps within the set time window; a second determination unit, used to determine the posture constraint between two timestamps based on the first posture with the closest timestamps and the initial posture to form a first residual equation; a third determination unit, used to determine the first correction result based on the first residual equation.

[0076] If the target posture is the second posture, the first determining unit is configured to determine the second posture with the closest timestamp and the initial posture within the set time window. The second determining unit is configured to determine a posture constraint between the two timestamps based on the second posture with the closest timestamp and the initial posture to form a second residual equation. The third determining unit is configured to determine the first correction result based on the second residual equation.

[0077] Optionally, the first correction result includes the fourth posture of the vehicle, and the second correction module 630 includes: a first determination unit, used to determine multiple combinations based on the timestamp of the fourth posture and the acquisition timestamp of the three-dimensional point, each combination including the fourth posture and the three-dimensional point with the timestamp closest to the timestamp; a conversion unit, used to convert the three-dimensional points in the target combination into the world coordinate system according to the fourth posture in the target combination to obtain the target point; a second determination unit, used to determine the target map including the target road sign closest to the target point according to the distance between the target point and the road sign in the preset map, the target map being one of the preset maps, and the accuracy of the preset map being lower than a preset threshold; an optimization unit, used to optimize the fourth posture according to the target point and the target map to obtain the second correction result.

[0078] Optionally, the device also includes: an acquisition module, used to acquire a point cloud of the road based on the vehicle-mounted laser radar within the set time window, and / or acquire an image of the road based on the vehicle-mounted camera; segmenting a road point cloud from the point cloud through a deep learning segmentation network, and each point in the road point cloud corresponds to a semantic category; and / or extracting road pixels from the image through a semantic segmentation network, and projecting the road pixels to a three-dimensional space through a dynamic inverse projection transformation based on the parameters of the vehicle-mounted camera to determine the three-dimensional coordinates of the road pixels, and each pixel of the road pixels corresponds to a semantic category; clustering the road point cloud and / or road pixels in the three-dimensional space, and fitting the road point cloud and / or road pixels in the three-dimensional space according to the semantic category to obtain road signs.

[0079] Optionally, the optimization unit is specifically used to: determine a third residual equation based on the distance from the target point to the target road sign and the error between the road sign composed of the target point and the target road sign; optimize the fourth posture based on the third residual equation to obtain the second correction result.

[0080] Optionally, the second correction result includes a fifth posture of the vehicle; the posture of the vehicle includes a third posture of the vehicle determined by relocalization. The second determination module 640 is specifically configured to: determine the third posture and the fifth posture with the closest timestamps; determine a posture constraint between the two timestamps based on the third posture and the fifth posture with the closest timestamps to form a fourth residual equation; and optimize the fifth posture based on the fourth residual equation to obtain the final positioning result.

[0081] Optionally, the first determination module 610 includes: an extraction unit, used to extract a first descriptor based on the point cloud of the preset road section through a first set algorithm, and / or extract a second descriptor based on the image of the preset road section through a second set algorithm when the vehicle is traveling on a preset road section; a matching unit, used to match the first descriptor and / or the second descriptor with the descriptor of a pre-stored initial key frame to obtain a candidate key frame; a first determination unit, used to determine the posture corresponding to the candidate key frame as the current posture of the vehicle; a second determination unit, used to determine the corrected posture and the confidence corresponding to the corrected posture based on the candidate key frame, the current posture and a pre-constructed map matching the preset road section, wherein the accuracy of the map matching the preset road section is higher than a preset threshold; a third determination unit, used to determine the corrected posture whose confidence exceeds a set value as the third posture of the vehicle.

[0082] Optionally, the first determination module 610 also includes a fourth determination unit, which is used to determine that the vehicle has left the preset road section if the confidence corresponding to the corrected posture is lower than a set value and the candidate key frame cannot be obtained within a preset time.

[0083] The positioning device provided in the embodiment of the present disclosure can execute the steps of the positioning method provided in the embodiment of the method of the present disclosure, and the execution steps and beneficial effects are not repeated here.

[0084] Figure 7 This is a schematic diagram of the structure of an electronic device in the embodiment of the present disclosure. Figure 7 , which shows a structural diagram of an electronic device 500 suitable for implementing the embodiments of the present disclosure. Figure 7 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0085] like Figure 7As shown, the electronic device 500 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes to implement the methods of the embodiments described in the present disclosure according to the program stored in the read-only memory (ROM) 502 or the program loaded from the storage device 508 into the random access memory (RAM) 503. Various programs and data required for the operation of the electronic device 500 are also stored in the RAM 503. The processing device 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0086] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart, thereby implementing the positioning method described above. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0087] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0088] The computer-readable medium may be included in the electronic device, or may exist independently and not be incorporated into the electronic device. The computer-readable medium carries one or more programs. When executed by the electronic device, the electronic device: when the vehicle is in motion, determines at least one posture of the vehicle using at least one positioning method within a set time window, wherein the postures determined by different positioning methods are different; performs a first correction on the initial posture obtained by the wheel speed odometer based on the target posture in the at least one posture, obtaining a first correction result; further corrects the first correction result based on road information acquired within the set time window, obtaining a second correction result, wherein the road information includes road signs, three-dimensional points constituting the road signs, and acquisition timestamps of the three-dimensional points; and obtains a final positioning result of the vehicle based on the second correction result.

[0089] Optionally, when the above one or more programs are executed by the electronic device, the electronic device may also execute other steps described in the above embodiments.

[0090] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0091] Solution 1: A positioning method, comprising:

[0092] When the vehicle is in a driving state, determining at least one posture of the vehicle by using at least one positioning method within a set time window, wherein the postures determined by different positioning methods are different;

[0093] Performing a first correction on the initial posture obtained by the wheel speed odometer based on the target posture in the at least one posture to obtain a first correction result;

[0094] Continuing to correct the first correction result based on road information acquired in the set time window to obtain a second correction result, the road information including road signs, three-dimensional points constituting the road signs, and acquisition timestamps of the three-dimensional points;

[0095] A final positioning result of the vehicle is obtained based on the second correction result.

[0096] Solution 2: According to the method of Solution 1, the at least one positioning method includes one or more of a laser radar wheel speed odometer positioning method, a visual inertial navigation odometer positioning method, and a repositioning method;

[0097] Determining the position of the vehicle by at least one positioning method when the vehicle is in a driving state includes:

[0098] Determining the first position of the vehicle by using a laser radar wheel speed odometer;

[0099] and / or, determining a second pose of the vehicle by a visual inertial navigation odometry;

[0100] and / or, when the vehicle is traveling on a preset road section, determining a third posture of the vehicle by repositioning;

[0101] The posture includes one or more of the first posture, the second posture and the third posture.

[0102] Solution 3: According to the method of Solution 1, the at least one posture includes a first posture of the vehicle obtained by a lidar wheel speed odometer, and / or a second posture of the vehicle obtained by a visual inertial navigation odometer;

[0103] If the target posture is the first posture, performing a first correction on the initial posture obtained by the wheel speed odometer based on the target posture in the at least one posture to obtain a first correction result posture initial posture includes:

[0104] Determining the first posture and the initial posture with the closest timestamps within the set time window;

[0105] Determining a pose constraint between two timestamps based on the first pose closest to the timestamp and the initial pose to form a first residual equation;

[0106] determining the first correction result according to the first residual equation;

[0107] If the target posture is the second posture, performing a first correction on the initial posture obtained by the wheel speed odometer based on the target posture in the at least one posture to obtain a first correction result posture initial posture includes:

[0108] Determining the second posture and the initial posture with the closest timestamps within the set time window;

[0109] Determining a pose constraint between two timestamps based on the second pose closest to the timestamp and the initial pose to form a second residual equation;

[0110] The first correction result is determined according to the second residual equation.

[0111] Solution 4: According to the method of Solution 1, the first correction result includes a fourth posture of the vehicle, and the first correction result is further corrected based on the road information obtained in the set time window to obtain a second correction result, including:

[0112] Determine a plurality of combinations according to the timestamp of the fourth posture and the acquisition timestamp of the three-dimensional point, each of the combinations including the timestamp closest to the fourth posture and the three-dimensional point;

[0113] According to the fourth posture of the target combination, the three-dimensional point in the target combination is converted into a world coordinate system to obtain a target point;

[0114] Determining, based on the distance between the target point and a road marker in a preset map, a target map including a target road marker closest to the target point, wherein the target map is one of the preset maps, and the accuracy of the preset map is lower than a preset threshold;

[0115] The fourth posture is optimized according to the target point and the target map to obtain the second correction result.

[0116] Solution 5: According to the method of Solution 4, before continuing to correct the first correction result based on the road information obtained in the set time window, the method further includes:

[0117] Acquiring a point cloud of the road based on a vehicle-mounted laser radar of the vehicle within the set time window, and / or acquiring an image of the road based on a vehicle-mounted camera;

[0118] Segmenting a road point cloud from the point cloud using a deep learning segmentation network, wherein each point in the road point cloud corresponds to a semantic category; and / or extracting road pixels from the image using a semantic segmentation network, and projecting the road pixels into a three-dimensional space using a dynamic inverse projection transformation based on parameters of the vehicle-mounted camera to determine the three-dimensional coordinates of the road pixels, wherein each pixel of the road pixels corresponds to a semantic category;

[0119] The road point cloud and / or road pixels in the three-dimensional space are clustered, and the road point cloud and / or road pixels in the three-dimensional space are fitted according to the semantic categories to obtain road signs.

[0120] Solution 6: According to the method of Solution 4, optimizing the fourth posture according to the target point and the target map to obtain the second correction result includes:

[0121] determining a third residual equation according to a distance from the target point to the target road sign and an error between the road sign formed by the target point and the target road sign;

[0122] The fourth posture is optimized based on the third residual equation to obtain the second correction result.

[0123] Solution 7: According to the method according to any one of solutions 1 to 6, the second correction result includes a fifth posture of the vehicle;

[0124] The posture of the vehicle includes a third posture of the vehicle determined by relocalization;

[0125] Determining a final positioning result of the vehicle based on the second correction result includes:

[0126] determining the third pose and the fifth pose having the closest timestamps;

[0127] determining a pose constraint between two time stamps based on the third pose and the fifth pose closest to the time stamp to form a fourth residual equation;

[0128] The fifth posture is optimized based on the fourth residual equation to obtain the final positioning result.

[0129] Solution 8: According to the method of Solution 2, when the vehicle is traveling on a preset road section, determining the third posture of the vehicle by repositioning, comprising:

[0130] When the vehicle is traveling on a preset road section, extracting a first descriptor based on a point cloud of the preset road section using a first set algorithm, and / or extracting a second descriptor based on an image of the preset road section using a second set algorithm;

[0131] Matching the first descriptor and / or the second descriptor with a pre-stored descriptor of an initial key frame to obtain a candidate key frame;

[0132] Determining the posture corresponding to the candidate key frame as the current posture of the vehicle;

[0133] Determining a corrected posture and a confidence level corresponding to the corrected posture based on the candidate keyframe, the current posture, and a pre-built map matching the preset road section, wherein the accuracy of the map matching the preset road section is higher than a preset threshold;

[0134] The corrected posture whose confidence exceeds a set value is determined as the third posture of the vehicle.

[0135] Solution 9: The method according to Solution 8, further comprising:

[0136] If the confidence level corresponding to the corrected posture is lower than a set value and the candidate key frame cannot be obtained within a preset time period, it is determined that the vehicle has left the preset road section.

[0137] Solution 10: A positioning device, comprising:

[0138] a first determining module, configured to determine, when the vehicle is in a driving state, at least one posture of the vehicle by using at least one positioning method within a set time window, wherein the postures determined by different positioning methods are different;

[0139] a first correction module, configured to perform a first correction on an initial posture obtained by the wheel speed odometer based on a target posture in the at least one posture, to obtain a first correction result;

[0140] a second correction module, configured to further correct the first correction result based on road information acquired during the set time window to obtain a second correction result, wherein the road information includes road signs, three-dimensional points constituting the road signs, and acquisition timestamps of the three-dimensional points;

[0141] A second determining module is configured to obtain a final positioning result of the vehicle based on the second correction result.

[0142] Solution 11. An electronic device, comprising:

[0143] one or more processors;

[0144] a storage device for storing one or more programs;

[0145] When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of options 1-9.

[0146] Solution 12: A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any one of Solutions 1-9.

[0147] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also includes other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

Claims

1. A positioning method, characterized in that: The method comprises: When the vehicle is in a driving state, determining at least one posture of the vehicle by using at least one positioning method within a set time window, wherein the postures determined by different positioning methods are different; Performing a first correction on the initial posture obtained by the wheel speed odometer based on the target posture in the at least one posture to obtain a first correction result; Continuing to correct the first correction result based on road information acquired in the set time window to obtain a second correction result, the road information including road signs, three-dimensional points constituting the road signs, and acquisition timestamps of the three-dimensional points; Obtaining a final positioning result of the vehicle based on the second correction result; The at least one positioning method includes one or more of a laser radar wheel speed odometer positioning method, a visual inertial navigation odometer positioning method, and a repositioning method when traveling on a preset road section; If the target posture is the first posture of the vehicle obtained by the laser radar wheel speed odometer, performing a first correction on the initial posture obtained by the wheel speed odometer based on the target posture in the at least one posture to obtain a first correction result includes: Determining the first posture and the initial posture with the closest timestamps within the set time window; Determining a pose constraint between two timestamps based on the first pose closest to the timestamp and the initial pose to form a first residual equation; The first correction result is determined according to the first residual equation.

2. The method according to claim 1, characterized in that Determining the position and posture of the vehicle by at least one positioning method when the vehicle is in a driving state includes: Determining the first position of the vehicle by using a laser radar wheel speed odometer; and / or, determining a second pose of the vehicle by a visual inertial navigation odometry; and / or, when the vehicle is traveling on a preset road section, determining a third posture of the vehicle by repositioning; The posture includes one or more of the first posture, the second posture and the third posture.

3. The method according to claim 2, characterized in that If the target posture is the second posture, performing a first correction on the initial posture obtained by the wheel speed odometer based on the target posture in the at least one posture to obtain a first correction result posture initial posture includes: Determining the second posture and the initial posture with the closest timestamps within the set time window; Determining a pose constraint between two timestamps based on the second pose closest to the timestamp and the initial pose to form a second residual equation; The first correction result is determined according to the second residual equation.

4. The method according to claim 1, wherein The first correction result includes the fourth posture of the vehicle, and the first correction result is further corrected based on the road information obtained in the set time window to obtain a second correction result, including: Determine a plurality of combinations according to the timestamp of the fourth posture and the acquisition timestamp of the three-dimensional point, each of the combinations including the timestamp closest to the fourth posture and the three-dimensional point; According to the fourth posture of the target combination, the three-dimensional point in the target combination is converted into a world coordinate system to obtain a target point; Determining, based on the distance between the target point and a road marker in a preset map, a target map including a target road marker closest to the target point, wherein the target map is one of the preset maps, and the accuracy of the preset map is lower than a preset threshold; The fourth posture is optimized according to the target point and the target map to obtain the second correction result.

5. The method according to claim 4, characterized in that Before continuing to correct the first correction result based on the road information acquired in the set time window, the method further includes: Acquiring a point cloud of the road based on a vehicle-mounted laser radar of the vehicle within the set time window, and / or acquiring an image of the road based on a vehicle-mounted camera; Segmenting a road point cloud from the point cloud using a deep learning segmentation network, wherein each point in the road point cloud corresponds to a semantic category; and / or extracting road pixels from the image using a semantic segmentation network, and projecting the road pixels into a three-dimensional space using a dynamic inverse projection transformation based on parameters of the vehicle-mounted camera to determine the three-dimensional coordinates of the road pixels, wherein each pixel of the road pixels corresponds to a semantic category; The road point cloud and / or road pixels in the three-dimensional space are clustered, and the road point cloud and / or road pixels in the three-dimensional space are fitted according to the semantic categories to obtain road signs.

6. The method according to claim 4, characterized in that The optimizing the fourth posture according to the target point and the target map to obtain the second correction result includes: determining a third residual equation according to a distance from the target point to the target road sign and an error between the road sign formed by the target point and the target road sign; The fourth posture is optimized based on the third residual equation to obtain the second correction result.

7. The method according to any one of claims 1 to 6, characterized in that The second correction result includes a fifth posture of the vehicle; The posture of the vehicle includes a third posture of the vehicle determined by relocalization; Determining a final positioning result of the vehicle based on the second correction result includes: determining the third pose and the fifth pose having the closest timestamps; determining a pose constraint between two time stamps based on the third pose and the fifth pose closest to the time stamp to form a fourth residual equation; The fifth posture is optimized based on the fourth residual equation to obtain the final positioning result.

8. The method according to claim 2, characterized in that When the vehicle is traveling on a preset road section, determining the third posture of the vehicle by repositioning the vehicle includes: When the vehicle is traveling on a preset road section, extracting a first descriptor based on a point cloud of the preset road section using a first set algorithm, and / or extracting a second descriptor based on an image of the preset road section using a second set algorithm; Matching the first descriptor and / or the second descriptor with a pre-stored descriptor of an initial key frame to obtain a candidate key frame; Determining the posture corresponding to the candidate key frame as the current posture of the vehicle; Determining a corrected posture and a confidence level corresponding to the corrected posture based on the candidate keyframe, the current posture, and a pre-built map matching the preset road section, wherein the accuracy of the map matching the preset road section is higher than a preset threshold; The corrected posture whose confidence exceeds a set value is determined as the third posture of the vehicle.

9. The method according to claim 8, characterized in that The method further comprises: If the confidence level corresponding to the corrected posture is lower than a set value and the candidate key frame cannot be obtained within a preset time period, it is determined that the vehicle has left the preset road section.

10. A positioning device, characterized in that: include: a first determining module, configured to determine, when the vehicle is in a driving state, at least one posture of the vehicle by using at least one positioning method within a set time window, wherein the postures determined by different positioning methods are different; a first correction module, configured to perform a first correction on an initial posture obtained by the wheel speed odometer based on a target posture in the at least one posture, to obtain a first correction result; a second correction module, configured to further correct the first correction result based on road information acquired during the set time window to obtain a second correction result, wherein the road information includes road signs, three-dimensional points constituting the road signs, and acquisition timestamps of the three-dimensional points; A second determining module, configured to obtain a final positioning result of the vehicle based on the second correction result; The at least one positioning method includes one or more of a laser radar wheel speed odometer positioning method, a visual inertial navigation odometer positioning method, and a repositioning method when traveling on a preset road section; If the target posture is the first posture of the vehicle obtained by the laser radar wheel speed odometer, performing a first correction on the initial posture obtained by the wheel speed odometer based on the target posture in the at least one posture to obtain a first correction result includes: Determining the first posture and the initial posture with the closest timestamps within the set time window; Determining a pose constraint between two timestamps based on the first pose closest to the timestamp and the initial pose to form a first residual equation; The first correction result is determined according to the first residual equation.

11. An electronic device, characterized in that: The electronic device comprises: one or more processors; a storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images