Vehicle pose determination method and device, storage medium and electronic equipment

Abstract

Embodiments of the invention disclose a vehicle pose determination method and apparatus, a storage medium and an electronic device. The method comprises the steps of obtaining a semantic map, environmental perception data and odometer data; processing the environmental perception data to obtain a semantic segmentation image containing semantic information and a distance field graph corresponding to the semantic segmentation image; matching a first road surface element of the semantic map with a second road surface element of the semantic segmentation image to obtain an element matching pair; based on the element matching pair, the distance field map and the odometer data, adjusting the vehicle predicted pose of the current frame to obtain a vehicle initial pose of the current frame; and determining the vehicle optimized pose of the current frame based on the vehicle initial pose, the vehicle optimized pose and the speedometer data of a preset number of frames before the current frame and the vehicle initial pose and the speedometer data of the current frame. According to the method and the device, the semantic map and the environment perception data can be fully utilized, and the accuracy and the positioning precision of the determined vehicle pose are effectively improved.

Description

Technical Field

[0001] The present disclosure relates to the field of intelligent driving technology, and in particular to a method, device, storage medium, and electronic device for determining a vehicle posture. Background Art

[0002] At present, intelligent driving technology is developing rapidly. Positioning of positioning devices such as vehicles is the basis of intelligent driving technology. The accuracy and reliability of vehicle positioning will directly affect vehicle planning and control performance, thereby affecting the functional experience of intelligent driving.

[0003] In related technologies, vehicle positioning is achieved through an inertial measurement unit algorithm. However, a series of cumulative error problems are prone to occur when positioning through an inertial measurement unit, resulting in inaccurate positioning. Summary of the Invention

[0004] In order to solve the above technical problems, the present disclosure is proposed. Embodiments of the present disclosure provide a vehicle posture determination method, device, storage medium, and electronic device to improve the accuracy and reliability of vehicle positioning during assisted driving.

[0005] According to a first aspect of an embodiment of the present disclosure, a method for determining a vehicle posture is provided, comprising:

[0006] Obtain semantic maps, environmental perception data, and odometry data;

[0007] Processing the environmental perception data to obtain a semantic segmentation image containing semantic information and a distance field map corresponding to the semantic segmentation image;

[0008] matching a first road surface element in the semantic map with a second road surface element in the semantic segmentation image to obtain an element matching pair;

[0009] Adjusting the predicted vehicle pose of the current frame based on the element matching pairs, the distance field map, and the odometer data to obtain an initial vehicle pose of the current frame;

[0010] The optimized posture of the vehicle in the current frame is determined based on the optimized posture and odometer data of the vehicle in a preset number of frames before the current frame, and the initial posture and odometer data of the vehicle in the current frame.

[0011] According to a second aspect of an embodiment of the present disclosure, there is provided a vehicle posture determination device, comprising:

[0012] Information acquisition module, used to obtain semantic maps, environmental perception data and odometry data;

[0013] a distance field map determination module, configured to process the environmental perception data to obtain a semantic segmentation image containing semantic information and a distance field map corresponding to the semantic segmentation image;

[0014] a matching module, configured to match a first road surface element in the semantic map with a second road surface element in the semantic segmentation image to obtain an element matching pair;

[0015] A posture adjustment module is used to adjust the predicted posture of the vehicle in the current frame based on the element matching pair, the distance field map and the odometer data to obtain the initial posture of the vehicle in the current frame;

[0016] A posture smoothing module is used to determine the optimized posture of the vehicle in the current frame based on the optimized posture and odometer data of the vehicle in a preset number of frames before the current frame, and the initial posture and odometer data of the vehicle in the current frame.

[0017] According to a third aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, wherein the storage medium stores a computer program, and the computer program is used to execute the above-mentioned vehicle posture determination method.

[0018] According to a fourth aspect of the embodiments of the present disclosure, there is provided an electronic device, comprising:

[0019] processor;

[0020] a memory for storing instructions executable by the processor;

[0021] The processor is used to read the executable instructions from the memory and execute the instructions to implement the above-mentioned vehicle posture determination method.

[0022] Based on the above embodiments of the present disclosure, during the assisted driving process, a semantic map, environmental perception data, and odometer data can be obtained in real time; the environmental perception data is processed to obtain a semantic segmentation image containing semantic information and a distance field map corresponding to the semantic segmentation image; a first road surface element in the semantic map is matched with a second road surface element in the semantic segmentation image to obtain an element matching pair; based on the element matching pair, the distance field map, and the odometer data, the predicted vehicle pose of the current frame is adjusted to obtain the vehicle initial pose of the current frame; and the optimized vehicle pose of the current frame is determined based on the vehicle optimized pose and odometer data of a preset number of frames before the current frame, as well as the vehicle initial pose and odometer data of the current frame. The technical solution disclosed in the present disclosure can fully utilize the semantic map and environmental perception data by matching the first road surface element in the vehicle's environmental perception data with the second road surface element in the semantic map, effectively improving the accuracy and positioning precision of the determined vehicle pose; on the other hand, smoothing and correcting the single-frame positioning result based on the positioning results of multiple frames of data can further improve the positioning precision of the vehicle, help improve vehicle planning and control performance, and thus improve the accuracy and reliability of assisted driving.

[0023] The technical solution of the present disclosure is further described in detail below through the accompanying drawings and examples. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The above and other purposes, features, and advantages of the present disclosure will become more apparent through a more detailed description of the embodiments of the present disclosure in conjunction with the accompanying drawings. The accompanying drawings are intended to provide a further understanding of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and are not intended to limit the present disclosure. In the drawings, the same reference numerals generally represent the same components or steps.

[0025] Figure 1 This is a platform structure diagram applicable to the present disclosure.

[0026] Figure 2 It is a flowchart of a vehicle posture determination method provided by an exemplary embodiment of the present disclosure.

[0027] Figure 3 2 is a flow chart of step 204 in a method for determining a vehicle posture according to another exemplary embodiment of the present disclosure.

[0028] Figure 4 2 is a flow chart of step 205 in a method for determining a vehicle posture according to another exemplary embodiment of the present disclosure.

[0029] Figure 5 2 is a flow chart of step 202 in a vehicle posture determination method provided by another exemplary embodiment of the present disclosure.

[0030] Figure 6 2 is a flow chart of step 203 in the vehicle posture determination method provided by another exemplary embodiment of the present disclosure.

[0031] Figure 7 It is a schematic diagram of a sliding smooth posture in a vehicle posture determination method provided by an exemplary embodiment of the present disclosure.

[0032] Figure 8 It is a schematic diagram of a semantic map of a parking lot in a method for determining a vehicle posture provided by an exemplary embodiment of the present disclosure.

[0033] Figure 9 It is a structural diagram of a vehicle posture determination device provided by an exemplary embodiment of the present disclosure.

[0034] Figure 10 It is a structural diagram of a vehicle posture determination device provided by another exemplary embodiment of the present disclosure.

[0035] Figure 11 is a structural diagram of an electronic device provided by an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0036] In order to explain the present disclosure, example embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure. It should be understood that the present disclosure is not limited to the example embodiments described herein.

[0037] It should be noted that the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.

[0038] Application Overview

[0039] In the process of realizing the present disclosure, the inventors discovered through research that when obtaining vehicle motion information through an inertial measurement unit and chassis system for posture estimation, the vehicle's linear velocity is directly obtained through the chassis system (such as wheel speed sensors, CAN bus, etc.), and the angular velocity is obtained by measuring the vehicle's rotational speed around each axis using the inertial measurement unit's gyroscope. Displacement is obtained by integrating the chassis linear velocity, and attitude angles (roll, pitch, and yaw) are obtained by integrating the gyroscope's angular velocity. However, due to inherent bias, noise, and temperature drift in the gyroscope, as well as measurement errors in the chassis velocity sensor, long-term integration can cause errors in position and attitude angles to accumulate over time, resulting in poor accuracy in the determined vehicle posture.

[0040] In the disclosed embodiment, environmental perception data can be acquired in real time using sensors deployed on the vehicle. A semantic segmentation image containing semantic information can be obtained based on the environmental perception data, and each second road surface element can be acquired from the semantic segmentation image. Based on a pre-constructed semantic map, each first road surface element can be acquired, thereby constructing an element matching pair based on the first road surface element and the second road surface element. The vehicle posture can be adjusted by combining the element matching pairs to improve the accuracy of the vehicle posture.

[0041] Based on the different sensor positions, the environmental perception data acquired includes but is not limited to forward-looking perception data, rear-view perception data, side-view perception data, and bird's-eye view (BEV) perception data. Furthermore, these perception data can be fused to obtain surround-view perception data or bird's-eye view perception data. By fusing multiple perception data, the robustness and spatiotemporal consistency of the perception data can be improved, thereby enhancing the stability and reliability of the perception system.

[0042] The disclosed technical solution determines the vehicle's posture by combining the first road surface element in the vehicle's environmental perception data with the second road surface element in the semantic map, thereby making full use of the semantic map and environmental perception data to effectively improve the accuracy and positioning precision of the determined vehicle posture. In addition, the positioning results of a single frame are smoothed and corrected based on the positioning results of multiple frames of data, which can further improve the positioning accuracy of the vehicle, help improve vehicle planning and control performance, and thereby improve the accuracy and reliability of assisted driving.

[0043] Exemplary Systems

[0044] Figure 1 This is a platform structure diagram applicable to the present disclosure, such as Figure 1As shown, multiple sensors 110, a chassis data communication bus 120, and an onboard computing platform 130 can be deployed on the vehicle as needed. The chassis data communication bus 120 can send chassis motion control instructions (including wheel speed, steering, and other data) to the odometer module 131 in the onboard computing platform 130. The odometer module 131 obtains odometer data such as the vehicle's distance traveled (the total distance traveled by the vehicle), the vehicle's motion status (including but not limited to vehicle speed, vehicle position, and posture) based on the received chassis motion control instructions. Multiple sensors 110 can be deployed at different locations and / or orientations on the vehicle according to perception requirements to perform environmental perception of corresponding areas in the vehicle's external environment (e.g., the left front area, the front area, the right front area, the left rear area, the rear area, the right rear area, the left side area of ​​the vehicle body, the rear side area of ​​the vehicle body, etc.), thereby obtaining a perception processing module 132. Multiple sensors 110 can include sensors of the same or different types, for example, visual sensors (i.e., cameras), laser radar (Light Detection and Ranging, LiDAR), etc.

[0045] The perception processing module 132 in the on-board computing platform 130 can perform perception processing on the environmental perception data to obtain a semantic segmentation image containing semantic information. Then, the positioning module 133 in the on-board computing platform 130 can use the technical solution of the present disclosure to determine the vehicle posture based on the semantic segmentation image containing semantic information, odometer data and semantic map.

[0046] Figure 1 This is only a structural diagram of an exemplary platform of the embodiment of the present disclosure. Based on the description of the embodiment of the present disclosure, those skilled in the art will know that the embodiment of the present disclosure can also adopt any other feasible implementation method. For example, the computing platform can also be partially or completely deployed on a cloud server or terminal device (such as a mobile terminal, tablet computer, PC, etc.), and receive environmental perception data collected by multiple sensors 110 through communication connection with the vehicle driving control system, and use the vehicle posture determination method provided by the embodiment of the present disclosure to process the data, obtain the vehicle posture and feed it back to the vehicle driving control system of the vehicle, thereby improving the vehicle planning and control performance.

[0047] The disclosed technical solution can be applied to determine the vehicle posture in driving scenarios, and can also be applied to determine the vehicle posture in parking scenarios. In order to more clearly describe the disclosed technical solution, the specific implementation of the disclosed technical solution is described below using the parking scenario as an example.

[0048] Exemplary Methods

[0049] Figure 2This is a flow chart of a method for determining a vehicle posture provided by an exemplary embodiment of the present disclosure, which can be applied to electronic devices (such as Figure 1 On the vehicle computing platform 130), such as Figure 2 As shown, the following steps are included:

[0050] Step 201: Acquire semantic maps, environmental perception data, and odometer data.

[0051] The semantic map is used to indicate a map that can provide a reference for vehicle positioning operations and can contain element information of multiple road elements. For example, a parking lot semantic map can include but is not limited to the location and identification information of each parking space in the parking lot, the location and identification of lane lines, etc. Figure 8 , which illustrates a semantic map of a parking environment.

[0052] In some embodiments, the semantic map can be obtained by perceptual processing based on the perception data collected during vehicle driving; in other embodiments, the semantic map can also be determined based on the parking lot map provided by the parking lot; in other embodiments, the semantic map can also be determined by the high-precision map provided by the high-precision map service provider.

[0053] For example, after a vehicle enters a parking lot, it can collect environmental data during driving and perform perception processing to obtain the position information of each parking space in the parking lot; or, RGB images and point cloud data of each area of ​​the parking lot can be obtained in advance based on cameras and lidars deployed at various locations in the parking lot; parking spaces are detected based on the RGB images and point cloud data to obtain a semantic map of the parking lot, and the semantic map of the parking lot is subsequently pushed to the vehicle before a vehicle enters the parking lot.

[0054] Among them, the environmental perception data can be environmental data obtained by sensors deployed on the vehicle to perform environmental perception on corresponding areas in the vehicle's external environment (for example, the left front area, the front area, the right front area, the left rear area, the rear area, the right rear area, the left side area of ​​the vehicle body, the rear side area of ​​the vehicle body, etc.).

[0055] Among them, the odometer data may include the vehicle's travel distance (the total distance traveled by the vehicle, which can be determined based on the number of wheel rotations) and the vehicle's motion status (including but not limited to vehicle speed, vehicle position and posture).

[0056] Step 202 : Process the environmental perception data to obtain a semantic segmentation image containing semantic information and a distance field map corresponding to the semantic segmentation image.

[0057] Among them, the categories of semantic segmentation may include: parking space lines, curbs, solid lane lines, dashed lane lines, speed bumps, and road arrows.

[0058] In some embodiments, when a variety of environmental perception data are acquired through different sensors, the multiple environmental perception data may be first fused through an inverse perspective transformation algorithm to obtain a surround view image or a bird's-eye view image.

[0059] Among them, the distance field map can represent the distance between each zero-point pixel in the semantic segmentation image and the non-zero-point pixel with semantic features (such as the pixel points corresponding to the parking space line). Through the distance from each zero-point pixel to the non-zero-point pixel with semantic features in the semantic segmentation image, as well as the distance from each non-zero-point pixel to the zero-point pixel, the semantic segmentation image can be distance transformed to obtain the distance field map.

[0060] In the embodiment of the present disclosure, it is possible to use Figure 5 The distance field map is determined in the manner described in the illustrated embodiment, which will not be described in detail here.

[0061] Step 203 : Match the first road surface element in the semantic map with the second road surface element in the semantic segmentation image to obtain an element matching pair.

[0062] Among them, road surface elements may include lane lines, road arrows, stop lines, parking space lines and other elements located on the road surface.

[0063] In some embodiments, the first road surface element of the semantic map is used to indicate each road surface element in the semantic map, and the second road surface element of the semantic segmentation image is used to indicate each road surface element in the semantic segmentation image.

[0064] In some embodiments, for each second road surface element, a matching element can be determined from the first road surface element in the semantic map to form an element matching pair. The first road surface element and the second road surface element in each element matching pair indicate the same road surface element in the real scene.

[0065] In some embodiments, it is possible to use Figure 6 The method described in the illustrated embodiment determines the element matching pairs, which will not be described in detail here.

[0066] Step 204 : Based on the element matching pairs, the distance field map, and the odometer data, the predicted vehicle pose of the current frame is adjusted to obtain the initial vehicle pose of the current frame.

[0067] The predicted vehicle pose for the current frame indicates the vehicle pose for the current frame predicted based on the optimized vehicle pose determined in the previous frame and the vehicle's odometer data. The predicted vehicle pose for the current frame can be obtained by adding the odometer data corresponding to the current frame to the optimized vehicle pose determined in the previous frame. The predicted vehicle pose for the initial frame can be obtained through relocalization, for example, using GPS positioning, or through positioning perception using environmental perception data.

[0068] In some embodiments, it is possible to use Figure 3 The method described in the illustrated embodiment determines the initial posture of the vehicle in the current frame, which will not be described in detail here.

[0069] Step 205 : Determine the optimized vehicle posture for the current frame based on the vehicle initial posture, optimized vehicle posture, and odometer data of a preset number of frames before the current frame, and the vehicle initial posture and odometer data of the current frame.

[0070] Among them, a sliding window method can be used to smooth and optimize the initial vehicle posture of the current frame based on the vehicle optimized posture of the historical frame and the odometer data to obtain the vehicle optimized posture of the current frame.

[0071] In an embodiment of the present disclosure, the sliding window may include the vehicle optimized posture and odometer data of a preset number of frames before the current frame. The vehicle initial posture of the current frame is smoothed and optimized based on the vehicle optimized posture and odometer data of the preset number of frames before the current frame to obtain the vehicle optimized posture of the current frame.

[0072] In some embodiments, it is possible to use Figure 4 The method described in the illustrated embodiment determines the optimized posture of the vehicle in the current frame, which will not be described in detail here.

[0073] Based on the above-mentioned embodiments of the present disclosure, by matching the first road surface element in the vehicle's environmental perception data with the second road surface element in the semantic map, the semantic map and environmental perception data can be fully utilized to effectively improve the accuracy and positioning accuracy of the determined vehicle posture; on the other hand, smoothing and correcting the single-frame positioning results based on the positioning results of multi-frame data can further improve the positioning accuracy of the vehicle positioning, help improve vehicle planning and control performance, and thereby improve the accuracy and reliability of assisted driving.

[0074] Figure 3 FIG. 2 is a flow chart of step 204 in a method for determining vehicle posture according to another exemplary embodiment of the present disclosure. Figure 3 As shown in the above Figure 2 Based on the embodiment shown, step 204 includes the following steps, each of which is described below.

[0075] Step 241 : Determine a first residual based on the predicted vehicle pose of the current frame and the adjusted initial vehicle pose of the current frame.

[0076] The adjusted initial vehicle posture ξ of the current frame is the posture obtained by adjusting the predicted vehicle posture ξ' of the current frame in step 204. The first residual can constrain the adjusted initial vehicle posture to be approximately equal to the predicted vehicle posture of the current frame.

[0077] In some embodiments, the first residual may be determined by equation (1).

[0078] e prior (ξ)=(ξ-ξ′), ξ′, ξ∈se2 Formula (1)

[0079]

[0080] In formula (1) to formula (3), represents the predicted vehicle pose of the current frame, Represents the initial posture of the vehicle in the current frame obtained by adjusting and Substituting into formula (1), we can get formula (2), e prior (ξ) represents the first residual, J acobian Indicates the derivative.

[0081] Step 242 : Project the semantic feature points of the semantic map onto the distance field map to obtain pixel values ​​of the projected points of the semantic feature points in the distance field map, and determine a second residual based on the pixel values ​​of the projected points in the distance field map and the grayscale distribution of the distance field map.

[0082] The semantic feature points in the semantic map are used to represent each pixel in the semantic map. Based on the projection transformation matrix from the vehicle coordinate system to the image coordinate system of the distance field map, the semantic feature points of the semantic map can be projected onto the distance field map.

[0083] Among them, the grayscale distribution of the distance field map is used to indicate the distribution of pixel values ​​in the distance field map, reflecting the distribution of road elements such as parking lines, lane lines, and road arrows in the image.

[0084] The process of projecting the semantic feature points of the semantic map onto the distance field map and determining the projection residual (second residual) can be seen in equations (4) to (8).

[0085]

[0086]

[0087] In formula (4) to formula (8), represents the conversion process from the point projection of the vehicle coordinate system to the image coordinates (u, v) of the distance field map, w, h represent the width and height of the semantic segmentation image and the distance field map, s (m / pixel) represents the scale from the vehicle coordinate system to the semantic segmentation image and the distance field map, m is the position of the center of the semantic segmentation image in the vehicle coordinate system, p w Represents a point in the map coordinate system, I(u,v) represents the grayscale value of the distance field map, e pc (ξ) represents the second residual.

[0088] By and Substituting into equation (4), we can obtain equation (5). Equation (7) is the formula for calculating the partial derivative of the image coordinates of the pixel point with respect to the initial vehicle pose, and equation (8) is the formula for calculating the partial derivative of the pixel value (grayscale value) of the projection point of the semantic feature point onto the distance field map with respect to the image coordinates of the pixel point. By substituting equations (7) and (8) into equation (6), we can obtain the value of the derivative of the second residual with respect to the optimized vehicle pose.

[0089] Step 243 : Determine a third residual based on the position and posture information of the first road surface element and the position and posture information of the second road surface element in the element matching pair.

[0090] The position and posture information of the first road surface element is used to represent the position and posture (orientation angle) of the first road surface element, and the position and posture information of the second road surface element is used to represent the position and posture (orientation angle) of the first road surface element.

[0091] For example, assuming that the road surface element is a parking space, the center point (x, y) of the parking space entrance line can be used as the position of the parking space, and the angle yaw along the long side of the parking space can be used as the orientation angle of the parking space.

[0092] During implementation, the residual between the position information of the first road surface element and the position information of the second road surface element can be determined by using equations (9) to (11).

[0093]

[0094] In formula (9)-formula (11), Represents the position information of the first road element in the semantic map coordinate system, represents the pose information of the second road surface element in the vehicle coordinate system, e slot (ξ) represents the third residual, where se2 is a two-dimensional Lie algebra, and its corresponding Lie group SE2 is used to represent the rigid body motion in two-dimensional space, which is constrained by the three degrees of freedom of x, y, and yaw.

[0095] By and Substituting into formula (9), we can obtain formula (10).

[0096] Step 244 : Based on the first residual, the second residual, and the third residual, the predicted vehicle pose of the current frame is adjusted to obtain the initial vehicle pose of the current frame.

[0097] In the disclosed embodiment, the predicted vehicle posture is adjusted under the constraints of the first residual, the second residual, and the third residual to obtain the initial vehicle posture of the current frame.

[0098] Specifically, a first optimization loss function can be determined based on the first residual, the second residual, and the third residual; based on the first pose optimization loss function, the predicted pose of the vehicle in the current frame is adjusted to obtain the initial pose of the vehicle in the current frame.

[0099] During implementation, the first optimization loss function can be determined by formula (12).

[0100]

[0101] By iteratively optimizing the predicted vehicle posture based on formula (12), the initial vehicle posture that makes the first optimization loss function converge can be obtained, that is, the initial vehicle posture of the current frame is obtained.

[0102] Based on the embodiments of the present disclosure, an element matching pair consisting of a first road surface element in the vehicle's environmental perception data and a second road surface element in the semantic map is implemented. The vehicle's predicted posture is adjusted using the posture offset residual (first residual), the semantic point reprojection residual (second residual) and the posture residual of the element matching pair, effectively improving the accuracy and positioning precision of the determined vehicle posture.

[0103] The initial vehicle posture obtained by adjusting the single-frame environment perception data and semantic map will cause positioning jitter due to vehicle bumps, surround perception errors, etc. Figure 4 In the illustrated embodiment, multiple frames of sliding window-based vehicle poses are introduced for smoothing to reduce the positioning jitter problem. Figure 4 FIG. 2 is a flow chart of step 205 in a method for determining vehicle posture provided by another exemplary embodiment of the present disclosure. Figure 2 Based on the illustrated embodiment, step 205 may include the following steps.

[0104] Step 251 : Determine a second posture optimization loss function based on the vehicle initial posture, vehicle optimized posture, and odometer data of a preset number of frames, and the vehicle initial posture and odometer data of the current frame.

[0105] Among them, the preset number of frames is a preset number of frames located before the current frame. For example, if the preset number is n, the vehicle's initial posture of the current frame is the vehicle's initial posture determined based on the environmental perception data collected at time t, then the vehicle's optimized posture of the preset number of frames can be the posture obtained by optimizing the vehicle's initial posture determined based on the environmental perception data collected at time t-1, t-2, t-3...tn.

[0106] In some embodiments, a fourth residual can be determined based on the vehicle's initial posture and the vehicle's optimized posture of a preset number of frames, as well as the vehicle's initial posture of the current frame and the vehicle's optimized posture of the current frame; a fifth residual can be determined based on the odometer increment corresponding to two consecutive frames of odometer data and the posture increment of two consecutive frames; and a second posture optimization loss function can be determined based on the fourth residual and the fifth residual.

[0107] Specifically, the fourth residual can be used to constrain that the optimized vehicle posture of the current frame obtained by sliding window optimization will not differ too much from the initial vehicle posture of the current frame. The fifth residual can be used to constrain that the odometer increment of the odometer data collected from two consecutive frames will not differ too much from the posture increment of the optimized vehicle posture of two consecutive frames.

[0108] During implementation, the fourth residual and the fifth residual can be determined by equations (13) and (14), respectively.

[0109] e prior (ξ″)=||ξ-ξ″||, ξ″, ξ″∈se2 Equation (13)

[0110] In formula (13), ξ″=[x″, y″, yaw″] represents the optimized pose of the vehicle in the current frame, and ξ represents the initial pose of the vehicle in the current frame.

[0111] e odo =ln(ΔTodoexp(ξ i-1 )exp(ξ i ) -1 ) Formula (14)

[0112] In formula (14), ΔT odo Represents the pose delta between the optimized poses of the vehicle in two consecutive frames.

[0113] According to the fourth residual and the fifth residual, the second pose optimization loss function can be determined, as shown in formula (15).

[0114]

[0115] By performing iterative optimization and solving the above formula (15), the optimized vehicle posture after multi-frame sliding optimization can be obtained.

[0116] See also Figure 7, which illustrates a schematic diagram of pose optimization through a sliding window. The single-frame prior pose indicated by label 71 is used to indicate the initial pose of the vehicle in a single frame, which is a priori information for determining the pose optimization variable (label 73). The odometry increment (label 72) is used to constrain the pose increment between the optimized poses of the vehicle in two adjacent frames to not deviate too much from the odometry increment. Figure 7 The illustrated process can realize sliding optimization of the initial posture of the vehicle to obtain the optimized posture of the vehicle.

[0117] Exemplarily, the sliding window includes 5 frames of vehicle optimized posture and odometer data. If the current frame is the 8th frame, the sliding window includes the vehicle optimized posture, odometer data of frames 4-7 and the vehicle initial posture and odometer data of frame 8. The fourth residual can be obtained by calculating the residual between the vehicle optimized pose and the vehicle initial pose of the 4th frame, the residual between the vehicle optimized pose and the vehicle initial pose of the 5th frame, the residual between the vehicle optimized pose and the vehicle initial pose of the 6th frame, and the residual between the vehicle optimized pose (to be solved) and the vehicle initial pose of the 8th frame (current frame), and summing the residuals corresponding to each frame; the fifth residual can be obtained by calculating the odometer increment corresponding to the odometer data of the 4th and 5th frames and the pose increment of the 4th and 5th frames, the odometer increment corresponding to the odometer data of the 5th and 6th frames and the pose increment of the 5th and 6th frames, the odometer increment corresponding to the odometer data of the 6th and 7th frames and the pose increment of the 6th and 7th frames, the odometer increment corresponding to the odometer data of the 7th and 8th frames and the pose increment of the 7th and 8th frames, and summing the residuals corresponding to the odometer increment and pose increment corresponding to each frame.

[0118] Step 252: Based on the second posture optimization loss function, the initial posture of the vehicle in the current frame is adjusted to obtain the optimized posture of the vehicle in the current frame.

[0119] Based on the embodiment of the present disclosure, by simultaneously constraining the sliding adjustment process through the fourth residual and the fifth residual, it is possible to avoid positioning jitter caused by vehicle bumps, surround perception errors, etc. in the optimized vehicle posture obtained in different frames. It is also possible to avoid the problem of inaccurate positioning caused by the cumulative error of the odometer data acquisition sensor, thereby ensuring that the obtained optimized vehicle posture can meet the requirements of smoothness and small error, and further improving the accuracy of the vehicle posture.

[0120] Figure 5 FIG. 2 is a flow chart of step 202 in a method for determining vehicle posture according to another exemplary embodiment of the present disclosure. Figure 5 As shown in the above Figure 2 Based on the illustrated embodiment, the operation of determining the corresponding distance field map according to the semantic segmentation image in step 202 may include the following steps.

[0121] Step 221: Determine a first transformation map corresponding to the semantic segmentation image based on the distance from the non-zero pixel to the zero pixel in the semantic segmentation image; and determine a second transformation map corresponding to the semantic segmentation image based on the distance from the zero pixel to the non-zero pixel in the semantic segmentation image.

[0122] In some embodiments, non-zero pixels in a semantic segmentation image indicate that they have a certain semantic category. Different semantic segmentation images can be obtained for different semantic segmentation categories. For example, by performing semantic segmentation on lane lines, a semantic segmentation image corresponding to the lane lines can be obtained; by performing semantic segmentation on road arrows, a semantic segmentation image corresponding to the road arrows can be obtained.

[0123] In some embodiments, since each semantic point in a semantic segmentation image typically forms a semantic point region with width, for each semantic segmentation image, a DT (Distance Transform) map can be calculated twice on the semantic segmentation image. By calculating the distance from non-zero pixels to zero pixels, a first transformation map D1 is obtained. The first transformation map D1 has a gradient change within the semantic point region with width. By calculating the distance from zero pixels to non-zero pixels, a second transformation map D2 is obtained. The second transformation map D2 has a gradient change outside the semantic point region with width.

[0124] Step 222: Determine a distance field map corresponding to the semantically segmented image based on the first transformation map, the second transformation map, and a preset constant value.

[0125] In some embodiments, D1 and D2 can be superimposed together to generate a final distance field map D3. The superimposed distance field map has gradient changes both inside and outside the semantic point area, ensuring that the subsequent iterative optimization solution of semantic point projection based on the distance field map can be optimized to the center position of the semantic point area with width.

[0126] In implementation, D1 and D2 can be superimposed using formula (16) to obtain the distance field map corresponding to the semantic segmentation image.

[0127] D3=D1-D2+k Formula (16)

[0128] In formula (16), k is a fixed constant related to the image resolution and scale of the environmental perception data. For example, it can be 10. 10 is an empirical value obtained by parameter adjustment and is a calibrable value.

[0129] Based on the embodiments of the present disclosure, by determining the first transformation map and the second transformation map corresponding to the semantic segmentation image inwardly and outwardly respectively, and then superimposing and fusing them, it is possible to generate a distance field map for road elements with width (such as lane lines and parking space lines), thereby providing support for the subsequent optimization of vehicle posture based on the distance field map.

[0130] Figure 6 This is a flow chart of step 203 in the vehicle posture determination method provided by another exemplary embodiment of the present disclosure. Figure 2 Based on the illustrated embodiment, step 203 includes the following steps.

[0131] Step 231: Determine the second road surface element and the second element identifier corresponding to the second road surface element included in the environmental perception data based on the semantic segmentation image; and determine the first road surface element and the first element identifier corresponding to the first road surface element based on the semantic map.

[0132] After collecting environmental perception data in real time and generating a semantic segmentation image, a corresponding second element identifier can be determined for each second road surface element in the semantic segmentation image. When constructing or acquiring the semantic map on the vehicle side, a corresponding first element identifier has been set for each first road surface element.

[0133] Step 232: Determine an element matching pair based on the first element identifier and the second element identifier.

[0134] In some embodiments, each time after determining the semantic segmentation image corresponding to a frame of environmental perception data, a first road surface element that matches the second element identifier can be searched in an element matching mapping table; in response to the absence of a first road surface element corresponding to the second element identifier, the posture deviation between the second road surface element and the first road surface element is calculated; based on the posture deviation, the first road surface element that matches the second road surface element is determined.

[0135] The element matching mapping table records the second element identifier and the first element identifier of the matched element matching pair. After acquiring a semantic segmentation image corresponding to a frame of environmental perception data, the element matching mapping table can be used to first check whether a first road surface element matches a second road surface element in the currently acquired semantic segmentation image. If so, the road surface element information of the matching first road surface element, such as the first road surface element's position and posture, can be directly obtained from the element matching mapping table. If not, the position and posture deviation between the second road surface element and each first road surface element can be calculated, and the first road surface element with a deviation less than a preset distance threshold can be determined as an element that matches the second road surface element, forming an element matching pair.

[0136] In the embodiment of the present disclosure, in order to improve the accuracy of element matching, the position deviation between the second road surface element and the first road surface element can be determined for multiple consecutive frames (such as three consecutive frames). If the position deviation between the second road surface element and the same first road surface element is less than a preset distance threshold for multiple consecutive frames (such as three consecutive frames), the first road surface element can be determined as the element that matches the second road surface element.

[0137] During implementation, the position deviation between the second road surface element and each first road surface element can be determined by formula (17).

[0138]

[0139] In formula (17), V sm represents the pose of the road element in the semantic map, ξ represents the predicted pose of the vehicle in the current frame, and V sp Indicates the pose of the second road surface element. Defined as the addition of se2 poses.

[0140] The three-dimensional Lie algebra se2 used in the above formulas, such as formula (1), formula (13), and formula (15), can be realized by formula (18).

[0141]

[0142] In formula (18), A and B are two vectors for performing se2 calculation.

[0143] After determining the position deviation between the first road surface element and the second road surface element by using the above formula (17), a matching element pair can be determined according to the value of the position deviation.

[0144] Based on the embodiments of the present disclosure, it is possible to determine an element matching pair between a first road surface element in a semantic map and a second road surface element in a semantic segmentation image according to the posture deviation, which helps to subsequently optimize the vehicle posture according to the element matching pair.

[0145] Any vehicle posture determination method provided in the embodiments of the present disclosure can be executed by any appropriate device with data processing capabilities, including but not limited to a terminal device and a server. Alternatively, any vehicle posture determination method provided in the embodiments of the present disclosure can be executed by a processor, such as a processor that executes any vehicle posture determination method mentioned in the embodiments of the present disclosure by invoking corresponding instructions stored in a memory. This will not be further described below.

[0146] Exemplary devices

[0147] Figure 9 FIG is a structural diagram of a vehicle posture determination device provided by an exemplary embodiment of the present disclosure. Figure 9 As shown, the vehicle posture determination device may include:

[0148] Information acquisition module 91, used to obtain semantic maps, environmental perception data and odometer data;

[0149] A distance field map determination module 92 is configured to process the environmental perception data to obtain a semantic segmentation image containing semantic information and a distance field map corresponding to the semantic segmentation image;

[0150] A matching module 93 is configured to match the first road surface element in the semantic map with the second road surface element in the semantic segmentation image to obtain an element matching pair;

[0151] A pose adjustment module 94 is configured to adjust the predicted pose of the vehicle in the current frame based on the element matching pairs, the distance field map, and the odometer data to obtain an initial pose of the vehicle in the current frame;

[0152] The posture smoothing module 95 is used to determine the optimized posture of the vehicle in the current frame based on the vehicle initial posture, vehicle optimized posture, and odometer data of a preset number of frames before the current frame, as well as the vehicle initial posture and odometer data of the current frame.

[0153] The information acquisition module 91 may include a processing module and a data acquisition device. The data acquisition device may be any type of acquisition sensor, such as a camera, a still camera, an odometer data acquisition sensor, or a lidar. Any sensor capable of acquiring image, video, or point cloud data is applicable to this embodiment. The processing module may determine a semantic map based on the data collected by the acquisition device.

[0154] The distance field map determining module 92 may be a functional module / processing module in a processor. The processing module may call an image processing algorithm stored in a memory to identify and process the environment perception data to obtain a distance field map.

[0155] The matching module 93, the posture adjustment module 94, and the posture smoothing module 95 can all be functional modules of the processor, or independent processing chips. The processing chips and functional modules can establish connections through electrical connections to transmit data or instructions.

[0156] Figure 10 FIG is a structural diagram of a vehicle posture determination device provided by another exemplary embodiment of the present disclosure. Figure 10 As shown in the above Figure 9 Based on the illustrated embodiment, in some implementations, the posture adjustment module 94 may include:

[0157] A first determining unit 941 is configured to determine a first residual based on the predicted vehicle pose of the current frame and the adjusted initial vehicle pose of the current frame;

[0158] A second determining unit 942 is configured to project the semantic feature point of the semantic map onto the distance field map, obtain a pixel value of the projected point of the semantic feature point in the distance field map, and determine a second residual based on the pixel value of the projected point in the distance field map and the grayscale distribution of the distance field map;

[0159] a third determining unit 943 , configured to determine a third residual based on the position and posture information of the first road surface element and the position and posture information of the second road surface element in the element matching pair;

[0160] The first adjustment unit 944 is used to adjust the predicted vehicle posture of the current frame based on the first residual, the second residual and the third residual to obtain the initial vehicle posture of the current frame.

[0161] In some embodiments, the first adjustment unit 944 can be used to: determine a first optimization loss function based on the first residual, the second residual, and the third residual; adjust the predicted vehicle posture of the current frame based on the first posture optimization loss function to obtain the initial vehicle posture of the current frame.

[0162] In some embodiments, the pose smoothing module 95 may include:

[0163] a fourth determining unit 951 for determining a second posture optimization loss function based on the vehicle initial posture, vehicle optimized posture, and odometer data of a preset number of frames, and the vehicle initial posture and odometer data of a current frame;

[0164] The second adjustment unit 952 is used to adjust the initial posture of the vehicle in the current frame based on the second posture optimization loss function to obtain the optimized posture of the vehicle in the current frame.

[0165] In some embodiments, the fourth determination unit 951 can be used to: determine a fourth residual based on the vehicle's initial posture and the vehicle's optimized posture of a preset number of frames, and the vehicle's initial posture of the current frame and the vehicle's optimized posture of the current frame; determine a fifth residual based on the odometer increment corresponding to two consecutive frames of odometer data and the posture increment of two consecutive frames; determine the second posture optimization loss function based on the fourth residual and the fifth residual.

[0166] In some implementations, the distance field map determination module 92 may include:

[0167] The fifth determining unit 921 is configured to determine a first transformation map corresponding to the semantic segmentation image based on the distance between the non-zero pixel and the zero pixel in the semantic segmentation image; and determine a second transformation map corresponding to the semantic segmentation image based on the distance between the zero pixel and the non-zero pixel in the semantic segmentation image.

[0168] The sixth determining unit 922 is configured to determine a distance field map corresponding to the semantically segmented image based on the first transformation map, the second transformation map, and a preset constant value.

[0169] In some embodiments, the matching module 93 may include:

[0170] The seventh determining unit 931 is configured to determine, based on the semantic segmentation image, a second road surface element and a second element identifier corresponding to the second road surface element included in the environmental perception data; and determine, based on the semantic map, a first road surface element and a first element identifier corresponding to the first road surface element;

[0171] The eighth determining unit 932 is configured to determine an element matching pair based on the first element identifier and the second element identifier.

[0172] In some embodiments, the eighth determination unit 932 is used to: search for a first pavement element that matches a second element identifier in an element matching mapping table; in response to the absence of a first pavement element corresponding to the second element identifier, calculate a posture deviation between the second pavement element and the first pavement element; and determine a first pavement element that matches the second pavement element based on the posture deviation.

[0173] The beneficial technical effects corresponding to the exemplary embodiment of this device can be found in the corresponding beneficial technical effects of the above exemplary method part, which will not be repeated here.

[0174] Exemplary electronic devices

[0175] Figure 11 A structural diagram of an electronic device provided in an embodiment of the present disclosure includes at least one processor 11 and a memory 12.

[0176] The processor 11 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0177] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), a hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may execute one or more computer program instructions to implement the vehicle posture determination method and / or other desired functions of the various embodiments of the present disclosure described above.

[0178] In one example, the electronic device may further include an input device 13 and an output device 14 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0179] The input device 13 may also include, for example, a keyboard, a mouse, etc.

[0180] The output device 14 can output various information to the outside, and may include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0181] Of course, to simplify, Figure 11 Only some of the components related to the present disclosure in the electronic device are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, the electronic device may further include any other appropriate components according to specific application scenarios.

[0182] Exemplary computer program products and computer-readable storage media

[0183] In addition to the above-mentioned methods and devices, embodiments of the present disclosure may also provide a computer program product, including computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the vehicle posture determination method of various embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section.

[0184] The computer program product may be written in any combination of one or more programming languages ​​to implement the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, and conventional procedural programming languages ​​such as C or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0185] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, the processor executes the steps in the vehicle posture determination method of various embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section.

[0186] Computer readable storage media can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium is, for example, but not limited to, a system, device or component comprising electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0187] The basic principles of the present disclosure have been described above in conjunction with specific embodiments. However, the advantages, strengths, and effects mentioned in this disclosure are merely illustrative and not restrictive, and should not be considered as essential to each embodiment of the present disclosure. Furthermore, the specific details disclosed above are provided for illustrative purposes and to facilitate understanding, rather than as limitations. These details do not limit the present disclosure to necessarily being implemented using these specific details.

[0188] Those skilled in the art may make various changes and modifications to the present disclosure without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is intended to include these modifications and variations.

Claims

1. A method for determining a vehicle posture, comprising: Obtain semantic maps, environmental perception data, and odometry data; Processing the environmental perception data to obtain a semantic segmentation image containing semantic information and a distance field map corresponding to the semantic segmentation image; matching a first road surface element in the semantic map with a second road surface element in the semantic segmentation image to obtain an element matching pair; Adjusting the predicted vehicle pose of the current frame based on the element matching pairs, the distance field map, and the odometer data to obtain an initial vehicle pose of the current frame; The optimized vehicle posture of the current frame is determined based on the vehicle initial posture, vehicle optimized posture, and odometer data of a preset number of frames before the current frame, and the vehicle initial posture and odometer data of the current frame.

2. The method according to claim 1, wherein The adjusting the predicted vehicle pose of the current frame based on the element matching pairs, the distance field map, and the odometer data to obtain the initial vehicle pose of the current frame includes: Determining a first residual based on the predicted vehicle pose of the current frame and the adjusted initial vehicle pose of the current frame; Projecting the semantic feature point of the semantic map onto the distance field map to obtain a pixel value of the projection point of the semantic feature point in the distance field map, and determining a second residual based on the pixel value of the projection point in the distance field map and the grayscale distribution of the distance field map; determining a third residual based on the pose information of the first road surface element and the pose information of the second road surface element in the element matching pair; Based on the first residual, the second residual, and the third residual, the predicted vehicle pose of the current frame is adjusted to obtain the initial vehicle pose of the current frame.

3. The method according to claim 2, wherein: The adjusting the predicted vehicle pose of the current frame based on the first residual, the second residual, and the third residual to obtain the initial vehicle pose of the current frame includes: Determining a first optimization loss function based on the first residual, the second residual, and the third residual; Based on the first posture optimization loss function, the predicted posture of the vehicle in the current frame is adjusted to obtain the initial posture of the vehicle in the current frame.

4. The method according to any one of claims 1 to 3, wherein: Determining the optimized vehicle pose of the current frame based on the vehicle initial pose, vehicle optimized pose, odometer data of a preset number of frames before the current frame, and the vehicle initial pose and odometer data of the current frame includes: Determining a second posture optimization loss function based on the vehicle initial posture, vehicle optimized posture, and odometer data of the preset number of frames, and the vehicle initial posture and odometer data of the current frame; Based on the second posture optimization loss function, the initial posture of the vehicle in the current frame is adjusted to obtain the optimized posture of the vehicle in the current frame.

5. The method according to claim 4, wherein Determining a second posture optimization loss function based on the vehicle initial posture, vehicle optimized posture, and odometer data of the preset number of frames, and the vehicle initial posture and odometer data of the current frame, includes: Determining a fourth residual based on the vehicle initial pose and the vehicle optimized pose of the preset number of frames, and the vehicle initial pose of the current frame and the vehicle optimized pose of the current frame; Determining a fifth residual based on an odometer increment corresponding to two consecutive frames of odometer data and a pose increment of the two consecutive frames; Determine the second pose optimization loss function based on the fourth residual and the fifth residual.

6. The method according to any one of claims 1 to 5, wherein: Processing the environmental perception data to obtain a distance field map corresponding to the semantic segmentation image includes: Determining a first transformation map corresponding to the semantic segmentation image based on a distance from a non-zero pixel to a zero pixel in the semantic segmentation image; Determining a second transformation map corresponding to the semantic segmentation image based on a distance from a zero-point pixel to a non-zero-point pixel in the semantic segmentation image; The distance field map corresponding to the semantic segmentation image is determined based on the first transformation map, the second transformation map, and a preset constant value.

7. The method according to any one of claims 1 to 6, wherein: The matching of the first road surface element of the semantic map and the second road surface element of the semantic segmentation image to obtain an element matching pair includes: determining, based on the semantic segmentation image, a second road surface element included in the environmental perception data and a second element identifier corresponding to the second road surface element; determining, based on the semantic map, the first road surface element and a first element identifier corresponding to the first road surface element; The element matching pair is determined based on the first element identifier and the second element identifier.

8. The method according to claim 7, wherein: The determining the element matching pair based on the first element identifier and the second element identifier includes: Searching an element matching mapping table for a first road surface element that matches the second element identifier; In response to the absence of the first road surface element corresponding to the second element identifier, calculating a position deviation between the second road surface element and the first road surface element; Based on the position deviation, the first road surface element that matches the second road surface element is determined.

9. A vehicle posture determination device, comprising: Information acquisition module, used to obtain semantic maps, environmental perception data and odometry data; a distance field map determination module, configured to process the environmental perception data to obtain a semantic segmentation image containing semantic information and a distance field map corresponding to the semantic segmentation image; a matching module, configured to match a first road surface element in the semantic map with a second road surface element in the semantic segmentation image to obtain an element matching pair; A posture adjustment module is used to adjust the predicted posture of the vehicle in the current frame based on the element matching pair, the distance field map and the odometer data to obtain the initial posture of the vehicle in the current frame; A posture smoothing module is used to determine the optimized posture of the vehicle in the current frame based on the vehicle initial posture, vehicle optimized posture, and odometer data of a preset number of frames before the current frame, as well as the vehicle initial posture and odometer data of the current frame.

10. A computer-readable storage medium storing a computer program for executing the vehicle posture determination method according to any one of claims 1 to 8.

11. An electronic device, comprising: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the vehicle posture determination method described in any one of claims 1-8 above.

Images