Position detection method, device, electronic device and storage medium

The posture detection method enhances laser positioning in autonomous driving by segmenting point cloud images and generating local maps, ensuring accurate vehicle positioning even in GPS-challenged environments.

CN114549633BActive Publication Date: 2024-12-03BEIJING XIAOMI MOBILE SOFTWARE CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111089402.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-09-16
Publication Date
2024-12-03
Estimated Expiration
2041-09-16

AI Technical Summary

Technical Problem

Existing laser positioning technologies in autonomous driving face challenges in areas with poor satellite signals and are affected by environmental obstructions, leading to low positioning accuracy.

Method used

A posture detection method involving semantic segmentation of point cloud images to extract target elements, matching these elements with vector maps, and utilizing vehicle-mounted radar to generate local point cloud maps for accurate vehicle positioning, even in areas with insufficient vector map coverage.

Benefits of technology

Improves positioning efficiency and accuracy by using lightweight vector maps and inertial device adjustments, maintaining high precision in environments with weak or no GPS signals.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114549633B_ABST
    Figure CN114549633B_ABST
Patent Text Reader

Abstract

The present application proposes a posture detection method, device, electronic device and storage medium, wherein the method includes: obtaining a first point cloud map, performing semantic segmentation based on the first point cloud map to extract target points belonging to object elements from the first point cloud map, matching the positions of the target points belonging to the object elements in the first point cloud map with the object elements marked in the vector map, and performing posture detection on the target vehicle from which the first point cloud map is acquired based on the positions of the target elements matching the positions of the target points in the vector map in the world coordinate system. In an embodiment of the present application, when there are target elements in the vector map that match the positions of the target points belonging to the object elements in the point cloud map, posture detection is performed on the target vehicle based on the positions of the target elements in the world coordinate system. Because a lightweight vector map is used for positioning, the efficiency and cost of positioning are improved while meeting the positioning accuracy compared to the situation where a large-volume positioning layer map is required.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to a posture detection method, device, electronic device and storage medium. Background Art

[0002] High-precision positioning is a key technology that can be used in autonomous driving. Related positioning technologies include visual positioning, laser positioning, and satellite navigation positioning based on GNSS (Global Navigation Satellite System) and INS (Inertial Navigation System). Among them, satellite navigation positioning has poor positioning effect in areas with poor satellite signals. Compared with laser positioning, visual positioning is easily affected by light and obstructions, and has low positioning accuracy. Therefore, laser positioning is one of the main positioning technologies, and how to improve the positioning effect of laser positioning is a technical problem that needs to be solved urgently. Summary of the invention

[0003] The present application aims to solve one of the technical problems in the related art at least to some extent.

[0004] The first embodiment of the present application proposes a posture detection method, including:

[0005] Get the first point cloud image;

[0006] Performing semantic segmentation according to the first point cloud image to extract target points belonging to object elements from the first point cloud image;

[0007] Positionally matching the target points belonging to the object elements in the first point cloud with the object elements marked in the vector map;

[0008] According to the position of the target element in the vector map that matches the position of the target point in the world coordinate system, the position and posture of the target vehicle acquired by the first point cloud image is detected.

[0009] In some embodiments of the present application, after position matching of the target points belonging to the object elements in the first point cloud map with the object elements marked in the vector map, the method further includes:

[0010] When there is no target element matching the position of the target point in the vector map, obtaining a plurality of frames of second point cloud images collected by the vehicle-mounted radar; wherein the plurality of frames of second point cloud images are collected within a set period of time before the first point cloud images are collected;

[0011] synthesizing the multiple frames of second point cloud images in the world coordinate system according to the predicted position of each sampling point in the multiple frames of second point cloud images in the world coordinate system to obtain a local point cloud map;

[0012] Position matching of each sampling point in the first point cloud map with a virtual plane where each sampling point in the local point cloud map is located;

[0013] The target vehicle is subjected to posture detection according to the position of the virtual plane matched by each sampling point in the first point cloud image in the world coordinate system.

[0014] In some embodiments of the present application, before position matching of each sampling point in the first point cloud map with the virtual plane where each sampling point in the local point cloud map is located, the method further includes:

[0015] Determine normal information of each sampling point in the local point cloud map according to the positions of each sampling point and its adjacent points in the local point cloud map in the world coordinate system;

[0016] According to the normal information of each sampling point in the local point cloud map, a virtual plane where each sampling point in the local point cloud map is located is determined.

[0017] In some embodiments of the present application, performing posture detection on the target vehicle acquired by acquiring the first point cloud image according to the position of the target element matching the target point position in the vector map in the world coordinate system includes:

[0018] Determine the position of the target point in the world coordinate system according to a posture observation value obtained by detecting the posture of the target vehicle in the world coordinate system according to the on-board inertial device of the target vehicle, and according to the relative position of the target point in the first point cloud image;

[0019] Determining a value of a first objective function according to a residual between a position of the target element in the vector map in a world coordinate system and a position of the target point in the world coordinate system;

[0020] According to the value of the first objective function, adjusting the position and posture observation value of the vehicle-mounted inertial device of the target vehicle to minimize the value of the first objective function;

[0021] According to the adjusted posture observation value, the predicted posture of the target vehicle at the sampling time corresponding to the first point cloud image is determined.

[0022] In some embodiments of the present application, the detecting of the position and posture of the target vehicle according to the position of the virtual plane matched by each sampling point in the first point cloud image in the world coordinate system includes:

[0023] Determine the position of each sampling point in the first point cloud image in the world coordinate system according to the position observation value obtained by detecting the position of the target vehicle in the world coordinate system according to the on-board inertial device of the target vehicle and the relative position of each sampling point in the first point cloud image;

[0024] Determine a value of a second objective function according to a residual between a position of the virtual plane in a world coordinate system and positions of each sampling point matched in the first point cloud image in the world coordinate system;

[0025] According to the value of the second objective function, adjusting the position and posture observation value of the vehicle-mounted inertial device of the target vehicle to minimize the value of the second objective function;

[0026] According to the adjusted posture observation value, the predicted posture of the target vehicle at the sampling time corresponding to the first point cloud image is determined.

[0027] In some embodiments of the present application, the value of the first objective function or the second objective function is positively correlated with at least one of the following:

[0028] The residual between the speed value in the posture observation value and the speed value detected by the wheel speed meter of the target vehicle;

[0029] The relative posture between the predicted posture at the sampling moment and the predicted posture at the historical moment, and the residual between the posture observation values ​​accumulated from the sampling moment to the historical moment.

[0030] In some embodiments of the present application, obtaining the first point cloud image includes:

[0031] Obtain a multi-frame point cloud image recently collected by the on-board radar of the target vehicle;

[0032] The most recently acquired multi-frame point cloud images are superimposed to obtain the first point cloud image.

[0033] The second aspect of the present application provides a posture detection device, including:

[0034] An acquisition module, used for acquiring a first point cloud image;

[0035] an extraction module, configured to perform semantic segmentation according to the first point cloud image, so as to extract target points belonging to object elements from the first point cloud image;

[0036] A matching module, used for matching the target points belonging to the object elements in the first point cloud image with the object elements marked in the vector map;

[0037] The first detection module is used to perform posture detection on the target vehicle obtained by collecting the first point cloud image according to the position of the target element matching the target point position in the vector map in the world coordinate system.

[0038] In some embodiments of the present application, the device further includes:

[0039] The acquisition module is further configured to acquire a plurality of frames of second point cloud images acquired by the vehicle-mounted radar when there is no target element matching the position of the target point in the vector map; wherein the plurality of frames of second point cloud images are acquired within a set period of time before the acquisition of the first point cloud image;

[0040] A synthesis module, configured to synthesize the plurality of frames of second point cloud images in the world coordinate system according to the predicted position of each sampling point in the plurality of frames of second point cloud images in the world coordinate system, so as to obtain a local point cloud map;

[0041] The matching module is further used to match the positions of the sampling points in the first point cloud map with the virtual planes where the sampling points in the local point cloud map are located;

[0042] The second detection module is further used to perform posture detection on the target vehicle according to the position of the virtual plane matched by each sampling point in the first point cloud image in the world coordinate system.

[0043] In some embodiments of the present application, the device further includes:

[0044] A determination module is used to determine the normal information of each sampling point in the local point cloud map according to the positions of each sampling point and its adjacent points in the local point cloud map in the world coordinate system; and determine the virtual plane where each sampling point in the local point cloud map is located according to the normal information of each sampling point in the local point cloud map.

[0045] In some embodiments of the present application, the first detection module is further used to:

[0046] Determine the position of the target point in the world coordinate system according to a posture observation value obtained by detecting the posture of the target vehicle in the world coordinate system according to the on-board inertial device of the target vehicle, and according to the relative position of the target point in the first point cloud image;

[0047] Determining a value of a first objective function according to a residual between a position of the target element in the vector map in a world coordinate system and a position of the target point in the world coordinate system;

[0048] According to the value of the first objective function, adjusting the position and posture observation value of the vehicle-mounted inertial device of the target vehicle to minimize the value of the first objective function;

[0049] According to the adjusted posture observation value, the predicted posture of the target vehicle at the sampling time corresponding to the first point cloud image is determined.

[0050] In some embodiments of the present application, the second detection module is further used to:

[0051] Determine the position of each sampling point in the first point cloud image in the world coordinate system according to the position observation value obtained by detecting the position of the target vehicle in the world coordinate system according to the on-board inertial device of the target vehicle and the relative position of each sampling point in the first point cloud image;

[0052] Determine a value of a second objective function according to a residual between a position of the virtual plane in a world coordinate system and positions of each sampling point matched in the first point cloud image in the world coordinate system;

[0053] According to the value of the second objective function, adjusting the position and posture observation value of the vehicle-mounted inertial device of the target vehicle to minimize the value of the second objective function;

[0054] According to the adjusted posture observation value, the predicted posture of the target vehicle at the sampling time corresponding to the first point cloud image is determined.

[0055] In some embodiments of the present application, the value of the first objective function or the second objective function is positively correlated with at least one of the following:

[0056] The residual between the speed value in the posture observation value and the speed value detected by the wheel speed meter of the target vehicle;

[0057] The relative posture between the predicted posture at the sampling moment and the predicted posture at the historical moment, and the residual between the posture observation values ​​accumulated from the sampling moment to the historical moment.

[0058] In some embodiments of the present application, the acquisition module is specifically used to:

[0059] Obtain a multi-frame point cloud image recently collected by the on-board radar of the target vehicle;

[0060] The most recently acquired multi-frame point cloud images are superimposed to obtain the first point cloud image.

[0061] The third aspect of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in the first aspect is implemented.

[0062] The fourth aspect of the present application provides a non-temporary computer-readable storage medium having a computer program stored thereon, which implements the method described in the first aspect when the computer program is executed by a processor.

[0063] A fifth aspect of the present application provides a computer program product, including computer instructions, which, when executed by a processor, implement the method described in the first aspect.

[0064] The technical solution provided by the embodiment of the present application has the following beneficial effects:

[0065] Obtain a first point cloud map, perform semantic segmentation based on the first point cloud map to extract target points belonging to object elements from the first point cloud map, match the positions of the target points belonging to object elements in the first point cloud map with the object elements marked in the vector map, and perform posture detection on the target vehicle from which the first point cloud map is acquired based on the position of the target elements matching the positions of the target points in the vector map in the world coordinate system. In an embodiment of the present application, when there are target elements in the vector map that match the positions of the target points belonging to object elements in the point cloud map, posture detection is performed on the target vehicle based on the positions of the target elements in the world coordinate system. Because a lighter vector map is used for positioning, the efficiency and cost of positioning are improved while meeting the positioning accuracy, compared to the situation where a large amount of positioning layer maps need to be specially produced.

[0066] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0068] Figure 1 A flow chart of a posture detection method provided in an embodiment of the present application;

[0069] Figure 2 A schematic diagram of each target point in the first point cloud map provided in an embodiment of the present application;

[0070] Figure 3 A schematic diagram of matching a target element between a first point cloud image and a vector map provided in an embodiment of the present application;

[0071] Figure 4 A flowchart of another posture detection method provided in an embodiment of the present application;

[0072] Figure 5 A flowchart of another posture detection method provided in an embodiment of the present application;

[0073] Figure 6 A schematic diagram of the result of position matching between each sampling point in the first point cloud map and the virtual plane where each sampling point in the local point cloud map is located, provided in an embodiment of the present application;

[0074] Figure 7 A flowchart of another posture detection method provided in an embodiment of the present application;

[0075] Figure 8 A schematic diagram of the structure of a posture detection device provided in an embodiment of the present application;

[0076] Fig. 9 A block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0077] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0078] The following describes the posture detection method, device, electronic device and storage medium of the embodiments of the present application with reference to the accompanying drawings.

[0079] Figure 1 A flow chart of a posture detection method provided in an embodiment of the present application.

[0080] like Figure 1 As shown, the method comprises the following steps:

[0081] Step 101: Obtain a first point cloud image.

[0082] The point cloud image is generated by acquiring point cloud data through data collection by a three-dimensional laser scanner, for example, a laser radar sensor, wherein a set of point data on the appearance surface of an object is point cloud data.

[0083] In one implementation of the embodiment of the present application, a frame of point cloud image most recently collected by an on-board radar of the target vehicle is obtained as a first point cloud image.

[0084] In another implementation of the embodiment of the present application, in order to increase the amount of information contained in the point cloud image, multiple frames of point cloud images recently collected by the on-board radar of the target vehicle are obtained, for example, 4 frames, and the recently collected multiple frames of point cloud images are superimposed to obtain a first point cloud image. As an implementation method, superposition of multiple frames of point cloud images refers to merging the data of multiple frames of point cloud images into one point cloud image to increase the amount of information contained in the first point cloud image.

[0085] Step 102: performing semantic segmentation according to the first point cloud image to extract target points belonging to the object elements from the first point cloud image.

[0086] The object elements include, for example, lane lines, poles, road surfaces, etc.

[0087] In one implementation of the embodiment of the present application, RangeNet++ is used for semantic segmentation, and a class label is assigned to each data point in the input first point cloud image through semantic segmentation to extract and mark the point cloud areas corresponding to object elements such as lane lines and poles. Figure 2 A schematic diagram of each target point in the first point cloud provided in an embodiment of the present application; Figure 2 As shown, the points in the gray circles represent lane lines, and the points in the white circles represent poles.

[0088] Among them, RangeNet++ calculates the nearest neighbors of each point in constant time and uses CPU-based computing to accurately infer the complete semantic segmentation of the lidar point cloud, which is faster than the sensor's frame rate and can run on any range image-based convolutional neural network (CNN) backbone.

[0089] Step 103 , position matching is performed between target points belonging to object elements in the first point cloud image and each object element marked in the vector map.

[0090] Among them, vector maps use straight lines and curves to describe graphics. The object elements of these graphics are described by some points, lines, rectangles, polygons, circles and arcs, etc., which are all obtained by mathematical formula calculation. Since vector maps can be obtained by formula calculation, the file size of vector maps is generally small. Compared with the rasterized point cloud Normal Distribution Transform (NDT) map, the vector map data volume is small and occupies less storage space.

[0091] In the embodiment of the present application, the first point cloud is composed of point data, and the object elements contained in the first point cloud are also composed of multiple point data. The target point belonging to the object element has been determined through semantic segmentation, and the position of the target point belonging to the object element in the first point cloud can be determined according to the position of the target point in the first point cloud. In the vector map, each object element has also been marked, so the position of each object element in the vector map can be determined.

[0092] Therefore, in the same coordinate system, the position of the target point belonging to the object element in the first point cloud image and the position of each object element marked in the vector map are matched to determine whether there is a target element in the vector map that matches the position of the target point. Since the vector map is a lightweight map in high-precision positioning, it occupies a small storage capacity, which improves the efficiency of subsequent posture determination and reduces costs. As an implementation method, Figure 3 A schematic diagram of a first point cloud image and a vector map for target element matching provided in an embodiment of the present application is provided. An octree is established for the first point cloud image to index the point cloud for subsequent search. A search is performed near object elements in the vector map, such as lane lines, poles, and road surfaces. If the same object element in the first point cloud image is found in the vector map, it is determined that there is a target element in the vector map that matches the target point position in the first point cloud image. Figure 3 As shown, Figure 3 The upper middle image shows the high-precision vector map and the first point cloud image, and the right image shows the matching results of the first point cloud image and each object element in the vector map, for example, Figure 3 In the figure, the position of the pole 1 in the vector map indicated by A is the same as the position of each target point indicating the pole 1 in the first point cloud map, thereby determining that there is a pole 1 in the vector map that is the same as the pole 1 in the first point cloud map.

[0093] Step 104 , performing posture detection on the target vehicle of the first point cloud image acquired according to the position of the target element matching the target point position in the vector map in the world coordinate system.

[0094] The position and posture of the target vehicle in the world coordinate system may include the coordinate position and / or posture of the target vehicle in the world coordinate system.

[0095] In an embodiment of the present application, if it is determined that there is a target element in the vector map that matches the position of the target point, the position of the target element in the vector map is converted to a position in the world coordinate system, wherein the position of the target element in the world coordinate system is an absolute position and does not change with the movement and change of objects in the surrounding environment. For example, it contains longitude and latitude information. Furthermore, according to the position of the target element in the world coordinate system, the position and posture of the target vehicle from which the first point cloud map is collected is detected, and it can be calculated that the target vehicle from which the first point cloud map is collected is based on what posture (angle) or position the target element is collected, thereby determining the position and posture of the target vehicle in the first point cloud map. Compared with determining the position and posture of the target vehicle by viewing angle positioning and determining the position and posture based on satellite navigation positioning, it is not affected by the environment, has high accuracy, and has good effect.

[0096] In the pose detection method of the embodiment of the present application, a first point cloud map is obtained, semantic segmentation is performed based on the first point cloud map to extract target points belonging to object elements from the first point cloud map, and the target points belonging to object elements in the first point cloud map are position-matched with each object element marked in the vector map, and pose detection is performed on the target vehicle whose first point cloud map is acquired based on the position of the target element that matches the position of the target point in the vector map in the world coordinate system. In the embodiment of the present application, when there is a target element in the vector map that matches the position of the target point belonging to the object element in the point cloud map, pose detection is performed on the target vehicle based on the position of the target element in the world coordinate system. Because a lightweight vector map is used for positioning, compared with the case where a large-volume positioning layer map is required, the efficiency and cost of positioning are improved while meeting the positioning accuracy.

[0097] Based on the above embodiments, this embodiment provides another posture detection method, which specifically explains how to perform posture detection on the target vehicle whose first point cloud image is collected based on the position of the target element determined by position matching in the world coordinate system. Figure 4 A flowchart of another posture detection method provided in an embodiment of the present application.

[0098] like Figure 4 As shown, the method may include the following steps:

[0099] Step 401, obtaining a first point cloud image.

[0100] Step 402: Perform semantic segmentation based on the first point cloud image to extract target points belonging to object elements from the first point cloud image.

[0101] Step 403 : Positionally match the target points belonging to the object elements in the first point cloud image with the object elements marked in the vector map.

[0102] Among them, steps 401 to 403 can refer to steps 101 to 103 in the aforementioned embodiment, and the principles are the same, so they will not be repeated in this embodiment.

[0103] Step 404, determining the position of the target point in the world coordinate system based on the posture observation value obtained by detecting the posture of the target vehicle in the world coordinate system based on the on-board inertial device of the target vehicle and the relative position of the target point in the first point cloud image.

[0104] Among them, the vehicle-mounted inertial devices, such as inertial sensors, include three-axis gyroscopes and three-axis accelerometers, which are sensors mainly used to detect and measure acceleration and rotational motion, and their principle is realized by adopting the law of inertia.

[0105] In an embodiment of the present application, the posture observation value obtained by detecting the posture of the target vehicle in the world coordinate system using the vehicle-mounted inertial device can be obtained by real-time detection of the posture of the target vehicle in the world coordinate system. As an implementation method, the posture observation value includes the position, velocity value, accelerometer deviation, gyroscope deviation and quaternion of the target vehicle in the world coordinate system, that is, the quaternion rotation matrix. Among them, the accelerometer deviation and gyroscope deviation indicate the device error of the vehicle-mounted inertial device during detection, so that when the posture observation value is subsequently corrected, the accuracy of the correction can be improved.

[0106] For example, the pose observations are represented by x i express, Among them, p i is the position of the target vehicle in the world coordinate system, v i is the speed, q i is a quaternion, b ai is the accelerometer bias, b gi is the gyro bias.

[0107] In an embodiment of the present application, a posture observation value obtained by detecting the posture of the target vehicle in a world coordinate system can be obtained based on the on-board inertial device of the target vehicle, and the first point cloud image is detected by the on-board radar of the target vehicle, so that the positional relationship between the first point cloud image and the posture observation value of the target vehicle is determined. Therefore, the position of the first point cloud image in the world coordinate system can be determined based on the posture observation value obtained by detecting the posture of the target vehicle in the world coordinate system. Since the relative position of the target point in the first point cloud image in the first point cloud image is determined, the position of the target point in the first point cloud image in the world coordinate system can be determined.

[0108] Step 405, determining the value of the first objective function according to the residual between the position of the target element in the vector map in the world coordinate system and the position of the target point in the world coordinate system.

[0109] In the embodiment of the present application, the first objective function r plm It can be determined by the following formula:

[0110]

[0111] Among them, PLM (point-line-matches) is point-line matching. Indicates the position of the target point in the first point cloud in the world coordinate system. and Respectively represent the positions of the two endpoints of the vector line indicating the target element in the vector map in the world coordinate system. The j in the graph refers to a point in the first point cloud. and The j in is the target element in the vector map, where and The j in is corresponding.

[0112] Step 406: According to the value of the first objective function, the position and posture observation values ​​of the on-board inertial device of the target vehicle are adjusted to minimize the value of the first objective function.

[0113] In the embodiment of the present application, according to the value of the first objective function, the posture observation value of the on-board inertial device of the target vehicle is continuously adjusted to optimize the posture observation value until the value of the first objective function is minimized, then the position observation value optimization is completed, that is, the accuracy of the posture observation value detected by the on-board inertial device of the target vehicle meets the set requirements, and the posture observation value optimization is completed.

[0114] Step 407, determining the predicted posture of the target vehicle at the sampling time corresponding to the first point cloud image according to the adjusted posture observation value.

[0115] Among them, the point cloud images collected by the on-board radar of the target vehicle at different sampling times are different, thereby corresponding to different first point cloud images.

[0116] In one implementation of the embodiment of the present application, the adjusted posture observation value can be used as the predicted posture of the target vehicle at the corresponding sampling time in the first point cloud map. In the embodiment of the present application, the first objective function is determined according to the residual between the position of the target element in the vector map in the world coordinate system and the position of the target point in the world coordinate system, and the observation state of the posture is optimized, and a high-precision positioning effect can be achieved in areas with weak or no GPS signals such as tunnels, viaducts, and underground, with strong robustness and higher accuracy.

[0117] In the posture detection method of the embodiment of the present application, when the coverage rate of each element in the vector map is sufficient, the posture observation value obtained by detecting the posture of the target vehicle in the world coordinate system according to the on-board inertial device of the target vehicle, and the position of the target point in the first point cloud map are determined according to the relative position of the target point in the first point cloud map. The value of the first objective function is determined through point-line constraints, that is, the residual between the position of the target element in the vector map in the world coordinate system and the position of the target point in the world coordinate system. The posture observation value of the on-board inertial device of the target vehicle is optimized and adjusted through the first objective function. Through multiple rounds of iterations, the predicted posture of the target vehicle at the sampling time corresponding to the first point cloud map is determined according to the adjusted posture observation value, thereby improving the accuracy of posture detection.

[0118] Based on the above embodiments, this embodiment provides another posture detection method. The above embodiments explain how to detect the posture of the target vehicle when there is a target element that matches the target point position in the vector map, while the embodiment of the present application explains how to detect the posture of the target vehicle when there is no target element that matches the target point position in the vector map. Figure 5 A flowchart of another posture detection method provided in an embodiment of the present application.

[0119] like Figure 5 As shown, the method may include the following steps:

[0120] Step 501: Obtain a first point cloud image.

[0121] Step 502: Perform semantic segmentation based on the first point cloud image to extract target points belonging to object elements from the first point cloud image.

[0122] Step 503 : Positionally match the target points belonging to the object elements in the first point cloud image with the object elements marked in the vector map.

[0123] Among them, steps 501 to 503 can refer to steps 101 to 103 in the aforementioned embodiment, and the principles are the same, so they will not be repeated in this embodiment.

[0124] Step 504 , determining whether there is a target element matching the target point position in the vector map, if so, executing step 505 , if not, executing step 506 .

[0125] In the embodiment of the present application, the coverage of object elements in the vector map is different. If the coverage rate of object elements in the vector map is large, the target element that matches the target point can be matched in the vector map, and step 505 is executed; if the coverage rate of object elements in the vector map is poor, for example, there are few object elements in rural roads, for example, there are few markings and few street lights, resulting in the line marking elements and street light elements in the vector map may not match the corresponding line marking elements or street light elements, then the target element that matches the target point cannot be matched in the vector map, and step 506 is executed.

[0126] Step 505 , performing posture detection on the target vehicle of the first point cloud image acquired according to the position of the target element matching the target point position in the vector map in the world coordinate system.

[0127] Specifically, reference may be made to the explanations in the aforementioned embodiments. The principles are the same and will not be repeated in this embodiment.

[0128] Step 506, obtaining a plurality of frames of second point cloud images collected by the vehicle-mounted radar; wherein the plurality of frames of second point cloud images are collected within a set period of time before the first point cloud images are collected.

[0129] Among them, the set time period can be set and adjusted according to needs and is not limited in the embodiments of the present application.

[0130] In one implementation of the embodiment of the present application, a second point cloud image of multiple frames collected by a vehicle-mounted radar within a set range of the vehicle-mounted radar of the target vehicle is obtained. The second point cloud image can be a single frame of the collected point cloud image, or can be determined by superimposing multiple frames of the collected point cloud images.

[0131] Step 507 , synthesizing the multiple frames of second point cloud images in the world coordinate system according to the predicted position of each sampling point in the multiple frames of second point cloud images in the world coordinate system, so as to obtain a local point cloud map.

[0132] Among them, the sampling points are the point clouds in the point cloud image.

[0133] In an embodiment of the present application, the relative position of each sampling point in the multi-frame second point cloud image is converted to the world coordinate system to obtain the predicted position of each sampling point in the world coordinate system, and then, the multi-frame second point cloud image is synthesized in the world coordinate system to obtain a local point cloud map, wherein each sampling point in the local point cloud map and the normal vector corresponding to each sampling point constitute a virtual plane.

[0134] Step 508 : Positionally match each sampling point in the first point cloud map with a virtual plane where each sampling point in the local point cloud map is located.

[0135] In an embodiment of the present application, each sampling point in the first point cloud map collected at the current sampling moment is positionally matched with the virtual plane where each sampling point in the local point cloud map is located, that is, point-to-plane position matching is performed, and a point-to-plane constraint is obtained according to the difference between the matching results. Based on the point-to-plane constraint, the target vehicle is subjected to posture detection, which will be described in detail in the specific subsequent embodiments.

[0136] In an embodiment of the present application, the virtual plane where each sampling point is located can be predetermined. As an implementation method, the normal information of each sampling point in the local point cloud map is determined according to the positions of each sampling point and its adjacent points in the local point cloud map in the world coordinate system, and the virtual plane where each sampling point in the local point cloud map is located is determined according to the normal information of each sampling point in the local point cloud map.

[0137] As another implementation method, a surface reconstruction technology is used to obtain a virtual plane corresponding to each sampling point from the acquired local point cloud map.

[0138] Step 509 , performing posture detection on the target vehicle according to the position of the virtual plane matched by each sampling point in the first point cloud image in the world coordinate system.

[0139] Figure 6 This is a schematic diagram of the result of position matching between each sampling point in the first point cloud map and the virtual plane where each sampling point in the local point cloud map is located, provided in an embodiment of the present application. Figure 6 The upper figure in the figure is a local point cloud map, and the lower figure is the result of position matching between each sampling point in the first point cloud map and the virtual plane where each sampling point in the local point cloud map is located. For example, D in the figure below is a matching result.

[0140] In the posture detection method of the embodiment of the present application, how to perform posture detection on the target vehicle is determined according to the different coverage of each element in the vector map, and when it is determined that there is no target element matching the target point position in the vector map, a local point cloud map is generated based on the acquired multi-frame second point cloud map, and each sampling point in the first point cloud map is matched with the virtual plane where each sampling point in the local point cloud map is located. According to the matched positions, the posture detection of the target vehicle is performed, thereby achieving stable positioning results when the coverage of object elements in the vector map is insufficient.

[0141] Based on the above embodiment, this embodiment provides another posture detection method. Figure 7 A flow chart of another posture detection method provided in an embodiment of the present application is as follows: Figure 7 As shown, step 509 includes the following steps:

[0142] Step 701, determining the position of each sampling point in the first point cloud image in the world coordinate system based on the posture observation value obtained by detecting the posture of the target vehicle in the world coordinate system according to the on-board inertial device of the target vehicle and the relative position of each sampling point in the first point cloud image.

[0143] Among them, the posture observation value includes the position, velocity value, accelerometer deviation, gyroscope deviation and quaternion of the target vehicle in the world coordinate system, that is, the quaternion rotation matrix. For the explanation of the posture observation value, please refer to the explanation in the above embodiment. The principle is the same and will not be repeated in this embodiment.

[0144] In an embodiment of the present application, a posture observation value obtained by detecting the posture of the target vehicle in a world coordinate system can be obtained based on the on-board inertial device of the target vehicle, and the first point cloud image is detected by the on-board radar of the target vehicle, so that the positional relationship between the first point cloud image and the posture observation value of the target vehicle is determined. Therefore, based on the posture observation value obtained by detecting the posture of the target vehicle in the world coordinate system, the position of the first point cloud image in the world coordinate system can be determined. Since the relative positional relationship between each sampling point in the first point cloud image is determined, the position of each sampling point in the first point cloud image in the world coordinate system can be determined.

[0145] Step 702: Determine the value of the second objective function according to the residual between the position of the virtual plane in the world coordinate system and the positions of each sampling point matched in the first point cloud image in the world coordinate system.

[0146] In the embodiment of the present application, the second objective function can be determined by the following formula:

[0147]

[0148] in, Indicates the position of each sampling point in the first point cloud in the world coordinate system, c j Indicates the position of the virtual plane in the world coordinate system, n j Represents the virtual plane normal vector information. Among them, The j in the figure refers to a sampling point in the first point cloud image, and c j The j in is a virtual plane in the local point cloud, where and c j The j in is corresponding.

[0149] Step 703: According to the value of the second objective function, the position and posture observation value of the vehicle-mounted inertial device of the target vehicle is adjusted to minimize the value of the second objective function.

[0150] In the embodiment of the present application, according to the value of the second objective function, the posture observation value of the on-board inertial device of the target vehicle is continuously adjusted to optimize the posture observation value until the value of the second objective function is minimized, then the optimization of the position observation value is completed, that is, the accuracy of the posture observation value detected by the on-board inertial device of the target vehicle meets the set requirements, and the optimization of the posture observation value is completed.

[0151] Step 704, determining the predicted posture of the target vehicle at the sampling time corresponding to the first point cloud image based on the adjusted posture observation value.

[0152] Among them, the point cloud images collected by the on-board radar of the target vehicle at different sampling times are different, thereby corresponding to different first point cloud images.

[0153] In one implementation of the embodiment of the present application, the adjusted posture observation value can be used as the predicted posture of the target vehicle at the corresponding sampling time in the first point cloud map. In the embodiment of the present application, the second objective function is determined according to the residual between the position of the virtual plane determined in the local map in the world coordinate system and the position of the sampling point in the world coordinate system, and the observation state of the posture is optimized, and a high-precision positioning effect can be achieved in areas with weak or no GPS signals such as tunnels, viaducts, and underground, with strong robustness and higher accuracy.

[0154] In the posture detection method of the embodiment of the present application, through point-plane constraints, that is, according to the residual between the position of the virtual plane in the world coordinate system and the positions of each sampling point matched in the first point cloud map in the world coordinate system, the value of the second objective function is determined, and the posture observation value of the on-board inertial device of the target vehicle is optimized and adjusted through the second objective function. Through multiple rounds of iterations, the predicted posture of the target vehicle at the sampling time corresponding to the first point cloud map is determined according to the adjusted posture observation value, thereby improving the accuracy of posture detection.

[0155] Based on the above embodiment, the embodiment of the present application proposes another position detection method, wherein the value of the first objective function or the second objective function is positively correlated with at least one of the following:

[0156] The residual between the speed value in the pose observation value and the speed value detected by the wheel speed meter of the target vehicle;

[0157] The relative posture between the predicted posture at the sampling moment and the predicted posture at the historical moment, and the residual between the accumulated posture observations from the sampling moment to the historical moment.

[0158] In one implementation of the embodiment of the present application, the value of the first objective function is the residual r between the speed value in the posture observation value and the speed value detected by the wheel speed meter of the target vehicle. ws, and the residual r between the relative pose between the predicted pose at the sampling moment and the predicted pose at the historical moment and the pose observation value accumulated from the sampling moment to the historical moment ip There is a positive relationship, so the first objective function can be expressed by the following formula:

[0159]

[0160] in, in Represents the rotation matrix from the IMU coordinate system to the vehicle body coordinate system, Represents the rotation matrix from the vehicle coordinate system to the IMU coordinate system, w i represents the angular velocity observed by the gyroscope, Represents the wheel speed meter observation value.

[0161]

[0162] Among them, x i represents the predicted observation value of the i-th frame detected by the vehicle-mounted inertial device, x i+1 represents the predicted observation value of the i+1th frame detected by the vehicle-mounted inertial device, ip represents (IMUPreintegration, IMU pre-integration), where r ip The purpose is to make the relative pose between the two lidar frames consistent with the pose observations accumulated by the IMU. i to x i+1 is the relative time of , and g is the acceleration due to gravity.

[0163] In one implementation of the embodiment of the present application, the value of the second objective function is the residual r between the speed value in the posture observation value and the speed value detected by the wheel speed meter of the target vehicle. ws , and the residual r between the relative pose between the predicted pose at the sampling moment and the predicted pose at the historical moment and the pose observation value accumulated from the sampling moment to the historical moment ip There is a positive relationship, so the second objective function can be expressed by the following formula:

[0164]

[0165] in, in Represents the rotation matrix from the IMU coordinate system to the vehicle body coordinate system, Represents the rotation matrix from the vehicle coordinate system to the IMU coordinate system, w i represents the angular velocity observed by the gyroscope, Represents the wheel speed meter observation value.

[0166]

[0167] Among them, x i represents the predicted observation value of the i-th frame detected by the vehicle-mounted inertial device, x i+1 represents the predicted observation value of the i+1th frame detected by the vehicle-mounted inertial device, ip represents (IMUPreintegration, IMU pre-integration), where r ip The purpose is to make the relative pose between the two lidar frames consistent with the pose observations accumulated by the IMU. i to x i+1 is the relative time of , and g is the acceleration due to gravity.

[0168] In the embodiment of the present application, when determining the values ​​of the first objective function and the second objective function, the residual r between the value and the speed value in the posture observation value and the speed value detected by the wheel speed meter of the target vehicle is calculated. ws , and the residual r between the relative pose between the predicted pose at the sampling moment and the predicted pose at the historical moment and the pose observation value accumulated from the sampling moment to the historical moment ip At least one of them has a positive relationship. By introducing the residuals between the speed values ​​of the wheel speed meter and the residuals accumulated at different times, the accuracy of the first objective function and the second objective function is improved, thereby improving the accuracy of the posture observation value and improving the positioning effect.

[0169] In order to implement the above embodiments, the present application also proposes a posture detection device.

[0170] Figure 8 A schematic diagram of the structure of a posture detection device provided in an embodiment of the present application.

[0171] like Figure 8 As shown, the device comprises:

[0172] The acquisition module 81 is used to acquire a first point cloud image.

[0173] The extraction module 82 is used to perform semantic segmentation according to the first point cloud image to extract target points belonging to object elements from the first point cloud image.

[0174] The matching module 83 is used to match the target points belonging to the object elements in the first point cloud image with the object elements marked in the vector map.

[0175] The first detection module 84 is used to perform posture detection on the target vehicle obtained by collecting the first point cloud image according to the position of the target element matching the target point position in the vector map in the world coordinate system.

[0176] Furthermore, in an implementation of the embodiment of the present application, the device further includes:

[0177] The acquisition module 81 is also used to obtain a multi-frame second point cloud map collected by the vehicle-mounted radar when there is no target element matching the target point position in the vector map; wherein the multi-frame second point cloud map is collected within a set time period before collecting the first point cloud map.

[0178] A synthesis module is used to synthesize the multiple frames of second point cloud images in the world coordinate system according to the predicted position of each sampling point in the multiple frames of second point cloud images in the world coordinate system to obtain a local point cloud map.

[0179] The matching module 83 is further used to match the positions of the sampling points in the first point cloud map with the virtual planes where the sampling points in the local point cloud map are located.

[0180] The second detection module is further used to perform posture detection on the target vehicle according to the position of the virtual plane matched by each sampling point in the first point cloud image in the world coordinate system.

[0181] In one implementation of the embodiment of the present application, the device further includes:

[0182] A determination module is used to determine the normal information of each sampling point in the local point cloud map according to the positions of each sampling point and its adjacent points in the local point cloud map in the world coordinate system; and determine the virtual plane where each sampling point in the local point cloud map is located according to the normal information of each sampling point in the local point cloud map.

[0183] In one implementation of the embodiment of the present application, the first detection module 84 is further configured to:

[0184] Determine the position of the target point in the world coordinate system according to the posture observation value obtained by detecting the posture of the target vehicle in the world coordinate system using the on-board inertial device of the target vehicle, and according to the relative position of the target point in the first point cloud map; determine the value of the first objective function according to the residual between the position of the target element in the vector map in the world coordinate system and the position of the target point in the world coordinate system; adjust the posture observation value of the on-board inertial device of the target vehicle according to the value of the first objective function to minimize the value of the first objective function; and determine the predicted posture of the target vehicle at the sampling time corresponding to the first point cloud map according to the adjusted posture observation value.

[0185] In one implementation of the embodiment of the present application, the second detection module is further used to:

[0186] The method comprises the following steps: determining the position of each sampling point in the first point cloud image in the world coordinate system according to the position observation value obtained by detecting the position of the target vehicle in the world coordinate system by the on-board inertial device of the target vehicle, and determining the position of each sampling point in the first point cloud image in the world coordinate system according to the relative position of each sampling point in the first point cloud image; determining the value of the second objective function according to the residual between the position of the virtual plane in the world coordinate system and the position of each sampling point matched in the first point cloud image in the world coordinate system; adjusting the position observation value of the on-board inertial device of the target vehicle according to the value of the second objective function to minimize the value of the second objective function; and determining the predicted position of the target vehicle at the sampling moment corresponding to the first point cloud image according to the adjusted position observation value.

[0187] In one implementation of the embodiment of the present application, the value of the first objective function or the second objective function is positively correlated with at least one of the following: the residual between the speed value in the posture observation value and the speed value detected by the wheel speed meter of the target vehicle; the relative posture between the predicted posture at the sampling moment and the predicted posture at the historical moment, and the residual between the accumulated posture observation values ​​from the sampling moment to the historical moment.

[0188] In one implementation of the embodiment of the present application, the acquisition module 81 is specifically used to:

[0189] Acquire a multi-frame point cloud image recently collected by the on-board radar of the target vehicle; and superimpose the multi-frame point cloud image recently collected to obtain the first point cloud image.

[0190] It should be noted that the above explanation of the method embodiment is also applicable to the device of this embodiment, and will not be repeated here.

[0191] The posture detection device of the embodiment of the present application obtains a first point cloud map, performs semantic segmentation based on the first point cloud map to extract target points belonging to object elements from the first point cloud map, matches the positions of the target points belonging to object elements in the first point cloud map with the object elements marked in the vector map, and performs posture detection on the target vehicle acquired by collecting the first point cloud map according to the position of the target elements matching the positions of the target points in the vector map in the world coordinate system. In the embodiment of the present application, when there are target elements in the vector map that match the positions of the target points belonging to object elements in the point cloud map, posture detection is performed on the target vehicle according to the positions of the target elements in the world coordinate system. Because a lightweight vector map is used for positioning, compared with the situation where a large-volume positioning layer map is required, the efficiency and cost of positioning are improved while meeting the positioning accuracy.

[0192] In order to implement the above embodiments, the present application also proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in the above method embodiments is implemented.

[0193] In order to implement the above embodiments, the present application also proposes a non-temporary computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method described in the above method embodiments is implemented.

[0194] In order to implement the above embodiments, the present application also proposes a computer program product. When the computer program in the computer program product is executed by a processor, the method described in the above method embodiments is implemented.

[0195] Fig. 9 This is a block diagram of an electronic device provided in an embodiment of the present application. For example, the electronic device 900 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0196] Reference Fig. 9 , the electronic device 900 may include one or more of the following components: a processing component 902 , a memory 904 , a power component 906 , a multimedia component 908 , an audio component 810 , an input / output (I / O) interface 912 , a sensor component 914 , and a communication component 916 .

[0197] The processing component 902 generally controls the overall operation of the electronic device 900, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 902 may include one or more processors 920 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 902 may include one or more modules to facilitate the interaction between the processing component 902 and other components. For example, the processing component 902 may include a multimedia module to facilitate the interaction between the multimedia component 908 and the processing component 802.

[0198] The memory 904 is configured to store various types of data to support operations on the electronic device 900. Examples of such data include instructions for any application or method operating on the electronic device 900, contact data, phone book data, messages, pictures, videos, etc. The memory 904 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0199] The power component 906 provides power to the various components of the electronic device 900. The power component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 800.

[0200] The multimedia component 908 includes a screen that provides an output interface between the electronic device 900 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 908 includes a front camera and / or a rear camera. When the electronic device 900 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0201] The audio component 910 is configured to output and / or input audio signals. For example, the audio component 910 includes a microphone (MIC), and when the electronic device 900 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 904 or sent via the communication component 916. In some embodiments, the audio component 910 also includes a speaker for outputting audio signals.

[0202] I / O interface 912 provides an interface between processing component 902 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0203] The sensor assembly 914 includes one or more sensors for providing various aspects of status assessment for the electronic device 900. For example, the sensor assembly 914 can detect the open / closed state of the electronic device 900, the relative positioning of components, such as the display and keypad of the electronic device 900, and the sensor assembly 914 can also detect the position change of the electronic device 900 or a component of the electronic device 900, the presence or absence of user contact with the electronic device 900, the orientation or acceleration / deceleration of the electronic device 900, and the temperature change of the electronic device 900. The sensor assembly 914 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 914 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 914 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0204] The communication component 916 is configured to facilitate wired or wireless communication between the electronic device 900 and other devices. The electronic device 900 can access a wireless network based on a communication standard, such as WiFi, 4G or 5G, or a combination thereof. In an exemplary embodiment, the communication component 916 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 916 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0205] In an exemplary embodiment, the electronic device 900 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0206] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, and the instructions can be executed by a processor 920 of an electronic device 900 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0207] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0208] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0209] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0210] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0211] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0212] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0213] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0214] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A posture detection method, characterized in that: The following steps are involved: Get the first point cloud image; Performing semantic segmentation according to the first point cloud image to extract target points belonging to object elements from the first point cloud image; Positionally matching the target points belonging to the object elements in the first point cloud with the object elements marked in the vector map; Performing posture detection on the target vehicle obtained by acquiring the first point cloud image according to the position of the target element matching the target point position in the vector map in the world coordinate system; After matching the target points belonging to the object elements in the first point cloud with the object elements marked in the vector map, the method further includes: When there is no target element matching the position of the target point in the vector map, obtaining a plurality of frames of second point cloud images collected by the vehicle-mounted radar; wherein the plurality of frames of second point cloud images are collected within a set period of time before the first point cloud images are collected; synthesizing the multiple frames of second point cloud images in the world coordinate system according to the predicted position of each sampling point in the multiple frames of second point cloud images in the world coordinate system to obtain a local point cloud map; Position matching of each sampling point in the first point cloud map with a virtual plane where each sampling point in the local point cloud map is located; The target vehicle is subjected to posture detection according to the position of the virtual plane matched by each sampling point in the first point cloud image in the world coordinate system.

2. The method according to claim 1, characterized in that Before position matching of each sampling point in the first point cloud map with the virtual plane where each sampling point in the local point cloud map is located, the method further includes: Determine normal information of each sampling point in the local point cloud map according to the positions of each sampling point and its adjacent points in the local point cloud map in the world coordinate system; According to the normal information of each sampling point in the local point cloud map, a virtual plane where each sampling point in the local point cloud map is located is determined.

3. The method according to claim 1, characterized in that: The performing posture detection on the target vehicle acquired by acquiring the first point cloud image according to the position of the target element matching the target point position in the vector map in the world coordinate system comprises: Determine the position of the target point in the world coordinate system according to a posture observation value obtained by detecting the posture of the target vehicle in the world coordinate system according to the on-board inertial device of the target vehicle, and according to the relative position of the target point in the first point cloud image; Determining a value of a first objective function according to a residual between a position of the target element in the vector map in a world coordinate system and a position of the target point in the world coordinate system; According to the value of the first objective function, adjusting the position and posture observation value of the vehicle-mounted inertial device of the target vehicle to minimize the value of the first objective function; According to the adjusted posture observation value, the predicted posture of the target vehicle at the sampling time corresponding to the first point cloud image is determined.

4. The method according to claim 1, characterized in that: The detecting of the position and posture of the target vehicle according to the position of the virtual plane matched by each sampling point in the first point cloud image in the world coordinate system includes: Determine the position of each sampling point in the first point cloud image in the world coordinate system according to the position observation value obtained by detecting the position of the target vehicle in the world coordinate system according to the on-board inertial device of the target vehicle and the relative position of each sampling point in the first point cloud image; Determine a value of a second objective function according to a residual between a position of the virtual plane in a world coordinate system and positions of each sampling point matched in the first point cloud image in the world coordinate system; According to the value of the second objective function, adjusting the position and posture observation value of the vehicle-mounted inertial device of the target vehicle to minimize the value of the second objective function; According to the adjusted posture observation value, the predicted posture of the target vehicle at the sampling time corresponding to the first point cloud image is determined.

5. The method according to claim 3 or 4, characterized in that: The value of the first objective function or the second objective function is positively correlated with at least one of the following: The residual between the speed value in the posture observation value and the speed value detected by the wheel speed meter of the target vehicle; The relative posture between the predicted posture at the sampling moment and the predicted posture at the historical moment, and the residual between the posture observation values ​​accumulated from the sampling moment to the historical moment.

6. The method according to any one of claims 1 to 4, characterized in that: The step of obtaining a first point cloud image includes: Obtain a multi-frame point cloud image recently collected by the on-board radar of the target vehicle; The most recently acquired multi-frame point cloud images are superimposed to obtain the first point cloud image.

7. A posture detection device, characterized in that: include: An acquisition module, used for acquiring a first point cloud image; an extraction module, configured to perform semantic segmentation according to the first point cloud image, so as to extract target points belonging to object elements from the first point cloud image; A matching module, used for matching the target points belonging to the object elements in the first point cloud image with the object elements marked in the vector map; A first detection module, configured to perform posture detection on the target vehicle obtained by acquiring the first point cloud image according to the position of the target element matching the target point position in the vector map in the world coordinate system; The device further comprises: The acquisition module is further configured to acquire a plurality of frames of second point cloud images acquired by the vehicle-mounted radar when there is no target element matching the position of the target point in the vector map; wherein the plurality of frames of second point cloud images are acquired within a set period of time before the acquisition of the first point cloud image; A synthesis module, configured to synthesize the plurality of frames of second point cloud images in the world coordinate system according to the predicted position of each sampling point in the plurality of frames of second point cloud images in the world coordinate system, so as to obtain a local point cloud map; The matching module is further used to match the positions of the sampling points in the first point cloud map with the virtual planes where the sampling points in the local point cloud map are located; The second detection module is further used to perform posture detection on the target vehicle according to the position of the virtual plane matched by each sampling point in the first point cloud image in the world coordinate system.

8. The device according to claim 7, characterized in that The device further comprises: A determination module is used to determine the normal information of each sampling point in the local point cloud map according to the positions of each sampling point and its adjacent points in the local point cloud map in the world coordinate system; and determine the virtual plane where each sampling point in the local point cloud map is located according to the normal information of each sampling point in the local point cloud map.

9. The device according to claim 7, characterized in that The first detection module is further used for: Determine the position of the target point in the world coordinate system according to a posture observation value obtained by detecting the posture of the target vehicle in the world coordinate system according to the on-board inertial device of the target vehicle, and according to the relative position of the target point in the first point cloud image; Determining a value of a first objective function according to a residual between a position of the target element in the vector map in a world coordinate system and a position of the target point in the world coordinate system; According to the value of the first objective function, adjusting the position and posture observation value of the vehicle-mounted inertial device of the target vehicle to minimize the value of the first objective function; According to the adjusted posture observation value, the predicted posture of the target vehicle at the sampling time corresponding to the first point cloud image is determined.

10. The device according to claim 7, characterized in that The second detection module is further used for: Determine the position of each sampling point in the first point cloud image in the world coordinate system according to the position observation value obtained by detecting the position of the target vehicle in the world coordinate system according to the on-board inertial device of the target vehicle and the relative position of each sampling point in the first point cloud image; Determine a value of a second objective function according to a residual between a position of the virtual plane in a world coordinate system and positions of each sampling point matched in the first point cloud image in the world coordinate system; According to the value of the second objective function, adjusting the position and posture observation value of the vehicle-mounted inertial device of the target vehicle to minimize the value of the second objective function; According to the adjusted posture observation value, the predicted posture of the target vehicle at the sampling time corresponding to the first point cloud image is determined.

11. The device according to claim 9 or 10, characterized in that The value of the first objective function or the second objective function is positively correlated with at least one of the following: The residual between the speed value in the posture observation value and the speed value detected by the wheel speed meter of the target vehicle; The relative posture between the predicted posture at the sampling moment and the predicted posture at the historical moment, and the residual between the posture observation values ​​accumulated from the sampling moment to the historical moment.

12. The device according to any one of claims 7 to 10, characterized in that: The acquisition module is specifically used for: Obtain a multi-frame point cloud image recently collected by the on-board radar of the target vehicle; The most recently acquired multi-frame point cloud images are superimposed to obtain the first point cloud image.

13. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 6 is implemented.

14. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.