High-precision map point cloud pose optimization method, device, equipment and medium

Abstract

The present disclosure provides a method, device, equipment and medium for optimizing the pose of a high-precision map point cloud, and relates to the field of computer technology, and in particular to the field of high-precision map technology. The specific technical solution is: obtaining the first pose of each frame of point cloud in at least one point cloud pair corresponding to the target environment; the point cloud pair includes two adjacent frames of point cloud in the multi-frame point cloud corresponding to the target environment; obtaining the relative pose between the two frames of point cloud in each point cloud pair; the relative pose is estimated by the odometer module; according to the first pose and the relative pose, the pose trajectory of at least one point cloud pair is optimized to obtain the second pose of each frame of point cloud. The technical solution disclosed in the present disclosure can make the pose trajectory smoother and overcome the influence of weak or missing GNSS signals on the pose.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, in particular to the field of high-precision map technology, and specifically to a method, device, equipment and medium for optimizing the pose of a high-precision map point cloud. Background Art

[0002] High-precision maps, also known as high-accuracy maps, are used by self-driving cars. They contain precise vehicle location information and rich road element data, helping cars predict complex road conditions, such as slope, curvature, and heading, to better avoid potential risks. Large-scale outdoor scene mapping generally relies on GNSS (Global Navigation Satellite System) to obtain the initial global pose, followed by loop detection and point cloud registration and fusion. However, in scenarios such as overpasses and boulevards, GNSS signals are easily lost or weak, causing the initial pose trajectory to drift and become uneven, resulting in a high failure rate for subsequent point cloud fusion. Summary of the Invention

[0003] The present invention provides a method, device, equipment and medium for optimizing the pose of a high-precision map point cloud.

[0004] According to a first aspect of the present disclosure, a point cloud pose optimization method is provided, comprising:

[0005] Obtaining the first pose of each frame of point cloud in at least one point cloud pair corresponding to the target environment; the point cloud pair includes two adjacent frames of point cloud in multiple frames of point cloud corresponding to the target environment;

[0006] Get the relative pose between the two frames of point cloud in each point cloud pair; the relative pose is estimated by the odometry module;

[0007] According to the above first pose and relative pose, the pose trajectory of at least one point cloud pair is optimized to obtain the second pose of each frame point cloud.

[0008] According to a second aspect of the present disclosure, a point cloud pose optimization device is provided, comprising:

[0009] A pose acquisition module is used to obtain the first pose of each frame of point cloud in at least one point cloud pair corresponding to the target environment; the point cloud pair includes two adjacent frames of point cloud in the multiple frames of point cloud corresponding to the target environment;

[0010] The relative pose acquisition module is used to obtain the relative pose between the two frames of point cloud in each point cloud pair; the relative pose is estimated by the odometry module;

[0011] The first optimization module is used to optimize the pose trajectory of at least one point cloud pair according to the first pose and relative pose to obtain the second pose of each frame of point cloud.

[0012] According to a third aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the point cloud pose optimization method provided by any embodiment of the present disclosure.

[0013] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the point cloud pose optimization method provided by any embodiment of the present disclosure.

[0014] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the point cloud pose optimization method provided by any embodiment of the present disclosure.

[0015] The technical solution disclosed herein can achieve at least the following beneficial effects:

[0016] The technical solution disclosed in the present invention optimizes the pose trajectory of the point cloud based on the relative pose estimated by the odometer, which can reduce the dependence on the GNSS signal and make the pose trajectory smoother, so as to overcome the impact of weak or missing GNSS signals on the pose, thereby improving the success rate of point cloud registration and fusion, and facilitating the accurate construction of high-precision maps of large-scale outdoor scenes.

[0017] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0019] Figure 1 1 is a flow chart of a point cloud pose optimization method provided by an embodiment of the present disclosure;

[0020] Figure 2 is a schematic diagram of a pose graph involved in an embodiment of the present disclosure;

[0021] Figure 3 1 is a flow chart of another point cloud pose optimization method provided by an embodiment of the present disclosure;

[0022] Figure 4 Schematic diagram of an example of a point cloud pose optimization method provided by an embodiment of the present disclosure;

[0023] Figure 5Schematic diagram of the structural framework of a point cloud pose optimization device provided by an embodiment of the present disclosure;

[0024] Figure 6 Schematic diagram of the structural framework of another point cloud pose optimization device provided by an embodiment of the present disclosure;

[0025] Figure 7 It is a schematic diagram of the structural framework of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0026] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0027] In the description of the embodiments of the present disclosure, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present disclosure, unless otherwise specified, "plurality" means two or more.

[0028] It should be further understood that the term “and / or” used in the embodiments of the present disclosure includes all or any elements and all combinations of one or more associated listed items.

[0029] Those skilled in the art will understand that, unless otherwise defined, all terms (including technical terms and scientific terms) used in the embodiments of the present disclosure have the same meaning as generally understood by those skilled in the art in the art to which the present disclosure belongs.

[0030] First, several terms involved in the embodiments of the present disclosure are introduced:

[0031] High-precision map: also known as high-precision map, high-resolution map or highly automated driving map, is a special map used for autonomous driving. It is composed of vector information such as lane models containing semantic information, road components, road attributes, and feature layers for multi-sensor positioning. It plays a core role in the entire autonomous driving field and can assist autonomous driving vehicles in determining their own position, drivable area, target type, driving direction, relative position of the preceding vehicle, perception of traffic light status and driving lanes, and other information.

[0032] Point cloud: Scanned data is recorded as points. Each point contains 3D coordinates and may also contain information such as color and reflection intensity. Color information is typically determined by capturing an image with an acquisition device and assigning the color information of corresponding pixels in the image to corresponding points in the point cloud. Reflection intensity information is obtained by capturing the echo intensity collected by the LiDAR receiver. This intensity information is related to the target's surface material, roughness, angle of incidence, and the instrument's emission energy and laser wavelength.

[0033] Pose: refers to the position and posture of the point cloud in a specified coordinate system.

[0034] The inventors of this disclosure discovered that when mapping large-scale outdoor scenes, GNSS signals are easily lost or weak on sections of road, such as overpasses and tree-lined roads. This causes initial pose trajectory drift and unevenness, leading to a high failure rate in subsequent point cloud fusion. Currently, manual intervention is often required for sections with poor GNSS signals, resulting in low efficiency in high-precision map production.

[0035] The following describes in detail the technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned technical problems with specific embodiments.

[0036] According to an embodiment of the present disclosure, the present disclosure provides a point cloud pose optimization method, such as Figure 1 As shown, the method includes:

[0037] S101, obtaining the first pose of each frame of point cloud in at least one point cloud pair corresponding to the target environment.

[0038] A point cloud pair consists of two adjacent frames of point clouds in the multi-frame point cloud corresponding to the target environment. Any two adjacent frames of point clouds in the multi-frame point cloud corresponding to the target environment form a point cloud pair. The multi-frame point cloud corresponding to the target environment can be acquired by a collection device within the target environment. The collection device can be a mobile device such as a vehicle, drone, or mobile robot.

[0039] The embodiment of the present disclosure can optimize the point cloud pose after the acquisition device acquires the point clouds of each point cloud pair, or can optimize the pose of the acquired point clouds in real time during the acquisition process of the acquisition device.

[0040] For each frame of point cloud, the first pose can be obtained by initializing the pose of the frame of point cloud by the GNSS pose corresponding to the frame of point cloud, that is, the GNSS pose is used as the first pose of the frame of point cloud.

[0041] In an optional embodiment, for the i-th (i is an arbitrary integer) frame point cloud, its timestamp t iThe timestamp of the GNSS pose may be out of sync, so it is impossible to obtain the GNSS pose with the same timestamp. First, obtain the timestamp of the GNSS pose with the same timestamp. i The GNSS poses of two adjacent timestamps are linearly interpolated to obtain a new GNSS pose. This new GNSS pose is used as the GNSS pose corresponding to the i-th frame of the point cloud. For example, for a point cloud with a timestamp of 10 seconds, there may not be a GNSS pose for the same time. If there are GNSS poses for the adjacent timestamps of 9 and 11 seconds, the GNSS poses of the 9th and 11th seconds can be linearly interpolated and the interpolated data can be used as the GNSS pose of the 10th second.

[0042] S102: Obtain the relative pose between two frames of point clouds in each point cloud pair.

[0043] The relative pose is estimated by the odometry module.

[0044] The odometry module in the disclosed embodiment may include at least one of LIO (LiDAR Inertial Odometry) and a visual odometry. The odometry module may estimate the relative pose between two adjacent frames of point cloud based on LiDAR measurement data and / or images, and optimize the pose trajectory based on the relative pose estimated by the odometry module, thereby overcoming the impact of weak or missing GNSS signals on the pose trajectory, thereby improving the success rate of point cloud registration and fusion, facilitating the accurate construction of high-precision maps of large-scale outdoor scenes, and reducing the frequency of manual intervention, thereby improving the automation level and production efficiency of high-precision maps.

[0045] S103, optimizing the pose trajectory of at least one point cloud pair according to the first pose of each frame of point cloud in each point cloud pair and the relative pose between two frames of point cloud in each point cloud pair, to obtain a second pose of each frame of point cloud in at least one point cloud pair.

[0046] The pose trajectory in the embodiment of the present disclosure can be presented in the form of a pose graph, referring to Figure 2 The pose graph example shown in the figure involves multiple nodes (such as Figure 2 ) and the edges between nodes (as shown in the circles in Figure 2 ), where a node is the pose of a frame of point cloud, and an edge between nodes is the relative pose of two nodes. Pose graph optimization can be performed to optimize the pose trajectory. Figure 2 The triangle in the figure represents the GNSS pose used to initialize each node, which is also the first pose after initialization.

[0047] In an optional embodiment, the following operations are performed on the first frame point cloud and the second frame point cloud in each point cloud pair: based on the relative pose between the first frame point cloud and the second frame point cloud, and the first pose of the second frame point cloud, the first pose residual of the current point cloud pair is constructed; the first pose residual is optimized to obtain the optimized second pose of the first frame point cloud and the second frame point cloud.

[0048] In one example, both the first frame point cloud and the second frame point cloud can be historical frame point clouds, which can be applicable to application requirements of first acquisition and then optimization, such as the mapping requirements of high-precision maps; in another example, the first frame point cloud can be a historical frame point cloud, and the second frame point cloud can be a current frame point cloud, which can be applicable to application requirements of real-time optimization during the acquisition process.

[0049] Based on this implementation, the disclosed embodiment can construct residuals for each point cloud pair, and by optimizing the residuals, achieve more accurate optimization of the pose trajectory.

[0050] In an optional embodiment, the first pose residual of the current point cloud pair is constructed based on the relative pose between the first frame point cloud and the second frame point cloud, and the first pose of the second frame point cloud, including: constructing a first residual item in the first pose residual of the current point cloud pair based on the relative pose between the first frame point cloud and the second frame point cloud; constructing a second residual item in the first pose residual of the current point cloud pair based on the first pose of the second frame point cloud.

[0051] In an optional embodiment, the first residual term in the first pose residual can be constructed by referring to the following expression:

[0052]

[0053] In expression (1), T i Represents the pose of the i-th frame point cloud (as the first frame point cloud in a point cloud pair), T i+1 Represents the pose of the i+1th frame point cloud (as the second frame point cloud in a point cloud pair), Indicates that based on T i and T i+1 Calculate the relative pose between the i-th frame point cloud and the i+1-th frame point cloud, T odom_ii+1 represents the relative pose between the point cloud of the i-th frame and the point cloud of the i+1-th frame estimated by the odometry; r odom_ii+1 Indicates that based on T i 、T i+1 and relative pose T odom_ii+1 The first residual term constructed is specifically used to characterize the relative pose actually measured or calculated and the relative pose T estimated by the odometry odom_ii+1 difference.

[0054] In an optional implementation, the second residual term may be constructed by referring to the following expression:

[0055]

[0056] In expression (2), T i+1 The meaning of is the same as expression (1); T gnss_i+1 represents the GNSS pose used to initialize the i+1 frame point cloud, that is, the first pose of the i+1 frame point cloud after initialization; r gnss_i+1 Represents the second residual term constructed based on the pose of the i+1th frame point cloud and the GNSS pose, which is specifically used to characterize the difference between the actually measured pose of the i+1th frame point cloud and the GNSS pose used to initialize the pose of the i+1th frame point cloud.

[0057] In another optional embodiment, T in expression (2) i+1 You can also use T i to replace.

[0058] In an optional embodiment, a specific method of optimizing the first pose residual may be to minimize the first pose residual. Specifically, the first residual term and the second residual term in the first pose residual may be minimized. The minimization may be expressed by the following expression:

[0059]

[0060] In expression (3), Ω odom Represents the first residual term r odom_ii+1 The weight, Ω gnss Represents the second residual term r gnss_i+1 The weights of the two residual terms can be set according to actual needs; T represents the transposition algorithm; the pose T of the i-th frame point can be obtained by calculating expression (3) i and the pose T of the i+1th frame i+1 The optimized value of .

[0061] When constructing the first pose residual, the embodiment of the present disclosure respectively considers the error of the point cloud pose information based on the first residual term based on the relative pose of the point cloud and the second residual term based on the point cloud pose, which can improve the accuracy of the optimization process.

[0062] This disclosure is for Figure 1 The order of the steps shown is not limited and can be adjusted according to actual needs. For example, steps S101 and S102 can be Figure 1 The steps are shown as follows, but can also be executed simultaneously.

[0063] According to an embodiment of the present disclosure, the present disclosure also provides a point cloud pose optimization method, such as Figure 3 As shown, the method includes:

[0064] S301, obtaining the first pose of each frame of point cloud in at least one point cloud pair corresponding to the target environment.

[0065] S302: Obtain the relative pose between two frames of point clouds in each point cloud pair.

[0066] S303: Optimize the pose trajectory of at least one point cloud pair according to the first pose of each frame of point cloud in each point cloud pair and the relative pose between two frames of point cloud in each point cloud pair to obtain a second pose of each frame of point cloud in at least one point cloud pair.

[0067] The specific implementation of steps S301 to S303 can refer to the above steps S101 to S103, which will not be repeated here.

[0068] S304: Optimize the pose trajectory of at least one point cloud pair based on the second pose of each frame of point cloud in each point cloud pair and the relative pose between two frames of point cloud in each point cloud pair to obtain a third pose of each frame of point cloud in at least one point cloud pair.

[0069] Based on the second pose of the point cloud obtained after the first optimization and the relative pose estimated by the odometry module, the trajectory in the pose graph is optimized for the second time. This can make the trajectory smoother, further reduce the dependence on GNSS signals, improve the success rate of point cloud registration and fusion, and facilitate the construction of more accurate high-precision maps.

[0070] In an optional embodiment, for at least a portion of the point clouds in each point cloud pair, if the absolute value of the distance difference between the second pose after the first optimization and the first pose before the first optimization is greater than a preset distance threshold, that is, the pose error before and after the first optimization is too large, then the reliability of the first pose (also known as the GNSS pose) is considered poor, and it is eliminated. The second pose is then used to replace the first pose for re-optimization. After the re-optimization, the smoothness of the pose trajectory can be greatly improved. The distance threshold can be set according to actual needs.

[0071] In an optional embodiment, the following operations are performed on the first frame point cloud and the second frame point cloud in each point cloud pair: when the absolute value of the distance difference between the second pose and the first pose of the second frame point cloud is greater than a preset distance threshold, the second pose residual of the current point cloud pair is constructed according to the second pose of the first frame point cloud and the second frame point cloud, the relative pose between the first frame point cloud and the second frame point cloud, and the first pose of the second frame point cloud; the second pose residual is optimized to obtain the optimized third pose of the first frame point cloud and the second frame point cloud.

[0072] Based on this implementation, the pose parameters of each point cloud pair can be updated, residuals can be calculated based on the new pose parameters, and optimization can be performed through the residuals to achieve a more accurate second optimization.

[0073] In an optional embodiment, the second pose of the first frame point cloud and the second frame point cloud, the relative pose between the first frame point cloud and the second frame point cloud, and the first pose of the second frame point cloud are used to construct the second pose residual of the current point cloud pair, including: constructing a first residual item in the second pose residual of the current point cloud pair according to the second pose of the first frame point cloud and the second frame point cloud, and the relative pose between the first frame point cloud and the second frame point cloud; constructing a second residual item in the second pose residual of the current point cloud pair according to the first pose and the second pose of the second frame point cloud.

[0074] In an optional embodiment, the construction method of the first residual term in the second pose residual can refer to the previous expression (1), where T in expression (1) i and T i+1 Update them to the second pose of the i-th frame point cloud and the second pose of the i+1-th frame point cloud respectively. The meanings of the other parameters are the same as before.

[0075] In an optional embodiment, the construction method of the second residual term in the second pose residual can refer to the previous expression (2), where T in expression (2) i+1 Update to the second pose of the i+1th frame point cloud. The meanings of the remaining parameters are the same as before.

[0076] In an optional embodiment, the specific method for optimizing the second posture residual may be to minimize the second posture residual. Specifically, the first residual term and the second residual term in the second posture residual may be minimized. The minimization method may refer to the previous expression (3).

[0077] When constructing the second pose residual, the embodiment of the present disclosure constructs a first residual term based on the relative pose of the point cloud and a second residual term based on the point cloud pose, respectively, which comprehensively considers the error of the point cloud pose information and can improve the accuracy of the second optimization process.

[0078] In an optional embodiment, the point cloud pose optimization method provided by the present disclosure further includes: determining whether the smoothness of the pose trajectory formed by the third pose meets a preset smoothness condition; if the smoothness does not meet the smoothness condition, continuing to optimize the pose trajectory formed by the third pose at least once until the smoothness of the optimized pose trajectory meets the above-mentioned smoothness condition.

[0079] In the case that the posture trajectory after the first two optimizations is not smooth enough, the embodiment of the present disclosure can continue to optimize the posture trajectory, thereby continuously improving the smoothness of the posture trajectory to meet actual needs.

[0080] The smoothness condition can be set according to actual needs. Each time the posture trajectory formed by the third posture is optimized, the corresponding parameters of expressions (1) to (2) can be iteratively updated by referring to the second optimization method mentioned above.

[0081] This disclosure is for Figure 3 The order of the steps shown is not limited and can be adjusted according to actual needs. For example, steps S301 and S302 can be Figure 3 The steps are shown as follows, but can also be executed simultaneously.

[0082] Figure 4 Shows a schematic diagram of the principle of optimizing the pose graph, refer to Figure 4 A specific example of the point cloud pose optimization method provided by the embodiment of the present disclosure is as follows:

[0083] Figure 4 The trajectory a in the figure is the expected pose trajectory. Due to the presence of blocked road sections in the actual scene, the GNSS signal is poor or even missing (such as Figure 4 ), resulting in a change in the pose trajectory, producing trajectory b. The concave portion of the line in trajectory b corresponds to the signal loss area. Through the first optimization of the embodiment of the present disclosure, trajectory c can be obtained, and the smoothness of trajectory c can be effectively improved. Trajectory c and trajectory b are far apart in the signal loss area, and the reliability of the GNSS pose constraint of trajectory b in the signal loss area is significantly lower. In this case, the GNSS pose constraint in the signal loss area of ​​trajectory b can be removed, and the pose trajectory can be optimized again using the pose after the first optimization, that is, the pose of trajectory c in the signal loss area, instead of the removed GNSS pose, to obtain a smoother trajectory d.

[0084] According to an embodiment of the present disclosure, the present disclosure also provides a point cloud pose optimization device, such as Figure 5 As shown, the device includes: an initial posture acquisition module 501, a relative posture acquisition module 502 and a first optimization module 503.

[0085] The initial pose acquisition module 501 is used to obtain the first pose of each frame of point cloud in at least one point cloud pair corresponding to the target environment; the point cloud pair includes two adjacent frames of point cloud in the multiple frames of point cloud corresponding to the target environment.

[0086] The relative pose acquisition module 502 is used to obtain the relative pose between two frames of point cloud in each point cloud pair; the relative pose is estimated by the odometer module.

[0087] The first optimization module 503 is used to optimize the pose trajectory of at least one point cloud pair according to the first pose of each frame of point cloud in each point cloud pair and the relative pose between two frames of point cloud in each point cloud pair to obtain the second pose of each frame of point cloud.

[0088] In an optional implementation, the first optimization module 503 is specifically configured to perform the following operations on the first frame point cloud and the second frame point cloud in each point cloud pair:

[0089] According to the relative pose between the first frame point cloud and the second frame point cloud, and the first pose of the second frame point cloud, the first pose residual of the current point cloud pair is constructed; the first pose residual is optimized to obtain the optimized second pose of the first frame point cloud and the second frame point cloud.

[0090] In an optional embodiment, when constructing the first pose residual of the current point cloud pair, the first optimization module 503 is specifically used to: construct a first residual item in the first pose residual of the current point cloud pair based on the relative pose between the first frame point cloud and the second frame point cloud; and construct a second residual item in the first pose residual of the current point cloud pair based on the first pose of the second frame point cloud.

[0091] In an optional embodiment, as Figure 6 As shown, the point cloud pose optimization device provided by the present disclosure further includes: an initial pose acquisition module 601, a relative pose acquisition module 602, a first optimization module 603 and a second optimization module 604.

[0092] The functions of the initial pose acquisition module 601, the relative pose acquisition module 602 and the first optimization module 603 can refer to the functions of the previous initial pose acquisition module 501, the relative pose acquisition module 502 and the first optimization module 503; the second optimization module 604 is used to optimize the pose trajectory of at least one point cloud pair according to the second pose of each frame of point cloud in each point cloud pair and the relative pose between two frames of point clouds in each point cloud pair, so as to obtain the third pose of each frame of point cloud in at least one point cloud pair.

[0093] In an optional implementation, the second optimization module 604 is specifically configured to perform the following operations on the first frame point cloud and the second frame point cloud in each point cloud pair:

[0094] When the absolute value of the distance difference between the second pose and the first pose of the second frame point cloud is greater than a preset distance threshold, the second pose residual of the current point cloud pair is constructed according to the second pose of the first frame point cloud and the second frame point cloud, the relative pose between the first frame point cloud and the second frame point cloud, and the first pose of the second frame point cloud; the second pose residual is optimized to obtain the optimized third pose of the first frame point cloud and the second frame point cloud.

[0095] In an optional embodiment, when constructing the second pose, the second optimization module 604 is specifically used to: construct a first residual item in the second pose residual of the current point cloud pair based on the second pose of the first frame point cloud and the second frame point cloud, and the relative pose between the first frame point cloud and the second frame point cloud; construct a second residual item in the second pose residual of the current point cloud pair based on the first pose and the second pose of the second frame point cloud.

[0096] In an optional embodiment, the point cloud pose optimization device provided by the present disclosure further includes: a third optimization module. The third optimization module is configured to determine whether the smoothness of the pose trajectory formed by the third pose satisfies a preset smoothness condition; if the smoothness does not satisfy the smoothness condition, the pose trajectory formed by the third pose is optimized at least once until the smoothness of the optimized pose trajectory satisfies the smoothness condition.

[0097] The functions of the modules in each device in the embodiments of the present disclosure can refer to the corresponding descriptions in the above method embodiments and will not be repeated here.

[0098] In the technical solutions disclosed herein, the acquisition, storage, and application of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0099] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a non-transitory computer-readable storage medium, and a computer program product.

[0100] The electronic device provided by the present disclosure includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the point cloud pose optimization method provided by any embodiment of the present disclosure.

[0101] The non-transitory computer-readable storage medium provided by the present disclosure stores computer instructions, which are used to enable a computer to execute the point cloud pose optimization method provided by any embodiment of the present disclosure.

[0102] The computer program product provided by the present disclosure includes a computer program, which, when executed by a processor, implements the point cloud pose optimization method provided by any embodiment of the present disclosure.

[0103] Figure 7A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0104] like Figure 7 As shown, the device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. Various programs and data required for the operation of the device 700 can also be stored in the RAM 703. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0105] Various components in device 700 are connected to I / O interface 705, including an input unit 706, such as a keyboard, mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a magnetic disk, optical disk, etc.; and a communication unit 709, such as a network card, modem, wireless communication transceiver, etc. The communication unit 709 allows device 700 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0106] The computing unit 701 can be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 701 performs the various methods and processes described above. For example, in some embodiments, the above methods can be implemented as computer software programs that are tangibly contained in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the method described above can be performed. Alternatively, in other embodiments, the computing unit 701 can be configured to perform the above methods in any other appropriate manner (e.g., by means of firmware).

[0107] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0108] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0109] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0110] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0111] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0112] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0113] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0114] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A point cloud pose optimization method, comprising: Obtain the first pose of each frame of at least one point cloud pair of the target environment; The point cloud pair includes two adjacent frames of point clouds in the multi-frame point cloud corresponding to the target environment; Obtain the relative pose between the two frames of point cloud in each point cloud pair; the relative pose is estimated by the odometry module; Optimizing the pose trajectory of the at least one point cloud pair according to the first pose and the relative pose to obtain a second pose of each frame of the point cloud; The step of optimizing the pose trajectory of the at least one point cloud pair according to the first pose and the relative pose to obtain the second pose of each frame of point cloud comprises: For each point cloud pair, perform the following operations on the first and second frames: Constructing a first pose residual of the current point cloud pair according to the relative pose between the first frame point cloud and the second frame point cloud, and the first pose of the second frame point cloud; Optimizing the first pose residual to obtain optimized second poses of the first frame point cloud and the second frame point cloud; The first pose residual includes a first residual term and a second residual term, the first residual term representing the difference between the relative pose actually measured or calculated and the relative pose estimated using the odometer module, and the second residual term representing the difference between the first pose of the second frame point cloud actually measured and the GNSS pose used to initialize the second frame point cloud pose.

2. The point cloud pose optimization method according to claim 1, wherein: The constructing the first pose residual of the current point cloud pair according to the relative pose between the first frame point cloud and the second frame point cloud, and the first pose of the second frame point cloud, includes: Constructing a first residual term in the first pose residual of the current point cloud pair according to the relative pose between the first frame point cloud and the second frame point cloud; According to the first pose of the second frame point cloud, a second residual term in the first pose residual of the current point cloud pair is constructed.

3. The point cloud pose optimization method according to claim 1 or 2, further comprising: The posture trajectory is optimized according to the second posture and the relative posture to obtain a third posture of the point cloud of each frame.

4. The point cloud pose optimization method according to claim 3, wherein: Optimizing the posture trajectory according to the second posture and the relative posture to obtain the third posture of each frame point cloud includes: For each point cloud pair, perform the following operations on the first and second frames: When the absolute value of the distance difference between the second pose and the first pose of the second frame point cloud is greater than a preset distance threshold, constructing a second pose residual of the current point cloud pair according to the second poses of the first frame point cloud and the second frame point cloud, the relative pose between the first frame point cloud and the second frame point cloud, and the first pose of the second frame point cloud; The second pose residual is optimized to obtain an optimized third pose of the first frame point cloud and the second frame point cloud.

5. The point cloud pose optimization method according to claim 4, wherein: The second pose residual of the current point cloud pair is constructed based on the second pose of the first frame point cloud and the second frame point cloud, the relative pose between the first frame point cloud and the second frame point cloud, and the first pose of the second frame point cloud, including: Constructing a first residual term in the second pose residual of the current point cloud pair according to the second pose of the first frame point cloud and the second frame point cloud, and the relative pose between the first frame point cloud and the second frame point cloud; Construct a second residual term in the second pose residual of the current point cloud pair according to the first pose and the second pose of the second frame point cloud.

6. The point cloud pose optimization method according to claim 3, further comprising: Determining whether the smoothness of the posture trajectory formed by the third posture meets a preset smoothness condition; When the smoothness does not satisfy the smoothness condition, the posture trajectory formed by the third posture is continuously optimized at least once until the smoothness of the optimized posture trajectory satisfies the smoothness condition.

7. A point cloud pose optimization device, comprising: An initial pose acquisition module is used to obtain the first pose of each frame of point cloud in at least one point cloud pair corresponding to the target environment; The point cloud pair includes two adjacent frames of point clouds in the multi-frame point cloud corresponding to the target environment; A relative pose acquisition module is used to obtain the relative pose between two frames of point cloud in each point cloud pair; the relative pose is estimated by the odometry module; A first optimization module is configured to optimize the pose trajectory of the at least one point cloud pair according to the first pose and the relative pose to obtain a second pose of each frame of the point cloud; The first optimization module is specifically configured to perform the following operations on the first frame point cloud and the second frame point cloud in each point cloud pair: Constructing a first pose residual of the current point cloud pair according to the relative pose between the first frame point cloud and the second frame point cloud, and the first pose of the second frame point cloud; Optimizing the first pose residual to obtain optimized second poses of the first frame point cloud and the second frame point cloud; The first pose residual includes a first residual term and a second residual term, the first residual term representing the difference between the relative pose actually measured or calculated and the relative pose estimated using the odometer module, and the second residual term representing the difference between the first pose of the second frame point cloud actually measured and the GNSS pose used to initialize the second frame point cloud pose.

8. The point cloud pose optimization device according to claim 7, wherein the first optimization module is specifically configured to: Constructing a first residual term in the first pose residual of the current point cloud pair according to the relative pose between the first frame point cloud and the second frame point cloud; According to the first pose of the second frame point cloud, a second residual term in the first pose residual of the current point cloud pair is constructed.

9. The point cloud pose optimization device according to claim 7 or 8, further comprising: The second optimization module is used to optimize the posture trajectory according to the second posture and the relative posture to obtain a third posture of the point cloud of each frame.

10. The point cloud pose optimization device according to claim 9, wherein: The second optimization module is specifically configured to perform the following operations on the first frame point cloud and the second frame point cloud in each point cloud pair: When the absolute value of the distance difference between the second pose and the first pose of the second frame point cloud is greater than a preset distance threshold, constructing a second pose residual of the current point cloud pair according to the second poses of the first frame point cloud and the second frame point cloud, the relative pose between the first frame point cloud and the second frame point cloud, and the first pose of the second frame point cloud; The second pose residual is optimized to obtain an optimized third pose of the first frame point cloud and the second frame point cloud.

11. The point cloud pose optimization device according to claim 10, wherein: The second optimization module is specifically used for: Constructing a first residual term in the second pose residual of the current point cloud pair according to the second pose of the first frame point cloud and the second frame point cloud, and the relative pose between the first frame point cloud and the second frame point cloud; Construct a second residual term in the second pose residual of the current point cloud pair according to the first pose and the second pose of the second frame point cloud.

12. The point cloud pose optimization device according to claim 9, further comprising: a third optimization module, configured to determine whether the smoothness of the posture trajectory formed by the third posture satisfies a preset smoothness condition; In the case that the smoothness does not satisfy the smoothness condition, the posture trajectory formed in the third posture is continuously optimized at least once until the smoothness of the optimized posture trajectory satisfies the smoothness condition.

13. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the point cloud pose optimization method according to any one of claims 1 to 6.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to enable the computer to execute the point cloud pose optimization method according to any one of claims 1 to 6.

15. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the point cloud pose optimization method according to any one of claims 1 to 6.

Images