Method, device and storage medium for removing ghost images from maps

Abstract

The present application provides a method, device and storage medium for removing ghost images in mapping, which are applied in the field of data processing. The method includes: displaying a target trajectory containing a ghost image area; in response to a user's operation of connecting a reference frame and a frame to be corrected in a pair of associated frames in the ghost image area, displaying the time point information of the associated frame; aligning the point cloud corresponding to the frame to be corrected to the point cloud corresponding to the reference frame according to preset point cloud alignment information to obtain a relative pose; and optimizing the ghost image of the target trajectory according to the relative pose. The present application can effectively remove ghost images in mapping.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a method, device and storage medium for removing ghost images from a map. Background Art

[0002] Autonomous driving technology specifically refers to providing autonomous driving vehicles with prior maps based on high-precision maps during driving, and performing real-time perception, positioning, and route planning. Simultaneous Localization and Mapping (SLAM) based on data collected during the autonomous driving process is a low-cost way to obtain high-precision maps. Specifically, autonomous driving vehicles use laser radar equipment and other equipment to scan the road and surrounding environment to collect point cloud data, providing high-quality point cloud data for simultaneous positioning and mapping; in addition, autonomous driving vehicles also use data collected by wheel speed meters, combined inertial navigation and other equipment for simultaneous positioning and mapping to generate postures.

[0003] In the process of implementing this application, the inventors found that the above technology has at least the following problems: in the process of using point cloud data for simultaneous positioning and mapping, cumulative errors will be generated as the mapping is carried out. Although most of the cumulative errors can be removed through closed-loop detection, it cannot guarantee 100% successful closed-loop; in addition, there will be large matching errors when performing push-to-position matching, such as sharp turns, and both cumulative errors and matching errors will lead to ghosting. Summary of the invention

[0004] The embodiments of the present application provide a method, device and storage medium for removing ghost images in mapping to improve the ghost image removal effect.

[0005] In a first aspect, an embodiment of the present application provides a method for removing ghosting in a map, including: displaying a target trajectory containing a ghosting area; in response to a user's operation on the ghosting area to connect a reference frame and a frame to be corrected in a pair of associated frames, displaying time point information of the associated frames; aligning the point cloud corresponding to the frame to be corrected to the point cloud corresponding to the reference frame according to preset point cloud alignment information to obtain a relative pose; and performing ghosting optimization on the target trajectory according to the relative pose.

[0006] In a second aspect, an embodiment of the present application provides a device for removing ghost images from a map, including:

[0007] A display module is used to display the target track including the ghosting area; and in response to the user's operation of connecting the reference frame and the frame to be corrected in a pair of associated frames on the ghosting area, display the time point information of the associated frames;

[0008] The processing module is used to align the point cloud corresponding to the frame to be corrected to the point cloud corresponding to the reference frame according to the preset point cloud alignment information to obtain the relative posture; and to perform ghosting optimization on the target trajectory according to the relative posture.

[0009] In a third aspect, an embodiment of the present application provides an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the computer program.

[0010] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program runs on an electronic device, the electronic device executes the method described in the first aspect.

[0011] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program. When the computer program is executed on an electronic device, the electronic device executes the method as described in the first aspect.

[0012] The present application provides a method, device and storage medium for removing ghost images in a map. The method includes displaying a target trajectory containing a ghost image area; in response to a user's operation of connecting a reference frame and a frame to be corrected in a pair of associated frames in the ghost image area, displaying the time point information of the associated frame; aligning the point cloud corresponding to the frame to be corrected to the point cloud corresponding to the reference frame according to preset point cloud alignment information to obtain a relative posture; and optimizing the ghost image of the target trajectory according to the relative posture. The relative posture is obtained by aligning the point cloud corresponding to the frame to be corrected to the point cloud corresponding to the reference frame, and optimizing the ghost image of the target trajectory according to the relative posture to effectively remove the ghost image in the map.

[0013] These and other aspects of the present application will become more apparent from the following description of the embodiment(s). BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0015] Figure 1 A schematic diagram of an application scenario of the method for removing ghost images in mapping provided in this application;

[0016] Figure 2 A schematic diagram of a process for removing ghost images from a map provided by an embodiment of the present application;

[0017] Figure 3 is a target trajectory schematic diagram;

[0018] Figure 4 A schematic diagram of an interface for removing ghost images from a map provided in an embodiment of the present application;

[0019] Figure 5 for Figure 4 A partial enlarged schematic diagram of the interface shown;

[0020] Figure 6 A schematic diagram of an interface of an operation panel of a ghost removal tool provided in an embodiment of the present application;

[0021] Figure 7 A schematic diagram of another interface of the operation panel of the ghost removal tool provided in an embodiment of the present application;

[0022] Figure 8 Another schematic diagram of an interface for removing ghost images from a map provided in an embodiment of the present application;

[0023] Fig. 9 A schematic diagram of point clouds of a pair of associated frames provided in an embodiment of the present application before point cloud alignment;

[0024] Fig.10 for Fig. 9 The schematic diagram of the point cloud of the associated frame after point cloud alignment is shown;

[0025] Fig.11 A schematic diagram of the structure of a device for removing ghost images from a map provided in an embodiment of the present application;

[0026] Fig.12 A schematic diagram of the structure of an electronic device provided in one embodiment of the present application.

[0027] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0028] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0029] First, for ease of understanding, some terms in this application are explained:

[0030] Ghosting is an abnormal mapping effect in point cloud maps. Specifically, when a map is constructed based on data collected by laser radar, cameras and other equipment to obtain information about an object in reality, the resulting map contains multiple objects. In other words, an object that should be displayed as one is displayed as multiple objects. This is when ghosting occurs. For example, a point cloud map corresponding to a telephone pole is displayed as two telephone poles, which is an error.

[0031] Compared with ordinary electronic maps, high-precision maps are more accurate and contain richer target objects.

[0032] Point cloud data is a collection of point data for each sampling point on the surface of an object obtained by a measuring instrument. Specifically, the point cloud data obtained according to the principle of laser measurement includes three-dimensional coordinates (XYZ) and laser reflection intensity (Intensity); the point cloud data obtained according to the principle of photogrammetry includes three-dimensional coordinates (XYZ) and color information (RGB); the point cloud data obtained by combining the principles of laser measurement and photogrammetry includes three-dimensional coordinates (XYZ), laser reflection intensity (Intensity) and color information (RGB). For example, for point cloud data obtained according to the principle of laser measurement, when the laser beam is irradiated on the surface of an object, the laser reflected by the object will carry information such as direction and distance; if the laser beam is scanned along a certain trajectory, the reflected laser point information will be recorded while scanning. Since the scanning is extremely fine, a large number of laser points can be obtained, thus forming point cloud data with a large number of points and relatively dense.

[0033] Closed loop detection, also known as loop closure detection, refers to the ability of an autonomous vehicle to recognize that it has been to a certain scene, thus closing the map loop.

[0034] Registration, two point cloud sets corresponding to the intersecting scenes are matched, coordinate transformed, etc., to align one scene with another scene, which can also be understood as eliminating ghosting.

[0035] The pose file contains the pose corresponding to each frame of point cloud data. Each frame of point cloud data corresponds to a pose, and the point cloud data corresponding to all poses are superimposed to form the entire point cloud map. The format of the pose file is that each pose occupies one line, for example:

[0036] 1589443722.909254,-0.005656,-0.000536,0.010376,0.999930,0.013034,1.739831,0.300687

[0037] Among them, "1589443722.909254" is the time information, which is used to find the position; "-0.005656, -0.000536, 0.010376" are the x, y, and z three-dimensional coordinates respectively; "0.999930, 0.013034, 1.739831, 0.300687" is the rotation angle information expressed by quaternion.

[0038] In closed loop detection, as the camera and other measuring instruments move, the calculated pose and triangulated point cloud positions are subject to error. The most effective way to remove the accumulated error is to find the closed loop and optimize all the results based on the closed loop.

[0039] At present, in order to reduce the matching error, in the process of simultaneous positioning and mapping, the matching methods used in mapping mainly include iterative closest point (ICP), normal distribution transformation (NDT), and feature extraction. Due to the characteristics of sharp turns and few scene features in the acquisition process, these make it very difficult to match the mapping position, so there will be ghosting in the single-pass mapping results. The single-pass ghosting phenomenon is usually corrected by improving the robustness of the algorithm, but it cannot guarantee that the ghosting will be 100% removed.

[0040] In addition, it is difficult to guarantee 100% correct detection of loop closure. As the map is constructed, the accumulated error continues to increase, which brings great difficulties to loop closure. For the loop closure problem, the current mainstream methods include iterative nearest point, normal distribution transformation, semantic information, global positioning system (GPS) information, and manual inspection of data annotation after collection (but there is no efficient open source software available). Among them:

[0041] Both the iterative closest point and normal distribution transformation calculate the point distance matching score by searching within a certain distance radius R, and then perform matching calculation to analyze whether a closed loop is detected. In these two methods, it is not easy to set a fixed value for the distance radius R. Sometimes the cumulative error is large, and the distance radius R needs to be enlarged, but the amount of calculation will increase a lot after enlargement. Moreover, even if the matching calculation reaches the corresponding relationship, the distance radius R is relatively large and the scene is repeated and single, and there may be multiple places with high matching scores. It is also unreliable to distinguish whether a closed loop is detected based on the matching score threshold of the point distance.

[0042] Based on semantic information, such as curbs, lane lines, buildings, signboards, etc., closed-loop detection is assisted. This method relies heavily on the accuracy of the recognition results and only works when the scene features are obvious, so it has limitations.

[0043] Using GPS information is a common method to assist closed-loop detection, but there are many high-rise buildings in cities. If the GPS signal is poor during driving, there is no guarantee that the GPS information can be used to assist closed-loop detection.

[0044] By manually checking the data annotation after collection, the program can find the corresponding closed-loop detection information. Compared with the above three methods, although the closed loop can be detected, the efficiency is lower.

[0045] Furthermore, the existing ghost removal software is relatively cumbersome to operate and not user-friendly. Moreover, these software are used to fix closed-loop detection problems and cannot remove ghosting within a single pass.

[0046] In summary, the simultaneous positioning and mapping technology cannot guarantee 100% correct matching during the process of positioning and matching, and large matching errors usually occur in challenging scenarios. For closed-loop detection, the current mainstream methods are difficult to guarantee 100% correct detection or are time-consuming and labor-intensive.

[0047] In view of the above problems, the present application provides a method, device and storage medium for removing ghost images in mapping, which can correct ghost images in synchronous positioning and mapping through human-computer interaction. The present application can be applied to various services related to ghost image removal.

[0048] For example, Figure 1 Schematic diagram of the application scenario of the method for removing ghost images in the map provided in this application. Figure 1 As shown, in actual applications, after the client 11 performs synchronous positioning and mapping based on point cloud data, the point cloud map corresponding to the target trajectory obtained contains ghosting, and the user needs to correct the ghosting through the client 11. The user determines the ghosting area in the target trajectory displayed by the client 11, and connects the associated frames on the ghosting area through input devices such as a mouse and keyboard or touch, where the associated frames include a reference frame and a frame to be corrected; in response to this user operation, the client 11 displays the time point information of the associated frame, and aligns the point cloud corresponding to the frame to be corrected to the point cloud corresponding to the reference frame according to the preset point cloud alignment information to obtain the relative posture, and then optimizes the target trajectory according to the relative posture to remove the ghosting. Afterwards, the client 11 generates a high-precision map based on the point cloud data and posture data after removing the ghosting, and sends it to the server 12 through the network. The autonomous driving vehicle 13 calls the high-precision map from the server 12 to assist autonomous driving.

[0049] It should be noted that the attached Figure 1 This is only a schematic diagram of an application scenario provided by the embodiment of the present application. Figure 1 does not limit the equipment included in Figure 1The positional relationship between the devices in the Figure 1 In the application scenario shown, the client 11 performs synchronous positioning and mapping, removes ghost images, and generates a high-precision map. The above operations can also be performed by different clients. Figure 1 The application scenario shown may also include a data storage device, which may be an external memory relative to the client 11 and the server 12, or an internal memory integrated in the client 11 or the server 12. In addition, the server 12 may be an independent server, or a service cluster.

[0050] The technical solution of the present application is described in detail below through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0051] Figure 2 The present invention provides a method for removing ghost images from a map, which is applied to a device for removing ghost images from a map. The device can be implemented in software and / or hardware. Figure 1 In the illustrated scenario, the device for removing ghost images in mapping can be integrated into a client, for example, the device for removing ghost images in mapping is a chip or circuit in the client; or, the device for removing ghost images in mapping is the client. Next, an example is given with the client as the execution subject. The client can be, for example, a laptop computer, a desktop computer, a workbench, a personal digital assistant (PDA, also called a "palmtop computer"), a mainframe computer, and other suitable computers.

[0052] like Figure 2 As shown, the method for removing ghost images in mapping includes the following steps:

[0053] S201, displaying a target trajectory including a ghost area.

[0054] For example, the target trajectory containing the ghost area is as follows Figure 3 As shown. Among them, Figure 3 The target trajectory shown is the result of synchronous positioning and mapping, and is the driving trajectory of the autonomous vehicle. The round trip of the target trajectory is approximately 4 meters ( Figure 3 The ghost shown in 3.970381) is the closed-loop ghost. Figure 3, the ghosting area includes a reference trajectory and a trajectory to be corrected. The effect to be achieved by the embodiment of the present application is to use the reference trajectory as a reference, and correct the trajectory to be corrected to coincide with the reference trajectory to remove the ghosting. Among them, the frame corresponding to the reference trajectory is the reference frame, and the frame corresponding to the trajectory to be corrected is the frame to be corrected. Among them, △X-0.337700 represents the difference in X coordinates; △Y-3.955700 represents the difference in Y coordinates; △Z-0.048200 represents the difference in Z coordinates; △XY 3.970088 represents the difference on the XY plane; △XZ 0.341122 represents the difference on the XZ plane; △ZY 3.955994 represents the difference on the ZY plane.

[0055] In terms of data input requirements, the data required by the embodiments of the present application include point cloud data and pose data obtained after simultaneous positioning and mapping, wherein the point cloud data is stored in a point cloud file and the pose data is stored in a pose file.

[0056] This step loads the point cloud data and displays the target trajectory corresponding to the point cloud data so that the user can view the ghosting area. For example, the 3D point cloud processing software CloudCompare has basic functions of opening point cloud data and displaying, but this application is not limited to this.

[0057] S202 : In response to a user operation on a ghosting area to connect a reference frame and a frame to be corrected in a pair of associated frames, time point information of the associated frames is displayed.

[0058] After the user knows the ghosting area of ​​the target trajectory, he can connect the reference frame and the frame to be corrected in a pair of associated frames in the ghosting area, that is, associate the reference frame with the frame to be corrected to perform point cloud alignment in a targeted manner. The operation of connecting associated frames is not limited in the embodiment of the present application, and an example is given below:

[0059] In one example, the user uses the mouse to position the cursor at a point on the reference trajectory, and selects the frame corresponding to the point as the reference frame to be associated by clicking the left mouse button or other similar operations; then, the user uses the mouse to position the cursor at a point on the trajectory to be corrected, and selects the frame corresponding to the point as the frame to be associated with correction by clicking the left mouse button or other similar operations, thereby achieving the connection of a pair of associated frames.

[0060] In another example, the user uses the mouse to position the cursor at a point on the reference trajectory, and selects the frame corresponding to the point as the reference frame to be associated by clicking the left mouse button or other similar operations; then, the user uses the mouse to continuously drag the cursor to a point on the trajectory to be corrected, and selects the frame corresponding to the point as the frame to be associated with correction by clicking the left mouse button or other similar operations, thereby realizing the connection of a pair of associated frames.

[0061] After the user performs the above operation, the client responds to the above operation and displays the time point information of the associated frames, that is, the time point information of the reference frame and the time point information of the frame to be corrected. Since each frame on the target trajectory occurs at a different time, the reference frame and / or the frame to be corrected can be distinguished by the time point information.

[0062] Optionally, the method may further include: in response to a user's operation of connecting a reference frame and a frame to be corrected in a pair of associated frames in the ghosting area, displaying an optimized edge corresponding to the associated frame on the screen, such as Figure 4 Further, after displaying the time point information of the associated frame, the method may further include: in response to the user selecting the associated frame, displaying the area where the associated frame is located in the center of the screen, such as Figure 5 shown.

[0063] S203. According to preset point cloud alignment information, align the point cloud corresponding to the frame to be corrected to the point cloud corresponding to the reference frame to obtain a relative pose.

[0064] Among them, the preset point cloud alignment information may include at least one of the following information: X in three-dimensional coordinates, Y in three-dimensional coordinates, Z in three-dimensional coordinates, laser reflection intensity (Intensity), R in color information (RGB), G in color information (RGB), and B in color information (RGB). For example, if only the X coordinate value is different between the point cloud data corresponding to the frame to be corrected and the point cloud data corresponding to the reference frame, the preset point cloud alignment information only includes the X coordinate value. In addition, since the X coordinate value is unknown to the user, the preset point cloud alignment information can be set smaller at the beginning, and the size of the preset point cloud alignment information can be adjusted later to achieve the purpose of aligning the point cloud corresponding to the frame to be corrected to the point cloud corresponding to the reference frame, thereby obtaining the relative pose of the frame to be corrected relative to the reference frame in the pair of associated frames.

[0065] In one possible implementation, the point cloud corresponding to the reference frame and the point cloud corresponding to the frame to be corrected can be displayed in different colors. For example, the point cloud corresponding to the frame to be corrected is represented by red, and the point cloud corresponding to the reference frame is represented by green. At this time, the client adjusts the red point cloud according to the preset point cloud alignment information until it is aligned with the green point cloud. The client will automatically calculate the relative pose between the two frames corresponding to the red point cloud and the green point cloud. Optionally, these relative poses can be stored in the correction information file under the local relative directory, such as fixfile.txt, as correction information for closed-loop correction or front-to-back registration error correction, for easy search.

[0066] S204, performing ghost optimization on the target trajectory according to the relative posture.

[0067] The target trajectory is optimized based on the relative position obtained by point cloud alignment, so that ghosting can be eliminated. After that, synchronous positioning and mapping are performed based on the ghost-optimized data to obtain a high-precision map to assist autonomous driving.

[0068] The method for removing ghost images in a map provided by the embodiment of the present application first displays the target trajectory including the ghost image area, and in response to the user's operation of connecting the reference frame and the frame to be corrected in a pair of associated frames in the ghost image area, displays the time point information of the associated frame; then, according to the preset point cloud alignment information, aligns the point cloud corresponding to the frame to be corrected to the point cloud corresponding to the reference frame to obtain the relative posture; finally, ghosting is optimized for the target trajectory according to the relative posture. By aligning the point cloud corresponding to the frame to be corrected to the point cloud corresponding to the reference frame to obtain the relative posture, and ghosting is optimized for the target trajectory according to the relative posture, the map ghosting is effectively removed, including ghosting caused by single-pass matching errors and ghosting caused by difficulty in closed-loop detection.

[0069] Based on the above, as a possible embodiment, the ghost removal method provided by this application can be implemented based on CloudCompare open source software. Based on the existing functions of CloudCompare open source software, a ghost removal tool plug-in is designed to perform closed-loop correction and correct the front and back registration errors. For example, the operation panel of the ghost removal tool is as follows: Figure 6 Reference Figure 6 The operation panel contains three areas, namely: Select correction points, Point cloud alignment and Ghost removal, where:

[0070] 1. Select the correction point

[0071] This area contains merge controls, add controls, delete controls, data cropping controls, and a panel list. Specifically, the merge control is used to merge point cloud trajectories; the add control is used to add a new associated frame, wherein the connection between the reference frame contained in the associated frame and the frame to be corrected is the optimized edge, that is, the constraint edge information when the point cloud is aligned; the delete control is used to delete the associated frame; the data cropping control is used to crop the point cloud data according to the corresponding time point information to retain the point cloud data corresponding to the associated frame. The time point information of a pair of associated frames added by the add control is displayed in the panel list; the delete control is used to delete the associated frame selected in the panel list. The panel list contains three columns of data, specifically: the serial number corresponding to the associated frame, the reference frame contained in the associated frame, and the frame to be corrected. Therefore, the above-mentioned display of the time point information of the associated frame may include: in the operation panel of the ghosting removal tool, displaying the time point information of the associated frame, such as Figure 7 The example given.

[0072] 2. Point Cloud Alignment

[0073] This area contains controls for rotating and / or translating the frame to be rectified, and can set the step size information for rotation and translation.

[0074] As described above, the preset point cloud alignment information includes the pose information set by the user in real time in the area. After the user selects the associated frame, the preset point cloud alignment information is set in the area. Afterwards, the client aligns the point cloud corresponding to the frame to be corrected to the point cloud corresponding to the reference frame according to the preset point cloud alignment information to obtain the relative pose. Specifically, the client performs the following steps until the point cloud of the frame to be corrected is aligned with the point cloud corresponding to the reference frame, and determines that the correction rotation matrix at this time is the relative pose:

[0075] According to the preset point cloud alignment information, the correction rotation matrix is ​​obtained;

[0076] Multiply the point cloud corresponding to the frame to be corrected by the correction rotation matrix to obtain the corrected point cloud;

[0077] Display the corrected point cloud;

[0078] If the corrected point cloud is not aligned with the point cloud corresponding to the reference frame, in response to the user's operation of updating the point cloud alignment information, the updated point cloud alignment information is used as new preset point cloud alignment information.

[0079] 3. Ghosting Removal

[0080] This area contains ghosting optimization and output controls for ghosting optimization, and virtual controls for plane optimization. When the virtual controls are selected as attributes, plane constraint optimization is performed on each frame of data after ghosting optimization. Optionally, the save address of the ghosting optimized data is pre-set for subsequent search, viewing or application. For example, the save address can be a storage medium inside the client, or it can also be a storage medium outside the client, such as a cloud server, etc.

[0081] In addition, the Operations panel contains an off control for turning off the Ghost Removal Tool, such as Figure 7 Shown in the upper right corner.

[0082] The ghost removal tool is simple to operate and easy to understand, and the manual interaction process is highly efficient. It is a plug-in developed based on the open source software CloudCompare. While using a large number of open source functions, it can more conveniently perform ghost removal for simultaneous positioning and mapping.

[0083] Next, the specific implementation of the mapping ghost removal method provided in this application is explained in conjunction with the application of the ghost removal tool.

[0084] In some embodiments, S201, displaying the target trajectory containing the ghosting area, may include: when detecting that the user clicks the merge control instruction, displaying the candidate point cloud file; when detecting that the user selects the target point cloud file in the candidate point cloud file instruction, obtaining multiple target point cloud files, the target point cloud files contain data corresponding to part of the trajectory in the target trajectory; merging the trajectories of the multiple target point cloud files and displaying the target trajectory. Specifically, when the user clicks the merge control, the client displays the file directory; after the user finds the directory where the point cloud file is located, the point cloud file is selected for merging. Since the amount of data to be processed is large, the data input for the usual operation is a lot of point cloud files, so a trajectory merge is required to facilitate the selection of subsequent associated frames.

[0085] When the user selects the associated frame (correction point), first select the frame to be corrected, and then select the reference frame. For example, the user clicks the add control, and then according to the trajectory point corresponding to the ghosting area, Figure 3 , add 5 pairs of associated frames, such as Figure 4 Further, S202, in response to the user's operation of connecting a reference frame and a frame to be corrected in a pair of associated frames in the ghosting area, displaying the time point information of the associated frames may include: when detecting the user's operation of connecting a reference frame and a frame to be corrected in a pair of associated frames in the ghosting area, displaying information on whether to confirm loading; if detecting the user's instruction to confirm loading, displaying the time point information of the associated frames. For example, when the user selects the second track point, the client will prompt whether to confirm loading, such as Figure 8 As shown, after the user confirms the loading, the client displays the time point information of the associated frame in the panel list, and the optimized edge is displayed in the ghosting area.

[0086] Optionally, if the user double-clicks an item in the sequence number, the client responds to the operation and automatically positions the optimized edge corresponding to the sequence number to the center of the screen.

[0087] When the user clicks the data cropping control, the client performs data cropping according to the selected associated frame, and retains the point cloud data corresponding to the associated frame and the frames adjacent to the associated frame. Therefore, in some embodiments, before aligning the point cloud corresponding to the frame to be corrected to the point cloud corresponding to the reference frame according to the preset point cloud alignment information to obtain the relative pose, the method may also include: in response to the data cropping operation performed by the user on the operation panel of the ghost removal tool, performing data cropping on the point cloud data corresponding to the target trajectory, retaining the point cloud data corresponding to the associated frame and the N frames adjacent to the associated frame before and after, and displaying a pair of point clouds corresponding to the associated frames on the screen, such as Fig. 9 As shown in the figure, the value of N is determined based on actual needs or historical experience.

[0088] If there are multiple pairs of associated frames, then the above-mentioned displaying the point cloud corresponding to a pair of associated frames on the screen includes: displaying the point cloud corresponding to the first pair of associated frames on the screen. At this time, the method may also include: aligning the point cloud corresponding to the frame to be corrected in the first pair of associated frames to the point cloud corresponding to the reference frame in the first pair of associated frames (such as Fig.10 After the point cloud alignment of the next group of associated frames is performed in response to the user's operation on the operation panel to perform point cloud alignment on the next group of associated frames, the relative pose corresponding to the first pair of associated frames is saved, and the point cloud corresponding to the second pair of associated frames is displayed on the screen. The step of aligning the point cloud corresponding to the frame to be corrected to the point cloud corresponding to the reference frame according to the preset point cloud alignment information to obtain the relative pose is performed for the point cloud corresponding to the second pair of associated frames, until the point cloud alignment of multiple pairs of associated frames is completed. Optionally, after the point cloud alignment of multiple pairs of associated frames is completed, the client can pop up a box to prompt that all alignments have been completed.

[0089] Optionally, the method may further include: loading point clouds corresponding to multiple pairs of associated frames through a progress bar program, and the loading order of the point clouds corresponding to the multiple pairs of associated frames is determined according to the associated time points of the associated frames. It can be understood that the associated time points can be expressed as serial numbers in the panel list, and the serial numbers of the associated frames that are associated first are arranged in the front and have smaller serial numbers; the serial numbers of the associated frames that are associated later are arranged in the back and have larger serial numbers.

[0090] In some embodiments, S204, performing ghost optimization on the target trajectory according to the relative posture, may include: obtaining a posture file, the posture file including the posture of each frame corresponding to the target trajectory; obtaining the posture change between adjacent frames according to the posture of each frame; performing ghost optimization on the target trajectory according to the relative posture, the posture of each frame and the posture change. The ghost optimization includes closed-loop optimization and front-back matching optimization.

[0091] Optionally, obtaining the pose file may include: displaying the candidate pose file in response to a ghost optimization operation performed by the user on the operation panel of the ghost removal tool; obtaining the pose file in response to a selection operation of a pose file in the candidate pose files by the user. For example, the user clicks on a virtual control labeled "ghost optimization and output". At this time, the ghost optimization operation is a click operation. In response to the operation, the client displays the candidate pose file in a pop-up box. After the user selects, the client obtains the pose file.

[0092] Among them, according to the relative posture, the posture of each frame and the posture change, the target trajectory is optimized for ghosting, including: taking the posture change as the optimization edge, setting the weight of the optimization edge to a preset value, and the posture of each frame as a G2O optimization node, and performing nonlinear optimization on the posture of each frame; according to the optimized posture, output the point cloud map data after ghosting is removed. Here, the G2O library is used for nonlinear optimization, each posture is a G2O optimization node, and the posture transformation between adjacent postures is the optimization edge. When optimizing, the weight of the optimization edge is assigned to a preset value, such as 1, to complete the nonlinear optimization.

[0093] In some embodiments, after performing ghost optimization on the target trajectory according to the relative posture in S204, the method may further include: performing plane constraint optimization on each frame data after ghost optimization. Optionally, this step may include: extracting the ground relative to each frame data after ghost optimization; calculating the normal vector of each frame data perpendicular to the ground, using the (0,0,1) vector and each normal vector to form a constraint edge, so as to horizontally optimize each frame data after ghost optimization by constraining the ground.

[0094] That is to say, the ghosting removal method provided in the embodiment of the present application supports plane constraint optimization. If the point cloud area is known to be close to a plane, the plane constraint can be checked for optimization to convert all point cloud data to the same height to prevent the ground from drifting. This method refers to the plane constraint method of the hdl_graph_slam open source framework. By extracting the ground from each frame of the point cloud, calculating the normal vector perpendicular to the ground, and using the (0,0,1) vector to form a constraint edge with each ground normal vector, the ground is constrained for horizontal optimization.

[0095] It should be noted that when each of the above operations is performed, corresponding log information will be printed to the log column for subsequent viewing.

[0096] In view of the above front-end operation logic and introduction, the following explains the client's calculation and processing logic:

[0097] 1) Select the correction point. In this process, the client reads all the selected point cloud files, merges them into a large trajectory file, and loads the trajectory to display in the current interface. By clicking the selected associated frame on the front end, the client obtains the time point information of the associated frame and displays it in the panel list. When performing data cropping operations, the point cloud data corresponding to each pair of associated frames will be recorded in the client memory, and the point cloud data corresponding to the first pair of associated frames will be displayed on the screen.

[0098] 2) Point cloud alignment. When the user rotates and translates the point cloud data on the front end, the client achieves the point cloud rotation effect by multiplying the point cloud by the correction rotation matrix, adding and subtracting the x, y, and z coordinates and updating the calculation results to the display interface, and finally achieves the rotation and translation effect. When the front end finally aligns the two point cloud data, the client will accumulate and calculate the difference in posture before and after alignment to obtain the relative posture, and record the relative posture.

[0099] 3) Ghost removal. This operation is implemented through the nonlinear optimization algorithm G2O library. The pose transformation between adjacent poses is used as the optimization edge, and the weight of the optimization edge is set to the preset value by default. The pose is used as the G2O optimization node, and finally the pose is subjected to back-end nonlinear optimization. After optimization, the corrected point cloud map data is output according to the new pose.

[0100] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0101] Fig.11 The present invention provides a schematic diagram of the structure of a device for removing ghost images in a map construction according to an embodiment of the present invention. The present invention provides a device for removing ghost images in a map construction according to an embodiment of the present invention, and the device can be integrated in a client. Fig.11 As shown, the mapping ghost removal device 110 includes: a display module 111 and a processing module 112. Among them:

[0102] The display module 111 is used to display the target track including the ghosting area; and, in response to the user's operation of connecting the reference frame and the frame to be corrected in a pair of associated frames on the ghosting area, display the time point information of the associated frames.

[0103] The processing module 112 is used to align the point cloud corresponding to the frame to be corrected to the point cloud corresponding to the reference frame according to preset point cloud alignment information to obtain a relative posture; and to perform ghosting optimization on the target trajectory according to the relative posture.

[0104] The device provided in the embodiment of the present application can be used to perform Figure 2 The implementation principle and technical effect of the method in the illustrated embodiment are similar and will not be described in detail here.

[0105] Optionally, the preset point cloud alignment information includes posture information set in real time by the user on the operation panel of the ghost removal tool.

[0106] Optionally, when the display module 111 is used to display the time point information of the associated frame, it is specifically used to: display the time point information of the associated frame in the operation panel of the ghost removal tool.

[0107] In some embodiments, when the processing module 112 is used to align the point cloud corresponding to the frame to be corrected to the point cloud corresponding to the reference frame according to the preset point cloud alignment information to obtain the relative pose, it is specifically used to:

[0108] Perform the following steps until the point cloud of the frame to be corrected is aligned with the point cloud corresponding to the reference frame, and the correction rotation matrix when the point cloud is aligned is determined as the relative pose:

[0109] According to the preset point cloud alignment information, the correction rotation matrix is ​​obtained;

[0110] Multiply the point cloud corresponding to the frame to be corrected by the correction rotation matrix to obtain the corrected point cloud;

[0111] Display the corrected point cloud;

[0112] If the corrected point cloud is not aligned with the point cloud corresponding to the reference frame, in response to the user's operation of updating the point cloud alignment information, the updated point cloud alignment information is used as new preset point cloud alignment information.

[0113] In some embodiments, when the processing module 112 is used to perform ghosting optimization on the target trajectory according to the relative posture, it is specifically used to: obtain a posture file, the posture file contains the posture of each frame corresponding to the target trajectory; according to the posture of each frame, obtain the posture change between adjacent frames; according to the relative posture, the posture of each frame and the posture change, perform ghosting optimization on the target trajectory.

[0114] In some embodiments, when the processing module 112 is used to obtain a pose file, it is specifically used to: display a candidate pose file in response to a ghosting optimization operation performed by a user on the operation panel of a ghosting removal tool; and obtain a pose file in response to a user selecting a pose file from the candidate pose files.

[0115] In some embodiments, when the processing module 112 is used to perform ghosting optimization on the target trajectory according to the relative posture, the posture of each frame and the posture change, it is specifically used to: use the posture change as the optimization edge, set the weight of the optimization edge to a preset value, use the posture of each frame as a G2O optimization node, and perform nonlinear optimization on the posture of each frame; and output the point cloud map data after ghosting is removed according to the optimized posture.

[0116] In some embodiments, the processing module 112 can also be used for: before aligning the point cloud corresponding to the frame to be corrected to the point cloud corresponding to the reference frame according to preset point cloud alignment information, and obtaining the relative posture, in response to the data cropping operation performed by the user on the operation panel of the ghosting removal tool, performing data cropping on the point cloud data corresponding to the target trajectory, retaining the point cloud data corresponding to the associated frame and the N frames adjacent to the associated frame, and triggering the display module 111 to display a pair of point clouds corresponding to the associated frames on the screen.

[0117] In some embodiments, if multiple pairs of associated frames are included, the display module 111 is specifically used to display the point cloud corresponding to a pair of associated frames on the screen. The processing module 112 is also used to: after aligning the point cloud corresponding to the frame to be corrected in the first pair of associated frames to the point cloud corresponding to the reference frame in the first pair of associated frames, in response to the user's operation on the operation panel to perform point cloud alignment on the next group of associated frames, save the relative posture corresponding to the first pair of associated frames, display the point cloud corresponding to the second pair of associated frames on the screen, and perform the step of aligning the point cloud corresponding to the frame to be corrected to the point cloud corresponding to the reference frame according to the preset point cloud alignment information for the point cloud corresponding to the second pair of associated frames to obtain the relative posture, until the point cloud alignment of multiple pairs of associated frames is completed.

[0118] In some embodiments, the processing module 112 may also be used to: load point clouds corresponding to multiple pairs of associated frames through a progress bar program, and the loading order of the point clouds corresponding to the multiple pairs of associated frames is determined according to the associated time points of the associated frames.

[0119] In some embodiments, the display module 111 may also be used to: in response to a user's operation of connecting a reference frame and a frame to be corrected in a pair of associated frames in the ghosting area, display an optimized edge corresponding to the associated frames on the screen.

[0120] In some embodiments, the display module 111 may also be configured to: after displaying the time point information of the associated frame, in response to a user selecting the associated frame, display the area where the associated frame is located in the center of the screen.

[0121] In some embodiments, when the display module 111 is used to respond to the user's operation of connecting a reference frame and a frame to be corrected in a pair of associated frames in the ghosting area and display the time point information of the associated frames, specifically: when the user's operation of connecting a reference frame and a frame to be corrected in a pair of associated frames in the ghosting area is detected, information on whether to confirm the loading is displayed; if the user's instruction to confirm the loading is detected, the time point information of the associated frames is displayed.

[0122] Optionally, the operation panel of the ghost removal tool includes a merge control for merging trajectories. In this case, when the display module is used to display the target trajectory containing the ghost area, it is specifically used to: display the candidate point cloud file when it is detected that the user clicks the merge control instruction; obtain multiple target point cloud files when it is detected that the user selects the target point cloud file in the candidate point cloud file, and the target point cloud file includes data corresponding to some trajectories in the target trajectory; merge the trajectories of the multiple target point cloud files and display the target trajectory.

[0123] Furthermore, the processing module 112 may also be used to: after ghost optimization is performed on the target trajectory according to the relative posture, plane constraint optimization is performed on each frame data after ghost optimization.

[0124] Furthermore, when the processing module 112 is used to perform plane constraint optimization on each frame data after ghosting optimization, it is specifically used to: extract the ground relative to each frame data after ghosting optimization; calculate the normal vector of each frame data perpendicular to the ground, and use the (0,0,1) vector and each normal vector to form a constraint edge to horizontally optimize each frame data after ghosting optimization by constraining the ground.

[0125] Optionally, the operation panel of the ghost removal tool includes a virtual control for plane optimization. When the virtual control is a selected attribute, plane constraint optimization is performed on each frame data after ghost optimization.

[0126] Optionally, the operation panel of the ghost removal tool includes a delete control for deleting associated frames, and / or an add control for adding new associated frames.

[0127] It should be noted that it should be understood that the division of the various modules of the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software called by processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. For example, the processing module can be a separately established processing element, or it can be integrated in a chip of the above device. In addition, it can also be stored in the memory of the above device in the form of program code, and called and executed by a processing element of the above device. The implementation of other modules is similar. In addition, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each module above can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.

[0128] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASIC), or one or more microprocessors (DSP), or one or more field programmable gate arrays (FPGA). For another example, when a module is implemented in the form of a processing element scheduling program code, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0129] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0130] Fig.12 Schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device is, for example, the client as described above. Fig.12As shown, the electronic device may include: a processor 81, a memory 82, a communication interface 83 and a system bus 84. The memory 82 and the communication interface 83 are connected to the processor 81 through the system bus 84 and communicate with each other. The memory 82 is used to store computer programs, the communication interface 83 is used to communicate with other devices, and the processor 81 is used to call the computer program in the memory to execute the solution described in the above-mentioned embodiment of the method for removing ghost images from a map.

[0131] Should Fig.12 The system bus 84 mentioned in the figure can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The system bus 84 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0132] The communication interface 83 is used to implement the communication between the database access device and other devices (such as clients, read-write libraries and read-only libraries).

[0133] The memory 82 may include a random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory.

[0134] The processor 81 can be a general-purpose processor, including a central processing unit, a network processor (NP), etc.; it can also be a digital signal processor DSP, an application-specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0135] An embodiment of the present application also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed on an electronic device, the electronic device executes the method for removing ghost images from a map as described in any of the above method embodiments.

[0136] An embodiment of the present application also provides a chip for running instructions, and the chip is used to execute the method for removing ghost images from mapping as described in any of the above method embodiments.

[0137] An embodiment of the present application also provides a computer program product, which includes a computer program, which is stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium, and when the at least one processor executes the computer program, it can implement the method for removing ghost images in mapping as described in any of the above method embodiments.

[0138] In the present application, "at least one" means one or more, and "plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship; in the formula, the character " / " indicates that the previous and next associated objects are in a "division" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0139] It is to be understood that the various numerical numbers involved in the embodiments of the present application are only for the convenience of description and are not intended to limit the scope of the embodiments of the present application. In the embodiments of the present application, the size of the sequence number of each process does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0140] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for removing ghost images in mapping, characterized in that: include: Displaying a target trajectory including a ghosting area, wherein the target trajectory in the ghosting area includes a reference trajectory and a trajectory to be corrected, a frame corresponding to the reference trajectory is a reference frame, and a frame corresponding to the trajectory to be corrected is a frame to be corrected; In response to a user's operation of connecting a reference frame and a frame to be corrected in a pair of associated frames in the ghosting area, displaying time point information of the associated frames; According to preset point cloud alignment information, align the point cloud corresponding to the frame to be corrected to the point cloud corresponding to the reference frame to obtain a relative pose; Obtaining a pose file, wherein the pose file includes the poses of each frame corresponding to the target trajectory; According to the poses of the frames, obtaining pose changes between adjacent frames; According to the relative posture, the posture of each frame and the posture change, ghosting optimization is performed on the target trajectory, wherein the ghosting optimization includes closed-loop optimization and front-back matching optimization; The ghosting optimization of the target trajectory according to the relative posture, the posture of each frame and the posture change includes: The relative posture is used as correction information for closed-loop optimization or front-to-back matching optimization, the posture change is used as an optimization edge, the weight of the optimization edge is set to a preset value, the posture of each frame is used as a G2O optimization node, and the posture of each frame is nonlinearly optimized; according to the optimized posture, the point cloud map data after ghosting is removed is output.

2. The method according to claim 1, characterized in that The preset point cloud alignment information includes the posture information set in real time by the user on the operation panel of the ghost removal tool.

3. The method according to claim 1, characterized in that The displaying the time point information of the associated frame includes: displaying the time point information of the associated frame in an operation panel of a ghost removal tool.

4. The method according to claim 1, characterized in that: The step of aligning the point cloud corresponding to the frame to be corrected to the point cloud corresponding to the reference frame according to the preset point cloud alignment information to obtain a relative pose includes: The following steps are performed until the point cloud of the frame to be corrected is aligned with the point cloud corresponding to the reference frame, and the correction rotation matrix when the point clouds are aligned is determined to be the relative pose: Obtaining a correction rotation matrix according to the preset point cloud alignment information; Multiplying the point cloud corresponding to the frame to be corrected by the correction rotation matrix to obtain a corrected point cloud; Displaying the corrected point cloud; If the corrected point cloud is not aligned with the point cloud corresponding to the reference frame, in response to the user's operation of updating the point cloud alignment information, the updated point cloud alignment information is used as new preset point cloud alignment information.

5. The method according to claim 1, characterized in that The obtaining of the pose file comprises: In response to a ghost optimization operation performed by a user on an operation panel of the ghost removal tool, displaying a candidate pose file; In response to a user selecting the pose file in the candidate pose files, the pose file is acquired.

6. The method according to any one of claims 1 to 5, characterized in that Before aligning the point cloud corresponding to the frame to be corrected to the point cloud corresponding to the reference frame according to the preset point cloud alignment information to obtain the relative pose, the method further includes: In response to the data cropping operation performed by the user on the operation panel of the ghost removal tool, the point cloud data corresponding to the target trajectory is cropped, the point cloud data corresponding to the associated frame and the N frames adjacent to and before the associated frame are retained, and a pair of point clouds corresponding to the associated frames are displayed on the screen.

7. The method according to claim 6, characterized in that If multiple pairs of associated frames are included, then displaying point clouds corresponding to a pair of associated frames on the screen includes: displaying point clouds corresponding to a first pair of associated frames on the screen; The method also includes: after aligning the point cloud corresponding to the frame to be corrected in the first pair of associated frames to the point cloud corresponding to the reference frame in the first pair of associated frames, in response to the user's operation on the operation panel to align the point clouds of the next group of associated frames, saving the relative posture corresponding to the first pair of associated frames, and displaying the point cloud corresponding to the second pair of associated frames on the screen, executing the step of aligning the point cloud corresponding to the frame to be corrected to the point cloud corresponding to the reference frame according to the preset point cloud alignment information to obtain the relative posture for the point cloud corresponding to the second pair of associated frames, until the point cloud alignment of the multiple pairs of associated frames is completed.

8. The method according to claim 7, characterized in that Also includes: The point clouds corresponding to the multiple pairs of associated frames are loaded through a progress bar program, and the loading order of the point clouds corresponding to the multiple pairs of associated frames is determined according to the associated time points of the associated frames.

9. The method according to any one of claims 1 to 5, characterized in that Also includes: In response to the user's operation of connecting the reference frame and the frame to be corrected in a pair of associated frames in the ghosting area, the optimized edges corresponding to the associated frames are displayed on the screen.

10. The method according to claim 9, characterized in that After displaying the time point information of the associated frame, the method further includes: In response to the user's operation of selecting the associated frame, the area where the associated frame is located is displayed in the center of the screen.

11. The method according to any one of claims 1 to 5, characterized in that The method of displaying the time point information of a pair of associated frames in response to the user's operation of connecting the reference frame and the frame to be corrected in the ghosting area comprises: When detecting a user's operation of connecting a reference frame and a frame to be corrected in a pair of associated frames in the ghosting area, displaying information on whether to confirm the loading; If a user confirmation instruction for loading is detected, the time point information of the associated frame is displayed.

12. The method according to any one of claims 1 to 5, characterized in that The Ghost Removal Tool's Operation Panel contains Merge Controls for merging tracks, which display target tracks containing ghost areas, including: When detecting that the user clicks the instruction of the merge control, displaying the candidate point cloud file; When detecting that a user has selected an instruction of a target point cloud file in the candidate point cloud file, acquiring a plurality of the target point cloud files, wherein the target point cloud files include data corresponding to a portion of the target trajectory; The multiple target point cloud files are merged into a track to display the target track.

13. The method according to any one of claims 1 to 5, characterized in that After performing ghost optimization on the target trajectory according to the relative posture, the method further includes: Plane constraint optimization is performed on each frame data after ghosting optimization.

14. The method according to claim 13, characterized in that The performing plane constraint optimization on each frame data after ghosting optimization includes: Extract the ground from each frame data after relative ghosting optimization; The normal vector of each frame data perpendicular to the ground is calculated, and a constraint edge is formed between the (0,0,1) vector and each normal vector, so as to horizontally optimize each frame data after ghosting optimization by constraining the ground.

15. The method according to claim 13, characterized in that The operation panel of the ghost removal tool includes a virtual control for plane optimization. When the virtual control is a selected attribute, plane constraint optimization is performed on each frame data after ghost optimization.

16. The method according to any one of claims 1 to 5, characterized in that The operation panel of the ghost removal tool includes a delete control for deleting the associated frame and an add control for adding a new associated frame.

17. A device for removing ghost images from a map, characterized in that: include: A display module is configured to display a target trajectory including a ghosting area, wherein the target trajectory in the ghosting area includes a reference trajectory and a trajectory to be corrected, the frame corresponding to the reference trajectory is a reference frame, and the frame corresponding to the trajectory to be corrected is a frame to be corrected; and, in response to a user's operation of connecting a reference frame and a frame to be corrected in a pair of associated frames on the ghosting area, display time point information of the associated frames; A processing module is used to align the point cloud corresponding to the frame to be corrected to the point cloud corresponding to the reference frame according to preset point cloud alignment information to obtain a relative pose; and obtain a pose file, wherein the pose file contains the poses of each frame corresponding to the target trajectory; according to the poses of each frame, obtain the pose changes between adjacent frames; according to the relative pose, the poses of each frame and the pose changes, perform ghosting optimization on the target trajectory, wherein the ghosting optimization includes closed-loop optimization and front-to-back matching optimization; when the processing module is used to perform ghosting optimization on the target trajectory according to the relative pose, the poses of each frame and the pose changes, the relative pose is used as correction information for closed-loop optimization or front-to-back matching optimization, the pose change is used as an optimization edge, the weight of the optimization edge is set to a preset value, the pose of each frame is used as a G2O optimization node, and the pose of each frame is nonlinearly optimized; according to the optimized pose, the point cloud map data after ghosting is removed is output.

18. An electronic device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 16 is implemented.

19. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 16.

20. A computer program product comprising a computer program, characterized in that When the computer program is executed on an electronic device, the electronic device is enabled to perform the method according to any one of claims 1 to 16.

Images