Ghost Detection Method for Point Cloud Map

Abstract

The present application discloses a method for ghost detection of point cloud maps, and belongs to the field of electronic information technology. The method includes: dividing the point cloud map to be detected into multiple grids; determining the query key frame from the local point cloud trajectory included in the target grid; determining at least one matching frame of the query key frame in the multi-frame point cloud trajectory; determining the registration results of the query key frame and each matching frame of the query key frame; in response to the existence of a registration result that meets the requirements, determining the target grid as a grid with ghosts. In the present application, by dividing the point cloud map into grids for detection, full coverage of ghost detection in the point cloud map is achieved, and the detection granularity is finer. The query key frame and the matching frame are selected for registration to improve the accuracy of detection, so that an accurate high-precision map can be constructed based on the point cloud map after ghost detection, and then the normal driving of the autonomous driving vehicle is guaranteed based on the accurate high-precision map. Since there is no need to extract specific elements, the restrictions of the scene on ghost detection are reduced.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of electronic information technology, and in particular to a method for detecting ghosting in a point cloud map. Background Art

[0002] At present, electronic high-precision maps have become an indispensable part of many travel scenarios, especially in the field of autonomous driving. Electronic high-precision maps are an important prerequisite for ensuring the normal driving of autonomous vehicles, and the construction of point cloud maps is an indispensable part of the production process of electronic high-precision maps. Point cloud is a collection of a series of scattered points with accurate angle and distance information presented by radar to collect information about objects in the surrounding environment. The map constructed using point cloud is called point cloud map. The quality of point cloud map determines the accuracy of electronic high-precision map, and the presence of ghosting in point cloud map determines the quality of point cloud map. Therefore, it is necessary to detect ghosting on point cloud map. Summary of the invention

[0003] The embodiments of the present application provide a method, device, equipment and storage medium for ghost detection of a point cloud map, which can be used to improve the accuracy of detection and reduce the restrictions of the scene on ghost detection. The technical solution is as follows:

[0004] On the one hand, an embodiment of the present application provides a method for detecting ghosting in a point cloud map, the method comprising:

[0005] Divide the point cloud map to be detected into multiple grids, the point cloud map includes multiple frame point cloud trajectories, and each grid includes a local point cloud trajectory in the multiple frame point cloud trajectories;

[0006] Determine a query keyframe from the local point cloud trajectory included in the target mesh;

[0007] Determine at least one matching frame of the query key frame in the multi-frame point cloud trajectory, where the matching frame is a local point cloud trajectory within the reference range where the query key frame is located;

[0008] Determine the registration results of the query key frame and each matching frame of the query key frame;

[0009] In response to the existence of a registration result that satisfies the requirement, the target mesh is determined as a mesh with ghosts.

[0010] In a possible implementation, determining a query key frame from a local point cloud trajectory included in a target grid includes:

[0011] Determine the number of road layers in the target grid;

[0012] A reference number of point cloud trajectories is determined from the local point cloud trajectories included in the target grid as query key frames, and the reference number is determined based on the number of road layers.

[0013] In a possible implementation, determining at least one matching frame of a query key frame in a multi-frame point cloud trajectory includes:

[0014] Filter out the local point cloud trajectory within the reference range from the multi-frame point cloud trajectory;

[0015] In response to the reference range including a plurality of local point cloud trajectories, a local point cloud trajectory satisfying a time interval among the plurality of local point cloud trajectories is determined as a matching frame of the query key frame.

[0016] In a possible implementation, determining the registration result of the query key frame and each matching frame of the query key frame includes:

[0017] Determine a subgraph of the query keyframe, where the subgraph of the query keyframe is obtained based on the point cloud trajectory included in the query keyframe;

[0018] For a query key frame and any matching frame, a subgraph of any matching frame is determined, where the subgraph of any matching frame is obtained based on a point cloud trajectory included in any matching frame;

[0019] Determine a plurality of pose transformation matrices based on a sub-graph of the query key frame and a sub-graph of any matching frame, wherein any pose transformation matrix includes at least one of a rotation transformation parameter and a translation transformation parameter for transforming the query key frame to any matching frame;

[0020] The pose transformation matrices that meet the requirements among multiple pose transformation matrices are averaged and the calculated result is used as the registration result between the query key frame and any matching frame.

[0021] In a possible implementation, determining a subgraph of a query key frame includes:

[0022] The point cloud tracks included in the query key frame and the related point cloud tracks of the query key frame are merged to obtain a subgraph of the query key frame, and the related point cloud tracks of the query key frame include the point cloud tracks adjacent to the query key frame timestamp in the target grid.

[0023] In a possible implementation, determining a sub-image of any matching frame includes:

[0024] The point cloud trajectory included in any matching frame is merged with the relevant point cloud trajectory of any matching frame to obtain a sub-image of any matching frame, and the relevant point cloud trajectory of any matching frame includes the point cloud trajectory adjacent to the timestamp of any matching frame in the grid where any matching frame is located.

[0025] In a possible implementation, multiple pose transformation matrices are determined based on a sub-image of a query key frame and a sub-image of any matching frame, including:

[0026] Determine multiple sets of local point cloud data corresponding to the sub-image of the query key frame and global point cloud data corresponding to the sub-image of any matching frame;

[0027] Multiple groups of local point cloud data are respectively aligned with the global point cloud data to obtain multiple pose transformation matrices. One group of local point cloud data corresponds to one pose transformation matrix. The pose transformation matrix corresponding to any group of local point cloud data includes at least one of the rotation transformation parameters and translation transformation parameters for transforming the query key frame to any matching frame through any group of local point cloud data.

[0028] In a possible implementation, determining multiple groups of local point cloud data corresponding to a sub-image of a query key frame includes:

[0029] In the global point cloud data included in the sub-graph of the query key frame, point cloud data that meets the ratio requirement is screened out according to different regions, and the point cloud data that meets the ratio requirement screened out in each region is used as a group of local point cloud data corresponding to the sub-graph of the query key frame.

[0030] In a possible implementation manner, after determining the registration results of the query key frame and each matching frame of the query key frame, the method further includes:

[0031] For any matching frame, a pose error of the query key frame is calculated according to the registration result of the query key frame and any matching frame and a registration transformation matrix, wherein the registration transformation matrix includes at least one of a rotation transformation parameter and a translation transformation parameter for transforming the query key frame to any matching frame through the global point cloud data of the query key frame;

[0032] In response to the pose error being greater than a pose error threshold, it is determined that a registration result between any matching frame and the query key frame meets the requirement.

[0033] In a possible implementation, after determining the target mesh as a mesh with ghosting, the method further includes:

[0034] Merge the grids with ghosts, and determine the ghost area based on the merged grids;

[0035] Repair the point cloud trajectory included in the ghost area.

[0036] On the other hand, a ghosting detection device for a point cloud map is provided, the device comprising:

[0037] A division module, used for dividing the point cloud map to be detected into a plurality of grids, the point cloud map includes a plurality of frames of point cloud trajectories, and each grid includes a local point cloud trajectory in the plurality of frames of point cloud trajectories;

[0038] A determination module, used for determining a query key frame from a local point cloud trajectory included in a target grid;

[0039] The determination module is further used to determine at least one matching frame of the query key frame in the multi-frame point cloud trajectory, the matching frame being a local point cloud trajectory within the reference range where the query key frame is located;

[0040] The determination module is further used to determine the registration results of the query key frame and each matching frame of the query key frame;

[0041] The determination module is further used to determine the target mesh as a mesh with ghosting in response to the existence of a registration result that meets the requirements.

[0042] In a possible implementation, the determination module is used to determine the number of road layers in the target grid; determine a reference number of point cloud trajectories from the local point cloud trajectories included in the target grid as query key frames, and the reference number is determined based on the number of road layers.

[0043] In a possible implementation, the device further includes: a screening module, configured to screen out local point cloud trajectories within a reference range from the multi-frame point cloud trajectories;

[0044] The determination module is configured to, in response to the reference range including a plurality of local point cloud trajectories, determine a local point cloud trajectory that satisfies a time interval among the plurality of local point cloud trajectories as a matching frame of the query key frame.

[0045] In a possible implementation, a determination module is used to determine a subgraph of a query key frame, where the subgraph of the query key frame is obtained based on a point cloud trajectory included in the query key frame; for the query key frame and any matching frame, a subgraph of any matching frame is determined, where the subgraph of any matching frame is obtained based on the point cloud trajectory included in any matching frame; multiple pose transformation matrices are determined based on the subgraph of the query key frame and the subgraph of any matching frame, where any pose transformation matrix includes at least one of a rotation transformation parameter and a translation transformation parameter for transforming the query key frame to any matching frame;

[0046] The device also includes: a calculation module, which is used to calculate the mean of the posture transformation matrices that meet the requirements among the multiple posture transformation matrices, and use the calculated result as the registration result of the query key frame and any matching frame.

[0047] In one possible implementation, the device also includes: a merging module, which is used to merge the point cloud trajectory included in the query key frame and the related point cloud trajectory of the query key frame to obtain a subgraph of the query key frame, and the related point cloud trajectory of the query key frame includes the point cloud trajectory adjacent to the query key frame timestamp in the target grid.

[0048] In one possible implementation, a merging module is used to merge the point cloud trajectory included in any matching frame with the related point cloud trajectory of any matching frame to obtain a sub-image of any matching frame, and the related point cloud trajectory of any matching frame includes the point cloud trajectory adjacent to the timestamp of any matching frame within the grid where any matching frame is located.

[0049] In a possible implementation, the determination module is used to determine multiple groups of local point cloud data corresponding to the sub-image of the query key frame and global point cloud data corresponding to the sub-image of any matching frame;

[0050] The device also includes: a registration module, which is used to align multiple groups of local point cloud data with the global point cloud data respectively to obtain multiple pose transformation matrices. A group of local point cloud data corresponds to one pose transformation matrix. The pose transformation matrix corresponding to any group of local point cloud data includes at least one of the rotation transformation parameters and translation transformation parameters that transform the query key frame to any matching frame through any group of local point cloud data.

[0051] In one possible implementation, a screening module is used to screen out point cloud data that meets the proportion requirements in different regions from the global point cloud data included in the sub-image of the query key frame, and use the point cloud data that meets the proportion requirements screened out in each region as a group of local point cloud data corresponding to the sub-image of the query key frame.

[0052] In a possible implementation, the calculation module is further used to calculate the pose error of the query key frame for any matching frame according to the registration result of the query key frame and any matching frame and the registration transformation matrix, wherein the registration transformation matrix includes at least one of the rotation transformation parameter and the translation transformation parameter for transforming the query key frame to any matching frame through the global point cloud data of the query key frame;

[0053] The determination module is also used to determine that the registration result of any matching frame and the query key frame meets the requirements in response to the posture error being greater than the posture error threshold.

[0054] In a possible implementation, the determination module is further used to merge the grids with ghosts, and determine the ghost area according to the merged grids;

[0055] The device also includes: a repair module, which is used to repair the point cloud trajectory included in the ghost area.

[0056] On the other hand, a computer device is provided, comprising a processor and a memory, wherein at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor so that the computer device implements any of the above-mentioned ghost detection methods for point cloud maps.

[0057] On the other hand, a computer-readable storage medium is also provided, in which at least one computer program is stored, and the at least one computer program is loaded and executed by a processor so that the computer implements any of the above-mentioned ghost detection methods for point cloud maps.

[0058] On the other hand, a computer program product or a computer program is also provided, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs any of the above-mentioned ghost detection methods for point cloud maps.

[0059] The technical solution provided by the embodiments of the present application brings at least the following beneficial effects:

[0060] By dividing the point cloud map into grids, the large-scale point cloud map is converted into point cloud trajectories within multiple small-scale grids, achieving full coverage of ghost detection in the point cloud map with finer detection granularity. The query keyframe and the matching frame are selected, and the query keyframe and the matching frame are aligned to improve the accuracy of detection. Therefore, an accurate high-precision map can be constructed based on the point cloud map after ghost detection, and then the normal driving of the autonomous driving vehicle is guaranteed based on the accurate high-precision map. Since there is no need to extract specific elements, the restrictions of the scene on ghost detection are reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0062] Figure 1 It is a schematic diagram of an implementation environment provided by an embodiment of the present application;

[0063] Figure 2 is a flow chart of a ghosting detection method for a point cloud map provided in an embodiment of the present application;

[0064] Figure 3 It is a schematic diagram of a point cloud map grid division provided in an embodiment of the present application;

[0065] Figure 4 It is a schematic diagram of generating central local point cloud data provided by an embodiment of the present application;

[0066] Figure 5 It is a schematic diagram of generating local point cloud data of an outer contour provided by an embodiment of the present application;

[0067] Figure 6 It is a schematic diagram of generating upper and lower local point cloud data provided by an embodiment of the present application;

[0068] Figure 7 It is a schematic diagram of generating left and right local point cloud data provided by an embodiment of the present application;

[0069] Figure 8 This is a schematic diagram of generating left diagonal local point cloud data provided by an embodiment of the present application;

[0070] Fig. 9 It is a schematic diagram of generating right diagonal local point cloud data provided by an embodiment of the present application;

[0071] Fig.10 It is a schematic diagram of a point cloud map after determining a grid with ghosting provided by an embodiment of the present application;

[0072] Fig.11 It is a schematic diagram of a ghosting detection device for a point cloud map provided in an embodiment of the present application;

[0073] Fig.12 It is a structural diagram of a server provided in an embodiment of the present application;

[0074] Fig.13 It is a schematic diagram of the structure of a ghosting detection device for a point cloud map provided in an embodiment of the present application. DETAILED DESCRIPTION

[0075] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0076] This application embodiment provides a method for detecting ghosting in a point cloud map. Figure 1 , which shows a schematic diagram of an implementation environment of the method provided in an embodiment of the present application. The implementation environment may include: a terminal 11 and a server 12.

[0077] Among them, the terminal 11 is installed with an application or webpage capable of detecting ghosting. When the application or webpage needs to detect ghosting, the method provided in the embodiment of the present application can be applied for display. The server 12 can store the point cloud map that needs to be detected, and the terminal 11 can obtain the point cloud map that needs to be detected from the server 12. Of course, the terminal 11 can also store the obtained point cloud map that needs to be detected.

[0078] Optionally, the terminal 11 may be a smart device such as a mobile phone, a tablet computer, a personal computer, etc. The server 12 may be a single server, or a server cluster consisting of multiple servers, or a cloud computing service center. The terminal 11 establishes a communication connection with the server 12 via a wired or wireless network.

[0079] Optionally, the terminal 11 can be any electronic product that can interact with the user through one or more methods such as keyboard, touchpad, touch screen, remote control, voice interaction or handwriting device, such as PC (Personal Computer), mobile phone, smart phone, PDA (Personal Digital Assistant), wearable device, PPC (Pocket PC), tablet computer, smart car machine, smart TV, smart speaker, etc. The server 12 can be a server, a server cluster composed of multiple servers, or a cloud computing service center. The terminal 11 establishes a communication connection with the server 12 through a wired or wireless network.

[0080] Those skilled in the art should understand that the above-mentioned terminal 11 and server 12 are only examples, and other existing or future terminals or servers that are applicable to the present application should also be included in the protection scope of the present application and are included here by reference.

[0081] The present application embodiment provides a method for detecting ghosting in a point cloud map. The flowchart of the method for detecting ghosting in a point cloud map is as follows: Figure 2 This method can be based on the above Figure 1 The implementation environment shown in the figure is implemented, and the ghost detection method of the point cloud map can be executed by the terminal or the server, or can be implemented by the terminal and the server interacting. The embodiment of the present application takes the server executing the method as an example to illustrate, see Figure 2 The method includes steps 201 to 205.

[0082] Step 201, dividing the point cloud map to be detected into multiple grids.

[0083] In the embodiment of the present application, before ghost detection is performed on the map, a point cloud map is first obtained. The point cloud map includes multiple frames of point cloud trajectories, and each grid includes a local point cloud trajectory in the multiple frames of point cloud trajectories. The embodiment of the present application does not limit the method of obtaining the point cloud map. For example, the area to be mapped is scanned multiple times by radar, and each time a scan is completed, a frame of point cloud trajectory is output, and multiple frames of point cloud trajectories constitute a point cloud map of the area.

[0084] When dividing the point cloud map into grids, including but not limited to determining the number and size of the divided grids according to the range spanned by the point cloud map and the required accuracy of the point cloud map. According to the determined number and size, the point cloud map is divided into a plurality of grids. Each grid includes all local point cloud tracks in the grid area, and the number of local point cloud tracks included in each grid is at least one, and the number of local point cloud tracks included in each grid can be the same or different.

[0085] The point cloud map to be detected is divided into multiple grids, which can reduce the large-scale point cloud map to multiple small-scale grids, and perform ghost detection on the small-scale point cloud trajectories included in the grids respectively. The detection granularity is finer, which can improve the accuracy of ghost detection and achieve full coverage of ghost detection.

[0086] by Figure 3 Taking the schematic diagram of grid division of the point cloud map shown as an example, all the point cloud trajectories in the figure constitute the point cloud map of the area. Within the scope of the point cloud map, the point cloud map is divided into multiple grids according to the length H meters and the width W meters.

[0087] Optionally, after the radar completes a scan and outputs a frame of point cloud trajectory, each frame of point cloud trajectory can be optimized by a mapping optimization algorithm to improve the accuracy of the point cloud pose in each frame of point cloud trajectory. Among them, the mapping optimization algorithm refers to an algorithm that removes the deviation of the point cloud pose and obtains an accurate point cloud pose. The point cloud pose is used to describe the transformation relationship between the world coordinate system and the camera coordinate system, including two transformation parameters, rotation and translation, usually represented by a matrix. The world coordinate system is a coordinate system defined before scanning, which can be used as a reference for scanning and constructing trajectories. The camera coordinate system refers to a coordinate system constructed with the radar as the origin. Since the radar moves during the scanning process, the camera coordinate system also changes with the movement of the radar, so the coordinates of the same frame of point cloud in the world coordinate system and the camera coordinate system are different, and there is a corresponding transformation relationship.

[0088] Step 202: determining a query key frame from the local point cloud trajectory included in the target grid.

[0089] The query keyframe is any frame of the point cloud trajectory in the local point cloud trajectory included in the target grid, and the target grid is any grid among the multiple grids. In the embodiment of the present application, the number of query keyframes in each grid is not necessarily the same, and the number of query keyframes in the grid can be determined based on the local point cloud trajectory included in the grid and the number required for detection. Optionally, the method for determining the query keyframe includes but is not limited to steps 2021-2022.

[0090] Step 2021, determine the number of road layers in the target grid.

[0091] The embodiment of the present application does not limit the method of determining the number of road layers in the target grid, including but not limited to determining the number of road layers L contained in the area according to the radar scanning area corresponding to the point cloud trajectory included in the target grid. For example, if there is a 3-story viaduct and a common two-way multi-lane in the radar scanning area corresponding to the point cloud trajectory included in a target grid, then the number of road layers L in the target grid is determined to be 3.

[0092] Step 2022: determining a reference number of point cloud trajectories from the local point cloud trajectories included in the target grid as query key frames, wherein the reference number is determined based on the number of road layers.

[0093] In a possible implementation, determining a reference number of point cloud trajectories as query key frames from the local point cloud trajectories included in the target grid includes: based on the number of road layers L in the target grid, selecting L*K frame point cloud trajectories from all point cloud trajectories included in the target grid as L*K query key frames of the target grid. K may be an adjustment coefficient, for example, selected according to the complexity of the point cloud trajectories in the grid.

[0094] Optionally, if there is no road in the radar scanning area corresponding to the point cloud trajectory included in the target grid, K frames of point cloud trajectory may be directly selected as K query key frames of the target grid.

[0095] Step 203 : determining at least one matching frame of the query key frame in the multi-frame point cloud trajectory, where the matching frame is a local point cloud trajectory within a reference range where the query key frame is located.

[0096] The present application embodiment does not limit the method for determining the number of matching frames of the query key frame, for example, it can be determined based on the number of point clouds contained in the query key frame, or it can be determined by the size of the target grid. Optionally, the method for determining at least one matching frame includes but is not limited to steps 2031-2032.

[0097] Step 2031 , selecting local point cloud trajectories within a reference range from the multi-frame point cloud trajectories.

[0098] The embodiment of the present application does not limit the reference range, and the reference range can be larger than the area of ​​the target grid or smaller than the area of ​​the target grid. Exemplarily, the first value is set according to a standard that can ensure the establishment of an association between multiple frame trajectories, and the target query key frame is taken as the center of the circle and the first value is taken as the radius to determine the reference range for screening the local point cloud trajectory.

[0099] When filtering out local point cloud trajectories within a reference range from multi-frame point cloud trajectories, the local point cloud trajectories near the query key frame can be filtered in the geometric space of the radar scanning area according to the determined reference range. The filtered local point cloud trajectories can be local point cloud trajectories in the target grid except the query key frame, or local point cloud trajectories in the adjacent grids of the target grid.

[0100] Exemplarily, the first value is set to 50 meters, the first value is used as the radius, the query key frame is used as the center of the circle, and the reference range for filtering the local point cloud trajectory near the query key frame is determined. This radius can cover common two-way multi-lane scenes and can establish associations between multi-frame trajectories, so that the coverage of ghost detection is more complete.

[0101] Step 2032 , in response to the reference range including multiple local point cloud trajectories, determining the local point cloud trajectories that meet the time interval among the multiple local point cloud trajectories as matching frames of the query key frame.

[0102] Optionally, before determining the local point cloud trajectory that satisfies the time interval, a time interval threshold T for selecting the matching frame is set. If the time interval between any two local point cloud trajectories among all the screened local point cloud trajectories is less than T, the point clouds contained in the two local point cloud trajectories are considered to be similar, so that when selecting the matching frame of the query key frame, any one of the two local point cloud trajectories is selected as the matching frame of the query key frame. If the time interval between any two local point cloud trajectories among all the screened local point cloud trajectories is greater than T, the point clouds contained in all the local point cloud trajectories are considered to be dissimilar, and all the screened local point cloud trajectories can be used as matching frames of the query key frame.

[0103] Regardless of whether the time interval between any two local point cloud trajectories is less than T, at least one local point cloud trajectory satisfying the time interval is selected from the selected local point cloud trajectories near the query key frame as a matching frame of the query key frame.

[0104] Exemplarily, three local point cloud trajectories, namely, local point cloud trajectory o, local point cloud trajectory s, and local point cloud trajectory p, are sequentially screened out within the reference range of the query key frame q, wherein, if the time interval between the screened out local point cloud trajectory o and the local point cloud trajectory s is less than T, and the time interval between the screened out local point cloud trajectory o and the local point cloud trajectory p is greater than T, the local point cloud trajectory s is removed, and the local point cloud trajectory o and the local point cloud trajectory p are selected as the two matching frames of the query key frame q.

[0105] Step 204: Determine the registration result of the query key frame and each matching frame of the query key frame.

[0106] After determining at least one matching frame of the query key frame, it is necessary to determine the registration result of the query key frame and each matching frame to reflect the matching degree between the query key frame and each matching frame. The registration result of the query key frame and any matching frame refers to a matrix that can reflect the transformation relationship between the query key frame and any matching frame, including at least one of the parameters of the translation transformation and the parameters of the rotation change. Optionally, the method for determining the registration result includes but is not limited to steps 2041-2044.

[0107] Step 2041 , determining a subgraph of the query key frame, where the subgraph of the query key frame is obtained based on the point cloud trajectory included in the query key frame.

[0108] In a possible implementation, the point cloud track included in the query key frame and the related point cloud track of the query key frame are merged to obtain a subgraph of the query key frame, and the related point cloud track of the query key frame includes the point cloud track adjacent to the query key frame timestamp in the target grid.

[0109] Among them, the timestamp can refer to the total number of seconds from 00:00:00 Greenwich time on January 1, 1970 (08:00:00 Beijing time on January 1, 1970) to the present. Data is generated by using digital signature technology. The object of the signature includes original file information, signature parameters, signature time and other information.

[0110] The query keyframe and the local point cloud trajectories adjacent to the query keyframe timestamp in the target grid are merged to obtain the query keyframe subgraph. Compared with the query keyframe of a single frame, the generated query keyframe subgraph has dense point clouds, complete spatial structure, wider range, and more common view areas, which is conducive to constructing line and surface constraints.

[0111] For example, the query keyframe q and the local point cloud trajectories adjacent to its timestamp in the target grid are merged to generate a subgraph S of the query keyframe q .

[0112] Step 2042: for any matching frame of the query key frame, determine a subgraph of any matching frame, where the subgraph of any matching frame is obtained based on a point cloud trajectory included in any matching frame.

[0113] In a possible implementation, consistent with the above-mentioned method for obtaining a sub-graph of a query key frame, the point cloud trajectory included in any matching frame is merged with the relevant point cloud trajectory of any matching frame to obtain a sub-graph of any matching frame, and the relevant point cloud trajectory of any matching frame includes the point cloud trajectory adjacent to the timestamp of any matching frame within the grid where any matching frame is located.

[0114] For example, the matching frame p and the local point cloud tracks adjacent to its timestamp in the grid where the matching frame is located are merged to generate a subgraph S of the matching framep .

[0115] Step 2043: determine multiple pose transformation matrices based on the sub-graph of the query key frame and the sub-graph of any matching frame, wherein any pose transformation matrix includes at least one of a rotation transformation parameter and a translation transformation parameter for transforming the query key frame to any matching frame.

[0116] In one possible implementation, the pose transformation matrix is ​​determined based on multiple sets of local point cloud data corresponding to the sub-image of the query key frame and global point cloud data corresponding to the sub-image of any matching frame. The present application does not limit the method for determining multiple sets of local point cloud data corresponding to the sub-image of the query key frame and global point cloud data corresponding to the sub-image of any matching frame.

[0117] Among them, the multiple groups of local point cloud data corresponding to the sub-graph of the query key frame refer to the point cloud data in any one or more areas that meet the proportion requirements in the global point cloud data included in the sub-graph of the query key frame, and the point cloud data includes but is not limited to the number of point clouds and the position and posture of each point in these point clouds. Exemplarily, the proportion requirement can be a pre-set proportion threshold, which is determined according to the required map accuracy. Correspondingly, the global point cloud data corresponding to the sub-graph of any matching frame includes but is not limited to the number of global point clouds of the matching frame and the position and posture of each point in these point clouds.

[0118] In one possible implementation, when determining multiple groups of local point cloud data corresponding to a sub-graph of a query key frame, point cloud data that meet the proportion requirements can be screened out according to different regions in the global point cloud data included in the sub-graph of the query key frame, and the point cloud data that meet the proportion requirements screened out in each region can be used as a group of local point cloud data corresponding to the sub-graph of the query key frame.

[0119] After generating a subgraph of the query keyframe and a subgraph of the matching frame for the query keyframe and any matching frame respectively, at least two groups of local point cloud data are generated for the subgraph of the query keyframe, and each group of local point cloud data corresponds to a different area of ​​the subgraph of the query keyframe. Each group of local point cloud data generated corresponds to at least one area of ​​the subgraph of the query keyframe, and different areas correspond to different generation methods, including but not limited to the center, outer contour, top and bottom, left and right, left diagonal, and right diagonal generation methods. When selecting the generation method of local point cloud data, it is necessary to ensure that the point cloud distribution in the generated local point cloud data is relatively uniform to prevent the situation where the point cloud of a certain dimension is missing, resulting in excessive deviation in the pose solution. Using local point cloud data for registration can avoid reducing the weight of other areas due to the rich structure of a certain area in the point cloud trajectory and dominating in the registration, resulting in incorrect registration results.

[0120] by Figure 4 Take the schematic diagram of generating central local point cloud data as an example. Figure 4The point cloud selected by the dotted line in the middle is a set of central local point clouds generated based on the sub-graph of the query key frame. The data contained in these point clouds is the data of the central local point cloud group. Figure 5-Figure 9 , respectively showing the results of generating local point cloud data for the sub-image of the query key frame using the generation methods of outer contour, up and down, left and right, left diagonal, and right diagonal. Figure 5-Figure 9 The point cloud data selected by the dotted line is the outer contour, top and bottom, left and right, left diagonal, and right diagonal local point cloud data generated according to the query key sub-frame.

[0121] Exemplarily, after generating at least two groups of local point cloud data of the sub-graph of the query key frame according to different regions, i.e., different generation methods, multiple groups of local point cloud data that meet the ratio requirement are screened out. Optionally, before screening out multiple groups of local point cloud data that meet the ratio requirement, a threshold value of the ratio of the overlapping area between any group of local point cloud data and the sub-graph of the matching frame is set as the ratio requirement according to the required map accuracy.

[0122] In a possible implementation, before screening multiple sets of local point cloud data that meet the ratio requirement, it is necessary to calculate the overlap ratio between each set of local point cloud data of the sub-image of the query key frame and the sub-image of any matching frame of the query key frame. The method of calculating the overlap ratio includes but is not limited to the following steps 1 to 3.

[0123] Step 1, determine each point in any set of local point cloud data corresponding to the sub-image of the query key frame, and calculate the projection point of each point in the target local point cloud data in the above matching frame sub-image according to the registration transformation matrix corresponding to the query key frame and any matching frame. Among them, the registration transformation matrix refers to the transformation relationship between the query key frame and each matching frame calculated by the mapping optimization algorithm after determining the query key frame and each matching frame, including at least one of the parameters of the rotation transformation and the parameters of the translation transformation. The relationship between the query key frame and each matching frame is not necessarily the same, so the query key frame and any matching frame have a corresponding registration transformation matrix.

[0124] Step 2: Determine the nearest neighbor of each projection point in the global point cloud data of the matching frame sub-image. Calculate the spatial distance between each projection point and its nearest neighbor, and compare each spatial distance with the set spatial distance threshold, and obtain all spatial distances less than the spatial distance threshold by screening. If any calculated spatial distance is greater than the set spatial distance threshold, it is considered that there is no corresponding relationship between the nearest neighbor corresponding to the spatial distance and the projection point corresponding to the nearest neighbor, and the spatial distance is discarded.

[0125] Step 3, determine the number of spatial distances that are screened out and are less than the spatial distance threshold, and determine the ratio of the number of spatial distances screened out to the number of points in the group of local point cloud data as the ratio of the overlapping area of ​​the group of local point cloud data and the matching frame sub-image.

[0126] For example, after the query key frame q and the matching frame p determine the matching relationship, the mapping optimization algorithm is used to determine the registration transformation matrix T from the query key frame q to the matching frame p. pq . According to the subgraph S of the query key frame q Generate 4 local point cloud data in the form of center, top and bottom, left and right, and outer contour. Determine the subgraph S of the query key frame q The central local point cloud data S q1 There are 4 points in it, namely A1, A2, A3, and A4. According to A'=TpqA, the subgraph S of each point in the matching frame is calculated. p The projection points A1', A2', A3', A4' in.

[0127] Exemplarily, after completing the above matching frame subgraph S p After calculating all the projection points in the matching frame subgraph S p The nearest neighbor of A1' is B1, the nearest neighbor of A2' is B2, the nearest neighbor of A3' is B3, and the nearest neighbor of A4' is B4. The spatial distances between each projection point and the corresponding nearest neighbor are calculated as d1, d2, d3, and d4 respectively. Compare each calculated spatial distance with the set spatial distance threshold, and discard d4 that is greater than the spatial distance threshold. Determine that the number of spatial distances screened out is 3, and the number of point clouds in this group of local point cloud data is 4, and obtain the central local point cloud data S q1 and matching frame subgraph S p The overlap ratio is

[0128] Optionally, after determining the overlapping area ratio of all local point cloud data of the sub-image of the query key frame and the sub-image of the matching frame, local point cloud data that meet the ratio requirement are screened out to determine multiple groups of local point cloud data corresponding to the sub-image of the query key frame.

[0129] In a possible implementation, if the ratio of the overlapped area between any group of local point cloud data and the matching frame sub-image calculated is greater than the ratio requirement, the result of the registration calculation between the group of local point cloud data and the matching frame sub-image is considered to be credible; if the ratio of the overlapped area between the calculated local point cloud data and the matching frame sub-image is less than the ratio threshold, it is considered that the result of the registration calculation between the local point cloud data and the matching frame sub-image is unreliable due to the small overlapped area. Therefore, each group of local point cloud data with a ratio of the overlapped area between the group of local point cloud data and the matching frame sub-image greater than the ratio requirement is used as a group of local point cloud data corresponding to the sub-image of the query key frame, and all local point cloud data with a ratio of the overlapped area greater than the ratio requirement are determined as multiple groups of local point cloud data corresponding to the sub-image of the query key frame.

[0130] For example, the above step of determining the overlap ratio between the local point cloud data and the matching frame sub-image needs to be repeated 4 times, that is, querying the key frame sub-image S q The four generated local point cloud data need to calculate the overlapping area ratio between the local point cloud data and the matching frame sub-image. After calculation, the central local point cloud data S q1 , upper and lower local point cloud data S q2 , S of the left and right local point cloud data q3 These three sets of local point cloud data and matching frame sub-image S p The overlapping area ratio is greater than the ratio requirement, and these three sets of local point cloud data are determined as the sub-graph S of the query key frame. q The corresponding three sets of local point cloud data. The local point cloud data S of the outer contour q4 and matching frame subgraph S p The overlapping area of ​​​​is less than the overlapping threshold, so the local point cloud data S of the outer contour is discarded q4 .

[0131] For the multiple groups of local point cloud data corresponding to the sub-image of the query key frame determined above and the global point cloud data of the sub-image of any matching frame, the multiple groups of local point cloud data are respectively aligned with the global point cloud data to obtain multiple pose transformation matrices. One group of local point cloud data corresponds to one pose transformation matrix. The pose transformation matrix corresponding to any group of local point cloud data includes at least one of the rotation transformation parameters and translation transformation parameters for transforming the query key frame to any matching frame through any group of local point cloud data.

[0132] Exemplarily, for the multiple groups of local point cloud data corresponding to the sub-image of the query key frame and the global point cloud data of the sub-image of any matching frame, a registration algorithm is used to perform point cloud registration. In a possible implementation, the multiple groups of local point cloud data corresponding to the sub-image of the query key frame and the global point cloud data of the sub-image of any matching frame are used for registration instead of the local point cloud data of the sub-image of any matching frame, so as to avoid the phenomenon that the registration result is unreliable due to the small overlap area between the multiple groups of local point cloud data corresponding to the sub-image of the query key frame and the local point cloud data of the sub-image of any matching frame. The embodiments of the present application do not limit the registration method. For example, the registration method may be: traditional methods such as point-to-plane ICP (Iterative Closest Point), point-to-point ICP, GICP (Generalized Iterative Closest Point), and deep learning methods such as D3Feat (Dense Detection and Description Feat), FCGF (Fully Convolutional Geometric Features), and PREDATOR (registration method for low-overlap three-dimensional point clouds). Each group of local point cloud data corresponding to the sub-image of the query key frame and the global point cloud data of any matching frame sub-image are subjected to registration calculation by any of the above-mentioned registration algorithms to obtain a corresponding pose transformation matrix. The pose transformation matrix corresponding to any group of local point cloud data includes at least one of the rotation transformation parameters and translation transformation parameters for transforming the query key frame to any matching frame through any group of local point cloud data.

[0133] Step 2044, calculating the mean of the pose transformation matrices that meet the requirements among the multiple pose transformation matrices, and using the calculated result as the registration result between the query key frame and any matching frame.

[0134] According to each group of local point cloud data corresponding to the sub-image of the query key frame and the global point cloud data of any matching frame sub-image, a corresponding pose transformation matrix is ​​obtained through registration calculation using any of the above-mentioned registration algorithms, and the first average transformation matrix of all pose transformation matrices of the query key frame and the matching frame is calculated, and the difference between each pose transformation matrix and the first average transformation matrix is ​​calculated respectively. The difference between each pose transformation matrix and the first average transformation matrix is ​​compared with the transformation matrix threshold, and the pose transformation matrix whose difference with the first average transformation matrix is ​​less than the transformation matrix threshold is screened out, and the mean is recalculated based on the screened pose transformation matrix to obtain the registration result of the query key frame and any matching frame.

[0135] Exemplarily, a query key frame and any matching frame of the query key frame can obtain a registration result through calculation, and the number of matching frames of the query key frame is the number of registration results finally obtained for the query key frame.

[0136] Among them, for special situations that arise, for example, when only two sets of local point cloud data corresponding to the sub-image of the query key frame have overlapping area ratios with the global point cloud data of the sub-image of the matching frame that meet the ratio requirements, and the differences between the two obtained pose transformation matrices and the first average transformation matrix exceed the transformation matrix threshold, at this time, the pose transformation matrix corresponding to the local point cloud data with the largest overlapping area ratio is selected as the alignment result.

[0137] In one possible implementation, after determining the registration results of the query key frame and each matching frame of the query key frame, for any matching frame, the pose error of the query key frame is calculated based on the registration results of the query key frame and any matching frame and the registration transformation matrix, and the registration transformation matrix includes at least one of the rotation transformation parameters and translation transformation parameters that transform the query key frame to any matching frame through the global point cloud data of the query key frame.

[0138] In a possible implementation, the inverse matrix of the registration transformation matrix is ​​multiplied by any registration result of the query key frame to obtain the pose error between the query key frame and the matching frame corresponding to the registration result.

[0139] For example, the registration result T of the matching frame p based on the query key frame q is f , according to T diff =(T pq ) -1 T f Calculate the pose error T between the query key frame q and the matching frame p diff , including rotation error and translation error.

[0140] After obtaining the pose errors between the query key frame and each matching frame of the query key frame, in response to the pose error being greater than a pose error threshold, it is determined that the registration result of any matching frame with the query key frame meets the requirement.

[0141] Compare each pose error of the query keyframe with the pose error threshold, and filter out the query keyframes whose maximum pose error exceeds the pose error threshold. The pose error threshold refers to a numerical value set in advance according to the required map accuracy, including a rotation threshold and a translation threshold, for example, the rotation threshold is an angle threshold. For the rotation error within the obtained pose error, first convert the rotation matrix into Euler angles. If the error in one of the three directions of the Euler angles is greater than the rotation threshold, it is considered that the pose error exceeds the pose error threshold; for the translation error within the obtained pose error, if the error in one of the three directions of xyz is greater than the translation threshold, it is considered that the pose error exceeds the pose error threshold. In a possible implementation, if at least one of the rotation error and translation error within the obtained pose error is greater than its corresponding rotation threshold or translation threshold, it is considered that the registration result calculated for the pose error meets the requirements.

[0142] Step 205: In response to the existence of a registration result that meets the requirements, the target mesh is determined to be a mesh with ghost images.

[0143] In a possible implementation, after the grids with ghosting are determined by using the method provided in the embodiment of the present application, the grids with ghosting may be merged, and the area of ​​the ghosting may be determined based on the merged grids.

[0144] For example, after determining that the registration result of any matching frame and the query keyframe meets the requirements, the target grid, that is, the grid where the query keyframe is located, is determined as a grid with ghosts. After determining all grids with ghosts in the point cloud map, adjacent ghost grids in the point cloud map are merged to obtain the area with ghosts in the point cloud map.

[0145] by Fig.10 Taking the schematic diagram of the point cloud map after determining the grid with ghost as shown, the grid with the cross symbol is the grid determined to have ghost. The adjacent grids with ghost in the point cloud map are merged to determine multiple areas with ghost, corresponding to the areas selected by the thick solid line in the figure.

[0146] Optionally, the method provided in the embodiment of the present application further includes: repairing the point cloud trajectory included in the ghosting area. For example, after determining the area where the ghosting exists in the point cloud map, the corresponding radar scanning area is determined according to the ghosting area. In this area, the radar re-scans multiple times to repair the point cloud trajectory corresponding to the area, remove the ghosting in the point cloud trajectory, and ensure the quality of the point cloud map.

[0147] In summary, the method provided in the embodiment of the present application, by dividing the point cloud map into grids, converts a large-scale point cloud map into point cloud trajectories within multiple small-scale grids with finer granularity, thereby achieving full coverage of ghost detection in the point cloud map. The query key frame and the matching frame are selected, the query key frame sub-graph and the matching frame sub-graph are synthesized, and at least one local point cloud data of the query key frame sub-graph is subjected to multiple rounds of registration calculations with the matching frame sub-graph to complete the ghost detection of the point cloud map. Multiple registrations improve the accuracy of the detection, so that an accurate high-precision map can be constructed based on the point cloud map after ghost detection, and then the normal driving of the autonomous driving vehicle is guaranteed based on the accurate high-precision map. Ghost detection is performed through the registration algorithm, and since there is no need to extract specific elements, the restrictions of the scene on ghost detection are reduced.

[0148] See also Fig.11 , an embodiment of the present application provides a ghosting detection device for a point cloud map, the device comprising:

[0149] A division module 1101 is used to divide the point cloud map to be detected into a plurality of grids, the point cloud map includes a plurality of frames of point cloud trajectories, and each grid includes a local point cloud trajectory in the plurality of frames of point cloud trajectories;

[0150] A determination module 1102, configured to determine a query key frame from a local point cloud trajectory included in a target grid;

[0151] The determination module 1102 is further configured to determine at least one matching frame of the query key frame in the multi-frame point cloud trajectory, where the matching frame is a local point cloud trajectory within the reference range where the query key frame is located;

[0152] The determination module 1102 is further used to determine the registration result of the query key frame and each matching frame of the query key frame;

[0153] The determination module 1102 is further configured to determine the target mesh as a mesh with ghosting in response to the existence of a registration result that meets the requirements.

[0154] In a possible implementation, the determination module 1102 is used to determine the number of road layers in the target grid; determine a reference number of point cloud trajectories from the local point cloud trajectories included in the target grid as query key frames, and the reference number is determined based on the number of road layers.

[0155] In a possible implementation, the device further includes: a screening module, configured to screen out local point cloud trajectories within a reference range from the multi-frame point cloud trajectories;

[0156] The determination module 1102 is configured to determine, in response to the reference range including multiple local point cloud trajectories, a local point cloud trajectory that satisfies a time interval among the multiple local point cloud trajectories as a matching frame of the query key frame.

[0157] In a possible implementation, the determination module 1102 is used to determine a subgraph of the query key frame, where the subgraph of the query key frame is obtained based on the point cloud trajectory included in the query key frame; for the query key frame and any matching frame, determine a subgraph of any matching frame, where the subgraph of any matching frame is obtained based on the point cloud trajectory included in any matching frame; determine multiple pose transformation matrices based on the subgraph of the query key frame and the subgraph of any matching frame, where any pose transformation matrix includes at least one of a rotation transformation parameter and a translation transformation parameter for transforming the query key frame to any matching frame;

[0158] The device also includes: a calculation module, which is used to calculate the mean of the posture transformation matrices that meet the requirements among the multiple posture transformation matrices, and use the calculated result as the registration result of the query key frame and any matching frame.

[0159] In one possible implementation, a merging module is used to merge the point cloud trajectory included in the query key frame and the related point cloud trajectory of the query key frame to obtain a subgraph of the query key frame, and the related point cloud trajectory of the query key frame includes the point cloud trajectory adjacent to the query key frame timestamp in the target grid.

[0160] In one possible implementation, a merging module is used to merge the point cloud trajectory included in any matching frame with the related point cloud trajectory of any matching frame to obtain a sub-image of any matching frame, and the related point cloud trajectory of any matching frame includes the point cloud trajectory adjacent to the timestamp of any matching frame within the grid where any matching frame is located.

[0161] In a possible implementation, the determination module 1102 is used to determine multiple groups of local point cloud data corresponding to the sub-image of the query key frame and global point cloud data corresponding to the sub-image of any matching frame;

[0162] The device also includes: a registration module, which is used to align multiple groups of local point cloud data with the global point cloud data respectively to obtain multiple pose transformation matrices. A group of local point cloud data corresponds to one pose transformation matrix. The pose transformation matrix corresponding to any group of local point cloud data includes at least one of the rotation transformation parameters and translation transformation parameters that transform the query key frame to any matching frame through any group of local point cloud data.

[0163] In one possible implementation, a screening module is used to screen out point cloud data that meets the proportion requirements in different regions from the global point cloud data included in the sub-image of the query key frame, and use the point cloud data that meets the proportion requirements screened out in each region as a group of local point cloud data corresponding to the sub-image of the query key frame.

[0164] In a possible implementation, the calculation module is further used to calculate the pose error of the query key frame for any matching frame according to the registration result of the query key frame and any matching frame and the registration transformation matrix, wherein the registration transformation matrix includes at least one of the rotation transformation parameter and the translation transformation parameter for transforming the query key frame to any matching frame through the global point cloud data of the query key frame;

[0165] The determination module 1102 is further configured to determine, in response to a posture error being greater than a posture error threshold, whether the registration result between any matching frame and the query key frame meets the requirement.

[0166] In a possible implementation, the determination module 1102 is further configured to merge the grids with ghost images, and determine the ghost image area according to the merged grids;

[0167] The device also includes: a repair module, which is used to repair the point cloud trajectory included in the ghost area.

[0168] The device provided in the embodiment of the present application divides the point cloud map into grids, converts a large-scale point cloud map into point cloud trajectories within multiple small-scale grids, and has a finer granularity, thereby achieving full coverage of ghost detection in the point cloud map. The query key frame and the matching frame are selected, the query key frame sub-graph and the matching frame sub-graph are synthesized, and at least one local point cloud data of the query key frame sub-graph is subjected to multiple rounds of registration calculations with the matching frame sub-graph to complete the ghost detection of the point cloud map. Multiple registrations improve the accuracy of the detection, so that an accurate high-precision map can be constructed based on the point cloud map after ghost detection, and then the normal driving of the autonomous driving vehicle is guaranteed based on the accurate high-precision map. Ghost detection is performed through the registration algorithm, and since there is no need to extract specific elements, the restrictions of the scene on ghost detection are reduced.

[0169] It should be noted that the device provided in the above embodiment only uses the division of the above functional modules as an example to implement its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and their specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0170] It should be noted that the terms "first", "second", etc. (if any) in the specification and claims of the present application are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0171] Fig.12 It is a structural diagram of a server provided in an embodiment of the present application. The server may have relatively large differences due to different configurations or performances, and may include one or more processors 1201 and one or more memories 1202. The processor 1201 is, for example, a CPU (Central Processing Units). Among them, at least one computer program is stored in the one or more memories 1202, and the at least one computer program is loaded and executed by the one or more processors 1201, so that the server implements the ghost detection method of the point cloud map provided in the above-mentioned various method embodiments. Of course, the server may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output. The server may also include other components for realizing device functions, which are not repeated here.

[0172] Fig.13 1 is a schematic diagram of a ghost detection device for a point cloud map provided in an embodiment of the present application. The device may be a terminal, such as a smart phone, a tablet computer, a laptop computer, or a desktop computer. The terminal may also be referred to as a user device, a portable terminal, a laptop terminal, a desktop terminal, or other names.

[0173] Typically, the terminal includes: a processor 1301 and a memory 1302 .

[0174] The processor 1301 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 1301 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 1301 may also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 1301 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1301 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0175] The memory 1302 may include one or more computer-readable storage media, which may be non-transitory. The memory 1302 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 1302 is used to store at least one instruction, which is used to be executed by the processor 1301 so that the terminal implements the method for ghosting detection provided in the method embodiment of the present application.

[0176] In some embodiments, the terminal may also optionally include: a peripheral device interface 1303 and at least one peripheral device. The processor 1301, the memory 1302 and the peripheral device interface 1303 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 1303 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 1304, a display screen 1305, a camera assembly 1306, an audio circuit 1307, a positioning assembly 1308 and a power supply 1309.

[0177] The peripheral device interface 1303 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 1301 and the memory 1302. In some embodiments, the processor 1301, the memory 1302, and the peripheral device interface 1303 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 1301, the memory 1302, and the peripheral device interface 1303 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0178] The radio frequency circuit 1304 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 1304 communicates with the communication network and other communication devices through electromagnetic signals. The radio frequency circuit 1304 converts the electrical signal into an electromagnetic signal for transmission, or converts the received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 1304 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The radio frequency circuit 1304 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: a metropolitan area network, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 1304 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0179] The display screen 1305 is used to display a UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 1305 is a touch display screen, the display screen 1305 also has the ability to collect touch signals on the surface or above the surface of the display screen 1305. The touch signal can be input to the processor 1301 as a control signal for processing. At this time, the display screen 1305 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 1305 can be one, which is set on the front panel of the terminal; in other embodiments, the display screen 1305 can be at least two, which are respectively set on different surfaces of the terminal or are folded; in other embodiments, the display screen 1305 can be a flexible display screen, which is set on the curved surface or folded surface of the terminal. Even, the display screen 1305 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 1305 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), etc.

[0180] The camera assembly 1306 is used to capture images or videos. Optionally, the camera assembly 1306 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize the panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 1306 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0181] The audio circuit 1307 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 1301 for processing, or input them into the RF circuit 1304 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the terminal. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 1301 or the RF circuit 1304 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 1307 may also include a headphone jack.

[0182] The positioning component 1308 is used to locate the current geographical location of the terminal to implement navigation or LBS (Location Based Service). The positioning component 1308 can be a positioning component based on the US GPS (Global Positioning System), China's Beidou system, Russia's Grenas system or the European Union's Galileo system.

[0183] The power supply 1309 is used to power various components in the terminal. The power supply 1309 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 1309 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.

[0184] In some embodiments, the terminal further includes one or more sensors 1310 , including but not limited to: an acceleration sensor 1311 , a gyroscope sensor 1312 , a pressure sensor 1313 , a fingerprint sensor 1314 , an optical sensor 1315 , and a proximity sensor 1316 .

[0185] The acceleration sensor 1311 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the terminal. For example, the acceleration sensor 1311 can be used to detect the components of gravity acceleration on the three coordinate axes. The processor 1301 can control the display screen 1305 to display the user interface in a horizontal view or a vertical view according to the gravity acceleration signal collected by the acceleration sensor 1311. The acceleration sensor 1311 can also be used for collecting game or user motion data.

[0186] The gyro sensor 1312 can detect the body direction and rotation angle of the terminal, and the gyro sensor 1312 can cooperate with the acceleration sensor 1311 to collect the user's 3D actions on the terminal. The processor 1301 can implement the following functions based on the data collected by the gyro sensor 1312: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0187] The pressure sensor 1313 can be set in the side frame of the terminal and / or the lower layer of the display screen 1305. When the pressure sensor 1313 is set in the side frame of the terminal, the user's holding signal of the terminal can be detected, and the processor 1301 performs left and right hand recognition or shortcut operation according to the holding signal collected by the pressure sensor 1313. When the pressure sensor 1313 is set in the lower layer of the display screen 1305, the processor 1301 controls the operability controls on the UI interface according to the user's pressure operation on the display screen 1305. The operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0188] The fingerprint sensor 1314 is used to collect the user's fingerprint, and the processor 1301 identifies the user's identity based on the fingerprint collected by the fingerprint sensor 1314, or the fingerprint sensor 1314 identifies the user's identity based on the collected fingerprint. When the user's identity is identified as a trusted identity, the processor 1301 authorizes the user to perform relevant sensitive operations, which include unlocking the screen, viewing encrypted information, downloading software, paying, and changing settings. The fingerprint sensor 1314 can be set on the front, back or side of the terminal. When a physical button or a manufacturer logo is set on the terminal, the fingerprint sensor 1314 can be integrated with the physical button or the manufacturer logo.

[0189] The optical sensor 1315 is used to collect the ambient light intensity. In one embodiment, the processor 1301 can control the display brightness of the display screen 1305 according to the ambient light intensity collected by the optical sensor 1315. Specifically, when the ambient light intensity is high, the display brightness of the display screen 1305 is increased; when the ambient light intensity is low, the display brightness of the display screen 1305 is reduced. In another embodiment, the processor 1301 can also dynamically adjust the shooting parameters of the camera assembly 1306 according to the ambient light intensity collected by the optical sensor 1315.

[0190] The proximity sensor 1316, also known as a distance sensor, is usually arranged on the front panel of the terminal. The proximity sensor 1316 is used to collect the distance between the user and the front of the terminal. In one embodiment, when the proximity sensor 1316 detects that the distance between the user and the front of the terminal is gradually decreasing, the processor 1301 controls the display screen 1305 to switch from the screen-on state to the screen-off state; when the proximity sensor 1316 detects that the distance between the user and the front of the terminal is gradually increasing, the processor 1301 controls the display screen 1305 to switch from the screen-off state to the screen-on state.

[0191] Those skilled in the art will understand that Fig.13 The structure shown in the figure does not constitute a limitation on the terminal, and may include more or less components than those shown in the figure, or combine certain components, or adopt a different component arrangement.

[0192] In an exemplary embodiment, a computer device is also provided, the computer device comprising a processor and a memory, wherein at least one computer program is stored in the memory. The at least one computer program is loaded and executed by one or more processors, so that the computer device implements any of the above ghost detection methods.

[0193] In an exemplary embodiment, a computer-readable storage medium is further provided, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor of a computer device to enable the computer to implement any of the above-mentioned ghost detection methods.

[0194] In one possible implementation, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0195] In an exemplary embodiment, a computer program product or a computer program is also provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs any of the above-mentioned ghost detection methods.

[0196] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions. For example, the grids, query key frames and matching frames involved in this application are all obtained with full authorization.

[0197] It should be understood that the "plurality" mentioned in this article refers to two or more. "And / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0198] The above description is only an exemplary embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for detecting ghosting in a point cloud map, characterized in that: The method comprises: dividing a point cloud map to be detected into a plurality of grids, wherein the point cloud map comprises a plurality of frame point cloud trajectories, and each grid comprises a local point cloud trajectory in the plurality of frame point cloud trajectories; determining a query key frame from the local point cloud trajectories included in a target grid; determining at least one matching frame of the query key frame in the plurality of frame point cloud trajectories, wherein the matching frame is a local point cloud trajectory within a reference range where the query key frame is located; determining a registration result of the query key frame and each matching frame of the query key frame; and in response to the existence of a registration result that satisfies the requirements, determining the target grid as a grid with ghosting; Determining the query key frame from the local point cloud trajectory included in the target grid includes: determining the number of road layers in the target grid; determining a reference number of point cloud trajectories from the local point cloud trajectory included in the target grid as the query key frame, wherein the reference number is determined based on the number of road layers; The step of determining at least one matching frame of the query key frame in the multi-frame point cloud trajectory comprises: selecting a local point cloud trajectory within the reference range from the multi-frame point cloud trajectory; in response to the reference range including a plurality of local point cloud trajectories, determining a local point cloud trajectory that satisfies a time interval among the plurality of local point cloud trajectories as a matching frame of the query key frame; The determination of the registration result of the query key frame and each matching frame of the query key frame includes: determining a subgraph of the query key frame, the subgraph of the query key frame is obtained based on the point cloud trajectory included in the query key frame; for any matching frame of the query key frame, determining a subgraph of any matching frame, the subgraph of any matching frame is obtained based on the point cloud trajectory included in the any matching frame; determining multiple pose transformation matrices based on the subgraph of the query key frame and the subgraph of any matching frame, any pose transformation matrix includes at least one of the rotation transformation parameters and the translation transformation parameters for transforming the query key frame to any matching frame; performing mean calculation on the pose transformation matrices that meet the requirements among the multiple pose transformation matrices, and using the calculated result as the registration result of the query key frame and any matching frame; After determining the registration results of the query key frame and each matching frame of the query key frame, the method further includes: for any matching frame, calculating the pose error of the query key frame according to the registration result of the query key frame and the any matching frame and a registration transformation matrix, the registration transformation matrix including at least one of a rotation transformation parameter and a translation transformation parameter for transforming the query key frame to the any matching frame through the global point cloud data of the query key frame; in response to the pose error being greater than a pose error threshold, determining that the registration result of the any matching frame with the query key frame meets the requirement.

2. The method according to claim 1, characterized in that The determining of the subgraph of the query key frame includes: merging the point cloud track included in the query key frame and the related point cloud track of the query key frame to obtain the subgraph of the query key frame, wherein the related point cloud track of the query key frame includes the point cloud track adjacent to the query key frame timestamp in the target grid.

3. The method according to claim 1, characterized in that The determining of the subgraph of any matching frame includes: merging the point cloud trajectory included in any matching frame with the related point cloud trajectory of any matching frame to obtain the subgraph of any matching frame, wherein the related point cloud trajectory of any matching frame includes the point cloud trajectory adjacent to the timestamp of any matching frame in the grid where the any matching frame is located.

4. The method according to claim 1, characterized in that: The method of determining multiple pose transformation matrices based on the sub-image of the query key frame and the sub-image of any matching frame includes: determining multiple groups of local point cloud data corresponding to the sub-image of the query key frame and global point cloud data corresponding to the sub-image of any matching frame; aligning the multiple groups of local point cloud data with the global point cloud data respectively to obtain multiple pose transformation matrices, one group of local point cloud data corresponds to one pose transformation matrix, and the pose transformation matrix corresponding to any group of local point cloud data includes at least one of the rotation transformation parameters and translation transformation parameters for transforming the query key frame to any matching frame through any group of local point cloud data.

5. The method according to claim 4, characterized in that The method of determining multiple groups of local point cloud data corresponding to the sub-graph of the query key frame includes: among the global point cloud data included in the sub-graph of the query key frame, point cloud data that meets the proportion requirements are screened out according to different regions, and the point cloud data that meets the proportion requirements screened out in each region are used as a group of local point cloud data corresponding to the sub-graph of the query key frame.

6. The method according to any one of claims 1 to 5, characterized in that: After determining the target grid as a grid with ghosts, the method further includes: merging the grids with ghosts, determining the ghost area according to the merged grids; and repairing the point cloud trajectory included in the ghost area.

Images