Marker mapping method, computer equipment and storage medium

Abstract

The invention discloses a marker mapping method, computer equipment and a storage medium. The method comprises the following steps: acquiring a plurality of point cloud data acquired by target equipment in a target area, wherein a marker is arranged in the target area; the point cloud data are classified, a first point cloud set and a second point cloud set are obtained, and the second point cloud set comprises the point cloud data collected from the markers; performing pose estimation by using the first point cloud set to obtain a current pose of the target equipment; performing pose correction on the point cloud data of the second point cloud set by using the current pose to obtain a target point cloud set; and based on the target point cloud set, determining a coordinate position of the marker to generate a point cloud map. According to the scheme, the marker mapping accuracy can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of pose estimation, and in particular to a method for mapping a landmark, a computer device, and a storage medium. Background Art

[0002] As equipment becomes more and more intelligent, it can realize functions such as positioning and navigation, and is widely used in many fields such as transportation and logistics, improving operational efficiency.

[0003] At present, it is usually necessary to add fixed markers to the scene. The device can use real-time positioning and mapping technology to estimate its own movement in the environment and build a map of the surrounding environment. When building a point cloud map containing markers, the accuracy of the constructed point cloud map is low due to the low accuracy of the position of the identified markers. Summary of the invention

[0004] The main technical problem solved by the present application is to provide a landmark mapping method, computer equipment and storage medium, which can improve the accuracy of pose estimation.

[0005] In order to solve the above problems, the first aspect of the present application provides a method for mapping a marker, which includes: obtaining a number of point cloud data collected by a target device in a target area, where a marker is set in the target area; classifying the number of point cloud data to obtain a first point cloud set and a second point cloud set, wherein the second point cloud set contains the point cloud data collected for the marker; using the first point cloud set to perform pose estimation to obtain the current pose of the target device; using the current pose, performing pose correction on each point cloud data of the second point cloud set to obtain a target point cloud set; based on the target point cloud set, determining the coordinate position of the marker to generate a point cloud map.

[0006] In order to solve the above-mentioned problem, the second aspect of the present application provides a computer device, which includes a memory and a processor coupled to each other, wherein program data is stored in the memory, and the processor is used to execute the program data to implement any step of the above-mentioned landmark mapping method.

[0007] In order to solve the above-mentioned problem, the third aspect of the present application provides a computer-readable storage medium, which stores program data that can be executed by a processor, and the program data is used to implement any step of the above-mentioned landmark mapping method.

[0008] The above scheme obtains a number of point cloud data collected by the target device in the target area, and there are markers in the target area. The point cloud data are classified to obtain a first point cloud set and a second point cloud set, wherein the second point cloud set includes the point cloud data collected for the marker, and the first point cloud set is used to perform pose estimation to obtain the current pose of the target device. The first point cloud set is used for pose estimation, so that the second point cloud set, i.e., the point cloud data of the marker, can be separated from the process of front-end matching and back-end optimization to obtain the current pose, thereby reducing the impact of inconsistent ranging errors between markers and non-markers on map optimization, reducing the source of errors in the mapping process, and using the current pose to correct the pose of each point cloud data of the second point cloud set to obtain the target point cloud set, which can improve the accuracy of the point cloud data set of the marker. Then, based on the target point cloud set, the coordinate position of the marker is determined, which improves the position accuracy of the acquired reflective column, thereby generating a point cloud map, and the marker can be used as a type of feature for mapping, thereby improving the accuracy of mapping the marker.

[0009] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in this application, the following briefly introduces the drawings required in the description of the embodiments. Obviously, the drawings described below are only some embodiments of this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0011] Figure 1 It is a flowchart of the first embodiment of the method for mapping landmarks of the present application;

[0012] Figure 2 This application Figure 1 A schematic diagram of a flow chart of an embodiment of step S15;

[0013] Figure 3 It is a flow chart of the second embodiment of the method for mapping landmarks of the present application;

[0014] Figure 4 It is a flowchart of the third embodiment of the method for mapping landmarks of the present application;

[0015] Figure 5 It is a structural schematic diagram of an embodiment of a posture estimation device of the present application;

[0016] Figure 6 It is a structural schematic diagram of an embodiment of a computer device of the present application;

[0017] Figure 7It is a structural diagram of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0019] The terms "first" and "second" in this application are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first" and "second" can explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices.

[0020] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0021] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the objects associated before and after are in an "or" relationship. In addition, "many" in this article means two or more than two. In addition, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of, for example, including at least one of A, B, and C, can mean including any one or more elements selected from the set consisting of A, B, and C.

[0022] The present application provides the following embodiments, and each embodiment is described in detail below.

[0023] It is understood that the method for mapping the landmark in the present application can be executed by a computer device, which can be any device with processing capabilities, such as a target device, a computer, a server, a mobile robot, etc., and the present application does not limit this. In some possible implementations, the method for mapping the landmark can be implemented by a processor calling program data stored in a memory.

[0024] See also Figure 1 , Figure 1 1 is a flow chart of the first embodiment of the method for mapping landmarks of the present application. The method may include the following steps:

[0025] S11: Acquire a number of point cloud data collected by a target device in a target area, where a marker is set.

[0026] The target device may be any movable device such as a mobile robot or a vehicle, for example, a handling robot, a service robot, a sweeping robot, etc. For example, the target device may include an AGV (Automated Guided Vehicle), an AMR (Autonomous Mobile Robot), or a sorting robot, etc. The following description of the present application takes the example that the target device includes a mobile robot, and the present application does not limit the target device.

[0027] Among them, the target device may be equipped with at least a radar device, which is used to collect point cloud data and can be used for posture estimation to achieve functions such as navigation and positioning. For example, the radar device may be a laser radar, a millimeter wave radar, etc. Exemplarily, this application uses a laser radar as an example for explanation, and the radar device may be a 3D (3 Dimensions, three-dimensional) laser radar sensor, a 2D (2 Dimensions, two-dimensional) laser radar sensor, etc. This application does not limit the radar device. In some application scenarios, the target device may also be equipped with at least one other mileage sensor, such as other mileage sensors including IMU (Inertial Measurement Unit, inertial measurement unit), wheel odometer, visual odometer, etc. This application does not limit this. The point cloud data collected by the radar device and the sensor data collected by at least one other mileage sensor can be combined for posture estimation to achieve functions such as navigation and positioning. This application does not limit the sensors installed on the target device.

[0028] Among them, the target area can be the environment where the target device is located, such as a storage environment, an indoor environment, a road environment, etc., and this application does not limit this. A marker is set in the target area, and the marker can be a reflective marker with high reflection intensity. The markers set in the target area can include one type or multiple types, and the number of markers can be single or multiple, etc. It is understandable that different types of markers can be set according to the actual environment, application scenarios, etc. of the target area, and this application does not limit the type, quantity, location, etc. of the markers set.

[0029] In some embodiments, the type of the marker includes at least one of a first type and a second type, wherein the first type indicates that the marker belongs to a plane structure type, and the second type indicates that the marker belongs to a three-dimensional structure type. The plane structure type and the three-dimensional structure type are relative. For example, the plane structure type indicates that the structure of the marker is flat and its height is less than a preset height. For example, the three-dimensional structure type indicates that the height of the structure of the marker is greater than or equal to a preset height, etc. Exemplarily, the marker may include at least one of the following: a reflective strip, a reflective plate, a reflective column, etc. Among them, the reflective strip and the reflective plate are markers of the first type, and the reflective column is a marker of the second type.

[0030] By scanning the target area with a radar device, a number of point cloud data collected by the target device in the target area can be obtained. In some application scenarios, a number of point cloud data Pscan of the current frame collected in the target area is obtained. Among them, the number of point cloud data of the current frame may include point cloud data collected by a single frame or multiple frames. In the case of point cloud data containing a single frame, pose estimation and landmark mapping can be performed on each frame to improve the accuracy of mapping; in the case of point cloud data containing multiple frames, the multiple frames can be multiple frames collected periodically or multiple frames collected by sliding windows. Using multiple frames for pose estimation and landmark mapping can reduce the number of mapping times, and combining multiple frames for mapping can improve the accuracy of mapping.

[0031] S12: Classify a number of point cloud data to obtain a first point cloud set and a second point cloud set, wherein the second point cloud set includes point cloud data collected for the landmark.

[0032] A number of point cloud data can be classified according to whether the point cloud data is collected from a marker, so as to obtain a first point cloud set and a second point cloud set, wherein the second point cloud set includes point cloud data collected from a marker, and the first point cloud set includes point cloud data collected from a non-marker.

[0033] In some embodiments, since the marker is a reflective marker with high reflection intensity, the point cloud data can be classified according to the reflection intensity, wherein the reflection intensity of the point cloud data in the first point cloud set is lower than the reflection intensity of the point cloud data in the second point cloud set. This application does not limit the classification method of the first point cloud set and the second point cloud set.

[0034] In some embodiments, a plurality of point cloud data Pscan are traversed to determine whether the reflection intensity of each point cloud data in the plurality of point cloud data is greater than a preset intensity threshold. If the reflection intensity of the point cloud data is greater than the preset intensity threshold, the point cloud corresponding to the point cloud data is considered to belong to a highlight point cloud. In response to the reflection intensity of the point cloud data being greater than the preset intensity threshold, the point cloud data may be classified and added to a second point cloud set HPoints_initial, which may also be referred to as a highlight point cloud set. Otherwise, in response to the reflection intensity of the point cloud data being not greater than the preset intensity threshold, the point cloud data may be classified and added to a first point cloud set CPoints, which may also be referred to as a common point cloud set.

[0035] In some embodiments, a number of data Pscans can be screened, and point cloud data with reflection intensity greater than a preset intensity threshold can be selected from the number of data Pscans, so as to be classified and added to the second point cloud set HPoints_initial, and the remaining point cloud data can be classified and added to the first point cloud set CPoints.

[0036] In some embodiments, after obtaining the first point cloud set CPoints and the second point cloud set HPoints_initial, the second point cloud set HPoints_initial can be screened. In the second point cloud set HPoints_initial, the point cloud data that meets the preset distance range is screened and retained as the screened second point cloud set HPoints_initial to retain scanning points with a suitable scanning distance. Among them, the preset distance range includes the scanning distance corresponding to the point cloud data being greater than the first distance threshold, and the scanning distance being less than the second distance threshold, wherein the second distance threshold is greater than the first distance threshold. In this way, point cloud data with a suitable scanning distance can be screened out, and the problem of decreased ranging accuracy when the radar device is too close or too far when scanning highly reflective markers is avoided.

[0037] In some embodiments, each point cloud data in the second point cloud set HPoints_initial satisfies the following conditions: the reflection intensity is greater than a preset intensity threshold and satisfies a preset distance range, wherein satisfying the preset distance range includes that the scanning distance corresponding to the point cloud data is greater than the first distance threshold and the scanning distance is less than the second distance threshold.

[0038] S13: Use the first point cloud set to perform pose estimation to obtain the current pose of the target device.

[0039] For the point cloud data contained in the first point cloud set CPoints, the current pose of the current frame is obtained by a pose estimation method, that is, the latest current pose of the target device in the current frame. The pose estimation method can use SLAM (Simultaneous Localization and Mapping) technology to perform pose estimation to obtain the current pose of the target device.

[0040] Exemplarily, the pose estimation method may include the ICP method (Iterative Closest Point), Kalman filtering (QSR-UKF), deep learning pose estimation method (such as PoseCNN), etc. The present application does not limit the pose estimation method.

[0041] In some embodiments, each point cloud data of the first point cloud set may be matched to obtain a plurality of matching point pairs, such as each point cloud data of the first point cloud set may be matched with the point cloud data of the historical frame to obtain a plurality of matching point pairs, each matching point pair comprising the first point cloud data of the first point cloud set and the second point cloud data of the historical frame. The historical frame may represent the previous frame or several preceding frames of the current frame, and the present application does not limit the historical frames, matching methods, etc.

[0042] Then, a relative change between the current frame and the historical frame is determined using a plurality of matching point pairs. The relative change can represent the transformation matrix relationship between the current frame and the historical frame. Specifically, a plurality of matching point pairs and the initial change can be used to construct a residual, and the residual is iteratively optimized to obtain a relative change. For example, the initial change is adjusted through multiple iterative optimizations to continuously reduce the residual until the residual is less than a preset residual threshold and / or the number of iterations reaches a preset number of conditions, and the adjusted initial change is used as the relative change.

[0043] Finally, based on the historical pose and relative change of the historical frame, the current pose of the target device in the current frame is obtained. The historical pose can represent the pose of the target device in the historical frame, and the current pose of the target device in the current frame can be obtained based on the product of the historical pose and relative change of the historical frame.

[0044] In some embodiments, after obtaining the current posture of the target device in the current frame, the current posture and the predicted posture of other mileage sensors can be fused to obtain the fused current posture. For example, the predicted posture of the mileage sensor includes the predicted posture of the IMU. The IMU includes an accelerometer and a gyroscope, which are mainly used to measure linear acceleration and angular velocity, respectively, so that the posture at the current moment can be estimated based on the posture change between the previous and next moments and the posture at the previous moment, that is, the predicted posture for the current frame is obtained. Fusion of the predicted posture of other mileage sensors can optimize the current posture and improve the accuracy of the acquired current posture.

[0045] S14: Using the current posture, correct the posture of each point cloud data of the second point cloud set to obtain a target point cloud set.

[0046] After the current pose is obtained by performing pose estimation through the first point cloud set, the current pose can be used to correct the pose of each point cloud data of the second point cloud set, and the coordinates of each point cloud data of the second point cloud set HPoints_initial can be corrected to obtain the target point cloud set.

[0047] In some embodiments, the current posture is used to perform posture conversion on each point cloud data of the second point cloud set. For example, the current posture is multiplied by each point cloud data of the second point cloud set to obtain the point cloud posture of each point cloud data in the current frame, that is, to obtain the coordinate set of the highlighted point cloud in the world coordinate system, and the set of each point cloud posture is used as the target point cloud set.

[0048] In some embodiments, since the current posture obtained above represents the posture in the world coordinate system, and the point cloud data of the second point cloud set represents the posture in the radar coordinate system of the radar device, the posture transformation can be performed on each point cloud data of the second point cloud set based on the current posture and the posture transformation relationship to obtain the point cloud posture of each point cloud data in the world coordinate system, that is, to obtain the target point cloud set. Among them, the posture transformation relationship can represent the transformation relationship between the radar coordinate system and the world coordinate system.

[0049] S15: Based on the target point cloud set, determine the coordinate position of the landmark to generate a point cloud map.

[0050] Each point cloud data contained in the filtered target point cloud set belongs to the point cloud data collected from the marker position. The highlight point clouds of the target point cloud set can be analyzed to determine the coordinate position of the marker, and then the coordinate position of the marker is marked in the point cloud map to obtain the point cloud map.

[0051] In some embodiments, see Figure 2, step S15 of the above embodiment can be further expanded. Based on the target point cloud set, the coordinate position of the marker is determined to generate a point cloud map. This embodiment may include the following steps:

[0052] S151: performing clustering processing on each point cloud data included in the target point cloud set to obtain a number of initial cluster sets.

[0053] The point cloud data contained in the target point cloud set are clustered to obtain a number of clusters, that is, a number of initial cluster sets. In the clustering process, a clustering algorithm can be used to cluster the target point cloud set. For example, the clustering algorithm includes: K-means algorithm, DBSCAN algorithm, hierarchical clustering, spectral clustering, etc. This application does not limit the clustering algorithm.

[0054] In some embodiments, among several initial clustering sets, each point cloud data belonging to the same initial clustering set meets a preset clustering condition, wherein the preset clustering condition includes: the point cloud distance between any two point cloud data is less than a preset distance measurement threshold. That is, any two highlight point cloud data whose point cloud distance between each other is less than the preset distance measurement threshold can be combined into one set.

[0055] S152: Screening several initial cluster sets to obtain a target cluster set.

[0056] Several initial clustering sets are screened to select a highlight point cloud set that is more representative and accurate for the marker, thereby obtaining a target clustering set.

[0057] In some embodiments, an initial cluster set that satisfies a preset screening condition can be screened out from a number of initial cluster sets to obtain a target cluster set. The preset screening condition includes at least one of the following: the number of point cloud data included is greater than a preset number threshold, and the number of key frames corresponding to the included point cloud data is greater than a preset number of frames. The preset number of frames is, for example, 3 frames, 5 frames, etc., and this application does not limit the preset number threshold and the preset number of frames.

[0058] Since the reflectivity of the marker is high, the highlight point clouds falling on the marker area are also more numerous and denser. This application uses the number of highlight point clouds as a screening condition, and the number of point cloud data contained in the screened initial cluster set is greater than the preset number threshold, which can improve the accuracy of the screened target cluster set.

[0059] Since the point cloud data detected in different frames may be different, some highlight point clouds may be detected in some frames but not in others, and this highlight point cloud may be misdetected. This application uses the number of key frames corresponding to the highlight point cloud as the screening condition, which can improve the accuracy of the screened target cluster set and reduce the misjudgment of the highlight point cloud.

[0060] Specifically, the number of times each point cloud data appears in each initial clustering set can be counted, that is, the point cloud data in the target clustering set of the historical frame can be matched and tracked to obtain the number of times each point cloud data appears, that is, the number of corresponding key frames that appear. If the number of key frames that appear is greater than the preset number of frames, it can be considered to be an accurate highlighted point cloud. In this way, the point cloud data in the target clustering set filtered for the current frame comes from different key frames of more than the preset number of frames. For example, the point cloud data in the target clustering set comes from different key frames of more than three frames.

[0061] Through the above method, a more accurate target cluster set can be screened, that is, a more accurate cluster set containing highlight point clouds.

[0062] S153: Using the target cluster set, determine the coordinate position of the landmark.

[0063] By analyzing each target cluster set, the coordinate position of the marker can be obtained. Among them, the center coordinates of each target cluster set can be obtained as the coordinate position of the marker. Alternatively, contour fitting is performed on each target cluster set, and the coordinate position of the marker is determined by the contour information obtained by fitting. This application does not limit this.

[0064] S154: Mark the coordinate position of the landmark into a preset grid map to obtain a point cloud map.

[0065] In some embodiments, the preset grid map is constructed by using the current position and some point cloud data. Before this step S154, the historical grid map can be refreshed in time sequence using some point cloud data of the current frame and the current position to obtain a refreshed preset grid map, that is, the current grid map.

[0066] Among them, the historical grid map referred to can be a probabilistic grid map. Exemplarily, taking the probabilistic grid map as an example, for each grid in the historical grid map, the probability of the grid being occupied (obstacle) or not occupied (free space) can be calculated based on the Bayesian formula according to a number of point cloud data. Then, the historical grid map is locally updated according to the point cloud data and the current posture. It includes integrating the newly observed point cloud data into the historical grid map, while considering the data of the previous historical frame to adjust the probability of occupation of each grid. This process is iterative, and each new point cloud data will lead to a local update of the grid map, thereby gradually constructing a more accurate grid map. In some application scenarios, the current posture can be used to convert a number of point cloud data into the world coordinate system, and then projected to the historical grid map for local map update to obtain a preset grid map.

[0067] After the coordinate positions of the markers are obtained, the coordinate positions of the markers can be marked in a preset grid map to generate a point cloud map that ultimately includes the coordinate positions of each marker.

[0068] The above scheme obtains a number of point cloud data collected by the target device in the target area, and there are markers in the target area. The point cloud data are classified to obtain a first point cloud set and a second point cloud set, wherein the second point cloud set includes the point cloud data collected for the marker, and the first point cloud set is used to perform pose estimation to obtain the current pose of the target device. The first point cloud set is used for pose estimation, so that the second point cloud set, i.e., the point cloud data of the marker, can be separated from the process of front-end matching and back-end optimization to obtain the current pose, thereby reducing the impact of inconsistent ranging errors between markers and non-markers on map optimization, reducing the source of errors in the mapping process, and using the current pose to correct the pose of each point cloud data of the second point cloud set to obtain the target point cloud set, which can improve the accuracy of the point cloud data set of the marker. Then, based on the target point cloud set, the coordinate position of the marker is determined, which improves the position accuracy of the acquired reflective column, thereby generating a point cloud map, and the marker can be used as a type of feature for mapping, thereby improving the accuracy of mapping the marker.

[0069] In addition, during the mapping process, the final current position of the landmark is determined only by the contour information of the ordinary point cloud set, so that the position of the landmark is consistent with the contour information in its probability grid map.

[0070] In some embodiments, in the above step S153, in the process of determining the coordinate position of the marker using the target cluster set, since the types of markers may be different, the coordinate position of the marker may be determined according to different types of markers and different types of position acquisition strategies. For details, please refer to the description of the following embodiment.

[0071] See also Figure 3 , Figure 3 1 is a flow chart of the second embodiment of the method for mapping landmarks of the present application. The method may include the following steps:

[0072] S21: Obtain the type of the marker corresponding to the target cluster set.

[0073] In some embodiments, the point cloud data included in each target cluster set may be analyzed to determine the type of marker corresponding to each target cluster set. For example, marker detection may be performed on each target cluster set to detect the type of marker corresponding to each target cluster set. For another example, marker fitting may be performed on each target cluster set to determine the type of marker corresponding to each target cluster set based on the fitted profile information.

[0074] In some embodiments, the type of the marker corresponding to the target cluster set can be determined in response to the type of the marker input to the target cluster set. That is, the type of the marker can be determined by the user inputting the type. In some application scenarios, after analyzing each target cluster set and failing to analyze the type to which the corresponding marker belongs, the type to which the corresponding marker belongs can be determined by receiving the type of the marker input to the target cluster set. In some application scenarios, the type of the marker can be input to the target area, and thus the corresponding marker type of the target cluster set can be determined in response to the type of the marker input to the target area. It is understandable that other methods can also be used to determine the type of the marker, and the present application does not limit the method for obtaining the type of the marker corresponding to the target cluster set.

[0075] The type of the marker includes at least one of a first type and a second type, wherein the first type indicates that the marker is a planar structure type, such as a reflective plate or a reflective strip, etc. The second type indicates that the marker is a three-dimensional structure type, such as a reflective column.

[0076] S22: Based on the position acquisition strategy corresponding to the type of the marker, the coordinate position of the marker is determined using the target cluster set.

[0077] The corresponding position acquisition strategy can be set for each type of marker, and the position acquisition strategies corresponding to different types can be the same or different. For example, when the type of the marker is the first type, the central coordinates of each target cluster set can be obtained as the coordinate position of the marker, so as to refresh the preset grid map using the central coordinates of each target cluster set, that is, mark it to the preset grid map, and obtain a point cloud map. For another example, when the type of the marker is the second type, the contour fitting is performed on each target cluster set, and the coordinate position of the marker is determined by the contour information obtained by fitting, and the preset grid map is refreshed using the coordinate position determined by the contour information of each target cluster set, that is, mark it to the preset grid map, and obtain a point cloud map. For another example, when the type of the marker corresponding to each target cluster set cannot be determined, the coordinate position of the marker can be obtained according to the input marker type or the preset marker type, and the coordinate position of the marker can be obtained according to the position acquisition strategy corresponding to each type, and then the coordinate position of the marker corresponding to the position acquisition strategy of each type is combined to obtain the final coordinate position of the marker. This application does not limit the method of obtaining the coordinate position of the marker.

[0078] The above scheme can determine the coordinate position of the marker according to the position acquisition strategy corresponding to the type of marker, using the target cluster set, which can improve the accuracy of determining the coordinate position of different types of markers. Therefore, by updating the coordinate position of the marker, that is, marking it to the preset grid map, the accuracy of the point cloud map generated for the marker can be improved.

[0079] In some embodiments, taking the example that the markers include a first type and a second type, the above embodiments are expanded to determine the coordinate positions of the markers of each different type respectively.

[0080] See also Figure 4 , Figure 4 1 is a flow chart of the third embodiment of the method for mapping landmarks of the present application. The method may include the following steps:

[0081] S31: Obtain the type of the marker corresponding to the target cluster set.

[0082] The type of the marker may include a first type or a second type, wherein the first type indicates that the marker is of a planar structure type, and the second type indicates that the marker is of a three-dimensional structure type.

[0083] Optionally, when the type of the marker includes the first type, the following step S32 and subsequent steps are performed.

[0084] Optionally, when the type of the marker includes the second type, the following step S33 and subsequent steps are performed.

[0085] S32: In response to the type of the marker being the first type, matching the target cluster set with the preset grid map to obtain a matching cluster set; obtaining preset coordinates of the matching cluster set to obtain the coordinate position of the marker.

[0086] If the type of the marker is the first type, such as a reflective strip, the target cluster set can be matched with the preset grid map to screen out the target cluster set that meets the overlap condition and obtain the matching cluster set.

[0087] In some embodiments, each target cluster set may be mapped to a preset grid map to obtain mapping point features. Then, the mapping point features are matched with the point cloud features of the preset grid map to screen out target cluster sets that meet the coincidence condition to obtain a matching cluster set. The coincidence condition includes that the mapping feature points of the target cluster set and the point cloud features of the preset grid map meet a matching relationship.

[0088] The preset coordinates of the matching cluster set are obtained, and the preset coordinates include the center coordinates or the centroid coordinates. The center coordinates can be obtained by weighted averaging the point cloud data in the matching cluster set, and the centroid coordinates represent the centroid of the matching cluster set. For example, taking the reflective strip as an example, the center coordinates of the matching cluster set can be used as the position coordinates of the corresponding reflective strip, so as to mark it in the preset grid map, and finally generate a point cloud map containing the coordinates of each reflective strip.

[0089] S33: In response to the type of the marker being the second type, construct a local truncation map for the target cluster set with a preset truncation distance; use the local truncation map to fit the marker to obtain contour information; obtain a preset contour position of the contour information to obtain the coordinate position of the marker.

[0090] If the type of the marker is the second type, such as a reflective column, a local truncated map is constructed for the target cluster set with a preset cutoff distance. Among them, TSDF (Truncated Signed Distance Function) can be used to build a map, project the target cluster set to the local truncated map, and then construct a local truncated map for the target cluster set with a preset cutoff distance. The local truncated map can also be called a TSDF map. The preset cutoff distance is the radius of the marker. For example, the reflective column can be in the shape of a cylinder, and the radius of the marker represents the radius of the cylinder. For example, the reflective column can be in the shape of a cube, and the radius of the marker represents the radius of the cube.

[0091] The TSDF map can be initialized, or the TSDF map of the historical frame can be obtained, and each target cluster set can be projected onto the TSDF map with a preset cutoff distance to obtain an updated TSDF map.

[0092] In some embodiments, the local truncated map may be a two-dimensional reconstruction map or a three-dimensional reconstruction map. It is understandable that the local truncated map may be determined according to a specific application scenario and may also include reconstruction maps of other dimensions, such as a 2.5-dimensional reconstruction map, etc. The present application does not impose any limitation on the local truncated map.

[0093] Exemplarily, when the local truncation map is a two-dimensional reconstructed map, the TSDF map is composed of some voxels, which is a two-dimensional grid array (voxel grid). Each grid point (voxel) stores the distance value and sign to the nearest surface, and its voxel can represent the projection position of the point cloud data. If the grid point is within the surface of the object, the distance value is negative; if it is outside the surface, the distance value is positive. By truncating this by a preset truncation distance, the TSDF map can be constructed for the target cluster set.

[0094] Exemplarily, when the local truncation map is a three-dimensional reconstructed map, the TSDF map is composed of a bunch of voxels, which can represent the projection position of the point cloud data. It is a grid array (voxelgrid) in a three-dimensional space, which can represent the three-dimensional surface model of the marker. Each voxel contains two variables, namely, the TSDF value (also called the voxel value, used to generate the reconstructed surface) and color information (used to attach a color texture to the reconstructed surface). Among them, the TSDF value contains the distance value and sign representing the voxel to the nearest surface. The TSDF value is calculated based on the preset cutoff distance. If the voxel is within the surface of the marker, the distance value is negative; if it is outside the surface, the distance value is positive. The TSDF value range is [-1,1]. Considering the voxels within a certain range (that is, the preset cutoff distance) from the surface of the marker, taking the preset cutoff distance as an example, the voxels beyond the preset cutoff distance are truncated and their distance information is not stored. This truncation range can affect the accuracy and reconstruction speed of the map.

[0095] Then, the point cloud data in the local truncated map is used to fit the markers, and the contour information of the markers can be fitted to obtain the contour information of each fitted marker. For example, taking the marker as a cylindrical reflective column as an example, a circle with a known reflective column diameter can be fitted.

[0096] The preset contour position of the contour information is obtained, wherein the preset contour position may be a center position, and the preset contour position may be used as the coordinate position of the marker. For example, the center position of each circle is used as the coordinate position of each marker. For example, taking a reflective column as an example, the center position of the circle may be used as the position coordinate of the corresponding reflective column, thereby marking it in a preset grid map, and finally generating a point cloud map containing the coordinates of each reflective column.

[0097] The above scheme, by matching the target cluster set with the preset grid map to obtain the matching cluster set, can screen a more accurate highlight point cloud set, and then obtain the preset coordinates of the matching cluster set to obtain the coordinate position of the marker, which can improve the position accuracy of the acquired reflective strip or reflective plate, thereby improving the accuracy of marker mapping. In addition, by establishing a local truncation map of the marker to fit the contour information of the marker, the random error of the scanning distance of the highlight area of ​​the marker is effectively eliminated, and the contour information of the reflective column can be better restored, which is conducive to the subsequent fitting of the reflective column and improving the position accuracy of the acquired reflective column, thereby improving the accuracy of marker mapping.

[0098] In some embodiments, for the above-constructed point cloud map containing the coordinate positions of the markers, in the subsequent process, the point cloud data of the markers can be detected, and then the coordinate positions of the markers marked by the point cloud map can be used for posture recognition to obtain the posture of the target device.

[0099] In some embodiments, when the target device enters the target area for the first time, any embodiment of the above-mentioned marker mapping method is executed to construct a point cloud map of the marker. When the target device enters the target area subsequently, the point cloud data of the detected marker can be used, and then the coordinate position of the marker marked by the point cloud map can be used to perform posture recognition to obtain the posture of the target device.

[0100] In some embodiments, the target device executes any embodiment of the above-mentioned landmark mapping method at a preset period to construct a point cloud map of the landmark, so as to reduce the error caused by the situation that the position of the landmark may change in different time periods. This application does not limit the specific period of landmark mapping.

[0101] It is understandable that in the above method of the specific implementation mode, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0102] For the above embodiments, the present application also provides a landmark mapping device for implementing the landmark mapping method of any of the above embodiments.

[0103] See also Figure 5 , Figure 5 4 is a schematic diagram of the structure of an embodiment of a posture estimation device of the present application. The posture estimation device 40 includes a point cloud acquisition module 41, a point cloud classification module 42, a posture estimation module 43, a coordinate correction module 44 and a map construction module 45. The modules can be interconnected.

[0104] The point cloud acquisition module 41 is used to acquire a number of point cloud data collected by a target device in a target area, where a marker is provided.

[0105] The point cloud classification module 42 is used to classify a plurality of point cloud data to obtain a first point cloud set and a second point cloud set, wherein the second point cloud set includes point cloud data collected for the marker.

[0106] The pose estimation module 43 is used to perform pose estimation using the first point cloud set to obtain the current pose of the target device.

[0107] The coordinate correction module 44 is used to use the current posture to correct the posture of each point cloud data of the second point cloud set to obtain a target point cloud set.

[0108] The map construction module 45 is used to determine the coordinate position of the landmark based on the target point cloud set to generate a point cloud map.

[0109] It should be noted that the mapping device for the markers provided in the above embodiment and the mapping method for the markers provided in the above embodiment belong to the same concept, wherein the specific manner in which each module and unit performs the operation has been described in detail in the method embodiment, and this application will not repeat it here. In practical applications, the mapping device for the markers provided in the above embodiment can allocate the above functions to different functional modules as needed, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above, and this application does not limit this.

[0110] For the above embodiment, the present application provides a computer device, see Figure 6 , Figure 6 1 is a schematic diagram of the structure of an embodiment of a computer device of the present application. The computer device 50 includes a memory 51 and a processor 52, wherein the memory 51 and the processor 52 are coupled to each other, the memory 51 stores program data, and the processor 52 is used to execute the program data to implement the steps of any embodiment of the above-mentioned landmark mapping method.

[0111] In this embodiment, the processor 52 may also be referred to as a CPU (Central Processing Unit). The processor 52 may be an integrated circuit chip having the ability to process signals. The processor 52 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or the processor 52 may also be any conventional processor, etc.

[0112] The method of the above embodiment can be implemented in the form of a computer program, so the present application proposes a computer readable storage medium, see Figure 7 , Figure 7 6 is a schematic diagram of a computer-readable storage medium of the present application. The computer-readable storage medium 60 stores program data 61 that can be run by a processor, and the program data 61 can be executed by the processor to implement the steps of any embodiment of the above-mentioned landmark mapping method.

[0113] The computer-readable storage medium 60 of this embodiment can be a medium that can store program data 61, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or it can also be a server that stores the program data 61. The server can send the stored program data 61 to other devices for execution, or it can also execute the stored program data 61 by itself.

[0114] In some embodiments, the functions or modules included in the device provided in the above embodiments of the present application can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and the present application will not repeat them here.

[0115] The above description of the various embodiments tends to emphasize the differences between the various embodiments. The same or similar points can be referenced to each other, and the present application will not repeat them here.

[0116] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0117] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0118] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0119] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium, which is a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of each implementation method of the present application.

[0120] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a computer-readable storage medium and executed by a computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the present application is not limited to any specific combination of hardware and software.

[0121] The above descriptions are merely embodiments of the present application and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for mapping a landmark, characterized in that: include: Acquire a plurality of point cloud data collected by a target device in a target area, wherein a marker is disposed in the target area; Classifying the plurality of point cloud data to obtain a first point cloud set and a second point cloud set, wherein the second point cloud set includes point cloud data collected for the marker; Using the first point cloud set to perform pose estimation to obtain a current pose of the target device; Using the current posture, performing posture correction on each point cloud data of the second point cloud set to obtain a target point cloud set; Based on the target point cloud set, the coordinate position of the marker is determined to generate a point cloud map.

2. The method according to claim 1, characterized in that The step of determining the coordinate position of the marker based on the target point cloud set to generate a point cloud map includes: Performing clustering processing on each point cloud data included in the target point cloud set to obtain a number of initial cluster sets; Screening the several initial cluster sets to obtain a target cluster set; Determine the coordinate position of the marker using the target cluster set; The coordinate position of the marker is marked in a preset grid map to obtain the point cloud map.

3. The method according to claim 2, characterized in that The using the target cluster set to determine the coordinate position of the marker includes: Obtaining the type of the marker corresponding to the target cluster set; Based on the position acquisition strategy corresponding to the type of the marker, the coordinate position of the marker is determined using the target cluster set.

4. The method according to claim 3, characterized in that The type of the marker includes a first type; the position acquisition strategy corresponding to the type of the marker, using the target cluster set, to determine the coordinate position of the marker, includes: In response to the type of the marker being the first type, matching the target cluster set with the preset grid map to obtain a matching cluster set; The preset coordinates of the matching cluster set are obtained to obtain the coordinate position of the marker.

5. The method according to claim 3, characterized in that: The type of the marker includes the second type; the position acquisition strategy corresponding to the type of the marker, using the target cluster set, to determine the coordinate position of the marker, includes: In response to the type of the marker being the second type, constructing a local truncation map for the target cluster set with a preset truncation distance; Using the local truncation map to fit the landmarks, and obtain contour information; The preset contour position of the contour information is acquired to obtain the coordinate position of the marker.

6. The method according to claim 3, characterized in that The type of the marker includes at least one of a first type and a second type, wherein the first type indicates that the marker belongs to a planar structure type, and the second type indicates that the marker belongs to a three-dimensional structure type.

7. The method according to claim 2, characterized in that In the plurality of initial clustering sets, each point cloud data belonging to the same initial clustering set meets a preset clustering condition, wherein the preset clustering condition includes: the point cloud distance between any two point cloud data is less than a preset distance measurement threshold; And / or, the preset grid map is obtained by constructing a map using the current posture and the plurality of point cloud data; And / or, screening the several initial cluster sets to obtain a target cluster set includes: From the several initial cluster sets, an initial cluster set that meets the preset filtering conditions is screened out to obtain the target cluster set; wherein the preset filtering conditions include at least one of the following: the number of point cloud data included is greater than a preset number threshold, and the number of key frames corresponding to the included point cloud data is greater than a preset frame number.

8. The method according to claim 1, characterized in that The using the first point cloud set to perform pose estimation to obtain the current pose of the target device includes: Matching each point cloud data of the first point cloud set to obtain a number of matching point pairs; Determine the relative change between the current frame and the historical frame using the plurality of matching point pairs; Based on the historical poses of the historical frames and the relative change amount, obtaining the current pose of the target device in the current frame; And / or, using the current posture to perform posture correction on each point cloud data of the second point cloud set to obtain a target point cloud set, including: The current posture is used to perform posture conversion on each point cloud data of the second point cloud set to obtain the point cloud posture of each point cloud data in the current frame as the target point cloud set.

9. The method according to claim 1, characterized in that: The reflection intensity of the point cloud data in the first point cloud set is lower than the reflection intensity of the point cloud data in the second point cloud set; and the classifying the plurality of point cloud data to obtain the first point cloud set and the second point cloud set comprises: In response to a reflection intensity of point cloud data among the plurality of point cloud data being greater than a preset intensity threshold, the point cloud data is classified and added to the second point cloud set; otherwise, the point cloud data is classified and added to the first point cloud set; And / or, after classifying the plurality of point cloud data to obtain the first point cloud set and the second point cloud set, the method further comprises: In the second point cloud set, point cloud data that meets a preset distance range is screened as a screened second point cloud set.

10. A computer device, characterized in that: The invention comprises a memory and a processor coupled to each other, wherein the memory stores program data, and the processor is used to execute the program data to implement the steps of the method according to any one of claims 1 to 9.

11. A computer-readable storage medium, characterized in that: Program data that can be executed by a processor are stored, and the program data is used to implement the steps of the method according to any one of claims 1 to 9.

Images