Three-dimensional mapping method, device and storage medium

Abstract

The present application discloses a three-dimensional mapping method, device and storage medium, which relates to the technical field of three-dimensional mapping. The method includes: obtaining a local point cloud map corresponding to the current frame based on a preset algorithm; obtaining a global point cloud map for the target site; for any first instance object in the local point cloud map, searching for a target instance object matching any first instance object among multiple second instance objects included in the global point cloud map, performing a point cloud merging operation on the first point cloud corresponding to any first instance object and the second point cloud corresponding to the target instance object to update the global point cloud map; when the preset screening conditions are met, performing a screening and clearing operation on any second instance object in the global point cloud map to update the global point cloud map. Among them, by utilizing the attribute of the number of merging times of the coordinate points, there is an accurate deduplication effect for objects to be detected of different sizes. In addition, the memory usage of the global point cloud map can be reduced, and the processing efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of three-dimensional mapping technology, and specifically to a three-dimensional mapping method, device and storage medium. Background Art

[0002] Three-dimensional mapping is a process of creating a model with three-dimensional data by using three-dimensional production software. This process can make a two-dimensional plane three-dimensional, making it more intuitive and three-dimensional. In the process of three-dimensional mapping, the robot continuously collects multiple frames of RGB-D images (including photos of depth maps), each frame of the picture can be converted into three-dimensional point cloud data in the picture area, and the global three-dimensional point cloud map is constructed and updated through continuous multi-frame point clouds. In the process of building a global point cloud map, the existing technology merges the new instance objects of the three-dimensional point cloud map of the continuous image frames with the old instance objects in the global point cloud map to ensure the dynamic update of the global point cloud map. However, when merging the new and old instance objects, a relatively simple clustering algorithm is used for merging, and the confidence distribution elements inside the point cloud are not taken into account. Although the ghosting can be eliminated in the end, it will cause position distortion. In addition, the clustering algorithm requires the setting of the hyperparameter of the clustering radius, which is usually set to the average size of the object to be detected. This means that the clustering-based deduplication method of the existing technology can only ensure that instance objects within a certain standard size can be deduplicated with high precision, but cannot accurately deduplicate smaller or larger instance objects, resulting in poor final mapping effect. At the same time, the number of point clouds in the global point cloud map will produce periodic squeezing phenomenon with the update of multiple consecutive frames, which makes the non-real-time downsampling cause heavy memory cost. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a three-dimensional mapping method, device and storage medium to solve the technical problems of position distortion, poor mapping effect and high memory cost in the process of updating the global point cloud map in the prior art.

[0004] In order to achieve the above-mentioned purpose, the first aspect of the present application provides a three-dimensional mapping method, the method comprising:

[0005] Acquire a current frame collected by the robot, and acquire a local point cloud map corresponding to the current frame for the target site based on a preset algorithm, wherein the local point cloud map includes first point clouds corresponding to a plurality of first instance objects respectively;

[0006] Acquire a global point cloud map for the target site, where the global point cloud map includes second point clouds corresponding to the plurality of second instance objects respectively;

[0007] For any first instance object, searching for a target instance object matching the first instance object among a plurality of second instance objects included in the global point cloud image;

[0008] For any first instance object, a point cloud merging operation is performed on a first point cloud corresponding to any first instance object and a second point cloud corresponding to the target instance object to update a global point cloud map, wherein the point cloud merging operation is used to merge any first coordinate point in the first point cloud into a target coordinate point in the second point cloud that is adjacent to the any first coordinate point, and to update a first merging number of the target coordinate point;

[0009] When the preset screening conditions are met, a screening and clearing operation is performed on any second instance object in the global point cloud map to update the global point cloud map, wherein the screening and clearing operation is used to delete the corresponding coordinate point according to the first merging number of the second coordinate point in the second point cloud corresponding to the any second instance object and the preset point cloud deletion condition.

[0010] A second aspect of the present application provides a three-dimensional mapping device, comprising:

[0011] a memory configured to store instructions;

[0012] The processor is configured to call instructions from the memory and implement the above-mentioned three-dimensional mapping method when executing the instructions.

[0013] A third aspect of the present application provides a machine-readable storage medium having instructions stored thereon, the instructions being used to enable a machine to execute the above-mentioned three-dimensional mapping method.

[0014] Through the above technical solution, the coordinate point level de-ghosting algorithm is realized by utilizing the coordinate point merging times attribute, which has an accurate de-ghosting effect on objects of different sizes to be detected. In addition, the merging of coordinate points in each frame achieves a certain downsampling effect, so that the number of points in the global point cloud map can be controlled in each frame, reducing the memory usage of the global point cloud map and improving the processing efficiency of the subsequent de-ghosting algorithm.

[0015] Other features and advantages of the embodiments of the present application will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present application, but do not constitute a limitation on the embodiments of the present application. In the accompanying drawings:

[0017] Figure 1 A schematic diagram of a process flow of a three-dimensional mapping method according to an embodiment of the present application is schematically shown;

[0018] Figure 2 The structure block diagram of a three-dimensional mapping device according to an embodiment of the present application is schematically shown;

[0019] Figure 3 A schematic diagram of the structure of a computer device according to an embodiment of the present application is shown schematically. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the specific implementation methods described herein are only used to illustrate and explain the embodiments of the present application, and are not used to limit the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0021] It should be noted that if the embodiments of the present application involve directional indications (such as up, down, left, right, front, back...), such directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0022] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present application, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0023] Figure 1 The following schematically shows a flow chart of a three-dimensional mapping method according to an embodiment of the present application. Figure 1 As shown, an embodiment of the present application provides a three-dimensional mapping method, which may include the following steps.

[0024] S102, obtaining a current frame collected by the robot, and obtaining a local point cloud map of the target site corresponding to the current frame based on a preset algorithm, wherein the local point cloud map includes first point clouds corresponding to a plurality of first instance objects.

[0025] It can be understood that a robot refers to a robot that is movable and can collect images. The target site refers to the three-dimensional space site designated by the technician for three-dimensional mapping. The current frame refers to the current image corresponding to the local space area collected by the robot when performing image acquisition for the target site at the current moment. The local point cloud map refers to the point cloud image of the local space corresponding to the current frame, and the local point cloud map contains the three-dimensional point cloud data of the local space area. Specifically, in the process of three-dimensional mapping, the robot can continuously collect multiple frames of images, which can be images containing depth information, such as RGB-D images. Each frame of RGB-D image is combined with the robot's camera posture through a target detection algorithm or a target segmentation algorithm, and then based on a two-dimensional to three-dimensional mapping algorithm, it can be converted into three-dimensional point cloud data (local point cloud map) in the space area corresponding to the RGB-D image, and the global three-dimensional point cloud map is constructed and updated through the three-dimensional point cloud data of continuous multiple frames of RGB-D images.

[0026] Specifically, the points in the three-dimensional point cloud data contained in the local point cloud map belong to different groups or categories, wherein the points in each group belong to the same object or the same area, and the point cloud is divided according to features such as space, geometry, and texture, so that the point clouds divided into the same category have similar features. An instance object refers to an object created by instantiating a class, and a first instance object refers to an object or area of ​​an instantiated class in a local point cloud map. Specifically, the local point cloud map includes multiple first instance objects. Among them, each first instance object has a corresponding first point cloud. For example, a local point cloud map may include instance objects such as tables, chairs, and water cups.

[0027] S104: Acquire a global point cloud map for the target site, where the global point cloud map includes second point clouds corresponding to a plurality of second instance objects.

[0028] It can be understood that the global point cloud map refers to the point cloud image of the spatial area corresponding to the entire target site. It is understandable that objects in the target site may move and change, so the local point cloud map corresponding to the current frame can update the global point cloud map. The global point cloud map includes multiple second instance objects, and the second instance object refers to an object or area of ​​an instantiated class in the global point cloud map, and each second instance object has a corresponding second point cloud. It can be understood that the first instance object is a new instance object at the current moment, and the second instance object is an old instance object at a historical time. Since the first instance object and the second instance object respectively have corresponding attributes and can be located at their corresponding spatial positions respectively, the global point cloud map can be updated by comparing and analyzing the first instance object and the second instance object.

[0029] S106: For any first instance object, search for a target instance object matching the first instance object among a plurality of second instance objects included in the global point cloud image.

[0030] It can be understood that if there is a first instance object in the local point cloud image that is the same as the second instance object in the global point cloud image, and if the global point cloud image is updated using the point cloud data of the local point cloud image, there will be duplicate instance objects in the global point cloud image, then the second instance object that matches the first instance object in the local point cloud image, i.e., the target instance object, can be found in the global point cloud image. The matching method with the first instance object includes calculating the spatial overlap between the two instance objects, the visual feature similarity, etc.

[0031] S108, for any first instance object, perform a point cloud merging operation on a first point cloud corresponding to any first instance object and a second point cloud corresponding to the target instance object to update a global point cloud map, wherein the point cloud merging operation is used to merge any first coordinate point in the first point cloud into a target coordinate point in the second point cloud that is adjacent to the any first coordinate point, and update the first merging times of the target coordinate point.

[0032] It can be understood that the first point cloud corresponding to the matched first instance object and the second point cloud corresponding to the target instance object are subjected to a point cloud merging operation. Specifically, the first point cloud contains a plurality of first coordinate points, and the second point cloud also contains a plurality of second coordinate points. Among them, each coordinate point has a unique identifier. For the matched first instance object and the target instance object, each first coordinate point in the first point cloud corresponding to the first instance object is traversed, that is, the spatial coordinates (x, y, z) of each first coordinate point are traversed. Then, for each first coordinate point, it can be confirmed whether there is a second coordinate point adjacent to the first coordinate point according to the spatial coordinates of the second coordinate point in the second point cloud. If searched, the second coordinate point searched by the second point cloud can be determined as the target coordinate point adjacent to the first coordinate point, and it can be considered that the image information contained in the target coordinate point is the same as that of the first coordinate point. Then, the target coordinate point can be merged with the first coordinate point. The merging can be to retain the target coordinate point in the global point cloud map and delete the first coordinate point. At the same time, each time the target coordinate point is merged, the first merging number of the target coordinate point needs to be updated in real time. The first merging number is the cumulative number of merging times that the coordinate point performs the point cloud merging operation. Then, after the first point cloud corresponding to each first instance object in the local point cloud map corresponding to the current frame completes the point cloud merging operation, an updated global point cloud map can be obtained. In this way, when merging point clouds, the attribute of the number of merging times as the confidence of spatial existence is retained for each coordinate point. The more first merging times a coordinate point has, the greater the possibility that it actually exists at the corresponding position. The merging of coordinate points in each frame achieves a certain downsampling effect, so that the number of point clouds in the global point cloud map can be controlled in each frame, reducing the memory usage of the global point cloud map and improving the processing efficiency of the subsequent de-ghosting algorithm.

[0033] S110, when a preset screening condition is met, a screening and clearing operation is performed on any second instance object in the global point cloud map to update the global point cloud map, wherein the screening and clearing operation is used to delete the corresponding coordinate point according to the first merging number of the second coordinate point in the second point cloud corresponding to the any second instance object and the preset point cloud deletion condition.

[0034] It can be understood that the global point cloud map updated by the above scheme may have the problem of instance object ghosting. The preset screening condition refers to the condition set by the technician to screen out the second instance object in the global point cloud map according to technical experience. The first point cloud corresponding to the first instance object in the local point cloud map corresponding to the current frame has completed the point cloud merging operation. After the global point cloud map is updated, the second instance object in the updated global point cloud map is further screened and cleared according to the preset screening condition to further update the global point cloud map, thereby solving the problem of instance object ghosting. Among them, the screening and clearing operation is used to delete the corresponding coordinate point according to the first merging number of the second coordinate point in the second point cloud corresponding to any second instance object and the preset point cloud deletion condition. The preset point cloud deletion condition refers to the condition for deleting the second coordinate point in the second point cloud specified in the preset screening condition. When the point cloud is merged, the attribute of the number of merging times as the confidence of spatial existence is retained for each coordinate point. The coordinate point with more first merging times indicates that it is more likely to exist in the corresponding position. Therefore, the deduplication algorithm based on the first merge number attribute can fully consider the confidence distribution inside the point cloud, avoid removing accurate coordinate points and retaining inaccurate coordinate points, improve the point cloud position accuracy of the instance object, and ultimately improve the accuracy of 3D mapping.

[0035] Through the above technical solution, the coordinate point level de-ghosting algorithm is realized by utilizing the coordinate point merging times attribute, which has an accurate de-ghosting effect on objects of different sizes to be detected. In addition, the merging of coordinate points in each frame achieves a certain downsampling effect, so that the number of points in the global point cloud map can be controlled in each frame, reducing the memory usage of the global point cloud map and improving the processing efficiency of the subsequent de-ghosting algorithm.

[0036] With respect to S106, in an embodiment of the present application, for any first instance object, searching for a target instance object that matches any first instance object among multiple second instance objects included in the global point cloud map includes: for any first instance object, determining the spatial overlap between each second instance object included in the global point cloud map and the any first instance object; for any first instance object, determining as a third instance object a second instance object among multiple second instance objects whose spatial overlap with the any first instance object is greater than a first preset threshold value; for any first instance object, determining the visual feature similarity between the third instance object and the any first instance object; for any first instance object, when the visual feature similarity between the third instance object and the any first instance object is greater than a second preset threshold value, determining the third instance object as the target instance object that matches the any first instance object.

[0037] It can be understood that the spatial overlap can be represented by spatial IoU (Intersection over Union), which is an indicator for measuring the overlap of two bounding boxes. The value range of spatial IoU is between 0 and 1, and the higher the value, the higher the overlap of the two bounding boxes. For any first instance object, the overlap between the first instance object and each second instance object can be calculated by the bounding box corresponding to the first instance object and the bounding box corresponding to each second instance object. The spatial overlap between the two can be determined by calculating the spatial IoU. The first preset threshold refers to the experience value set by the technician for the spatial overlap based on technical experience. If there is a second instance object with a spatial overlap greater than the first preset threshold between the first instance objects in the global point cloud map, the second instance object can be selected as the third instance object. Further, for any first instance object, a third instance object whose visual feature similarity with any first instance object is greater than the second preset threshold is selected in the third instance object as the target instance object. It can be understood that visual feature similarity refers to a measure of the proximity of two images or objects in visual features. This concept is based on low-level features of images, such as color, texture, shape, etc. Specifically, the calculation of visual feature similarity usually involves core elements such as global similarity, local similarity and structural similarity. Among them, global similarity considers features such as color, texture and shape of the entire image. Local similarity focuses on the features of specific areas in the image, such as the details of objects or scenes. Structural similarity can measure the structural relationship between images, such as the layout of objects and scenes. Then, when the spatial overlap and visual feature similarity meet the conditions set by the technicians, the instance object in the global point cloud map can be considered to be the target instance object that matches the first instance object.

[0038] For S108, in an embodiment of the present application, for any first instance object, performing a point cloud merging operation on a first point cloud corresponding to any first instance object and a second point cloud corresponding to a target instance object includes: for any first instance object, determining a neighborhood space corresponding to any first coordinate point of a first point cloud corresponding to the first instance object; for any first instance object, determining whether there is any second coordinate point included in a second point cloud corresponding to the target instance object in the neighborhood space corresponding to any first coordinate point; for any first instance object, if there is any second coordinate point in the neighborhood space corresponding to any first coordinate point, determining a second coordinate point in the neighborhood space corresponding to any first coordinate point in the second point cloud corresponding to the target instance object as a target coordinate point, retaining the target coordinate point in the global point cloud map, and updating the first merging times of the target coordinate point to perform the point cloud merging operation.

[0039] It can be understood that for the matched first instance object and target instance object, each first coordinate point in the first point cloud corresponding to the first instance object is traversed, that is, the spatial coordinates (x, y, z) of each first coordinate point are traversed, and a neighborhood space (x±a, y±a, z±a) of the first coordinate point is searched to see whether any second coordinate point in the second point cloud corresponding to the target instance object is included. If found, the second coordinate point of the second point cloud searched can be determined as the target coordinate point adjacent to the first coordinate point, and it can be considered that the image information contained in the target coordinate point is the same as that contained in the first coordinate point. Specifically, when searching for the second point cloud corresponding to the target instance object, there may be multiple second coordinate points in the neighborhood space corresponding to the first coordinate point. In order to improve the computational efficiency, the searched first coordinate point can be used as the target coordinate point, and then the search for the next first coordinate point can be continued. Then, the target coordinate point can be merged with the first coordinate point. The merge can be to retain the target coordinate point in the global point cloud map and delete the first coordinate point. At the same time, each time the target coordinate point is merged, the first merge count of the target coordinate point needs to be updated in real time. The first merging number is the cumulative number of merging times that the coordinate point performs the point cloud merging operation. Then, after the first point cloud corresponding to each first instance object in the local point cloud map corresponding to the current frame completes the point cloud merging operation, an updated global point cloud map can be obtained. In this way, when merging point clouds, the attribute of the number of merging times as the confidence of spatial existence is retained for each coordinate point. The more first merging times a coordinate point has, the greater the possibility that it actually exists at the corresponding position. The merging of coordinate points in each frame achieves a certain downsampling effect, so that the number of point clouds in the global point cloud map can be controlled in each frame, reducing the memory usage of the global point cloud map and improving the processing efficiency of the subsequent de-ghosting algorithm.

[0040] In an embodiment of the present application, for any first instance object, performing a point cloud merging operation on a first point cloud corresponding to any first instance object and a second point cloud corresponding to a target instance object also includes: for any first instance object, when any second coordinate point included in the second point cloud does not exist in a neighborhood space corresponding to any first coordinate point, adding any first coordinate point to a global point cloud map to perform a point cloud merging operation.

[0041] Specifically, the spatial coordinates (x, y, z) of each first coordinate point are traversed, and a neighborhood space (x±a, y±a, z±a) of the first coordinate point is searched to see whether any second coordinate point in the second point cloud corresponding to the target instance object is included. If no second coordinate point is found, the first coordinate point can be added to the global point cloud map to perform the point cloud merging operation. In this way, the new image information appearing in the local point cloud map can be added to the global point cloud map to achieve dynamic update of the global point cloud map.

[0042] In an embodiment of the present application, each second instance object corresponds to a visual feature, and the visual feature includes multiple dimensional vectors. The method also includes: when a point cloud merging operation is completed for all first coordinate points included in the first point cloud corresponding to any first instance object, for any first instance object, an instance merging operation is performed on any first instance object and the target instance object, and the second merging number of the target instance object is updated, and the instance merging operation is used to fuse the visual features of any first instance object and the target instance object; for any first instance object, a merging weight corresponding to any first instance object is determined according to the second merging number of the target instance object corresponding to any first instance object; for any first instance object, the value of each dimensional vector included in the visual feature corresponding to the target instance object is updated according to the merging weight corresponding to any first instance object to update the global point cloud map.

[0043] It can be understood that the instance merging operation is a merging operation of instance objects between the local point cloud map and the global point cloud map, and specifically can be a merging of visual features corresponding to the instance objects. That is, the visual features of any first instance object are fused with the corresponding target instance object. Among them, the fusion of visual features can be achieved by updating the value of the dimension vector. It can be understood that since the spatial overlap and visual feature similarity of any first instance object and the corresponding target instance object reach a preset threshold, the two can be defined as essentially the same instance object. Specifically, the second merging number of target instance objects can be used to calculate the merging weight corresponding to the instance merging operation of the first instance object, which is used as the basis for the instance merging operation. The merging weight can be obtained by normalizing the second merging number. The second merging number refers to the cumulative merging number of instance objects performing the instance merging operation. Then, for any first instance object, the value of each dimensional vector included in the visual features corresponding to the target instance object can be updated according to the merging weight corresponding to any first instance object to update the global point cloud map.

[0044] In an embodiment of the present application, for any first instance object, the value of each dimensional vector included in the visual feature corresponding to the target instance object is updated according to the merging weight corresponding to any first instance object, so as to update the global point cloud image, including updating the value of each dimensional vector according to formula (1):

[0045] (1)

[0046] in, It refers to the updated value of the j-th dimension vector included in the visual features corresponding to the target instance object. refers to the merge weight corresponding to any first instance object, It refers to the value of the i-th dimension vector j before the update, including the visual features corresponding to the target instance object. It refers to the value of the j-th dimension vector included in the visual features corresponding to any first instance object.

[0047] In an embodiment of the present application, for any first instance object, determining the merging weight corresponding to any first instance object according to the second merging number of target instance objects corresponding to any first instance object includes calculating the merging weight corresponding to any first instance object according to the following formula (2):

[0048] (2)

[0049] in, Refers to the merging weight corresponding to any first instance object, and p refers to the second merging number of target instance objects corresponding to any first instance object. It can be understood that the more second merging numbers of target instance objects corresponding to the first instance object, the smaller the corresponding merging weight value, then the dimensional vector of the visual features corresponding to the target instance object has a greater impact on the value of the updated dimensional vector, while the dimensional vector of the visual features corresponding to the first instance object has a smaller impact on the value of the updated dimensional vector. That is, the more second merging numbers of the target instance object, the more image information of the old instance object will be retained after the update of the global point cloud map, thereby improving the stability of the image information in the three-dimensional map and improving the accuracy of the three-dimensional map.

[0050] With respect to S110, in an embodiment of the present application, when a preset screening condition is met, a screening and clearing operation is performed on any second instance object in the global point cloud map to update the global point cloud map, including: for any second instance object, determining the first merging times corresponding to any second coordinate point in the second point cloud corresponding to any second instance object in the current frame and each target historical frame before the current frame, and generating a cache array corresponding to any second coordinate point according to the multiple first merging times corresponding to the any second coordinate point; when the preset screening condition is met, for any second instance object, determining the merging growth rate corresponding to any second coordinate point in the current frame according to the cache array corresponding to any second coordinate point of the second point cloud corresponding to any second instance object; for any second instance object, performing a screening and clearing operation on any second instance object according to the merging growth rate corresponding to any second coordinate point of the second point cloud corresponding to any second instance object and the preset point cloud deletion condition to update the global point cloud map.

[0051] It can be understood that the target historical frame refers to a historical frame selected from multiple historical frames acquired before the current frame. For example, it can be the first 3 frames or the first 5 frames of the current frame. Specifically, when performing a point cloud merging operation, if there is a target coordinate point corresponding to the first coordinate point, the cache array corresponding to the target coordinate point can also be updated. The cache array stores the first merging times corresponding to each second coordinate point of the second point cloud in each historical frame and the current frame, respectively, which can reflect the growth of point cloud merging in the process of updating the global point cloud map frame by frame from the local point cloud map, that is, the merging growth rate. The preset point cloud deletion condition can be a deletion condition set for the merging growth rate.

[0052] Specifically, after performing the point cloud merging operation and instance merging operation of each m-frame, there will still be many duplicate coordinate points. Then, it is necessary to perform a screening and clearing operation based on the first merging times for the second point cloud corresponding to each second instance object in the global point cloud map, so as to solve the object ghosting problem. Then, for each second point cloud of each second instance object, each second coordinate point in the second point cloud can be polled. For each second coordinate point, a cache array of size m+1 is maintained in advance to store the value of the second merging times of the second coordinate point after the most recent m frames are merged. For example, for the second coordinate point with id 001, it maintains a cache array of {100,101,102,103,103,104} (assuming m=5), which is used to represent that it has accumulated 104 merges after the current frame, and 103 merges in the previous frame. By analogy, the cache array will be updated every frame, and each element in the array will be moved forward one position, and the last element will be supplemented with the number of merges after the current frame is merged. For each second coordinate point, check the second merge times m frames ago (that is, the value of the first element of the cache array) and the value of the second merge times of the current frame (that is, the value of the last element of the cache array). Furthermore, the merge growth rate of the second coordinate point can be calculated based on the cache array. Then, for any second instance object, according to the merge growth rate corresponding to any second coordinate point of the second point cloud corresponding to any second instance object, it can be determined whether its merge growth rate meets the preset point cloud deletion condition, so that it can be determined whether to perform a screening and clearing operation on the second instance object according to the judgment result, so as to update the global point cloud map. In this way, the coordinate points can be accurately screened and cleared, thereby solving the ghosting problem of the instance object.

[0053] In an embodiment of the present application, the preset screening condition refers to that the frame number of the current frame meets the preset interval frame number. For any second instance object, determining the merge growth rate corresponding to any second coordinate point in the second point cloud in the current frame according to the cache array corresponding to any second coordinate point in the second point cloud includes: for any second instance object, determining the first target merge number and the second target merge number in the cache array corresponding to any second coordinate point of the second point cloud corresponding to any second instance object, wherein the first target merge number refers to the first merge number corresponding to the first target frame before the preset interval frame number of the current frame, and the second target merge number refers to the first merge number corresponding to the current frame after the point cloud merging operation is completed; for any second instance object, determining the first merge growth rate corresponding to any second coordinate point according to the first target merge number and the second target merge number corresponding to any second coordinate point of the second point cloud corresponding to the second instance object.

[0054] It can be understood that the preset filtering condition means that the number of frames of the current frame meets the preset interval number of frames. For example, the preset filtering condition is to perform a filtering and clearing operation every m frames, and the interval between the number of frames of the current frame and the number of frames that were last filtered and cleared is m frames, and the filtering and clearing operation can be performed on the global point cloud map updated by the current frame.

[0055] For any second instance object, determine the first target merge number and the second target merge number in the cache array corresponding to any second coordinate point of the second point cloud corresponding to any second instance object, wherein the first target merge number refers to the first merge number corresponding to the first target frame before the preset interval frame number of the current frame, and the second target merge number refers to the first merge number corresponding to the current frame after the point cloud merge operation is completed. For example, for the cache array of {100,101,102,103,103,104}, the first target merge number is 100, and the second target merge number refers to 104. Further, for any second instance object, determine the first merge growth rate corresponding to any second coordinate point of the second point cloud corresponding to the second instance object based on the first target merge number and the second target merge number corresponding to the second coordinate point. It can be understood that the first merge growth rate refers to the growth rate of the recent number of merges.

[0056] In an embodiment of the present application, for any second instance object, determining the first merged growth rate corresponding to any second coordinate point of the second point cloud corresponding to the second instance object according to the first target merge times and the second target merge times includes calculating the first merged growth rate corresponding to the any second coordinate point according to the following formula (3):

[0057] (3)

[0058] in, It refers to the first merged growth rate corresponding to the i-th second coordinate point of the second point cloud corresponding to any second instance object, It refers to the number of first target merging times corresponding to the i-th second coordinate point of the second point cloud corresponding to any second instance object. It refers to the number of second target merging times corresponding to the i-th second coordinate point of the second point cloud corresponding to any second instance object, and m refers to the preset interval frame number.

[0059] In an embodiment of the present application, the point cloud merging operation is also used to update the number of surviving frames of the target coordinate point in the second point cloud corresponding to any target instance object. For any second instance object, determining the merging growth rate corresponding to any second coordinate point in the second point cloud in the current frame according to the cache array corresponding to any second coordinate point in the second point cloud includes: for any second instance object, determining the number of surviving frames corresponding to any second coordinate point in the second point cloud corresponding to any second instance object in the current frame; for any second instance object, determining the second merging growth rate corresponding to any second coordinate point in the second point cloud corresponding to any second instance object according to the first merging times and the number of surviving frames in the current frame.

[0060] It can be understood that when performing a point cloud merging operation, if there is a target coordinate point corresponding to the first coordinate point, the number of survival frames corresponding to the target coordinate point can also be updated. Specifically, the first merging number of the target coordinate point is increased by 1 (the default initial value is 0), the number of survival frames is increased by 1 (the default initial value is 0), and the cache array corresponding to the target coordinate point is updated at the same time. For any second instance object, the number of survival frames corresponding to any second coordinate point of the second point cloud corresponding to any second instance object in the current frame can be determined, and then the second merging growth rate corresponding to any second coordinate point can be determined based on the first merging number and the number of survival frames of any second coordinate point of the second point cloud corresponding to any second instance object in the current frame. The second merging growth rate refers to the growth rate of the accumulated number of merging times.

[0061] In an embodiment of the present application, for any second instance object, determining the second merged growth rate corresponding to any second coordinate point of the second point cloud corresponding to any second instance object according to the first merge times and the number of survival frames in the current frame includes calculating the second merged growth rate corresponding to the any second coordinate point according to the following formula (4):

[0062] (4)

[0063] in, It refers to the second merged growth rate corresponding to the i-th second coordinate point of the second point cloud corresponding to any second instance object, It refers to the first merging number of the i-th second coordinate point of the second point cloud corresponding to any second instance object in the current frame. It refers to the number of frames in which the i-th second coordinate point of the second point cloud corresponding to any second instance object survives in the current frame.

[0064] In an embodiment of the present application, the merge growth rate includes a first merge growth rate, and the function expression of the preset point cloud deletion condition is shown in the following formula (5):

[0065] (5)

[0066] in, It refers to the first merged growth rate corresponding to the i-th second coordinate point of the second point cloud corresponding to any second instance object, is the number of checkpoints between the current frame and the forward look. is the first deletion force control parameter, and Z is a positive integer. The larger the value, the stronger the deletion intensity is set, and the stronger the deduplication effect is. However, this may also lead to excessive deduplication. It can be set to 2 / j (round down) to achieve a balanced deduplication effect and retain important image information.

[0067] In an embodiment of the present application, the merge growth rate includes a second merge growth rate, and the function expression of the preset point cloud deletion condition is shown in the following formula (6):

[0068] (6)

[0069] in, It refers to the second merged growth rate corresponding to the i-th second coordinate point of the second point cloud corresponding to any second instance object, refers to the second merge growth rate corresponding to the kth second coordinate point of the second point cloud corresponding to any second instance object, N refers to the total number of second coordinate points of the second point cloud corresponding to any second instance object, Refers to the second deletion force control parameter. The second deletion force control parameter It is usually set between 0 and 1. The larger the value, the stronger the deletion force and the stronger the deduplication effect. However, it may also lead to excessive deduplication. It can be set to 1 / 3 to achieve a balanced deduplication effect and retain important image information.

[0070] The function expression (5) of the above-mentioned preset point cloud deletion condition is deletion condition 1, and the function expression (6) is deletion condition 2. For each second coordinate point in the global point cloud map, if it meets deletion condition 1 or 2, the second coordinate point is deleted. At the same time, its corresponding cache array, survival frame number, merge growth rate and other attributes are also deleted. If both deletion conditions are met, the second coordinate point is retained in the global point cloud map to update the global point cloud map.

[0071] In an embodiment of the present application, the method also includes: for any second instance object, determining the number of second target merges in the cache array corresponding to any second coordinate point of the second point cloud corresponding to any second instance object; for any second instance object, if the number of second target merges corresponding to any second coordinate point of the second point cloud corresponding to any second instance object is greater than or equal to the critical number of frames to be retained, retaining any second coordinate point in the updated global point cloud map.

[0072] It can be understood that in order to avoid the phenomenon that after the observation angle of the instance object changes, the number of merges slows down due to the fact that some parts of the instance object cannot be observed for a long time, which eventually leads to the phenomenon of accidental deletion, a condition for refusing deletion can also be set. That is, for any second instance object, determine the number of second target merges in the cache array corresponding to any second coordinate point of the second point cloud corresponding to any second instance object, and the number of second target merges corresponding to any second coordinate point of the second point cloud corresponding to any second instance object is greater than or equal to the critical number of frames to be retained. In the case of , any second coordinate point is retained in the updated global point cloud map. That is to say, even if the second coordinate point meets the preset point cloud deletion condition, if it is finally determined that it also meets the condition of refusing deletion, it will not be deleted in the end, but retained in the global point cloud map, thereby realizing the update of the global point cloud map. In this way, more key information of the global point cloud map can be retained during the update of the global point cloud map.

[0073] Through the above technical solution, when merging point clouds, the merge count attribute as the confidence of spatial existence is retained for each coordinate point. The more merged times a coordinate point has, the more likely it is that it actually exists at the corresponding position. The de-duplication algorithm based on the merge count attribute can fully consider the confidence distribution inside the point cloud, avoid removing accurate coordinate points and retaining inaccurate coordinate points, improve the point cloud position accuracy of the instance object, and ultimately improve the accuracy of 3D mapping. In addition, by utilizing the merge count attribute of the coordinate point, a de-ghosting algorithm based on the coordinate point level is implemented, which has an accurate de-duplication effect on objects to be detected of different sizes. At the same time, the merging of coordinate points in each frame achieves a certain downsampling effect, so that the number of point clouds in the global point cloud map can be controlled in each frame, reducing the memory usage of the global point cloud map and improving the processing efficiency of the subsequent de-ghosting algorithm.

[0074] Figure 1 FIG. 1 is a flow chart of a three-dimensional mapping method in one embodiment. It should be understood that although Figure 1The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0075] Figure 2 The structure block diagram of a three-dimensional mapping device according to an embodiment of the present application is schematically shown. Figure 2 As shown, the embodiment of the present application provides a three-dimensional mapping method, which may include:

[0076] a memory configured to store instructions; and

[0077] The processor is configured to call instructions from the memory and implement the above-mentioned three-dimensional mapping method when executing the instructions.

[0078] Specifically, in the embodiment of the present application, the processor may be configured to:

[0079] A current frame collected by the robot is obtained, and a local point cloud map corresponding to the current frame for the target site is obtained based on a preset algorithm, wherein the local point cloud map includes first point clouds corresponding to multiple first instance objects respectively; a global point cloud map for the target site is obtained, wherein the global point cloud map includes second point clouds corresponding to multiple second instance objects respectively; for any first instance object, a target instance object matching the any first instance object is searched among multiple second instance objects included in the global point cloud map; for any first instance object, a point cloud merging operation is performed on a first point cloud corresponding to any first instance object and a second point cloud corresponding to the target instance object to update the global point cloud map, wherein the point cloud merging operation is used to merge any first coordinate point in the first point cloud into a target coordinate point in the second point cloud adjacent to the any first coordinate point, and update the first merging number of the target coordinate point; when a preset screening condition is met, a screening and clearing operation is performed on any second instance object in the global point cloud map to update the global point cloud map, wherein the screening and clearing operation is used to delete the corresponding coordinate point according to the first merging number of the second coordinate point in the second point cloud corresponding to the any second instance object and a preset point cloud deletion condition.

[0080] In an embodiment of the present application, the processor may be configured to:

[0081] For any first instance object, determine the spatial overlap between each second instance object included in the global point cloud image and the any first instance object; for any first instance object, determine a second instance object among multiple second instance objects whose spatial overlap with the any first instance object is greater than a first preset threshold as a third instance object; for any first instance object, determine the visual feature similarity between the third instance object and the any first instance object; for any first instance object, when the visual feature similarity between the third instance object and the any first instance object is greater than a second preset threshold, determine the third instance object as a target instance object that matches the any first instance object.

[0082] In an embodiment of the present application, the processor may be configured to:

[0083] For any first instance object, determine the neighborhood space corresponding to any first coordinate point of the first point cloud corresponding to the first instance object; for any first instance object, determine whether there is any second coordinate point included in the second point cloud corresponding to the target instance object in the neighborhood space corresponding to any first coordinate point; for any first instance object, if there is any second coordinate point in the neighborhood space corresponding to any first coordinate point, determine the second coordinate point in the neighborhood space corresponding to the any first coordinate point in the second point cloud corresponding to the target instance object as the target coordinate point, retain the target coordinate point in the global point cloud map, and update the first merging number of the target coordinate point to perform the point cloud merging operation.

[0084] In an embodiment of the present application, the processor may be configured to:

[0085] For any first instance object, when any second coordinate point included in the second point cloud does not exist in the neighborhood space corresponding to any first coordinate point, the any first coordinate point is added to the global point cloud map to perform a point cloud merging operation.

[0086] In an embodiment of the present application, each second instance object corresponds to a visual feature, the visual feature includes multiple dimensional vectors, and the processor may be configured to:

[0087] When the point cloud merging operation is completed for all the first coordinate points included in the first point cloud corresponding to any first instance object, for any first instance object, an instance merging operation is performed on any first instance object and the target instance object, and the second merging number of the target instance object is updated. The instance merging operation is used to fuse the visual features of the any first instance object and the target instance object; for any first instance object, a merging weight corresponding to any first instance object is determined according to the second merging number of the target instance object corresponding to any first instance object; for any first instance object, the value of each dimensional vector included in the visual features corresponding to the target instance object is updated according to the merging weight corresponding to any first instance object to update the global point cloud map.

[0088] In an embodiment of the present application, the processor may be configured to:

[0089] Update the value of each dimension vector according to formula (1):

[0090] (1)

[0091] in, It refers to the updated value of the j-th dimension vector included in the visual features corresponding to the target instance object. refers to the merge weight corresponding to any first instance object, It refers to the value of the i-th dimension vector j before the update, including the visual features corresponding to the target instance object. It refers to the value of the j-th dimension vector included in the visual features corresponding to any first instance object.

[0092] In an embodiment of the present application, the processor may be configured to:

[0093] The merging weight corresponding to any first instance object is calculated according to the following formula (2):

[0094] (2)

[0095] in, It refers to the merging weight corresponding to any first instance object, and p refers to the second merging number of target instance objects corresponding to any first instance object.

[0096] In an embodiment of the present application, the processor may be configured to:

[0097] When the preset screening conditions are met, for any second instance object, the first number of merging times corresponding to any second coordinate point in the second point cloud corresponding to any second instance object in the current frame and each target historical frame before the current frame is determined, and a cache array corresponding to any second coordinate point is generated according to the multiple first merging times corresponding to the any second coordinate point; for any second instance object, the merging growth rate corresponding to any second coordinate point in the current frame is determined according to the cache array corresponding to any second coordinate point of the second point cloud corresponding to any second instance object; for any second instance object, a screening and clearing operation is performed on any second instance object according to the merging growth rate corresponding to any second coordinate point of the second point cloud corresponding to any second instance object and the preset point cloud deletion condition, so as to update the global point cloud map.

[0098] In an embodiment of the present application, the preset screening condition refers to that the number of frames of the current frame meets the preset interval number of frames, and the processor may be configured as follows:

[0099] For any second instance object, determine the first target merging number and the second target merging number in the cache array corresponding to any second coordinate point of the second point cloud corresponding to any second instance object, wherein the first target merging number refers to the first merging number corresponding to the first target frame before the preset interval frame number of the current frame, and the second target merging number refers to the first merging number corresponding to the current frame after the point cloud merging operation is completed; for any second instance object, determine the first merging growth rate corresponding to any second coordinate point of the second point cloud corresponding to the second instance object based on the first target merging number and the second target merging number corresponding to the second coordinate point.

[0100] In an embodiment of the present application, the processor may be configured to:

[0101] The first combined growth rate corresponding to any second coordinate point is calculated according to the following formula (3):

[0102] (3)

[0103] in, It refers to the first merged growth rate corresponding to the i-th second coordinate point of the second point cloud corresponding to any second instance object, It refers to the number of first target merging times corresponding to the i-th second coordinate point of the second point cloud corresponding to any second instance object. It refers to the number of second target merging times corresponding to the i-th second coordinate point of the second point cloud corresponding to any second instance object, and m refers to the preset interval frame number.

[0104] In an embodiment of the present application, the point cloud merging operation is also used to update the number of survival frames of the target coordinate point in the second point cloud corresponding to any target instance object, and the processor can be configured to:

[0105] For any second instance object, determine the number of surviving frames corresponding to any second coordinate point of the second point cloud corresponding to any second instance object in the current frame; for any second instance object, determine the second merged growth rate corresponding to any second coordinate point based on the first merge times and the number of surviving frames of any second coordinate point of the second point cloud corresponding to any second instance object in the current frame.

[0106] In an embodiment of the present application, the processor may be configured to:

[0107] The second combined growth rate corresponding to any second coordinate point is calculated according to the following formula (4):

[0108] (4)

[0109] in, It refers to the second merged growth rate corresponding to the i-th second coordinate point of the second point cloud corresponding to any second instance object, It refers to the first merging number of the i-th second coordinate point of the second point cloud corresponding to any second instance object in the current frame. It refers to the number of frames in which the i-th second coordinate point of the second point cloud corresponding to any second instance object survives in the current frame.

[0110] In an embodiment of the present application, the merge growth rate includes a first merge growth rate, and the function expression of the preset point cloud deletion condition is shown in the following formula (5):

[0111] (5)

[0112] in, It refers to the first merged growth rate corresponding to the i-th second coordinate point of the second point cloud corresponding to any second instance object, is the number of checkpoints between the current frame and the forward look. refers to the first deletion force control parameter, and Z refers to a positive integer.

[0113] In an embodiment of the present application, the merge growth rate includes a second merge growth rate, and the function expression of the preset point cloud deletion condition is shown in the following formula (6):

[0114] (6)

[0115] in, It refers to the second merged growth rate corresponding to the i-th second coordinate point of the second point cloud corresponding to any second instance object, refers to the second merge growth rate corresponding to the kth second coordinate point of the second point cloud corresponding to any second instance object, N refers to the total number of second coordinate points of the second point cloud corresponding to any second instance object, Refers to the second deletion intensity control parameter.

[0116] In an embodiment of the present application, the processor may be configured to:

[0117] For any second instance object, determine the number of second target merges in the cache array corresponding to any second coordinate point of the second point cloud corresponding to any second instance object; for any second instance object, if the number of second target merges corresponding to any second coordinate point of the second point cloud corresponding to any second instance object is greater than or equal to the retention critical number of frames, retain the any second coordinate point in the updated global point cloud map.

[0118] An embodiment of the present application also provides a machine-readable storage medium, on which instructions are stored, and the instructions are used to enable a machine to execute the above-mentioned three-dimensional mapping method.

[0119] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 3 As shown. The computer device includes a processor A01, a network interface A02, a memory (not shown in the figure) and a database (not shown in the figure) connected through a system bus. Among them, the processor A01 of the computer device is used to provide computing and control capabilities. The memory of the computer device includes an internal memory A03 and a non-volatile storage medium A04. The non-volatile storage medium A04 stores an operating system B01, a computer program B02 and a database (not shown in the figure). The internal memory A03 provides an environment for the operation of the operating system B01 and the computer program B02 in the non-volatile storage medium A04. The database of the computer device is used to store three-dimensional mapping data. The network interface A02 of the computer device is used to communicate with an external terminal through a network connection. When the computer program B02 is executed by the processor A01, a three-dimensional mapping method is implemented.

[0120] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0121] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0122] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0123] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0124] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0125] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0126] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0127] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0128] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0129] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A three-dimensional mapping method, characterized in that: The method comprises: Acquire a current frame collected by the robot, and acquire a local point cloud map corresponding to the current frame for the target site based on a preset algorithm, wherein the local point cloud map includes first point clouds corresponding to a plurality of first instance objects respectively; Acquire a global point cloud map for the target site, wherein the global point cloud map includes second point clouds corresponding to a plurality of second instance objects respectively; For any first instance object, searching for a target instance object matching the first instance object among a plurality of second instance objects included in the global point cloud image; For any first instance object, a point cloud merging operation is performed on a first point cloud corresponding to any first instance object and a second point cloud corresponding to the target instance object to update the global point cloud map, wherein the point cloud merging operation is used to merge any first coordinate point in the first point cloud into a target coordinate point in the second point cloud that is adjacent to the any first coordinate point, and update a first merging number of the target coordinate point; When a preset screening condition is met, a screening and clearing operation is performed on any second instance object in the global point cloud map to update the global point cloud map, wherein the screening and clearing operation is used to delete the corresponding coordinate point according to the first merging number of the second coordinate point in the second point cloud corresponding to the any second instance object and the preset point cloud deletion condition.

2. The three-dimensional mapping method according to claim 1, characterized in that: The step of searching, for any first instance object, a target instance object matching any first instance object among a plurality of second instance objects included in the global point cloud image comprises: For any first instance object, determining a spatial overlap between each second instance object included in the global point cloud image and the first instance object; For any first instance object, determine a second instance object among the plurality of second instance objects whose spatial overlap with the first instance object is greater than a first preset threshold as a third instance object; For any first instance object, determining a visual feature similarity between the third instance object and any first instance object; For any first instance object, when the visual feature similarity between the third instance object and the any first instance object is greater than a second preset threshold, the third instance object is determined to be a target instance object matching the any first instance object.

3. The three-dimensional mapping method according to claim 1, characterized in that: The performing of a point cloud merging operation on a first point cloud corresponding to any first instance object and a second point cloud corresponding to the target instance object for any first instance object includes: For any first instance object, determine a neighborhood space corresponding to any first coordinate point of a first point cloud corresponding to the first instance object; For any first instance object, determining whether there is any second coordinate point included in the second point cloud corresponding to the target instance object in the neighborhood space corresponding to the any first coordinate point; For any first instance object, when any second coordinate point exists in the neighborhood space corresponding to any first coordinate point, the second coordinate point in the neighborhood space corresponding to the any first coordinate point in the second point cloud corresponding to the target instance object is determined as the target coordinate point, the target coordinate point is retained in the global point cloud map, and the first merging number of the target coordinate point is updated to perform the point cloud merging operation.

4. The three-dimensional mapping method according to claim 3, characterized in that: The step of performing a point cloud merging operation on a first point cloud corresponding to any first instance object and a second point cloud corresponding to the target instance object for any first instance object further includes: For any first instance object, if any second coordinate point included in the second point cloud does not exist in the neighborhood space corresponding to any first coordinate point, the any first coordinate point is added to the global point cloud map to perform the point cloud merging operation.

5. The three-dimensional mapping method according to claim 1, characterized in that: Each second instance object corresponds to a visual feature, wherein the visual feature includes a plurality of dimensional vectors, and the method further includes: When all first coordinate points included in the first point cloud corresponding to any first instance object complete the point cloud merging operation, for any first instance object, perform an instance merging operation on the any first instance object and the target instance object, and update the second merging number of the target instance object, wherein the instance merging operation is used to fuse the visual features of the any first instance object with the target instance object; For any first instance object, determining a merging weight corresponding to the any first instance object according to the second merging number of the target instance object corresponding to the any first instance object; For any first instance object, the value of each dimensional vector included in the visual feature corresponding to the target instance object is updated according to the merging weight corresponding to the any first instance object, so as to update the global point cloud image.

6. The three-dimensional mapping method according to claim 5, characterized in that: For any first instance object, the value of each dimensional vector included in the visual feature corresponding to the target instance object is updated according to the merging weight corresponding to the any first instance object to update the global point cloud image, including updating the value of each dimensional vector according to formula (1): （1） in, It refers to the updated value of the j-th dimension vector included in the visual feature corresponding to the target instance object, refers to the merge weight corresponding to any first instance object, It refers to the value of the i-th dimension vector j before the update, including the visual features corresponding to the target instance object. It refers to the value of the j-th dimension vector included in the visual features corresponding to any first instance object.

7. The three-dimensional mapping method according to claim 5, characterized in that: The step of determining the merging weight corresponding to any first instance object according to the second merging number of target instance objects corresponding to any first instance object includes calculating the merging weight corresponding to any first instance object according to the following formula (2): （2） in, refers to the merging weight corresponding to any of the first instance objects, and p refers to the second merging number of target instance objects corresponding to any of the first instance objects.

8. The three-dimensional mapping method according to claim 1, characterized in that: When a preset screening condition is met, performing a screening and clearing operation on any second instance object in the global point cloud image to update the global point cloud image includes: For any second instance object, determine the first merging times corresponding to any second coordinate point in the second point cloud corresponding to the any second instance object in the current frame and each target historical frame before the current frame, and generate a cache array corresponding to the any second coordinate point according to the multiple first merging times corresponding to the any second coordinate point; When the preset screening condition is met, for any second instance object, determining the merged growth rate corresponding to any second coordinate point in the current frame according to the cache array corresponding to any second coordinate point of the second point cloud corresponding to the any second instance object; For any second instance object, the screening and clearing operation is performed on the any second instance object according to the merged growth rate corresponding to any second coordinate point of the second point cloud corresponding to the any second instance object and the preset point cloud deletion condition to update the global point cloud map.

9. The three-dimensional mapping method according to claim 8, characterized in that: The preset screening condition refers to that the number of frames of the current frame meets the preset interval number of frames, and for any second instance object, determining the merged growth rate corresponding to any second coordinate point in the current frame according to the cache array corresponding to any second coordinate point in the second point cloud includes: For any second instance object, determine the first target merging number and the second target merging number in the cache array corresponding to any second coordinate point of the second point cloud corresponding to the any second instance object, wherein the first target merging number refers to the first merging number corresponding to the first target frame before the preset interval frame number of the current frame, and the second target merging number refers to the first merging number corresponding to the current frame after the point cloud merging operation is completed; For any second instance object, a first merging growth rate corresponding to any second coordinate point of the second point cloud corresponding to the second instance object is determined according to the first target merging times and the second target merging times corresponding to the any second coordinate point.

10. The three-dimensional mapping method according to claim 9, characterized in that: The determining, for any second instance object, of the first merged growth rate corresponding to any second coordinate point of the second point cloud corresponding to the second instance object according to the first target merged number and the second target merged number corresponding to the second coordinate point includes calculating the first merged growth rate corresponding to the second coordinate point according to the following formula (3): （3） in, It refers to the first merged growth rate corresponding to the i-th second coordinate point of the second point cloud corresponding to any second instance object, It refers to the number of first target merging times corresponding to the i-th second coordinate point of the second point cloud corresponding to any second instance object. It refers to the number of second target merging times corresponding to the i-th second coordinate point of the second point cloud corresponding to any second instance object, and m refers to the preset interval frame number.

11. The three-dimensional mapping method according to claim 8, characterized in that: The point cloud merging operation is also used to update the number of survival frames of the target coordinate point in the second point cloud corresponding to any target instance object, and for any second instance object, determining the merging growth rate corresponding to any second coordinate point in the current frame according to the cache array corresponding to any second coordinate point in the second point cloud includes: For any second instance object, determine the number of survival frames corresponding to any second coordinate point of the second point cloud corresponding to the any second instance object in the current frame; For any second instance object, a second merge growth rate corresponding to any second coordinate point of the second point cloud corresponding to the any second instance object is determined according to the first merge times and the number of survival frames in the current frame.

12. The three-dimensional mapping method according to claim 11, characterized in that: The determining, for any second instance object, according to the first merging times and the number of survival frames of any second coordinate point of the second point cloud corresponding to the any second instance object in the current frame, of the second merged growth rate corresponding to the any second coordinate point includes calculating the second merged growth rate corresponding to the any second coordinate point according to the following formula (4): （4） in, It refers to the second combined growth rate corresponding to the i-th second coordinate point of the second point cloud corresponding to any second instance object, It refers to the first merging number of the i-th second coordinate point of the second point cloud corresponding to any second instance object in the current frame, It refers to the number of frames in which the i-th second coordinate point of the second point cloud corresponding to any second instance object survives in the current frame.

13. The three-dimensional mapping method according to claim 8, characterized in that: The merge growth rate includes a first merge growth rate, and the function expression of the preset point cloud deletion condition is shown in the following formula (5): （5） in, It refers to the first combined growth rate corresponding to the i-th second coordinate point of the second point cloud corresponding to any second instance object, is the number of checkpoints that are forward from the current frame. refers to the first deletion force control parameter, and Z refers to a positive integer.

14. The three-dimensional mapping method according to claim 8, characterized in that: The merge growth rate includes a second merge growth rate, and the function expression of the preset point cloud deletion condition is shown in the following formula (6): （6） in, It refers to the second combined growth rate corresponding to the i-th second coordinate point of the second point cloud corresponding to any second instance object, refers to the second merged growth rate corresponding to the kth second coordinate point of the second point cloud corresponding to any second instance object, N refers to the total number of second coordinate points of the second point cloud corresponding to any second instance object, Refers to the second deletion intensity control parameter.

15. The three-dimensional mapping method according to claim 8, characterized in that: The method further comprises: For any second instance object, determine the second target merging times in the cache array corresponding to any second coordinate point of the second point cloud corresponding to the any second instance object; For any second instance object, if the number of second target merging corresponding to any second coordinate point of the second point cloud corresponding to the any second instance object is greater than or equal to the retention critical number of frames, the any second coordinate point is retained in the updated global point cloud map.

16. A three-dimensional mapping device, characterized in that: include: a memory configured to store instructions; A processor is configured to call the instructions from the memory and implement the three-dimensional mapping method according to any one of claims 1 to 15 when executing the instructions.

17. A machine-readable storage medium, characterized in that: The machine-readable storage medium stores instructions, which are used to enable a machine to execute the three-dimensional mapping method according to any one of claims 1 to 15.

Images