A SLAM navigation positioning and three-dimensional reconstruction method and device

Abstract

The present application discloses a SLAM navigation positioning and three-dimensional reconstruction method and device, which relates to the field of SLAM three-dimensional reconstruction and pose estimation. The method includes using a YOLOv8 target detection network to identify and detect dynamic and static targets in the environment in real time, thereby making dynamic and static judgments of feature points according to dynamic and static target detection frames; using static feature points that exclude dynamic feature points to estimate pose trajectories, then making dynamic and static judgments of key frame pixel points according to dynamic and static target detection frames, generating a fully static three-dimensional point cloud map according to static pixel points, and performing SLAM navigation positioning according to pose trajectories and three-dimensional point cloud maps of fully static environments. The present invention can improve pose estimation accuracy and mapping accuracy in dynamic environments, and can realize real-time operation of the SLAM system, thereby ensuring the accuracy and real-time performance of SLAM navigation positioning.

Description

Technical Field

[0001] The present application relates to the field of SLAM three-dimensional reconstruction and posture estimation, and in particular to a SLAM navigation positioning and three-dimensional reconstruction method and device. Background Art

[0002] SLAM (Simultaneous Localization And Mapping) was proposed in 1988 to solve the problem of robots building environmental maps and positioning themselves in maps in real time when moving in unknown environments. Because the starting point of technology development is applied to robots, the SLAM system is expected to work in static or low-dynamic indoor environments. However, with the development of technology, the application field of SLAM technology has gradually expanded, and the use environment has gradually changed from static low-interference indoor environments to complex and highly dynamic indoor and outdoor environments. During operation, the system will frequently be disturbed by dynamic targets such as pedestrians, animals and vehicles, causing the SLAM system to produce mismatches in the process of matching the feature points of the upper and lower frames. When these mismatches are too many, it will affect the camera posture positioning of the system, and it will not be able to complete the expected tracking, positioning and mapping work.

[0003] At present, there are studies on dynamic visual SLAM algorithms. In order to solve the problem that dynamic targets affect the robustness of the system and the accuracy of position positioning, the research directions of visual SLAM based on deep learning include target detection and semantic segmentation. In existing studies, the dynamic targets are segmented by using the deep learning neural network Mask R-CNN, and then the potential dynamic targets are segmented by using multi-view geometry. Finally, only the obtained static feature points are used for tracking and mapping. However, this method cannot build a semantic information map that meets the actual use needs; in existing studies, the impact of dynamic targets on the system is reduced by integrating the SegNet real-time semantic segmentation network and consistency detection, and the positioning accuracy and robustness are improved compared with traditional visual SLAM in complex dynamic environments. However, due to the use of semantic segmentation networks, the calculation is very time-consuming and cannot meet the real-time requirements; in existing studies, the sparse direct method and semantic segmentation are integrated, and the dynamic targets are detected and excluded by dilating the mask through the morphological filter, but the real-time performance is poor.

[0004] Analyzing the existing visual SLAM technology, the dynamic visual SLAM system using deep learning can solve the problem of dynamic objects affecting pose calculation and trajectory estimation in a dynamic environment. However, technically, first of all, because of the use of complex neural networks to segment dynamic and static targets, the system requires higher-level hardware, otherwise it cannot meet the real-time performance of the system. In addition, due to the interference of dynamic targets, it is impossible to reconstruct an undisturbed map in a dynamic environment in three dimensions, and it cannot be read and applied by robots. In view of the current social demand for unmanned driving navigation, how to meet the system's real-time performance, trajectory positioning accuracy, and mapping accuracy has become a new challenge, and a new innovative solution is urgently needed. Summary of the invention

[0005] The purpose of this application is to provide a SLAM navigation positioning and three-dimensional reconstruction method and device, which can improve the pose estimation accuracy and three-dimensional mapping accuracy of the SLAM system in a dynamic environment, and can realize the real-time operation of the SLAM system, ensuring the accuracy and real-time performance of SLAM navigation positioning.

[0006] To achieve the above objectives, this application provides the following solutions:

[0007] In a first aspect, the present application provides a SLAM navigation positioning and three-dimensional reconstruction method, comprising:

[0008] Get each frame of the image collected at the current moment, and use the YOLOv8 target detection network to determine the target detection box and target category of each target in each frame of the image;

[0009] Dividing the target detection frame into a dynamic target detection frame and a static target detection frame according to the target category;

[0010] Dividing the feature points detected in each frame of the image into dynamic feature points and static feature points according to the dynamic target detection frame and the static target detection frame;

[0011] Performing posture calculation according to the static feature points to obtain a posture trajectory;

[0012] Using the dynamic target detection frame and the static target detection frame, each pixel in each key frame image under the posture trajectory at the current moment is determined to be a dynamic pixel point and a static pixel point; the key frame image refers to a number of images selected from all frame images under the posture trajectory at the current moment;

[0013] Generate a 3D point cloud map of the current fully static environment based on static pixel points;

[0014] SLAM navigation and positioning is performed according to the posture trajectory and the three-dimensional point cloud map of the fully static environment.

[0015] In a second aspect, the present application provides a SLAM navigation positioning and three-dimensional reconstruction device, comprising:

[0016] The target recognition module is used to obtain each frame of the image collected at the current moment, and use the YOLOv8 target detection network to determine the target detection box and target category of each target in each frame of the image;

[0017] A detection frame dynamic and static division module, used for dividing the target detection frame into a dynamic target detection frame and a static target detection frame according to the target category;

[0018] A feature point dynamic and static classification module, used to classify the feature points detected in each frame of the image into dynamic feature points and static feature points according to the dynamic target detection frame and the static target detection frame;

[0019] A posture calculation module, used to perform posture calculation according to the static feature points to obtain a posture trajectory;

[0020] A pixel point dynamic and static classification module is used to use the dynamic target detection frame and the static target detection frame to determine whether each pixel point in each key frame image under the posture trajectory at the current moment is a dynamic pixel point or a static pixel point;

[0021] Point cloud map generation module, which generates a three-dimensional point cloud map of the current fully static environment based on static pixel points;

[0022] The navigation and positioning module is used to perform SLAM navigation and positioning according to the posture trajectory and the three-dimensional point cloud map of the full static environment.

[0023] In a third aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-mentioned SLAM navigation positioning and three-dimensional reconstruction method.

[0024] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned SLAM navigation positioning and three-dimensional reconstruction method.

[0025] In a fifth aspect, the present application provides a computer program product, including a computer program, which implements the above-mentioned SLAM navigation positioning and three-dimensional reconstruction method when executed by a processor.

[0026] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0027] The present application provides a SLAM navigation positioning and three-dimensional reconstruction method and device. First, the target detection frame in each frame image is identified by using the YOLOv8 target detection network, and the dynamic and static target detection frame is determined according to the target category. After that, the feature points in each frame image can be judged to be dynamic feature points or static feature points according to the dynamic and static target detection frame, and the dynamic feature points can be excluded. Only the posture calculation is performed according to the retained static feature points, which reduces the interference of dynamic targets during posture calculation in the SLAM system and ensures the accuracy of posture calculation. In addition, the pixel points in each key frame image are judged to be dynamic pixel points or static pixel points according to the dynamic and static target detection frame, and the dynamic pixel points can be excluded. Only the three-dimensional point cloud map is generated according to the retained static pixel points, which reduces the interference of dynamic targets during the generation of the three-dimensional point cloud map and ensures the accuracy of the generation of the three-dimensional point cloud map. It can be seen that the present invention uses the dynamic target recognition information output by the YOLOv8 target detection network to ensure the real-time detection, and the dynamic point cloud points of each frame are filtered out during the three-dimensional point cloud reconstruction process, and the point cloud map of the full static environment can be generated in real time, which is convenient for the robot to read and use. In addition, the recognition and exclusion of dynamic targets can be realized under the premise of ensuring real-time performance, and the pose estimation accuracy of the SLAM system in a dynamic interference environment can be optimized. Therefore, the present invention not only ensures the accuracy of SLAM navigation and positioning, but also meets the real-time requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0029] Figure 1 This is an application environment diagram of a SLAM navigation positioning and three-dimensional reconstruction method in an embodiment of the present application;

[0030] Figure 2 A schematic diagram of a process of a SLAM navigation positioning and three-dimensional reconstruction method provided in one embodiment of the present application;

[0031] Figure 3 A schematic diagram of a process for navigation and positioning of a SLAM system provided in an embodiment of the present application;

[0032] Figure 4 A schematic diagram of an example of YOLOv8-Detect target recognition provided in an embodiment of the present application;

[0033] Figure 5A schematic diagram of the pose trajectory effect of applying the YOLOv8-SLAM algorithm to the TUM dataset provided in an embodiment of the present application;

[0034] Figure 6 A schematic diagram of the mapping effect of a three-dimensional point cloud map on a TUM dataset and an EDH3D dataset provided in an embodiment of the present application;

[0035] Figure 7 A schematic diagram of functional modules of a SLAM navigation positioning and three-dimensional reconstruction device provided in another embodiment of the present application;

[0036] Figure 8 A schematic diagram of the structure of a computer device provided in accordance with another embodiment of the present application. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0038] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0039] The SLAM navigation positioning and three-dimensional reconstruction method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown, the terminal communicates with the server through the network. The data storage system can store the data that the server needs to process. The data storage system can be set up separately, integrated on the server, or placed on the cloud or other servers. The terminal can send each frame of image collected at the current moment to the server. After the server receives the image, the server uses the YOLOv8 target detection network to determine the target detection frame and target category of each target in each frame of image; divide the target detection frame into a dynamic target detection frame and a static target detection frame according to the target category; divide the feature points detected in each frame of image into dynamic feature points and static feature points according to the dynamic target detection frame and the static target detection frame; perform pose calculation according to the static feature points to obtain a pose trajectory; use the dynamic target detection frame and the static target detection frame to determine whether each pixel in each key frame image under the pose trajectory at the current moment is a dynamic pixel point or a static pixel point; the key frame image refers to a number of images selected from all frame images under the pose trajectory at the current moment; generate a three-dimensional point cloud map of the full static environment at the current moment according to the static pixel points; perform SLAM navigation and positioning according to the pose trajectory and the three-dimensional point cloud map of the full static environment. The server can feed back the obtained pose trajectory and the three-dimensional point cloud map of the full static environment to the terminal. In addition, in some embodiments, the SLAM navigation positioning and three-dimensional reconstruction method can also be implemented independently by a server or a terminal. For example, the terminal can directly perform pose trajectory estimation and generate a three-dimensional point cloud map of the fully static environment for each frame of image collected at the current moment, or the server can obtain each frame of image collected at the current moment from the data storage system, and perform pose trajectory estimation and generate a three-dimensional point cloud map of the fully static environment.

[0040] The terminals may be, but are not limited to, various desktop computers, laptops, smart phones, tablet computers, IoT devices and portable wearable devices. IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server may be implemented as an independent server or a server cluster consisting of multiple servers, or a cloud server.

[0041] In an exemplary embodiment, Figure 2 and Figure 3 As shown, a SLAM navigation positioning and three-dimensional reconstruction method is provided. The method is executed by a computer device, and can be executed by a computer device such as a terminal or a server alone, or can be executed by a terminal and a server together. In the embodiment of the present application, the method is applied to Figure 1 The server in the example is used to illustrate, including the following steps 101 to 107. Among them:

[0042] Step 101, obtain each frame of image collected at the current moment, and use the YOLOv8 target detection network to determine the target detection box and target category of each target in each frame of image, such as Figure 4 shown.

[0043] Step 102: Divide the target detection frame into a dynamic target detection frame and a static target detection frame according to the target category.

[0044] The visual SLAM system divides the detection results of the YOLOv8 target detection network (YOLOv8-Detect) into dynamic target detection frames and static target detection frames, and defines the static target detection frame as , the dynamic target detection box is ,in, and Indicates the left and top coordinate values ​​of the static target detection box; and Indicates the coordinate values ​​of the right and bottom of the static target detection box; and Indicates the left and top coordinate values ​​of the dynamic target detection box; and Indicates the coordinate values ​​of the right and bottom of the dynamic target detection box.

[0045] Step 103 , dividing the feature points detected in each frame of the image into dynamic feature points and static feature points according to the dynamic target detection frame and the static target detection frame, and excluding the dynamic feature points.

[0046] Step 104, calculate the posture according to the static feature points to obtain the posture trajectory, such as Figure 5 As shown, Figure 5 This is the result of running under the high dynamic TUM dataset Walking_xyz. Figure 5 The left figure in the figure is a comparison between the estimated trajectory of ORB-SLAM3 and the real trajectory, and the right figure is a comparison between the estimated trajectory of the SLAM of the present invention and the real trajectory, wherein the dotted line is the real trajectory. It can be seen that ORB-SLAM3 is seriously affected by dynamic interference, and the calculated trajectory deviates seriously from the real trajectory, and the error is extremely serious. However, the trajectory estimation of the present invention removes the interference of dynamic factors in a high dynamic environment, and the calculated trajectory is basically consistent with the actual trajectory without interference.

[0047] The pose estimation theory in the SLAM system is: after excluding dynamic feature points, set the feature points that meet the conditions Space point , then the relationship between the feature points and the space points is:

[0048] ;

[0049] Among them, s is the scale factor, K is the intrinsic parameter matrix of the camera itself, is the Lie algebra of the position, e is the error, all errors are summed up to construct the least squares problem:

[0050] ;

[0051] Among them, i is the feature point number, n is the number of feature points; because are all static feature points, so are all fixed values, so by solving the least squares problem, a more accurate camera pose can be obtained , and then the pose trajectory is obtained.

[0052] Step 105, using the dynamic target detection frame and the static target detection frame, each pixel in each key frame image under the posture trajectory at the current moment is determined to be a dynamic pixel and a static pixel; a key frame image refers to a number of images selected from all frame images under the posture trajectory at the current moment, and is an image with key information for navigation and positioning. The key frame image is identified by the visual SLAM system through feature points, and is an inherent function of the visual SLAM system.

[0053] Step 106: Generate a 3D point cloud map of the current fully static environment based on the static pixels. Figure 6 shown. Figure 6 (a) and (c) in the figure use the 3D point cloud map generated by the traditional visual SLAM system, which does not exclude dynamic targets. Figure 6 In (b) and (d), the YOLOv8 target detection network is used to exclude dynamic targets, and the visual SLAM system is used to generate a 3D point cloud map based only on static pixels.

[0054] Step 107: Perform SLAM navigation and positioning according to the posture trajectory and the three-dimensional point cloud map of the full static environment.

[0055] Implement the above-mentioned steps 101 to 107, and on the basis of the feature point tracking, key frame extraction generation, local mapping, pose estimation and loop detection threads of the original visual SLAM system, add target detection function and three-dimensional reconstruction function. Wherein, in the link of obtaining image information, the visual SLAM system and YOLOv8-Detect run in parallel, YOLOv8-Detect identifies the dynamic target in the target, and the visual SLAM system identifies feature points, and can realize the detection information output and feature point identification of the dynamic target at the same time through information interaction, so as to further perform pose estimation and three-dimensional reconstruction, and adopt the YOLOv8-SLAM system framework to complete all processes in real time, and can obtain more accurate trajectory information and a three-dimensional point cloud map of a fully static environment. In addition, in the feature point generation link of the ORB-SLAM system, a strategy for fusing dynamic target detection information is designed to exclude the feature points identified on the dynamic target, and effectively avoid the interference caused by the existence of dynamic feature points to pose estimation. In the fully static point cloud generation stage, a three-dimensional point cloud reconstruction and visual SLAM system sharing dynamic detection target information link is designed. During the point cloud generation process, the key frame image information of the visual SLAM system is received and the dynamic targets in the image information are excluded, thereby obtaining a three-dimensional point cloud map of the fully static environment.

[0056] In another exemplary embodiment of the present application, in step 103, because the detection frame information is a rectangular structure, it is inevitable that static feature points will be mistakenly deleted when identifying irregular dynamic targets such as people or animals. Therefore, in the dynamic feature point determination rule, in order to avoid the situation where the dynamic target detection frame contains the static target detection frame and the static feature points are mistakenly deleted. The specific implementation method is as follows:

[0057] 1) First determine the coordinates of the feature points Is it satisfied? If the condition is not met, it is directly judged as a static feature point and retained. If the condition is met, it is then judged whether it is met. If the condition is not met, it is judged as a dynamic feature point and deleted. If the condition is met, it is judged as a static feature point.

[0058] 2) In order to prevent the dynamic target detection frame to which the feature point belongs from being completely contained in the static target detection frame, resulting in the dynamic feature point being completely retained, when the feature point belongs to both a dynamic target detection frame and a static target detection frame, it will first determine whether the dynamic target detection frame to which the feature point belongs is completely contained in the static target detection frame, and then determine whether to retain or delete the feature point in the dynamic area frame.

[0059] Therefore, in step 103, the feature points detected in each frame of the image are divided into dynamic feature points and static feature points according to the dynamic target detection frame and the static target detection frame, specifically including:

[0060] (1) For each feature point in each frame of the image, determine whether the feature point is within the dynamic target detection frame to obtain a first determination result.

[0061] (2) If the first judgment result is no, the feature point is regarded as a static feature point.

[0062] (3) If the first judgment result is yes, determine whether the feature point is within the static target detection frame to obtain a second judgment result.

[0063] (4) If the second judgment result is no, the feature point is regarded as a dynamic feature point.

[0064] (5) If the second judgment result is yes, it is determined whether the dynamic target detection frame to which the feature point belongs is completely contained in the static target detection frame to which it belongs, thereby obtaining a third judgment result.

[0065] (6) If the third judgment result is yes, the feature point is regarded as a dynamic feature point.

[0066] (7) If the third judgment result is no, the feature point is regarded as a static feature point.

[0067] In another exemplary embodiment of the present application, a three-dimensional dense point cloud mapping thread is added to the visual SLAM system, and the YOLOv8-Detect detection information is shared to reconstruct a three-dimensional point cloud map of a fully static environment. The specific steps are as follows:

[0068] 1) Map construction Use the PCL point cloud library to build the map. The format of the point cloud map is set to pcl::pointRGBXYZ. By traversing each pixel q in the key frame, the PCL library obtains the point cloud data , generate point cloud point Q. In the data , , Generated by the pixel coordinates and depth information of pixel point q, , , It is generated by the color information of pixel point q. The two can be regarded as the corresponding relationship between two-dimensional space points and three-dimensional space points. The correlation relationship can be described by the following formula:

[0069] ;

[0070] Among them, K represents the camera intrinsic parameter, and Z is the proportional factor between the depth value and the actual space distance.

[0071] 2) While traversing each pixel, read the static target detection frame of YOLOv8 Dynamic target detection box information, if Then command the PCL point cloud library to read the pixel information to generate the three-dimensional space point Q corresponding to the pixel point q, otherwise skip the point cloud generation and traverse the next pixel point.

[0072] 3) In order to avoid the problem of mistakenly deleting static target pixels within the dynamic target detection frame, a similar rule to that for deleting dynamic feature points is used. and , the pixel is still considered static. Finally, a point cloud map is generated based on the static pixel.

[0073] Therefore, step 105, using the dynamic target detection frame and the static target detection frame, determines the dynamic pixel point and the static pixel point for each pixel point in each key frame image under the posture trajectory at the current moment, specifically including:

[0074] (a) for each pixel point in each key frame image under the posture trajectory at the current moment, determine whether the pixel point is within the dynamic target detection frame to obtain a fourth determination result.

[0075] (b) If the fourth determination result is no, the pixel point is regarded as a static pixel point.

[0076] (c) If the fourth judgment result is yes, then determine whether the pixel point is within the static object detection frame to obtain a fifth judgment result.

[0077] (d) If the fifth judgment result is yes, the pixel point is regarded as a static pixel point.

[0078] (e) If the fifth judgment result is no, the pixel point is regarded as a dynamic pixel point.

[0079] In another exemplary embodiment of the present application, step 106, generating a three-dimensional point cloud map of the fully static environment at the current moment according to the static pixel points, specifically includes:

[0080] (i) Traverse each key frame image at the current moment in turn.

[0081] (ii) For each static pixel point in each key frame at the current moment, the PCL point cloud library is used to generate a corresponding three-dimensional point from the static pixel point.

[0082] (iii) Repeated point cloud filtering is performed on the 3D points of two adjacent key frames.

[0083] (iiii) Generate a 3D point cloud map of the fully static environment at the current moment based on the filtered 3D points corresponding to each key frame.

[0084] The SLAM navigation positioning and three-dimensional reconstruction method based on the target recognition algorithm provided in this embodiment can be applied in complex dynamic scenes such as streets, shopping malls, indoors, etc. It can eliminate the interference of dynamic targets for navigation equipment in dynamic and complex environments, provide accurate posture information, and can construct a map of static background for the navigation equipment in a dynamic environment, thereby enhancing the practicality of the navigation equipment in static and dynamic environments.

[0085] Based on the same inventive concept, the embodiment of the present application also provides a SLAM navigation positioning and 3D reconstruction device for implementing the above-mentioned SLAM navigation positioning and 3D reconstruction method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in one or more SLAM navigation positioning and 3D reconstruction device embodiments provided below can refer to the limitations of the SLAM navigation positioning and 3D reconstruction method above, and will not be repeated here.

[0086] In an exemplary embodiment, Figure 7 As shown, a SLAM navigation positioning and three-dimensional reconstruction device is provided, including:

[0087] The target recognition module M1 is used to obtain each frame of image collected at the current moment, and use the YOLOv8 target detection network to determine the target detection box and target category of each target in each frame of image.

[0088] The detection frame dynamic and static division module M2 is used to divide the target detection frame into a dynamic target detection frame and a static target detection frame according to the target category.

[0089] The feature point dynamic and static classification module M3 is used to classify the feature points detected in each frame of the image into dynamic feature points and static feature points according to the dynamic target detection frame and the static target detection frame.

[0090] The posture calculation module M4 is used to perform posture calculation according to the static feature points to obtain a posture trajectory.

[0091] The pixel dynamic and static classification module M5 is used to use the dynamic target detection frame and the static target detection frame to determine whether each pixel in each key frame image under the posture trajectory at the current moment is a dynamic pixel or a static pixel.

[0092] The point cloud map generation module M6 generates a three-dimensional point cloud map of the fully static environment at the current moment based on static pixel points.

[0093] The navigation and positioning module M7 is used to perform SLAM navigation and positioning according to the posture trajectory and the three-dimensional point cloud map of the full static environment.

[0094] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 8 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store posture trajectories and three-dimensional point cloud maps of a fully static environment. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a SLAM navigation positioning and three-dimensional reconstruction method is implemented.

[0095] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components. In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0096] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0097] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0098] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0099] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0100] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0101] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0102] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A SLAM navigation positioning and three-dimensional reconstruction method, characterized in that: The SLAM navigation positioning and three-dimensional reconstruction method comprises: Get each frame of the image collected at the current moment, and use the YOLOv8 target detection network to determine the target detection box and target category of each target in each frame of the image; Dividing the target detection frame into a dynamic target detection frame and a static target detection frame according to the target category; Dividing the feature points detected in each frame of the image into dynamic feature points and static feature points according to the dynamic target detection frame and the static target detection frame; Performing posture calculation according to the static feature points to obtain a posture trajectory; Using the dynamic target detection frame and the static target detection frame, each pixel in each key frame image under the posture trajectory at the current moment is determined to be a dynamic pixel point and a static pixel point; the key frame image refers to a number of images selected from all frame images under the posture trajectory at the current moment; Generate a 3D point cloud map of the current fully static environment based on static pixel points; Perform SLAM navigation and positioning according to the posture trajectory and the three-dimensional point cloud map of the full static environment; The feature points detected in each frame of the image are divided into dynamic feature points and static feature points according to the dynamic target detection frame and the static target detection frame, specifically including: For each feature point in each frame of the image, determining whether the feature point is within the dynamic target detection frame to obtain a first determination result; If the first judgment result is no, the feature point is regarded as a static feature point; If the first judgment result is yes, then judging whether the feature point is within the static target detection frame, and obtaining a second judgment result; If the second judgment result is no, the feature point is regarded as a dynamic feature point; If the second judgment result is yes, then judging whether the dynamic target detection frame to which the feature point belongs is completely contained in the static target detection frame to which it belongs, and obtaining a third judgment result; If the third judgment result is yes, the feature point is regarded as a dynamic feature point; If the third judgment result is no, the feature point is regarded as a static feature point; The method of using the dynamic target detection frame and the static target detection frame to determine the dynamic pixel point and the static pixel point for each pixel point in each key frame image under the posture trajectory at the current moment specifically includes: For each pixel point in each key frame image under the posture trajectory at the current moment, determine whether the pixel point is within the dynamic target detection frame to obtain a fourth determination result; If the fourth judgment result is no, the pixel point is regarded as a static pixel point; If the fourth judgment result is yes, then judging whether the pixel point is within the static target detection frame, and obtaining a fifth judgment result; If the fifth judgment result is yes, the pixel point is regarded as a static pixel point; If the fifth judgment result is no, the pixel point is regarded as a dynamic pixel point.

2. The SLAM navigation positioning and three-dimensional reconstruction method according to claim 1, characterized in that: Generate a 3D point cloud map of the current fully static environment based on static pixel points, including: Traverse each key frame image at the current moment in turn; For each static pixel point in each key frame at the current moment, the PCL point cloud library is used to generate a corresponding three-dimensional point from the static pixel point; Perform repeated point cloud filtering on the three-dimensional points of two adjacent key frames; A 3D point cloud map of the fully static environment at the current moment is generated based on the filtered 3D points corresponding to each key frame.

3. A SLAM navigation positioning and three-dimensional reconstruction device, characterized in that: The SLAM navigation positioning and three-dimensional reconstruction device comprises: The target recognition module is used to obtain each frame of the image collected at the current moment, and use the YOLOv8 target detection network to determine the target detection box and target category of each target in each frame of the image; A detection frame dynamic and static division module, used for dividing the target detection frame into a dynamic target detection frame and a static target detection frame according to the target category; A feature point dynamic and static classification module, used to classify the feature points detected in each frame of the image into dynamic feature points and static feature points according to the dynamic target detection frame and the static target detection frame; The feature points detected in each frame of the image are divided into dynamic feature points and static feature points according to the dynamic target detection frame and the static target detection frame, specifically including: For each feature point in each frame of the image, determining whether the feature point is within the dynamic target detection frame to obtain a first determination result; If the first judgment result is no, the feature point is regarded as a static feature point; If the first judgment result is yes, then judging whether the feature point is within the static target detection frame, and obtaining a second judgment result; If the second judgment result is no, the feature point is regarded as a dynamic feature point; If the second judgment result is yes, then judging whether the dynamic target detection frame to which the feature point belongs is completely contained in the static target detection frame to which it belongs, and obtaining a third judgment result; If the third judgment result is yes, the feature point is regarded as a dynamic feature point; If the third judgment result is no, the feature point is regarded as a static feature point; A posture calculation module, used to perform posture calculation according to the static feature points to obtain a posture trajectory; A pixel point dynamic and static classification module is used to use the dynamic target detection frame and the static target detection frame to determine whether each pixel point in each key frame image under the posture trajectory at the current moment is a dynamic pixel point or a static pixel point; The method of using the dynamic target detection frame and the static target detection frame to determine the dynamic pixel point and the static pixel point for each pixel point in each key frame image under the posture trajectory at the current moment specifically includes: For each pixel point in each key frame image under the posture trajectory at the current moment, determine whether the pixel point is within the dynamic target detection frame to obtain a fourth determination result; If the fourth judgment result is no, the pixel point is regarded as a static pixel point; If the fourth judgment result is yes, then judging whether the pixel point is within the static target detection frame, and obtaining a fifth judgment result; If the fifth judgment result is yes, the pixel point is regarded as a static pixel point; If the fifth judgment result is no, the pixel point is regarded as a dynamic pixel point; Point cloud map generation module, which generates a three-dimensional point cloud map of the current fully static environment based on static pixel points; The navigation and positioning module is used to perform SLAM navigation and positioning according to the posture trajectory and the three-dimensional point cloud map of the full static environment.

4. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the SLAM navigation positioning and three-dimensional reconstruction method according to any one of claims 1 to 2.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the SLAM navigation positioning and three-dimensional reconstruction method described in any one of claims 1 to 2 is implemented.

6. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the SLAM navigation positioning and three-dimensional reconstruction method described in any one of claims 1 to 2 is implemented.

Images