A static dense point cloud map construction method, system, device, medium and product

Abstract

This application discloses a method, system, apparatus, medium, and product for constructing a static dense point cloud map, relating to the field of map construction technology. The method includes: for any frame of a color image in the first N-1 frames of a continuous RGB-D image sequence; inputting the color image into a pre-trained YOLOv9 object detection network to obtain object detection results; dividing the color image into hierarchical grids to obtain multiple uniform grid units and performing FAST corner detection to obtain multiple feature points and their pixel coordinates, and combining this with a depth image to determine the world coordinates of each feature point; based on the object detection results, filtering each feature point through a multi-condition dynamic feature point filtering mechanism to obtain all static feature points and generate a static single-frame point cloud; fusing the static single-frame point clouds corresponding to the first N-1 frames of color images to obtain a static dense point cloud map. This application achieves high-precision positioning and high-quality dense mapping.

Description

Technical Field

[0001] This application relates to the field of map construction technology, and in particular to a method, system, device, medium and product for constructing static dense point cloud maps. Background Technology

[0002] Simultaneous Localization and Mapping (SLAM) is one of the core technologies in fields such as robot navigation and autonomous driving. Among them, visual SLAM has become an important research direction due to its advantages of low cost and rich information. Static dense point cloud maps, as the key output of visual SLAM, can provide high-precision 3D models for applications such as environmental perception and path planning.

[0003] Traditional visual SLAM algorithms are based on the assumption of a "static environment," enabling them to construct high-quality dense point cloud maps in static scenes. However, dynamic objects commonly found in real-world environments (such as pedestrians and vehicles) can severely interfere with the map building process, leading to increased localization errors, tracking failures, and even "ghosting" in the map. This makes it difficult for traditional methods to obtain clean and complete static dense point cloud maps in dynamic environments.

[0004] To address the aforementioned issues, SLAM improvement methods for dynamic scenes can be mainly categorized into three types: 1) Geometric information-based methods utilize multi-view geometric constraints to identify dynamic feature points, offering high computational efficiency but limited accuracy in dynamic judgment, and poor performance in handling low-speed or stationary dynamic objects; 2) Semantic information-based methods identify dynamic objects and remove relevant feature points through object detection or semantic segmentation networks, achieving high recognition accuracy but incurring high computational overhead and relying on a large amount of training data; 3) Multi-sensor fusion methods combine the advantages of sensors such as visual cameras, inertial measurement units, and lidar to enhance system robustness, but increase system complexity and data alignment difficulty, affecting real-time performance.

[0005] In summary, existing technologies struggle to achieve a balance between the accuracy of dynamic point cloud removal, the integrity of static background preservation, and the quality and real-time performance of the constructed map. Therefore, there is an urgent need for a static dense point cloud map construction method capable of accurately removing dynamic point clouds in dynamic environments while maximizing the preservation of static point clouds, to address the issues of low positioning accuracy and poor grounding in dynamic scenarios. Figure 1 Technical issues related to poor consistency. Summary of the Invention

[0006] The purpose of this application is to provide a method, system, device, medium and product for constructing a static dense point cloud map, so as to accurately identify and exclude dynamic targets, achieve more stable and reliable pose tracking, and ultimately generate a high-precision fully static environmental point cloud map.

[0007] To achieve the above objectives, this application provides the following solution.

[0008] Firstly, this application provides a method for constructing a static dense point cloud map, including: Obtain a continuous RGB-D image sequence, wherein the RGB-D image sequence includes N frames of color images and a depth image corresponding to each frame of color image; N≥2; Determine any one of the color images from the previous N-1 frames as the current color image, and determine the depth image corresponding to the current color image as the current depth image; The current color image is input into a pre-trained YOLOv9 object detection network to obtain the object detection results of the current color image; the object detection results include: the object category and bounding box of the object; The current color image is divided into layers into a grid, resulting in multiple uniform grid units of the current color image; FAST corner detection is performed within each grid cell to obtain multiple feature points and pixel coordinates of each feature point on the current color image. Based on the pixel coordinates of each feature point in the current color image and the depth value of the corresponding position of each feature point in the current depth image, determine the world coordinates of each feature point in the current color image; Based on the target detection results of the current color image, a multi-condition dynamic feature point filtering mechanism is used to filter each feature point in the current color image to obtain all static feature points on the current color image. The multi-condition dynamic feature point filtering mechanism includes: determining the type of bounding box in the target detection result according to the object category and a preset object category set, making a preliminary judgment based on the type of the bounding box where the feature point is located, and determining the corresponding feature points of the secondary judgment feature points obtained from the preliminary judgment in each color image after the current color image through ORB feature matching, and making a secondary judgment. The preset object category set includes a dynamic object category set and a static object category set; the type of bounding box is a dynamic object bounding box or a static object bounding box. Based on the world coordinates of all static feature points on the current color image, generate a static single-frame point cloud corresponding to the current color image; By fusing the static single-frame point clouds corresponding to the first N-1 color images in a continuous RGB-D image sequence, a static dense point cloud map is obtained.

[0009] Secondly, this application provides a static dense point cloud map construction system to implement the above-described static dense point cloud map construction method. The static dense point cloud map construction system includes: The image acquisition module is used to acquire a continuous RGB-D image sequence, wherein the RGB-D image sequence includes N frames of color images and a depth image corresponding to each frame of color image; N≥2; The definition module is used to determine any one of the color images in the previous N-1 frames as the current color image, and to determine the depth image corresponding to the current color image as the current depth image; The object detection module is used to input the current color image into the pre-trained YOLOv9 object detection network to obtain the object detection results of the current color image; the object detection results include: the object category and bounding box of the object; The grid division module is used to divide the current color image into layers into grids, resulting in multiple uniform grid units of the current color image; The corner detection module is used to perform FAST corner detection in each grid cell to obtain multiple feature points and the pixel coordinates of each feature point on the current color image. The coordinate transformation module is used to determine the world coordinates of each feature point in the current color image based on the pixel coordinates of each feature point in the current color image and the depth value of the corresponding position of each feature point in the current depth image; The filtering module is used to filter feature points in the current color image based on the target detection results of the current color image through a multi-condition dynamic feature point filtering mechanism to obtain all static feature points on the current color image. The multi-condition dynamic feature point filtering mechanism includes: determining the type of bounding box in the target detection results according to the object category and a preset set of object categories, making a preliminary judgment based on the type of the bounding box where the feature point is located, and determining the corresponding feature points of the secondary judgment feature points obtained from the preliminary judgment in each subsequent color image through ORB feature matching, and making a secondary judgment. The preset set of object categories includes a set of dynamic object categories and a set of static object categories; the type of bounding box is either a dynamic object bounding box or a static object bounding box. The single-frame point cloud construction module is used to generate a static single-frame point cloud corresponding to the current color image based on the world coordinates of all static feature points on the current color image. The map building module is used to fuse the static single-frame point clouds corresponding to the first N-1 frames of color images in a continuous RGB-D image sequence to obtain a static dense point cloud map.

[0010] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the static dense point cloud map construction method described above.

[0011] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the static dense point cloud map construction method described above.

[0012] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the static dense point cloud map construction method described above.

[0013] According to the specific embodiments provided in this application, the following technical effects are disclosed: This application discloses a method, system, device, medium, and product for constructing a static dense point cloud map. It acquires a continuous RGB-D image sequence and uses the first N-1 frames as the processing object, ensuring that each color image has a next frame for feature point displacement verification. The YOLOv9 object detection network is used to identify object categories and bounding boxes. Combined with a preset set of dynamic and static object categories, the bounding boxes are divided into dynamic or static bounding boxes, thus making a preliminary judgment based on the type of the bounding box where the feature point is located, achieving preliminary identification of dynamic objects. For feature points located in the overlapping area of ​​dynamic and static bounding boxes, ORB feature matching is used to determine their corresponding feature points in subsequent color images and a secondary judgment is performed to further accurately distinguish between dynamic and static feature points. Through the multi-condition dynamic feature point filtering mechanism of the above preliminary and secondary judgments, static feature points are accurately identified, avoiding interference from dynamic objects on feature matching and pose estimation. A static single-frame point cloud is generated based on the world coordinates of the retained static feature points, and the static single-frame point clouds of the first N-1 frames are fused to finally construct a static dense point cloud map without interference from dynamic objects. This effectively improves the pose estimation accuracy and tracking robustness of the visual SLAM system in dynamic environments, and enables the construction of high-precision point cloud maps in fully static environments. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a schematic flowchart of a static dense point cloud map construction method provided in an embodiment of this application; Figure 2 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0016] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0017] The purpose of this application is to provide a method, system, device, medium and product for constructing a static dense point cloud map, which aims to accurately identify and exclude dynamic targets, achieve more stable and reliable pose tracking, and generate a high-precision fully static environmental point cloud map.

[0018] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0019] In one exemplary embodiment, such as Figure 1 As shown, a method for constructing a static dense point cloud map is provided, including the following steps.

[0020] Step 1: Obtain a continuous RGB-D image sequence, which includes N frames of color images and a depth image corresponding to each frame of color image; N≥2.

[0021] Step 2: Determine any one of the color images from the previous N-1 frames as the current color image, and determine the depth image corresponding to the current color image as the current depth image.

[0022] Step 3: Input the current color image into the pre-trained YOLOv9 object detection network to obtain the object detection results of the current color image.

[0023] The object detection results include: the object category and the bounding box.

[0024] Step 4: Divide the current color image into layers and grids to obtain multiple uniform grid cells of the current color image.

[0025] Step 5: Perform FAST corner detection within each grid cell to obtain multiple feature points and pixel coordinates of each feature point on the current color image.

[0026] As an optional implementation, step 5 includes the following steps.

[0027] Step 51: Select any grid cell as the current grid cell.

[0028] Step 52: Determine the texture richness of the current mesh cell.

[0029] Step 53: Adaptively adjust the detection threshold of the Feature from Accelerated Segment Test (FAST) algorithm based on the texture richness of the current grid cell to obtain the improved FAST algorithm.

[0030] Specifically, texture richness is evaluated by calculating gradient variance. For sparse textured grid cells, the detection threshold is automatically lowered to ensure that a sufficient number of effective feature points can still be extracted from the grid cells. For dense textured grid cells, the detection threshold is appropriately increased to filter redundant corner points and improve computational efficiency.

[0031] Step 54: Use the improved FAST algorithm to perform FAST corner detection on the current grid cell to obtain the feature points and pixel coordinates of the feature points on the current grid cell, thereby obtaining multiple feature points and pixel coordinates of each feature point on the current color image.

[0032] Step 6: Based on the pixel coordinates of each feature point in the current color image and the depth value of the corresponding position of each feature point in the current depth image, determine the world coordinates of each feature point in the current color image.

[0033] As an optional implementation, step 6 includes the following steps.

[0034] Step 61: Based on the camera intrinsic parameters, transform the pixel coordinates of the current feature point from the pixel coordinate system to the image coordinate system to obtain the image coordinates of the current feature point.

[0035] Specifically, the formula for calculating the image coordinates of feature points is: ; in, The image coordinates of the feature points are the components of the image coordinates on the X-axis; the image coordinates are the coordinates in the image coordinate system. The image coordinates of the feature points are represented by the Y-axis components. The pixel coordinates of the feature point are the components of the pixel coordinates on the X-axis; the pixel coordinates are the coordinates in the pixel coordinate system. The pixel coordinates of the feature point are represented by the Y-axis components; , , and All are camera internal parameters; The component of the offset of the origin of the image coordinate system in the pixel coordinate system on the X-axis; The component of the offset of the origin of the image coordinate system in the pixel coordinate system on the Y-axis; The physical size of a pixel along the X-axis; This represents the physical size of a pixel along the Y-axis.

[0036] Step 62: Determine the depth value of the current feature point at the corresponding position in the current depth image.

[0037] Step 63: Determine the camera coordinates of the current feature point based on the camera intrinsic parameters, the image coordinates of the current feature point, and the depth value of the corresponding position of the current feature point in the current depth image.

[0038] Specifically, the formula for calculating the camera coordinates of feature points is: ; in, The camera coordinates of the feature points are the components of the camera coordinates on the X-axis; the camera coordinates are the coordinates in the camera coordinate system. This represents the depth value of the feature point at its corresponding location in the depth image. The camera coordinates of the feature points are represented by the Y-axis components. The camera coordinates of the feature points are represented by the Y-axis component.

[0039] Step 64: Transform the camera coordinates of the current feature point from the camera coordinate system to the world coordinate system to obtain the world coordinates of the current feature point.

[0040] Specifically, the formula for calculating the world coordinates of a feature point is: ; in, The world coordinates of the feature point are the coordinates in the world coordinate system. The camera coordinates of the feature points; This is the translation vector between the camera coordinate system and the world coordinate system; This is the rotation matrix between the camera coordinate system and the world coordinate system.

[0041] Step 7: Based on the target detection results of the current color image, filter each feature point in the current color image through a multi-condition dynamic feature point filtering mechanism to obtain all static feature points on the current color image.

[0042] The multi-condition dynamic feature point filtering mechanism includes: determining the type of bounding box in the target detection result based on the object category and a preset set of object categories, making a preliminary judgment based on the type of the bounding box where the feature point is located, and determining the corresponding feature points of the secondary judgment feature points obtained from the preliminary judgment in each color image after the current color image through ORB feature matching, and making a secondary judgment; the preset set of object categories includes a set of dynamic object categories and a set of static object categories; the type of bounding box is a dynamic object bounding box or a static object bounding box.

[0043] As an optional implementation, in step 7, the type of bounding box in the target detection result is determined according to the object category and the preset object category set, and a preliminary judgment is made based on the type of the bounding box where the feature point is located, including the following steps.

[0044] Step 711: Determine the set of object categories to which each object in the target detection result belongs, and obtain the type of the bounding box of the corresponding object; the type of the bounding box is either a static bounding box or a dynamic bounding box.

[0045] Step 712: Determine any feature point in the current color image as the initial feature point.

[0046] Step 713: If the feature point is initially determined not to be located in any bounding box or is only located in a static bounding box, then the initially determined feature point is determined as a static feature point.

[0047] Step 714: If the feature point initially determined is located in both the static bounding box and the dynamic bounding box, then the feature point initially determined is determined as the feature point for secondary determination.

[0048] As an optional implementation, step 7, the secondary determination, includes the following steps.

[0049] Step 721: Determine the current color image as the initial image.

[0050] Step 722: Based on the target detection results of the initial image and the target detection results of each color image following the initial image, perform multi-frame filtering according to the frame order to obtain multiple target images; wherein, the filtering process for any current frame image includes: Step 7221: Determine whether the stopping condition is met; the stopping condition is either the first sub-condition or the second sub-condition. The first sub-condition is that the corresponding feature point of the secondary determination feature point in the current frame image is not located in any bounding box or is only located in a static bounding box. The second sub-condition is that the number of images from the initial image to the current frame reaches a preset number. Step 7222: If so, all color images from the initial image to the current frame image are determined as target images; Step 7223: If not, update the current frame image to the color image of the next frame, and return to step 7221 until multiple target images are obtained.

[0051] Step 723: Determine the distance between the secondary judgment feature point and the world coordinates of the corresponding feature point in the target image located in two adjacent frames, and determine the average distance of all distances.

[0052] Specifically, the formula for calculating the distance between the world coordinates of any feature point and its corresponding feature point in two adjacent color images is as follows: ; in, For feature points located at the th The world coordinates of the corresponding feature points in the frame color image and the feature points located at the th frame The distance to the world coordinates of corresponding feature points in a frame of color image; For feature points located at the th The X-axis component of the world coordinates of the corresponding feature points in a frame of color image; For feature points located at the th The X-axis component of the world coordinates of the corresponding feature points in a frame of color image; For feature points located at the th The Y-axis component of the world coordinates of the corresponding feature points in a frame of color image; For feature points located at the th The Y-axis component of the world coordinates of the corresponding feature points in a frame of color image; For feature points located at the th The Z-axis component of the world coordinates of the corresponding feature points in a frame of color image; For feature points located at the th The Z-axis component of the world coordinates of the corresponding feature points in a frame of color image.

[0053] Step 724: If the average distance is less than the displacement threshold, then the secondary judgment feature point is determined as a static feature point.

[0054] Step 8: Based on the world coordinates of all static feature points on the current color image, generate a static single-frame point cloud corresponding to the current color image.

[0055] Step 9: Fuse the static single-frame point clouds corresponding to the first N-1 frames of color images in the continuous RGB-D image sequence to obtain a static dense point cloud map.

[0056] As an optional implementation, step 9 includes the following steps.

[0057] Step 91: For any current position in the world coordinate system, determine the point cloud corresponding to the current position in the static single-frame point cloud of the previous N-1 frames of color images, and obtain multiple point clouds corresponding to the current position.

[0058] Step 92: Determine the world coordinates of each point cloud corresponding to the current position and the world coordinates of the geometric center of each point cloud corresponding to the current position.

[0059] Step 93: Determine the weight of the corresponding point cloud based on the world coordinates of each point cloud and the world coordinates of the geometric center of the current position.

[0060] Specifically, the formula for calculating the weight of any point cloud corresponding to any location is as follows: ; in, For any position corresponding to the first The weight of each point cloud; For any position corresponding to the first World coordinates of a point cloud; The world coordinates of the geometric centers of the point cloud corresponding to the current location; For Euclidean distance.

[0061] Step 94: Based on the weights of each point cloud corresponding to the current location and the world coordinates, determine the fused point cloud at the current location.

[0062] Specifically, the formula for calculating the fused point cloud at any location is: ; in, The point cloud after fusion at the current location; This represents the number of point clouds corresponding to the current location.

[0063] Step 95: Combine the merged point clouds from all locations in the world coordinate system to obtain a static dense point cloud map.

[0064] In one exemplary embodiment, a static dense point cloud map construction system is provided to implement the above-described static dense point cloud map construction method. The static dense point cloud map construction system includes: The image acquisition module is used to acquire a continuous RGB-D image sequence, which includes N frames of color images and a depth image corresponding to each frame of color image; N≥2; The definition module is used to determine any one of the color images in the previous N-1 frames as the current color image, and to determine the depth image corresponding to the current color image as the current depth image; The object detection module is used to input the current color image into the pre-trained YOLOv9 object detection network to obtain the object detection results of the current color image; the object detection results include: the object category and bounding box of the object; The grid division module is used to divide the current color image into layers into grids, resulting in multiple uniform grid units of the current color image; The corner detection module is used to perform FAST corner detection in each grid cell to obtain multiple feature points and the pixel coordinates of each feature point on the current color image. The coordinate transformation module is used to determine the world coordinates of each feature point in the current color image based on the pixel coordinates of each feature point in the current color image and the depth value of the corresponding position of each feature point in the current depth image; The filtering module is used to filter feature points in the current color image based on the target detection results of the current color image through a multi-condition dynamic feature point filtering mechanism to obtain all static feature points on the current color image. The multi-condition dynamic feature point filtering mechanism includes: determining the type of bounding box in the target detection result according to the object category and a preset set of object categories, making a preliminary judgment based on the type of the bounding box where the feature point is located, and determining the corresponding feature points of the secondary judgment feature points obtained from the preliminary judgment in each subsequent color image through ORB feature matching, and making a secondary judgment. The preset set of object categories includes a set of dynamic object categories and a set of static object categories; the type of bounding box is either a dynamic object bounding box or a static object bounding box. The single-frame point cloud construction module is used to generate a static single-frame point cloud corresponding to the current color image based on the world coordinates of all static feature points on the current color image. The map building module is used to fuse the static single-frame point clouds corresponding to the first N-1 frames of color images in a continuous RGB-D image sequence to obtain a static dense point cloud map.

[0065] In one exemplary embodiment, a computer device is provided, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a static dense point cloud map construction method.

[0066] In one exemplary embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements a method for constructing a static dense point cloud map.

[0067] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements a method for constructing a static dense point cloud map.

[0068] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 2As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media to run. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a method for constructing a static dense point cloud map.

[0069] Those skilled in the art will understand that Figure 2 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0070] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0071] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0072] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0073] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0074] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for constructing a static dense point cloud map, characterized in that, The method for constructing a static dense point cloud map includes: Obtain a continuous RGB-D image sequence, wherein the RGB-D image sequence includes N frames of color images and a depth image corresponding to each frame of color image; N≥2; Determine any one of the color images from the previous N-1 frames as the current color image, and determine the depth image corresponding to the current color image as the current depth image; The current color image is input into a pre-trained YOLOv9 object detection network to obtain the object detection results of the current color image; the object detection results include: the object category and bounding box of the object; The current color image is divided into layers into a grid, resulting in multiple uniform grid units of the current color image; FAST corner detection is performed within each grid cell to obtain multiple feature points and pixel coordinates of each feature point on the current color image. Based on the pixel coordinates of each feature point in the current color image and the depth value of the corresponding position of each feature point in the current depth image, determine the world coordinates of each feature point in the current color image; Based on the target detection results of the current color image, a multi-condition dynamic feature point filtering mechanism is used to filter each feature point in the current color image to obtain all static feature points on the current color image. The multi-condition dynamic feature point filtering mechanism includes: determining the type of bounding box in the target detection result according to the object category and a preset object category set, making a preliminary judgment based on the type of the bounding box where the feature point is located, and determining the corresponding feature points of the secondary judgment feature points obtained from the preliminary judgment in each color image after the current color image through ORB feature matching, and making a secondary judgment. The preset object category set includes a dynamic object category set and a static object category set; the type of bounding box is a dynamic object bounding box or a static object bounding box. Based on the world coordinates of all static feature points on the current color image, generate a static single-frame point cloud corresponding to the current color image; By fusing the static single-frame point clouds corresponding to the first N-1 color images in a continuous RGB-D image sequence, a static dense point cloud map is obtained.

2. The static dense point cloud map construction method according to claim 1, characterized in that, FAST corner detection is performed within each grid cell to obtain multiple feature points and their pixel coordinates on the current color image, including: Define any grid cell as the current grid cell; Determine the texture richness of the current mesh cell; The detection threshold of the FAST algorithm is adaptively adjusted based on the texture richness of the current grid cell to obtain the improved FAST algorithm; The improved FAST algorithm is used to perform FAST corner detection on the current grid cell to obtain the feature points and pixel coordinates of the feature points on the current grid cell, thereby obtaining multiple feature points and pixel coordinates of each feature point on the current color image.

3. The static dense point cloud map construction method according to claim 1, characterized in that, Based on the pixel coordinates of each feature point in the current color image and the depth value of the corresponding position of each feature point in the current depth image, the world coordinates of each feature point in the current color image are determined, including: Based on the camera intrinsic parameters, the pixel coordinates of the current feature point are transformed from the pixel coordinate system to the image coordinate system to obtain the image coordinates of the current feature point; Determine the depth value of the current feature point at its corresponding location in the current depth image; The camera coordinates of the current feature point are determined based on the camera intrinsic parameters, the image coordinates of the current feature point, and the depth value of the corresponding position of the current feature point in the current depth image. Transform the camera coordinates of the current feature point from the camera coordinate system to the world coordinate system to obtain the world coordinates of the current feature point.

4. The static dense point cloud map construction method according to claim 1, characterized in that, Based on the object category and a pre-defined set of object categories, the type of bounding box in the target detection result is determined, and a preliminary judgment is made based on the type of the bounding box containing the feature point, including: Determine the set of object categories to which each object in the target detection results belongs, and obtain the type of the bounding box of the corresponding object; the type of the bounding box is either a static bounding box or a dynamic bounding box. Select any feature point in the current color image as the initial feature point; If the feature point is initially determined not to be located in any bounding box or only in a static bounding box, then the initially determined feature point is determined as a static feature point. If the feature point initially determined is located in both the static bounding box and the dynamic bounding box, then the feature point initially determined is determined as the feature point for secondary determination.

5. The static dense point cloud map construction method according to claim 4, characterized in that, Secondary determination, including: The current color image is selected as the initial image; Based on the target detection results of the initial image and the target detection results of each subsequent color image, multi-frame filtering is performed in frame order to obtain multiple target images; wherein the filtering process for any current frame image includes: Determine whether the stopping condition is met; the stopping condition is either the first sub-condition or the second sub-condition. The first sub-condition is that the corresponding feature point of the secondary determination feature point in the current frame image is not located in any bounding box or is only located in a static bounding box. The second sub-condition is that the number of images from the initial image to the current frame reaches a preset number. If so, all color images from the initial image to the current frame image will be determined as the target image; If not, the current frame image is updated to the color image of the next frame, and "determine if the stopping condition is met" is returned until multiple target images are obtained; Determine the distance between the secondary judgment feature point and the world coordinates of the corresponding feature point in the target image located in two adjacent frames, and determine the average distance of all distances; If the average distance is less than the displacement threshold, the secondary judgment feature point is determined as a static feature point.

6. The static dense point cloud map construction method according to claim 1, characterized in that, The static single-frame point clouds corresponding to the first N-1 frames of color images in a continuous RGB-D image sequence are fused to obtain a static dense point cloud map, including: For any current position in the world coordinate system, determine the point cloud corresponding to the current position in the static single-frame point cloud of the previous N-1 frames of color images, and obtain multiple point clouds corresponding to the current position. Determine the world coordinates of each point cloud corresponding to the current position and the world coordinates of the geometric center of each point cloud corresponding to the current position. The weights of the corresponding point clouds are determined based on the world coordinates of each point cloud and the world coordinates of its geometric center at the current location, respectively. Based on the weights of each point cloud corresponding to the current location and the world coordinates, determine the fused point cloud at the current location; By combining the point clouds from all locations in the world coordinate system, a static dense point cloud map is obtained.

7. A static dense point cloud map construction system, to implement the static dense point cloud map construction method according to any one of claims 1-6, characterized in that, The static dense point cloud map construction system includes: The image acquisition module is used to acquire a continuous RGB-D image sequence, wherein the RGB-D image sequence includes N frames of color images and a depth image corresponding to each frame of color image; N≥2; The definition module is used to determine any one of the color images in the previous N-1 frames as the current color image, and to determine the depth image corresponding to the current color image as the current depth image; The object detection module is used to input the current color image into the pre-trained YOLOv9 object detection network to obtain the object detection results of the current color image; the object detection results include: the object category and bounding box of the object; The grid division module is used to divide the current color image into layers into grids, resulting in multiple uniform grid units of the current color image; The corner detection module is used to perform FAST corner detection in each grid cell to obtain multiple feature points and the pixel coordinates of each feature point on the current color image. The coordinate transformation module is used to determine the world coordinates of each feature point in the current color image based on the pixel coordinates of each feature point in the current color image and the depth value of the corresponding position of each feature point in the current depth image; The filtering module is used to filter feature points in the current color image based on the target detection results of the current color image through a multi-condition dynamic feature point filtering mechanism to obtain all static feature points on the current color image. The multi-condition dynamic feature point filtering mechanism includes: determining the type of bounding box in the target detection result according to the object category and a preset object category set, making a preliminary judgment based on the type of the bounding box where the feature point is located, and determining the corresponding feature points of the secondary judgment feature points obtained from the preliminary judgment in each subsequent color image through ORB feature matching, and making a secondary judgment. The preset object category set includes a dynamic object category set and a static object category set; the type of bounding box is either a dynamic object bounding box or a static object bounding box. The single-frame point cloud construction module is used to generate a static single-frame point cloud corresponding to the current color image based on the world coordinates of all static feature points on the current color image. The map building module is used to fuse the static single-frame point clouds corresponding to the first N-1 frames of color images in a continuous RGB-D image sequence to obtain a static dense point cloud map.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the static dense point cloud map construction method according to any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the static dense point cloud map construction method according to any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the static dense point cloud map construction method according to any one of claims 1-6.

Images