A method and system for dense 3D reconstruction by fusing imaging radar point clouds and vision.

Abstract

This invention provides a method and system for dense 3D reconstruction by fusing imaging radar point cloud and visual image data, belonging to the field of 3D reconstruction technology. First, imaging radar point cloud data and visual image data are simultaneously acquired to form a dual-source data set. Next, a pixel-level depth correlation is established between the two, defining the pixel depth range. Then, visual texture detail information is extracted to fill in sparse areas of the imaging radar point cloud, forming intermediate dense point cloud data. Next, visual texture color information is mapped to the intermediate point cloud, and the spatial point depth positions are adjusted to generate dense point cloud data with coordinated depth and texture. Finally, all spatial point information is extracted to restore the target scene shape and texture, forming a color dense 3D reconstruction model. This invention effectively integrates the advantages of both types of data to achieve high-precision dense 3D reconstruction.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional reconstruction technology, and more specifically, to a dense three-dimensional reconstruction method and system that fuses imaging radar point clouds and visual data. Background Technology

[0002] In the field of 3D reconstruction technology, with the ever-increasing demands for accuracy and realism in target scene reconstruction, obtaining high-quality, dense 3D reconstruction models has become a research hotspot. Currently, common 3D reconstruction methods are mainly divided into two categories: those based on imaging radar point cloud data and those based on visual image data.

[0003] 3D reconstruction methods based on imaging radar point cloud data: Imaging radar can quickly acquire spatial point cloud information of a target scene, is insensitive to changes in ambient lighting, and can operate under complex lighting conditions. However, due to its inherent principles and hardware limitations, the point cloud data acquired by imaging radar often suffers from sparsity, especially in areas such as edges, corners, or regions with poor reflectivity. The sparse distribution of point clouds makes it difficult to fully represent the geometry of the target scene, resulting in a lack of detail and completeness in the reconstructed model.

[0004] 3D reconstruction methods based on visual image data leverage the rich texture and color information contained in visual images to intuitively reflect the surface features of a target scene. However, visual methods are highly dependent on ambient lighting conditions. Insufficient lighting, strong light reflection, or occlusion can degrade image quality, affecting the accuracy of feature extraction and matching, and consequently reducing the precision of 3D reconstruction. Furthermore, visual images themselves lack depth information, requiring the use of other techniques to acquire depth, a process prone to introducing errors that can negatively impact the reconstruction results.

[0005] Some existing studies have attempted to simply fuse imaging radar point cloud data and visual image data. However, during the fusion process, the deep correlation between the two types of data was not fully considered, resulting in poor fusion results. This approach fails to effectively address the issues of sparse point clouds and insufficient utilization of texture information, making it difficult to generate high-quality, dense 3D reconstruction models. Summary of the Invention

[0006] In view of the aforementioned problems, and in conjunction with the first aspect of the present invention, embodiments of the present invention provide a dense 3D reconstruction method that fuses imaging radar point clouds and visual data, the method comprising: Simultaneously acquire imaging radar point cloud data and visual image data to form a time-synchronized dual-source data set; Each spatial point in the imaging radar point cloud data is mapped to the corresponding image pixel position in the visual image data, establishing a pixel-level depth association between the imaging radar point cloud data and the visual image data. Based on this pixel-level depth association, the depth range corresponding to each pixel in the visual image data is defined, and the depth range is bound to the corresponding pixel. Based on pixel-level deep correlation, texture detail information corresponding to sparse regions of imaging radar point cloud data is extracted from visual image data. Combined with the spatial distribution correlation of spatial points around sparse regions in imaging radar point cloud data, the extracted texture detail information is transformed into spatial point cloud information. The transformed spatial point cloud information is then filled into sparse regions in imaging radar point cloud data. The original spatial points and the completed spatial points are integrated to form intermediate dense point cloud data. The texture and color information in the visual image data is mapped pixel by pixel to the corresponding spatial points in the intermediate dense point cloud data, establishing a one-to-one correspondence between spatial points and texture and color information. The ground truth depth value in the imaging radar point cloud data is retrieved, the depth position of each spatial point in the intermediate dense point cloud data is adjusted, and the adjusted spatial points, depth information and texture and color information are integrated to generate dense point cloud data with depth and texture coordination. The spatial location, depth, and texture color information of all spatial points in the dense point cloud data with depth and texture synergy are extracted. All spatial points are arranged according to the actual spatial relationship of the target scene to restore the spatial geometry and surface texture features of the target scene and integrate them to form a color dense 3D reconstruction model of the target scene.

[0007] Furthermore, embodiments of the present invention also provide a dense 3D reconstruction system that fuses imaging radar point clouds and visual data, characterized in that it includes: A processor; a machine-readable storage medium for storing machine-executable instructions of the processor; wherein the processor is configured to perform the above-described dense 3D reconstruction method of fused imaging radar point cloud and vision by executing the machine-executable instructions.

[0008] In another aspect, embodiments of the present invention also provide a computer program product, the computer program product including machine-executable instructions stored in a computer-readable storage medium, a processor of a dense 3D reconstruction system for fusion imaging radar point clouds and vision reading the machine-executable instructions from the computer-readable storage medium, the processor executing the machine-executable instructions, causing the dense 3D reconstruction system for fusion imaging radar point clouds and vision to perform the above-described dense 3D reconstruction method for fusion imaging radar point clouds and vision.

[0009] Based on the above, firstly, imaging radar point cloud data and visual image data are collected synchronously to form a time-synchronized dual-source data set. Then, a pixel-level depth correlation relationship is established between the imaging radar point cloud data and the visual image data, and the depth range corresponding to each pixel is defined. Based on this pixel-level depth correlation relationship, texture detail information in the visual image is extracted and converted into spatial point cloud information to fill the sparse areas of the imaging radar point cloud data, effectively solving the problem of sparse imaging radar point clouds. At the same time, the rich texture information of the visual image is fully utilized, so that the dense point cloud data in the middle has both a complete spatial structure and rich detailed features. The texture and color information of the visual image is mapped to the intermediate dense point cloud data, and the depth ground truth from the imaging radar point cloud data is retrieved to adjust the depth position of the spatial points, generating dense point cloud data with coordinated depth and texture. This further improves the accuracy and realism of the point cloud data, enabling it to not only have precise spatial position and depth information, but also realistic texture and color. Finally, all spatial points are arranged according to the actual spatial relationship of the target scene to restore the spatial geometry and surface texture features of the target scene, and integrated to form a color dense 3D reconstruction model. This achieves a complete conversion from raw data to a high-quality 3D model. The generated model has the characteristics of high precision, high realism and completeness, greatly improving the quality of 3D reconstruction. Attached Figure Description

[0010] Figure 1 This is a schematic diagram of the execution flow of the dense 3D reconstruction method for fused imaging radar point cloud and vision provided in an embodiment of the present invention.

[0011] Figure 2 This is a schematic diagram of exemplary hardware and software components of the dense 3D reconstruction system for fused imaging radar point clouds and vision provided in an embodiment of the present invention. Detailed Implementation

[0012] Figure 1 This is a flowchart illustrating a dense 3D reconstruction method for fused imaging radar point clouds and visual data provided in one embodiment of the present invention, which will be described in detail below.

[0013] Step S110: Synchronously acquire imaging radar point cloud data and visual image data to form a time-synchronized dual-source data set.

[0014] An imaging radar and a visual acquisition device are deployed, and their time synchronization is achieved through a hardware synchronization trigger mechanism, ensuring that the time deviation of each data acquisition is controlled within the microsecond range. The imaging radar device is configured to continuously acquire spatial point cloud data of the target scene at a fixed sampling frequency. Each frame of point cloud data contains multiple spatial sampling points, and each sampling point carries three-dimensional spatial position information and reflection intensity information. The visual acquisition device is configured to acquire color image data of the target scene at the same sampling frequency as the imaging radar device. Each image frame consists of pixels arranged in rows and columns, and each pixel contains color information for the red, green, and blue channels. During data acquisition, when the imaging radar device initiates a sampling, the visual acquisition device is simultaneously triggered to acquire an image. The two sets of data are marked with the same timestamp. For example, after receiving a trigger signal, the imaging radar device transmits a detection signal to the target scene and receives the echo. After signal processing, it generates a frame of point cloud data, while the visual acquisition device captures a scene image at the same moment. Both are assigned the same timestamp information and stored as a single data unit. By continuously executing the above process, the point cloud data and image data under multiple timestamps are arranged in chronological order to form a time-synchronized dual-source data set. Each element in this dual-source data set contains point cloud data and image data corresponding to the timestamp.

[0015] Step S120: Map each spatial point in the imaging radar point cloud data to the corresponding image pixel position in the visual image data, establish a pixel-level depth association between the imaging radar point cloud data and the visual image data, define the depth range corresponding to each pixel in the visual image data based on the pixel-level depth association, and bind the depth range to the corresponding pixel.

[0016] For each set of point cloud data and image data in the time-synchronized dual-source dataset, the spatial transformation parameters between the imaging radar device and the visual acquisition device are first obtained, including relative position and attitude relationships. For each spatial point in the point cloud data, it is transformed from the radar coordinate system to the camera coordinate system according to the spatial transformation parameters. Then, the 3D coordinates in the camera coordinate system are projected onto the image plane using the camera's intrinsic parameters to obtain the pixel coordinates of that spatial point in the image. The depth information of each spatial point is associated with its corresponding pixel coordinates, and the depth values ​​of all spatial points corresponding to each pixel coordinate are recorded. For each pixel in the image, the depth values ​​of all spatial points mapped to that pixel are counted, and the minimum and maximum depth values ​​are determined as the depth range of that pixel. The coordinates of each pixel and its corresponding depth range are stored in an associated data structure to achieve the binding of depth range and pixel.

[0017] Step S121: Extract the three-dimensional spatial coordinates and depth information of each spatial point in the imaging radar point cloud data, extract the pixel coordinates and texture features of each image in the visual image data, establish corresponding associations according to the acquisition time sequence, and bind the spatial point corresponding to each imaging radar point cloud data to the image region corresponding to the same time node and the same spatial location in the visual image data to form time-series associated data.

[0018] Imaging radar point cloud data and visual image data are separated from the dual-source dataset. Each frame of point cloud data is analyzed to extract the 3D spatial coordinates of each spatial point. These coordinates consist of three components, representing the position in three directions within the spatial coordinate system. Simultaneously, the depth information of each spatial point, representing its distance from the imaging radar device, is extracted. Each frame of visual image data is analyzed to obtain the image's size information and determine the coordinates of each pixel within the image. These coordinates consist of horizontal and vertical components. The texture features of each pixel, including its color and grayscale information, are also extracted. Using the timestamp information recorded during data acquisition, point cloud data and image data at the same timestamp are associated to establish a temporal correspondence. For the associated point cloud and image data, based on the 3D coordinates of the spatial points and the imaging principle of the image, the approximate region corresponding to each spatial point in the image is determined. The spatial points are then bound to this image region, ensuring that each spatial point is associated with an image region at the same time point and spatial location. This ultimately forms temporally correlated data containing spatial points, pixel coordinates, texture features, and timestamps.

[0019] Step S122: Extract the depth information of the imaging radar point cloud data from the temporal correlation data, use the depth information as the truth benchmark, combine the scanning parameters of the imaging radar and the imaging parameters of the visual camera, determine the correspondence between spatial points in the imaging radar point cloud data and pixels in the visual image data, transform the correspondence into a specific execution process, set the operation steps for mapping each spatial point to the image plane, and form a spatial mapping process.

[0020] Depth information from imaging radar point cloud data is extracted from time-series correlation data. Since imaging radar directly measures the distance to the target, its depth information has high reliability and is therefore used as the ground truth. Scanning parameters of the imaging radar, including scan angle range, number of scan lines, and scan frequency, are collected, along with imaging parameters of the vision camera, including focal length, pixel size, and principal point position. A coordinate transformation model between the imaging radar and the vision camera is established based on their installation positions and attitude relationships. Based on this model, the transformation steps from the radar coordinate system to the camera coordinate system, and the projection steps from the camera coordinate system to the image pixel coordinate system, are determined. These steps are arranged sequentially to form a specific execution flow for mapping spatial points to the image plane. This flow details the order and method of coordinate transformation and projection calculations required for each spatial point during the mapping process, such as coordinate translation, coordinate rotation, and perspective projection, thus forming the spatial mapping process.

[0021] Step S123: Invoke the spatial mapping process to map each spatial point in the imaging radar point cloud data to the corresponding image plane of the visual image data one by one, calculate the pixel coordinates of each spatial point in the visual image data, and bind the depth information of each spatial point with the calculated pixel coordinates to form a spatial point pixel coordinate depth true value association unit.

[0022] The spatial mapping process is initiated, sequentially processing each spatial point in the imaging radar point cloud data. For each spatial point, its three-dimensional coordinates in the radar coordinate system are first obtained, and then transformed to the camera coordinate system according to the coordinate transformation steps specified in the spatial mapping process. In the camera coordinate system, based on the imaging parameters of the visual camera, the pixel coordinates of the spatial point on the image plane are calculated using perspective projection. These pixel coordinates consist of a horizontal pixel index and a vertical pixel index. The calculated pixel coordinates are then bound to the depth information of the spatial point, forming an association unit containing the spatial point identifier, pixel coordinates, and depth information. The above operation is performed on all spatial points, generating multiple such association units. Each association unit accurately records the position of a spatial point in the image and its corresponding ground truth depth value. These association units together constitute the spatial point pixel coordinate depth ground truth association unit.

[0023] Step S1231: Extract each spatial point from the imaging radar point cloud data. Each spatial point is extracted separately. Each spatial point contains three-dimensional spatial coordinates and depth information. The extracted spatial points are organized to form individual spatial point data.

[0024] The process iterates through all spatial points in the imaging radar point cloud data, extracting the 3D spatial coordinates and depth information for each point. Each point's 3D spatial coordinates consist of three numerical components, representing its position in different directions within 3D space. The depth information is a single numerical value representing the distance from the imaging radar. Each extracted spatial point is treated as an independent data unit and arranged according to its original order or spatial relationship within the point cloud data, forming a separate spatial point data set. This separate spatial point data set is an ordered collection where each element is a spatial point containing both 3D spatial coordinates and depth information.

[0025] Step S1232: Call each spatial point in the individual spatial point data, substitute its three-dimensional spatial coordinates into the spatial mapping process, and perform pixel coordinate calculation operations step by step according to the mapping logic and parameter settings set in the spatial mapping process. During the calculation process, combine the scanning parameters of the imaging radar and the imaging parameters of the vision camera to obtain the pixel coordinates of the spatial point in the corresponding image plane of the visual image data, and form spatial point pixel coordinate calculation data.

[0026] Each spatial point is selected sequentially from the individual spatial point data to obtain its three-dimensional spatial coordinates. These coordinates are then input into a spatial mapping process. Following the mapping logic defined in the process, and combining the scanning parameters of the imaging radar (such as scanning angle and scanning range) and the imaging parameters of the visual camera (such as focal length, pixel size, and principal point coordinates), coordinate transformation and projection calculations are performed. During the calculation, the three-dimensional coordinates in the radar coordinate system are first transformed to the camera coordinate system. Then, based on the camera imaging model, the coordinates in the camera coordinate system are projected onto the image plane to obtain the corresponding pixel coordinates. The calculated pixel coordinates of each spatial point are associated with the identifier of that spatial point to form spatial point pixel coordinate calculation data. This data records the pixel coordinate information corresponding to each spatial point.

[0027] Step S1233: Extract the pixel coordinates and corresponding spatial points from the spatial point pixel coordinate calculation data, integrate the pixel coordinates corresponding to each spatial point with the visual image data, locate the specific image frame and image region of the pixel coordinates in the visual image data, and bind the pixel coordinates, corresponding image frames and image regions with the spatial points to form spatial point pixel coordinate image association data.

[0028] Each spatial point and its corresponding pixel coordinates are extracted from the spatial point pixel coordinate calculation data. The image frame to which the spatial point belongs is determined based on the timestamp information in the temporal association data. For example, if the timestamp of a spatial point is T0, the image frame with timestamp T0 is searched as the corresponding image frame. In the determined image frame, the specific position of the pixel is located by the horizontal index u and the vertical index v of the pixel coordinates. For example, u=300 and v=450 means that the pixel is located in the 300th column and 450th row of the image frame. According to the texture distribution characteristics of the image frame, the image frame is divided into multiple image regions, such as the central region, the edge region, and the upper left corner region. The image region to which the pixel belongs is determined by the range of the region where the pixel coordinates are located. The pixel coordinates (u, v), the corresponding image frame identifier (such as Frame_001), the image region (such as the central region), and the spatial point are bound together to form an association record containing the spatial point ID, pixel coordinates, image frame identifier, and image region. After performing the above operations on all spatial points, all association records are summarized to form spatial point pixel coordinate image association data.

[0029] Step S12331: Extract each pixel coordinate and corresponding spatial point from the spatial point pixel coordinate calculation data. Extract the pixel coordinates corresponding to each spatial point separately. At the same time, extract the three-dimensional spatial coordinates and depth ground value of the spatial point and integrate them to form a temporary association unit.

[0030] The pixel coordinates of spatial points are calculated by traversing the data. For each spatial point, its pixel coordinates (u, v) are extracted individually, along with its 3D spatial coordinates (X, Y, Z) and depth ground truth D. This information is then integrated into a temporary data structure containing the spatial point ID, pixel coordinates (u, v), 3D coordinates (X, Y, Z), and depth ground truth D, forming a temporary associated unit. For example, the temporary associated unit for spatial point ID P001 contains (u=300, v=450), (X=10.2, Y=5.3, Z=8.7), and D=8.7.

[0031] Step S12332: Substitute the pixel coordinates in the temporary association unit into the coordinate system of the visual image data, combine the imaging parameters of the visual camera, determine the specific image frame of the pixel coordinates in the visual image data according to the acquisition time sequence, and bind the pixel coordinates, the corresponding image frame and the spatial point to form pixel coordinate image frame association data.

[0032] The pixel coordinates (u, v) in the temporary association unit are substituted into the coordinate system of the visual image data. This system has the top left corner of the image as the origin, the horizontal axis as the u-axis, and the vertical axis as the v-axis. Combining the imaging parameters of the visual camera (such as frame rate and exposure time) and the acquisition time series records, an image frame matching the spatial point timestamp is searched. For example, if the spatial point timestamp is T0, the image frame Frame_001 with the timestamp closest to T0 is found in the time series records of the visual image data, and the pixel coordinates are determined to belong to Frame_001. The pixel coordinates (u, v), the image frame identifier Frame_001, and the spatial point ID are bound to form pixel coordinate image frame association data. Each record contains the spatial point ID, (u, v), and Frame_001.

[0033] Step S12333: Call the pixel coordinate image frame association data, locate the specific position of the pixel coordinate in the determined image frame, clarify the horizontal and vertical coordinate positions of the pixel coordinate in the image frame, divide the image region according to the texture distribution of the image frame, determine the image region where the pixel coordinate is located, and bind the pixel coordinate, image frame, image region and spatial point to form pixel coordinate image region association data.

[0034] For each record in the pixel coordinate image frame association data, its specific position in the corresponding image frame (e.g., Frame_001) is located using pixel coordinates (u, v). For example, u=300 corresponds to the 300th column of the horizontal coordinate, and v=450 corresponds to the 450th row of the vertical coordinate. Based on the texture distribution characteristics of the image frame (e.g., gradient changes, color distribution), the image frame is divided into multiple image regions. For example, the image frame is divided into 9 regions using a 3×3 grid, and each region is (W / 3)×(H / 3) pixels in size. By determining which grid region the pixel coordinates (u, v) fall into, the image region to which it belongs is determined. For example, the grid in the 2nd row and 2nd column corresponds to the center region. The pixel coordinates (u, v), the image frame Frame_001, the "center region" of the image region, and the spatial point ID are bound together to form the pixel coordinate image region association data.

[0035] Step S12334: Extract the pixel coordinates and corresponding image regions from the pixel coordinate image region association data, extract the texture features of the image region, associate the texture features with the depth ground truth of the corresponding spatial point, and bind the texture features, pixel coordinates, image frames, image regions and spatial points to form pixel coordinate image region texture association data.

[0036] Pixel coordinates (u, v) and corresponding image regions (such as the center region) are extracted from the pixel coordinate image region association data. Texture features are then extracted from this image region, including the gray-level co-occurrence matrix, edge orientation histogram, and color mean. For example, the texture features of the center region can be represented as an energy value of 0.3, an entropy value of 0.8 in the gray-level co-occurrence matrix, a 0° orientation percentage of 30% in the edge orientation histogram, and a color mean (R=120, G=130, B=140). These texture features are then associated with the ground truth depth D of the corresponding spatial point, for example, D=8.7, and bound to the pixel coordinates (u, v), image frame Frame_001, the image region "center region," and the spatial point ID to form pixel coordinate image region texture association data.

[0037] Step S12335: Call the pixel coordinate image region texture association data and spatial mapping process, compare the pixel coordinate mapping result with the operation requirements of the spatial mapping process, compare the matching of texture features with the ground truth of depth, adjust the calculation result of pixel coordinates, and form pixel coordinate mapping adjustment data.

[0038] The process involves calling the pixel coordinate image region texture association data and spatial mapping procedure to check whether the calculation of pixel coordinates (u, v) conforms to the coordinate transformation and projection operation requirements in the spatial mapping procedure, such as whether the camera intrinsic and extrinsic parameters are correctly applied. Simultaneously, the matching between texture features and ground truth depth is analyzed, such as whether the ground truth depth in smooth texture regions is continuous, and whether there is a step in the ground truth depth in abrupt texture regions. If a pixel coordinate calculation deviation is found (e.g., exceeding image boundaries) or a mismatch between texture and depth (e.g., the ground truth depth does not change at texture boundaries), the pixel coordinates are recalculated according to the spatial mapping procedure. For example, the rotation matrix parameters in the camera extrinsic parameters are adjusted to form pixel coordinate mapping adjustment data, and the adjusted pixel coordinates (u', v') are recorded.

[0039] Step S12336: Call the pixel coordinate mapping adjustment data, repeat the above steps for the pixel coordinates corresponding to each spatial point, determine the image frame, image region and texture features corresponding to each pixel coordinate, bind the pixel coordinates, image frame, image region, texture features and depth ground truth corresponding to each spatial point to form complete associated data of spatial point pixel coordinates.

[0040] Based on the adjusted pixel coordinates (u', v') in the pixel coordinate mapping adjustment data, steps S12332 to S12335 are re-executed to determine the new image frame, image region, and texture features. For example, the adjusted pixel coordinates (u'=302, v'=448) may correspond to the center region of image frame Frame_001, and the texture features are updated to an energy value of 0.32 and an entropy value of 0.78. The adjusted pixel coordinates (u', v'), image frame, image region, texture features, and depth ground truth D are bound together to form complete spatial point pixel coordinate association data. Each record contains a spatial point ID, (u', v'), Frame_001, "center region", texture features, and D=8.7.

[0041] Step S12337: Summarize the complete association data of the spatial point pixel coordinates corresponding to all spatial points, integrate the mapping information of all spatial points, and form a complete association set of spatial point pixel coordinates.

[0042] Collect complete spatial point pixel coordinate association data for all spatial points, sort by spatial point ID, and form a set containing all spatial point mapping information. For example, the set contains association records from P001 to P1000, and each record contains pixel coordinates, image frame, image region, texture features, and depth ground truth, forming a complete spatial point pixel coordinate association set data.

[0043] Step S12338: Call the complete association set of spatial point pixel coordinates data, analyze the mapping distribution of all spatial points, check whether the mapping of spatial points covers the main image area of ​​the visual image data, check the uniformity of the mapping distribution, and adjust the mapping parameters and pixel coordinate calculation results of spatial points to make the mapping distribution of spatial points more uniform.

[0044] Statistical analysis is performed on the complete associated set of spatial point pixel coordinates to calculate the number of spatial point mappings in each image region. For example, if the central region has 500 points and the edge region has 100 points, it is determined whether the mapping covers the main regions (such as the central region and the surrounding areas). The uniformity is evaluated by calculating the mapping density (number of points / region area) of each region. If the density in the central region is 5 points / pixel and the density in the edge region is 1 point / pixel, the parameters in the spatial mapping process are adjusted (such as increasing the sampling weight of the edge region), and the pixel coordinates are recalculated to keep the difference in mapping density between regions within a set threshold (such as ±2 points / pixel).

[0045] Step S12339: Call the adjusted spatial point pixel coordinate complete association set data, integrate the adjusted mapping parameters and pixel coordinate calculation results to form spatial point mapping optimization data.

[0046] The adjusted mapping parameters (such as camera extrinsic parameters and sampling weights) in step S12338 and the recalculated pixel coordinates (u'', v'') are integrated to update the corresponding fields in the complete association set of spatial point pixel coordinates, forming spatial point mapping optimization data to ensure that the mapping distribution is uniform and covers the main image area.

[0047] Step S123310: Integrate the spatial point mapping optimization data with the complete spatial point pixel coordinate association set data, update the complete spatial point pixel coordinate association set data, and form spatial point pixel coordinate image association data.

[0048] The adjusted parameters and pixel coordinates in the spatial point mapping optimization data replace the original values ​​in the complete spatial point pixel coordinate association set data. The updated set data is the spatial point pixel coordinate image association data, which contains the optimized pixel coordinates of all spatial points, image frames, image regions, texture features, and depth ground truth association information.

[0049] Step S1234: Extract each spatial point from the spatial point pixel coordinate image association data, extract the depth information of the spatial point, and bind the depth information with the corresponding pixel coordinates, image frame and image region of the spatial point to form a spatial point pixel coordinate depth ground value association unit.

[0050] Each spatial point is obtained from the spatial point pixel coordinate image association data, and its depth information is extracted. This depth information is then bound to the corresponding pixel coordinates, the image frame to which the spatial point belongs, and the image region in which it is located, forming a complete association unit containing the spatial point, pixel coordinates, image frame, image region, and depth information, namely, the spatial point pixel coordinate depth ground truth association unit.

[0051] Step S1235: Summarize all the formed spatial point pixel coordinate depth ground truth association units, integrate the information of all association units, and form a set of spatial point pixel coordinate depth ground truth association units. Each association unit contains a spatial point, pixel coordinates, image frame, image region and depth ground truth, forming the association unit set data.

[0052] All individual spatial point pixel coordinate depth ground truth correlation units are collected together, and these correlation units are organized and sorted to form a set. Each element in this set is a correlation unit, containing various information about the spatial point, as well as corresponding image-related information and depth ground truth, thus forming a correlation unit set data.

[0053] Step S1236: Call the associated unit set data, analyze the association of all pixels in the visual image data, locate the pixels in the visual image data that are not mapped to spatial points, extract the unmapped pixels, combine them with the depth ground truth of the surrounding mapped pixels, delineate the depth reference range of the unmapped pixels, bind the unmapped pixels with the corresponding depth reference range, and form unmapped pixel depth reference data.

[0054] The associated unit set data is analyzed to determine whether each pixel in the visual image data has a corresponding spatial point mapping. For pixels not mapped to a spatial point, mapped pixels within a certain range are located, and their corresponding ground truth depth values ​​are obtained. Based on the distribution of the ground truth depth values ​​of the surrounding mapped pixels, such as by taking the minimum and maximum values ​​or by interpolation, a depth reference range for the unmapped pixels is determined. The coordinates of the unmapped pixels are then bound to this depth reference range to form unmapped pixel depth reference data.

[0055] Step S1237: Bind each unmapped pixel in the unmapped pixel depth reference data to its corresponding depth reference range to form a supplementary association unit. The supplementary association unit contains the coordinates of the unmapped pixel, the image frame, the image region, and the depth reference range. The structure is consistent with that of the spatial point pixel coordinate depth ground value association unit, forming supplementary association unit data.

[0056] Following the structure of the spatial point pixel coordinate depth ground truth association unit, a supplementary association unit is created for each unmapped pixel in the unmapped pixel depth reference data. The supplementary association unit contains the coordinates of the unmapped pixel, the image frame to which it belongs, the image region it is located in, and the depth reference range, making its structure consistent with that of the spatial point pixel coordinate depth ground truth association unit, thus forming supplementary association unit data.

[0057] Step S1238: Integrate the supplementary association unit data with the spatial point pixel coordinate depth ground value association unit set to form a spatial point pixel coordinate depth ground value association unit set. The association unit set contains the association units corresponding to all pixels in the visual image data, forming a complete association unit set data.

[0058] The supplementary association units in the supplementary association unit data are added to the spatial point pixel coordinate depth true value association unit set, so that the set contains the association units corresponding to all pixels in the visual image data, whether the pixels are mapped to spatial points or not, and thus form a complete association unit set data.

[0059] Step S1239: Integrate the complete set of associated units data with the visual image data so that each pixel in the visual image data can retrieve the depth information in its corresponding associated unit. Bind the integrated visual image data with the complete set of associated units data to form integrated visual image and associated unit data.

[0060] This involves integrating the complete set of associated units with the visual image data to establish a mapping relationship between each pixel in the visual image data and its corresponding associated unit in the complete set of associated units. This allows each pixel to quickly locate its corresponding associated unit and obtain its depth information when processing visual image data, resulting in integrated visual image and associated unit data.

[0061] Step S12310: Integrate the complete set of associated unit data with the imaging radar point cloud data, and bind the integrated complete set of associated unit data with the imaging radar point cloud data to form integrated data of associated units and radar point cloud.

[0062] The complete set of associated unit data is integrated with the imaging radar point cloud data to establish the correspondence between the associated units and the spatial points in the point cloud data. This ensures that each spatial point in the point cloud data can be associated with its corresponding associated unit, forming integrated data of associated units and radar point cloud.

[0063] Step S124: Extract the pixel coordinates in the spatial point pixel coordinate depth ground truth association unit, locate the neighboring pixels around the pixel coordinates in the visual image data, analyze the texture similarity of the neighboring pixels, divide the pixels with the same texture similarity into the same pixel group, bind each pixel group to the corresponding spatial point in the imaging radar point cloud data, assign the depth information of the spatial point to the corresponding pixel group, and form pixel group depth ground truth association data.

[0064] Pixel coordinates are extracted from the spatial point pixel coordinate depth ground truth association unit. For each pixel coordinate, its neighboring pixels are determined in the visual image data. The range of neighboring pixels can be set according to the actual situation, such as a 3x3 pixel area or a 5x5 pixel area. The texture features of these neighboring pixels are extracted, and the texture similarity between neighboring pixels is analyzed by calculating the similarity between texture features. Pixels with texture similarity reaching a set threshold are divided into the same pixel group, and each pixel group contains multiple pixels with similar textures. Each pixel group is bound to the corresponding spatial point in the imaging radar point cloud data, that is, the pixel group is composed of the pixels mapped from the spatial point and its similar texture pixels. The depth information of the spatial point is assigned to the corresponding pixel group, so that all pixels in the pixel group are associated with the depth information, forming pixel group depth ground truth association data.

[0065] Step S125: Extract the depth ground value corresponding to each pixel group from the pixel group depth ground value association data. Combine the texture distribution features of the pixels in the pixel group to define the depth range corresponding to all pixels in the pixel group. With the depth ground value corresponding to the pixel group as the core, adjust the upper and lower limits of the depth range in combination with the texture distribution features, and bind the defined depth range to the pixel group to form pixel group depth range association data.

[0066] The depth ground truth value for each pixel group is obtained from the pixel group depth ground truth correlation data. The texture distribution characteristics of the pixels within that pixel group are analyzed, such as texture density, orientation, and rate of change. Based on the depth ground truth value and texture distribution characteristics, the depth range of all pixels within the pixel group is defined. The initial depth range can be set as a certain interval centered on the depth ground truth value. Then, the upper and lower limits of the depth range are adjusted according to the texture distribution characteristics. For example, when the texture distribution is relatively uniform, the depth range can be appropriately reduced; when the texture changes drastically, the depth range can be appropriately expanded. The adjusted depth range is then bound to the pixel group to form pixel group depth range correlation data.

[0067] Step S126: Extract the pixel group and corresponding depth range from the pixel group depth range association data, bind each pixel in each pixel group to the depth range corresponding to that pixel group, and integrate the bound pixels and depth ranges to form pixel depth range binding data.

[0068] Pixel groups and their corresponding depth ranges are extracted from the pixel group depth range association data. All pixels within each pixel group are traversed, and each pixel is bound to the depth range of that pixel group. All bound pixels and depth ranges are then integrated to form pixel depth range binding data, which records the depth range information corresponding to each pixel.

[0069] Step S127: Call the pixel depth range binding data, integrate the depth range binding information of all pixels, and form a depth estimation constraint process for visual image data. This depth estimation constraint process takes the true depth value of imaging radar point cloud data as the core and the texture features of pixels as an aid, and sets the specific operation standards and range limits for depth estimation of each pixel.

[0070] Based on pixel depth range binding data, this paper integrates the depth range information of all pixels to construct a depth estimation constraint process for visual image data. This process uses the ground truth depth value of imaging radar point cloud data as the core reference to ensure the accuracy of depth estimation; simultaneously, it incorporates pixel texture features as auxiliary information to adjust the depth estimation. The process sets specific operational standards for depth estimation for each pixel, such as the depth estimate value must be within the bound depth range, and range limitations, such as how to adjust the depth range when texture features change significantly.

[0071] Step S128: Integrate the depth estimation constraint process with the visual image data, so that the depth estimation operation of the visual image data is strictly executed according to the depth estimation constraint process, and limit the depth estimation value of each pixel to the depth range bound to that pixel, thus forming the depth-constrained visual image data.

[0072] The depth estimation constraint process is applied to the depth estimation of visual image data. During depth estimation, the operational standards and range limitations set in the process are strictly followed to ensure that the depth estimate of each pixel does not exceed its bound depth range. After this processing, depth-constrained visual image data is generated.

[0073] Step S129: Call the visual image data after depth estimation constraints, perform preliminary depth estimation operation according to the depth estimation constraint process, generate a preliminary depth map of the visual image data, each pixel value in the preliminary depth map corresponds to the depth estimate value of the pixel, and the depth estimate value is within the depth range bound to the pixel, thus forming preliminary depth map data.

[0074] Using the depth-estimated constrained visual image data, preliminary depth estimation is performed according to the steps in the depth estimation constraint process. By analyzing and calculating the texture features and color information in the image, a depth estimate is generated for each pixel, which must be within the depth range bound to that pixel. The depth estimates of all pixels are combined to form a preliminary depth map of the visual image data, i.e., preliminary depth map data.

[0075] Step S1210: Integrate the preliminary depth map data with the imaging radar point cloud data, and associate the integrated preliminary depth map data with the imaging radar point cloud data to form preliminary depth map and radar point cloud associated data.

[0076] The depth information in the preliminary depth map data is integrated with the spatial point depth information in the imaging radar point cloud data to establish a correlation between the two. For example, the depth estimate of a pixel in the preliminary depth map is compared and correlated with the ground truth depth value of the corresponding spatial point to form preliminary depth map-radar point cloud correlation data. This preliminary depth map-radar point cloud correlation data integrates the depth estimation information of the image and the ground truth depth information of the point cloud.

[0077] Step S130: Based on pixel-level depth correlation, extract texture detail information corresponding to sparse areas of imaging radar point cloud data from visual image data. Combine the spatial distribution correlation of spatial points around sparse areas in imaging radar point cloud data, convert the extracted texture detail information into spatial point cloud information, fill the sparse areas in imaging radar point cloud data with the converted spatial point cloud information, and integrate the original spatial points and the completed spatial points to form intermediate dense point cloud data.

[0078] Based on pixel-level depth correlation, the sparse regions in the imaging radar point cloud data are identified as corresponding image regions in the visual image data. Texture analysis is then performed on these image regions to extract detailed texture information, such as texture direction, density, and shape. Simultaneously, the spatial distribution of points surrounding the sparse regions is analyzed, including the spatial distribution correlations such as point position, spacing, and arrangement. Using these spatial distribution correlations as constraints, the extracted texture details are converted into spatial point cloud information, i.e., corresponding spatial points are generated based on the texture features. The generated spatial point cloud information is then used to fill in the sparse regions of the point cloud data and integrated with the original spatial points to form intermediate dense point cloud data, which has a higher density than the original point cloud data.

[0079] Step S131: Call the pixel-level depth correlation relationship to analyze the distribution of spatial points in the imaging radar point cloud data, locate the sparse regions in the imaging radar point cloud data, extract the spatial location information of the sparse regions, and form sparse region spatial location data.

[0080] Pixel-level depth correlation is used to analyze the spatial point distribution in imaging radar point cloud data. Sparse regions in the point cloud data are identified by calculating the density of spatial points, such as counting the number of spatial points per unit volume. When the spatial point density of a certain region is lower than a set threshold, that region is defined as a sparse region. Spatial location information of the sparse regions is extracted, including the region's boundary coordinates and center coordinates, forming sparse region spatial location data.

[0081] Step S132: Call the spatial location data of the sparse region, locate the image region corresponding to the sparse region in the visual image data through pixel-level depth correlation, extract the depth information in the preliminary depth map data corresponding to the image region, and integrate the image region with the corresponding depth information to form the image and depth data corresponding to the sparse region.

[0082] Based on the spatial location information in the sparse region spatial location data, the corresponding image region is found in the visual image data through pixel-level depth correlation. The depth information of this image region is extracted from the preliminary depth map data, and the texture information of the image region is integrated with the corresponding depth information to form the image and depth data corresponding to the sparse region.

[0083] Step S133: Extract the image region corresponding to the sparse region from the image and depth data, and extract the texture direction, texture density, texture color transition and texture boundary features within the image region. During the extraction process, follow the pixel-level depth correlation relationship, and match each extracted texture detail information with the spatial location of the sparse region in the imaging radar point cloud data to form target corresponding data.

[0084] Image regions are extracted from sparse regions corresponding to image and depth data, and texture features are extracted from these regions. Specifically, this includes extracting texture direction (the direction of texture extension); texture density (the number of texture elements per unit area); texture color transition (the variation in texture color); and texture boundary features (the shape and position of the boundaries between texture regions). During the extraction process, based on pixel-level depth correlation, it is ensured that each extracted texture detail corresponds to a specific spatial location in the sparse region of the imaging radar point cloud data, forming target-correspondence data. This target-correspondence data establishes a correspondence between texture detail information and spatial location.

[0085] Step S134: Analyze the texture detail information in the data corresponding to the target, extract the continuous distribution relationship of the texture detail information in space, divide the continuous texture region according to the continuous distribution relationship, and associate each continuous texture region with the corresponding spatial location to form the spatial location data of the continuous texture region.

[0086] The texture details in the target data are analyzed to study the spatial continuity of texture direction, density, and color transitions, determining the correlation of continuous distribution of texture details. Based on these correlations, the image region is divided into multiple texture-continuous regions, each with good continuity of texture features. Each texture-continuous region is then associated with its corresponding spatial location, forming spatial location data of the texture-continuous regions.

[0087] Step S135: Extract each texture continuous region from the spatial location data of texture continuous regions, associate each texture continuous region with the surrounding spatial points of sparse regions in the imaging radar point cloud data, extract the spatial location, spacing and depth variation correlation of the surrounding spatial points, and form the surrounding spatial point distribution correlation data.

[0088] Each continuous texture region is extracted from the spatial location data of the continuous texture region. Spatial points surrounding sparse regions in the imaging radar point cloud data are found, and the continuous texture regions are associated with these surrounding spatial points. Spatial distribution correlations, such as the spatial coordinates of the surrounding spatial points, the spacing between spatial points, and the changes in depth values, are extracted to form the surrounding spatial point distribution correlation data.

[0089] Step S136: Call the surrounding spatial point distribution association data and texture continuous region spatial location data, combine the texture continuous distribution association relationship of each texture continuous region and the depth information in the preliminary depth map data, determine the spatial point generation operation process corresponding to each texture continuous region. This spatial point generation operation process sets the specific determination method of spatial point generation density, spatial location and depth information, forming a spatial point generation process.

[0090] By integrating surrounding spatial point distribution correlation data and texture continuous region spatial location data, and considering the texture continuity distribution correlation of each texture continuous region as well as the depth information in the preliminary depth map data, a specific operational procedure for spatial point generation is determined. This procedure specifies how the generation density of spatial points is determined based on texture density and the spacing between surrounding points, how the spatial location is determined based on texture direction and the location of surrounding points, and how the depth information is determined based on the preliminary depth map and the depth changes of surrounding points, thus forming a spatial point generation process.

[0091] Step S1361: Extract each surrounding spatial point from the surrounding spatial point distribution correlation data, analyze the spatial location, depth information and spatial spacing of each surrounding spatial point, organize the correlation relationship of the spacing, direction and depth changes of the surrounding spatial points, and form detailed data of the surrounding spatial point distribution.

[0092] Each surrounding spatial point in the associated spatial point distribution data is analyzed to obtain its spatial coordinates and depth information, and the distance between adjacent spatial points is calculated. By organizing the above data, the correlations such as the distribution pattern of the distance between surrounding spatial points, the overall orientation, and the changing trend of depth values ​​are determined, forming detailed data on the distribution of surrounding spatial points.

[0093] Step S1362: Call the detailed data of the distribution of surrounding spatial points to determine the basic generation density of the spatial points. The basic generation density is based on the spatial spacing of the surrounding spatial points and is adjusted in combination with the texture density of the texture continuous region. For regions with high texture density, the value of the basic generation density is increased, and for regions with low texture density, the value of the basic generation density is decreased. The adjusted basic generation density is bound to the texture continuous region to form basic generation density binding data.

[0094] Based on the spatial spacing in the detailed data of surrounding spatial point distribution, the basic generation density of spatial points is determined; for example, the smaller the spatial spacing, the higher the basic generation density. Then, the basic generation density is adjusted in conjunction with the texture density of the texture contiguous region. When the texture density is high, the basic generation density is appropriately increased; when the texture density is low, the basic generation density is appropriately decreased. The adjusted basic generation density is then bound to the corresponding texture contiguous region to form basic generation density binding data.

[0095] Step S1363: Call the texture direction feature in the texture continuous distribution association data, bind the texture direction feature with the spatial point generation direction, set the spatial point generation direction to be consistent with the texture direction in the region where the texture direction is continuous, set the spatial point generation direction to change synchronously with the texture direction in the region where the texture direction changes, bind the generation direction with the texture continuous region to form generation direction binding data.

[0096] Texture orientation features are extracted from continuously distributed texture data, and these features are correlated with the generation direction of spatial points. In regions where the texture orientation is continuous, the generation direction of spatial points is set to be the same as the texture orientation; in regions where the texture orientation changes, the generation direction of spatial points is adjusted synchronously with the change in texture orientation. The determined generation orientation is bound to the continuous texture region to form generation orientation binding data.

[0097] Step S1364: Call the preliminary depth map data, extract the depth information corresponding to each continuous texture region, combine the depth ground truth in the surrounding spatial point distribution detail data, organize the depth change correlation relationship corresponding to each continuous texture region, and form depth change correlation data.

[0098] Depth information corresponding to each continuous texture region is obtained from the initial depth map data. At the same time, the depth ground truth is combined with the depth values ​​in the surrounding spatial point distribution details data. The changes in depth values ​​within the continuous texture region are analyzed, such as whether the depth values ​​gradually increase or decrease, and the rate of change. The correlation of depth changes is then organized to form depth change correlation data.

[0099] Step S1365: Call the depth change association data to determine the depth determination method of spatial points at different positions in each continuous texture area, and bind the depth determination method to the continuous texture area to form depth determination method binding data.

[0100] Based on depth variation correlation data, the calculation method for spatial point depth at different locations within a continuous texture region is determined. For example, in regions with linearly varying depth, the depth of spatial points can be determined using linear interpolation; in regions with non-linearly varying depth, methods such as curve fitting can be used to determine the depth. These depth determination methods are then bound to the corresponding continuous texture regions to form depth determination method binding data.

[0101] Step S1366: Call the texture boundary features in the texture continuous distribution association data, transform the texture boundary features into the generation range boundary of spatial points, the generation range boundary fits the boundary of the texture continuous region, and at the same time connects with the distribution range of surrounding spatial points, bind the generation range boundary to the texture continuous region, and form generation range boundary binding data.

[0102] Texture boundary features are extracted from the texture continuous distribution association data and transformed into the boundaries of the spatial point generation range. The generation range boundary should closely fit the boundary of the texture continuous region and naturally connect with the distribution range of surrounding spatial points, avoiding gaps or overlaps. The generation range boundary is then bound to the texture continuous region to form generation range boundary binding data.

[0103] For example, step S1366-1: extract the texture boundary features of each texture continuous region, extract the texture boundary line between the texture continuous region and other regions, extract the boundary contour and boundary range information of the texture continuous region, organize the texture boundary lines, and form texture continuous region boundary line data.

[0104] Each continuous texture region is analyzed to identify its texture boundaries with other regions. These boundaries are where texture features change significantly. The coordinate information of the boundary contour shape and boundary range of the continuous texture regions is extracted, and the texture boundaries are organized to form the boundary line data of the continuous texture regions.

[0105] Step S1366-2: Call the boundary line data of the continuous texture region, outline the boundary contour of each continuous texture region according to the texture boundary line, take the texture change position in the texture boundary line as the node of the boundary contour, connect all the nodes in sequence to form the boundary contour of the continuous texture region, and organize the boundary contour to form the boundary contour data of the continuous texture region.

[0106] Using texture boundaries in the continuous texture region boundary data, and with texture change locations on the boundaries as nodes, these nodes are connected sequentially to form the boundary contour of the continuous texture region. The coordinate information of this boundary contour is then processed to form the boundary contour data of the continuous texture region.

[0107] Step S1366-3: Map the boundary contour data of the continuous texture region to a spatial coordinate system. Combine the pixel-level depth correlation and the depth information in the preliminary depth map data to calculate the spatial coordinates corresponding to the boundary contour of the continuous texture region. Determine the spatial boundary range corresponding to the boundary contour of the continuous texture region. Organize the spatial boundary range to form the spatial boundary data of the continuous texture region.

[0108] The boundary contour data of the continuous texture region is mapped from the image coordinate system to the spatial coordinate system. In this process, the coordinates of each node on the boundary contour in the spatial coordinate system are calculated by combining pixel-level depth correlation and depth information from the preliminary depth map data. Based on these spatial coordinates, the spatial boundary range of the continuous texture region is determined, and the resulting data forms the spatial boundary data of the continuous texture region.

[0109] Step S1366-4: Extract the spatial location information of the surrounding spatial points of the sparse region, organize the spatial distribution range of the surrounding spatial points, calculate the spatial coordinate extreme values ​​of the surrounding spatial points, determine the distribution boundary of the surrounding spatial points, organize the distribution range and distribution boundary of the surrounding spatial points, and form the distribution range data of the surrounding spatial points.

[0110] Obtain the spatial coordinates of points surrounding the sparse region. By statistically analyzing the maximum and minimum values ​​of these coordinates, determine the spatial distribution range and boundaries of these points. After organizing this information, generate data on the distribution range of the surrounding points.

[0111] Step S1366-5: Call the spatial boundary data of the continuous texture region and the distribution range data of the surrounding spatial points, analyze the positional relationship between the surrounding spatial points and the spatial boundary range of the continuous texture region, determine the connection position between the surrounding spatial points and the continuous texture region, calculate the spatial coordinates of the connection position, and form the connection data between the continuous texture region and the surrounding points.

[0112] By comparing the spatial boundary data of the continuous texture region with the distribution range data of surrounding spatial points, the relative positions of the surrounding spatial points and the spatial boundary of the continuous texture region are analyzed. The connection points between the two are identified, that is, the intersection points of the distribution range of surrounding spatial points and the spatial boundary range of the continuous texture region. The spatial coordinates of these connection points are calculated to form the connection data between the continuous texture region and the surrounding points.

[0113] Step S1366-6: Call the data connecting the continuous texture region and surrounding points. Based on the spatial boundary range of the continuous texture region, extend the boundary in the direction of the distribution range of the surrounding spatial points to determine the preliminary boundary of the generation range of the spatial points. Organize the spatial coordinates of the preliminary boundary to form the preliminary boundary data of the generation range.

[0114] Starting with the spatial boundary of the continuous texture region, and based on the connection positions between the continuous texture region and surrounding points, the boundary is extended towards the distribution range of surrounding spatial points to determine the preliminary boundary of the spatial point generation range. The spatial coordinates of the preliminary boundary are then organized to form the preliminary boundary data of the generation range.

[0115] Step S1366-7: Call the preliminary boundary data of the generated range, check whether there are existing original spatial points of imaging radar point cloud data within the preliminary boundary, extract the coordinates of the original spatial points within the preliminary boundary, adjust the spatial coordinates of the preliminary boundary to avoid existing original spatial points, and form the preliminary boundary adjustment data of the generated range.

[0116] Check whether there are original spatial points from the imaging radar point cloud data within the boundary determined by the preliminary boundary data of the generated range. If so, extract the coordinates of these original spatial points and adjust the spatial coordinates of the preliminary boundary to make the boundary of the generated range avoid these original spatial points, thus preventing the newly generated spatial points from overlapping with the original spatial points, forming the preliminary boundary adjustment data of the generated range.

[0117] Step S1366-8: Invoke the preliminary boundary adjustment data of the generated range and the texture density distribution features of the texture continuous region, adjust the details of the preliminary boundary, and form the boundary adjustment data of the generated range.

[0118] By combining the initial boundary adjustment data of the generated range with the texture density distribution characteristics of the texture continuous region, the details of the initial boundary are adjusted. For example, in areas with high texture density, the boundary range can be appropriately reduced to ensure the generation density of spatial points; in areas with low texture density, the boundary range can be appropriately expanded. After adjustment, the generated range boundary adjustment data is formed.

[0119] Step S1366-9: Call the generation range boundary adjustment data to determine the generation range of spatial points, so that the boundary of the generation range matches the spatial boundary range of the continuous texture area, and at the same time forms a seamless connection with the distribution range of surrounding spatial points. Organize the spatial coordinates of the generation range to form spatial point generation range data.

[0120] Based on the data adjustment according to the generated range boundary, the final generated range of spatial points is determined. The boundary of this generated range must match the spatial boundary range of the continuous texture region and seamlessly connect with the distribution range of surrounding spatial points to ensure that the generated spatial points can be naturally integrated into the original point cloud data. After organizing the spatial coordinates of the generated range, the spatial point generated range data is formed.

[0121] Step S1366-10: Integrate the spatial point generation range data with the texture features of the texture continuous region, organize the integrated generation range data, and form the final generation range data of the texture continuous region.

[0122] The spatial point generation range data is integrated with the texture features of the continuous texture region, such as texture direction and density, so that the generation range can better reflect the texture features. The integrated generation range data is then organized to form the final generation range data of the continuous texture region.

[0123] Step S1367: Integrate basic generation density binding data, generation direction binding data, depth determination method binding data, and generation range boundary binding data to construct the specific steps of the spatial point generation operation process corresponding to each continuous texture region, set the order and specific operation requirements of spatial point cloud generation, and form the basic steps of the spatial point generation process.

[0124] This process integrates basic generation density binding data, generation direction binding data, depth determination method binding data, and generation range boundary binding data. Following the logical order of spatial point generation, it constructs the specific steps of the spatial point generation operation flow for each continuous texture region. The order of spatial point cloud generation is determined, for example, first determining the generation range, then the generation density, and then generating the position and depth of spatial points. The specific operational requirements for each step are clearly defined, forming the basic steps of the spatial point generation process.

[0125] Step S1368: Integrate the texture density and texture color transition features in the texture continuous distribution association data into the basic steps of the spatial point generation process, improve the spatial point generation density adjustment operation, and add the improved generation density adjustment operation to the basic steps of the spatial point generation process to form the improved steps of the spatial point generation process.

[0126] Texture density and texture color transition features are extracted from continuous texture distribution correlation data. These features are then integrated into the basic steps of the spatial point generation process to refine the spatial point generation density adjustment operation. For example, the rate of change of generation density is adjusted according to the severity of texture color transition. This refined generation density adjustment operation is added to the basic steps of the spatial point generation process, forming a refined step in the spatial point generation process.

[0127] Step S1369: Integrate the depth adjustment requirements in the depth change correlation data into the spatial point generation process improvement steps, and combine the texture color transition features to fine-tune the derived spatial point depth information to form a complete spatial point generation process.

[0128] The depth adjustment requirements from the depth change correlation data, such as the trend and magnitude of depth changes, are integrated into the refinement steps of the spatial point generation process. Simultaneously, by combining texture and color transition features, the spatial point depth information obtained based on the depth determination method is fine-tuned to better match the depth information with the texture and color transition, resulting in a more complete spatial point generation process.

[0129] Step S13610: Bind the improved spatial point generation process to each continuous texture region, set the spatial point generation process corresponding to each continuous texture region, and summarize the spatial point generation processes corresponding to all continuous texture regions to form a set of spatial point generation processes.

[0130] The refined spatial point generation process is bound one-to-one with each continuous texture region, and a dedicated spatial point generation process is set for each continuous texture region. All spatial point generation processes corresponding to all continuous texture regions are collected to form a spatial point generation process set, so that corresponding spatial point generation operations can be performed on different continuous texture regions in the future.

[0131] Step S137: Call the spatial point generation process to convert the texture detail information of each continuous texture region into spatial point cloud information. Determine the generation density of spatial points based on the texture density in the texture detail information. Determine the spatial orientation of spatial points based on the texture orientation. Determine the depth position of spatial points based on the depth information in the preliminary depth map data. Determine the distribution range of spatial points based on the texture boundary features to form the converted spatial point cloud information.

[0132] Following the spatial point generation process, the texture detail information of each continuous texture region is processed. The number and density of generated spatial points are determined based on the texture density; a higher texture density results in a greater number and density of generated spatial points. The arrangement direction of the spatial points in space is determined based on the texture direction. The depth position of each spatial point is determined based on the depth information in the preliminary depth map data. The distribution range of the spatial points is determined based on the texture boundary features, ensuring that the spatial point cloud information can accurately reflect the texture detail features, thus forming the transformed spatial point cloud information.

[0133] Step S138: Integrate the transformed spatial point cloud information with the surrounding spatial points of sparse areas in the imaging radar point cloud data, adjust the position of the spatial points in the transformed spatial point cloud information, and associate the integrated spatial point cloud information with the surrounding points to form a complete point cloud and surrounding point association data.

[0134] The transformed spatial point cloud information is integrated with the spatial points surrounding the sparse region. By adjusting the positions of the transformed spatial points, the newly generated spatial points are made to maintain spatial coordination and coherence with the surrounding spatial points. The integrated spatial point cloud information is then correlated with the surrounding points to form complete point cloud and surrounding point correlation data.

[0135] Step S139: Call the data associated with the completed point cloud and surrounding points, summarize the spatial point cloud information generated by converting each continuous texture region, fill the corresponding sparse region in the imaging radar point cloud data with the summarized spatial point cloud information, and initially integrate the filled point cloud data with the original imaging radar point cloud data to form preliminary completed point cloud data.

[0136] Based on the correlation data between the completed point cloud and surrounding points, the spatial point cloud information generated for each continuous texture region is summarized. The summarized spatial point cloud information is then used to fill in the corresponding sparse regions in the imaging radar point cloud data. Finally, the filled point cloud data is initially integrated with the original imaging radar point cloud data to form preliminary completed point cloud data.

[0137] Step S1310: Call the preliminary point cloud data, adjust the spatial distribution of all spatial points, and integrate the adjusted point cloud data to form intermediate dense point cloud data.

[0138] The spatial distribution of all spatial points in the initially completed point cloud data is adjusted, such as optimizing the position of the spatial points to make the overall distribution of the point cloud more uniform and reasonable. The adjusted point cloud data is then integrated to form intermediate dense point cloud data, which has a significantly higher point cloud density than the original point cloud data.

[0139] Step S140: Map the texture and color information in the visual image data pixel by pixel to the corresponding spatial points in the intermediate dense point cloud data, establish a one-to-one correspondence between spatial points and texture and color information, retrieve the ground truth depth value in the imaging radar point cloud data, adjust the depth position of each spatial point in the intermediate dense point cloud data, integrate the adjusted spatial points, depth information and texture and color information, and generate dense point cloud data with depth and texture coordination.

[0140] For each spatial point in the intermediate dense point cloud data, its corresponding pixel coordinates in the visual image data are determined based on pixel-level depth correlation. The texture color information (RGB color value) corresponding to these pixel coordinates is then mapped to the spatial point, establishing a one-to-one correspondence between the spatial point and the texture color information. Next, the ground truth depth values ​​of the original spatial points in the imaging radar point cloud data are retrieved. Using these ground truth values ​​as a benchmark, the depth positions of all spatial points in the intermediate dense point cloud data are adjusted to make the depth information more accurate. Finally, the adjusted spatial points, their corresponding depth information, and texture color information are integrated to generate dense point cloud data with coordinated depth and texture information. Each spatial point in this dense point cloud data possesses accurate depth information and corresponding texture color information.

[0141] Step S141: Call the pixel-level depth association relationship to determine the pixel coordinates in the visual image data corresponding to each spatial point in the intermediate dense point cloud data, and bind each spatial point to the corresponding pixel coordinates to form spatial point pixel coordinate binding data.

[0142] By leveraging pixel-level depth correlation, the corresponding pixel coordinates in the visual image data are found for each spatial point in the intermediate dense point cloud data. The identifier of the spatial point is bound to the corresponding pixel coordinates to form spatial point pixel coordinate binding data, which records the correspondence between spatial points and pixel coordinates.

[0143] Step S142: Call the spatial point pixel coordinate binding data, extract the pixel coordinates corresponding to each spatial point, and map the texture color information of each pixel in the visual image data to the corresponding spatial point in the intermediate dense point cloud data pixel by pixel. During the mapping process, follow the pixel-level depth association relationship, bind the mapped spatial points with the texture color information, and form spatial point texture color binding data.

[0144] Based on the spatial point pixel coordinate binding data, the pixel coordinates corresponding to each spatial point are extracted, and the texture color information of that pixel coordinate is obtained from the visual image data. The texture color information is then mapped pixel-by-pixel to the corresponding spatial point in the intermediate dense point cloud data, strictly adhering to pixel-level depth correlation during the mapping process to ensure accuracy. The mapped spatial points are then bound to the texture color information to form spatial point texture color binding data.

[0145] Step S143: Integrate the spatial point texture color binding data with the pixel-level depth association to link spatial points, texture color information and depth information. Organize the integrated association data to form spatial point texture depth association data, with each spatial point bound to unique texture color information and corresponding depth information.

[0146] By integrating spatial point texture color binding data with pixel-level depth association, spatial points are associated not only with texture color information but also with depth information. After processing, spatial point texture depth association data is formed, in which each spatial point is bound to unique texture color information and corresponding depth information.

[0147] Step S144: Extract the ground truth depth value from the imaging radar point cloud data, associate the ground truth depth value with the spatial points in the spatial point texture depth association data, directly bind the spatial points in the intermediate dense point cloud data that correspond to the original spatial points in the imaging radar point cloud data with the ground truth depth value of the imaging radar point cloud data, and bind the spatial points generated by the completion in the intermediate dense point cloud data with the depth reference value based on the ground truth depth value of the imaging radar point cloud data to form spatial point depth truth value binding data.

[0148] The ground truth depth values ​​of original spatial points are extracted from the imaging radar point cloud data, and these ground truth depth values ​​are then associated with spatial points in the spatial point texture depth association data. For spatial points in the intermediate dense point cloud data that correspond to the original spatial points, the ground truth depth values ​​of the imaging radar are directly bound to those spatial points. For spatial points generated through completion, depth reference values ​​are obtained based on the ground truth depth values ​​of the surrounding original spatial points through interpolation or other calculation methods, and these reference values ​​are bound to the completed spatial points to form spatial point depth ground truth bound data.

[0149] Step S145: Call the spatial point depth ground truth binding data, preliminary depth map data and spatial point texture color binding data, adjust the depth position of each spatial point in the intermediate dense point cloud data, refer to the distribution characteristics of the texture color information corresponding to the spatial point during the adjustment process, and adjust the spatial point depth position to make it change continuously in the area where the texture color information changes continuously, and bind the adjusted spatial point depth position with the texture color information to form spatial point depth texture adjustment data.

[0150] By integrating ground truth depth data, preliminary depth map data, and spatial point texture and color binding data, the depth position of each spatial point in the intermediate dense point cloud data is adjusted. During the adjustment process, the distribution characteristics of the texture and color information corresponding to the spatial points are analyzed. When the texture and color information changes continuously in a certain area, the depth position of the spatial points in that area is adjusted accordingly to also exhibit a continuous changing trend. The adjusted spatial point depth positions are then bound to the texture and color information to form spatial point depth texture adjustment data.

[0151] Step S1451: Extract all ground truth depth values ​​from the imaging radar point cloud data, associate the ground truth depth values ​​with the spatial points in the spatial point depth value binding data, organize the reference ground truth depth values ​​corresponding to each spatial point, and form spatial point reference depth value data.

[0152] Collect all ground truth depth values ​​from the imaging radar point cloud data, associate these ground truth depth values ​​with the spatial points in the spatial point depth value binding data, determine the reference ground truth depth value corresponding to each spatial point, and form spatial point reference depth value data.

[0153] Step S1452: Call the spatial point texture color binding data, group all spatial points in the intermediate dense point cloud data according to their corresponding texture color information, divide spatial points with similar texture color information into the same group, extract the depth information from the preliminary depth map data corresponding to each group, bind each group of spatial points with the corresponding depth information, and form spatial point group depth association data.

[0154] Based on the texture and color information in the spatial point texture and color binding data, the spatial points in the intermediate dense point cloud data are grouped. Spatial points with similar texture and color information are grouped together; for example, by calculating color similarity, spatial points with similarity higher than a set threshold are grouped together. The depth information corresponding to each group in the preliminary depth map data is extracted, and each group of spatial points is bound to this depth information to form spatial point group depth association data.

[0155] Step S1453: Call each spatial point group in the spatial point group depth association data, analyze the distribution characteristics of the texture color information corresponding to the spatial points in the group, analyze the continuous change of texture color information, divide the region of continuous texture color change and the region of abrupt texture color change, and form spatial point group texture change region data.

[0156] Each spatial point group in the deep correlation data is analyzed to study the distribution characteristics of texture color information corresponding to spatial points within the group, and to determine whether the texture color information changes continuously or abruptly. Based on the analysis results, regions with continuous texture color changes and regions with abrupt texture color changes are identified, forming spatial point group texture change region data.

[0157] Step S1454: Call the spatial point group texture change area data and spatial point reference depth true value data to determine the depth adjustment benchmark for each spatial point group. The depth adjustment benchmark is based on the depth true value of the imaging radar point cloud data and is corrected by combining the depth information in the preliminary depth map data. The corrected depth adjustment benchmark is bound to each spatial point group to form spatial point group depth adjustment benchmark data.

[0158] By combining spatial point group texture variation region data and spatial point reference depth ground truth data, a depth adjustment benchmark is determined for each spatial point group. The depth ground truth of the imaging radar point cloud data is used as the core reference, while also incorporating depth information from the preliminary depth map data for correction. For example, if there is a difference between the depth information in the preliminary depth map data and the depth ground truth, appropriate adjustments are made based on the magnitude of the difference. The corrected depth adjustment benchmark is then bound to each spatial point group to form spatial point group depth adjustment benchmark data.

[0159] Step S1455: Call the spatial point group depth adjustment reference data and spatial point group texture change area data. For spatial points in areas where texture color changes continuously, adjust the depth position of each spatial point according to the depth adjustment reference and the continuous change trend of texture color. Bind the adjusted spatial point depth position with texture color information to form continuous area spatial point adjustment data.

[0160] For spatial points within a region where texture color changes continuously, the depth position of each spatial point is adjusted based on the depth adjustment reference data in the spatial point group depth adjustment reference data, combined with the continuous change trend of texture color. For example, when the texture color changes continuously from light to dark, the depth position is also adjusted continuously from near to far. The adjusted spatial point depth position is then bound to the texture color information to form continuous region spatial point adjustment data.

[0161] Step S1456: Call the spatial point group depth adjustment reference data and spatial point group texture change area data. For spatial points in the area of ​​texture color mutation, adjust the depth position of each spatial point according to the depth adjustment reference and the characteristics of texture color mutation. Bind the adjusted spatial point depth position with texture color information to form spatial point adjustment data for mutation area.

[0162] For spatial points within regions of abrupt texture color changes, the depth position of each spatial point is adjusted based on the depth adjustment reference in the spatial point group depth adjustment reference data and the characteristics of the texture color change. For example, at the boundary where a texture color change occurs, the depth position of the spatial point may also change accordingly, and the depth value is adjusted based on the degree and direction of the change. The adjusted spatial point depth position is then bound to the texture color information to form spatial point adjustment data for the abrupt change region.

[0163] Step S1457: Integrate the spatial point adjustment data of continuous region with the spatial point adjustment data of abrupt region, and perform depth adjustment on all spatial points in each spatial point group one by one. During the adjustment process, refer to the depth position of the surrounding spatial points and adjust the depth position of each spatial point to keep it continuous with the depth position of the surrounding spatial points, thus forming the spatial point group depth adjustment completion data.

[0164] The data for adjusting spatial points in continuous and abrupt regions are integrated together, and each spatial point within each group is processed individually. When adjusting the depth position of each spatial point, the depth positions of its surrounding spatial points are referenced to ensure continuity between the adjusted depth position and those of the surrounding points, avoiding significant jumps or discontinuities. After these adjustments, the depth adjustment data for each spatial point group is completed.

[0165] Step S1458: Call the spatial point group depth adjustment completion data, extract the spatial points generated by the completion, combine their corresponding texture color information and the depth true value of the surrounding original spatial points, adjust their depth position again, integrate the adjusted completed spatial points, and form the secondary adjustment data of the completed spatial points.

[0166] Extract the completed spatial points from the depth adjustment data of the spatial point group. Combine the texture and color information corresponding to these spatial points with the ground truth depth values ​​of the surrounding original spatial points, and then further adjust the depth positions of the completed spatial points. This adjustment is more refined to ensure that the depth positions of the completed spatial points match the ground truth depth values ​​of the surrounding original spatial points more closely. Integrate the adjusted completed spatial points to form the secondary adjustment data for completed spatial points.

[0167] Step S1459: Integrate the secondary adjustment data of the completed spatial points with the original spatial point data in the spatial point group depth adjustment data, perform depth distribution analysis on all adjusted spatial points in the intermediate dense point cloud data, organize the correlation of depth changes of all spatial points, and form spatial point depth distribution analysis data.

[0168] The data from the secondary adjustment of completed spatial points are integrated with the original spatial point data from the completed depth adjustment data of the spatial point group to obtain all adjusted spatial points in the intermediate dense point cloud data. The depth distribution of these spatial points is analyzed to study the variation patterns and correlations of depth values, forming spatial point depth distribution analysis data.

[0169] Step S14510: Integrate all spatial points in the spatial point depth distribution analysis data, and organize the integrated spatial point data to form spatial point depth texture adjustment data.

[0170] All spatial points in the spatial point depth distribution analysis data are integrated and organized according to spatial location and other methods to form spatial point depth texture adjustment data. This spatial point depth texture adjustment data contains all spatial point information that has been adjusted in depth and bound to texture color.

[0171] Step S146: Extract the completed spatial points from the spatial point depth texture adjustment data, and adjust their depth positions again based on their corresponding texture color information and the depth ground truth of the surrounding original spatial points. Integrate the adjusted completed spatial points to form the completed spatial point depth adjustment data.

[0172] Spatial points generated through completion are extracted from the spatial point depth texture adjustment data. Based on the texture color information of these spatial points and the ground truth depth values ​​of surrounding original spatial points, their depth positions are further adjusted. By comparing the texture color differences between the completed spatial points and the surrounding original spatial points, and considering the changing trends of the ground truth depth values, the depth positions are fine-tuned to better integrate the completed spatial points into the original point cloud data. The adjusted completed spatial points are then integrated to form the completed spatial point depth adjustment data.

[0173] Step S147: Integrate the original spatial point data in the completed spatial point depth adjustment data and the spatial point depth texture adjustment data, adjust the depth position of all spatial points, so that the depth distribution of spatial points and the texture color distribution in the visual image data form a synergistic relationship. In areas where the texture color is the same or similar, the depth position of the spatial points remains consistent or changes continuously. In areas where the texture color is significantly different, the depth position of the spatial points changes accordingly, forming spatial point depth texture synergistic data.

[0174] The original spatial point data from the completed spatial point depth adjustment data and the spatial point depth texture adjustment data are integrated together, and the depth positions of all spatial points are adjusted as a whole. The goal of the adjustment is to establish a synergistic relationship between the depth distribution of spatial points and the texture color distribution in the visual image data. That is, when the texture colors are the same or similar, the depth positions of the spatial points remain consistent or show continuous changes; when there are significant differences in texture colors, the depth positions of the spatial points also change accordingly to reflect different surface structures. After the above adjustments, spatial point depth texture synergistic data is formed.

[0175] Step S148: Integrate the spatial point depth texture co-processing data with the texture color information of the visual image data, and remap the texture color information corresponding to each spatial point to the spatial point to form spatial point texture depth co-processing verification data.

[0176] By integrating spatial point depth texture co-processing data with texture and color information from visual image data, the texture and color information corresponding to each spatial point is re-mapped to ensure the accuracy and completeness of the texture and color information. This forms spatial point texture depth co-processing verification data, which is used to verify the texture and depth information of spatial points.

[0177] Step S149: Call the spatial point texture depth co-verification data, extract the spatial position information, depth information and texture color information of each spatial point, and integrate the three to form a depth and texture co-verified point cloud unit. Each point cloud unit contains spatial position, depth information and texture color information, forming depth and texture co-verified point cloud unit data.

[0178] Spatial location, depth, and texture color information of each spatial point are extracted from the spatial point texture depth co-verification data, and these three pieces of information are integrated into a single point cloud unit. Each point cloud unit fully contains the spatial location, depth, and texture color information of the spatial points, forming depth and texture co-verification point cloud unit data.

[0179] Step S1410: Integrate the depth and texture co-processing point cloud unit data, arrange all point cloud units according to the spatial distribution relationship of the target scene, and form dense point cloud data with depth and texture co-processing.

[0180] All point cloud units in the depth and texture co-processing point cloud unit data are arranged according to the spatial distribution relationship of the target scene, such as by the order of spatial coordinates or by the structural features of the scene. Through the above integration, dense point cloud data with depth and texture co-processing is formed, which can comprehensively reflect the spatial structure and surface texture features of the target scene.

[0181] Step S150: Extract the spatial location information, depth information, and texture color information of all spatial points in the dense point cloud data of depth and texture collaboration, arrange all spatial points according to the actual spatial relationship of the target scene, restore the spatial geometry and surface texture features of the target scene, and integrate them to form a color dense 3D reconstruction model of the target scene.

[0182] Spatial location, depth, and texture color information of all spatial points are extracted from dense point cloud data that combines depth and texture. Based on the actual spatial relationships of the target scene, such as the relative positions and connections between objects, all spatial points are arranged and organized. Through this method, the spatial geometry of the target scene is reconstructed, including the shape, size, and position of objects, while also presenting surface texture features such as color and pattern. Integrating this information, a colorful, dense 3D reconstruction model of the target scene is finally formed.

[0183] Step S151: Extract all point cloud units from the dense point cloud data that combines depth and texture. Each point cloud unit contains spatial location information, depth information, and texture color information. Arrange all point cloud units according to their spatial location information to form preliminary spatial structure data of the target scene.

[0184] All point cloud units are extracted from dense point cloud data that combines depth and texture. Each point cloud unit contains spatial location information, depth information, and texture color information. Based on the spatial location information, all point cloud units are initially arranged in three-dimensional space to form the preliminary spatial structure of the target scene. After processing, the preliminary spatial structure data is obtained.

[0185] Step S152: Call the preliminary spatial structure data, extract the spatial position information and depth information of each point cloud unit, connect all point cloud units to each other according to the spatial position and depth information of the point cloud units, construct the surface mesh of the target scene, and organize the constructed surface mesh to form the scene surface mesh data.

[0186] Using preliminary spatial structure data, the spatial location and depth information of each point cloud unit are extracted. Based on this information, a mesh construction algorithm is used to connect adjacent point cloud units to form a surface mesh of the target scene. The surface mesh consists of multiple polygonal patches, which can approximate the surface shape of the scene. The constructed surface mesh is then organized to form the scene surface mesh data.

[0187] Step S153: Call the scene surface mesh data, analyze the spatial angle and curvature changes of each mesh surface, combine the depth information of the point cloud unit, refine the details of the surface mesh, supplement the detailed structure of the mesh surface, and organize the refined surface mesh to form refined surface mesh data.

[0188] Each mesh face in the scene surface mesh data is analyzed, calculating the spatial angle and curvature changes of the mesh face. Combining the depth information of the point cloud units, the surface mesh details are refined; for example, the number of mesh faces is increased in areas of high curvature to more accurately represent the surface curvature. The detailed structure of the mesh faces is supplemented, enabling the surface mesh to more realistically reflect the surface features of the target scene. The refined surface mesh is then organized to form refined surface mesh data.

[0189] Step S154: Call the refined surface mesh data and the dense point cloud data with depth and texture coordination, extract the texture color information of each point cloud unit, map the texture color information to the corresponding position of the surface mesh one by one, so that each mesh face of the surface mesh has the corresponding texture color, organize the mapped surface mesh to form texture-mapped surface mesh data.

[0190] By combining refined surface mesh data and dense point cloud data with depth and texture synergy, texture color information is extracted for each point cloud unit. Using texture mapping technology, this texture color information is mapped one by one to the corresponding positions on the surface mesh, ensuring that each mesh face has its corresponding texture color. The mapped surface mesh is then organized to form texture-mapped surface mesh data.

[0191] Step S155: Call the texture mapping surface mesh data, adjust the texture mapping effect of the surface mesh according to the distribution characteristics of texture color information, and organize the adjusted surface mesh to form texture-optimized surface mesh data.

[0192] Analyze the distribution characteristics of texture color information in the texture-mapped surface mesh data, such as color uniformity and contrast. Based on these characteristics, adjust the texture mapping effect of the surface mesh, such as adjusting the texture scaling ratio, rotation angle, or color balance, to improve the texture display effect. Organize the adjusted surface mesh to form texture-optimized surface mesh data.

[0193] Step S156: Call the texture optimization surface mesh data, integrate the spatial geometric information and texture color information of the surface mesh to form the initial three-dimensional structure of the target scene, and organize the initial three-dimensional structure to form the initial three-dimensional structure data of the scene.

[0194] By integrating the surface mesh spatial geometry information (such as vertex coordinates and mesh connectivity) and texture color information from the texture-optimized surface mesh data, an initial 3D structure of the target scene is constructed. This initial 3D structure can preliminarily reflect the spatial morphology and surface texture of the target scene. The initial 3D structure is then organized to form the initial 3D structure data of the scene.

[0195] Step S157: Call the initial 3D structure data of the scene and the dense point cloud data of depth and texture collaboration, refer to the point cloud unit information, supplement the detailed features of the target scene, add the supplemented detailed features to the initial 3D structure, and organize the supplemented initial 3D structure to form detailed supplemented 3D structure data.

[0196] By combining the initial 3D structural data of the scene with dense point cloud data that integrates depth and texture, and referencing detailed information from point cloud units, such as small bumps and depressions, these detailed features are added to the initial 3D structure, making it more complete and refined. The supplemented initial 3D structure is then organized to form detailed supplementary 3D structural data.

[0197] Step S158: Call up the detailed supplementary three-dimensional structural data, perform overall optimization on the initial three-dimensional structure, organize the optimized three-dimensional structure, and form density-optimized three-dimensional structural data.

[0198] The detailed 3D structural data is supplemented and optimized overall. For example, the mesh density is adjusted, increasing the number of meshes in important areas and decreasing the number of meshes in less important areas, in order to improve efficiency while maintaining model accuracy. The optimized 3D structure is then organized to form density-optimized 3D structural data.

[0199] Step S159: Call the density-optimized 3D structure data, optimize the texture and color performance of the initial 3D structure, adjust the brightness, contrast and saturation of the texture and color, and organize the optimized 3D structure to form texture and color-optimized 3D structure data.

[0200] The texture and color representation in the density-optimized 3D structural data is optimized by adjusting parameters such as brightness, contrast, and saturation to make the texture and color more realistic and natural. The optimized 3D structure is then organized to form texture and color-optimized 3D structural data.

[0201] Step S1510: Call the texture and color optimization 3D structure data, integrate the optimized 3D structure to form a dense 3D reconstruction model of the target scene. The dense 3D reconstruction model of the target scene contains the complete spatial geometry of the target scene and the real surface texture and color features.

[0202] By integrating texture, color, and optimized 3D structural data, a final dense 3D reconstruction model of the target scene is formed. This model fully encompasses the spatial geometry of the target scene, such as the shape, position, and size of objects, as well as realistic surface texture and color features, enabling it to clearly and accurately present the 3D morphology of the target scene.

[0203] In one exemplary embodiment, a dense 3D reconstruction system fusing imaging radar point clouds and visual data is provided. This system can be a terminal, server, etc., and its internal structure diagram can be as follows: Figure 2As shown, the dense 3D reconstruction system for fused imaging radar point clouds and visual perception includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computational and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, near-field communication, or other technologies. When the computer program is executed by the processor, it implements a dense 3D reconstruction method for fused imaging radar point clouds and visual perception. The display unit is used to form a visually visible image and can be a display screen, projection device, or virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device can be a touch layer covering the display screen, or a button, trackball, or touchpad set on the shell of a dense 3D reconstruction system that fuses imaging radar point clouds and vision, or an external keyboard, touchpad, or mouse, etc.

[0204] It should be noted that, in order to simplify the description of the present invention and thus help to understand one or more embodiments of the invention, multiple features may sometimes be grouped into one embodiment, drawing or description thereof in the foregoing description of the embodiments of the present invention.

Claims

1. A dense 3D reconstruction method fusing imaging radar point clouds and visual data, characterized in that, The method includes: Simultaneously acquire imaging radar point cloud data and visual image data to form a time-synchronized dual-source data set; Each spatial point in the imaging radar point cloud data is mapped to the corresponding image pixel position in the visual image data, establishing a pixel-level depth association between the imaging radar point cloud data and the visual image data. Based on this pixel-level depth association, the depth range corresponding to each pixel in the visual image data is defined, and the depth range is bound to the corresponding pixel. Based on pixel-level deep correlation, texture detail information corresponding to sparse regions of imaging radar point cloud data is extracted from visual image data. Combined with the spatial distribution correlation of spatial points around sparse regions in imaging radar point cloud data, the extracted texture detail information is transformed into spatial point cloud information. The transformed spatial point cloud information is then filled into sparse regions in imaging radar point cloud data. The original spatial points and the completed spatial points are integrated to form intermediate dense point cloud data. The texture and color information in the visual image data is mapped pixel by pixel to the corresponding spatial points in the intermediate dense point cloud data, establishing a one-to-one correspondence between spatial points and texture and color information. The ground truth depth value in the imaging radar point cloud data is retrieved, the depth position of each spatial point in the intermediate dense point cloud data is adjusted, and the adjusted spatial points, depth information and texture and color information are integrated to generate dense point cloud data with depth and texture coordination. The spatial location, depth, and texture color information of all spatial points in the dense point cloud data with depth and texture synergy are extracted. All spatial points are arranged according to the actual spatial relationship of the target scene to restore the spatial geometry and surface texture features of the target scene and integrate them to form a color dense 3D reconstruction model of the target scene.

2. The dense 3D reconstruction method based on fused imaging radar point clouds and visual data according to claim 1, characterized in that, The process of mapping each spatial point in the imaging radar point cloud data to the corresponding image pixel position in the visual image data, establishing a pixel-level depth association between the imaging radar point cloud data and the visual image data, defining the depth range corresponding to each pixel in the visual image data based on this pixel-level depth association, and binding the depth range to the corresponding pixel includes: The three-dimensional spatial coordinates and depth information of each spatial point in the imaging radar point cloud data are extracted, and the pixel coordinates and texture features of each image in the visual image data are extracted. Corresponding associations are established according to the acquisition time sequence, and the spatial points corresponding to each imaging radar point cloud data are bound to the image regions corresponding to the same time node and the same spatial location in the visual image data to form time-series associated data. The depth information of the imaging radar point cloud data in the temporal correlation data is extracted. This depth information is used as the truth benchmark. Combined with the scanning parameters of the imaging radar and the imaging parameters of the visual camera, the correspondence between spatial points in the imaging radar point cloud data and pixels in the visual image data is determined. This correspondence is transformed into a specific execution process. The operation steps for mapping each spatial point to the image plane are set to form a spatial mapping process. The spatial mapping process is invoked to map each spatial point in the imaging radar point cloud data to the corresponding image plane of the visual image data one by one. The pixel coordinates of each spatial point in the visual image data are calculated, and the depth information of each spatial point is bound to the calculated pixel coordinates to form a spatial point pixel coordinate depth true value association unit. Pixel coordinates are extracted from the depth ground truth association unit of spatial point pixel coordinates. The neighboring pixels around the pixel coordinates in the visual image data are located. The texture similarity of the neighboring pixels is analyzed. Pixels with the same texture similarity are divided into the same pixel group. Each pixel group is bound to the corresponding spatial point in the imaging radar point cloud data. The depth information of the spatial point is assigned to the corresponding pixel group to form pixel group depth ground truth association data. Extract the depth ground value corresponding to each pixel group from the pixel group depth ground value association data, combine the texture distribution features of the pixels in the pixel group, define the depth range corresponding to all pixels in the pixel group, take the depth ground value corresponding to the pixel group as the core, combine the texture distribution features to adjust the upper and lower limits of the depth range, bind the defined depth range to the pixel group, and form pixel group depth range association data. Extract the pixel group and its corresponding depth range from the pixel group depth range association data, bind each pixel in each pixel group to the depth range corresponding to that pixel group, and integrate the bound pixels and depth range to form pixel depth range binding data; The pixel depth range binding data is called, and the depth range binding information of all pixels is integrated to form a depth estimation constraint process for visual image data. This depth estimation constraint process takes the true depth value of imaging radar point cloud data as the core and the texture features of pixels as an aid, and sets the specific operation standards and range limits for depth estimation of each pixel. By integrating the depth estimation constraint process with the visual image data, the depth estimation operation of the visual image data is strictly executed according to the depth estimation constraint process, and the depth estimation value of each pixel is limited to the depth range bound to that pixel, thus forming the visual image data with depth estimation constraint. The visual image data with depth estimation constraints is called, and a preliminary depth estimation operation is performed according to the depth estimation constraint process to generate a preliminary depth map of the visual image data. Each pixel value in the preliminary depth map corresponds to the depth estimation value of the pixel, and the depth estimation value is within the depth range bound to the pixel, thus forming the preliminary depth map data. The preliminary depth map data is integrated with the imaging radar point cloud data, and the integrated preliminary depth map data is correlated with the imaging radar point cloud data to form preliminary depth map and radar point cloud correlated data.

3. The dense 3D reconstruction method based on fused imaging radar point clouds and visual data according to claim 1, characterized in that, The process involves extracting texture detail information from visual image data corresponding to sparse regions of imaging radar point cloud data based on pixel-level depth correlation. This information is then combined with the spatial distribution correlation of spatial points surrounding the sparse regions in the imaging radar point cloud data. The extracted texture detail information is converted into spatial point cloud information, which is then used to fill in the sparse regions of the imaging radar point cloud data. Finally, the original spatial points and the filled-in spatial points are integrated to form intermediate dense point cloud data. This includes: By calling pixel-level depth correlation, the distribution of spatial points in imaging radar point cloud data is analyzed, sparse regions in imaging radar point cloud data are located, spatial location information of sparse regions is extracted, and sparse region spatial location data is formed. By calling the spatial location data of the sparse region and locating the image region corresponding to the sparse region in the visual image data through pixel-level depth correlation, the depth information in the preliminary depth map data corresponding to the image region is extracted, and the image region is integrated with the corresponding depth information to form the image and depth data corresponding to the sparse region. Extract the image region corresponding to the sparse region from the image and depth data, and extract the texture direction, texture density, texture color transition and texture boundary features within the image region. During the extraction process, follow the pixel-level depth correlation relationship, and match each extracted texture detail information with the spatial location of the sparse region in the imaging radar point cloud data to form target corresponding data. Analyze the texture detail information in the data corresponding to the target, extract the continuous spatial distribution relationship of the texture detail information, divide the continuous texture region according to the continuous distribution relationship, and associate each continuous texture region with the corresponding spatial location to form the spatial location data of the continuous texture region. Extract each texture continuous region from the spatial location data of texture continuous regions, associate each texture continuous region with the surrounding spatial points of sparse regions in the imaging radar point cloud data, and extract the spatial location, spacing and depth variation correlation of the surrounding spatial points to form the surrounding spatial point distribution correlation data. Call the surrounding spatial point distribution association data and texture continuous region spatial location data, combine the texture continuous distribution association relationship of each texture continuous region and the depth information in the preliminary depth map data, determine the spatial point generation operation process corresponding to each texture continuous region, and set the specific determination method of spatial point generation density, spatial location and depth information in the spatial point generation operation process; The spatial point generation process is invoked to convert the texture detail information of each continuous texture region into spatial point cloud information. The generation density of spatial points is determined based on the texture density in the texture detail information, the spatial orientation of spatial points is determined based on the texture orientation, the depth position of spatial points is determined based on the depth information in the preliminary depth map data, and the distribution range of spatial points is determined based on the texture boundary features, thus forming the converted spatial point cloud information. The transformed spatial point cloud information is integrated with the surrounding spatial points of sparse areas in the imaging radar point cloud data. The positions of the spatial points in the transformed spatial point cloud information are adjusted, and the integrated spatial point cloud information is associated with the surrounding points to form the data of the complete point cloud and the associated surrounding points. The system calls the data associated with the complete point cloud and surrounding points, summarizes the spatial point cloud information generated by each continuous texture region, fills the corresponding sparse region in the imaging radar point cloud data with the summarized spatial point cloud information, and initially integrates the filled point cloud data with the original imaging radar point cloud data to form preliminary complete point cloud data. The initial point cloud data is called to complete the data, adjust the spatial distribution of all spatial points, and integrate the adjusted point cloud data to form intermediate dense point cloud data.

4. The dense 3D reconstruction method based on fused imaging radar point clouds and visual data according to claim 1, characterized in that, The process of mapping texture and color information from visual image data pixel by pixel to corresponding spatial points in intermediate dense point cloud data, establishing a one-to-one correspondence between spatial points and texture and color information, retrieving ground truth depth values ​​from imaging radar point cloud data, adjusting the depth position of each spatial point in intermediate dense point cloud data, and integrating the adjusted spatial points, depth information, and texture and color information to generate dense point cloud data with depth and texture synergy includes: By calling pixel-level depth correlation, the pixel coordinates in the visual image data corresponding to each spatial point in the intermediate dense point cloud data are determined, and each spatial point is bound to the corresponding pixel coordinates to form spatial point pixel coordinate binding data. Call the spatial point pixel coordinate binding data, extract the pixel coordinates corresponding to each spatial point, and map the texture color information of each pixel in the visual image data to the corresponding spatial point in the intermediate dense point cloud data pixel by pixel. During the mapping process, follow the pixel-level depth association relationship, and bind the mapped spatial points with texture color information to form spatial point texture color binding data. Integrate spatial point texture color binding data with pixel-level depth association to link spatial point, texture color information and depth information. Organize the integrated association data to form spatial point texture depth association data, with each spatial point bound to unique texture color information and corresponding depth information. Extract the ground truth depth value from the imaging radar point cloud data, associate the ground truth depth value with the spatial points in the spatial point texture depth association data, directly bind the spatial points in the intermediate dense point cloud data that correspond to the original spatial points in the imaging radar point cloud data with the ground truth depth value of the imaging radar point cloud data, and bind the spatial points generated by the completion in the intermediate dense point cloud data with the depth reference value based on the ground truth depth value of the imaging radar point cloud data to form spatial point depth truth value binding data; Call the spatial point depth ground value binding data, preliminary depth map data and spatial point texture color binding data, adjust the depth position of each spatial point in the intermediate dense point cloud data, refer to the distribution characteristics of the texture color information corresponding to the spatial point and the area where the texture color information changes continuously, adjust the spatial point depth position to make it change continuously, bind the adjusted spatial point depth position with the texture color information to form spatial point depth texture adjustment data; Extract the completed spatial points from the spatial point depth texture adjustment data, and adjust their depth positions again based on their corresponding texture color information and the depth ground truth of the surrounding original spatial points. Integrate the adjusted completed spatial points to form the completed spatial point depth adjustment data. The original spatial point data in the supplementary spatial point depth adjustment data and the spatial point depth texture adjustment data are integrated. The depth position of all spatial points is adjusted so that the depth distribution of spatial points and the texture color distribution in the visual image data form a synergistic relationship. In areas where the texture color is the same or similar, the depth position of the spatial point remains consistent or changes continuously. In areas where the texture color is significantly different, the depth position of the spatial point changes accordingly, thus forming spatial point depth texture synergistic data. By integrating spatial point depth texture co-processing data with texture color information from visual image data, and mapping the texture color information corresponding to each spatial point back to the spatial point, spatial point texture depth co-processing verification data is formed. The spatial point texture depth co-verification data is called, and the spatial position, depth information and texture color information of each spatial point are extracted. The three are integrated to form a depth and texture co-verified point cloud unit. Each point cloud unit contains spatial position, depth information and texture color information, forming depth and texture co-verified point cloud unit data. The depth and texture collaborative point cloud unit data are integrated, and all point cloud units are arranged according to the spatial distribution relationship of the target scene to form dense point cloud data with depth and texture collaboration.

5. The dense 3D reconstruction method based on fused imaging radar point clouds and visual data according to claim 2, characterized in that, The aforementioned spatial mapping process maps each spatial point in the imaging radar point cloud data to the corresponding image plane in the visual image data, calculates the pixel coordinates of each spatial point in the visual image data, and binds the depth information of each spatial point with the calculated pixel coordinates to form a spatial point pixel coordinate depth ground truth association unit, including: Extract each spatial point from the imaging radar point cloud data, extract each spatial point separately, each spatial point contains three-dimensional spatial coordinates and depth information, organize the individually extracted spatial points to form individual spatial point data; Each spatial point in the individual spatial point data is called, its three-dimensional spatial coordinates are substituted into the spatial mapping process, and the pixel coordinate calculation operation is executed step by step according to the mapping logic and parameter settings set in the spatial mapping process. During the calculation process, the scanning parameters of the imaging radar and the imaging parameters of the vision camera are combined to obtain the pixel coordinates of the spatial point in the corresponding image plane of the visual image data, thus forming the spatial point pixel coordinate calculation data. Extract the pixel coordinates and corresponding spatial points from the spatial point pixel coordinate calculation data, integrate the calculated pixel coordinates of each spatial point with the visual image data, locate the specific image frame and image region of the pixel coordinates in the visual image data, and bind the pixel coordinates, corresponding image frames and image regions with the spatial points to form spatial point pixel coordinate image association data. Extract each spatial point from the spatial point pixel coordinate image association data, extract the depth information of the spatial point, and bind the depth information with the pixel coordinates, image frame and image region corresponding to the spatial point to form a spatial point pixel coordinate depth ground value association unit. All the spatial point pixel coordinate depth ground truth association units are summarized and the information of all association units is integrated to form a set of spatial point pixel coordinate depth ground truth association units. Each association unit contains a spatial point, pixel coordinates, image frame, image region and depth ground truth, forming the association unit set data. The associated unit set data is called to analyze the association of all pixels in the visual image data, locate the pixels in the visual image data that are not mapped to spatial points, extract the unmapped pixels, combine them with the depth ground truth of the surrounding mapped pixels, delineate the depth reference range of the unmapped pixels, and bind the unmapped pixels to the corresponding depth reference range to form unmapped pixel depth reference data. Each unmapped pixel in the unmapped pixel depth reference data is bound to its corresponding depth reference range to form a supplementary association unit. The supplementary association unit contains the coordinates of the unmapped pixel, the image frame, the image region, and the depth reference range. It has the same structure as the spatial point pixel coordinate depth ground value association unit, forming supplementary association unit data. The supplementary association unit data is integrated with the spatial point pixel coordinate depth true value association unit set to form the spatial point pixel coordinate depth true value association unit set. The association unit set contains the association units corresponding to all pixels in the visual image data, forming a complete association unit set data. The complete set of associated units data is integrated with the visual image data, so that each pixel in the visual image data can retrieve the depth information in its corresponding associated unit. The integrated visual image data is then bound to the complete set of associated units data to form integrated visual image and associated unit data. The complete set of associated unit data is integrated with the imaging radar point cloud data, and the integrated complete set of associated unit data is bound to the imaging radar point cloud data to form the integrated data of associated units and radar point cloud.

6. The dense 3D reconstruction method based on fused imaging radar point clouds and visual data according to claim 3, characterized in that, The process involves calling surrounding spatial point distribution association data and texture continuous region spatial location data, combining the texture continuous distribution association relationship of each texture continuous region with the depth information in the preliminary depth map data, to determine the spatial point generation operation process corresponding to each texture continuous region. This process sets the specific determination methods for the spatial point generation density, spatial location, and depth information, forming a spatial point generation process, including: Extract each surrounding spatial point from the surrounding spatial point distribution correlation data, analyze the spatial location, depth information and spatial spacing of each surrounding spatial point, organize the correlation relationship of the spacing, direction and depth changes of the surrounding spatial points, and form detailed data of the surrounding spatial point distribution; The basic generation density of spatial points is determined by calling the detailed data of the distribution of surrounding spatial points. The basic generation density is based on the spatial spacing of surrounding spatial points and is adjusted in combination with the texture density of the texture continuous region. The value of the basic generation density is increased in areas with high texture density and decreased in areas with low texture density. The adjusted basic generation density is bound to the texture continuous region to form basic generation density binding data. Call the texture direction feature in the texture continuous distribution association data, bind the texture direction feature with the spatial point generation direction. In the region where the texture direction is continuous, set the spatial point generation direction to be consistent with the texture direction. In the region where the texture direction changes, set the spatial point generation direction to change synchronously with the texture direction. Bind the generation direction with the continuous texture region to form generation direction binding data. Call the preliminary depth map data, extract the depth information corresponding to each continuous texture region, combine the depth ground value in the surrounding spatial point distribution detail data, organize the depth change correlation relationship corresponding to each continuous texture region, and form depth change correlation data. Call the depth change association data to determine the depth determination method of spatial points at different locations within each continuous texture region, and bind the depth determination method to the continuous texture region to form depth determination method binding data; The texture boundary features in the texture continuous distribution association data are called, and the texture boundary features are transformed into the generation range boundary of spatial points. The generation range boundary fits the boundary of the texture continuous region and connects with the distribution range of surrounding spatial points. The generation range boundary is bound to the texture continuous region to form the generation range boundary binding data. By integrating basic generation density binding data, generation direction binding data, depth determination method binding data, and generation range boundary binding data, the specific steps of the spatial point generation operation process corresponding to each continuous texture region are constructed, and the order and specific operation requirements of spatial point cloud generation are set to form the basic steps of the spatial point generation process. The texture density and texture color transition features in the texture continuous distribution associated data are integrated into the basic steps of the spatial point generation process, and the spatial point generation density adjustment operation is improved. The improved generation density adjustment operation is added to the basic steps of the spatial point generation process to form the improved steps of the spatial point generation process. The depth adjustment requirements in the depth change correlation data are incorporated into the spatial point generation process improvement steps. Combined with texture color transition features, the derived spatial point depth information is fine-tuned to form a complete spatial point generation process. The improved spatial point generation process is bound to each continuous texture region. A spatial point generation process corresponding to each continuous texture region is set, and the spatial point generation processes corresponding to all continuous texture regions are summarized to form a set of spatial point generation processes.

7. The dense 3D reconstruction method based on fused imaging radar point clouds and visual data according to claim 4, characterized in that, The process involves calling spatial point depth ground truth binding data, preliminary depth map data, and spatial point texture color binding data to adjust the depth position of each spatial point in the intermediate dense point cloud data. During the adjustment process, the distribution characteristics of the texture color information corresponding to the spatial point are referenced, and the spatial point depth position is adjusted to make it continuously change in areas where the texture color information changes continuously. The adjusted spatial point depth position is then bound to the texture color information to form spatial point depth texture adjustment data, including: Extract all ground truth depth values ​​from the imaging radar point cloud data, associate the ground truth depth values ​​with the spatial points in the data, organize the reference ground truth depth values ​​corresponding to each spatial point, and form spatial point reference depth ground truth data. Call the spatial point texture color binding data, group all spatial points in the intermediate dense point cloud data according to their corresponding texture color information, divide spatial points with similar texture color information into the same group, extract the depth information from the preliminary depth map data corresponding to each group, bind each group of spatial points with the corresponding depth information, and form spatial point group depth association data. Call each spatial point group in the spatial point group deep association data, analyze the distribution characteristics of the texture color information corresponding to the spatial points in the group, analyze the continuous change of texture color information, divide the region of continuous texture color change and the region of abrupt texture color change, and form spatial point group texture change region data. The depth adjustment benchmark for each spatial point group is determined by calling the texture change region data of the spatial point group and the reference depth ground value data of the spatial point. The depth adjustment benchmark is based on the depth ground value of the imaging radar point cloud data and is corrected by combining the depth information in the preliminary depth map data. The corrected depth adjustment benchmark is bound to each spatial point group to form the spatial point group depth adjustment benchmark data. Call the spatial point group depth adjustment reference data and spatial point group texture change area data. For spatial points in areas where texture color changes continuously, adjust the depth position of each spatial point according to the depth adjustment reference and the continuous change trend of texture color. Bind the adjusted spatial point depth position with texture color information to form continuous area spatial point adjustment data. Call the spatial point group depth adjustment reference data and spatial point group texture change area data. For spatial points in areas with abrupt changes in texture color, adjust the depth position of each spatial point according to the depth adjustment reference and the characteristics of the texture color change. Bind the adjusted spatial point depth position with texture color information to form spatial point adjustment data for the change area. The continuous region spatial point adjustment data and the abrupt region spatial point adjustment data are integrated, and the depth of all spatial points in each spatial point group is adjusted one by one. During the adjustment process, the depth position of each spatial point is adjusted to keep it continuous with the depth position of the surrounding spatial points, thus forming the spatial point group depth adjustment data. The spatial point group depth adjustment data is called, the completed spatial points are extracted, and their depth positions are adjusted again by combining their corresponding texture color information and the depth ground truth of the surrounding original spatial points. The adjusted completed spatial points are integrated to form the secondary adjustment data of the completed spatial points. The data of secondary adjustment of spatial points is integrated with the original spatial point data in the data of spatial point group depth adjustment. The depth distribution analysis of all adjusted spatial points in the intermediate dense point cloud data is performed, and the correlation of depth changes of all spatial points is sorted out to form spatial point depth distribution analysis data. All spatial points in the spatial point depth distribution analysis data are integrated, and the integrated spatial point data is then organized to form spatial point depth texture adjustment data.

8. The dense 3D reconstruction method based on fused imaging radar point clouds and visual data according to claim 1, characterized in that, The spatial location information, depth information, and texture color information of all spatial points in the extracted depth and texture co-existing dense point cloud data are arranged according to the actual spatial relationships of the target scene. This process restores the spatial geometry and surface texture features of the target scene, integrating them to form a color-dense 3D reconstruction model of the target scene, including: Extract all point cloud units from dense point cloud data that combines depth and texture. Each point cloud unit contains spatial location information, depth information, and texture color information. Arrange all point cloud units according to their spatial location information to form preliminary spatial structure data of the target scene. The initial spatial structure data is called, the spatial position information and depth information of each point cloud unit are extracted, and all point cloud units are connected to each other based on the spatial position and depth information of the point cloud units to construct the surface mesh of the target scene. The constructed surface mesh is then organized to form the scene surface mesh data. The scene surface mesh data is called up, the spatial angle and curvature changes of each mesh surface are analyzed, and the depth information of the point cloud unit is combined to refine the details of the surface mesh, supplement the detailed structure of the mesh surface, and organize the refined surface mesh to form refined surface mesh data. The finely refined surface mesh data and the dense point cloud data with depth and texture coordination are called up, the texture color information of each point cloud unit is extracted, and the texture color information is mapped to the corresponding position of the surface mesh one by one, so that each mesh face of the surface mesh has the corresponding texture color. The mapped surface mesh is sorted out to form texture-mapped surface mesh data. The texture mapping surface mesh data is called, and the texture mapping effect of the surface mesh is adjusted according to the distribution characteristics of texture color information. The adjusted surface mesh is then organized to form texture-optimized surface mesh data. The texture is used to optimize the surface mesh data, integrate the spatial geometric information and texture color information of the surface mesh to form the initial three-dimensional structure of the target scene, and organize the initial three-dimensional structure to form the initial three-dimensional structure data of the scene. The initial 3D structural data of the scene and the dense point cloud data with depth and texture are called. The point cloud unit information is referenced to supplement the detailed features of the target scene. The supplemented detailed features are added to the initial 3D structure. The supplemented initial 3D structure is then organized to form the detailed supplemented 3D structural data. The detailed 3D structural data is called to perform overall optimization of the initial 3D structure. The optimized 3D structure is then organized to form density-optimized 3D structural data. The density-optimized 3D structure data is called to optimize the texture and color performance of the initial 3D structure. The brightness, contrast and saturation of the texture and color are adjusted, and the optimized 3D structure is organized to form texture and color-optimized 3D structure data. The texture and color are used to optimize the 3D structure data, and the optimized 3D structure is integrated to form a dense 3D reconstruction model of the target scene. This dense 3D reconstruction model of the target scene contains the complete spatial geometry and real surface texture and color features of the target scene.

9. The dense 3D reconstruction method based on fused imaging radar point clouds and visual data according to claim 5, characterized in that, The step of extracting pixel coordinates and corresponding spatial points from the spatial point pixel coordinate calculation data, integrating the calculated pixel coordinates corresponding to each spatial point with the visual image data, locating the specific image frame and image region of the pixel coordinates in the visual image data, and binding the pixel coordinates, corresponding image frames, and image regions with the spatial points to form spatial point pixel coordinate image association data, including: Extract each pixel coordinate and corresponding spatial point from the spatial point pixel coordinate calculation data, extract the pixel coordinates corresponding to each spatial point separately, and simultaneously extract the three-dimensional spatial coordinates and depth ground value of the spatial point and integrate them to form a temporary association unit; Substitute the pixel coordinates in the temporary association unit into the coordinate system of the visual image data, combine the imaging parameters of the visual camera, determine the specific image frame of the pixel coordinates in the visual image data according to the acquisition time sequence, and bind the pixel coordinates, the corresponding image frame and the spatial point to form pixel coordinate image frame association data. Call the pixel coordinate image frame association data, locate the specific position of the pixel coordinate in the determined image frame, clarify the horizontal and vertical coordinate positions of the pixel coordinate in the image frame, divide the image region according to the texture distribution of the image frame, determine the image region where the pixel coordinate is located, and bind the pixel coordinate, image frame, image region and spatial point to form pixel coordinate image region association data. Extract the pixel coordinates and corresponding image regions from the pixel coordinate image region association data, extract the texture features of the image region, associate the texture features with the depth ground truth of the corresponding spatial point, and bind the texture features, pixel coordinates, image frame, image region and spatial point to form pixel coordinate image region texture association data; The pixel coordinate image region texture association data and spatial mapping process are called, the pixel coordinate mapping results are compared with the operation requirements of the spatial mapping process, the matching of texture features and depth ground truth is compared, the pixel coordinate calculation results are adjusted, and pixel coordinate mapping adjustment data is formed. The pixel coordinate mapping adjustment data is called, and the aforementioned steps are repeated for the pixel coordinates corresponding to each spatial point to determine the image frame, image region and texture features corresponding to each pixel coordinate. The pixel coordinates, image frame, image region, texture features and depth ground truth corresponding to each spatial point are bound to form complete associated data of spatial point pixel coordinates. The complete association data of the pixel coordinates of all spatial points is summarized and the mapping information of all spatial points is integrated to form a complete association set of spatial point pixel coordinates. Call the complete association set of spatial point pixel coordinates, analyze the mapping distribution of all spatial points, check whether the mapping of spatial points covers the main image area of ​​the visual image data, check the uniformity of the mapping distribution, and adjust the mapping parameters and pixel coordinate calculation results of spatial points to make the mapping distribution of spatial points more uniform. The adjusted spatial point pixel coordinates complete association set data is called, and the adjusted mapping parameters and pixel coordinate calculation results are integrated to form spatial point mapping optimization data. The spatial point mapping optimization data is integrated with the complete spatial point pixel coordinate association set data, and the complete spatial point pixel coordinate association set data is updated to form spatial point pixel coordinate image association data.

10. A dense 3D reconstruction system fusing imaging radar point clouds and visual data, characterized in that, include: processor; A machine-readable storage medium for storing machine-executable instructions of the processor; The processor is configured to perform the dense 3D reconstruction method of fusion imaging radar point cloud and vision as described in any one of claims 1 to 9 by executing the machine-executable instructions.

Images