A method for constructing a three-dimensional model of a site by fusing SLAM point cloud and image features

Abstract

The present invention relates to the technical field of three-dimensional construction of ruins, specifically a method for constructing a three-dimensional model of a ruins by integrating SLAM point cloud and image features, comprising the following steps: acquiring depth images and RGB data, aligning by timestamp and eliminating inconsistent data, calculating frame rate and node ratio, establishing a binding relationship between image and point cloud, extracting three-dimensional coordinate points, identifying and classifying geometric structures, evaluating texture and edge response, adjusting resolution, analyzing static and dynamic structures, eliminating dynamic interference, calculating geometric curvature and fitting error, optimizing three-dimensional model display, and obtaining a detail restoration score. In the present invention, by accurately synchronizing point cloud data and RGB images, strict verification of timestamp matching and data consistency is allowed, significantly reducing data errors and noise, ensuring the accuracy and reliability of the model, identifying structural elements and refining dense areas of geometric structures for analysis, providing richer data level support, and providing higher quality data for archaeological research.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional construction of ruins, and in particular to a method for constructing a three-dimensional model of ruins by integrating SLAM point cloud and image features. Background Art

[0002] The technology of 3D construction of historical sites mainly focuses on using modern surveying and image processing technologies to reconstruct 3D models of historical sites. This field integrates multiple technologies such as geographic information systems (GIS), photogrammetry, laser scanning (LiDAR), computer vision and machine learning. The purpose is to create accurate 3D digital copies. Through 3D models, researchers can conduct detailed research and analysis without causing physical interference to the site itself. The model can not only be used for scientific research and education, but also for the protection and restoration of sites, especially when the original site has been damaged or is at risk of collapse.

[0003] Among them, the method of constructing a three-dimensional model of the site is a method that uses point cloud data generated by SLAM (simultaneous localization and mapping) technology and combines it with high-resolution image features to create a three-dimensional model of the archaeological site. Its main purpose is to provide higher-quality data support for archaeological and historical research. At the same time, accurate three-dimensional models can also be used for the digital preservation of cultural heritage, which will help the protection, educational display and possible restoration work of the site in the future.

[0004] Although existing technologies integrate a variety of high-end technologies, they have limitations in processing the integration and accuracy improvement of heterogeneous data. In particular, without fully utilizing SLAM technology, it is difficult to achieve high-precision spatial data alignment, resulting in deviations between the details of the three-dimensional model and the actual site. In addition, the existing technology lacks an effective dynamic resolution adjustment mechanism, resulting in the inability to adjust the scanning strategy according to the specific situation during the data collection process, generating too much invalid or redundant data, increasing the difficulty and cost of subsequent data processing. At the same time, it fails to effectively identify dynamic and static structures, causing the model to accumulate data errors under the influence of natural environmental factors such as vegetation, affecting the accuracy and practicality of the model and increasing the complexity of modeling. Summary of the Invention

[0005] The purpose of the present invention is to solve the shortcomings of the prior art and propose a method for constructing a three-dimensional model of a site by integrating SLAM point cloud and image features.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a method for constructing a three-dimensional model of a site by integrating SLAM point cloud and image features, comprising the following steps:

[0007] S1: Acquire the depth image data and RGB image data in the site space, align the RGB image with the 3D point cloud data generated by the SLAM system according to the timestamp, calculate the ratio of the image frame rate to the number of scanning nodes, determine the effective matching data area, establish the binding relationship between the image and the point cloud, and obtain the image point cloud binding coefficient;

[0008] S2: Based on the image point cloud binding coefficient, identify dense areas of geometric structure, analyze the spatial direction and boundary characteristics of the area, compare them with geometric rules, identify and classify structural elements in the ruins, and obtain geometric structure recognition indicators;

[0009] S3: Evaluate the texture direction change and edge response of each image block based on the geometric structure recognition index, adjust the corresponding scanning resolution by analyzing the local complexity of the image data, and refine the pixel granularity and scanning frequency during the image acquisition process to obtain a resolution adjustment index;

[0010] S4: Using the resolution adjustment index, static and dynamic structures are identified, displacement vectors and density distributions in the point cloud data are continuously monitored, interference caused by dynamic factors is eliminated, non-static structures are removed, and a point cloud stability rating is obtained.

[0011] The improvements of the present invention are that the image point cloud binding coefficient includes data synchronization rate, matching accuracy, and data consistency; the geometric structure recognition index specifically includes structure discrimination, classification accuracy, and positioning accuracy; the resolution adjustment index includes resolution range, pixel adjustment ratio, and complexity response information; the point cloud stability rating specifically refers to static verification degree, dynamic exclusion efficiency, and environmental interference recognition information.

[0012] The present invention is improved in that the steps of obtaining the image point cloud binding coefficient are specifically as follows:

[0013] S111: Acquire depth image data and RGB image data in the site space, align the RGB image with the three-dimensional point cloud data generated by the SLAM system according to the timestamp, remove RGB image frames and point cloud frame data with inconsistent timestamps, and calculate the ratio of the image frame rate to the number of point cloud nodes to obtain the image point cloud frame rate ratio;

[0014] S112: Based on the image point cloud frame rate ratio, determine whether it is within the node matching ratio baseline range. If it is outside the range, mark the data area of the ratio, and perform a joint judgment based on the point cloud density and image frame rate continuity to select data segments that meet the time continuity and point cloud density requirements to obtain a qualified matching area indicator.

[0015] S113: Constructing a mapping relationship between the image frame and the point cloud node based on the qualified matching area index, using the formula:

[0016] ;

[0017] Obtain the degree of binding between the image features and the point cloud, and obtain the image point cloud binding coefficient, where: Representative The image point cloud binding coefficient of the region, Representative In the region The number of point cloud nodes corresponding to the image features, Representative In the region Frame number of image frames, Representative The average number of image frames per region, Representative The total number of point cloud matching nodes in the region, Representative The time span corresponding to the image frame in each region, Representative In the region The spatial density of point cloud nodes, Representative The average spatial density of point cloud nodes in the region, It is a region The total number of image features in , It is a region The total number of image frames in , It is a region The total number of point cloud nodes.

[0018] The present invention is improved in that the steps of obtaining the geometric structure recognition index are specifically as follows:

[0019] S211: Based on the image point cloud binding coefficient, extract the three-dimensional coordinate points corresponding to the image, calculate the spatial density, voxel distribution and depth continuity of the point cloud, determine the high-density area in the point cloud, and analyze the density difference and voxel number in the area to obtain the aggregation degree of the high-density area;

[0020] S212: Based on the high-density region aggregation degree, combined with the tangent plane normal vector and boundary coordinate deviation of the point cloud in the local region, the formula is used:

[0021] ;

[0022] Calculation area Structural orientation differentiation , we get the boundary directional stability, where Indicates area Neidi The Euclidean distance between the point and the center of the region, Indicates a point The boundary direction coordinate difference, that is, the offset between the point and the predetermined boundary direction, Indicates the Point in area The modulus of the tangent plane normal vector in reflects the orientation or direction characteristics of the point on the tangent plane. Indicates a point The offset in the direction of the boundary, Indicates area The number of boundary points within ;

[0023] S213: Calling the boundary direction stability, classifying the regional structure, numbering the structural types of the different areas according to the boundary characteristics, comparing the overlap between the classified areas and the geometric rules, identifying the structural elements in the ruins, and obtaining the geometric structure identification index.

[0024] The present invention is improved in that the step of obtaining the resolution adjustment index is specifically as follows:

[0025] S311: Evaluate the texture direction change and edge response of each image block based on the geometric structure recognition index, perform local feature analysis on the image data, extract the amount of texture direction change and the edge response degree, and obtain texture change and response degree;

[0026] S312: Based on the texture change and responsiveness, the complexity between the different regions is compared to determine whether the scanning resolution needs to be adjusted, using the formula:

[0027] ;

[0028] Get resolution adjustment parameters ,in, represents the average edge response, is the edge response threshold, Indicates the degree of change in texture direction, is the statistical standard deviation of the variation in texture orientation;

[0029] S313: Based on the resolution adjustment parameter, the corresponding scanning resolution is adjusted, and the pixel granularity and scanning frequency in the acquisition process are refined to obtain a resolution adjustment index.

[0030] The present invention is improved in that the steps for obtaining the point cloud stability rating are specifically as follows:

[0031] S411: Analyzing each type of structure in the point cloud data using the resolution adjustment index, identifying features of differential structures by analyzing point cloud resolution changes, determining the difference between static and dynamic structures, and obtaining static and dynamic structure recognition results;

[0032] S412: Based on the static and dynamic structure identification results, the dynamic changes of the point cloud data are monitored. The data in the stable area are screened for the displacement and density fluctuation ranges, the interference of dynamic factors is eliminated, and non-static structures are removed. The formula is used:

[0033] ;

[0034] Get a point cloud stability rating ,in, Representative The displacement of a point, is the mean value of the displacement, and are the maximum and minimum displacement values, Representative The density value of the point, is the mean of the density values, and are the maximum and minimum density values, respectively. is the total number of data points.

[0035] The present invention is improved in that the steps further include:

[0036] S5: Based on the point cloud stability rating, static point cloud data is selected, the geometric curvature and surface fitting error of each point cloud are calculated, the structure of the 3D model is annotated based on the calculation results, the 3D structure display effect is optimized, and a score for the restoration of the site details is obtained;

[0037] The site detail restoration score includes shape restoration quality, surface fit, and detail annotation completeness.

[0038] The present invention has been improved in that the steps for obtaining the site detail restoration score are specifically as follows:

[0039] S511: Selecting static point cloud data based on the point cloud stability rating, calculating the geometric curvature value and surface fitting error value of each point cloud, performing spatial local feature analysis on the point cloud, identifying the difference between the point cloud and the fitting model, and obtaining the point cloud geometric features;

[0040] S512: Based on the geometric features of the point cloud, the point cloud is screened using an error benchmark, and the point cloud is analyzed according to its geometric form and error level to screen point clouds with high stability, thereby obtaining a point cloud classification result.

[0041] S513: Based on the point cloud classification results, the structure of the three-dimensional model is annotated, and the three-dimensional structure display effect is optimized in combination with the stability and fitting error of the point cloud to obtain a site detail restoration score.

[0042] Compared with the prior art, the advantages and positive effects of the present invention are:

[0043] In the present invention, by accurately synchronizing point cloud data and RGB images, strict verification of timestamp matching and data consistency is allowed, which significantly reduces data errors and noise, ensures the accuracy and reliability of the model, identifies structural elements and refines the analysis of dense areas of geometric structures. This not only makes the visual effect more realistic, but also provides richer data level support, providing higher quality data for archaeological research. The application of the resolution adjustment index enables the image acquisition process to be dynamically adjusted according to the complexity of the data, effectively optimizes the image quality, and reduces the burden of data processing. By screening out static structure data for model reconstruction, the stability and accuracy of the model are improved, making the application of three-dimensional models in site protection and restoration work more reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 Flow chart of the main steps proposed by the present invention;

[0045] Figure 2 This is a flow chart for obtaining the image point cloud binding coefficient in the present invention;

[0046] Figure 3 This is a flow chart for obtaining geometric structure identification indicators in the present invention;

[0047] Figure 4 This is a flowchart for obtaining the resolution adjustment index in the present invention;

[0048] Figure 5 A flowchart for obtaining a point cloud stability rating in the present invention;

[0049] Figure 6 This is a flow chart for obtaining the score of the site detail restoration degree in the present invention. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0051] In the description of the present invention, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings and are only for the convenience of describing the present invention and simplifying the description. They do not indicate or imply that the devices or elements referred to must have a specific direction, be constructed and operate in a specific direction, and therefore should not be understood as limiting the present invention. In addition, in the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0052] Example

[0053] See also Figure 1 The present invention provides a technical solution: a method for constructing a three-dimensional model of a site by integrating SLAM point cloud and image features, comprising the following steps:

[0054] S1: Acquire the depth image data and RGB image data in the site space, align the RGB image with the 3D point cloud data generated by the SLAM system according to the timestamp, remove data points with inconsistent time correspondence, calculate the ratio of the image frame rate to the number of scanning nodes, determine the valid matching data area, establish the binding relationship between the image and the point cloud, and obtain the image point cloud binding coefficient;

[0055] S2: Based on the image point cloud binding coefficient, the 3D coordinate points corresponding to the image are extracted from the point cloud data to identify the dense areas of geometric structure. The spatial direction and boundary characteristics of the area are analyzed and compared with the geometric rules to identify and classify the structural elements in the ruins and obtain the geometric structure recognition index.

[0056] S3: Based on the geometric structure recognition index, the texture direction change and edge response of each image block are evaluated. By analyzing the local complexity of the image data, the corresponding scanning resolution is adjusted. The pixel granularity and scanning frequency during the image acquisition process are refined to obtain the resolution adjustment index.

[0057] S4: Analyze the structure in the point cloud data using the resolution adjustment index, identify static and dynamic structures, continuously monitor the displacement vectors and density distribution in the point cloud data, eliminate interference caused by dynamic factors, remove non-static structures, and obtain a point cloud stability rating;

[0058] S5: Based on the point cloud stability rating, static point cloud data is selected, and the geometric curvature and surface fitting error of each point cloud are calculated. The structure of the 3D model is annotated based on the calculation results, and the 3D structure display effect is optimized to obtain the site detail restoration score.

[0059] The image point cloud binding coefficient includes data synchronization rate, matching accuracy, and data consistency. The geometric structure recognition indicators include structural discrimination, classification accuracy, and positioning accuracy. The resolution adjustment index includes resolution range, pixel adjustment ratio, and complexity response information. The point cloud stability rating specifically refers to static verification degree, dynamic exclusion efficiency, and environmental interference identification information. The site detail restoration score includes shape recovery quality, surface fit, and detail annotation completeness.

[0060] See also Figure 2 , the steps for obtaining the image point cloud binding coefficient are as follows:

[0061] S111: Acquire depth image data and RGB image data in the site space, align the RGB image with the three-dimensional point cloud data generated by the SLAM system according to the timestamp, remove RGB image frames and point cloud frame data with inconsistent timestamps, and calculate the ratio of the image frame rate to the number of point cloud nodes to obtain the image point cloud frame rate ratio;

[0062] First, align the RGB image with the three-dimensional point cloud data generated by the SLAM system. Ensure that each frame of image data is synchronized with the corresponding point cloud data through timestamp matching, and eliminate those image frames and point cloud frame data with inconsistent timestamps to ensure data consistency. Then, calculate the ratio between each pair of image frames and point cloud nodes. By analyzing the relationship between the image frame rate and the number of point cloud nodes, the image point cloud frame rate ratio is obtained. Assuming that in a specific area, the alignment timestamp of RGB image data and point cloud data is 13:01:30, the image frame rate is 15FPS, and the number of point cloud nodes is 1500, in this area, by calculating the ratio of image frame rate to the number of point cloud nodes (15 / 1500=0.01), the image point cloud frame rate ratio can be obtained. This ratio reflects the correspondence between image data and point cloud data, providing a basic basis for subsequent data screening, and obtaining the image point cloud frame rate ratio.

[0063] S112: Based on the image point cloud frame rate ratio, determine whether it is within the node matching ratio baseline range. If it is out of range, mark the data area of the ratio, and make a joint judgment based on the point cloud density and image frame rate continuity to select data segments that meet the time continuity and point cloud density requirements to obtain the qualified matching area index;

[0064] After obtaining the image point cloud frame rate ratio, it is further determined whether the ratio is within the baseline range of the node matching ratio. The setting of the baseline range is adjusted according to experimental data and actual application scenarios. For example, a threshold range is set, assuming it is 0.005 to 0.02. If the ratio exceeds this range, the data area is marked as unqualified and eliminated. Further combined with the point cloud density and image frame rate continuity, a joint judgment is made. For example, for a certain area, the point cloud density is low and the image frame rate changes discontinuously, then the area is judged as an unqualified area. Through screening, only data segments that meet the requirements of time continuity and point cloud density can be retained. For example, the point cloud density in a data segment is 1200 nodes per cubic meter and the image frame rate is 12FPS. If the density value and frame rate meet the set threshold conditions (for example: the point cloud density is greater than 1000 nodes per cubic meter and the image frame rate is greater than 10FPS), this part of the data passes the screening and obtains the qualified matching area indicator.

[0065] S113: Based on the qualified matching area index, a mapping relationship between the image frame and the point cloud node is constructed using the formula:

[0066] ;

[0067] Obtain the degree of binding between the image features and the point cloud, and obtain the image point cloud binding coefficient, where: Representative The image point cloud binding coefficient of the region represents the strength of association between the image and the corresponding point cloud. Representative In the region The number of point cloud nodes corresponding to the image features, Representative In the region Frame number of image frames, Representative The average number of image frames per region, Representative The total number of point cloud matching nodes in the region, Representative The time span corresponding to the image frame in each region is the length of time during which the image data is collected. Representative In the region The spatial density of point cloud nodes, Representative The average spatial density of point cloud nodes in the region, It is a region The total number of image features in , It is a region The total number of image frames in , It is a region The total number of midpoint cloud nodes;

[0068] If you get the following data, area The total number of image features in is 3 (3 image features are extracted by SIFT), No. In the region The number of point cloud nodes corresponding to each image feature is 120, 100, and 130 respectively. area The total number of image frames is 4 frames. No. In the region The number of image frames is 510, 490, 500, and 520 respectively. No. The average number of image frames in a region is calculated as , No. The total number of point cloud matching nodes in the region is assumed to be 1000. No. The time span corresponding to each regional image frame is 4 seconds. No. The total number of point cloud nodes in the region is 3. No. Region No. The spatial density of point cloud nodes (points / cubic meter) is 20, 25, and 22 respectively. No. The average point cloud node spatial density of the region is calculated as , calculate the molecular part:

[0069] ;

[0070] ;

[0071] ;

[0072] ;

[0073] ;

[0074] Calculate the denominator:

[0075] ;

[0076] ;

[0077] ;

[0078] Substitute into the formula to calculate the binding coefficient:

[0079] ;

[0080] The results show that the region The image point cloud binding coefficient is 0.0931, which is within the normal range (the binding coefficient reference range is ), indicating that there is a strong matching relationship between the image and the point cloud in this area, which can be used to build a fusion model in the future.

[0081] See also Figure 3 , the steps for obtaining the geometric structure recognition index are as follows:

[0082] S211: Based on the image point cloud binding coefficient, extract the 3D coordinate points corresponding to the image, calculate the spatial density, voxel distribution and depth continuity of the point cloud, determine the high-density area in the point cloud, and analyze the density difference and voxel number in the area to obtain the aggregation degree of the high-density area;

[0083] There is a set of image data and point cloud data, where the image data comes from drones or remote sensors, and the point cloud data is obtained through LiDAR technology. By establishing a spatial mapping relationship between the image and the point cloud, each pixel in the image can be mapped to a point in three-dimensional space. The core of this step is to ensure that the image points and point cloud data can be accurately connected, so that each point in the point cloud can correspond to an accurate three-dimensional coordinate in the image. Technologies such as back projection can be used. Then, the point cloud data is further analyzed by calculating the spatial density, voxel distribution and depth continuity of the point cloud. The spatial density can be evaluated by counting the number of points in a unit volume, which helps to judge the density of the point cloud. For example, if there are more points in a certain volume unit, it means that the density of the area is higher. Voxel distribution refers to the grid division of the point cloud in space, and the distribution of points in each voxel is analyzed. , it can be judged whether there is obvious aggregation phenomenon in the area. Depth continuity involves the spatial continuity between points in the point cloud, which is used to identify whether there are obvious blanks or faults. Through the steps, the high-density area aggregation degree of the area can be accurately obtained and calculated. For example, consider the point cloud data of an area. Suppose that 100 points are counted in a small grid and 50 points are counted in another area. The spatial density of the former is obviously higher than that of the latter. This density difference can be directly used to measure the aggregation degree of the area. The aggregation degree is calculated based on the density difference of the points in each grid, and the spatial characteristics of the overall high-density area are further obtained. Finally, by analyzing the spatial aggregation of high-density areas, the high-density areas in the point cloud can be identified and classified, and the high-density area aggregation degree of the area can be calculated. This indicator reflects the spatial distribution characteristics of point cloud data and helps identify areas with different densities.

[0084] S212: Based on the high-density area aggregation degree, combined with the tangent plane normal vector and boundary coordinate deviation of the point cloud in the local area, the formula is used:

[0085] ;

[0086] Calculation area Structural orientation differentiation , we get the boundary directional stability, where Indicates area Neidi The Euclidean distance between the point and the center of the region, Indicates a point The boundary direction coordinate difference, that is, the offset between the point and the predetermined boundary direction, Indicates the Point in area The modulus of the tangent plane normal vector in reflects the orientation or direction characteristics of the point on the tangent plane. Indicates a point The offset in the direction of the boundary, Indicates area The number of boundary points within ;

[0087] There are two points and ,point , : Euclidean distance from the center of the region , boundary direction coordinate difference , the modulus of the tangent plane normal vector , the offset in the direction of the boundary ;point : Euclidean distance from the center of the region , boundary direction coordinate difference , the modulus of the tangent plane normal vector , the offset in the direction of the boundary , substitute the formula to calculate the structural direction differentiation , calculate the first summation :

[0088] ;

[0089] Calculate the second summation :

[0090] ;

[0091] Calculate the structural orientation differentiation

[0092] ;

[0093] Get the structural orientation differentiation ,The value reflects the structural difference and boundary stability of the ,region. A higher differentiation value indicates that the structure of the ,region has a large change, which is caused by the uneven distribution of the ,point cloud or the irregularity of the boundary.

[0094] S213: Invoke boundary directional stability to classify regional structures, number the structural types of different regions according to boundary characteristics, compare the overlap between the classified regions and geometric rules, identify structural elements in the ruins, and obtain geometric structure identification indicators;

[0095] The boundary directional stability calculated in the previous step is called and the regional structure is classified according to this stability. First, by comparing the boundary directional stability of different regions, regions with significant differences are identified. The different regions are further classified according to their boundary characteristics. For example, some regions have relatively smooth boundaries, while other regions have obvious fractures or irregular boundaries. By classifying the boundary characteristics, the structural elements in the point cloud can be numbered and identified. Subsequently, by comparing the classified regions with the preset geometric rules, an overlap analysis is performed to determine which regions conform to the established structural pattern and identify structural elements with historical or cultural value in the region. During the overlap analysis, by comparing the classified regions with a set of geometric rules (such as building outlines, site outlines, etc.), structural elements that conform to the rules can be identified. Elements are ruins of ancient buildings, parts of infrastructure, or other structures with special forms. Finally, by numbering and classifying the regions, a geometric structure recognition index can be obtained. This index reflects the identification results of the structural elements in the region and can determine the spatial layout and type of each structure in the site, providing a basis for subsequent research.

[0096] See also Figure 4 , the steps for obtaining the resolution adjustment index are as follows:

[0097] S311: Evaluate the texture direction change and edge response of each image block based on the geometric structure recognition index, perform local feature analysis on the image data, extract the amount of texture direction change and edge response degree, and obtain texture change and response degree;

[0098] According to the geometric structure recognition index, the texture direction change and edge response of each image block are evaluated. First, multiple image blocks are selected in the image, and the texture direction change in each block is calculated. Pixel gradient, texture co-occurrence matrix and other methods are usually used to measure the direction and intensity differences of texture. In actual operation, if the texture change in an image block is large, it means that the edge information of the area is complex, such as the rapid change of object boundaries or texture patterns in the image. For the evaluation of edge response, an edge detection algorithm such as the Sobel operator is used to calculate the edge response intensity in each image block. By analyzing the response value, the edge clarity in the area is obtained. For example, if the edge response intensity of an image is high, it means that the area has a clear structural boundary, otherwise it means that the area is relatively smooth. The texture change and edge response are combined for analysis to obtain the texture change amount and edge response of the image block, providing a basis for subsequent processing.

[0099] S312: Based on the texture change and responsiveness, the complexity between the different areas is compared to determine whether the scanning resolution needs to be adjusted. The formula is:

[0100] ;

[0101] Get resolution adjustment parameters ,in, It stands for average edge response, which indicates the average value of edge response intensity in the evaluated image block. It is used to measure edge clarity and the obviousness of image edges. is the edge response threshold, which is used to determine whether the edge response in the image block exceeds the standard. Indicates the degree of change in texture direction, which measures the degree of change in texture direction in the image block. is the statistical standard deviation of the variation in texture direction;

[0102] By analyzing the texture direction changes and edge response strength of different regions, we can determine which regions have higher complexity. In actual implementation, if the texture changes in some regions are large and the edge response is strong, it means that the image information in the region is more complex and requires a higher resolution to accurately capture the details. For example, the following values are available: the average edge response of the image block , edge response threshold , standard deviation of texture variation , the statistical standard deviation of texture direction variation , substitute the value into the formula for calculation:

[0103] ;

[0104] This result indicates that the image blocks are more complex, which means that it is necessary to obtain a finer image by increasing the scanning resolution.

[0105] S313: Based on the resolution adjustment parameter, the corresponding scanning resolution is adjusted, and the pixel granularity and scanning frequency during the acquisition process are refined to obtain a resolution adjustment index;

[0106] Conclusion , this value indicates that the details of the image block are more complex and the scanning resolution needs to be improved. In order to adjust the resolution, there are two main ways, one is to increase the pixel density of the scan, and the other is to increase the scanning frequency. If the current scanning resolution is 200DPI, and according to the adjustment parameter calculation, it is recommended to increase it to a higher resolution, in order to ensure finer image capture, you can choose to increase the resolution to 300DPI, that is, scan more pixels per inch, or increase the scanning frequency, for example, from 5 times per second to 7 times per second, so as to ensure full collection of image details. Improving the resolution or increasing the scanning frequency can effectively ensure that the image quality is improved, especially when processing complex textures and areas with strong edge response, which can avoid the loss of image details, thereby improving the accuracy and expressiveness of the image.

[0107] See also Figure 5 ,The steps for obtaining the point cloud stability rating are as follows:

[0108] S411: Analyze each type of structure in the point cloud data using the resolution adjustment index. By analyzing the change in point cloud resolution, identify the characteristics of the differential structures, determine the difference between static and dynamic structures, and obtain static and dynamic structure recognition results.

[0109] Each type of structure in the point cloud data is analyzed based on the resolution adjustment index. The resolution adjustment index is used to identify changes in accuracy and detail in the point cloud data, thereby determining the key characteristics of each type of structure. Structures are buildings, bridges, or other fixed objects, as well as specific areas in natural scenery. During the analysis process, the spatial distribution of the point cloud data is first evaluated in detail. By calculating the distribution changes of each point in space, relatively stable areas and areas with large dynamic changes are identified, and then static and dynamic structures are identified. For example, if the point cloud data of a building is analyzed, the various parts of the building should remain relatively stable, while surrounding trees or vehicles will have large displacements. The resolution adjustment index can help identify structural differences. During identification, dynamic factors (such as wind and people) need to be distinguished from static structures (such as buildings, roads, rocks, etc.). The generated static / dynamic structure recognition results can clearly classify information. The analysis results assign different stability labels to each structure category, confirming which areas are static structures and which areas are dynamic structures.

[0110] S412: Based on the results of static and dynamic structure recognition, monitor the dynamic changes of point cloud data, filter the data in stable areas according to the displacement and density fluctuation range, eliminate the interference of dynamic factors, and eliminate non-static structures. The formula is used:

[0111] ;

[0112] Get a point cloud stability rating ,in, Representative The displacement of a point, is the mean value of the displacement, and are the maximum and minimum displacement values, respectively. Representative The density value of each point, that is, the density information, is the mean of the density values, and are the maximum and minimum density values, respectively. is the total number of data points;

[0113] There are two monitoring points, monitoring point 1 and monitoring point 2, and the following data are measured:

[0114] Monitoring point 1:

[0115] Displacement ( ) = 0.08m, density value ( ) = 450kg / m³;

[0116] Monitoring point 2:

[0117] Displacement ( ) = 0.15m, density value ( ) = 460kg / m³;

[0118] Calculate the displacement ( )’s maximum, minimum, and mean values:

[0119] m, m, m;

[0120] Calculate the density value ( )’s maximum, minimum, and mean values:

[0121] kg / m³, kg / m³, kg / m³;

[0122] For monitoring point 1:

[0123] ;

[0124] ;

[0125] For monitoring point 2:

[0126] ;

[0127] ;

[0128] Calculate the stability score, for the first point:

[0129] ;

[0130] For the second point:

[0131] ;

[0132] Stability Rating:

[0133] ;

[0134] If the stability score threshold is 0.6, the result is , the result is less than the threshold value 0.6, so the point cloud data is considered unstable, so the point cloud data is judged to be an unstable area. , then the point cloud is considered stable if , the point cloud is considered unstable.

[0135] See also Figure 6 The specific steps for obtaining the site detail restoration score are as follows:

[0136] S511: Based on the point cloud stability rating, select static point cloud data, calculate the geometric curvature value and surface fitting error value of each point cloud, perform spatial local feature analysis on the point cloud, identify the difference between the point cloud and the fitting model, and obtain the point cloud geometric features;

[0137] The position data of each point cloud point is collected, and the geometric curvature of each point is obtained using computational geometry methods. For example, assuming the coordinates of a group of points in the point cloud data are (x1, y1, z1), (x2, y2, z2), and (x3, y3, z3), the three-point method is used to calculate the local geometric curvature. The calculation method is: curvature = 1 / (r), where r is the curvature radius between each three points. By analyzing the distance between the points, it is possible to determine whether the point belongs to a high curvature area, and then identify the difference with the fitted model. For example, if the curvature value is greater than the set threshold, it means that the point is in an unstable area. Then, the surface fitting method is used to calculate the surface fitting error. The error with the fitted model is used to determine the difference between the point cloud and the model. If the error value is large, it indicates that the point cloud area is unstable. The obtained point cloud geometric features represent the stability of the point cloud. In this process, the curvature threshold and the fitting error benchmark can be set to screen out point cloud areas that meet the stability requirements. According to the above calculation process, the obtained point cloud geometric features reflect the stability and accuracy of the point cloud.

[0138] S512: Based on the geometric features of the point cloud, the point cloud is screened using an error benchmark, and the point cloud is analyzed based on its geometric form and error level to screen the point cloud with high stability and obtain the point cloud classification result;

[0139] First, the error baseline is determined, usually a value within a set of standard deviations. Based on the difference between the surface fitting error value and the standard error, each point is analyzed using a point-by-point comparison method. For example, assuming the fitting error baseline is 0.01, when the error value of the point cloud is greater than this baseline, it can be considered a point cloud with a large error. Otherwise, it is considered a point cloud with good stability. In this way, point clouds that meet the requirements are screened out and classified. For example, if the error is greater than 0.01, the point cloud is classified as unstable, while the point cloud with an error less than 0.01 is classified as stable. The geometric shape of the point cloud needs to be considered during the classification process. For example, a point cloud with a regular and flat shape is considered to have good stability, while a point cloud with a complex shape and large error is considered to have poor stability. The classification result can directly affect the subsequent modeling accuracy and stability. Therefore, ensuring the accuracy of the screening is crucial to the entire process. The obtained point cloud classification result, which selects stable point clouds based on error and shape, accurately reflects the reliability and applicability of the point cloud.

[0140] S513: Based on the point cloud classification results, the structure of the 3D model is annotated. The 3D structure display effect is optimized based on the stability and fitting error of the point cloud to obtain a score for the restoration of the site details.

[0141] Based on the point cloud classification results, point clouds with higher stability and smaller fitting errors are first annotated. The annotation process is to find the part that best matches the three-dimensional model by extracting features from the point cloud. For example, if the point cloud has good stability and small errors, it can be directly used for detailed reconstruction of the three-dimensional model. For point clouds with large errors, this part needs to be optimized or deleted. When optimizing the display effect, smoothing technology can be used to remove unstable point cloud data and display stable point clouds with high precision to enhance the detail restoration of the model. Through this annotation and optimization, the detail restoration of the site is obtained. This value represents the accuracy of the three-dimensional model detail restoration based on the existing point cloud data.

[0142] The above are merely preferred embodiments of the present invention and do not limit the present invention in any other form. Any technician familiar with the profession may use the technical content disclosed above to change or modify it into an equivalent embodiment with equivalent changes and apply it to other fields. However, any simple modification, equivalent change and modification made to the above embodiment based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall still fall within the scope of protection of the technical solution of the present invention.

Claims

1. A method for constructing a three-dimensional model of a site by integrating SLAM point cloud and image features, characterized in that: The following steps are involved: S1: Acquire the depth image data and RGB image data in the site space, align the RGB image with the 3D point cloud data generated by the SLAM system according to the timestamp, calculate the ratio of the image frame rate to the number of scanning nodes, determine the effective matching data area, establish the binding relationship between the image and the point cloud, and obtain the image point cloud binding coefficient; S2: Based on the image point cloud binding coefficient, identify dense areas of geometric structure, analyze the spatial direction and boundary characteristics of the area, compare them with geometric rules, identify and classify structural elements in the ruins, and obtain geometric structure recognition indicators; S3: Evaluate the texture direction change and edge response of each image block based on the geometric structure recognition index, adjust the corresponding scanning resolution by analyzing the local complexity of the image data, and refine the pixel granularity and scanning frequency during the image acquisition process to obtain a resolution adjustment index; S4: Using the resolution adjustment index, static and dynamic structures are identified, displacement vectors and density distributions in the point cloud data are continuously monitored, interference caused by dynamic factors is eliminated, non-static structures are removed, and a point cloud stability rating is obtained.

2. The method for constructing a three-dimensional model of a site by integrating SLAM point cloud and image features according to claim 1, wherein: The image point cloud binding coefficient includes data synchronization rate, matching accuracy, and data consistency. The geometric structure recognition index specifically includes structural discrimination, classification accuracy, and positioning accuracy. The resolution adjustment index includes resolution range, pixel adjustment ratio, and complexity response information. The point cloud stability rating specifically refers to static verification degree, dynamic exclusion efficiency, and environmental interference recognition information.

3. The method for constructing a three-dimensional model of a site by integrating SLAM point cloud and image features according to claim 1, wherein: The steps for obtaining the image point cloud binding coefficient are as follows: S111: Acquire depth image data and RGB image data in the site space, align the RGB image with the three-dimensional point cloud data generated by the SLAM system according to the timestamp, remove RGB image frames and point cloud frame data with inconsistent timestamps, and calculate the ratio of the image frame rate to the number of point cloud nodes to obtain the image point cloud frame rate ratio; S112: Based on the image point cloud frame rate ratio, determine whether it is within the node matching ratio baseline range. If it is outside the range, mark the data area of the ratio, and perform a joint judgment based on the point cloud density and image frame rate continuity to select data segments that meet the time continuity and point cloud density requirements to obtain a qualified matching area indicator. S113: Constructing a mapping relationship between the image frame and the point cloud node based on the qualified matching area index, using the formula: ； Obtain the degree of binding between the image features and the point cloud, and obtain the image point cloud binding coefficient, where: Representative The image point cloud binding coefficient of the region, Representative In the region The number of point cloud nodes corresponding to the image features, Representative In the region Frame number of image frames, Representative The average number of image frames per region, Representative The total number of point cloud matching nodes in the region, Representative The time span corresponding to the image frame in each region, Representative In the region The spatial density of point cloud nodes, Representative The average spatial density of point cloud nodes in the region, It is a region The total number of image features in , It is a region The total number of image frames in It is a region The total number of point cloud nodes.

4. The method for constructing a three-dimensional model of a site by integrating SLAM point cloud and image features according to claim 1, wherein: The steps for obtaining the geometric structure recognition index are specifically as follows: S211: Based on the image point cloud binding coefficient, extract the three-dimensional coordinate points corresponding to the image, calculate the spatial density, voxel distribution and depth continuity of the point cloud, determine the high-density area in the point cloud, and analyze the density difference and voxel number in the area to obtain the aggregation degree of the high-density area; S212: Based on the high-density region aggregation degree, combined with the tangent plane normal vector and boundary coordinate deviation of the point cloud in the local region, the formula is used: ； Calculation area Structural orientation differentiation , we get the boundary directional stability, where Indicates area Neidi The Euclidean distance between the point and the center of the region, Indicates a point The boundary direction coordinate difference, that is, the offset between the point and the predetermined boundary direction, Indicates the Point in area The modulus of the tangent plane normal vector in reflects the orientation or direction characteristics of the point on the tangent plane. Indicates a point The offset in the direction of the boundary, Indicates area The number of boundary points within ; S213: Calling the boundary direction stability, classifying the regional structure, numbering the structural types of the different areas according to the boundary characteristics, comparing the overlap between the classified areas and the geometric rules, identifying the structural elements in the ruins, and obtaining the geometric structure identification index.

5. The method for constructing a three-dimensional model of a site by integrating SLAM point cloud and image features according to claim 1, wherein: The steps for obtaining the resolution adjustment index are specifically as follows: S311: Evaluate the texture direction change and edge response of each image block based on the geometric structure recognition index, perform local feature analysis on the image data, extract the amount of texture direction change and the edge response degree, and obtain texture change and response degree; S312: Based on the texture change and responsiveness, the complexity between the different regions is compared to determine whether the scanning resolution needs to be adjusted, using the formula: ； Get resolution adjustment parameters ,in, represents the average edge response, is the edge response threshold, Indicates the degree of change in texture direction, is the statistical standard deviation of the variation in texture direction; S313: Based on the resolution adjustment parameter, the corresponding scanning resolution is adjusted, and the pixel granularity and scanning frequency in the acquisition process are refined to obtain a resolution adjustment index.

6. The method for constructing a three-dimensional model of a site by integrating SLAM point cloud and image features according to claim 1, wherein: The steps for obtaining the point cloud stability rating are as follows: S411: Analyzing each type of structure in the point cloud data using the resolution adjustment index, identifying features of differential structures by analyzing point cloud resolution changes, determining the difference between static and dynamic structures, and obtaining static and dynamic structure recognition results; S412: Based on the static and dynamic structure identification results, the dynamic changes of the point cloud data are monitored. The data in the stable area are screened for the displacement and density fluctuation ranges, the interference of dynamic factors is eliminated, and non-static structures are removed. The formula is used: ； Get a point cloud stability rating ,in, Representative The displacement of a point, is the mean value of the displacement, and are the maximum and minimum displacement values, Representative The density value of the point, is the mean of the density values, and are the maximum and minimum density values, respectively. is the total number of data points.

7. The method for constructing a three-dimensional model of a site by integrating SLAM point cloud and image features according to claim 1, wherein: The steps also include: S5: Based on the point cloud stability rating, static point cloud data is selected, the geometric curvature and surface fitting error of each point cloud are calculated, the structure of the 3D model is annotated based on the calculation results, the 3D structure display effect is optimized, and a score for the restoration of the site details is obtained; The site detail restoration score includes shape restoration quality, surface fit, and detail annotation completeness.

8. The method for constructing a three-dimensional model of a site by integrating SLAM point cloud and image features according to claim 7, wherein: The specific steps for obtaining the site detail restoration score are as follows: S511: Selecting static point cloud data based on the point cloud stability rating, calculating the geometric curvature value and surface fitting error value of each point cloud, performing spatial local feature analysis on the point cloud, identifying the difference between the point cloud and the fitting model, and obtaining the point cloud geometric features; S512: Based on the geometric features of the point cloud, the point cloud is screened using an error benchmark, and the point cloud is analyzed according to its geometric form and error level to screen point clouds with high stability, thereby obtaining a point cloud classification result. S513: Based on the point cloud classification results, the structure of the three-dimensional model is annotated, and the three-dimensional structure display effect is optimized in combination with the stability and fitting error of the point cloud to obtain a site detail restoration score.

Images