Intelligent Detection Method and System for Surface Defects of Ancient City Walls Based on 3D Reconstruction

Abstract

This invention belongs to the fields of cultural relic protection and computer vision technology. Addressing the problems of low efficiency, poor accuracy, and lack of spatial positioning in traditional methods for detecting defects in ancient city walls, this invention discloses an intelligent detection method and system for surface defects of ancient city walls based on 3D reconstruction. The method includes: processing the texture information of a loaded 3D reconstructed model of the ancient city wall using a multi-strategy texture restoration method; analyzing the model's geometric structure to determine the wall's orientation and dimensional parameters; generating virtual camera positions based on geometric features and a meshed serpentine scanning strategy; rendering RGB and depth images at each virtual camera position; performing defect detection using a YOLO model; mapping the 2D detection results to 3D space to generate 3D defect marker color blocks; and fusing multi-viewpoint results to output the detection results of surface defects on the ancient city wall. This invention achieves efficient, accurate, and automated detection of defects in ancient city walls, providing effective technical support for cultural relic protection.

Description

Technical Field

[0001] This invention belongs to the fields of cultural relic protection and computer vision technology, specifically relating to an intelligent detection method and system for surface defects of ancient city walls based on three-dimensional reconstruction. Background Technology

[0002] Ancient city walls, as important historical and cultural heritage, carry rich historical information and cultural value. However, due to long-term exposure to the natural environment, they inevitably suffer from various erosions such as weathering, cracking, and peeling, seriously threatening the safety of the cultural relics themselves. Timely detection and accurate assessment of these damages are of great significance for developing scientific protection and restoration plans. Traditional detection of ancient city wall defects mainly relies on on-site visual inspection and manual recording by professionals. This method has the following significant problems: (1) Low efficiency: For city walls that stretch for several kilometers or even tens of kilometers, manual inspection is time-consuming and labor-intensive. A complete inspection of a section of the city wall often takes several days or even weeks; (2) High subjectivity: Affected by the experience and judgment of the inspectors, there is a lack of unified quantitative standards. Different people may have different evaluations of the same defect; (3) Safety hazards: It is difficult to approach high or dangerous areas, and there are safety risks such as people falling; (4) Lack of digital records: Traditional methods are difficult to establish effective digital archives, which is not conducive to long-term monitoring and historical tracing of defects; (5) Inaccurate spatial positioning: The description of the defect location recorded manually is vague and it is difficult to accurately locate the three-dimensional spatial coordinates.

[0003] In recent years, with the rapid development of computer vision and deep learning technologies, image-based disease detection methods have gradually emerged. However, existing two-dimensional image detection methods have the following limitations: (1) lack of spatial information: they can only identify diseases on a two-dimensional plane and cannot provide the three-dimensional spatial location of diseases; (2) single perspective: a single image can only cover a local area and it is difficult to obtain the overall distribution of diseases; (3) occlusion problem: due to the limitation of the shooting angle, some areas may be occluded and cannot be detected. Summary of the Invention

[0004] This invention provides an intelligent detection method and system for surface defects of ancient city walls based on three-dimensional reconstruction, aiming to solve the technical problems of low efficiency, poor accuracy, lack of spatial positioning capability, and inability of existing two-dimensional detection methods to provide three-dimensional spatial information in traditional ancient city wall defect detection methods.

[0005] The first aspect of this invention provides an intelligent detection method for surface defects of ancient city walls based on three-dimensional reconstruction, comprising the following steps: The obtained 3D reconstruction model of the ancient city wall is loaded and subjected to multi-strategy texture restoration processing. The geometric structural features of the processed 3D reconstruction model are analyzed to determine the main extension direction, thickness, length and height of the ancient city wall. Based on the determined main extension direction, thickness, length and height of the ancient city wall, and combined with a gridded serpentine scanning strategy, multiple virtual camera positions are generated. The processed 3D reconstruction model is rendered at each virtual camera position to obtain multiple RGB images and their corresponding multiple depth images; The pre-trained YOLO object detection model was used to detect diseases in each acquired RGB image to obtain the coordinates of the two-dimensional detection box and the disease category. Based on the corresponding depth image, the coordinates of each two-dimensional detection box and the disease category are mapped to three-dimensional space through coordinate transformation, and three-dimensional disease marker color blocks are generated on the surface of the processed three-dimensional reconstruction model. By integrating all the coordinates of the two-dimensional detection frames and the disease categories, as well as the generated three-dimensional disease marker color blocks, the detection results of the surface diseases of the ancient city wall are output.

[0006] Preferably, the process of loading the acquired 3D reconstruction model of the ancient city wall and performing multi-strategy texture restoration includes the following strategies: Strategy 1: Use the TriangleModel data structure to load the acquired 3D reconstruction model and associated texture information to obtain the processed 3D reconstruction model; Strategy 2: If Strategy 1 fails, the traditional TriangleMesh method is used to load the obtained 3D reconstruction model and associated texture information. When texture information is missing, the texture files in the same directory are automatically searched. After the texture files are found, a texture mapping relationship is established to obtain the processed 3D reconstruction model.

[0007] Preferably, when texture information is missing, the automatic search for texture files in the same directory, the establishment of texture mapping relationships after finding the texture files, and the resulting processed 3D reconstruction model include the following steps: When texture information is missing, automatically search for texture files in the same directory. Once a texture file is found, perform the following operations: If the missing texture information is UV coordinates, the missing UV coordinates are generated based on the wall plane features and added to the texture information to obtain the processed 3D reconstruction model. If the missing texture information is the entire texture and vertex colors on the texture image, the dominant hue is extracted from the texture image and combined with the vertex height information to generate the vertex colors. The dominant hue and vertex colors are then added to the texture information to obtain the processed 3D reconstruction model.

[0008] Preferably, the step of analyzing the geometric structural features of the processed 3D reconstructed model to determine the main extension direction, thickness, and height of the ancient city wall includes the following steps: Calculate the axial bounding box of the processed 3D reconstructed model to obtain the minimum and maximum boundary points of the 3D reconstructed model; Based on the obtained minimum and maximum boundary points, calculate the dimension vector of the ancient city wall. Based on the calculated dimension vector of the ancient city wall, the main extension direction, thickness, length, and height of the ancient city wall are determined.

[0009] Preferably, the process of generating multiple virtual camera positions based on the determined main extension direction, thickness, length, and height of the ancient city wall, combined with a gridded serpentine scanning strategy, includes the following steps: Based on the determined thickness of the ancient city wall, the distance between the virtual camera and the wall surface is calculated; Multiple horizontal sampling points are arranged in the even-numbered rows of the vertical direction in a forward direction, and a serpentine scanning strategy is adopted to arrange multiple horizontal sampling points in the odd-numbered rows of the vertical direction in a reverse direction. For all generated horizontal sampling points, generate grids in both the horizontal and vertical directions to obtain multiple grids; For each generated grid, its normalized coordinates are calculated, resulting in multiple normalized coordinates; Multiple virtual camera positions are generated based on the distance between the virtual camera and the wall, the main extension direction, length and height of the ancient city wall, and multiple normalized coordinates.

[0010] Preferably, the step of rendering the processed 3D reconstructed model at each virtual camera location to obtain multiple RGB images and their corresponding multiple depth images includes the following steps: Set the internal and external parameters of the virtual camera at each virtual camera location; The virtual camera's intrinsic and extrinsic parameters include an extrinsic parameter matrix and an intrinsic parameter matrix; Based on the set intrinsic and extrinsic parameters of the virtual camera, the processed 3D reconstruction model is rendered graphically at each virtual camera position. Based on multiple rendered 3D reconstruction models, multiple RGB images and their corresponding depth images are obtained.

[0011] Preferably, the step of mapping the coordinates of each two-dimensional detection box and the disease category to three-dimensional space through coordinate transformation based on the corresponding depth image, and generating disease marker color blocks on the surface of the processed three-dimensional reconstructed model, includes the following steps: Based on the corresponding depth image, obtain the depth value corresponding to the coordinates of each two-dimensional detection box; Based on the acquired depth values, the coordinates of each two-dimensional detection box are converted into camera three-dimensional coordinates through the inverse transformation of the intrinsic parameter matrix; The camera's 3D coordinates are converted to world coordinates through the inverse transformation of the extrinsic parameter matrix; Calculate the size of the three-dimensional disease marker color block based on the size of each two-dimensional detection frame; Based on the outer normal vector of the wall surface, combined with the three-dimensional dimensions, and colored according to the disease type, three-dimensional disease marker color blocks are generated on the surface of the processed three-dimensional reconstruction model.

[0012] Preferably, the process of fusing all two-dimensional detection frame coordinates and disease categories, along with the generated three-dimensional disease marker color blocks, to output the surface disease detection results of the ancient city wall includes the following steps: Based on the coordinates of each two-dimensional detection box and the disease category, draw bounding boxes of different colors and disease category labels on the corresponding RGB image to generate an labeled two-dimensional detection image; All labeled 2D detection images are stitched together in serpentine scanning path order to form a 2D panoramic image; By combining the generated two-dimensional panoramic image and three-dimensional disease marker color blocks, the detection results of surface diseases of the ancient city wall are output.

[0013] The second aspect of this invention provides an intelligent detection system for surface defects of ancient city walls based on three-dimensional reconstruction, implemented using the aforementioned intelligent detection method for surface defects of ancient city walls based on three-dimensional reconstruction, comprising: The 3D model preprocessing module is used to load the acquired 3D reconstruction model of the ancient city wall, perform multi-strategy texture restoration processing, analyze the geometric structural features of the processed 3D reconstruction model, and determine the main extension direction, thickness, length and height of the ancient city wall. The intelligent viewpoint planning module is used to generate multiple virtual camera positions based on the determined main extension direction, thickness, length and height of the ancient city wall, combined with a gridded serpentine scanning strategy. The virtual view rendering module is used to render the processed 3D reconstructed model at each virtual camera position to obtain multiple RGB images and their corresponding multiple depth images. The disease intelligent detection module is used to perform disease detection on each acquired RGB image using a pre-trained YOLO object detection model, and obtain the coordinates of the two-dimensional detection box and the disease category. The 3D mapping and annotation module is used to map the coordinates of each 2D detection box and the disease category to 3D space based on the corresponding depth image through coordinate transformation, and generate 3D disease marker color blocks on the surface of the processed 3D reconstruction model. The result fusion output module is used to fuse all the coordinates of the two-dimensional detection boxes and the disease categories, as well as the generated three-dimensional disease marker color blocks, and output the detection results of the surface diseases of the ancient city wall.

[0014] Preferably, the 3D model preprocessing module includes: The model loading unit is used to load the acquired 3D reconstruction model of the ancient city wall; The texture restoration processing unit is used to perform multi-strategy texture restoration processing on the loaded 3D reconstruction model to obtain the processed 3D reconstruction model. The geometric analysis unit is used to analyze the geometric structural features of the processed 3D reconstruction model and determine the main extension direction, thickness, length and height of the ancient city wall.

[0015] Preferably, the intelligent viewpoint planning module includes: The parameter calculation unit is used to calculate the distance between the virtual camera and the wall surface based on the determined thickness of the ancient city wall. The sampling point optimization setting unit arranges multiple horizontal sampling points in the even-numbered rows of the vertical direction in a forward direction, and adopts a serpentine scanning strategy to arrange multiple horizontal sampling points in the odd-numbered rows of the vertical direction in a reverse direction. The mesh generation unit is used to generate meshes in the horizontal and vertical directions for all generated horizontal sampling points, resulting in multiple meshes. The normalized coordinate calculation unit is used to calculate the normalized coordinates of each generated grid, resulting in multiple normalized coordinates; The virtual camera position generation unit is used to generate multiple virtual camera positions based on the distance of the virtual camera from the wall, the main extension direction, length and height of the ancient city wall, and multiple normalized coordinates.

[0016] Preferably, the virtual view rendering module includes: The virtual camera parameter setting unit is used to set the internal and external parameters of the virtual camera at each virtual camera location. The virtual camera's intrinsic and extrinsic parameters include an extrinsic parameter matrix and an intrinsic parameter matrix; The graphics rendering unit is used to perform graphics rendering on the processed 3D reconstructed model at each virtual camera position based on the set intrinsic and extrinsic parameters of the virtual camera. The image capture unit is used to obtain multiple RGB images and their corresponding depth images based on multiple rendered 3D reconstruction models.

[0017] Preferably, the intelligent disease detection module includes: The YOLO inference unit is used to perform forward inference on each acquired RGB image using a pre-trained YOLO object detection model to obtain an inferred image. The post-processing unit is used to perform non-maximum suppression on the inference image and filter to obtain the disease detection results; The annotation drawing unit is used to draw detection boxes and disease labels on the RGB image based on the disease detection results, and obtain the coordinates of the two-dimensional detection boxes and the disease category.

[0018] Preferably, the 3D mapping annotation module includes: The depth value calculation unit is used to obtain the depth value corresponding to the coordinates of each two-dimensional detection box based on the corresponding depth image; The coordinate transformation unit is used to convert the coordinates of each two-dimensional detection box into camera three-dimensional coordinates through the inverse transformation of the intrinsic parameter matrix based on the acquired corresponding depth values; and to convert the camera three-dimensional coordinates into world coordinates through the inverse transformation of the extrinsic parameter matrix. The color block size calculation unit is used to calculate the size of the three-dimensional disease marker color block based on the size of each two-dimensional detection frame; The color block generation unit is used to generate three-dimensional disease marker color blocks on the surface of the processed three-dimensional reconstruction model based on the wall's outer normal vector, combined with three-dimensional dimensions, and coloring according to the disease category.

[0019] Preferably, the result fusion output module includes: The two-dimensional detection image generation unit is used to draw bounding boxes of different colors and disease category labels on the corresponding RGB image according to the coordinates of each two-dimensional detection box and the disease category, and generate an labeled two-dimensional detection image. The image stitching unit is used to stitch all the labeled two-dimensional detection images into a two-dimensional panoramic image in serpentine scanning path order; The detection result output unit is used to combine the generated two-dimensional panoramic image and three-dimensional disease marker color blocks to output the detection results of surface diseases of the ancient city wall.

[0020] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements the above-described intelligent detection method for surface defects of ancient city walls based on three-dimensional reconstruction.

[0021] A fourth aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the above-mentioned intelligent detection method for surface defects of ancient city walls based on three-dimensional reconstruction.

[0022] Compared with the prior art, the advantages of this invention are as follows: (1) This invention automatically loads and preprocesses the 3D reconstruction model of the ancient city wall, enabling the processing of 3D models of different qualities and formats without manual intervention. Furthermore, through a multi-strategy texture restoration design, it can automatically adapt to situations such as missing textures and missing UV coordinates; automatically unify multi-material models into a single material; and by analyzing the geometric structural features of the processed 3D reconstruction model, it can automatically identify the main extension direction of the wall (X-axis or Y-axis); automatically calculate the wall length, thickness, and height; automatically determine the wall bounding box and center point; and automatically calculate the camera distance and perform virtual view rendering based on the wall's geometric features. Moreover, through the design of a pre-trained YOLO target detection model, it can automatically identify multiple types of defects, automatically perform coordinate transformations, generate 3D defect marker color blocks, and automatically generate structured statistical reports. Therefore, this invention shortens the complete detection time from 24 hours using traditional manual methods to approximately 1-2 hours (including 3D reconstruction) through the entire automated processing flow, increasing efficiency several times and significantly reducing labor and time costs.

[0023] (2) Achieve precise spatial positioning: The camera's three-dimensional coordinates are converted into world coordinates through the inverse transformation of the extrinsic parameter matrix, providing precise three-dimensional world coordinates for each disease. The positioning accuracy reaches the centimeter level (16cm RMSE), which facilitates the formulation of targeted repair plans and the estimation of engineering quantities, and supports the spatial analysis and hotspot identification of diseases.

[0024] (3) Improve detection accuracy: The pre-trained YOLO target detection model is used to detect the defects of the ancient city wall, which is far superior to the subjectivity and instability of manual visual inspection, and can identify a variety of defect types.

[0025] (4) Solving the problem of missing texture information: Innovative multi-strategy texture recovery technology was designed, including automatic texture search, UV coordinate generation, texture-based vertex shading, etc., to ensure that visual representation suitable for detection can be obtained under various model quality conditions.

[0026] (5) Optimize viewpoint planning strategy: In view of the slender structure of the ancient city wall, an adaptive viewpoint generation method based on geometric structure and a serpentine scanning path are proposed. Compared with the traditional uniform spherical sampling method, the number of effective viewpoints is reduced by about 60%, while the detection coverage is significantly improved and the path length is reduced.

[0027] (6) Establishing a precise mapping from two-dimensional to three-dimensional: Innovatively combining deep learning detection with three-dimensional geometric transformation, using depth maps and camera parameters to achieve a precise conversion from two-dimensional pixels to three-dimensional world coordinates, and for the first time realizing the three-dimensional spatial annotation of detection results in the field of cultural relic disease detection.

[0028] (7) Provide multimodal result expression: generate two-dimensional annotated images, panoramic stitched images, interactive three-dimensional visualization models, exportable PLY files and structured statistical reports to meet the diverse needs of different users (cultural heritage experts, restoration personnel, management decision-makers).

[0029] (8) Reduce safety risks: The non-contact virtual inspection method avoids the safety hazards of people climbing to high places or dangerous areas, and at the same time reduces the contact damage to the cultural relics during the inspection process, which is in line with the principle of minimum intervention in cultural relic protection.

[0030] (9) Establish digital archives: The generated three-dimensional models and disease annotations can be permanently saved, support multi-time comparison analysis, realize dynamic monitoring of disease development, and provide data support for the formulation of long-term protection strategies.

[0031] (10) It has good scalability: The system adopts a modular design and can handle models of different scales (200,000 to 10 million faces). It can be extended to other types of cultural relics and buildings (grottoes, ancient buildings, stone cultural relics, etc.) and has broad application prospects. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of a gridded serpentine scan for intelligent viewpoint planning;

[0033] Figure 2 This is an example image of a single-viewpoint detection result;

[0034] Figure 3 Here is an example of a panoramic stitched image;

[0035] Figure 4 It is a visualization of the disease marking effect in a 3D model;

[0036] Figure 5 This is a schematic diagram showing the test results for different types of diseases; where blue represents weathering diseases, green represents cracking diseases, and yellow represents mortar loss diseases.

[0037] Figure 6 This is a visualization of the correspondence between the YOLOv10 target detection bounding box and the 3D disease marker color block;

[0038] Figure 7 This is a schematic diagram of the disease statistics report;

[0039] Figure 8 This is a schematic diagram of the system provided by the present invention; Detailed Implementation

[0040] The following will be described in conjunction with embodiments of the present invention. Figures 1 to 8The technical solutions in the embodiments of the present invention are clearly and completely described herein. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention. Example 1

[0041] This invention provides an intelligent detection method for surface defects of ancient city walls based on three-dimensional reconstruction, comprising the following steps: S1. Load the obtained 3D reconstruction model of the ancient city wall and perform multi-strategy texture restoration processing. Analyze the geometric structural features of the processed 3D reconstruction model to determine the main extension direction, thickness, length and height of the ancient city wall. In this embodiment of the invention, the three-dimensional reconstruction model of the ancient city wall is obtained through existing technologies such as oblique photogrammetry, laser scanning, and structured light scanning, and its format supports common three-dimensional file formats such as OBJ, PLY, and STL.

[0042] Specifically, in S1, the acquired 3D reconstruction model of the ancient city wall is loaded and subjected to multi-strategy texture restoration processing, including the following strategies: Strategy 1: Prioritize using the TriangleModel data structure from the Open3D library to load the obtained 3D reconstruction model and associated texture information to obtain the processed 3D reconstruction model; this method can completely preserve the multi-mesh structure and associated texture information.

[0043] Strategy 2: If Strategy 1 fails, the traditional TriangleMesh method is used to load the acquired 3D reconstruction model and associated texture information. When texture information is missing, texture files in the same directory are automatically searched. Once a texture file is found, a texture mapping relationship is established to obtain the processed 3D reconstruction model. Texture file search supports common image formats such as JPG and PNG.

[0044] Specifically, when texture information is missing, the system automatically searches for texture files in the same directory. Once a texture file is found, the following operations are performed: If the missing texture information is UV coordinates, then the missing UV coordinates are generated based on the wall plane features, and the generated UV coordinates are added to the texture information to obtain the processed 3D reconstruction model. Let the set of vertices of the city wall model be . , where x i This indicates the position of the vertex in the horizontal direction, or in other words, its position along the main axis of the ancient city wall. This value usually represents the length or east-west direction of the ancient city wall; y iThis represents the vertical height of the vertex, that is, the vertical distance from the ground to the top of the ancient city wall. This value represents the height or vertical direction of the ancient city wall; z i This indicates the position of the vertex in another horizontal direction, that is, the thickness dimension perpendicular to the extension direction of the ancient city wall. This value usually represents the width direction or the north-south direction of the ancient city wall. The formula for calculating the UV coordinates is: (1); In the formula, To prevent small constants from being divided by zero, This is the horizontal coordinate of the texture map, representing the position of the texture image in the horizontal direction, corresponding to the x-direction (the length direction of the ancient city wall) in the 3D reconstruction model. This is the vertical coordinate of the texture map, representing the position of the texture image in the vertical direction, corresponding to the z-direction (width / depth of the ancient city wall) in the 3D reconstruction model. These represent the maximum and minimum values ​​of the vertex's position in the horizontal direction, respectively. These represent the maximum and minimum values ​​of the vertex's position in another horizontal direction, respectively. This method projects the 3D coordinates onto the XZ plane and normalizes them to the range [0,1]. If the missing texture information is the entire texture and vertex colors on the texture image, then the dominant hue is extracted from the texture image and combined with the vertex height information to generate the vertex colors. Then, the dominant hue and vertex colors are added to the texture information to obtain the processed 3D reconstruction model.

[0045] Specifically, when the 3D reconstructed model has neither texture nor vertex color, the dominant color tone is extracted from the texture image and combined with geometric information to generate pseudo-color. Let the texture image be... After downsampling to 100×100 resolution, the average color is calculated. The calculation formula is: (2); Vertex color Modulation is performed based on altitude information, and the specific formula is shown below: (3); Where N(0,0.02) represents Gaussian noise with a mean of 0 and a standard deviation of 0.02, used to simulate the natural texture changes on the surface of bricks and stones.

[0046] The multi-strategy texture loading method described above ensures that usable model representations can be obtained under various data quality conditions.

[0047] In embodiment S1 of this invention, the geometric structural features of the processed three-dimensional reconstruction model are analyzed to determine the main extension direction, thickness, and height of the ancient city wall. This specifically includes the following steps: Step 1: Calculate the axial bounding box (AABB) of the processed 3D reconstructed model to obtain the minimum boundary points of the 3D reconstructed model. and the maximum boundary point ,Right now: (4); in, The lowest point height in the 3D reconstruction model represents the minimum Y-coordinate among all vertices of the ancient city wall. This value typically corresponds to the bottom / foundation location of the ancient city wall. The height of the highest point in the 3D reconstruction model represents the maximum value of the Y-coordinate among all vertices of the ancient city wall, which usually corresponds to the top / highest point of the ancient city wall. The formula for calculating the center coordinates of the ancient city wall is shown below: (5); By calculating the center coordinates C of the ancient city wall center Serving as the center point for geometric transformations such as rotation and scaling, this value provides a reference for subsequent model transformations. Using the center as a reference point during model operations maintains the symmetry of the transformation. Furthermore, this value helps in virtual camera perspective positioning, determining the optimal viewpoint for observing the ancient city wall, calculating the focal point the virtual camera should be aligned with, and ensuring the ancient city wall is centered in the field of view.

[0048] Based on the obtained minimum boundary point and the maximum boundary point Calculate the dimension vector of the ancient city wall; the formula for calculating the dimension vector is: (6); in, This represents the span / length of the ancient city wall along the X-axis, i.e., the total length of the ancient city wall (if the X-axis is the direction in which the city wall extends). This represents the span / height of the ancient city wall along the Y-axis, i.e., the total height of the ancient city wall (from the foundation to the highest point). This represents the span / thickness of the ancient city wall along the Z-axis, i.e., the total width / thickness of the city wall (the depth of the wall). Based on the calculated dimension vector of the ancient city wall, the main extension direction, thickness, length, and height of the ancient city wall are determined; the specific determination method is as follows: (1) Calculated according to formula (6) and The size relationship determines the main extension direction of the ancient city wall: when > When the wall is in a certain direction, the main extension direction of the ancient city wall is the X-axis direction; otherwise, it is the Y-axis direction.

[0049] (2) Determine wall parameters: Length of the ancient city wall: ; Thickness of the ancient city wall: ; The height of the ancient city wall: .

[0050] S2. Based on the determined main extension direction, thickness, length, and height of the ancient city wall, and combined with a gridded serpentine scanning strategy, multiple virtual camera positions are generated to form a set of viewpoints covering the wall surface, such as... Figure 1 As shown; specifically including the following steps: S21. Based on the determined thickness of the ancient city wall, calculate the distance between the virtual camera and the wall surface. The distance between the camera and the wall is adaptively determined based on the wall thickness to ensure that each viewpoint can capture the wall from directly in front. The calculation formula is as follows: (7); in, This is a distance coefficient, with a value ranging from 0.05 to 0.15; this distance ensures both a sufficient field of view and a high image resolution.

[0051] S22. Arrange multiple horizontal sampling points in the even-numbered rows of the vertical direction in a forward direction, and use a serpentine scanning strategy to arrange multiple horizontal sampling points in the odd-numbered rows of the vertical direction in a reverse direction. In this embodiment of the invention, when arranging horizontal sampling points, the horizontal sampling interval is first determined. and vertical sampling interval The calculation is performed using the following formula: (8); in, and These represent the number of sampling points in the horizontal and vertical directions, respectively. In this embodiment, we take... =10, =4.

[0052] To reduce the movement distance of the virtual camera, this invention employs a serpentine scanning strategy, arranging horizontal sampling points in reverse order along odd-numbered rows in the vertical direction. (9) in, These are the optimized column numbers; these are the actual column positions used by the sampling points after applying the serpentine strategy. h represents the original column number, i.e., the original horizontal coordinate before serpentine optimization. This value indicates the column position when normally arranged from left to right, and h∈[0, 1]. -1]; v is the row number, i.e., the current row number (vertical index), indicating which row it is in the vertical direction (odd row). This value is used to determine whether the current row is odd or even, and v∈[0, 1 / 2 ... -1]); This strategy minimizes the movement distance between adjacent viewpoints, reducing the path length by approximately 45% compared to traditional raster scanning, thus significantly improving processing efficiency.

[0053] S23. For all the generated horizontal sampling points, generate grids in the horizontal and vertical directions respectively to obtain multiple grids; S24. For each generated grid, calculate its normalized coordinates to obtain multiple normalized coordinates; the formula for calculating the normalized coordinates is: (10); in, This is a horizontally normalized coordinate, representing the relative position of the current grid cell in the horizontal direction. The value ranges from [0,1], where 0 represents the leftmost position and 1 represents the rightmost position. The vertically normalized coordinates represent the relative position of the current grid in the vertical direction. The value ranges from [0,1], where 0 represents the bottom edge and 1 represents the top edge. S25. Based on the distance between the virtual camera and the wall, the main extension direction, length and height of the ancient city wall, and multiple normalized coordinates, generate multiple virtual camera positions. Specifically, when the ancient city wall extends along the X-axis, the virtual camera position... for: (10); in, This is the Y-coordinate (height coordinate) of the wall's center. This value is the Y component of the previously calculated wall center coordinate Ccenter, specifically representing the center position of the 3D reconstructed model in the vertical direction (height direction), that is, the average height of the lowest and highest points of the ancient city wall. It is the midpoint of the wall in the vertical direction, and its function is to determine the height at which the virtual camera should observe the city wall. And observe the target location for: (11); Using the above virtual camera position and observe the target location The calculation determined the position and orientation of the virtual camera, and together they determined the virtual camera's line-of-sight vector LP. Additionally, through virtual camera position Observe the target location The three elements, Up (the upward direction, usually (0, 1, 0)), can construct a complete camera view matrix, thus fully defining the camera's pose and orientation virtually. This ensures that the virtual camera is always perpendicular to the wall, without tilting or shifting. Moreover, regardless of the virtual camera's position on the grid, the observed target is always on the wall's center line, guaranteeing that the wall remains in the center of the field of view for optimal shooting results.

[0054] The virtual camera position as the wall extends along the Y-axis. for: (12); in, This is the X-coordinate (horizontal center position) of the wall's center coordinates Ccenter. Specifically, it represents the center position of the 3D reconstructed model along the X-axis (horizontal direction), which is the midpoint between the leftmost and rightmost ends of the wall in the X-direction. It is the center position of the wall in the X direction (horizontal), used to determine where the virtual camera should observe the wall in the X direction when extending along the Y axis; And observe the target location for: (13); S3. Render the processed 3D reconstructed model at each virtual camera location to obtain multiple RGB images and their corresponding depth images; specifically including the following steps: S31. Set the internal and external parameters of the virtual camera at each virtual camera location; The virtual camera's intrinsic and extrinsic parameters include an extrinsic parameter matrix and an intrinsic parameter matrix, and the specific setting method is as follows: Calculate camera attitude parameters: The forward vector is: ; Where L is the observation target position L(h,v), representing the coordinates of the target point observed by the virtual camera; P is the virtual camera position P(h,v), representing the position coordinates of the virtual camera in three-dimensional space.

[0055] The upward vector is: (Z-axis of world coordinate system); The rightward vector is: ; The corrected upper vector is .

[0056] Construct the rotation matrix using the vectors calculated above. (Where r corresponds to the rightward vector, u' to the corrected up vector, and f to the forward vector) and translation vector Then, the extrinsic parameter matrix is ​​constructed. .

[0057] Construct an intrinsic parameter matrix, assuming the image resolution is W×H (1920×1080 in this embodiment) and the field of view is... The focal length is The principal point coordinates are The intrinsic parameter matrix is .in, This represents the focal length of the virtual camera in the horizontal direction (X-axis), measured in pixels. This value reflects how distances in three-dimensional space are projected onto the pixel coordinates in the X-direction of the image. This is the focal length of the virtual camera in the vertical direction (Y-axis), also measured in pixels. This value reflects how three-dimensional space is projected onto the image along the Y-axis. The X-coordinate of the principal point of the image (the intersection of the optical axis and the image plane) is usually located at the center of the image, i.e., half the width. For example, for a 1920×1080 image, = 1920 / 2 = 960, This is the Y-coordinate of the principal point of the image, usually located at the image center, which is half the height. For example, for a 1920×1080 image, = 1080 / 2 = 540.

[0058] S32. Based on the internal and external parameters of the virtual camera, the processed 3D reconstructed model is rendered at each virtual camera position. Specifically, the rendering process is performed using the Open3D visualization framework. The rendering process includes: (1) Geometric transformation: transforming the model vertices from the world coordinate system to the camera coordinate system; (2) Perspective projection: projecting 3D points onto the 2D image plane; (3) Rasterization: converting triangles into pixels; (4) Fragment shading: calculating the color value of each pixel.

[0059] S33. Based on multiple rendered 3D reconstruction models, obtain multiple RGB images. and its corresponding multiple depth images The RGB image records the color information of the scene, with a resolution of 1920×1080 and 8-bit RGB; the depth image records the depth value corresponding to each pixel (i.e., the distance from the virtual camera to the surface of the ancient city wall).

[0060] S4. Use the pre-trained YOLO object detection model to detect diseases in each acquired RGB image, and obtain the coordinates of the two-dimensional detection box and the disease category. Among them, the surface defects of the city wall include weathering, cracks, and mortar loss.

[0061] In this embodiment of the invention, the pre-trained YOLO object detection model is obtained through the following method: Collect a large number of real images containing the defects of the ancient city wall, and use annotation tools (such as LabelImg) to annotate each image. Here, it is necessary to annotate the bounding box coordinates of the defects and the type of defects (weathering / cracks / mortar loss); The collected dataset was divided into a training set, a test set, and a validation set in a ratio of 8:1:1. Select a YOLO series model, such as YOLOv8 or YOLOv10, and set relevant training parameters, such as learning rate, batch size, and number of iterations. Train the selected YOLO series model on the training set to obtain the trained model. Test the accuracy and recall of the trained model on the validation set, adjust the model hyperparameters to optimize performance, and save the optimal model weights. After completing the above steps, you will have a pre-trained YOLO object detection model. The test set is mainly used to evaluate the generalization ability of the pre-trained YOLO object detection model and provide objective performance metrics. The test set is completely isolated throughout the training process and is only used once after the model training and tuning are completed.

[0062] In this embodiment of the invention, S4 includes the following steps: S41. Preprocess each acquired RGB image, specifically including normalization (scaling pixel values ​​to the [0,1] range), resizing (adjusting to the YOLO model input size (e.g., 640×640)), and format conversion (RG→Tensor format); to obtain the preprocessed image; S42. Input the preprocessed image into the pre-trained YOLO object detection model for processing and result extraction to obtain the detection result; S43. Perform non-maximum suppression (removing duplicate detection boxes), coordinate mapping (mapping the detection box coordinates back to the original image size of 1920×1080) on each detection result, and information extraction (specifically including the two-dimensional detection box coordinates, confidence level, and disease category). Finally, output the disease detection result, which includes the disease category, confidence level, two-dimensional detection box coordinates, and detection image.

[0063] S5. Based on the corresponding depth image, map the coordinates of each two-dimensional detection box and the disease category to three-dimensional space through coordinate transformation, and generate three-dimensional disease marker color blocks on the surface of the processed three-dimensional reconstruction model; specifically including the following steps: S51. Based on the corresponding depth image, obtain the coordinates of each two-dimensional detection box (i.e., the center pixel coordinates). The corresponding depth value ; S52. Based on the acquired depth values, the coordinates of each 2D detection box are converted into camera 3D coordinates (i.e., 3D coordinates in the camera coordinate system) through the inverse transformation of the intrinsic parameter matrix. The expression for the camera 3D coordinates is: (14); in, This represents the X-coordinate (horizontal direction) of the defect point in the camera coordinate system. This represents the Y-coordinate (vertical direction) of the defect point in the camera coordinate system. The Z-coordinate (depth direction) of the defect point in the camera coordinate system; S53. Convert the camera's 3D coordinates to world coordinates through the inverse transformation of the extrinsic parameter matrix. The transformation formula is: (15); in, These are the coordinates of the defect points in the world coordinate system (usually the coordinate system of the city wall model). The three-dimensional coordinates in the camera coordinate system obtained by formula (14) are calculated using the following formula: ; This invention converts the coordinates of each two-dimensional detection box into world coordinates through coordinate transformation, with the following objectives: (1) By converting to world coordinates, the spatial location and recording of defects can be realized. By recording the specific location of defects on the city wall model, it is easier for subsequent maintenance personnel to accurately find the location of defects. (2) Since the camera coordinate systems of different viewpoints are different, the same defect may be detected in multiple viewpoints. Therefore, it is necessary to transform to a unified world coordinate system in order to determine whether it is the same defect (removing duplicates). (3) Mark the detected defects onto the original 3D model of the city wall. Only by knowing the location of the defects in the model coordinate system (world coordinate system) can the defect area be highlighted on the model and a 3D visualization of the defect markings be generated.

[0064] (4) Since the size changes due to different viewing angles when calculated in the camera coordinate system, the actual physical size of the disease can be accurately calculated by converting to the unified world coordinate system.

[0065] S54. Calculate the size of the three-dimensional defect marker color block based on the size of each two-dimensional detection frame. The specific calculation formula is as follows: (16); in, The width of the three-dimensional disease marker color block. The height of the three-dimensional disease marker color block. The width of the 2D detection box. The height of the 2D detection box. This is the size scaling factor; S55. Based on the outer normal vector of the wall surface, combined with the three-dimensional dimensions, and colored according to the disease type, generate three-dimensional disease marker color blocks on the surface of the processed three-dimensional reconstruction model.

[0066] In this embodiment of the invention, the normal vector of the outer side of the wall The value is determined by calculating the average of all virtual camera positions, and the specific determination method is as follows: Calculate the average of all virtual camera positions. The calculation formula is: (17); in, Σ represents the total number of viewpoints, where Σ represents the summation over all grid positions (h,v).

[0067] When the wall extends along the X-axis: like ,but ;otherwise ; in, for The Y component, which is the average of the Y coordinates of all virtual camera positions, specifically represents the average height position of the virtual camera. for The Y component, i.e., the Y coordinate of the wall center, is calculated using the following formula: .

[0068] When the wall extends along the Y-axis: like ,but ;otherwise .

[0069] in for The X component, which is the average of the X coordinates of all virtual camera positions, specifically represents the average position of the virtual camera in the X direction. for The X component, i.e., the X coordinate of the wall center, is calculated using the following formula: .

[0070] The center position of the generated 3D disease marker color block is ,in, The offset distance of the 3D disease marker color block from the outer side of the wall is 5cm; the width of the 3D disease marker color block is... The height is The thickness is T=8cm, and its color is based on the disease category (RGB value).

[0071] In this embodiment of the invention, the three-dimensional disease marker color block takes into account the spatial orientation of the wall. By analyzing the distribution of camera positions, the outer direction of the wall is automatically determined to ensure that the disease marker is always located on the visible side. The color block is offset outward by a certain distance to avoid graphic rendering problems caused by overlapping with the wall.

[0072] S6. Integrate all 2D detection frame coordinates and disease categories, as well as the generated 3D disease marker color blocks, to output the detection results of surface diseases on the ancient city wall. This includes the following steps: S61. Based on the coordinates of each two-dimensional detection box and the disease category, draw bounding boxes of different colors and disease category labels on the corresponding RGB image to generate an labeled two-dimensional detection image, such as... Figure 2 As shown; the files are named wall_view_001.jpg to wall_view_040.jpg.

[0073] S62. Stitch all the labeled 2D detection images together in serpentine scanning path order to form a 2D panoramic image, such as... Figure 3 As shown; the original image was scaled to 30% (576×324 pixels), with a grid layout of 10 columns × 4 rows, and the total size of the stitched image was 5760×1296 pixels, and finally saved as wall_detection_montage.jpg.

[0074] S63. Combining the generated 2D panoramic image and 3D defect marker color blocks, output the detection results of surface defects on the ancient city wall. Specifically, the processed 3D reconstruction model is integrated with the 3D defect marker color blocks, and interactively displayed using the Open3D visualizer, supporting rotation, scaling, and translation operations, such as... Figure 4 As shown, different colors are used to distinguish the types of diseases. Weathering disease is blue, crack disease is green, and fissure loss disease is yellow, as shown below. Figure 5 As shown.

[0075] Merge the original mesh (i.e., the original geometric data of the 3D reconstruction model of the ancient city wall loaded in step S1) and the color patch mesh (i.e., 3D defect marker color patches, i.e., colored patches marking defect areas on the processed 3D model, these color patches covering the surface of the original city wall model), export it as a PLY format, with the number of vertices being the original number of vertices + the number of color patch vertices (247×8), and the number of patches being the original number of patches + the number of color patch patches (247×12), and finally save it as wall_with_detection_patches.ply, as shown below. Figure 6 As shown.

[0076] The detection results of surface defects on the ancient city wall will be automatically generated into a structured text report, which will be saved as detection_report.txt. Figure 7 As shown.

[0077] Before outputting the text report, for the same defect in the overlapping area of ​​adjacent viewpoints, deduplication is performed as follows: (1) Spatial clustering: Calculate the Euclidean distance between the 3D center points of all detection results. If the distance is less than a threshold... If they are of the same category, they are grouped together.

[0078] (2) Confidence fusion: For multiple test results in the same group, calculate the weighted average confidence level: (18); in, The confidence score for detecting the disease at the i-th viewpoint, with a value ranging from [0, 1], and k being the total number of times the same disease is detected. For example, if a disease is detected at 3 different viewpoints, then k = 3. The weights are... ,and The angle between the line of sight and the normal vector of the wall.

[0079] (3) Location fusion: The weighted center is taken from the merged location: (19); in, This represents the final convergence location of the disease in three-dimensional space; it is the optimal location estimate obtained by integrating multiple detection results. The three-dimensional location of the disease was detected for the i-th viewpoint. Example 2

[0080] This invention provides an intelligent detection method for surface defects of ancient city walls based on 3D reconstruction. The specific steps are basically the same as in Embodiment 1, the only difference being the setting of the viewpoint planning parameters. Specifically, this embodiment targets larger sections of the city wall (length exceeding 150 meters) and employs a denser sampling strategy: number of horizontal sampling points: Number of vertical sampling points: Total number of viewpoints: 75.

[0081] Correspondingly, the processing time increased to approximately 68 seconds, but the detection coverage increased to 98.1%, enabling the detection of more minor defects.

[0082] This embodiment also adds an adaptive sampling strategy: after identifying densely affected areas in the initial scan, additional viewpoints are added to these areas for a second, more detailed scan, further improving detection accuracy. Example 3

[0083] This invention provides an intelligent detection method for surface diseases of ancient city walls based on three-dimensional reconstruction. The specific steps are basically the same as those in Embodiment 1. The only difference is that, in addition to the three basic diseases of weathering, cracks, and fissure loss, this embodiment also supports the detection of three other types of diseases: biological diseases (plant erosion, moss attachment), salting diseases (salting out, efflorescence), and pollution diseases (graffiti, stains).

[0084] Meanwhile, the YOLO model training dataset in this embodiment of the invention has been expanded to 6 categories, and the total number of samples has increased to 5000 images. Different disease types are identified by different colors in the detection results: weathering is blue (0,0,255); crack is green (0,255,0); seamLoss is yellow (255,255,0); biological is purple (128,0,128); salting is orange (255,165,0); and pollution is red (255,0,0). Example 4

[0085] like Figure 8 As shown, this embodiment of the invention provides an intelligent detection system for surface defects of ancient city walls based on three-dimensional reconstruction, implemented using the method of Embodiment 1 above, including: The 3D model preprocessing module is used to load the acquired 3D reconstruction model of the ancient city wall, perform multi-strategy texture restoration processing, analyze the geometric structural features of the processed 3D reconstruction model, and determine the main extension direction, thickness, length and height of the ancient city wall. The intelligent viewpoint planning module is used to generate multiple virtual camera positions based on the determined main extension direction, thickness, length and height of the ancient city wall, combined with a gridded serpentine scanning strategy. The virtual view rendering module is used to render the processed 3D reconstructed model at each virtual camera position to obtain multiple RGB images and their corresponding multiple depth images. The disease intelligent detection module is used to perform disease detection on each acquired RGB image using a pre-trained YOLO object detection model, and obtain the coordinates of the two-dimensional detection box and the disease category. The 3D mapping and annotation module is used to map the coordinates of each 2D detection box and the disease category to 3D space based on the corresponding depth image through coordinate transformation, and generate 3D disease marker color blocks on the surface of the processed 3D reconstruction model. The result fusion output module is used to fuse all the coordinates of the two-dimensional detection boxes and the disease categories, as well as the generated three-dimensional disease marker color blocks, and output the detection results of the surface diseases of the ancient city wall.

[0086] In this embodiment of the invention, the three-dimensional model preprocessing module includes: The model loading unit is used to load the acquired 3D reconstruction model of the ancient city wall; The texture restoration processing unit is used to perform multi-strategy texture restoration processing on the loaded 3D reconstruction model to obtain the processed 3D reconstruction model. The geometric analysis unit is used to analyze the geometric structural features of the processed 3D reconstruction model and determine the main extension direction, thickness, length and height of the ancient city wall.

[0087] In this embodiment of the invention, the intelligent viewpoint planning module includes: The parameter calculation unit is used to calculate the distance between the virtual camera and the wall surface based on the determined thickness of the ancient city wall. The sampling point optimization setting unit arranges multiple horizontal sampling points in the even-numbered rows of the vertical direction in a forward direction, and adopts a serpentine scanning strategy to arrange multiple horizontal sampling points in the odd-numbered rows of the vertical direction in a reverse direction. The mesh generation unit is used to generate meshes in the horizontal and vertical directions for all generated horizontal sampling points, resulting in multiple meshes. The normalized coordinate calculation unit is used to calculate the normalized coordinates of each generated grid, resulting in multiple normalized coordinates; The virtual camera position generation unit is used to generate multiple virtual camera positions based on the distance of the virtual camera from the wall, the main extension direction, length and height of the ancient city wall, and multiple normalized coordinates.

[0088] In this embodiment of the invention, the virtual view rendering module includes: The virtual camera parameter setting unit is used to set the internal and external parameters of the virtual camera at each virtual camera location. The virtual camera's intrinsic and extrinsic parameters include an extrinsic parameter matrix and an intrinsic parameter matrix; The graphics rendering unit is used to perform graphics rendering on the processed 3D reconstructed model at each virtual camera position based on the set intrinsic and extrinsic parameters of the virtual camera. The image capture unit is used to obtain multiple RGB images and their corresponding depth images based on multiple rendered 3D reconstruction models.

[0089] In this embodiment of the invention, the intelligent disease detection module includes: The YOLO inference unit is used to perform forward inference on each acquired RGB image using a pre-trained YOLO object detection model to obtain an inferred image. The post-processing unit is used to perform non-maximum suppression on the inference image and filter to obtain the disease detection results; The annotation drawing unit is used to draw detection boxes and disease labels on the RGB image based on the disease detection results, and obtain the coordinates of the two-dimensional detection boxes and the disease category.

[0090] In this embodiment of the invention, the three-dimensional mapping and annotation module includes: The depth value calculation unit is used to obtain the depth value corresponding to the coordinates of each two-dimensional detection box based on the corresponding depth image; The coordinate transformation unit is used to convert the coordinates of each two-dimensional detection box into camera three-dimensional coordinates through the inverse transformation of the intrinsic parameter matrix based on the acquired corresponding depth values; and to convert the camera three-dimensional coordinates into world coordinates through the inverse transformation of the extrinsic parameter matrix. The color block size calculation unit is used to calculate the size of the three-dimensional disease marker color block based on the size of each two-dimensional detection frame; The color block generation unit is used to generate three-dimensional disease marker color blocks on the surface of the processed three-dimensional reconstruction model based on the wall's outer normal vector, combined with three-dimensional dimensions, and coloring according to the disease category.

[0091] In this embodiment of the invention, the result fusion output module includes: The two-dimensional detection image generation unit is used to draw bounding boxes of different colors and disease category labels on the corresponding RGB image according to the coordinates of each two-dimensional detection box and the disease category, and generate an labeled two-dimensional detection image. The image stitching unit is used to stitch all the labeled two-dimensional detection images into a two-dimensional panoramic image in serpentine scanning path order; The detection result output unit is used to combine the generated two-dimensional panoramic image and three-dimensional disease marker color blocks to output the detection results of surface diseases of the ancient city wall. Example 5

[0092] This invention provides a computer-readable storage medium storing a computer program thereon, characterized in that the computer program, when executed by a processor, implements the intelligent detection method for surface defects of ancient city walls based on three-dimensional reconstruction as described in Embodiment 1 above.

[0093] The computer-readable storage medium is any medium capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk. Example 6

[0094] This invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. The device is characterized in that, when the processor executes the computer program, it implements the intelligent detection method for surface defects of ancient city walls based on three-dimensional reconstruction as described in Embodiment 1 above.

[0095] The electronic devices mentioned are personal computers, workstations, servers, tablets, smartphones, and other devices with computing capabilities.

[0096] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for intelligent detection of surface defects on ancient city walls based on three-dimensional reconstruction, characterized in that, Includes the following steps: The obtained 3D reconstruction model of the ancient city wall is loaded and subjected to multi-strategy texture restoration processing. The geometric structural features of the processed 3D reconstruction model are analyzed to determine the main extension direction, thickness, length and height of the ancient city wall. Based on the determined main extension direction, thickness, length and height of the ancient city wall, and combined with a gridded serpentine scanning strategy, multiple virtual camera positions are generated. The processed 3D reconstruction model is rendered at each virtual camera position to obtain multiple RGB images and their corresponding multiple depth images; The pre-trained YOLO object detection model was used to detect diseases in each acquired RGB image to obtain the coordinates of the two-dimensional detection box and the disease category. Based on the corresponding depth image, the coordinates of each two-dimensional detection box and the disease category are mapped to three-dimensional space through coordinate transformation, and three-dimensional disease marker color blocks are generated on the surface of the processed three-dimensional reconstruction model. By integrating all the coordinates of the two-dimensional detection frames and the disease categories, as well as the generated three-dimensional disease marker color blocks, the detection results of the surface diseases of the ancient city wall are output.

2. The intelligent detection method for surface defects of ancient city walls based on three-dimensional reconstruction according to claim 1, characterized in that, The process involves loading the acquired 3D reconstruction model of the ancient city wall and performing multi-strategy texture restoration processing, including the following strategies: Strategy 1: Use the TriangleModel data structure to load the acquired 3D reconstruction model and associated texture information to obtain the processed 3D reconstruction model; Strategy 2: If Strategy 1 fails, the traditional TriangleMesh method is used to load the obtained 3D reconstruction model and associated texture information. When texture information is missing, the texture files in the same directory are automatically searched. After the texture files are found, a texture mapping relationship is established to obtain the processed 3D reconstruction model.

3. The intelligent detection method for surface defects of ancient city walls based on three-dimensional reconstruction according to claim 2, characterized in that, When texture information is missing, the system automatically searches for texture files in the same directory. Once a texture file is found, a texture mapping relationship is established to obtain the processed 3D reconstruction model. This includes the following steps: When texture information is missing, automatically search for texture files in the same directory. Once a texture file is found, perform the following operations: If the missing texture information is UV coordinates, the missing UV coordinates are generated based on the wall plane features and added to the texture information to obtain the processed 3D reconstruction model. If the missing texture information is the entire texture and vertex colors on the texture image, the dominant hue is extracted from the texture image and combined with the vertex height information to generate the vertex colors. The dominant hue and vertex colors are then added to the texture information to obtain the processed 3D reconstruction model.

4. The intelligent detection method for surface defects of ancient city walls based on three-dimensional reconstruction as described in claim 1, characterized in that, The analysis of the geometric structural features of the processed 3D reconstructed model to determine the main extension direction, thickness, and height of the ancient city wall includes the following steps: Calculate the axial bounding box of the processed 3D reconstructed model to obtain the minimum and maximum boundary points of the 3D reconstructed model; Based on the obtained minimum and maximum boundary points, calculate the dimension vector of the ancient city wall. Based on the calculated dimension vector of the ancient city wall, the main extension direction, thickness, length, and height of the ancient city wall are determined.

5. A method for intelligent detection of surface defects of ancient city walls based on three-dimensional reconstruction, as described in claim 1, is characterized in that... The process of generating multiple virtual camera positions based on the determined main extension direction, thickness, length, and height of the ancient city wall, combined with a gridded serpentine scanning strategy, includes the following steps: Based on the determined thickness of the ancient city wall, the distance between the virtual camera and the wall surface is calculated; Multiple horizontal sampling points are arranged in the even-numbered rows of the vertical direction in a forward direction, and a serpentine scanning strategy is adopted to arrange multiple horizontal sampling points in the odd-numbered rows of the vertical direction in a reverse direction. For all generated horizontal sampling points, generate grids in both the horizontal and vertical directions to obtain multiple grids; For each generated grid, its normalized coordinates are calculated, resulting in multiple normalized coordinates; Multiple virtual camera positions are generated based on the distance between the virtual camera and the wall, the main extension direction, length and height of the ancient city wall, and multiple normalized coordinates.

6. A method for intelligent detection of surface defects of ancient city walls based on three-dimensional reconstruction, as described in claim 1, is characterized in that... The process of rendering the processed 3D reconstructed model at each virtual camera location to obtain multiple RGB images and their corresponding depth images includes the following steps: Set the internal and external parameters of the virtual camera at each virtual camera location; The virtual camera's intrinsic and extrinsic parameters include an extrinsic parameter matrix and an intrinsic parameter matrix; Based on the set intrinsic and extrinsic parameters of the virtual camera, the processed 3D reconstruction model is rendered graphically at each virtual camera position. Based on multiple rendered 3D reconstruction models, multiple RGB images and their corresponding depth images are obtained.

7. A method for intelligent detection of surface defects of ancient city walls based on three-dimensional reconstruction, as described in claim 6, is characterized in that... Based on the corresponding depth image, the coordinates of each two-dimensional detection box and the disease category are mapped to three-dimensional space through coordinate transformation, and disease marker color blocks are generated on the surface of the processed three-dimensional reconstructed model. This includes the following steps: Based on the corresponding depth image, obtain the depth value corresponding to the coordinates of each two-dimensional detection box; Based on the acquired depth values, the coordinates of each two-dimensional detection box are converted into camera three-dimensional coordinates through the inverse transformation of the intrinsic parameter matrix; The camera's 3D coordinates are converted to world coordinates through the inverse transformation of the extrinsic parameter matrix; Calculate the size of the three-dimensional disease marker color block based on the size of each two-dimensional detection frame; Based on the outer normal vector of the wall surface, combined with the three-dimensional dimensions, and colored according to the disease type, three-dimensional disease marker color blocks are generated on the surface of the processed three-dimensional reconstruction model.

8. A method for intelligent detection of surface defects of ancient city walls based on three-dimensional reconstruction, as described in claim 1, is characterized in that... The process of integrating all two-dimensional detection frame coordinates and disease categories, along with the generated three-dimensional disease marker color blocks, to output the detection results of surface diseases on the ancient city wall includes the following steps: Based on the coordinates of each two-dimensional detection box and the disease category, draw bounding boxes of different colors and disease category labels on the corresponding RGB image to generate an labeled two-dimensional detection image; All labeled 2D detection images are stitched together in serpentine scanning path order to form a 2D panoramic image; By combining the generated two-dimensional panoramic image and three-dimensional disease marker color blocks, the detection results of surface diseases of the ancient city wall are output.

9. An intelligent detection system for surface defects of ancient city walls based on three-dimensional reconstruction, characterized in that, The method for intelligent detection of surface defects of ancient city walls based on three-dimensional reconstruction, as described in claims 1-8, includes: The 3D model preprocessing module is used to load the acquired 3D reconstruction model of the ancient city wall, perform multi-strategy texture restoration processing, analyze the geometric structural features of the processed 3D reconstruction model, and determine the main extension direction, thickness, length and height of the ancient city wall. The intelligent viewpoint planning module is used to generate multiple virtual camera positions based on the determined main extension direction, thickness, length and height of the ancient city wall, combined with a gridded serpentine scanning strategy. The virtual view rendering module is used to render the processed 3D reconstructed model at each virtual camera position to obtain multiple RGB images and their corresponding multiple depth images. The disease intelligent detection module is used to perform disease detection on each acquired RGB image using a pre-trained YOLO object detection model, and obtain the coordinates of the two-dimensional detection box and the disease category. The 3D mapping and annotation module is used to map the coordinates of each 2D detection box and the disease category to 3D space based on the corresponding depth image through coordinate transformation, and generate 3D disease marker color blocks on the surface of the processed 3D reconstruction model. The result fusion output module is used to fuse all the coordinates of the two-dimensional detection boxes and the disease categories, as well as the generated three-dimensional disease marker color blocks, and output the detection results of the surface diseases of the ancient city wall.

10. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that... When the processor executes the computer program, it implements the intelligent detection method for surface defects of ancient city walls based on three-dimensional reconstruction as described in any one of claims 1 to 8.

Images