Surface damage three-dimensional topography splicing method based on multi-strategy fusion optimization

Abstract

This invention discloses a method for stitching 3D surface damage morphology based on multi-strategy fusion optimization. Addressing the problems of registration failure, morphology distortion, and blurred contours in existing technologies for stitching damage on weakly textured and highly reflective metal surfaces, this invention first performs adaptive preprocessing of the damage region and dual-domain feature fusion on the 3D point cloud and 2D texture image to achieve robust initial registration. Then, non-rigid fine registration is achieved through damage morphology structural similarity guidance. Finally, global optimization and adaptive fusion while preserving the damage contour output a high-fidelity 3D surface damage morphology. This invention focuses on damage perception throughout the process, significantly improving registration success rate and damage region stitching accuracy through cross-modal feature collaboration, damage-constrained registration, and global and local dual-layer optimization. It effectively suppresses cumulative errors and maintains sharp damage contours, enabling high-precision reconstruction of high-curvature and large-size surface damage, providing reliable support for digital modeling and fault diagnosis of mechanical friction pairs.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional measurement technology, and specifically to a method for stitching together three-dimensional morphology of surface damage based on multi-strategy fusion optimization. Background Technology

[0002] As a key component in hydro-generator units, sliding bearings transmit power and provide support; their operating condition directly affects the safety and stability of the entire unit. Under frequent start-ups and shutdowns, variable loads, and complex operating conditions, sliding bearing surfaces often exhibit various forms of damage, such as fatigue spalling, cavitation pitting, and ablation. Under complex contact conditions, fatigue microcracks easily initiate on the sliding surface, cavitation can lead to material spalling, and poor lubrication can cause localized overheating and ablation. Currently, commonly used two-dimensional detection methods are insufficient to obtain depth information of damage and have limitations in assessing surface damage of irregular shapes. Although three-dimensional detection technology can achieve high-precision reconstruction of surface morphology, its limited field of view in a single measurement cannot completely cover large-sized damage areas. Therefore, multi-view three-dimensional data stitching is needed to achieve full surface morphology measurement.

[0003] In the prior art, the patent with document number CN110415342B provides a three-dimensional point cloud reconstruction device and method based on multiple fusion sensors. This patent is a general-purpose three-dimensional point cloud reconstruction method based on multiple fusion sensors. It uses multiple sensors such as lidar, visible light and infrared to collect data. After voxel filtering for noise reduction, 3D-SIFT key point extraction, KD-Tree and RANSAC coarse registration, and rigid / non-rigid fine registration, the point cloud fusion reconstruction is completed.

[0004] The patent with document number CN120031790A provides a structural damage detection method and system based on multimodal fusion. This patent is a multimodal fusion structural damage detection method that acquires two-dimensional images and three-dimensional point clouds, segments the images using nnU-Net, maps the labels to the point clouds, and performs classification and quantitative analysis on crack / surface damage.

[0005] However, the existing technology has the following drawbacks:

[0006] In CN110415342B, preprocessing, registration, and fusion do not distinguish between damaged areas and normal backgrounds, leading to feature loss and registration failure on weakly textured / highly reflective metal damaged surfaces. It only uses voxel grid filtering, lacking image region enhancement and curvature-aware filtering, resulting in easily filtered-out damage edges and insufficient damage feature contrast. Coarse registration relies solely on the number of interior points for evaluation, while fine registration lacks damage morphology constraints, resulting in large alignment errors and microscopic morphology distortion. CN120031790A only performs damage detection without 3D stitching and reconstruction, failing to achieve multi-viewpoint point cloud registration and stitching, and unable to generate complete 3D morphology for large-size, high-curvature damage. Furthermore, it lacks algorithm optimization for weakly textured, high-curvature, and highly reflective metal surfaces such as hydroelectric generator bearings, making it unsuitable for complex industrial damage scenarios.

[0007] Therefore, developing a three-dimensional topography stitching method that features prominent characteristics, high damage contrast, and adaptability to complex industrial damage scenarios has become an urgent technical problem to be solved in this field. Summary of the Invention

[0008] To address the above problems, this invention provides a method for stitching together three-dimensional surface damage morphology based on multi-strategy fusion optimization, specifically including the following steps: S10. Receive the pre-collected three-dimensional point cloud data of the surface damage area and its corresponding two-dimensional texture image, and perform multi-modal feature extraction and initial matching processing on the three-dimensional point cloud data and its corresponding two-dimensional texture image for the damage area to obtain the optimal initial transformation matrix. S101. Accept the pre-collected three-dimensional point cloud data of the surface damage area and its corresponding two-dimensional texture image, and preprocess it.

[0009] S102. Perform dual-domain extraction and adaptive fusion of damage features on the preprocessed 3D point cloud data and its corresponding 2D texture image to obtain hybrid features.

[0010] S103. Initial registration is performed based on the sampling consistency of hybrid features to obtain the initial transformation matrix. .

[0011] S20. For the optimal initial transformation matrix Non-rigid fine registration guided by damage morphology and structural similarity is performed to output accurately registered point cloud pairs.

[0012] S201, Using the optimal initial transformation matrix Prepare damage area projection and structural similarity measurement data to output depth projection and intensity projection images.

[0013] S202. Construct an optimization objective based on the depth projection image and intensity projection image, fusing damage-aware structural similarity constraints, to obtain the objective function. .

[0014] S203. Iteratively solve and constrain the damage feature point pairs using the high-confidence damage feature point pairs and the composite objective function to obtain a high-precision transformation matrix and a precise registration point cloud pair.

[0015] S30. Based on the accurate registration of point cloud pairs, and after global optimization and adaptive fusion processing to preserve the damage contour, a high-fidelity three-dimensional surface damage morphology map is output.

[0016] S301. Based on multiple precisely registered point cloud pairs and the relative transformation relationships between each point cloud, perform global pose graph optimization processing to preserve the damaged contour, and output the optimized global pose and multiple point clouds in a unified coordinate system.

[0017] S302. Based on a unified coordinate system, perform damage-sensitive multi-scale adaptive fusion processing on multiple point clouds to obtain a preliminary fusion model, denoted as [the preliminary fusion model is missing here]. , It is in the form of point cloud or mesh, containing geometric coordinates and texture information.

[0018] S303. Based on the preliminary fusion model and the damaged areas identified therefrom, local geometric consistency optimization processing is performed to obtain a three-dimensional morphology map of the surface damage.

[0019] Based on the above-described surface damage 3D morphology stitching method based on multi-strategy fusion optimization, a non-transitory computer-readable storage medium is provided storing one or more programs, the programs comprising: instructions that, when executed by one or more processors of an electronic device, cause the electronic device to: The system receives pre-collected three-dimensional point cloud data of the surface damage area and its corresponding two-dimensional texture image, and performs multimodal feature extraction and initial matching processing on the three-dimensional point cloud data and its corresponding two-dimensional texture image for the damage area to obtain the optimal initial transformation matrix. The optimal initial transformation matrix is ​​subjected to non-rigid fine registration guided by the similarity of damage morphology and structure, and the precise registered point cloud pairs are output. Based on the accurate registration of point cloud pairs, and through global optimization and adaptive fusion processing that preserves the damage contour, a high-fidelity three-dimensional morphology map of surface damage is output.

[0020] Based on the aforementioned surface damage three-dimensional morphology stitching method based on multi-strategy fusion optimization, an electronic device is provided, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for performing the following operations: the one or more programs include instructions that, when executed by the one or more processors of the electronic device, cause the electronic device to: The system receives pre-collected three-dimensional point cloud data of the surface damage area and its corresponding two-dimensional texture image, and performs multimodal feature extraction and initial matching processing on the three-dimensional point cloud data and its corresponding two-dimensional texture image for the damage area to obtain the optimal initial transformation matrix. The optimal initial transformation matrix is ​​subjected to non-rigid fine registration guided by the similarity of damage morphology and structure, and the precise registered point cloud pairs are output. Based on the accurate registration of point cloud pairs, and through global optimization and adaptive fusion processing that preserves the damage contour, a high-fidelity three-dimensional morphology map of surface damage is output.

[0021] Compared with the prior art, the beneficial effects of the present invention include: (1) By adaptive enhancement of the damaged area and cross-modal hybrid feature extraction, the traditional algorithm solves the problems of feature loss and matching failure caused by sparse texture and uneven lighting on the metal damaged surface, thus improving the registration robustness.

[0022] (2) Non-rigid fine registration with damage-aware D-SSIM constraint is adopted to prioritize the alignment of damage morphology. Compared with the general registration algorithm, the alignment error of the damage area is reduced, and the microscopic three-dimensional morphology of cracks, spalling and cavitation pits is accurately restored.

[0023] (3) By optimizing the global pose map for damage perception, the cumulative error of multi-view stitching is effectively suppressed, the limitation of single measurement field of view is broken, and the complete three-dimensional shape reconstruction of high curvature and large-size damage area is realized.

[0024] (4) Local geometry optimization adopts the strategy of sharpening the damage edge and smoothing the interior to avoid blurring and distortion of the damage contour after fusion. The reconstructed model can be directly used for accurate quantitative measurement of damage depth, area and length.

[0025] (5) Dynamically weighted fusion of two-dimensional texture and three-dimensional geometric features based on damage confidence, it can stably output high-quality results under different damage types and different surface conditions, and is suitable for complex industrial scenarios such as hydropower unit bearings. Attached Figure Description

[0026] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0027] Figure 1This is a flowchart of the surface damage three-dimensional morphology stitching method based on multi-strategy fusion optimization of the present invention.

[0028] Figure 2 This is a comparison image of the adaptive contrast enhancement effect of the damaged area in the surface damage three-dimensional morphology stitching method based on multi-strategy fusion optimization of the present invention.

[0029] Figure 3 This is a schematic diagram of cross-modal feature mapping in the surface damage three-dimensional morphology stitching method based on multi-strategy fusion optimization of the present invention.

[0030] Figure 4 This is a schematic diagram illustrating the principle of damage morphology structure similarity constraint in the surface damage three-dimensional morphology stitching method based on multi-strategy fusion optimization of the present invention.

[0031] Figure 5 This is a diagram showing the fusion and optimization stitching effect in the surface damage three-dimensional morphology stitching method based on multi-strategy fusion optimization of the present invention. Detailed Implementation

[0032] The overall flowchart of the surface damage three-dimensional morphology stitching method based on multi-strategy fusion optimization described in this invention is as follows: Figure 1 As shown, the specific implementation is as follows: Example 1 S10. Receive pre-collected three-dimensional point cloud data of the surface damage area and its corresponding two-dimensional texture image, and perform multi-modal feature extraction and initial matching processing on the three-dimensional point cloud data and its corresponding two-dimensional texture image for the damage area to obtain the optimal initial transformation matrix. .

[0033] In an embodiment of the present invention, step S10 involves receiving pre-collected three-dimensional point cloud data of the surface damage area and its corresponding two-dimensional texture image, and performing multimodal feature extraction and initial matching processing on the three-dimensional point cloud data and its corresponding two-dimensional texture image for the damage area to obtain the optimal initial transformation matrix. This includes steps S101 to S103.

[0034] S101. Accept the pre-collected three-dimensional point cloud data of the surface damage area and its corresponding two-dimensional texture image, and preprocess it.

[0035] In an embodiment of the present invention, the three-dimensional point cloud data of the surface damage area and its corresponding two-dimensional texture image are acquired by a structured light three-dimensional scanning system.

[0036] In an embodiment of the present invention, the preprocessing includes: The two-dimensional texture image is processed using a contrast enhancement algorithm that adapts to damaged regions.

[0037] In an embodiment of the present invention, the processing of the two-dimensional texture image using a contrast enhancement algorithm that adapts to damaged regions includes: The suspected damaged area was initially located through edge detection and region growing. Subsequently, a weighted CLAHE was applied within this area, with its contrast clipping limit set. Exceeding the background area limit (set up ), to enhance the visibility of damage features such as scratches, peeling, and cavitation pits (e.g. Figure 2 (as shown), while avoiding excessive background enhancement that introduces noise.

[0038] Curvature-aware statistical filtering is applied to the 3D point cloud data.

[0039] In an embodiment of the present invention, the application of curvature-aware statistical filtering to the three-dimensional point cloud data includes: Calculate the local curvature of each point. For high curvature points (which may correspond to damage edges), relax the threshold for the distance to its neighboring points to avoid filtering out true damage features. For low curvature points (flat background), use a strict threshold to filter out outlier noise points.

[0040] In an embodiment of the present invention, the preprocessing calculation formula is as follows: ; in, For the original two-dimensional texture image in pixels grayscale value at that location The area of ​​damage initially detected. Contrast cropping limit, This is higher than the background area limit. To enhance the image in The grayscale value at that location.

[0041] S102. Perform dual-domain extraction and adaptive fusion of damage features on the preprocessed 3D point cloud data and its corresponding 2D texture image to obtain hybrid features.

[0042] In an embodiment of the present invention, S102, performing dual-domain extraction and adaptive fusion of damage features on the preprocessed three-dimensional point cloud data and its corresponding two-dimensional texture image, includes: Extract KAZE feature point set from preprocessed 2D texture image And calculate the FPFH feature on the corresponding 3D point cloud data; The KAZE feature points of the 2D texture image are back-projected into 3D space, and the nearest 3D points in the space are found and bound together. The weights are then fused to obtain a hybrid feature descriptor. .

[0043] In an embodiment of the present invention, the calculation formula for the hybrid feature descriptor is as follows: ; in, The damage confidence score, with a value range of [0,1], represents the probability that the feature point belongs to the damaged region (calculated from local curvature, depth gradient, etc.). The normalized KAZE feature descriptor vector; The normalized FPFH feature descriptor vector; This is a vector concatenation operation; The fused hybrid feature descriptor (cross-modal feature mapping process as follows) Figure 3 (As shown). This fusion strategy makes the algorithm rely more on texture information where damage features are obvious, and more on three-dimensional shape information where geometric features are obvious.

[0044] S103. Initial registration is performed based on the sampling consistency of hybrid features to obtain the initial transformation matrix. .

[0045] In an embodiment of the present invention, step S103 involves initial registration based on the sampling consistency of hybrid features to obtain an initial transformation matrix. ,include: Source cloud to be pieced together and target point cloud Using its hybrid feature descriptors as input, an improved SAC-IA algorithm is employed; random sampling and matching point pairs are performed, and a transformation matrix is ​​calculated to obtain the initial transformation matrix. In the process of randomly sampling and matching point pairs, damage confidence is given priority. High-resolution points are used to improve the discriminative power and robustness of the matching. The evaluation function of the transformation matrix not only calculates the number of interior points but also introduces a term for the alignment error of the damaged region.

[0046] In an embodiment of the present invention, the formula for calculating the alignment error term of the damaged region is as follows: ; in, The value is the improved SAC-IA scoring function value. The larger the value, the better the registration quality. It represents the alignment error term of the damaged area. A higher value indicates better registration quality; This represents the number of interior points under the current transformation matrix (consistent point pairs that satisfy the distance threshold). A subset identified as damage feature points; The weighting coefficient (balancing the number of in-points and the damage alignment error); This represents the number of interior points under the current transformation matrix (consistent point pairs that satisfy the distance threshold). The subset identified as damage feature points (i.e., point pairs with high damage confidence); Damage points in the source point cloud Transformed matrix The coordinates after; In the target point cloud The corresponding matching point.

[0047] This scoring function-guided algorithm, while ensuring global alignment, pays more attention to the local alignment accuracy of the damaged area, and ultimately outputs the optimal initial transformation matrix. .

[0048] S20. For the optimal initial transformation matrix Non-rigid fine registration guided by damage morphology and structural similarity is performed to output accurately registered point cloud pairs.

[0049] In an embodiment of the present invention, S20, the optimal initial transformation matrix is... Non-rigid fine registration guided by damage morphology and structural similarity is performed to output accurately registered point cloud pairs; including: S201, Using the optimal initial transformation matrix Prepare damage area projection and structural similarity measurement data to output depth projection and intensity projection images.

[0050] In an embodiment of the present invention, step S201 utilizes the optimal initial transformation matrix. Preparation is performed for damage area projection and structural similarity measurement to output depth projection images and intensity projection images, including: Using the optimal initial transformation matrix Source cloud Coarsely align to target point cloud Nearby, obtained Extract the overlapping region of the two point clouds and project it onto a two-dimensional plane along the average normal to generate a depth projection image. and (Pixel values ​​represent depth) and intensity projection images and (Pixel value is reflection intensity).

[0051] S202. Construct an optimization objective based on the depth projection image and intensity projection image, fusing damage-aware structural similarity constraints, to obtain the objective function. .

[0052] In an embodiment of the present invention, step S202, constructing an optimization objective that fuses damage-aware structural similarity constraints using the depth projection image and intensity projection image to obtain a composite objective function, includes: Calculate damage-sensing D-SSIM and construct a composite objective function.

[0053] It should be noted that one of the core improvements of this invention lies in transforming the general SSIM metric into damage-aware weighted structural similarity (D-SSIM). Traditional SSIM treats all regions in an image equally, while D-SSIM assigns higher weights to damaged regions, making the optimization process more focused on matching the damage morphology. Defining the damage weight map... The weight of each pixel The weight of regions with obvious damage features is determined by the local curvature and depth gradient of their corresponding 3D points. Therefore, D-SSIM is defined as: ; in, Damage weight map at pixels The value at that location is determined by the local curvature and depth gradient of the corresponding 3D point; For Within the local window centered on the weight Weighted mean under the influence of the action; For weighted variance; For weighted covariance; It is a constant used to avoid the denominator being zero (take...). , (Pixel dynamic range); D-SSIM is the damage-aware structural similarity index. The closer the value is to 1, the more similar the two images are in the damage region. Its constraint principle is as follows: Figure 4 As shown. The composite objective function for fine registration is: ; in, Let be the rotation matrix and translation vector to be solved; For the first Matching points (points in the source point cloud after initial transformation and corresponding points in the target point cloud); In the target point cloud The normal vector of a point; These are the weighting coefficients for each item, balancing geometric distance, depth structural similarity, and strength structural similarity; A depth image generated by projecting the source point cloud and the target point cloud; For intensity projection images; This represents the total error, which needs to be minimized.

[0054] This function, through the D-SSIM term, forces the stitched damaged areas to maintain coherence and consistency in both three-dimensional depth and two-dimensional texture appearance.

[0055] S203. Iteratively solve and constrain the damage feature point pairs using the high-confidence damage feature point pairs and the composite objective function to obtain a high-precision transformation matrix and a precise registration point cloud pair.

[0056] In an embodiment of the present invention, step S203, which iteratively solves the problem using the high-confidence damage feature point pairs and the composite objective function while constraining the damage feature point pairs to obtain a high-precision transformation matrix and a precise registration point cloud pair, includes: in each iteration of establishing point correspondence, in addition to using KD-Tree for nearest neighbor search, an additional hard constraint is added between the correspondences between the high-confidence damage feature points extracted in S102. These feature point pairs participate in error calculation but are protected during dynamic threshold elimination to ensure that the matching of the damage region is not erroneously eliminated. The Gauss-Newton method is used to solve the above nonlinear optimization problem until convergence is achieved to obtain a precise registration point cloud pair, i.e., to obtain the high-precision transformation matrix. .

[0057] S30. Based on the accurate registration of point cloud pairs, and after global optimization and adaptive fusion processing to preserve the damage contour, a high-fidelity three-dimensional surface damage morphology map is output.

[0058] In an embodiment of the present invention, step S30, based on accurately registered point cloud pairs and performing global optimization and adaptive fusion processing to preserve the damage contour, outputs a high-fidelity three-dimensional surface damage topography map, including: S301. Based on multiple precisely registered point cloud pairs and the relative transformation relationships between each point cloud, perform global pose optimization processing to preserve the damaged contour, and output the optimized global pose and multiple point clouds in a unified coordinate system (denoted as S301). ).

[0059] In an embodiment of the present invention, step S301, based on multiple precisely registered point cloud pairs and the relative transformation relationships between each point cloud, optimizes the global pose map while preserving the damaged contour, and outputs the optimized global pose and multiple point clouds in a unified coordinate system, including: Construct a damage-aware pose graph and adjust the edge covariance matrix. And minimize global error.

[0060] When constructing the pose graph, connect the edges covariance matrix It is no longer isotropic. According to the edges... The abundance and distribution of damage features in the overlapping regions of the two associated point clouds are adjusted: if the overlapping region is rich in damage features, then less uncertainty (i.e., higher confidence) is assigned to the local principal direction of the feature distribution, resulting in stronger constraints. The optimization problem is rewritten as: ; in, This is the spatial distribution matrix of the damage characteristics; The pose (transformation matrix) to be optimized for the point cloud in each frame; It is a set of edges in the pose graph, where each edge represents a relative transformation between two frames. ; For the edge The covariance matrix is ​​based on the spatial distribution matrix of damage features in the overlapping region. Adaptive adjustment reduces uncertainty (stronger constraints) in the direction of damage enrichment. For the logarithmic mapping over the Lie algebra, the pose error is converted into a vector; The square of the Mahalanobis distance, i.e. This optimization provides stronger constraints on the stitching of densely damaged areas, thereby better preserving the geometric integrity of the damage profile.

[0061] S302. Based on a unified coordinate system, perform damage-sensitive multi-scale adaptive fusion processing on multiple point clouds to obtain a preliminary fusion model, denoted as [the preliminary fusion model is missing here]. It can be in the form of point clouds or meshes, containing geometric coordinates and texture information.

[0062] In an embodiment of the present invention, step S302 involves performing damage-sensitive multi-scale adaptive fusion based on multiple point clouds in a unified coordinate system to obtain a preliminary fusion model, including: The damage perception confidence score is calculated and weighted fusion is used to generate a preliminary fusion model.

[0063] In an embodiment of the present invention, under a unified coordinate system, for a point in the overlapping region, its origin from the first... Damage perception confidence of patch cloud : ; in, This represents the local registration residual at that point. The visual salience of the damage in the local area where the point is located (calculated from the projected image). This is the adjustment coefficient. The confidence level will increase accordingly in areas of significant damage. The point attribute fusion formula is: Finally, guided by the weights generated from the confidence graph, Laplace pyramid fusion is applied to generate a preliminary fusion model.

[0064] S303. Based on the preliminary fusion model and the damaged areas identified therefrom, local geometric consistency optimization processing is performed to obtain a three-dimensional morphology map of the surface damage.

[0065] In an embodiment of the present invention, step S303, based on the preliminary fusion model and the identified damage regions therein, performs local geometric consistency optimization processing oriented towards damage features to obtain a three-dimensional morphology map of surface damage, including: Detecting the damaged region and boundary and minimizing the local energy function .

[0066] After completing global fusion, local geometric consistency optimization is performed on the damaged areas (such as pits and cracks) identified in the initial fusion model. A local tangent plane is established centered on each damage feature point, and the consistency of its neighboring points in the normal direction is checked. A local energy function is then constructed. The aim is to smooth out the subtle, step-like geometric distortions that may result from the fusion of data from different perspectives, while strictly maintaining the original sharpness of the damage features: ; in, Points within the damaged area The coordinates to be optimized in the normal direction, Its Laplacian operator (for smoothing). This is its original coordinate in the initial fusion model. As a damage edge indicator factor, at the damage boundary detected by local curvature To close the smooth, maintain the sharp edges, and flatten the area inside the damage. To allow for moderate smoothing, and For weight parameters, It is the set of all damage feature points identified by the damage detection algorithm in the preliminary fusion model. This is achieved by minimizing... This process achieves final refinement of the geometric model of the damaged area, eliminating fusion imperfections while ensuring that key geometric features of the damage itself (such as edges and depth) are not blurred. The result is a high-fidelity 3D morphology of the surface damage that is rich in detail, has clear contours, and is generally smooth. The final stitching and optimization effect is as follows: Figure 5 As shown.

[0067] This invention relates to a damage feature-driven cross-modal fusion: by co-processing two-dimensional images and three-dimensional point clouds, and specifically enhancing the feature representation of damaged areas, it overcomes the matching problem caused by texture destruction and feature weakening on damaged surfaces, and significantly improves the success rate and stability of initial registration.

[0068] Precise alignment with dual constraints of morphology and texture: The algorithm innovatively introduces the two-dimensional structural similarity of the damaged area as a soft constraint into the three-dimensional fine registration process, enabling the algorithm to perceive and correct local non-rigid deformations, achieving high-fidelity alignment of the micromorphology of the damage, and surpassing the limitations of traditional rigid registration.

[0069] Intelligent fusion of global optimization and damage contour preservation: By suppressing cumulative errors through global pose optimization and combining damage-sensitive adaptive fusion strategies, the overall model can be seamlessly stitched together while effectively maintaining the sharpness and geometric integrity of the damage contour, providing a high-precision three-dimensional data foundation for subsequent quantitative damage measurement and assessment.

[0070] It should be understood that although the above description follows a certain order, these steps are not necessarily executed in that order. Unless otherwise expressly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, some steps in this embodiment may include multiple steps or multiple stages, which are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be performed alternately or in turn with other steps or at least a portion of the steps or stages in other steps.

[0071] Example 2 An embodiment of the present invention also provides an electronic device, including: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing the following operations: the one or more programs include instructions that, when executed by the one or more processors of the electronic device, cause the electronic device to: S10. Receive the pre-collected three-dimensional point cloud data of the surface damage area and its corresponding two-dimensional texture image, and perform multi-modal feature extraction and initial matching processing on the three-dimensional point cloud data and its corresponding two-dimensional texture image for the damage area to obtain the optimal initial transformation matrix. S20. Perform non-rigid fine registration processing on the optimal initial transformation matrix guided by the similarity of damage morphology and structure, and output the accurately registered point cloud pairs. S30. Based on the accurate registration of point cloud pairs, and after global optimization and adaptive fusion processing to preserve the damage contour, a high-fidelity three-dimensional surface damage morphology map is output.

[0072] The communication bus of the aforementioned electronic devices can be a standard bus for interconnecting peripheral components or an extended industrial standard structure bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not indicate that there is only one bus or one type of bus.

[0073] The communication interface is used for communication between the aforementioned terminal and other devices.

[0074] The memory may include random access memory or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0075] The processors mentioned above can be general-purpose processors, including central processing units, network processors, etc.; they can also be digital signal processors, application-specific integrated circuits, field-programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0076] The electronic devices include user equipment and network equipment. The user equipment includes, but is not limited to, computers, smartphones, and PDAs. The network equipment includes, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing, which is a type of distributed computing consisting of a super virtual computer composed of a group of loosely coupled computers. The computer equipment can operate independently to implement the present invention, or it can connect to a network and interact with other computer equipment within the network to implement the present invention. The network in which the computer equipment is located includes, but is not limited to, the Internet, wide area networks (WANs), metropolitan area networks (MANs), local area networks (LANs), and VPN networks.

[0077] It should also be understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0078] Example 3 In embodiments of the present invention, a non-transitory computer-readable storage medium is also provided for storing one or more programs, said one or more programs including instructions that, when executed by one or more processors of an electronic device, cause the electronic device to: S10. Receive the pre-collected three-dimensional point cloud data of the surface damage area and its corresponding two-dimensional texture image, and perform multi-modal feature extraction and initial matching processing on the three-dimensional point cloud data and its corresponding two-dimensional texture image for the damage area to obtain the optimal initial transformation matrix. S20. Perform non-rigid fine registration processing on the optimal initial transformation matrix guided by the similarity of damage morphology and structure, and output the accurately registered point cloud pairs. S30. Based on the accurate registration of point cloud pairs, and after global optimization and adaptive fusion processing to preserve the damage contour, a high-fidelity three-dimensional surface damage morphology map is output.

[0079] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Furthermore, any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include at least one of non-volatile and volatile memory.

[0080] It should be understood that, as used herein, the singular form "a" is intended to include the plural form as well, unless the context clearly supports an exception. It should also be understood that, as used herein, "and / or" refers to any and all possible combinations of one or more of the associatedly listed items. The embodiment numbers disclosed above are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0081] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of the invention (including the claims) is limited to these examples. Within the framework of the invention, technical features of the above embodiments or different embodiments can be combined, and many other variations of different aspects of the invention exist, which are not provided in the details for the sake of brevity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the protection scope of the invention.

Claims

1. A method for stitching together three-dimensional surface damage morphology based on multi-strategy fusion optimization, characterized in that, Includes the following steps: S10. Receive and preprocess the three-dimensional point cloud data of the surface damage area and its corresponding two-dimensional texture image, and perform multimodal feature extraction and initial matching processing on the preprocessed three-dimensional point cloud data and its corresponding two-dimensional texture image for the damage area to obtain the optimal initial transformation matrix. S20. Perform non-rigid fine registration processing on the optimal initial transformation matrix guided by the similarity of damage morphology structure, and output accurately registered point cloud pairs, including: The optimal initial transformation matrix is ​​used to prepare for the projection of the damaged area and the measurement of structural similarity. An optimization objective is constructed that incorporates damage-sensing structural similarity constraints to obtain a composite objective function; Iterative solution and damage feature point pair constraints are used to obtain accurate registration point cloud pairs; S30. Based on the accurate registration of point cloud pairs, global optimization and adaptive fusion processing are performed to preserve the damage contour, and a high-fidelity three-dimensional surface damage morphology map is output.

2. The surface damage three-dimensional morphology stitching method based on multi-strategy fusion optimization according to claim 1, characterized in that, The two-dimensional texture image is processed using a contrast enhancement algorithm that adapts to damaged regions. Suspected damaged regions are initially located through edge detection and region growing. Then, a weighted CLAHE algorithm is applied within these regions, with a contrast cropping limit set. Exceeding the background area limit .

3. The surface damage three-dimensional morphology stitching method based on multi-strategy fusion optimization according to claim 1, characterized in that, Curvature-aware statistical filtering is applied to the 3D point cloud data. Specifically, the local curvature of each point is calculated. For high curvature points, the threshold for the distance to their neighboring points is relaxed to avoid filtering out real damage features. For low curvature points, a strict threshold is used to filter out outlier noise points.

4. The surface damage three-dimensional morphology stitching method based on multi-strategy fusion optimization according to claim 1, characterized in that, Extract KAZE feature point set from preprocessed 2D texture image And calculate the FPFH feature on the corresponding 3D point cloud data; The KAZE feature points of the 2D texture image are back-projected into 3D space, and the nearest 3D points in the space are found and bound together. The weights are then fused to obtain a hybrid feature descriptor. , The expression is as follows: ； in, The damage confidence score ranges from [0,1], representing the probability that the feature point belongs to the damaged region. The normalized KAZE feature descriptor vector; The normalized FPFH feature descriptor vector; This is a vector concatenation operation.

5. The surface damage three-dimensional morphology stitching method based on multi-strategy fusion optimization according to claim 1, characterized in that, The method for obtaining the optimal initial transformation matrix is ​​as follows: Source cloud to be pieced together and target point cloud The algorithm takes the input of the feature descriptor and its hybrid feature descriptor, and adopts the improved SAC-IA algorithm; it randomly samples and matches point pairs, and performs transformation matrix to obtain the initial transformation matrix; Among these, when randomly sampling and matching point pairs, damage confidence is selected first. High points are used to improve the discriminative power and robustness of the matching.

6. The surface damage three-dimensional morphology stitching method based on multi-strategy fusion optimization according to claim 5, characterized in that, The evaluation function of the transformation matrix also introduces a damaged region alignment error term, which is calculated using the following formula: ； in, The value is the improved SAC-IA scoring function value. The larger the value, the better the registration quality. It represents the alignment error term of the damaged area. A higher value indicates better registration quality; This represents the number of interior points under the current transformation matrix. A subset identified as damage feature points; These are the weighting coefficients; This represents the number of interior points under the current transformation matrix. A subset identified as damage feature points; Damage points in the source point cloud Transformed matrix The coordinates after; In the target point cloud The corresponding matching point.

7. The surface damage three-dimensional morphology stitching method based on multi-strategy fusion optimization according to claim 1, characterized in that, When constructing the optimization objective that integrates damage-aware structural similarity constraints, the general SSIM metric is improved to damage-aware weighted structural similarity D-SSIM. The expression for D-SSIM is: ； in, Damage weight map at pixels The value at that location is determined by the local curvature and depth gradient of the corresponding 3D point; For Within the local window centered on the weight Weighted mean under the influence of the action; For weighted variance; For weighted covariance; The constant is used to avoid the denominator being zero; D-SSIM is the damage-sensing structural similarity index, and the closer the value is to 1, the more similar the two images are in the damage area.

8. The surface damage three-dimensional morphology stitching method based on multi-strategy fusion optimization according to claim 7, characterized in that, The method described above obtains the composite objective function through the D-SSIM term, which mandates that the stitched damaged regions maintain coherence and consistency in both three-dimensional depth and two-dimensional texture appearance. The expression for the composite objective function is as follows: ； in, Let be the rotation matrix and translation vector to be solved; For the first For matching points; In the target point cloud The normal vector of a point; These are the weighting coefficients for each item, balancing geometric distance, depth structural similarity, and strength structural similarity; A depth image generated by projecting the source point cloud and the target point cloud; For intensity projection images; This represents the total error, which needs to be minimized.

9. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method described in any one of claims 1-8.

10. A computer electronic device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1-8.

Images