Three-dimensional reconstruction method, repair method and device for damaged structure

Abstract

The invention discloses a three-dimensional reconstruction method, a restoration method and a device for a damaged structure, and belongs to the technical field of data processing. The reconstruction method comprises the following steps: acquiring three-dimensional point cloud data of a damaged structure based on laser scanning equipment; converting the three-dimensional point cloud data projection into a two-dimensional image, and performing direction gradient feature extraction on the two-dimensional image by using a histogram of oriented gradient HOG mode to obtain HOG features; constructing a gradient field based on the three-dimensional point cloud data, and correcting the gradient field by using the HOG features to obtain a corrected gradient field; and reconstructing a three-dimensional model of the damaged structure by using the corrected gradient field in combination with a Poisson equation. The damage information acquisition accuracy can be improved, and then the repair reliability is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a three-dimensional reconstruction method, a repair method and a device for a damaged structure. Background Art

[0002] In the related art, for damaged structural parts, it is necessary to rely on manual visual inspection and simple measuring tools, which makes it difficult to accurately obtain detailed damage information in the damaged structure, resulting in poor repair reliability. Summary of the invention

[0003] The present invention provides a three-dimensional reconstruction method, a repair method and a device for a damaged structure, which can improve the accuracy of obtaining damage information and thus improve the reliability of repair. The technical solution is as follows: In one aspect, a method for three-dimensional reconstruction of a damaged structure is provided, the method comprising: Acquire three-dimensional point cloud data of damaged structures based on laser scanning equipment; The three-dimensional point cloud data is projected and converted into a two-dimensional image, and directional gradient features are extracted from the two-dimensional image using a histogram of directional gradients (HOG) method to obtain HOG features; Constructing a gradient field based on the three-dimensional point cloud data, and correcting the gradient field using the HOG feature to obtain a corrected gradient field; The three-dimensional model of the damaged structure is reconstructed by using the modified gradient field in combination with the Poisson equation.

[0004] In another aspect, a method for repairing a damaged structure is provided, comprising: Reconstructing a three-dimensional model of the damaged structure using any of the three-dimensional reconstruction methods for the damaged structure described above; Obtain a three-dimensional model of the undamaged structure; Determining a three-dimensional model of a repair component for repairing the damaged structure using the three-dimensional model of the damaged structure and the three-dimensional model of the undamaged structure; The three-dimensional model of the repair component is imported into a 3D printer for 3D printing to obtain a physical repair component for repairing the damaged structure.

[0005] In another aspect, a device for repairing a damaged structure is provided, comprising: A first acquisition unit, configured to reconstruct a three-dimensional model of the damaged structure using any of the three-dimensional reconstruction methods for the damaged structure described above; A second acquisition unit is used to acquire a three-dimensional model of an undamaged structure; a determining unit, configured to determine a three-dimensional model of a repair component for repairing the damaged structure by using the three-dimensional model of the damaged structure and the three-dimensional model of the undamaged structure; The import unit is used to import the three-dimensional model of the repair component into a 3D printer for 3D printing to obtain a physical repair component for repairing the damaged structure.

[0006] In another aspect, a three-dimensional reconstruction device for a damaged structure is provided, the device comprising: An acquisition unit, used for acquiring three-dimensional point cloud data of a damaged structure based on a laser scanning device; An extraction unit, used for projecting the three-dimensional point cloud data into a two-dimensional image, and extracting directional gradient features from the two-dimensional image using a histogram of directional gradients (HOG) method to obtain HOG features; A construction unit, configured to construct a gradient field based on the three-dimensional point cloud data, and to correct the gradient field using the HOG feature to obtain a corrected gradient field; A reconstruction unit is used to reconstruct the three-dimensional model of the damaged structure by using the corrected gradient field in combination with the Poisson equation.

[0007] On the other hand, a computer device is provided, which includes a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to implement the steps of the above-mentioned three-dimensional reconstruction method of the damaged structure and the damaged structure repair method.

[0008] On the other hand, a computer-readable storage medium is provided, wherein a computer program is stored in the storage medium, and when the computer program is executed by a processor, the steps of the above-mentioned three-dimensional reconstruction method of the damaged structure and the repair method of the damaged structure are implemented.

[0009] On the other hand, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the steps of the above-mentioned three-dimensional reconstruction method of a damaged structure and the method for repairing a damaged structure.

[0010] The technical solution provided by the present invention can at least bring the following beneficial effects: The present invention provides a 3D reconstruction method for a damaged structure. When reconstructing a 3D model of the damaged structure, a laser scanning device can be used to accurately obtain 3D point cloud data of the damaged structure. Two methods are used to determine the gradient information of the damaged structure. The first method is to directly construct a gradient field that can realize 3D model reconstruction using the 3D point cloud data. The second method is to extract directional gradient features from a 2D image converted by projection of the 3D point cloud data to obtain HOG features. The gradient field is then corrected using the HOG features, so that the corrected gradient field can more accurately reflect the real gradient changes of the damaged structure, thereby improving the accuracy of the 3D model of the reconstructed damaged structure. The present invention provides a method for repairing a damaged structure, which uses a three-dimensional reconstruction method for a damaged structure to reconstruct an accurate three-dimensional model of the damaged structure, and then uses the three-dimensional model of the undamaged structure and the three-dimensional model of the damaged structure to obtain a three-dimensional model of the damaged part, that is, a three-dimensional model of a repair component used to repair the damaged structure. Since the reconstructed three-dimensional model of the damaged structure has high accuracy, the three-dimensional model of the repair component can accurately describe detailed damage information in the damaged structure, and then after 3D printing using the three-dimensional model of the repair component, a physical repair component used to repair the damaged structure can be obtained, thereby improving the repair reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0012] Figure 1 is a flow chart of a method for three-dimensional reconstruction of a damaged structure provided by an embodiment of the present invention; Figure 2 is a flow chart of a method for repairing a damaged structure provided by an embodiment of the present invention; Figure 3 is a structural diagram of a three-dimensional reconstruction device for a damaged structure provided by an embodiment of the present invention; Figure 4 is a structural diagram of a damaged structure repair device provided by an embodiment of the present invention; Figure 5 It is a hardware architecture diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0013] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0014] As mentioned above, it is difficult to accurately obtain damage information in damaged structures through manual visual inspection and simple measurement tools. In addition, the Poisson reconstruction method based on the gradient field focuses more on the overall structure and is not sensitive enough to local damage.

[0015] Based on this, the inventive concept of the present invention is to extract HOG features from images using the Histogram of Directed Gradients (HOG) method. Since HOG features are more sensitive to local damage, the gradient field is corrected using HOG features, so that in the process of Poisson reconstruction using the gradient field, the reconstructed three-dimensional model can more accurately reflect the real gradient changes in the damaged area, thereby improving the accuracy of the reconstructed three-dimensional model of the damaged structure.

[0016] Please refer to Figure 1 , an embodiment of the present invention provides a three-dimensional reconstruction method of a damaged structure, the method comprising: Step 100, obtaining three-dimensional point cloud data of the damaged structure based on a laser scanning device; Step 102, projecting the three-dimensional point cloud data into a two-dimensional image, and extracting directional gradient features from the two-dimensional image using a histogram of directional gradients (HOG) method to obtain HOG features; Step 104, constructing a gradient field based on the three-dimensional point cloud data, and correcting the gradient field using the HOG feature to obtain a corrected gradient field; Step 106: Reconstruct the three-dimensional model of the damaged structure by using the corrected gradient field combined with the Poisson equation.

[0017] In an embodiment of the present invention, when reconstructing a three-dimensional model of a damaged structure, a laser scanning device can be used to accurately obtain three-dimensional point cloud data of the damaged structure, and two methods are used to determine the gradient information of the damaged structure. The first method is to directly construct a gradient field that can realize three-dimensional model reconstruction using the three-dimensional point cloud data, and the second method is to extract directional gradient features for a two-dimensional image converted by projection of the three-dimensional point cloud data to obtain HOG features, and then use the HOG features to correct the gradient field, so that the corrected gradient field can more accurately reflect the real gradient changes of the damaged structure, thereby improving the accuracy of the three-dimensional model of the reconstructed damaged structure.

[0018] In the description Figure 1 Before describing the execution methods of each step shown, the application scenario of the embodiment of the present invention is described first. The application scenario of the embodiment of the present invention is at least to repair damage to mechanical parts in the field of mechanical engineering and aviation equipment parts in the field of aerospace. For example, mechanical parts form damaged structures under long-term operation or abnormal stress, and aviation equipment parts are subjected to huge aerodynamic and structural stresses during flight, resulting in fatigue or structural damage. When damage occurs in a component, a three-dimensional model of a repair component for repairing the damaged structure can be obtained by three-dimensionally reconstructing the damaged structure, and the repair component can be printed out using 3D printing technology to repair the damaged structure.

[0019] Described below Figure 1How the various steps are performed.

[0020] First, with respect to step 100 , three-dimensional point cloud data of the damaged structure is acquired based on a laser scanning device.

[0021] In an embodiment of the present invention, the laser scanning device can use a device with a higher resolution, for example, a resolution of 0.1 mm. By using the laser scanning device to perform a comprehensive scan of the damaged structure, accurate three-dimensional point cloud data can be obtained, thereby providing a data basis for the subsequent reconstruction of the three-dimensional model of the damaged structure.

[0022] Furthermore, in order to improve data accuracy, the three-dimensional point cloud data obtained by the laser scanning device can also be purified to effectively remove interference information such as background objects and improve data purity, so that the three-dimensional point cloud data after the purification operation can be used for subsequent steps.

[0023] Then, for step 102, the three-dimensional point cloud data is projected and converted into a two-dimensional image, and directional gradient features are extracted from the two-dimensional image using the HOG method to obtain HOG features.

[0024] The reconstruction of three-dimensional models requires the use of gradient features, and the reconstruction of three-dimensional models can be achieved by constructing a gradient field. Considering that there is a certain error in directly using three-dimensional point cloud data to construct a gradient field, the directional gradient histogram method is used to extract directional gradient features from two-dimensional images to correct the directly constructed gradient field, so that the gradient field used to reconstruct the three-dimensional model can better reflect the real gradient changes.

[0025] In one embodiment of the present invention, taking into account the different damage categories of damaged parts in the damaged structure, in order to enable the corrected gradient field to more accurately restore the characteristics of the damaged parts, the directional gradient feature extraction of the two-dimensional image using the directional gradient histogram HOG method to obtain the HOG feature can specifically include: using an improved fully convolutional network model to extract features from the two-dimensional image to output a feature map that can reflect the feature pattern corresponding to the damage category; using the HOG method to extract directional gradient features from the feature map to obtain the HOG feature.

[0026] Since the feature map used to extract the HOG feature has a feature pattern that can reflect the damage category, and the feature map is obtained by outputting the improved fully convolutional network model, the HOG feature extracted from the feature map has more detailed and realistic gradient information at the damage site. Therefore, after the gradient field is corrected using the HOG feature, the gradient information of the corrected gradient field at the damage site is more accurate, thereby improving the reconstruction accuracy of the three-dimensional model.

[0027] In an embodiment of the present invention, the training method of the improved fully convolutional network model includes: Acquire multiple training samples; the training samples include two-dimensional sample images marked with damage categories; wherein the two-dimensional sample images are obtained by projecting and transforming three-dimensional sample point cloud data of the sample damage structure; The fully convolutional network is trained using the multiple training samples in a supervised training manner, so that the fully convolutional network learns the characteristic patterns corresponding to different damage categories, outputs a feature map that can reflect the characteristic patterns corresponding to the corresponding damage categories, and trains an improved fully convolutional network model.

[0028] Since different damage categories have different characteristic patterns, the fully convolutional network is trained using two-dimensional sample images labeled with damage categories, so that the fully convolutional network can learn the characteristic patterns corresponding to different damage categories. Then, when extracting features from the input two-dimensional image in practical applications, the extracted feature map has a characteristic pattern that conforms to its damage category. When the HOG feature is subsequently extracted, the damage category can be determined based on the characteristic pattern in the feature map, making the features of the extracted HOG feature in the gradient direction more accurate.

[0029] In one embodiment of the present invention, the damage category includes at least one of cracks, peeling, deformation, corrosion and wear.

[0030] In one embodiment of the present invention, the architecture of the improved fully convolutional network model and the processing method of the input two-dimensional image thereof include: The macro feature extraction layer is used to quickly extract the macro features of the input two-dimensional image to reduce the resolution of the required output feature map; the macro feature extraction layer can be implemented using a 7×7 convolution kernel and a convolution layer with a stride of 2.

[0031] The multi-scale downsampling layer includes sequentially connected pooling layers of different sizes, which is used to perform multi-scale downsampling on the output of the macro feature extraction layer to capture abstract information of different scales; The upsampling layer is implemented using a transposed convolution layer. For example, a 4×4 convolution kernel and a transposed convolution layer with a stride of 2 are used to gradually restore the resolution of the feature map of the output of the multi-scale downsampling layer. Multiple convolution optimization layers are sequentially located between the above layers to adjust and optimize feature information; The multi-scale feature fusion layer is used to splice and fuse the corresponding block-level feature map in the downsampling with the similar scale feature map after upsampling in the channel dimension according to the division of 8×8 cell units and 2×2 blocks; The final output layer is implemented by a 1×1 convolutional layer to output a feature map of the same size as the input 2D image.

[0032] Among them, the improved fully convolutional network model is trained using training samples of multiple different damage categories. Therefore, when the improved fully convolutional network model is used to extract features from an input two-dimensional image, the output image has a characteristic pattern that conforms to its damage category.

[0033] Furthermore, after obtaining the feature map, the feature map can be extracted by using the Histogram of Directed Gradients (HOG) method to obtain HOG features. The HOG features can be used as key scanning features in the repair process of damaged parts. The HOG feature extraction process can specifically include: The first step is to evenly divide the feature map into D1×D1 pixel cells, where D1 is a positive integer, such as 8×8 cells.

[0034] The second step is to use the central difference method to calculate the gradient amplitude and gradient direction of each pixel in each cell unit; The third step is to divide the calculated gradient direction into multiple histogram intervals. According to the gradient direction of each pixel, its gradient amplitude is accumulated into the corresponding histogram interval, so as to construct the gradient direction histogram of each cell unit. The fourth step is to group the adjacent D2×D2 cells into a block, and perform L2-norm normalization on the gradient direction histogram of the cell units in each block; D2 is a positive integer; for example, 2×2 cell units; In the fifth step, the normalized gradient direction histograms of all blocks are combined in a certain order to form a feature vector that describes the entire image. This feature vector is the HOG feature.

[0035] Since the feature map has a characteristic pattern that matches the damage category, the extracted HOG feature can accurately describe the gradient information at the damage site.

[0036] Finally, step 104 "constructing a gradient field based on the three-dimensional point cloud data, and using the HOG feature to correct the gradient field to obtain a corrected gradient field" and step 106 "using the corrected gradient field in combination with the Poisson equation to reconstruct the three-dimensional model of the damaged structure" are explained at the same time.

[0037] In the embodiment of the present invention, in order to reconstruct the three-dimensional model of the damaged structure, the Poisson algorithm can be used to achieve this, wherein a gradient field needs to be constructed first to serve as the source term in the Poisson equation. Specifically: Step 1040: preprocessing the three-dimensional point cloud data; In one implementation, a distance-based outlier removal method can be used to calculate the average distance from each point in the 3D point cloud data to its nearest neighbor, and points whose distance is greater than a set number of standard deviations of the average distance are considered outliers and removed. For example, the set number is 3 times.

[0038] Step 1042: Calculate the gradient field according to the three-dimensional point cloud data, determine the gradient information by fitting the normal vector of the local surface estimation point, and obtain the gradient field based on the gradient information of all the three-dimensional point cloud data.

[0039] Furthermore, the gradient field constructed directly using 3D point cloud data has gradient information about the damaged part of the damaged structure based on the damaged structure as a whole, which has a certain error overall. In other words, it does not focus on more detailed processing of the damaged part, making the details of the damaged part in the gradient field unclear. However, when repairing the damaged part, more accurate gradient information is needed to obtain the repaired part, so that the repaired part fits the damaged part more closely and improves the reliability of the repair.

[0040] Based on this, in an embodiment of the present invention, the gradient field is corrected using the HOG feature in step 102. Since the HOG feature is extracted from a feature map having a feature pattern that conforms to the damage category, the HOG feature also has features that conform to the damage category. The HOG feature can better reflect more detailed gradient information of the damage site.

[0041] In one implementation, the method of correcting the gradient field using the HOG feature may include: Based on the HOG feature, the gradient direction is divided into multiple intervals, and the direction with the largest magnitude is selected from the multiple intervals as the main direction; Determining the gradient amplitude of a point in the target interval corresponding to the main direction in the gradient field; The gradient amplitude of the target interval in the HOG feature is used to compare and fuse the gradient amplitude of the points in the target interval in the gradient field, so as to obtain a gradient field for reflecting the real gradient change of the damaged structure.

[0042] Since the gradients in certain directions in the HOG feature dominate the histogram, the structures in these directions are more important and are taken as the main directions. Since the edges of the damaged structure at the damaged position are irregular, the target interval corresponding to the main direction in the HOG feature at the damaged position can be regarded as the damaged position. Therefore, the gradient amplitude at the damaged position is used to compare and fuse the gradient amplitude at the corresponding position of the gradient field, so that the gradient information of the gradient field at the damaged position can better reflect the real gradient change.

[0043] In one implementation, the contrast fusion may be implemented by weighted averaging.

[0044] In this way, after contrast fusion, the corrected gradient field at the damage location can not only reflect the gradient change of the overall structure (from the gradient field before correction), but also highlight the directionality of the local structure (from the fusion result of the HOG feature and the gradient field).

[0045] In the embodiments of the present invention, there are at least the following two effects: First, the gradient information of the damaged structure is determined by two methods. The first method is to use the HOG feature extraction method, and the second method is to use the method of constructing the gradient field. The two methods are compared and fused, so that the corrected gradient field can better reflect the real gradient changes of the damaged structure and improve the reconstruction accuracy of the three-dimensional model.

[0046] Second, since the HOG features extracted by the first method are extracted from feature maps with feature patterns that conform to the damage category, the HOG features can more accurately reflect the gradient information of the damaged part. After using the HOG features to correct the gradient field in the second method, the gradient information of the corrected gradient field at the damaged part is more real and accurate, thereby ensuring the accuracy of the repaired parts in the subsequent repair process.

[0047] After obtaining the corrected gradient field, the three-dimensional model reconstruction is completed using the corrected gradient field and Poisson's equation. Specifically: Step 1060: define an initial indicator function, divide the space into discrete grids, and assign initial values ​​to grid points according to the three-dimensional point cloud data; In the embodiment of the present invention, when the space is divided into discrete grids, the neighborhood radius r can be determined according to the distribution density of the three-dimensional point cloud data. For each point cloud point Pi, a local neighborhood Ni is constructed with it as the center and r as the radius, so that the entire space is covered by these local neighborhoods, completing the discretization representation of the space. This step utilizes the distribution information of the point cloud points to divide the space into local areas centered on the point.

[0048] In an embodiment of the present invention, when assigning initial values ​​to grid points, for each grid point in a local neighborhood Ni, the initial value can be set based on the attributes of the point cloud point Pi and the neighborhood information. If Pi is judged to be inside the object (accurately judging the position state of the point based on the connectivity analysis of the point cloud data, the positional relationship with the known structure boundary, and other methods), the initial value of the grid point in Ni is set to a small positive value, such as 0.1; if Pi is outside the object, the initial value of the grid point in the neighborhood is set to a large positive value, such as 1.0. For point cloud points close to the surface of the object, a distance-based linear interpolation function is used to calculate the initial value based on the estimated distance from the point to the surface. The closer the distance to the surface, the closer the value is to 0. When the distance is greater than a certain threshold, a fixed value of 0.5 is taken. In this way, by considering the different position situations of the point cloud points, reasonable initial values ​​are assigned to the grid points to guide the subsequent reconstruction process.

[0049] Further, after the initial value assigned to the grid point is obtained based on the above method, in order to ensure that the initial value is more reasonable and improve the accuracy of the three-dimensional model reconstruction achieved after solving the Poisson equation using the initial value, the initial value used to solve the Poisson equation is obtained after adjustment based on the HOG features of the points in the neighborhood of each grid point. Specifically, it can include: using the HOG features in step 102, calculating the HOG feature vector statistics of the points in the neighborhood of each grid point, and adjusting the initial value of the grid point according to the distribution of the statistical information.

[0050] Specifically, the neighborhood range of each grid point is determined, and for each point in the neighborhood, the corresponding HOG feature vector is calculated according to the HOG feature calculation method, and then the HOG feature vector statistical information of each point in the neighborhood is obtained by using the mean, variance, etc.; According to the distribution of statistical information, if in the neighborhood of the target grid point, the calculated gradient amplitude ratio of the first direction interval is higher than that of other intervals, and the first direction is close to the gradient direction at the damage location, it is further determined that the target grid point is located on the surface of the object, and the initial value of the target grid point is adjusted closer to 0; for the neighborhood of the non-target grid point, since the HOG feature vector is evenly distributed, the initial value of the non-target grid point is adjusted closer to 1.

[0051] In one implementation, the adjustment can be made according to the following formula: Initial Value new = Initial value old ×(1−α×gradient ratio) Among them, α is the adjustment coefficient, and the gradient ratio is the amplitude ratio of the first direction interval.

[0052] For example, suppose that within a certain grid point neighborhood, the calculated gradient amplitude in the 0° - 20° direction interval accounts for a much higher proportion than other intervals, and this direction is close to the known damage direction (for example, the difference is not greater than the set value). If the grid point is close to the surface of the object, its initial value may be adjusted from 0.5 to a value closer to 0, such as 0.2. This is because damage may cause surface feature changes. The closer to the damaged surface, the closer the indicator function value should be to 0. If the distribution of HOG feature vectors in the neighborhood is relatively uniform and there is no obvious characteristic direction, the initial value of the grid point can be appropriately increased to make it closer to the initial value setting outside the object (such as 1.0). If the calculated means of each dimension are close, the variance is small, and the gradient amplitude in a specific direction is evenly distributed, it means that the structural changes in the area are not obvious and it is more likely to be a normal area. The initial value of the grid point can be adjusted from 0.5 to 0.8.

[0053] It should be noted that the adjustment range can be determined according to the degree of change.

[0054] Step 1062: Solve the Poisson equation using the initial value to obtain an indicator function value vector.

[0055] It should be noted that the method of solving the Poisson equation using initial values ​​is a well-known technique in the art and will not be described in detail in this embodiment.

[0056] Step 1064: Based on the indicator function value vector, a surface model is extracted, and a reconstructed three-dimensional model of the damaged structure is obtained through model optimization.

[0057] Please refer to Figure 2 The embodiment of the present invention further provides a method for repairing a damaged structure, which may include: Step 200, reconstructing a three-dimensional model of the damaged structure using any of the above three-dimensional reconstruction methods for the damaged structure; Step 202, obtaining a three-dimensional model of an undamaged structure; Step 204, using the three-dimensional model of the damaged structure and the three-dimensional model of the undamaged structure, determining a three-dimensional model of a repair component for repairing the damaged structure; Step 206: import the three-dimensional model of the repair component into a 3D printer for 3D printing to obtain a physical repair component for repairing the damaged structure.

[0058] Among them, the three-dimensional model of the undamaged structure can be obtained through methods such as historical data calling or symmetric structure generation.

[0059] Since the 3D model of the damaged structure has damage, while the 3D model of the undamaged structure is complete and has no damage, the 3D model of the damaged part can be obtained by subtracting the two 3D models, that is, the 3D model of the repair part used to repair the damaged structure can be obtained. The 3D model of the repair part is imported into a 3D printer to print a physical repair part.

[0060] It should be noted that after obtaining the three-dimensional model of the repair part, the three-dimensional model of the repair part can also be post-processed based on printing requirements, such as adding support structures for 3D printing, adjusting the model position and direction, etc., to achieve 3D printing of the repair part.

[0061] In the embodiment of the present invention, a three-dimensional reconstruction method for a damaged structure is used to reconstruct an accurate three-dimensional model of the damaged structure, and then the three-dimensional model of the undamaged structure and the three-dimensional model of the damaged structure are used to obtain a three-dimensional model of the damaged part, that is, a three-dimensional model of a repair component for repairing the damaged structure. Since the reconstructed three-dimensional model of the damaged structure has high accuracy, the three-dimensional model of the repair component can accurately describe the detailed damage information in the damaged structure, and then after 3D printing using the three-dimensional model of the repair component, a physical repair component for repairing the damaged structure can be obtained, thereby improving the repair reliability.

[0062] Please refer to Figure 3 , an embodiment of the present invention provides a three-dimensional reconstruction device for a damaged structure, the device comprising: An acquisition unit 300 is used to acquire three-dimensional point cloud data of a damaged structure based on a laser scanning device; An extraction unit 302 is used to convert the three-dimensional point cloud data into a two-dimensional image through projection, and extract directional gradient features from the two-dimensional image using a histogram of directional gradients (HOG) method to obtain HOG features; A construction unit 304 is used to construct a gradient field based on the three-dimensional point cloud data, and to correct the gradient field using the HOG feature to obtain a corrected gradient field; The reconstruction unit 306 is used to reconstruct the three-dimensional model of the damaged structure by using the corrected gradient field in combination with the Poisson equation.

[0063] In one embodiment of the present invention, the extraction unit is specifically used to perform feature extraction on the two-dimensional image using an improved fully convolutional network model to output a feature map that can reflect the characteristic pattern corresponding to the damage category; and use the HOG method to extract directional gradient features from the feature map to obtain HOG features.

[0064] In one embodiment of the present invention, the training method of the improved fully convolutional network model includes: Acquire multiple training samples; the training samples include two-dimensional sample images marked with damage categories; wherein the two-dimensional sample images are obtained by projecting and transforming three-dimensional sample point cloud data of the sample damage structure; The fully convolutional network is trained using the multiple training samples in a supervised training manner, so that the fully convolutional network learns the characteristic patterns corresponding to different damage categories, outputs a feature map that can reflect the characteristic patterns corresponding to the corresponding damage categories, and trains an improved fully convolutional network model.

[0065] In one embodiment of the present invention, the damage category includes at least one of cracks, peeling, deformation, corrosion and wear.

[0066] In one embodiment of the present invention, the initial value used to solve the Poisson equation is obtained by adjusting the HOG features of the points in the neighborhood of each grid point.

[0067] Please refer to Figure 4 , an embodiment of the present invention provides a repair device for a damaged structure, the device comprising: A first acquisition unit 400 is used to reconstruct a three-dimensional model of the damaged structure using any of the above three-dimensional reconstruction methods for the damaged structure; A second acquisition unit 402 is used to acquire a three-dimensional model of an undamaged structure; A determination unit 404 is used to determine a three-dimensional model of a repair component for repairing the damaged structure by using the three-dimensional model of the damaged structure and the three-dimensional model of the undamaged structure; The import unit 406 is used to import the three-dimensional model of the repair component into a 3D printer for 3D printing to obtain a physical repair component for repairing the damaged structure.

[0068] It should be noted that the three-dimensional reconstruction device of the damaged structure and the repair device of the damaged structure provided in the above-mentioned embodiments are only illustrated by the division of the above-mentioned functional modules. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the three-dimensional reconstruction device of the damaged structure provided in the above-mentioned embodiments and the three-dimensional reconstruction method embodiment of the damaged structure belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here. Similarly, the repair device of the damaged structure provided in the above-mentioned embodiments and the repair method embodiment of the damaged structure belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0069] The embodiment of the present application also provides a computer device, please refer to Figure 5The computer device includes a processor and a memory, in which at least one instruction, at least one program, a code set or an instruction set is stored, and the at least one instruction, at least one program, a code set or an instruction set is loaded and executed by the processor to implement the three-dimensional reconstruction method of the damaged structure and the repair method of the damaged structure provided by the above-mentioned method embodiments.

[0070] An embodiment of the present application also provides a computer-readable storage medium, on which is stored at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by a processor to implement the three-dimensional reconstruction method of the damaged structure and the repair method of the damaged structure provided in the above-mentioned method embodiments.

[0071] An embodiment of the present application also provides a computer program product, which includes a computer program. A processor of a computer device reads the computer program from a computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the three-dimensional reconstruction method of the damaged structure and the repair method of the damaged structure described in any of the above embodiments.

[0072] For the convenience of description, the above system or device is described by dividing it into various modules or units according to its functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0073] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly contributed to the prior art in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0074] Finally, it should be noted that, in this article, relational terms such as first, second, third and fourth are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0075] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A three-dimensional reconstruction method for a damaged structure, characterized in that: include: Acquire three-dimensional point cloud data of damaged structures based on laser scanning equipment; The three-dimensional point cloud data is projected and converted into a two-dimensional image, and directional gradient features are extracted from the two-dimensional image using a histogram of directional gradients (HOG) method to obtain HOG features; Constructing a gradient field based on the three-dimensional point cloud data, and correcting the gradient field using the HOG feature to obtain a corrected gradient field; The three-dimensional model of the damaged structure is reconstructed by using the modified gradient field in combination with the Poisson equation.

2. The method according to claim 1, characterized in that The method of extracting directional gradient features from the two-dimensional image using a histogram of directional gradients (HOG) to obtain HOG features includes: Using an improved fully convolutional network model to extract features from the two-dimensional image, so as to output a feature map that can reflect the feature pattern corresponding to the damage category; The HOG method is used to extract directional gradient features from the feature map to obtain HOG features.

3. The method according to claim 2, characterized in that The training method of the improved fully convolutional network model includes: Acquire multiple training samples; the training samples include two-dimensional sample images marked with damage categories; wherein the two-dimensional sample images are obtained by projecting and transforming three-dimensional sample point cloud data of the sample damage structure; The fully convolutional network is trained using the multiple training samples in a supervised training manner, so that the fully convolutional network learns the characteristic patterns corresponding to different damage categories, outputs a feature map that can reflect the characteristic patterns corresponding to the corresponding damage categories, and trains an improved fully convolutional network model.

4. The method according to claim 3, characterized in that The damage category includes at least one of cracks, peeling, deformation, corrosion and wear.

5. The method according to any one of claims 1 to 4, characterized in that: The initial value used to solve the Poisson equation is obtained by adjusting the HOG features of the points in the neighborhood of each grid point.

6. A method for repairing a damaged structure, characterized in that: include: Reconstructing a three-dimensional model of the damaged structure using the three-dimensional reconstruction method of the damaged structure described in any one of claims 1 to 5; Obtain a three-dimensional model of the undamaged structure; Determining a three-dimensional model of a repair component for repairing the damaged structure using the three-dimensional model of the damaged structure and the three-dimensional model of the undamaged structure; The three-dimensional model of the repair component is imported into a 3D printer for 3D printing to obtain a physical repair component for repairing the damaged structure.

7. A three-dimensional reconstruction device for a damaged structure, characterized in that: The device comprises: An acquisition unit, used for acquiring three-dimensional point cloud data of a damaged structure based on a laser scanning device; An extraction unit, used for projecting the three-dimensional point cloud data into a two-dimensional image, and extracting directional gradient features from the two-dimensional image using a histogram of directional gradients (HOG) method to obtain HOG features; A construction unit, configured to construct a gradient field based on the three-dimensional point cloud data, and to correct the gradient field using the HOG feature to obtain a corrected gradient field; A reconstruction unit is used to reconstruct the three-dimensional model of the damaged structure by using the corrected gradient field in combination with the Poisson equation.

8. A repair device for a damaged structure, characterized in that: include: A first acquisition unit, configured to reconstruct a three-dimensional model of the damaged structure using the three-dimensional reconstruction method of the damaged structure described in any one of claims 1 to 5; A second acquisition unit is used to acquire a three-dimensional model of an undamaged structure; a determining unit, configured to determine a three-dimensional model of a repair component for repairing the damaged structure by using the three-dimensional model of the damaged structure and the three-dimensional model of the undamaged structure; The import unit is used to import the three-dimensional model of the repair component into a 3D printer for 3D printing to obtain a physical repair component for repairing the damaged structure.

9. A computer device, characterized in that: The computer device includes a memory and a processor, the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to implement the steps of any one of the methods described in claims 1-6.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 6 are implemented.

Images