Equipment surface defect detection method and system based on three-dimensional space reconstruction

Abstract

The invention discloses an equipment surface defect detection method and system based on three-dimensional space reconstruction. The method comprises the following steps: acquiring multi-view spatial data of standard equipment and equipment to be detected, and respectively constructing three-dimensional models containing geometric structures and surface features; registering the three-dimensional point cloud data of the standard equipment and the to-be-detected equipment into the same world coordinate system, and automatically generating two-dimensional rendering pictures of the standard equipment and the to-be-detected equipment with the same visual angle and pose in the same coordinate system; and further performing surface defect detection on the two-dimensional rendering pictures corresponding to the two-dimensional rendering pictures, and visually presenting a defect detection result at a corresponding position of the three-dimensional model. According to the invention, artificial leak detection and error detection are effectively avoided, and the efficiency and accuracy of equipment surface defect detection are remarkably improved.

Description

Technical Field

[0001] This invention relates to the fields of computer vision, three-dimensional spatial reconstruction and intelligent detection technology, and more specifically, to a method and system for detecting surface defects in equipment based on three-dimensional spatial reconstruction.

[0002] This method and system are suitable for automated detection of surface defects in industrial equipment, machinery and their components, and can be widely applied in manufacturing inspection, quality control and equipment maintenance. Background Technology

[0003] In equipment manufacturing and maintenance, the surface quality of products is directly related to structural safety and service life. Existing equipment surface defect detection technologies have mainly gone through the following development stages: the manual or single-position two-dimensional detection stage, and the stage of fixed-position multiple reconstructions and manual verification.

[0004] In the manual or single-camera 2D inspection stage, local 2D images are typically acquired by manual visual inspection or using a fixed-view industrial camera. Inspectors then manually interpret the defects or use image algorithms for assistance. This method is limited by the fixed shooting angle or limited coverage, making it difficult to achieve full coverage of the equipment surface. Defects located in obscured areas, back areas, or complex curved surfaces are easily missed. This not only results in low inspection efficiency but also relies heavily on the experience of the inspectors and is difficult to adapt to equipment with complex structures or large sizes.

[0005] In the stage of multiple reconstructions and manual verification at fixed locations, structured light, laser scanning, or multiple imaging methods are introduced to acquire 3D data in an attempt to compensate for the insufficient coverage of single-view detection. However, such solutions usually require multiple scans or step-by-step reconstructions of the equipment under fixed equipment conditions. The data acquisition process is complex, the reconstruction cycle is long, and the final defect interpretation still requires manual inspection and comparison in a 2D image interface or point cloud view, making it difficult to achieve truly efficient and unified detection.

[0006] In general, existing surface defect detection technologies for equipment suffer from the following problems: reliance on fixed equipment and viewing angles limits the detection coverage, making it difficult to avoid defect omissions; manual inspection relies on the experience and subjective judgment of inspectors, making it difficult to avoid missed or incorrect detections; defect results are often presented in two-dimensional coordinates or partial screenshots, making it difficult to achieve intuitive defect location and visualization; the inspection process is complex, inefficient, and costly to verify. Therefore, there is an urgent need for a detection method and system that can automatically align and compare standard equipment with the equipment under inspection in a unified three-dimensional coordinate system, achieving precise defect location and intuitive display, to effectively avoid human error and improve the efficiency and accuracy of surface defect detection. Summary of the Invention

[0007] The purpose of this invention is to provide a method and system for detecting surface defects of equipment based on three-dimensional spatial reconstruction. By registering the three-dimensional models of standard equipment and the equipment to be tested to the same world coordinate system, and automatically generating two-dimensional rendering images under unified viewpoint and pose conditions for defect detection, the invention achieves automated and accurate detection of surface defects of equipment, effectively avoids human error and omissions, and significantly improves detection efficiency and accuracy.

[0008] To achieve the above objectives, the present invention adopts the following technical solution.

[0009] This invention provides a method for detecting surface defects in equipment based on three-dimensional spatial reconstruction, comprising the following steps: S1. Acquire multi-view spatial data of standard equipment and equipment under test; S2. Construct three-dimensional models containing geometric structures and surface features for both the standard equipment and the equipment to be tested; S3. Register the 3D point cloud data of the standard equipment and the equipment to be tested to the same world coordinate system; S4. Automatically generate 2D rendered images with the same viewpoint and pose in the same coordinate system; S5. Perform surface defect detection on the corresponding two-dimensional rendering images of the standard equipment and the equipment to be tested. S6. Visualize the defect detection results at the corresponding locations in the 3D model.

[0010] Preferably, the multi-view spatial data in S1 includes, but is not limited to, image data, depth data, point cloud data, or fused data acquired by industrial cameras, depth cameras, structured light devices, laser scanning devices, vision and ranging fusion devices, mobile terminals, or combinations thereof.

[0011] Preferably, the construction of the three-dimensional model in S2 includes, but is not limited to, point cloud models, mesh models, voxel models, implicit representation models, or any combination thereof generated based on multi-view geometry, point cloud reconstruction, voxel modeling, implicit field modeling, neural radiation fields or combinations thereof, or improved forms of three-dimensional modeling methods.

[0012] Preferably, the registration in S3 includes, but is not limited to, using feature matching, iterative optimization, deep learning, or a combination thereof, to achieve spatial alignment of the standard device and the device to be detected in a unified world coordinate system.

[0013] Preferably, the automatic generation of two-dimensional rendering images in S4 refers to the automatic generation of two sets of two-dimensional rendering images with the same viewpoint pose by virtual cameras around the standard device and the device to be tested in the same coordinate system at 360°, ensuring that the two viewpoints are consistent and the poses are matched; multiple virtual cameras are distributed on the sphere surrounding the standard device and the device to be tested in the same coordinate system.

[0014] Preferably, the surface defects detected in S5 include, but are not limited to, scratches, cracks, dents, corrosion, defects, color differences, surface anomalies, structural deviations, manufacturing errors, or combinations thereof; And / or, the surface defect detection in S5 includes, but is not limited to, methods based on image processing, methods based on deep learning, methods based on comparative analysis, or combinations thereof.

[0015] Preferably, the defect detection results in S6 include, but are not limited to, marking defects on the 3D model using color, text or symbols, mapping defect probability or severity with heat maps, providing local magnified / cross-sectional views, and supporting interactive 3D browsing with rotation and scaling, so as to realize the visualization of defect location, shape and risk level.

[0016] The present invention also provides a device surface defect detection system based on three-dimensional spatial reconstruction, including but not limited to: The data acquisition module is used to acquire multi-view spatial data of standard equipment and equipment under test; The 3D modeling module is used to construct 3D models of the standard equipment and the equipment to be tested, respectively, containing geometric structure information and surface feature information. The spatial registration module is used to register the 3D point cloud data of the standard equipment and the equipment under test to the same world coordinate system; The rendering synchronization module is used to automatically generate 2D rendered images of a standard device and a device to be tested with the same viewpoint and pose in the same world coordinate system. The defect detection module is used to perform surface defect detection on the corresponding two-dimensional rendering images of the standard equipment and the equipment under test; The results display module is used to visualize the results of defect detection at the corresponding locations in the 3D model.

[0017] The system is used to implement the above-mentioned method for detecting surface defects of equipment based on three-dimensional spatial reconstruction.

[0018] Preferably, each module is implemented in hardware, software, or a combination of hardware and software; And / or, the system is deployed on a local computing device, an edge computing device, a cloud computing platform, or a combination thereof.

[0019] Preferably, the equipment is industrial equipment, mechanical equipment, precision components, manufactured parts, or a combination thereof; And / or, the defect detection system is applicable to equipment with different structural forms, different sizes and different detection scenarios.

[0020] Compared with the prior art, the beneficial effects of the present invention are as follows: It provides a method and system for detecting surface defects of equipment based on three-dimensional spatial reconstruction. By registering the three-dimensional models of standard equipment and the equipment to be tested to the same world coordinate system, and automatically generating two-dimensional rendering images for defect detection under unified viewpoint and pose conditions, it realizes automated and accurate detection of surface defects of equipment, effectively avoids human error in detection and significantly improves detection efficiency and accuracy. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 A flowchart of a device surface defect detection method based on three-dimensional spatial reconstruction provided in an embodiment of the present invention.

[0023] Figure 2 This is a schematic diagram for automatically generating 2D rendered images.

[0024] Figure 3 This is a diagram showing the results of three-dimensional defect localization.

[0025] Figure 4 A block diagram of a device surface defect detection system based on three-dimensional spatial reconstruction provided in an embodiment of the present invention. Detailed Implementation

[0026] In view of the shortcomings of the prior art, the inventors of this invention, through long-term research and extensive practice, have proposed the technical solution of this invention. The following will further explain and illustrate this technical solution, its implementation process, and its principles.

[0027] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0028] Furthermore, in the description of this invention, it should be understood that the terms "upper," "lower," "inner," "outer," "horizontal," "vertical," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0029] In the description of this specification, the references to terms such as "an embodiment," "a particular embodiment," or "the embodiment" indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0030] This invention provides a method for detecting surface defects in equipment based on three-dimensional spatial reconstruction. (See attached document.) Figure 1 This includes the following steps S1-S6.

[0031] S1. Acquire multi-view spatial data of standard equipment and equipment under test.

[0032] S2. Construct three-dimensional models containing geometric structures and surface features for both the standard equipment and the equipment to be tested.

[0033] S3. Register the 3D point cloud data of the standard equipment and the equipment to be tested to the same world coordinate system.

[0034] S4. Automatically generate 2D rendered images with the same viewpoint and pose in the same coordinate system.

[0035] S5. Perform surface defect detection on the corresponding two-dimensional rendering images of the standard equipment and the equipment to be tested.

[0036] S6. Visualize the defect detection results at the corresponding locations in the 3D model.

[0037] The standard equipment and the equipment to be tested are of the same type, such as an elevator traction machine. In step S4, a two-dimensional rendering image of the elevator traction machine is automatically generated, such as... Figure 2 As shown, the result of the three-dimensional localization of the defect in step S6 is as follows: Figure 3 As shown.

[0038] Based on the same inventive concept, this invention also provides a device surface defect detection system based on three-dimensional spatial reconstruction, such as... Figure 4 As shown, including but not limited to the following modules.

[0039] The data acquisition module is used to acquire multi-view spatial data of standard equipment and equipment under test.

[0040] The 3D modeling module is used to construct 3D models of the standard equipment and the equipment to be tested, respectively, containing geometric structure information and surface feature information.

[0041] The spatial registration module is used to register the 3D point cloud data of the standard equipment and the equipment under test to the same world coordinate system.

[0042] The rendering synchronization module is used to automatically generate 2D rendered images of a standard device and a device under test with the same viewpoint and pose in the same world coordinate system.

[0043] The defect detection module is used to perform surface defect detection on the corresponding two-dimensional rendering images of the standard equipment and the equipment to be tested.

[0044] The results display module is used to visualize the results of defect detection at the corresponding locations in the 3D model.

[0045] The modules work together to achieve automated detection of surface defects on the equipment.

[0046] Compared with existing equipment surface defect detection technologies, this method and system have at least the following beneficial effects.

[0047] (1) Unify the three-dimensional spatial reference and eliminate the influence of differences in viewpoint and posture.

[0048] By registering the 3D models of the standard equipment and the equipment under test to the same world coordinate system and generating 2D rendered images under unified viewpoint and pose conditions, the detection errors caused by different equipment placement positions and acquisition angles are effectively eliminated, thereby improving the consistency and reliability of the detection results.

[0049] (2) Based on automated comparison detection, it effectively avoids human error and false detection.

[0050] By automating the comparative analysis of two-dimensional rendered images of standard equipment and equipment under test under uniform observation conditions, the reliance on human experience and subjective judgment is significantly reduced, fundamentally reducing the probability of missed detections and false detections.

[0051] (3) Combining two-dimensional detection with three-dimensional mapping enables precise defect localization.

[0052] Mapping the two-dimensional defect detection results back to the corresponding positions in the three-dimensional model makes the location and distribution of defects in three-dimensional space intuitively visible, avoiding the problem of difficulty in accurately locating defects by relying solely on two-dimensional results.

[0053] (4) Multi-perspective data fusion to improve defect detection coverage.

[0054] By acquiring multi-view spatial data and constructing a complete 3D model, the information loss caused by occlusion or blind spots can be effectively reduced, and the coverage of equipment surface defect detection can be improved.

[0055] (5) The detection process is highly automated, which significantly improves detection efficiency and accuracy.

[0056] The entire inspection process is completed automatically by the system, from data acquisition, 3D modeling, spatial registration to defect detection and visualization, reducing manual intervention, shortening the inspection cycle, and significantly improving the efficiency and accuracy of equipment surface defect detection.

[0057] (6) It has strong applicability and good engineering and industrial application value.

[0058] This invention does not rely on specific 3D reconstruction algorithms or detection models, is applicable to various equipment types and application scenarios, and has good scalability and promotional value.

[0059] The technical solution of the present invention will be further described in detail below with reference to several preferred embodiments and accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Test methods in the following embodiments that do not specify specific conditions are generally performed under conventional conditions.

[0060] The specific implementation process of the method of the present invention is illustrated below with reference to specific embodiments. The method includes the following steps S100-S600.

[0061] Step S100: Obtain multi-view spatial data of the standard device and multi-view spatial data of the device under test.

[0062] Step S200: Based on the acquired multi-view spatial data, construct three-dimensional models containing geometric structures and surface features for the standard device and the device to be tested, respectively. The three-dimensional models contain a large amount of three-dimensional point cloud data.

[0063] Step S300: Register the 3D point cloud data of the standard equipment and the equipment to be tested to the same world coordinate system.

[0064] Step S400: Based on the registered 3D model, automatically generate a 2D rendered image with the same viewpoint and pose in the same coordinate system.

[0065] Step S500: Compare the corresponding two-dimensional rendering images of the standard equipment and the equipment to be tested to achieve surface defect detection.

[0066] Step S600: Visualize the defect detection results at the corresponding positions in the 3D model.

[0067] Spatial data is used to represent information about an object’s location, shape, size distribution, and other aspects, including but not limited to the appearance and spatial information of the device to be tested.

[0068] As an example, step S200 may include: performing regional importance analysis on the multi-view spatial data to determine the importance of each spatial region and selecting key spatial regions related to the detection of surface defects on the equipment; for each spatial region: setting a corresponding reconstruction coefficient for it according to the importance of the spatial region, the reconstruction coefficient including reconstruction accuracy and / or modeling weight; and performing three-dimensional reconstruction on the multi-view spatial data according to the reconstruction coefficient of each spatial region, so as to perform refined modeling of key spatial regions and approximate modeling of non-key spatial regions, thereby generating a three-dimensional model.

[0069] The higher the importance of a spatial region, the greater its reconstruction accuracy and the greater its modeling weight. By setting different reconstruction coefficients, it is possible to achieve refined modeling of key spatial regions and approximate modeling of non-key spatial regions, thereby further improving detection efficiency while ensuring detection accuracy.

[0070] As an example, step S400 may include: projecting the three-dimensional spatial representation units in the three-dimensional model onto the screen space; for each three-dimensional spatial representation unit: calculating its contribution weight to the screen pixels based on its projection position, scale parameters, and visibility relationship in the screen space; and weighting and accumulating the pixel contributions of each three-dimensional spatial representation unit at the same pixel position according to the contribution weight to generate a rasterized image (i.e., a two-dimensional rendered image) from the corresponding viewpoint.

[0071] For each 3D spatial representation unit, the more central its projection position in screen space, the larger its scale parameter, and the clearer its visibility, the greater its contribution weight to screen pixels.

[0072] Rasterization rendering of 3D models enables high-fidelity approximation of continuous radiation fields in screen space, realistically reproducing the device's geometry, edge continuity, and lighting appearance on discrete pixel grids.

[0073] It should be understood that the above embodiments are merely illustrative of the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent changes or modifications made in accordance with the spirit and essence of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A method for detecting surface defects in equipment based on three-dimensional spatial reconstruction, characterized in that, include: S1. Acquire multi-view spatial data of standard equipment and equipment under test; S2. Construct three-dimensional models containing geometric structures and surface features for both the standard equipment and the equipment to be tested; S3. Register the 3D point cloud data of the standard equipment and the equipment to be tested to the same world coordinate system; S4. Automatically generate 2D rendered images with the same viewpoint and pose in the same coordinate system; S5. Perform surface defect detection on the corresponding two-dimensional rendering images of the standard equipment and the equipment to be tested. S6. Visualize the defect detection results at the corresponding locations in the 3D model.

2. The method according to claim 1, characterized in that, The multi-view spatial data in S1 includes, but is not limited to, image data, depth data, point cloud data, or fused data acquired by industrial cameras, depth cameras, structured light devices, laser scanning devices, vision and ranging fusion devices, mobile terminals, or combinations thereof.

3. The method according to claim 1, characterized in that, The construction of the three-dimensional model in S2 includes, but is not limited to, point cloud models, mesh models, voxel models, implicit representation models, or any combination thereof generated based on multi-view geometry, point cloud reconstruction, voxel modeling, implicit field modeling, neural radiation fields or combinations thereof, or improved forms of three-dimensional modeling methods.

4. The method according to claim 1, characterized in that, The registration in S3 includes, but is not limited to, using feature matching, iterative optimization, deep learning, or a combination thereof, to achieve spatial alignment of the standard device and the device to be detected in a unified world coordinate system.

5. The method according to claim 1, characterized in that, The automatic generation of 2D rendering images in S4 refers to the automatic generation of two sets of 2D rendering images with the same viewpoint pose by virtual cameras around the standard device and the device to be tested in the same coordinate system at 360°, ensuring that the viewpoints of the two are consistent and the poses are matched; multiple virtual cameras are distributed on the sphere surrounding the standard device and the device to be tested in the same coordinate system.

6. The method according to claim 1, characterized in that, The surface defects detected in S5 include, but are not limited to, scratches, cracks, dents, corrosion, defects, color differences, surface anomalies, structural deviations, manufacturing errors, or combinations thereof; And / or, the surface defect detection in S5 includes, but is not limited to, methods based on image processing, methods based on deep learning, methods based on comparative analysis, or combinations thereof.

7. The method according to claim 1, characterized in that, The defect detection results in S6 include, but are not limited to, using color, text or symbols to mark defects on the 3D model, mapping defect probability or severity with heat maps, providing local magnified / cross-sectional views, and supporting interactive 3D browsing with rotation and scaling, to achieve a visual presentation of defect location, shape and risk level.

8. A surface defect detection system for equipment based on three-dimensional spatial reconstruction, characterized in that, Including but not limited to: The data acquisition module is used to acquire multi-view spatial data of standard equipment and equipment under test; The 3D modeling module is used to construct 3D models of the standard equipment and the equipment to be tested, respectively, containing geometric structure information and surface feature information. The spatial registration module is used to register the 3D point cloud data of the standard equipment and the equipment under test to the same world coordinate system; The rendering synchronization module is used to automatically generate 2D rendered images of a standard device and a device to be tested with the same viewpoint and pose in the same world coordinate system. The defect detection module is used to perform surface defect detection on the corresponding two-dimensional rendering images of the standard equipment and the equipment under test; The results display module is used to visualize the results of defect detection at the corresponding locations in the 3D model. The system is used to implement the equipment surface defect detection method based on three-dimensional spatial reconstruction as described in any one of claims 1-7.

9. The system according to claim 8, characterized in that, Each module is implemented using hardware, software, or a combination of hardware and software. And / or, the system is deployed on a local computing device, an edge computing device, a cloud computing platform, or a combination thereof.

10. The system according to claim 8, characterized in that, The equipment is industrial equipment, mechanical equipment, precision components, manufactured parts, or a combination thereof; And / or, the defect detection system is applicable to equipment with different structural forms, different sizes and different detection scenarios.

Images