Method and device for on-line detection of three-dimensional defects on the surface of a belt based on the fusion of a line laser and vision

Abstract

The application discloses a kind of based on line laser and vision fusion's belt surface three-dimensional defect on-line detection method and device.The method includes: line laser projector projects line laser stripe to belt surface, binocular camera synchronously collects left and right view (S1);Based on epipolar constraint and Steger algorithm, subpixel level stripe center positioning is carried out (S2);Based on triangulation principle, three-dimensional point cloud reconstruction is carried out (S3);Through normal vector estimation and curvature analysis, defect geometric feature is extracted (S4);Geometric feature and texture feature are fused, and the type of defect is identified using multi-feature fusion classifier (S5).The application realizes the high-precision three-dimensional detection and intelligent classification of belt surface defects, the comprehensive detection rate reaches 98%, the classification accuracy reaches 97.2%, the positioning accuracy is better than ±0.11mm, and can be widely applied to the on-line detection of belt conveyor in mining, port, power and other industries.

Description

Technical Field

[0001] This invention belongs to the field of industrial visual inspection and non-destructive testing technology, specifically relating to an online detection method and device for three-dimensional defects on the surface of a belt conveyor based on line laser active projection and binocular vision fusion. Background Technology

[0002] Belt conveyors are widely used material handling equipment in industries such as mining, ports, power, and metallurgy. During long-term operation, belts are prone to surface defects such as cracks, scratches, dents, bulges, and detachment due to material impact, mechanical friction, and aging. If these defects are not detected and addressed promptly, they can lead to belt breakage, material spillage, and even safety accidents, causing significant economic losses. Therefore, real-time and accurate online detection of belt surface defects is of significant engineering importance.

[0003] Existing belt defect detection technologies mainly include the following categories: (1) Detection method based on two-dimensional vision: Comparative document D1 (CN114359246A) discloses a conveyor belt detection method based on stereo vision and convolutional neural network. This method uses visible light images for two-dimensional defect recognition, but cannot obtain three-dimensional depth information of the defect. It has limited detection capability for surface height variation defects (such as pits and bulges) and is greatly affected by changes in ambient light.

[0004] (2) Three-dimensional detection method based on line laser: Reference document D2 (CN102297658B) discloses a three-dimensional information detection method based on dual-line laser, which uses line structured light for three-dimensional measurement. However, this method is not optimized for the online belt inspection scenario, lacks real-time and robustness assurance measures, and does not involve defect feature fusion and classification. Reference document D4 discloses a three-dimensional surface defect online detection method based on line structured light scanning, but it only uses a monocular camera to acquire stripe images and does not adopt a binocular fusion strategy, thus limiting the accuracy of three-dimensional reconstruction.

[0005] (3) Three-dimensional detection in other fields: Reference document D3 (CN101639452B) discloses a three-dimensional detection method for rail surface defects. It is aimed at rail detection scenarios. The surface characteristics, running speed and defect types of the detection objects are quite different from those of belts, and it cannot be directly applied to belt online detection.

[0006] In summary, the existing technologies have the following shortcomings: (1) Two-dimensional vision methods lack three-dimensional geometric information and are not capable of detecting highly variable defects; (2) Monocular laser methods have limited accuracy in three-dimensional reconstruction; (3) There is a lack of multi-feature fusion defect classification methods for belt scenarios; (4) The real-time performance and environmental adaptability of online detection need to be improved. Summary of the Invention

[0007] The technical problem to be solved by the present invention is to overcome the shortcomings of existing belt surface defect detection methods in terms of three-dimensional information acquisition, detection accuracy, defect classification capability and online real-time performance, and to provide a detection method and device that can acquire the three-dimensional morphology of belt surface online, accurately quantify the geometric features of defects and perform intelligent classification.

[0008] To address the aforementioned technical problems, this invention provides an online detection method for three-dimensional defects on belt surfaces based on the fusion of line laser and vision. A line laser projector projects line laser stripes onto the surface of the belt, simultaneously triggering a binocular camera to acquire left and right view images; the projection angle of the line laser projector and the optical axis of the binocular camera are 30°-60°, and the projection width of the stripes onto the belt surface is 1-3mm.

[0009] Initial matching of the left and right views is performed based on epipolar geometric constraints. The Steger algorithm is used to extract the subpixel-level center line of the stripe image. The matching accuracy is optimized by epipolar constraints to obtain the subpixel coordinates of the stripe center. The Steger algorithm is based on the eigenvalue analysis of the Hessian matrix and performs subpixel interpolation in the stripe normal direction.

[0010] Based on the principle of triangulation, three-dimensional point cloud reconstruction is performed using calibrated binocular camera parameters and matched stripe center point pairs to obtain three-dimensional topographic data of the belt surface; the point cloud is globally optimized using the bundle adjustment method to improve reconstruction accuracy.

[0011] Normal vector estimation and curvature analysis are performed on the 3D point cloud to extract the geometric features of the defect region, including defect depth, area and volume; the normal vector estimation adopts the PCA neighborhood analysis method, and the curvature analysis includes Gaussian curvature and mean curvature calculation.

[0012] The three-dimensional geometric features and two-dimensional texture features are fused together and input into a fusion classifier for defect type identification. The fusion classifier includes a geometric feature branch, a texture feature branch, and an attention fusion layer, and outputs the defect type and confidence level.

[0013] This invention leverages the powerful feature representation capabilities of large-scale visual models, extracting high-level semantic features of targets through a pre-trained visual Transformer model. Compared to the small-scale backbone networks used in traditional few-shot methods, it achieves more discriminative and generalizable feature representations, significantly improving target recognition accuracy under few-shot conditions. Experiments show that, in 1-shot and 5-shot settings, the recognition accuracy of this invention is improved by 12.3% and 8.7% compared to traditional methods, respectively.

[0014] By adopting the above technical solution, the present invention has the following beneficial effects: (1) High accuracy of 3D reconstruction: By actively projecting line laser and fusing binocular vision, combined with epipolar constraints and Steger subpixel extraction, the accuracy of 3D reconstruction can reach ±0.1mm, which is more than 50% higher than that of monocular line laser method.

[0015] (2) Strong defect detection capability: By extracting the three-dimensional geometric features of defects through normal vector estimation and curvature analysis, the depth, area and volume of defects can be accurately quantified, and the height variation defects such as pits and bulges can be effectively detected, overcoming the limitations of two-dimensional methods.

[0016] (3) High classification accuracy: The multi-feature fusion classifier combines three-dimensional geometric features and two-dimensional texture features, and the defect classification accuracy reaches more than 96.5%, which is significantly better than the single feature classification method.

[0017] (4) Good online real-time performance: The optimized algorithm process and parallel processing architecture can process up to 30 frames of images per second, which meets the online detection requirements of belt conveyors.

[0018] (5) Strong environmental adaptability: The line laser active light source reduces the dependence on ambient light, and the Steger algorithm has low requirements for the quality of stripe images. It can still work stably in harsh environments such as dust and vibration. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the overall structure of the detection system of the present invention.

[0020] Figure 2 This is a schematic diagram of the line laser triangulation principle of the present invention.

[0021] Figure 3 This is a flowchart of the detection method of the present invention.

[0022] Figure 4 This is a diagram showing the relationship between binocular vision calibration and coordinate system in this invention.

[0023] Figure 5 This is a schematic diagram of the three-dimensional point cloud reconstruction and defect feature extraction of the present invention.

[0024] Figure 6 This is a block diagram of the multi-feature fusion classifier structure of the present invention.

[0025] Figure 7 This is a flowchart of a method and apparatus for online detection of three-dimensional defects on belt surfaces based on line laser and vision fusion, according to one embodiment of the present invention. Detailed Implementation

[0026] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The following embodiments are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0027] Example 1 The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0028] like Figure 1 As shown, the online detection system for three-dimensional defects on the belt surface of the present invention mainly includes: a line laser projector (wavelength 650nm, power 50mW) mounted on a bracket above the belt conveyor, a left camera and a right camera (industrial CMOS camera, resolution 2048×1536, frame rate 30fps), a light source controller, and an image processing unit. The installation distance between the line laser projector and the binocular camera is 300mm, the projection angle is 45° with the optical axis, and the baseline distance is 350mm.

[0029] The testing process is as follows: (1) System calibration stage: The binocular camera is calibrated with intrinsic parameters (focal length, principal point, distortion coefficient) and stereo calibration (relative pose R, T) using a checkerboard calibration board. The calibration accuracy is better than 0.05 pixels.

[0030] (2) Online inspection stage: The belt runs at a speed of 2 m / s, a line laser projector projects a straight line stripe onto the belt surface, and a binocular camera simultaneously acquires left and right views at 30 fps. For example... Figure 2 As shown, the line laser stripes form bright light stripes on the surface of the belt, with a stripe width of approximately 2 mm.

[0031] (3) Image processing stage: Epipolar correction is performed on the acquired left and right views, and the Steger algorithm is used to extract the stripe center line (sub-pixel accuracy 0.1 pixels). The left and right views are matched by epipolar constraints.

[0032] (4) Three-dimensional reconstruction stage: such as Figure 4-5 As shown, the coordinates of three-dimensional points are calculated based on the principle of triangulation, and the point cloud is optimized by the bundle adjustment method, achieving a three-dimensional reconstruction accuracy of ±0.08mm.

[0033] (5) Defect Detection Stage: Perform normal vector estimation (K=20 neighborhood) and curvature analysis on the point cloud to extract geometric features such as defect depth, area, and volume. Fuse the geometric features with two-dimensional texture features (GLCM gray-level co-occurrence matrix, LBP local binary pattern, HOG histogram of oriented gradients) and input them into a multi-feature fusion classifier, such as... Figure 6 As shown, the output shows the defect type and confidence level.

[0034] Example 2 The difference between this embodiment and Embodiment 1 is that the line laser projector has a wavelength of 532nm (green light), which provides a better signal-to-noise ratio in bright industrial environments; the binocular camera has a resolution of 4096×3072 and a frame rate of 15fps, making it suitable for high-precision detection scenarios. The baseline distance is set to 450mm, and the projection angle is 35°.

[0035] In this embodiment, the detection performance for different defect types is shown in the table below: Table 1 Performance data of various defect detection types in Example 2 As shown in Table 1, the method of the present invention has a comprehensive detection rate of 98.0% for all types of defects, a classification accuracy of 97.2%, and a positioning accuracy better than ±0.11mm, which can meet the requirements of online belt inspection in industrial settings.

Claims

1. A method and apparatus for online detection of three-dimensional defects on belt surfaces based on line laser and vision fusion, characterized in that: This method projects line laser stripes onto the surface of a running conveyor belt using a line laser projector, and simultaneously triggers the left and right cameras of a binocular camera to acquire left and right view images, respectively. After image acquisition, the method performs initial matching of the left and right views based on epipolar geometric constraints, and uses the Steger algorithm to extract the sub-pixel centerline of the stripe image. Then, it optimizes the stripe center using epipolar constraints to obtain the sub-pixel coordinates. Based on this, the method utilizes the calibrated intrinsic and extrinsic parameters of the binocular camera, along with matched stripe center point pairs, to reconstruct a three-dimensional point cloud, thereby obtaining the three-dimensional topographic data of the belt surface. Subsequently, normal vector estimation and curvature analysis are performed on the reconstructed three-dimensional point cloud to extract the geometric features of the defect region. The extracted geometric features include defect depth, defect area, and defect volume. Finally, the method performs multi-feature fusion between the extracted three-dimensional geometric features and the two-dimensional texture features extracted from the original image, and inputs the fused features into a multi-feature fusion classifier, which outputs the defect type identification result and its corresponding confidence score.

2. The online detection method and apparatus for three-dimensional defects on belt surfaces based on line laser and vision fusion as described in claim 1, characterized in that, The projection angle of the line laser projector and the optical axis of the binocular camera are set to 30 to 60 degrees, the projection width of the line laser stripe on the belt surface is 1 to 3 millimeters, and the baseline distance of the binocular camera is 200 to 500 millimeters.

3. The online detection method and apparatus for three-dimensional defects on belt surfaces based on line laser and vision fusion as described in claim 2, characterized in that, The specific process of Steger subpixel extraction is as follows: First, calculate the Hessian matrix of the stripe image, obtain the eigenvector direction corresponding to the largest eigenvalue as the normal direction of the stripe, and then perform second-order Taylor expansion subpixel interpolation in this normal direction to obtain the subpixel coordinates of the stripe center.

4. The online detection method and apparatus for three-dimensional defects on belt surfaces based on line laser and vision fusion as described in claim 3, characterized in that, After the 3D point cloud reconstruction, the bundle adjustment method is used to optimize the 3D point cloud globally to minimize the reprojection error and make the 3D reconstruction accuracy within ±1 mm.

5. The online detection method and apparatus for three-dimensional defects on belt surfaces based on line laser and vision fusion according to claim 4, characterized in that, The normal vector estimation adopts the principal component analysis neighborhood analysis method. For each point, it searches for its nearest K neighboring points, calculates the eigenvector corresponding to the smallest eigenvalue of the covariance matrix formed by these neighboring points, and uses the eigenvector as the normal vector of the point. The curvature analysis includes calculating Gaussian curvature and mean curvature.

6. The online detection method and apparatus for three-dimensional defects on belt surfaces based on line laser and vision fusion as described in claim 5, characterized in that, The multi-feature fusion classifier specifically includes a fully connected branch for geometric features, a fully connected branch for texture features, an attention mechanism fusion layer, and a Softmax classification layer; wherein, the attention mechanism fusion layer is used to perform weighted fusion of geometric features and texture features, and then the fused features are sent to the Softmax classification layer for defect type discrimination.

7. An online detection device for three-dimensional defects on belt surfaces based on line laser and vision fusion, characterized in that, The device includes: Line laser projector module for projecting line laser stripes onto the surface of the belt; A binocular camera module is used to simultaneously acquire left and right view images of the belt surface; An image processing module is used to perform the detection method as described in any one of claims 1 to 6; and a result output module is used to output the defect type, defect location, and defect quantization parameters.

8. An electronic device, characterized in that, include: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to perform the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the method as described in any one of claims 1 to 6.

Images