Defect detection method, device and equipment based on multi-field light source and binocular camera and storage medium

Abstract

The invention discloses a defect detection method, device and equipment based on a multi-field light source and a binocular camera and a storage medium, and relates to the technical field of image recognition, and the defect detection method based on the multi-field light source and the binocular camera comprises the following steps: collecting initial image data of multiple groups of binocular cameras for the same defect area under light sources of a bright field, a dark field and a side field, three-dimensional images under three light sources are generated through stereo correction and matching, and then defect detection is carried out to obtain a bright field defect detection result, a dark field defect detection result and a side field defect detection result; and when at least two of the detection results are inconsistent, fusing the three three-dimensional images, and performing defect detection on the fused image to obtain a target defect detection result. Through the closed-loop process of multi-light three-dimensional parallelism and conflict fusion re-detection, the detection accuracy and the recognition rate of the surface defects of the plates and the roller products are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, in particular to a defect detection method and device based on multi-field light source and binocular camera, equipment and storage medium. BACKGROUND

[0002] In the industrial fields of high-speed continuous production of steel, aluminum processing and papermaking, once the surface of plate and roll products appears defects such as peeling, scratching, concave printing, roll printing or zinc slag, it will not only directly affect the mechanical properties and appearance quality of the final product, but also may cause belt breaking, scrapping or even safety accidents in subsequent deep processing links; with the continuous improvement of production line speed, manual sampling inspection cannot meet the demand of full-width, real-time and high-precision quality control, and the industry urgently hopes to introduce automatic detection means that can identify various surface defects online, at high speed and accurately.

[0003] Three types of machine vision detection schemes are commonly used on the market at present: the first type is a monocular industrial camera cooperating with a single angle coaxial bright field light source, which continuously photographs the moving plate; the second type increases multi-angle illumination on the basis of the monocular camera, and obtains multiple images of the same scene under different lighting conditions by sequentially lighting the bright field, dark field and side field light sources; the third type uses a pair of strictly calibrated binocular cameras to form a stereo vision system under the condition of a single bright field light source, which synchronously images the surface, and then detects defects by calculating the depth information.

[0004] The monocular single field scheme has limited imaging contrast due to single view angle and single illumination, and it is difficult to distinguish subtle defects on low contrast surfaces such as galvanized and aluminum plated surfaces, resulting in a large number of missed detections; the monocular multi-field scheme introduces multi-angle illumination, but monocular imaging lacks depth information, and the defect position, shape and gray scale distribution under different illuminations differ significantly, making it difficult to accurately align multiple images in the later stage, and the same defect is often misreported as multiple targets, resulting in chaotic recognition results; the binocular single field scheme can provide depth data, but it is limited by single illumination angle and cannot simultaneously consider low contrast defects and three-dimensional topography defects, and depth calculation is prone to failure in low texture areas, so the overall recognition rate is still limited. Therefore, how to improve the detection accuracy and recognition rate of surface defects of plate and roll products has become a problem to be solved.

[0005] The above content is only used to assist in understanding the technical solutions of the present application, and does not represent the acknowledgement of the above content as prior art. SUMMARY

[0006] The present application aims to provide a defect detection method and device based on multi-field light source and binocular camera, equipment and storage medium, which aims to solve the technical problem of how to improve the detection accuracy and recognition rate of surface defects of plate and roll products.

[0007] To achieve the above object, the application provides a defect detection method based on multi-field light sources and binocular cameras, which comprises the following steps: Obtaining initial defect image data of a same defect region collected by a plurality of binocular cameras under bright field light source, dark field light source and side field light source environments respectively; Performing stereo correction and stereo matching on the initial defect image data to obtain bright field three-dimensional image, dark field three-dimensional image and side field three-dimensional image; Performing defect detection on the bright field three-dimensional image, the dark field three-dimensional image and the side field three-dimensional image to obtain bright field defect detection result, dark field defect detection result and side field defect detection result; When at least two of the bright field defect detection result, the dark field defect detection result and the side field defect detection result are inconsistent, fusing the bright field three-dimensional image, the dark field three-dimensional image and the side field three-dimensional image, and performing defect detection on the fused image to obtain target defect detection result.

[0008] In an embodiment, when at least two of the bright field defect detection result, the dark field defect detection result and the side field defect detection result are inconsistent, the step of fusing the bright field three-dimensional image, the dark field three-dimensional image and the side field three-dimensional image, and performing defect detection on the fused image to obtain target defect detection result comprises the following steps: When at least two of the bright field defect detection result, the dark field defect detection result and the side field defect detection result are inconsistent, performing spatial registration on the bright field three-dimensional image, the dark field three-dimensional image and the side field three-dimensional image to obtain a set of registered images; Performing weighted fusion on the set of registered images to obtain a single fused defect image; Performing defect detection on the single fused defect image to obtain target defect detection result.

[0009] In an embodiment, the step of performing weighted fusion on the set of registered images to obtain a single fused defect image comprises the following steps: Calculating bright field weight based on local contrast of the bright field three-dimensional image; Calculating dark field weight based on saturation and brightness of the dark field three-dimensional image; Calculating side field weight based on edge enhancement features of the side field three-dimensional image; Performing weighted superposition calculation on images in the set of registered images according to the bright field weight, the dark field weight and the side field weight to obtain a single fused defect image.

[0010] In one embodiment, the step of spatially registering the bright-field 3D image, the dark-field 3D image, and the side-field 3D image to obtain a registered image set includes: Feature points are extracted from the bright-field 3D image, the dark-field 3D image, and the side-field 3D image; The feature points of the bright-field 3D image are matched with the feature points of the dark-field 3D image to obtain the first matching result; The feature points of the bright-field 3D image are matched with the feature points of the side-field 3D image to obtain a second matching result; Based on the first matching result and the second matching result, spatial transformation is performed on the dark field 3D image and the side field 3D image to obtain a registered image set.

[0011] In one embodiment, the step of matching feature points of the bright-field 3D image with feature points of the dark-field 3D image to obtain a first matching result includes: Based on the feature points of the bright-field 3D image and the feature points of the dark-field 3D image, a bright-field descriptor and a dark-field descriptor are generated using a scale-invariant feature transformation algorithm. The similarity distance between the bright field descriptor and the dark field descriptor is calculated using a nearest neighbor search algorithm. A preliminary set of matching point pairs is generated based on the feature points whose similarity distance is less than a preset matching threshold; The spatial consistency of matching point pairs in the preliminary matching point pair set is verified by a random sampling consensus algorithm, and the verified matching point pairs are taken as the first matching results.

[0012] In one embodiment, the step of performing defect detection on the bright-field 3D image, the dark-field 3D image, and the side-field 3D image to obtain bright-field defect detection results, dark-field defect detection results, and side-field defect detection results includes: Obtain a historical defect sample set, where each sample includes a 3D image sample and a corresponding defect category label; The convolutional neural network model is trained based on the historical defect sample set to obtain a defect detection model; Feature extraction is performed on the bright-field 3D image, the dark-field 3D image, and the side-field 3D image to obtain bright-field feature vector, dark-field feature vector, and side-field feature vector; The defect detection model performs parallel classification processing on the bright field feature vector, the dark field feature vector, and the side field feature vector to obtain bright field defect detection results, dark field defect detection results, and side field defect detection results.

[0013] In one embodiment, the initial defect image data includes an initial bright-field image, an initial dark-field image, and an initial side-field image; The step of performing stereo correction and stereo matching on the initial defect image data to obtain a bright-field three-dimensional image, a dark-field three-dimensional image, and a side-field three-dimensional image includes: Camera calibration is performed on the initial bright-field image, the initial dark-field image, and the initial side-field image to obtain the bright-field camera intrinsic parameter matrix and bright-field distortion coefficients. Based on the bright-field camera intrinsic parameter matrix and the bright-field distortion coefficient, stereo correction is performed on the initial bright-field image, the initial dark-field image and the initial side-field image, and region matching calculation is performed on the correction results to obtain the bright-field disparity map, the dark-field disparity map and the side-field disparity map. Calculate the depth information of the bright-field parallax map, the dark-field parallax map, and the side-field parallax map; The depth information is converted into three-dimensional point cloud coordinates, and bright-field three-dimensional images, dark-field three-dimensional images, and side-field three-dimensional images are constructed based on the three-dimensional point cloud coordinates.

[0014] Furthermore, to achieve the above objectives, this application also proposes a defect detection device based on a multi-field light source and a binocular camera, the device comprising: The data acquisition module is used to acquire initial defect image data of the same defect area collected by multiple sets of binocular cameras under bright field light source, dark field light source and side field light source environments; The preprocessing module is used to perform stereo correction and stereo matching on the initial defect image data to obtain a bright-field three-dimensional image, a dark-field three-dimensional image, and a side-field three-dimensional image. The first defect detection module is used to perform defect detection on the bright field three-dimensional image, the dark field three-dimensional image and the side field three-dimensional image to obtain bright field defect detection results, dark field defect detection results and side field defect detection results. The second defect detection module is used to fuse the bright-field three-dimensional image, the dark-field three-dimensional image, and the side-field three-dimensional image when there is inconsistency among at least two of the bright-field defect detection results, the dark-field defect detection results, and the side-field defect detection results, and to perform defect detection on the fused image to obtain the target defect detection result.

[0015] Furthermore, to achieve the above objectives, this application also proposes a defect detection device based on a multi-field light source and a binocular camera. The device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. The computer program is configured to implement the steps of the defect detection method based on a multi-field light source and a binocular camera as described above.

[0016] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the steps of the defect detection method based on multi-field light source and binocular camera as described above.

[0017] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the defect detection method based on multi-field light source and binocular camera as described above.

[0018] One or more technical solutions proposed in this application have at least the following technical effects: First, the defect detection system drives multiple binocular cameras to sequentially capture images of the same defect area under bright field, dark field, and side field light sources, acquiring three sets of initial defect image data. This allows low-contrast defects to be developed under different lighting conditions, increasing the detection probability. Then, stereo correction and stereo matching are performed on each pair of left and right images to reconstruct three-dimensional images of the bright field, dark field, and side field, binding two-dimensional grayscale with depth to eliminate deformation errors from monocular imaging. Next, the same defect detection model is used to infer the three three-dimensional images separately, obtaining three defect categories and confidence levels, achieving parallel screening. When at least two of the three results are inconsistent, the system aligns the three three-dimensional images according to spatial coordinates and fuses them into a single composite image. The detection model is then re-run on the fused image, outputting a unique target defect detection result, integrating complementary features from various lighting conditions and suppressing misjudgments. Through this closed-loop process of "multi-light three-dimensional parallel processing + conflict fusion re-inspection," the detection accuracy and recognition rate of surface defects in sheet and roll products are improved. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating an embodiment of the defect detection method based on multi-field light sources and binocular cameras provided in this application. Figure 2 This is a schematic diagram of the defect detection system layout provided in Embodiment 1 of the defect detection method based on multi-field light sources and binocular cameras in this application; Figure 3This is a schematic diagram of the single-field image defect detection process provided in Embodiment 1 of the defect detection method based on multi-field light sources and binocular cameras in this application; Figure 4 This is a schematic diagram of the multi-field image defect detection process provided in Embodiment 1 of the defect detection method based on multi-field light sources and binocular cameras in this application; Figure 5 This is a schematic diagram of the short defect detection effect provided in Embodiment 1 of the defect detection method based on multi-field light source and binocular camera of this application; Figure 6 This is a schematic diagram of the long defect detection effect provided in Embodiment 1 of the defect detection method based on multi-field light source and binocular camera of this application; Figure 7 This is a flowchart illustrating Embodiment 2 of the defect detection method based on multi-field light sources and binocular cameras in this application; Figure 8 This is a schematic diagram of the module structure of a defect detection device based on a multi-field light source and a binocular camera, according to an embodiment of this application. Figure 9 This is a schematic diagram of the device structure of the hardware operating environment involved in the defect detection method based on multi-field light source and binocular camera in the embodiments of this application.

[0022] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0023] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0024] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0025] It should be noted that the executing entity of this application embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device or defect detection system capable of realizing the above functions. The following description uses a defect detection system as an example to illustrate this embodiment and the subsequent embodiments.

[0026] Based on this, embodiments of this application provide a defect detection method based on multi-field light sources and a binocular camera, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the defect detection method based on multi-field light sources and binocular cameras in this application.

[0027] In this embodiment, the defect detection method based on multi-field light sources and binocular cameras includes steps S10~S40: Step S10: Acquire initial defect image data of the same defect area from multiple sets of binocular cameras under bright field light source, dark field light source and side field light source environments.

[0028] It should be noted that the defect detection system includes the outer frame of the detection device, a line scan camera and its components, two sets of line scan structured light sources and their adjustment device components, and a control unit component. The outer frame of the detection device refers to the rigid metal skeleton used to support and fix all detection components. It is usually welded from aluminum profiles or steel structures, possessing sufficient strength and stability, providing an installation reference and protective housing for the camera, light source, adjustment mechanism, and electrical control equipment. The line scan camera is an industrial camera that uses a single-row pixel sensor. It acquires only a narrow band image of the object at a time, stitching it together line by line through the relative movement of the object and the camera to form a complete two-dimensional image. It is suitable for high-speed, large-format, continuous roll material online inspection. The line scan structured light source refers to shaping laser or other high-brightness light into an extremely narrow linear beam through optical elements such as cylindrical mirrors and Powell prisms, and projecting it onto the surface being measured at a certain angle. It is used in conjunction with the line scan camera to produce high-contrast illumination of contours or defect features. The adjustment device assembly refers to a mechanical unit composed of slide rails, precision screws, rotary hinges, and locking mechanisms. It allows for precise adjustment and locking of the spatial position, pitch angle, and working distance of the line scan camera and line scan structured light source to adapt to different product specifications and testing requirements. The control unit assembly refers to an electrical system integrating an industrial computer, image acquisition card, motion control module, light source driver power supply, and communication interface. It is used to uniformly control camera exposure, light source brightness, and trigger timing, and to perform real-time image processing and data exchange.

[0029] Please refer to Figure 2 , Figure 2 This is a schematic diagram of the defect detection system layout provided in Embodiment 1 of the defect detection method based on multi-field light sources and binocular cameras of this application. The diagram shows a strip (metal plate) horizontally passing through the detection area, with bright-field and side-field lights installed above it to provide illumination from different angles to highlight potential defects. Three sets of cameras are symmetrically arranged on both sides of the strip: a dark-field or transition-field camera, a side-field camera, and a bright-field camera. Each set of cameras consists of two CCD (Charge-Coupled Device) cameras for stereo imaging. The optical axes of these cameras converge on the strip surface, ensuring accurate stereo observation of the same defect area. Rollers are located below the strip to support and guide it through the detection area. The entire layout aims to improve the accuracy and reliability of defect detection through multi-angle illumination and stereo vision technology.

[0030] A binocular camera refers to a stereo vision unit consisting of two industrial cameras with identical parameters. The two cameras are installed at a fixed baseline distance and angle, and the three-dimensional morphological information of the measured surface is acquired through parallax calculation. A bright-field light source is a coaxial or paraxial illumination source that directly illuminates the measured surface at a small angle to the camera's optical axis, used to highlight the overall texture and main details of the material. A dark-field light source is a diffused or low-angle illumination source projected from the side at a large tilt angle, making flat areas of the surface appear dark, while small protrusions, depressions, or particles appear bright due to scattered light entering the lens. A side-field light source is a directional illumination source located to the side of the measured surface at a tilt angle of 20° to 60°, used to enhance the contrast of edges, scratches, and three-dimensional morphology. A defect area refers to a localized two-dimensional area and its corresponding three-dimensional spatial location on the surface of sheet or roll products that has been identified or manually marked as having defects such as peeling, scratches, indentations, or roll marks. Initial defect image data refers to the original defect area images and their accompanying depth or parallax information, which were acquired by each group of binocular cameras under three light source conditions: bright field, dark field, and side field, and have not yet been fused or post-processed.

[0031] Understandably, the defect detection system illuminates three independent light sources—bright field, dark field, and side field—in sequence while keeping the physical position of the defect area unchanged. Each time the light source is switched, the system synchronously triggers all binocular cameras and completes one exposure, writing the left and right images of each camera and the corresponding timestamps into the cache. After all three light sources have been polled, the system summarizes the three pairs of original images acquired by each camera under bright field, dark field, and side field conditions, as well as the depth map calculated from the disparity, to obtain the initial defect image dataset for the defect area.

[0032] Step S20: Perform stereo correction and stereo matching on the initial defect image data to obtain a bright-field three-dimensional image, a dark-field three-dimensional image, and a side-field three-dimensional image.

[0033] It should be noted that a bright-field 3D image refers to image data acquired by a binocular camera under bright-field illumination, and generated after stereo calibration and stereo matching, containing the complete 3D coordinates of the measured surface and the corresponding bright-field texture grayscale information. A dark-field 3D image refers to image data acquired by a binocular camera under dark-field illumination, and generated after stereo calibration and stereo matching, containing the complete 3D coordinates of the measured surface and the corresponding dark-field texture grayscale information. A side-field 3D image refers to image data acquired by a binocular camera under side-field illumination, and generated after stereo calibration and stereo matching, containing the complete 3D coordinates of the measured surface and the corresponding side-field texture grayscale information.

[0034] As an example, the initial defect image data includes an initial bright-field image, an initial dark-field image, and an initial side-field image. The step of performing stereo correction and stereo matching on the initial defect image data to obtain a bright-field 3D image, a dark-field 3D image, and a side-field 3D image includes: performing camera calibration on the initial bright-field image, the initial dark-field image, and the initial side-field image to obtain a bright-field camera intrinsic parameter matrix and a bright-field distortion coefficient; performing stereo correction on the initial bright-field image, the initial dark-field image, and the initial side-field image based on the bright-field camera intrinsic parameter matrix and the bright-field distortion coefficient, and performing region matching calculation on the correction results to obtain a bright-field disparity map, a dark-field disparity map, and a side-field disparity map; calculating the depth information of the bright-field disparity map, the dark-field disparity map, and the side-field disparity map; converting the depth information into 3D point cloud coordinates, and constructing the bright-field 3D image, the dark-field 3D image, and the side-field 3D image based on the 3D point cloud coordinates.

[0035] The initial bright-field image refers to the two original two-dimensional images (left and right) simultaneously acquired by the binocular camera under bright-field illumination for a defective area. The initial dark-field image refers to the two original two-dimensional images (left and right) simultaneously acquired under dark-field illumination for the same area. The initial side-field image refers to the two original two-dimensional images (left and right) simultaneously acquired under side-field illumination for the same area. The bright-field camera intrinsic parameter matrix is ​​a 3×3 matrix describing the internal geometric parameters of the binocular camera in a bright-field environment, such as focal length, principal point, and pixel size. The bright-field distortion coefficients are a set of calibration parameters characterizing the radial and tangential distortion of the camera lens. The correction result refers to the left and right image pairs obtained after distortion correction and row alignment of the initial bright-field, initial dark-field, and initial side-field images using the aforementioned intrinsic parameter matrix and distortion coefficients. The bright-field disparity map, dark-field disparity map, and side-field disparity map refer to the horizontal disparity value matrices calculated pixel-by-pixel on the corresponding corrected left and right image pairs, used to reflect the pixel offset of the same spatial point in the left and right images. Depth information refers to the actual distance from each pixel to the camera coordinate system calculated using the triangulation principle based on each disparity map. 3D point cloud coordinates refer to a set of discrete points represented in a 3D Cartesian coordinate system, obtained by combining depth information with the camera model; each point corresponds to a physical location on the measured surface.

[0036] First, the defect detection system acquires multiple bright-field, dark-field, and side-field left and right images with different poses on the same calibration board. Using Zhang Zhengyou's calibration method, the intrinsic parameter matrix fx fy cx cy (focal length and principal point) and the five-dimensional distortion coefficients k1 k2 p1 p2 k3 (radial and tangential) of the binocular camera are calculated in one go to ensure that the same set of geometric parameters are used under the three sets of light sources and to avoid depth errors caused by parameter differences in the future. Secondly, the system reads the initial bright field, initial dark field, and initial side field left and right images, performs distortion correction based on the obtained intrinsic parameter matrix and distortion coefficients, and then uses the stereo correction algorithm (Bouguet method) to calculate the rotation matrix R and translation vector T to generate a stereo correction mapping table. The left and right images are remapped to ensure that corresponding rows of the left and right images are strictly aligned, ensuring that subsequent matching is only performed in the horizontal direction, reducing the amount of computation and improving the matching accuracy. After correction, the semi-global matching (SGM) algorithm is used on each corrected image to calculate the matching cost pixel by pixel with a 5×5 Census window and 64 levels of disparity range. After multi-path cost aggregation and a winner-take-all strategy, the bright field, dark field, and side field disparity maps are output respectively. Then, the system substitutes the disparity value d of each pixel into the formula depth = (f × B) / d (where f is the focal length and B is the baseline length) to obtain the actual depth value. It then combines the depth map with the corrected RGB information of the left image and generates 3D point cloud coordinates using the camera coordinate system back-projection formula. Finally, it reconstructs 3D images of the bright field, dark field, and side field according to the grid index for subsequent fusion and defect analysis. The camera coordinate system back-projection formula is as follows: X=(u-cx)×depth / fx, Y=(v-cy) × depth / fy, Z = depth Where X, Y, and Z refer to the three-dimensional coordinates of the target point in the camera coordinate system; u refers to the horizontal pixel column number of the point in the image coordinate system (counting from the top left corner of the image); v refers to the vertical pixel row number of the point in the image coordinate system (counting from the top left corner of the image); cx refers to the horizontal pixel coordinates of the camera principal point in the image coordinate system, representing the intersection of the optical axis and the image plane; cy refers to the vertical pixel coordinates of the camera principal point in the image coordinate system; fx refers to the equivalent focal length of the camera in the horizontal direction (in pixels, fx = f / dx, where f is the focal length and dx is the physical width of a single pixel); fy refers to the equivalent focal length of the camera in the vertical direction (in pixels, fy = f / dy, where dy is the physical height of a single pixel); depth refers to the depth value of the pixel, that is, the straight-line distance from the camera optical center to the corresponding object point.

[0037] Step S30: Perform defect detection on the bright-field 3D image, the dark-field 3D image, and the side-field 3D image to obtain bright-field defect detection results, dark-field defect detection results, and side-field defect detection results.

[0038] It should be noted that the bright-field defect detection result refers to the list of defects output after algorithmic identification in a bright-field 3D image. Each record includes the defect category, confidence score, 3D contour coordinates, and its geometric features under the bright-field texture. The dark-field defect detection result refers to the list of defects output after algorithmic identification in a dark-field 3D image. Each record includes the defect category, confidence score, 3D contour coordinates, and its geometric features under the dark-field texture. The side-field defect detection result refers to the list of defects output after algorithmic identification in a side-field 3D image. Each record includes the defect category, confidence score, 3D contour coordinates, and its geometric features under the side-field texture.

[0039] Please refer to Figure 3 , Figure 3 This is a schematic diagram of the single-field image defect detection process provided in Embodiment 1 of the defect detection method based on multi-field light sources and binocular cameras of this application. First, an image of the surface of the object to be detected is acquired using a binocular CCD camera. Next, stereo correction is performed on the acquired image to eliminate distortion and ensure left and right image alignment. Then, a deep learning algorithm is used to detect defects in the corrected image, identifying potential defects in the image. Afterward, target stereo matching and depth calculation are performed to determine the three-dimensional position and depth information of the defects. Next, the detected defects are precisely located. Finally, the defects are detected and classified, and information such as the type and severity of the defects is output. The entire process automates the processing from image acquisition to defect recognition, improving detection efficiency and accuracy.

[0040] As an example, the step of performing defect detection on the bright-field 3D image, the dark-field 3D image, and the side-field 3D image to obtain bright-field defect detection results, dark-field defect detection results, and side-field defect detection results includes: acquiring a historical defect sample set, wherein each sample includes a 3D image sample and a corresponding defect category label; training a convolutional neural network model based on the historical defect sample set to obtain a defect detection model; extracting features from the bright-field 3D image, the dark-field 3D image, and the side-field 3D image to obtain bright-field feature vectors, dark-field feature vectors, and side-field feature vectors; and performing parallel classification processing on the bright-field feature vectors, the dark-field feature vectors, and the side-field feature vectors using the defect detection model to obtain bright-field defect detection results, dark-field defect detection results, and side-field defect detection results.

[0041] A historical defect sample set refers to a collection of 3D image data collected from production lines or experiments in the past, with manually labeled defect categories, used to train and validate detection models. A 3D image sample refers to a point cloud or voxel matrix of a defect region, reconstructed from a binocular camera under bright-field, dark-field, or side-field conditions, containing the 3D coordinates of each pixel and corresponding texture information. A defect category label refers to a textual or numerical identifier corresponding to each 3D image sample, describing the specific type of defect, such as "peeling," "scratching," or "dentation." A defect detection model refers to the weights and overall network structure obtained after training a convolutional neural network using the historical defect sample set, capable of judging the defect category of new input feature vectors. A bright-field feature vector refers to a high-dimensional numerical sequence extracted and compressed from a bright-field 3D image, used to characterize the geometric and textural features of defects under bright-field illumination. A dark-field feature vector refers to a high-dimensional numerical sequence extracted and compressed from a dark-field 3D image, used to characterize the geometric and textural features of defects under dark-field illumination. Side field feature vectors refer to high-dimensional numerical sequences extracted and compressed from side field 3D images, used to characterize the geometric and textural features of defects under side field illumination.

[0042] First, the defect detection system reads a historical defect sample set (e.g., 12,000 labeled 3D point cloud samples), divides it into a training set and a validation set at an 8:2 ratio, resamples the samples to a fixed voxel grid (e.g., 128×128×64), and performs online random rotation (±5°) and elastic deformation to expand the data. Then, the voxel grid is input into a 3D convolutional neural network (3D-CNN, e.g., 5 layers of 3×3×3 convolutions + BN (Batch Normalization) + ReLU (Rectified Linear Unit)), employing cross-entropy loss and adding Focal loss. The Loss (Focus Loss) method suppresses the weights of easily separable samples. The system is trained using the Adam optimizer (e.g., initial learning rate 0.001, cosine annealing) until the validation set accuracy is ≥98%, and the weights are saved as the defect detection model. Next, the system resamples the online input bright-field, dark-field, and side-field 3D images using voxelization consistent with the training phase. Feature maps are extracted via forward propagation using a 3D-CNN with shared weights, and then global average pooling is used to compress the feature maps into fixed-length one-dimensional vectors (e.g., 256-dimensional), resulting in bright-field, dark-field, and side-field feature vectors, ensuring comparability of the three illumination features in the same feature space. Finally, the system feeds the three feature vectors in parallel into the classification branch of the trained model, outputting the probability of each type of defect through a matrix multiplication and softmax operation. The category and confidence level corresponding to the highest probability are taken to directly generate bright-field, dark-field, and side-field defect detection results for subsequent fusion and judgment, thus ensuring a balance between detection speed and accuracy.

[0043] Step S40: When at least two of the bright-field defect detection results, dark-field defect detection results, and side-field defect detection results are inconsistent, the bright-field three-dimensional image, the dark-field three-dimensional image, and the side-field three-dimensional image are fused, and defect detection is performed on the fused image to obtain the target defect detection result.

[0044] It should be noted that the target defect detection result refers to the final, unique and comprehensive defect category and confidence output obtained by fusing the three sets of three-dimensional images and running the detection model again when there are discrepancies in the three illumination results of bright field, dark field and side field. This result is used as the final judgment basis for the defect area.

[0045] Understandably, the defect detection system first compares the category labels of the three outputs: bright field, dark field, and side field. If at least two labels are different, the corresponding 3D point clouds are immediately aligned according to spatial coordinates, and the texture with the highest confidence is extracted point by point and merged into a single fused image. Then, the system sends the fused image into the same defect detection model to perform forward inference again, and obtains a unique category and confidence, which is output as the final target defect detection result and written into the report.

[0046] Please refer to Figure 4 , Figure 4 This diagram illustrates the multi-field image defect detection process provided in Embodiment 1 of the defect detection method based on multi-field light sources and binocular cameras of this application. The diagram shows the multi-field image defect detection process, including processing steps in three parts: bright field, dark field, and side field. First, in the binocular CCD image acquisition stage, the binocular camera in each scene captures an image of the steel strip surface. Next, in the image processing stage within the FPGA (Field-Programmable Gate Array), stereo correction, information extraction, and stereo matching algorithms are performed on the image to obtain disparity and depth maps, and virtual viewpoints are synthesized using the disparity maps. Subsequently, the deep learning defect detection stage performs feature extraction and defect identification on the image. In the target stereo matching and depth calculation stage, the precise location and depth information of the defect are determined. Finally, in the bright field FPGA, dark field and side field image data are fused to determine the defect location; in the dark field FPGA, the defect location is determined and the image grayscale deviation value is recorded; and in the side field FPGA, the defect location and grayscale deviation value are determined. All information is aggregated into the FPGA and processed by a 3x3 convolution kernel and three IP processing units (IP processing refers to pre-designed, reusable circuit modules or functional units within the FPGA) for defect data buffering and reprocessing, including defect re-detection and reclassification. The final output is the defect information storage and display results. This process achieves efficient fusion of multi-field images and defect detection, improving the accuracy and reliability of detection.

[0047] Please refer to Figure 5 , Figure 5 This diagram illustrates the detection effect of short defects according to Embodiment 1 of the defect detection method based on multi-field light sources and a binocular camera in this application. The diagram shows the detection effect of short defects under different lighting conditions. The left image was acquired under bright-field conditions, showing the imaging of the short defect under single illumination, where defect features may not be clearly visible. The right image is the result of fusing images acquired under bright-field, side-field, and dark-field conditions. Through multi-field image fusion technology, the contour and details of the defect are enhanced, making the defect features clearer and more prominent, thereby improving the accuracy and reliability of defect detection. The defect in the diagram is more accurately located and identified in the fused image, verifying the effectiveness of multi-field image fusion technology in improving the performance of short defect detection.

[0048] Please refer to Figure 6 , Figure 6 This is a schematic diagram illustrating the detection effect of a long defect provided in Embodiment 1 of the defect detection method based on multi-field light sources and a binocular camera of this application. The diagram shows a comparison of the detection effect of a long defect under different lighting conditions. The image on the left was acquired under bright field conditions, showing the imaging of the long defect under a single light source, where some parts of the defect may not be obvious. The image on the right is the result of fusing images acquired under bright field, side field, and dark field conditions. Through multi-field image fusion technology, the outline and details of the defect are clearer, enhancing the visibility and accuracy of defect recognition, thereby verifying the effectiveness of multi-field image fusion in improving the detection effect of long defects.

[0049] This embodiment provides a defect detection method based on multi-field light sources and binocular cameras. First, the defect detection system drives multiple sets of binocular cameras to sequentially capture images of the same defect area under bright-field, dark-field, and side-field light sources, acquiring three sets of initial defect image data. This allows low-contrast defects to be developed under different lighting conditions, increasing the detection probability. Subsequently, stereo correction and stereo matching are performed on each set of left and right image pairs to reconstruct three-dimensional images of bright-field, dark-field, and side-field, binding two-dimensional grayscale with depth to eliminate deformation errors from monocular imaging. Then, the same defect detection model is used to infer the three three-dimensional images separately, obtaining three defect categories and confidence levels, achieving parallel screening. When at least two of the three results are inconsistent, the system aligns the three three-dimensional images according to spatial coordinates and fuses them into a single comprehensive image. The detection model is then re-run on the fused image to output a unique target defect detection result, integrating complementary features of each illumination and suppressing misjudgments. Through the closed-loop process of "multi-light three-dimensional parallel + conflict fusion re-inspection," the detection accuracy and recognition rate of surface defects in sheet and roll products are improved.

[0050] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in Embodiment 1 above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 7 , Figure 7 This is a flowchart illustrating the second embodiment of the defect detection method based on multi-field light sources and a binocular camera according to this application. Step S40 of the defect detection method based on multi-field light sources and a binocular camera includes steps S41 to S43: Step S41: When at least two of the bright-field defect detection results, the dark-field defect detection results, and the side-field defect detection results are inconsistent, the bright-field three-dimensional image, the dark-field three-dimensional image, and the side-field three-dimensional image are spatially registered to obtain a registered image set.

[0051] It should be noted that the registered image set refers to a set of three-dimensional images with pixel-level correspondence formed by aligning the bright field, dark field, and side field three-dimensional images to the same coordinate system through spatial transformation, which is used for subsequent fusion and re-detection.

[0052] As an example, the step of spatially registering the bright-field 3D image, the dark-field 3D image, and the side-field 3D image to obtain a registered image set includes: extracting feature points from the bright-field 3D image, the dark-field 3D image, and the side-field 3D image; matching the feature points of the bright-field 3D image with the feature points of the dark-field 3D image to obtain a first matching result; matching the feature points of the bright-field 3D image with the feature points of the side-field 3D image to obtain a second matching result; and performing spatial transformation on the dark-field 3D image and the side-field 3D image based on the first matching result and the second matching result to obtain the registered image set.

[0053] Feature points are sets of spatial point coordinates extracted from 3D images that possess significant geometric features (such as edges, corners, curvature extrema, etc.), used as a reference for image registration. The first matching result is a list of corresponding point pairs calculated based on feature point similarity between the primary and secondary 3D images. The second matching result is a list of corresponding point pairs calculated based on feature point similarity between the primary and secondary 3D images.

[0054] First, the system performs ISS (Intrinsic Shape Signatures) detection (non-maximum suppression radius = 3 voxels, threshold = 0.5) on the bright field, dark field, and side field 3D images to extract key points. For each point, an FPFH (Fast Point Feature Histogram) descriptor (neighborhood radius = 5 voxels, dimension = 33) is calculated, using dense features to ensure robustness in subsequent matching. Second, using the bright field key points as a benchmark, the FPFH vectors are compared one by one with the dark field key points. Point pairs with the smallest distance and a ratio below 0.7 are selected to form initial correspondences. Then, a subset is repeatedly extracted using random sampling to calculate the rigid body model, retaining the transformation with the most interior points as the first matching result. The same process is repeated for the side field to obtain the second matching result, thereby eliminating erroneous correspondences and improving geometric consistency. Finally, the system uses SVD (Singular Value Decomposition) to solve for the rotation R and translation t of the two sets of interior points respectively. The resulting rigid body transformation is applied to the coordinates of each point in the dark field and side field 3D images to complete the coordinate mapping and output a registered image set that is completely aligned with the bright field, ensuring that subsequent fusion does not need to consider positional deviations.

[0055] As an example, the step of matching feature points of the bright-field 3D image with feature points of the dark-field 3D image to obtain a first matching result includes: generating a bright-field descriptor and a dark-field descriptor using a scale-invariant feature transformation algorithm based on the feature points of the bright-field 3D image and the feature points of the dark-field 3D image; calculating the similarity distance between the bright-field descriptor and the dark-field descriptor using a nearest neighbor search algorithm; generating a preliminary set of matching point pairs based on the feature points whose similarity distance is less than a preset matching threshold; verifying the spatial consistency of the matching point pairs in the preliminary set of matching point pairs using a random sampling consensus algorithm; and taking the verified matching point pairs as the first matching result.

[0056] The Scale-Invariant Feature Transform (SIFT) algorithm is an algorithm that detects keypoints at different scales and generates 128-dimensional feature descriptors, ensuring that the features remain stable under changes in rotation, scale, and illumination. Bright-field descriptors are 128-dimensional vectors extracted from bright-field 3D images using SIFT, containing gradient direction information around the keypoints, and are used for subsequent matching. Dark-field descriptors are 128-dimensional vectors extracted from dark-field 3D images using the same SIFT process, and their similarity is compared with the bright-field descriptors. The nearest neighbor search algorithm calculates the Euclidean distance between two sets of descriptors, finding the dark-field descriptor with the smallest distance for each bright-field descriptor, achieving fast matching. The similarity distance refers to the Euclidean distance between the descriptor vectors; the smaller the value, the more similar the features. The preset matching threshold is the maximum distance limit set by the system; only descriptor pairs with distances less than this threshold are accepted, used to filter obvious errors. The preliminary matching point pair set is the list of bright-field and dark-field keypoint correspondences retained after threshold filtering. The Random Sample Consensus algorithm... Consensus (RANSAC) refers to calculating the spatial transformation model and counting the number of points within the model through multiple random samplings, eliminating mismatches, and retaining geometrically consistent point pairs; spatial consistency means that the matched point pairs should satisfy the same rigid body transformation relationship in three-dimensional space to ensure the accuracy of image registration.

[0057] First, the system performs SIFT on each feature point of the bright-field and dark-field 3D images: extreme points are detected within a 3x pyramid scale space, a principal direction is assigned to each point, and a 128-dimensional vector is generated, forming bright-field and dark-field descriptors to ensure comparability under rotation and scale changes. Second, a KD-Tree (k-dimensional tree) is used to index all dark-field descriptors. For each bright-field descriptor, the two nearest points are searched, and the ratio of the nearest to the second nearest distance is calculated. If the ratio is less than a preset matching threshold (e.g., 0.7), the descriptor is retained; otherwise, it is discarded, quickly filtering out high-confidence point pairs to form a preliminary set of matching point pairs. Finally, three pairs of points are randomly selected from this set to estimate the rigid body transformation matrix. The remaining points are projected onto this matrix, and the number of interior points with an error less than the tolerance (e.g., 2 mm) is counted. This sampling is repeated up to the maximum number of times (e.g., 1000 times), and the model with the most interior points is selected as the final solution. The verified point pairs are output as the first matching result to eliminate mismatches and ensure geometric consistency.

[0058] Step S42: Perform weighted fusion on the registered image set to obtain a single fused defect image.

[0059] It should be noted that a single fused defect image refers to a comprehensive image generated by superimposing the registered bright field, dark field, and side field three-dimensional images according to weighting coefficients. This image contains the dominant features of each illumination and has consistent coordinates, and is used to complete the final defect detection in one go.

[0060] As an example, the step of weighted fusion of the registered image set to obtain a single fused defective image includes: calculating the bright field weight based on the local contrast of the bright field 3D image; calculating the dark field weight based on the saturation and brightness of the dark field 3D image; calculating the side field weight based on the edge enhancement features of the side field 3D image; and performing weighted superposition calculation on the images in the registered image set according to the bright field weight, the dark field weight, and the side field weight to obtain a single fused defective image.

[0061] Local contrast refers to the normalized value calculated by taking the difference between the maximum and minimum gray values ​​in the neighborhood of each voxel in a bright-field 3D image, and is used to quantify the degree of detail prominence in that region. Bright-field weight is a fusion coefficient assigned to a bright-field 3D image based on the normalized local contrast; higher contrast results in a greater weight. Dark-field weight is a normalized coefficient obtained by non-linearly stretching the product of saturation and brightness in a dark-field 3D image, allowing low-brightness, high-saturation defects in the dark field to contribute more significantly. Edge enhancement features refer to the gradient magnitude calculated by the Sobel operator in a side-field 3D image, used to highlight shape edge information. Side-field weight is a fusion coefficient assigned to a side-field 3D image after normalizing the edge enhancement features; a larger gradient results in a greater weight.

[0062] First, the system opens a 3×3×3 window for each voxel of the bright-field 3D image, calculates the maximum and minimum gray-level difference within the window and normalizes it to 0-1 to obtain a local contrast map, which is directly used as the bright-field weight, giving higher contribution to areas with prominent details. Second, for the dark-field 3D image, the saturation and brightness products are calculated point by point, and then normalized after nonlinear stretching using Gamma correction (e.g., γ=0.8) to obtain the dark-field weight, which increases the weight of low-brightness, high-saturation defect areas. Then, the gradient magnitude of the side-field 3D image is calculated using the Sobel operator (e.g., 3×3×3 kernel), and normalized after Gaussian smoothing to form the side-field weight, which enhances edge information. Finally, the three sets of weights are multiplied by the gray-level values ​​of the corresponding registered images, stacked by voxels and divided by the weight sum to output a single fused defect image, taking into account the advantages of each illumination and avoiding edge artifacts.

[0063] Step S43: Perform defect detection on the single fused defect image to obtain the target defect detection result.

[0064] Understandably, firstly, the system resamples a single fused defect image into a fixed voxel grid (e.g., 128×128×64) and normalizes it to 0-1 to ensure that the input size is consistent with the training data; secondly, the voxel grid is fed into a pre-trained 3D-CNN, which extracts high-dimensional features in one forward propagation and outputs the confidence score of each type of defect; finally, the category with the highest confidence score and its score are taken as the target defect detection result, thus integrating various illumination information in one inference and avoiding multiple conflicting results.

[0065] In this embodiment, when the defect categories of the bright field, dark field, and side field 3D images diverge, spatial registration is first performed on the three sets of images to ensure that all voxel coordinates correspond one-to-one and eliminate positional deviations. Then, weights are calculated and superimposed according to local contrast, saturation and brightness product, and edge gradient to generate a single fused defect image that integrates defect features under various illuminations. Finally, the fused image is input into the trained model for inference and outputs a unique target defect detection result, avoiding multiple result conflicts and improving the detection accuracy and recognition rate of surface defects in sheet and roller products.

[0066] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the defect detection method based on multi-field light sources and binocular cameras in this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0067] This application also provides a defect detection device based on multi-field light sources and a binocular camera; please refer to [reference needed]. Figure 8 The defect detection device based on multi-field light sources and binocular cameras includes: The data acquisition module 10 is used to acquire initial defect image data of the same defect area collected by multiple sets of binocular cameras under bright field light source, dark field light source and side field light source environments respectively; Preprocessing module 20 is used to perform stereo correction and stereo matching on the initial defect image data to obtain a bright field three-dimensional image, a dark field three-dimensional image and a side field three-dimensional image; The first defect detection module 30 is used to perform defect detection on the bright field three-dimensional image, the dark field three-dimensional image and the side field three-dimensional image to obtain bright field defect detection results, dark field defect detection results and side field defect detection results. The second defect detection module 40 is used to fuse the bright-field three-dimensional image, the dark-field three-dimensional image, and the side-field three-dimensional image when there is inconsistency among at least two of the bright-field defect detection results, the dark-field defect detection results, and the side-field defect detection results, and to perform defect detection on the fused image to obtain the target defect detection result.

[0068] The defect detection device based on multi-field light sources and binocular cameras provided in this application, employing the defect detection method based on multi-field light sources and binocular cameras described in the above embodiments, can solve the technical problem of how to improve the detection accuracy and recognition rate of surface defects in sheet and roller products. Compared with the prior art, the beneficial effects of the defect detection device based on multi-field light sources and binocular cameras provided in this application are the same as those of the defect detection method based on multi-field light sources and binocular cameras provided in the above embodiments, and other technical features in the defect detection device based on multi-field light sources and binocular cameras are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0069] This application provides a defect detection device based on a multi-field light source and a binocular camera. The defect detection device based on a multi-field light source and a binocular camera includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the defect detection method based on a multi-field light source and a binocular camera in the above embodiment 1.

[0070] The following is for reference. Figure 9 This document illustrates a structural schematic diagram of a defect detection device based on a multi-field light source and a binocular camera, suitable for implementing embodiments of this application. The defect detection device based on a multi-field light source and a binocular camera in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 9 The defect detection device based on multi-field light source and binocular camera shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0071] like Figure 9As shown, the defect detection device based on multi-field light sources and a binocular camera may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in ROM (Read Only Memory) 1002 or a program loaded from storage device 1003 into RAM (Random Access Memory) 1004. RAM 1004 also stores various programs and data required for the operation of the defect detection device based on multi-field light sources and a binocular camera. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via bus 1005. Input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, LCDs (Liquid Crystal Displays), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the defect detection equipment based on multi-field light sources and binocular cameras to wirelessly or wiredly communicate with other devices to exchange data. Although the figure shows a defect detection equipment based on multi-field light sources and binocular cameras with various systems, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems can be implemented alternatively.

[0072] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0073] The defect detection device based on multi-field light sources and binocular cameras provided in this application, employing the defect detection method based on multi-field light sources and binocular cameras described in the above embodiments, can solve the technical problem of how to improve the detection accuracy and recognition rate of surface defects in sheet and roller products. Compared with the prior art, the beneficial effects of the defect detection device based on multi-field light sources and binocular cameras provided in this application are the same as those of the defect detection method based on multi-field light sources and binocular cameras provided in the above embodiments, and other technical features in this defect detection device based on multi-field light sources and binocular cameras are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0074] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0075] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0076] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the defect detection method based on multi-field light source and binocular camera in the above embodiments.

[0077] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable Read Only Memory or Flash Memory), optical fibers, CD-ROM (CD-Read Only Memory), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0078] The aforementioned computer-readable storage medium may be included in a defect detection device based on a multi-field light source and a binocular camera; or it may exist independently and not assembled into a defect detection device based on a multi-field light source and a binocular camera.

[0079] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by a defect detection device based on a multi-field light source and a binocular camera, the defect detection device based on a multi-field light source and a binocular camera performs the following actions: acquiring initial defect image data of the same defect area collected by multiple sets of binocular cameras under bright-field, dark-field, and side-field light source environments; performing stereo correction and stereo matching on the initial defect image data to obtain a bright-field three-dimensional image, a dark-field three-dimensional image, and a side-field three-dimensional image; performing defect detection on the bright-field three-dimensional image, the dark-field three-dimensional image, and the side-field three-dimensional image to obtain a bright-field defect detection result, a dark-field defect detection result, and a side-field defect detection result; and when at least two of the bright-field defect detection result, the dark-field defect detection result, and the side-field defect detection result are inconsistent, fusing the bright-field three-dimensional image, the dark-field three-dimensional image, and the side-field three-dimensional image, and performing defect detection on the fused image to obtain a target defect detection result.

[0080] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including LAN (Local Area Network) or WAN (Wide Area Network)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0081] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0082] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0083] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the aforementioned defect detection method based on multi-field light sources and a binocular camera. This solves the technical problem of how to improve the detection accuracy and recognition rate of surface defects in sheet and roll products. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the defect detection method based on multi-field light sources and a binocular camera provided in the above embodiments, and will not be repeated here.

[0084] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the defect detection method based on multi-field light sources and binocular cameras as described above.

[0085] The computer program product provided in this application can solve the technical problem of how to improve the detection accuracy and recognition rate of surface defects in sheet and roll products. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the defect detection method based on multi-field light source and binocular camera provided in the above embodiments, and will not be repeated here.

[0086] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for defect detection based on multi-field light source and binocular camera, characterized in that, The method comprises: acquiring a plurality of groups of binocular cameras to respectively collect initial defect image data of a same defect region under bright field light source, dark field light source and side field light source environments; performing stereoscopic correction and stereoscopic matching on the initial defect image data to obtain bright field three-dimensional images, dark field three-dimensional images and side field three-dimensional images; performing defect detection on the bright field three-dimensional images, the dark field three-dimensional images and the side field three-dimensional images to obtain bright field defect detection results, dark field defect detection results and side field defect detection results; when at least two of the bright field defect detection results, the dark field defect detection results and the side field defect detection results are inconsistent, fusing the bright field three-dimensional images, the dark field three-dimensional images and the side field three-dimensional images, and performing defect detection on the fused images to obtain target defect detection results.

2. The method of claim 1, wherein, The step of fusing the bright field three-dimensional images, the dark field three-dimensional images and the side field three-dimensional images when at least two of the bright field defect detection results, the dark field defect detection results and the side field defect detection results are inconsistent, and performing defect detection on the fused images to obtain target defect detection results comprises: when at least two of the bright field defect detection results, the dark field defect detection results and the side field defect detection results are inconsistent, performing spatial registration on the bright field three-dimensional images, the dark field three-dimensional images and the side field three-dimensional images to obtain a set of registered images; performing weighted fusion on the set of registered images to obtain a single fused defect image; performing defect detection on the single fused defect image to obtain target defect detection results.

3. The method of claim 2, wherein, The step of performing weighted fusion on the set of registered images to obtain a single fused defect image comprises: calculating bright field weights based on local contrast of the bright field three-dimensional images; calculating dark field weights based on saturation and brightness of the dark field three-dimensional images; calculating side field weights based on edge enhancement features of the side field three-dimensional images; performing weighted superposition calculation on images in the set of registered images according to the bright field weights, the dark field weights and the side field weights to obtain a single fused defect image.

4. The method of claim 2, wherein, The step of performing spatial registration on the bright field three-dimensional images, the dark field three-dimensional images and the side field three-dimensional images to obtain a set of registered images comprises: extracting feature points of the bright field three-dimensional images, the dark field three-dimensional images and the side field three-dimensional images; matching the feature points of the bright field three-dimensional images with the feature points of the dark field three-dimensional images to obtain a first matching result; matching the feature points of the bright field three-dimensional images with the feature points of the side field three-dimensional images to obtain a second matching result; performing spatial transformation on the dark field three-dimensional images and the side field three-dimensional images according to the first matching result and the second matching result to obtain a set of registered images.

5. The method of claim 4, wherein, The step of matching the feature points of the bright field three-dimensional images with the feature points of the dark field three-dimensional images to obtain a first matching result comprises: According to the feature points of the bright field three-dimensional image and the feature points of the dark field three-dimensional image, bright field descriptors and dark field descriptors are generated through a scale-invariant feature transform algorithm; Similarity distances of the bright field descriptors and the dark field descriptors are calculated through a nearest neighbor search algorithm; A preliminary matching point pair set is generated according to the feature points with the similarity distances less than a preset matching threshold; Spatial consistencies of matching point pairs in the preliminary matching point pair set are verified through a random sample consensus algorithm, and the matching point pairs passing the verification are taken as a first matching result.

6. The method of claim 1, wherein, The steps of performing defect detection on the bright field three-dimensional image, the dark field three-dimensional image and the side field three-dimensional image to obtain bright field defect detection results, dark field defect detection results and side field defect detection results include: A historical defect sample set is obtained, wherein each sample includes a three-dimensional image sample and a corresponding defect category label; A convolutional neural network model is trained according to the historical defect sample set to obtain a defect detection model; Feature extraction is performed on the bright field three-dimensional image, the dark field three-dimensional image and the side field three-dimensional image to obtain bright field feature vectors, dark field feature vectors and side field feature vectors; Parallel classification processing is performed on the bright field feature vectors, the dark field feature vectors and the side field feature vectors through the defect detection model to obtain bright field defect detection results, dark field defect detection results and side field defect detection results.

7. The method of any one of claims 1 to 6, wherein, The initial defect image data includes an initial bright field image, an initial dark field image and an initial side field image; The steps of performing stereo rectification and stereo matching on the initial defect image data to obtain a bright field three-dimensional image, a dark field three-dimensional image and a side field three-dimensional image include: Camera calibration is performed on the initial bright field image, the initial dark field image and the initial side field image to obtain a bright field camera intrinsic parameter matrix and a bright field distortion coefficient; Based on the bright field camera intrinsic parameter matrix and the bright field distortion coefficient, stereo rectification is performed on the initial bright field image, the initial dark field image and the initial side field image, and region matching calculation is performed on the rectification results to obtain a bright field disparity map, a dark field disparity map and a side field disparity map; Depth information of the bright field disparity map, the dark field disparity map and the side field disparity map is calculated; The depth information is converted into three-dimensional point cloud coordinates, and a bright field three-dimensional image, a dark field three-dimensional image and a side field three-dimensional image are constructed according to the three-dimensional point cloud coordinates.

8. A defect detection apparatus based on multi-field light source and binocular camera, characterized in that, The device includes: A data acquisition module is configured to acquire a plurality of groups of initial defect image data of a same defect region collected by a binocular camera under a bright field light source, a dark field light source and a side field light source environment; A preprocessing module is configured to perform stereo rectification and stereo matching on the initial defect image data to obtain a bright field three-dimensional image, a dark field three-dimensional image and a side field three-dimensional image; A first defect detection module is configured to perform defect detection on the bright field three-dimensional image, the dark field three-dimensional image and the side field three-dimensional image to obtain bright field defect detection results, dark field defect detection results and side field defect detection results. The second defect detection module is configured to fuse the bright-field three-dimensional image, the dark-field three-dimensional image and the side-field three-dimensional image when there is inconsistency in at least two of the bright-field defect detection result, the dark-field defect detection result and the side-field defect detection result, and perform defect detection on the fused image to obtain a target defect detection result.

9. A defect detection apparatus based on multi-field light source and binocular camera, characterized in that, The device comprises a memory, a processor and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the defect detection method based on the multi-field light source and the binocular camera according to any one of claims 1 to 7.

10. A storage medium, characterized by The storage medium is a computer readable storage medium, and the storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the defect detection method based on the multi-field light source and the binocular camera according to any one of claims 1 to 7.

Images