A method and system for online defect detection in complex scenarios

Abstract

This invention provides an online defect detection method and system for complex scenarios, relating to the field of defect detection technology. The method includes: activating a first optical inspection instrument to acquire image data of a first target to be inspected, obtaining a first target image; performing superpixel segmentation on the first target image to obtain a first segmentation result, the first segmentation result including N superpixel images; extracting features from the N superpixel images to obtain an image feature matrix; inputting the image feature matrix into a defect detection model to output a target defect image; and optimizing the target defect image using a preset optimization scheme to obtain an online detection result. This invention solves the technical problem in existing technologies where poor image processing results lead to poor defect detection performance, achieving the technical effect of reducing the impact of the environment on defect detection and improving the defect detection effect.

Description

Technical Field

[0001] This invention relates to the field of defect detection technology, and specifically to an online defect detection method and system for complex scenarios. Background Technology

[0002] Defect detection is an important means of routine equipment maintenance. Timely detection of surface defects can effectively extend the service life of equipment. With the development of machine vision technology, defect detection is more effective in simple environments with simple backgrounds, and can accurately and effectively detect surface defects. However, for surface defect detection in complex scenes, the complexity of the detection environment leads to less clear images and the presence of interference factors such as shadows, which increases the difficulty of defect detection.

[0003] In summary, existing technologies suffer from poor defect detection performance due to unreasonable settings in the image acquisition process for complex scenarios and inadequate image processing. Summary of the Invention

[0004] This invention provides an online defect detection method and system for complex scenes, which solves the technical problem in the prior art that the defect detection effect is poor due to unreasonable settings of the image acquisition process in complex scenes and poor image processing effect.

[0005] According to a first aspect of the present invention, an online defect detection method for complex scenes is provided, comprising: activating a first optical inspection instrument to acquire image data of a first target to be inspected, thereby obtaining a first target image, wherein the first optical inspection instrument includes multiple beam emission sources and multiple cameras; performing superpixel segmentation on the first target image to obtain a first segmentation result, wherein the first segmentation result includes N superpixel images, where N is an integer greater than 0; performing feature extraction on the N superpixel images to obtain an image feature matrix; inputting the image feature matrix into a defect detection model to output a target defect image; and performing image optimization on the target defect image using a preset optimization scheme to obtain an online detection result.

[0006] According to a second aspect of the present invention, an online defect detection system for complex scenes is provided, comprising: an image acquisition module, wherein the image acquisition module is used to activate a first optical inspection instrument to acquire image data of a first target to be inspected, thereby obtaining a first target image, wherein the first optical inspection instrument includes multiple beam emission sources and multiple cameras; a superpixel segmentation module, wherein the superpixel segmentation module is used to perform superpixel segmentation on the first target image to obtain a first segmentation result, wherein the first segmentation result includes N superpixel images, where N is an integer greater than 0; a feature extraction module, wherein the feature extraction module is used to extract features from the N superpixel images to obtain an image feature matrix; a defect detection module, wherein the defect detection module is used to input the image feature matrix into a defect detection model and output a target defect image; and an image optimization module, wherein the image optimization module is used to optimize the target defect image through a preset optimization scheme to obtain an online detection result.

[0007] According to the present invention, an online defect detection method for complex scenarios involves activating a first optical inspection instrument to acquire image data of a first target to be inspected, obtaining a first target image. The first optical inspection instrument includes multiple beam emitters and multiple cameras. The first target image is subjected to superpixel segmentation to obtain a first segmentation result, which includes N superpixel images, where N is an integer greater than 0. Feature extraction is performed on the N superpixel images to obtain an image feature matrix. The image feature matrix is ​​input into a defect detection model to output a target defect image. The target defect image is then optimized using a preset optimization scheme to obtain an online detection result. This achieves the technical effect of reducing the influence of the environment on defect detection and improving the defect detection effect. It should be understood that the content described in this section is not intended to identify key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily apparent from the following description. Attached Figure Description

[0008] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely exemplary. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0009] Figure 1 A flowchart illustrating an online defect detection method for complex scenarios provided in an embodiment of the present invention;

[0010] Figure 2 This is a schematic diagram of the process for obtaining the first target image in an embodiment of the present invention;

[0011] Figure 3 This is a schematic diagram of the process for obtaining the image feature matrix in an embodiment of the present invention;

[0012] Figure 4 This is a schematic diagram of the structure of an online defect detection system for complex scenarios provided in an embodiment of the present invention.

[0013] Figure labeling: Image acquisition module 11, superpixel segmentation module 12, feature extraction module 13, defect detection module 14, image optimization module 15. Detailed Implementation

[0014] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0015] In order to solve the technical problem that the existing technology has poor defect detection effect due to unreasonable settings of the image acquisition process in complex scenes and poor image processing effect, the inventors of the present invention have creatively obtained an online defect detection method and system for complex scenes.

[0016] Example 1

[0017] Figure 1 A diagram illustrating an online defect detection method for complex scenarios provided by an embodiment of the present invention is shown below. Figure 1 As shown, the method includes:

[0018] Step S100: Start the first optical detection instrument to acquire image data of the first target to be inspected and obtain the first target image. The first optical detection instrument includes multiple beam emission sources and multiple cameras.

[0019] Among them, such as Figure 2 As shown, step S100 of this embodiment of the invention includes:

[0020] Step S110: Collect real-time light intensity information for the first target to be inspected;

[0021] Step S120: Obtain the surface material information of the first target to be inspected, and perform light reflectivity analysis based on the surface material information to obtain the light reflectivity coefficient;

[0022] Step 130: Obtain the size information of the first target to be inspected;

[0023] Step S40: Determine the image acquisition scheme for the multiple beam emission sources and the multiple cameras based on the real-time light intensity information, the light reflection intensity coefficient, and the size information;

[0024] Step 150: Control the first optical detection instrument to acquire image data according to the image acquisition scheme, and obtain the first target image.

[0025] In this embodiment of the invention, step S140 includes:

[0026] Step S141: Analyze the light source intensity and light source distribution of the multiple beam emission sources based on the real-time light intensity information and the light reflection intensity coefficient;

[0027] Step S142: Using image shadow intensity as the objective function, optimize the light source intensity analysis results and light source distribution analysis results to obtain optimized light source intensity information and light source distribution information;

[0028] Step S143: Determine the moving direction and moving speed of the first optical detection instrument based on the size information;

[0029] Step S144: The image acquisition scheme is composed of the light source intensity information, the light source distribution information, the moving direction, and the moving speed.

[0030] Specifically, this embodiment provides an online defect detection method for complex scenarios, which can be used to assist in the routine maintenance of equipment. In complex scenarios where lighting is low, direct image acquisition would result in blurry images, making it impossible to extract useful information and leading to poor defect detection. Therefore, it is necessary to acquire image data of a first target to be inspected using a first optical inspection instrument. The first target to be inspected is a metal or non-metal object such as a pipe or part to be inspected. The first optical inspection instrument is a device used for image acquisition, including multiple beam emitters and multiple cameras, capable of multi-angle image acquisition and possessing automatic acquisition and transmission functions. During image acquisition, the illumination intensity of the beam emitters and the movement scheme of the first optical inspection instrument can be set according to the ambient light and the optical properties and size of the first target to be inspected, thereby improving the usability of the first target image.

[0031] Specifically, real-time light intensity information is collected for the first target under inspection, which represents the brightness of the environment in which the first target is located. Further, the surface material information of the first target is acquired. This surface material information includes various materials such as metal and plastic, the specific type to be determined based on the actual situation. Different surface materials have different light reflection intensities. Based on this, a light reflectance analysis is performed to obtain a light reflection intensity coefficient, characterizing the light reflection intensity of the surface material of the first target. Finally, the size information of the first target is acquired, which represents the shape of the surface area of ​​the first target.

[0032] Further, based on the real-time light intensity information, the light reflection intensity coefficient, and the size information, an image acquisition scheme is determined for the multiple light beam emission sources and the multiple cameras. The image acquisition scheme includes light source intensity information, light source distribution information, movement direction, and movement speed. Based on the image acquisition scheme, the first optical detection instrument is controlled to acquire image data to obtain a first target image, ensuring the integrity and clarity of the image acquisition.

[0033] Specifically, the process of determining the image acquisition scheme is as follows: First, based on the real-time light intensity information and the light reflection intensity coefficient, the light source intensity and distribution of the multiple beam emission sources are analyzed. In simple terms, the real-time light intensity information refers to the brightness of the ambient light, and the light reflection intensity coefficient refers to the light reflection intensity of the first target object. To reduce the influence of the environment on the image and make the acquired image easier to extract features, it is necessary to illuminate the first target object with light from multiple beam emission sources to compensate for the ambient light. However, if the illumination direction and intensity are not set accurately when illuminating with multiple beam emission sources, shadows will appear in the image. These shadows will affect subsequent image feature extraction. The reason for the shadows is that the emitted light is too strong, and the first target object cannot completely absorb the emitted light. Therefore, it is necessary to adjust the light intensity and illumination direction to reduce image shadows. The light source intensity is the light intensity emitted by the multiple beam emission sources, and the light source distribution is the illumination direction of the multiple beam emission sources. For example, a cross-line light source, where two light sources illuminate each other, can effectively improve the illumination effect. Specifically, multiple sets of light source intensity analysis results and light source distribution analysis results are obtained. The image shading degree is used as the objective function. The image shading degree is the proportion of the shadow area in the entire image. The light source intensity analysis result and light source distribution analysis result with the minimum image shading degree are obtained as the light source intensity information and light source distribution information.

[0034] Further, the moving direction and speed of the first optical inspection instrument are determined based on the size information. Simply put, during defect detection, the size of the first target to be inspected may be large, such as a section of pipe. If image acquisition is only performed at one location, it only increases the distance between the first optical inspection instrument and the target, but the resulting image is not conducive to extracting defect features. Therefore, it is necessary to determine the moving direction and speed of the first optical inspection instrument based on the size information. The moving direction can be determined according to the actual situation, and the moving speed refers to the moving distance over a certain time period, such as moving 20 centimeters every 5 seconds. The moving time period can be determined based on the camera's shooting frequency; for example, if the camera can acquire an image every 3 seconds, it can be set to move 20 centimeters every 3 seconds. Finally, the image acquisition scheme is composed of the light source intensity information, the light source distribution information, the moving direction, and the moving speed. Subsequently, the first optical inspection instrument is controlled to acquire image data according to the image acquisition scheme, improving the accuracy of image acquisition and reducing the impact of the environment on defect detection.

[0035] Step S200: Perform superpixel segmentation on the first target image to obtain a first segmentation result, wherein the first segmentation result includes N superpixel images, where N is an integer greater than 0;

[0036] In this embodiment of the invention, step S200 includes:

[0037] Step S210: Perform pixel feature recognition on the first target image to obtain pixel recognition results;

[0038] Step S220: Perform feature clustering on the pixel recognition results to obtain the N superpixel images;

[0039] Step S230: Use the N superpixel images as the first segmentation result.

[0040] Specifically, a superpixel is a collection of pixels. It is a local region formed by several pixels that are physically adjacent and have similar features in an image. Superpixel segmentation is the process of grouping some pixels with similar features in the first target image together as a whole to form a superpixel image.

[0041] In layman's terms, the first target image contains multiple pixels. Superpixel segmentation is the process of clustering these pixels. A superpixel image contains multiple adjacent pixels with similar features. Based on this, N superpixel images are obtained, where N is a positive integer. For example, an image can be segmented into 100 superpixel images. Subsequent image processing uses these superpixel images as the smallest unit, improving processing speed. In defective images, surface defect areas and defect-free backgrounds often have different colors, textures, and other features. Superpixel segmentation can effectively delineate defective areas. The superpixel segmentation algorithm can be chosen freely, such as the SLIC algorithm, SEEDS algorithm, LSC algorithm, etc. When performing superpixel segmentation, the number of superpixel images can be limited. The specific limit can be determined based on practical experience, such as limiting N to be greater than 0 and less than 300, to improve image processing efficiency.

[0042] Specifically, pixel feature recognition is performed on the first target image. Pixel feature recognition is used to identify the position and color of pixels in the first target image. The pixel recognition result includes the position and color features of all pixels. Further, feature clustering is performed on the pixel recognition result to aggregate pixels with adjacent positions and similar colors together to obtain the N superpixel images. Each superpixel image contains multiple adjacent pixels. The N superpixel images are used as the first segmentation result to provide basic data for subsequent image feature extraction and improve image processing efficiency.

[0043] Step S300: Extract features from the N superpixel images to obtain an image feature matrix;

[0044] Among them, such as Figure 3 As shown, step S300 of this embodiment of the invention includes:

[0045] Step S310: Extract the first superpixel image from the N superpixel images, extract color features from the first superpixel image, and obtain the first color feature, which includes RGB color features and HSV color features;

[0046] Step S320: Perform filter feature extraction on the first superpixel image to obtain the texture features of the first image;

[0047] Step S330: Extract directional scale features from the first superpixel image using a directional controllable pyramid to obtain the first directional scale features;

[0048] Step S340: Construct a first superpixel image feature matrix using the first color feature, the first image texture feature, and the first directional scale feature;

[0049] Step S350: By analogy, obtain the N superpixel image feature matrices corresponding to the N superpixel images, and use them as the image feature matrices.

[0050] Specifically, multidimensional features are extracted from the N superpixel images. The multidimensional features include color features, frequency features, and orientation scale features. The results of the multidimensional feature extraction are used to construct an image feature matrix.

[0051] Specifically, any superpixel image is extracted from the N superpixel images as the first superpixel image. Color features are extracted from the first superpixel image to obtain a first color feature. The first color feature includes RGB color features and HSV color features. RGB color features refer to the color values ​​of red, green and blue. HSV color features refer to hue and saturation. Each superpixel is composed of multiple pixels. The color features of multiple pixels in the same superpixel image will have slight differences. The average value of the color features of multiple pixels in the superpixel image is calculated and used as the color feature of the superpixel image.

[0052] Feature extraction is performed using Gabor filters, which are particularly effective at extracting object texture features. Gabor filters can effectively describe changes in brightness and contrast in an image, making them especially suitable for describing image texture. Therefore, by inputting the first superpixel image into a Gabor filter, the first image texture features can be obtained. These first image texture features are the brightness and contrast features of the first superpixel image. Similarly, the average value of the brightness and contrast features of multiple pixels within the first superpixel image is also taken as the first image texture feature.

[0053] The first directional scale feature is obtained by extracting directional scale features from the first superpixel image using a directionally controllable pyramid. A directionally controllable pyramid is an image processing tool that decomposes an image into a series of image subbands of different scales and directions through linear decomposition. The first directional scale feature is obtained by applying a 2-scale, 6-directional filter to the image and extracting features from each superpixel.

[0054] A first superpixel image feature matrix is ​​constructed using the first color feature, the first image texture feature, and the first directional scale feature. Specifically, a 1×n or 1×n feature matrix can be constructed based on the number of features contained in the first color feature, the first image texture feature, and the first directional scale feature, where n is the number of features. Alternatively, three sub-feature matrices can be constructed based on the first color feature, the first image texture feature, and the first directional scale feature respectively, and then these three sub-feature matrices can be combined to form a 1×3 or 3×1 first superpixel image feature matrix. It should be noted that the above feature types and numbers can be adjusted according to actual conditions. Similarly, N superpixel image feature matrices corresponding to the N superpixel images are obtained as the image feature matrix, providing data support for subsequent defect detection.

[0055] Step S400: Input the image feature matrix into the defect detection model and output the target defect image;

[0056] In this embodiment of the invention, step S400 includes:

[0057] Step S410: Construct the defect detection model, which includes a matrix factorization layer, a saliency value calculation layer, and a target defect image extraction layer;

[0058] Step S420: Input the image feature matrix into the matrix decomposition layer to obtain a low-rank matrix, a sparse matrix, and a noise matrix;

[0059] Step S430: Input the sparse matrix into the saliency value calculation layer to obtain the N image saliency values ​​of the N superpixel images;

[0060] Step S440: The N image saliency values ​​are sorted in descending order by the target defect image extraction layer, and M superpixel images with saliency values ​​greater than or equal to a preset saliency value are obtained as the target defect images, where M is a positive integer greater than 0 and M≤N.

[0061] In this embodiment of the invention, step S420 includes:

[0062] Step S421: Collect and acquire a set of defect images that are identical to the first target model to be inspected;

[0063] Step S422: Set image transformation rules;

[0064] Step S423: Perform data augmentation on the defective image set according to the image transformation rules to obtain a sample image set;

[0065] Step S424: Extract features from the sample image set to obtain the sample image feature matrix;

[0066] Step S425: Perform matrix decomposition on the sample image matrix to obtain the sample low-rank matrix, sample sparse matrix, and sample noise matrix;

[0067] Step S426: After labeling the sample image set, the sample low-rank matrix, the sample sparse matrix, and the sample noise matrix, use them as the construction dataset to construct the matrix decomposition layer;

[0068] Step S427: Input the image feature matrix into the constructed matrix decomposition layer to obtain the low-rank matrix, the sparse matrix, and the noise matrix.

[0069] Specifically, the image feature matrix is ​​input into the defect detection model, and the target defect image is output. The defect detection model is a neural network model in machine learning, which includes a matrix factorization layer, a saliency calculation layer, and a target defect image extraction layer. The target defect image refers to M superpixel images selected from N superpixel images, where M is a positive integer greater than 0 and M≤N.

[0070] Specifically, a defect detection model is constructed, comprising a matrix factorization layer, a saliency calculation layer, and a target defect image extraction layer. The matrix factorization layer decomposes the image feature matrix into a low-rank matrix, a sparse matrix, and a noise matrix. The sparse matrix represents the defect region, and the low-rank matrix represents the defect-free region. Defect-free regions are typically strongly correlated and located in a low-dimensional subspace; therefore, they can be represented by a low-rank matrix. However, defect regions are different; they can be considered salient targets and represented by a sparse matrix. Specifically, an image decomposition algorithm (such as the LS-MAD algorithm) can be selected. The matrix factorization layer is then trained and validated using a sample set. Furthermore, the image feature matrix is ​​input into the matrix factorization layer to obtain the low-rank matrix, the sparse matrix, and the noise matrix.

[0071] The sparse matrix is ​​input into the saliency calculation layer to obtain N saliency values ​​for the N superpixel images. The saliency value calculation formula is as follows: a i =||d i ||, where a i d represents the image saliency value of the i-th superpixel. i The feature matrix representing the i-th superpixel, ||d iThe norm of the feature matrix of the i-th superpixel is represented by the norm of the vector, which is the sum of the squares of the vector elements to the power of 1 / 2. In other words, the sparse matrix can be represented by the eigenvector. The magnitude of the vector is calculated as N salient values ​​of the image. The salient value calculation formula is embedded in the salient value calculation layer.

[0072] The target defect image extraction layer sorts the N image saliency values ​​in descending order, and obtains M superpixel images with saliency values ​​greater than or equal to a preset saliency value as the target defect images, where M is a positive integer greater than 0 and M≤N, that is, a i The larger the value, the larger the superpixel d. i The greater the probability of a defective area, the more accurate the detection method becomes. Therefore, staff can set a preset saliency value based on the actual situation. The preset saliency value is a reference data for judging defective images. Thus, M superpixel images with saliency values ​​greater than or equal to the preset saliency value are used as the target defect images to ensure the accuracy of defect detection.

[0073] Specifically, the construction process of the matrix decomposition layer is as follows: A set of defect images identical to the first target to be inspected (e.g., identical pipes, parts, etc.) is acquired. This set includes defect images of different types (e.g., cracks), sizes, and shapes. Image transformation rules are set according to the actual situation. These rules include geometric transformations such as flipping, rotating, scaling, cropping, and deformation; color transformations; and operations such as adding noise, blurring, erasing, and filling. Alternatively, new samples can be synthesized from multiple samples according to certain principles. The specific rules can be customized; that is, if the number of sample images in the defect image set is too small, data augmentation is required. Based on the image transformation rules, geometric transformations such as flipping, rotating, scaling, cropping, and deformation, color transformations, and operations such as adding noise, blurring, erasing, and filling are performed on the images in the defect image set to augment the defect image set. The augmented defect image set is then used as the sample image set.

[0074] Further utilizing the same method as described above for obtaining the image feature matrix, feature extraction is performed on the sample image set to obtain the sample image feature matrix. Then, matrix decomposition is performed on the sample image matrix to obtain a sample low-rank matrix, a sample sparse matrix, and a sample noise matrix. After data annotation of the sample image set, the sample low-rank matrix, the sample sparse matrix, and the sample noise matrix, this serves as the construction dataset for building the matrix decomposition layer. That is, the sample image set, the sample low-rank matrix, the sample sparse matrix, and the sample noise matrix have a one-to-one correspondence. Based on this, the construction dataset can be divided into a training set and a validation set. The matrix decomposition layer is trained using data from the training set. After training, the output accuracy of the matrix decomposition layer is validated using data from the validation set to obtain a matrix decomposition layer with satisfactory accuracy. Finally, the image feature matrix is ​​input into the constructed matrix decomposition layer to obtain the low-rank matrix, the sparse matrix, and the noise matrix, providing data support for subsequent defect detection.

[0075] Step S500: Optimize the target defect image using a preset optimization scheme to obtain online detection results.

[0076] Specifically, image optimization is the process of enhancing defective images and weakening defect-free images, thereby improving the accuracy of defect detection. Specifically, a low-rank matrix and a sparse matrix are obtained. The low-rank matrix represents defect-free regions, and the sparse matrix represents defective regions. Using the same method as calculating the image saliency value of the sparse matrix, the low-rank saliency value of the low-rank matrix is ​​calculated. The low-rank saliency value of the low-rank matrix is ​​always smaller than the image saliency value of the sparse matrix. With the goal of the low-rank saliency value being much smaller than the image saliency value, the eigenvalues ​​of the low-rank and sparse matrices are adjusted. This is used as a preset optimization scheme to increase the saliency value of defective regions while decreasing the saliency value of defect-free regions. The optimized target defect image is used as the online detection result, thus improving the defect detection effect.

[0077] Based on the above analysis, the present invention provides an online defect detection method for complex scenarios. In this embodiment, a first optical inspection instrument is activated to acquire image data of a first target to be inspected, thereby obtaining a first target image. The first optical inspection instrument includes multiple beam emission sources and multiple cameras. The first target image is subjected to superpixel segmentation to obtain a first segmentation result. The first segmentation result includes N superpixel images, where N is an integer greater than 0. Feature extraction is performed on the N superpixel images to obtain an image feature matrix. The image feature matrix is ​​input into a defect detection model to output a target defect image. The target defect image is then optimized using a preset optimization scheme to obtain an online detection result. This achieves the technical effect of reducing the influence of the environment on defect detection and improving the defect detection effect.

[0078] Example 2

[0079] Based on the same inventive concept as the online defect detection method for a complex scenario in the foregoing embodiments, such as Figure 4 As shown, the present invention also provides an online defect detection system for complex scenarios, the system comprising:

[0080] Image acquisition module 11, the image acquisition module 11 is used to start the first optical detection instrument to acquire image data of the first target to be inspected, and obtain the first target image, wherein the first optical detection instrument includes multiple beam emission sources and multiple cameras;

[0081] The superpixel segmentation module 12 is used to perform superpixel segmentation on the first target image to obtain a first segmentation result, wherein the first segmentation result includes N superpixel images, where N is an integer greater than 0.

[0082] Feature extraction module 13 is used to extract features from the N superpixel images to obtain an image feature matrix;

[0083] Defect detection module 14, which is used to input the image feature matrix into the defect detection model and output the target defect image;

[0084] Image optimization module 15 is used to optimize the target defect image through a preset optimization scheme to obtain online detection results.

[0085] Furthermore, the system also includes:

[0086] A real-time ambient light analysis module is used to collect real-time light intensity information of the first target to be inspected.

[0087] The light reflectance analysis module is used to acquire the surface material information of the first target under inspection, perform light reflectance analysis based on the surface material information, and obtain the light reflectance intensity coefficient.

[0088] A size information acquisition module, wherein the size information acquisition module is used to acquire the size information of the first target to be inspected;

[0089] An image acquisition scheme determination module is used to determine an image acquisition scheme for the plurality of light beam emission sources and the plurality of cameras based on the real-time light intensity information, the light reflection intensity coefficient, and the size information.

[0090] The first target image acquisition module is used to control the first optical detection instrument to acquire image data according to the image acquisition scheme and acquire the first target image.

[0091] Furthermore, the system also includes:

[0092] A beam emission source analysis module is used to analyze the light source intensity and light source distribution of the plurality of beam emission sources based on the real-time light intensity information and the light reflection intensity coefficient.

[0093] A light source analysis and optimization module is used to optimize the light source intensity analysis results and light source distribution analysis results with image shadow intensity as the objective function, so as to obtain optimized light source intensity information and light source distribution information.

[0094] An instrument movement analysis module is used to determine the movement direction and speed of the first optical detection instrument based on the size information.

[0095] An information combination module is used to compose the image acquisition scheme using the light source intensity information, the light source distribution information, the movement direction, and the movement speed.

[0096] Furthermore, the system also includes:

[0097] A pixel feature recognition module is used to perform pixel feature recognition on the first target image to obtain pixel recognition results.

[0098] A feature clustering module is used to perform feature clustering on the pixel recognition results to obtain the N superpixel images;

[0099] The first segmentation result acquisition module is used to obtain the N superpixel images as the first segmentation result.

[0100] Furthermore, the system also includes:

[0101] The color feature extraction module is used to extract a first superpixel image from the N superpixel images, perform color feature extraction on the first superpixel image, and obtain a first color feature, which includes RGB color features and HSV color features.

[0102] A filtering feature extraction module is used to extract filtering features from the first superpixel image to obtain the first image texture features;

[0103] An orientation scale feature extraction module is used to extract orientation scale features from the first superpixel image through an orientation-controllable pyramid to obtain first orientation scale features.

[0104] The first feature matrix acquisition module is used to construct a first superpixel image feature matrix using the first color feature, the first image texture feature, and the first directional scale feature.

[0105] The image feature matrix acquisition module is used to acquire, by analogy, N superpixel image feature matrices corresponding to the N superpixel images, as the image feature matrix.

[0106] Furthermore, the system also includes:

[0107] A defect detection model building module is used to build the defect detection model, which includes a matrix factorization layer, a saliency value calculation layer, and a target defect image extraction layer.

[0108] A matrix decomposition module is used to input the image feature matrix into the matrix decomposition layer to obtain a low-rank matrix, a sparse matrix, and a noise matrix.

[0109] A saliency value calculation module is used to input the sparse matrix into the saliency value calculation layer to obtain N image saliency values ​​of the N superpixel images;

[0110] The superpixel filtering module is used to sort the N image saliency values ​​in descending order through the target defect image extraction layer, and obtain M superpixel images with image saliency values ​​greater than or equal to a preset saliency value as the target defect images, where M is a positive integer greater than 0 and M≤N.

[0111] Furthermore, the system also includes:

[0112] A defect image set acquisition module, which is used to collect and acquire a defect image set that is the same as the first target model to be inspected;

[0113] An image transformation rule setting module, which is used to set image transformation rules;

[0114] The data augmentation module is used to augment the defective image set according to the image transformation rules to obtain a sample image set;

[0115] A sample image feature extraction module is used to extract features from the sample image set and obtain a sample image feature matrix.

[0116] A sample image matrix decomposition module is used to perform matrix decomposition on the sample image matrix to obtain a low-rank sample matrix, a sparse sample matrix, and a noise sample matrix.

[0117] A matrix factorization layer construction module is used to construct the matrix factorization layer by annotating the sample image set, the sample low-rank matrix, the sample sparse matrix, and the sample noise matrix as the construction dataset.

[0118] The second matrix decomposition module is used to input the image feature matrix into the constructed matrix decomposition layer to obtain the low-rank matrix, the sparse matrix, and the noise matrix.

[0119] The specific example of the online defect detection method for complex scenarios in the aforementioned Embodiment 1 is also applicable to the online defect detection system for complex scenarios in this embodiment. Through the foregoing detailed description of the online defect detection method for complex scenarios, those skilled in the art can clearly understand the online defect detection system for complex scenarios in this embodiment. Therefore, for the sake of brevity, it will not be described in detail here.

[0120] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this invention can be achieved, and this is not limited herein.

[0121] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for online detection of defects in complex scenes, characterized in that, The method comprises: starting a first optical detection instrument to collect image data of a first target to be detected, to obtain a first target image, wherein the first optical detection instrument comprises a plurality of light beam emitting sources and a plurality of cameras; performing superpixel segmentation on the first target image to obtain a first segmentation result, the first segmentation result comprising N superpixel images, N being an integer greater than 0; performing feature extraction on the N superpixel images to obtain an image feature matrix; inputting the image feature matrix into a defect detection model to output a target defect image; performing image optimization on the target defect image through a preset optimization scheme to obtain an online detection result; the starting of the first optical detection instrument to collect image data of the first target to be detected to obtain the first target image comprises: collecting real-time light intensity information of the first target to be detected; obtaining surface material information of the first target to be detected, performing illumination reflectivity analysis according to the surface material information, and obtaining an illumination reflection intensity coefficient; obtaining size information of the first target to be detected; determining an image collection scheme of the plurality of light beam emitting sources and the plurality of cameras according to the real-time light intensity information, the illumination reflection intensity coefficient and the size information; controlling the first optical detection instrument to collect image data according to the image collection scheme to obtain the first target image; the determination of the image collection scheme of the plurality of light beam emitting sources and the plurality of cameras according to the real-time light intensity information, the illumination reflection intensity coefficient and the size information comprises: performing light source intensity and light source distribution analysis on the plurality of light beam emitting sources according to the real-time light intensity information and the illumination reflection intensity coefficient; optimizing the light source intensity analysis result and the light source distribution analysis result by taking image shading degree as a target function to obtain optimized light source intensity information and light source distribution information; determining a moving direction and a moving speed of the first optical detection instrument according to the size information; composing the image collection scheme by the light source intensity information, the light source distribution information, the moving direction and the moving speed.

2. The method of claim 1, wherein, the superpixel segmentation on the first target image to obtain the first segmentation result comprises: performing pixel feature recognition on the first target image to obtain a pixel recognition result; performing feature clustering on the pixel recognition result to obtain the N superpixel images; taking the N superpixel images as the first segmentation result.

3. The method of claim 1, wherein, the feature extraction on the N superpixel images to obtain the image feature matrix comprises: extracting a first superpixel image from the N superpixel images, performing color feature extraction on the first superpixel image to obtain a first color feature, the first color feature comprising an RGB color feature and an HSV color feature; performing filter feature extraction on the first superpixel image to obtain a first image texture feature; performing direction scale feature extraction on the first superpixel image through a direction controllable pyramid to obtain a first direction scale feature; Assemble a first superpixel image feature matrix with the first color feature, the first image texture feature and the first direction scale feature; Similarly, obtain N superpixel image feature matrices corresponding to the N superpixel images as the image feature matrix.

4. The method of claim 1, wherein, The inputting of the image feature matrix into the defect detection model and the outputting of the target defect image comprises: The defect detection model comprises a matrix decomposition layer, a significant value calculation layer and a target defect image extraction layer; The image feature matrix is inputted into the matrix decomposition layer to obtain a low-rank matrix, a sparse matrix and a noise matrix; The sparse matrix is inputted into the significant value calculation layer to obtain N image significant values of the N superpixel images; The N image significant values are sequentially sorted in descending order by the target defect image extraction layer, and M superpixel images with image significant values greater than or equal to a preset significant value are obtained as the target defect image, wherein M is a positive integer greater than 0 and M≤N.

5. The method of claim 4, wherein, The inputting of the image feature matrix into the matrix decomposition layer to obtain a low-rank matrix, a sparse matrix and a noise matrix comprises: A defect image set identical to the first target to be inspected is collected and obtained; An image transformation rule is set; The defect image set is data-augmented according to the image transformation rule to obtain a sample image set; Feature extraction is performed on the sample image set to obtain a sample image feature matrix; Matrix decomposition is performed on the sample image matrix to obtain a sample low-rank matrix, a sample sparse matrix and a sample noise matrix; After data labeling of the sample image set, the sample low-rank matrix, the sample sparse matrix and the sample noise matrix, the matrix decomposition layer is constructed as a construction dataset; The image feature matrix is inputted into the constructed matrix decomposition layer to obtain the low-rank matrix, the sparse matrix and the noise matrix.

6. A system for online detection of defects in complex scenes, characterized in that, The system comprises: An image acquisition module, which is configured to start a first optical detection instrument to acquire image data of a first target to be inspected, and obtain a first target image, wherein the first optical detection instrument comprises a plurality of light beam emitting sources and a plurality of cameras; A superpixel segmentation module, which is configured to perform superpixel segmentation on the first target image, and obtain a first segmentation result comprising N superpixel images, wherein N is an integer greater than 0; A feature extraction module, which is configured to perform feature extraction on the N superpixel images, and obtain an image feature matrix; A defect detection module, which is configured to input the image feature matrix into a defect detection model, and output a target defect image; An image optimization module, which is configured to perform image optimization on the target defect image by using a preset optimization scheme, and obtain an online detection result; The system further comprises: A real-time ambient light analysis module, which is configured to collect real-time light intensity information of the first target to be inspected; An illumination reflectivity analysis module is configured to acquire surface material information of the first target to be detected, perform illumination reflectivity analysis according to the surface material information, and obtain an illumination reflection intensity coefficient; A size information acquisition module is configured to acquire size information of the first target to be detected; An image acquisition scheme determination module is configured to determine an image acquisition scheme for the multiple light beam emitting sources and the multiple cameras according to the real-time light intensity information, the illumination reflection intensity coefficient, and the size information; A first target image acquisition module is configured to control the first optical detection instrument to acquire image data and acquire the first target image according to the image acquisition scheme; A light beam emitting source analysis module is configured to perform light source intensity and light source distribution analysis on the multiple light beam emitting sources according to the real-time light intensity information and the illumination reflection intensity coefficient; A light source analysis optimization module is configured to take image shading degree as a target function, optimize the light source intensity analysis result and the light source distribution analysis result, and obtain optimized light source intensity information and light source distribution information; An instrument movement analysis module is configured to determine a movement direction and a movement speed of the first optical detection instrument according to the size information; An information combination module is configured to combine the light source intensity information, the light source distribution information, the movement direction, and the movement speed to form the image acquisition scheme.

Images