Surface defect detection method, equipment and storage medium

Abstract

The present application discloses a surface defect detection method, device and storage medium, which relates to the field of image processing technology, including: obtaining a hyperspectral image to be detected, dividing the hyperspectral image into sub-images, and obtaining multiple initial sub-images; determining the average spectrum of each initial sub-image, inputting the average spectrum of the initial sub-image into a target sub-image classifier, and judging whether the initial sub-image has defects; in the case that the initial sub-image has defects, determining the optimal characteristic band of the initial sub-image, and obtaining the target sub-image; inputting the target sub-image into a target defect detector, and obtaining the defect detection result of the hyperspectral image. The present application achieves the technical effect of improving the efficiency of surface defect detection of industrial products.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a surface defect detection method, device and storage medium. Background Art

[0002] With the development of science and technology and the continuous progress of society, on the one hand, various industrial production lines are moving towards more efficient directions such as automation and intelligence, while on the other hand, people's requirements for the appearance and quality of various industrial products are getting higher and higher. In the increasingly efficient and intelligent industrial production, how to quickly and accurately detect defects such as the appearance quality of various industrial products has become an important issue that needs to be solved urgently.

[0003] At present, when inspecting the surface of a product, the original image needs to be segmented and all the segmented images need to be inspected. If the segmented size is too large, the image inspection result will be poor. If the segmented size is too small, a lot of inspection time will be consumed for one original image. Therefore, the existing method for inspecting the surface of a product still has the problem of low inspection efficiency. Summary of the invention

[0004] The main purpose of the present application is to provide a surface defect detection method, device and storage medium, aiming to solve the technical problem of low detection efficiency of existing methods for defect detection on the surface of industrial products.

[0005] To achieve the above objectives, the present application proposes a surface defect detection method, the method comprising:

[0006] Acquire a hyperspectral image to be detected, and divide the hyperspectral image into sub-images to obtain a plurality of initial sub-images;

[0007] Determine an average spectrum of each of the initial sub-images, input the average spectrum of the initial sub-images into a target sub-image classifier, and judge whether the initial sub-images have defects;

[0008] In the case where the initial sub-image has defects, determining the optimal characteristic band of the initial sub-image to obtain a target sub-image;

[0009] The target sub-image is input into a target defect detector to obtain a defect detection result of the hyperspectral image.

[0010] In one embodiment, before the step of acquiring a hyperspectral image to be detected and dividing the hyperspectral image into sub-images to obtain a plurality of initial sub-images, the step further includes:

[0011] Acquire a sample spectral image, and separate a color channel image from the sample spectral image;

[0012] Annotating defects in the color channel image;

[0013] Defect data of the marked positions are extracted, and a defect database is established based on the defect data.

[0014] In one embodiment, before the step of determining the average spectrum of each of the initial sub-images, inputting the average spectrum of the initial sub-images into the target sub-image classifier, and judging whether the initial sub-images have defects, the step further includes:

[0015] Dividing the sample spectral image into sub-images to obtain sample sub-images;

[0016] An average spectrum of the sample sub-image is determined, and an initial sub-image classifier is trained based on the average spectrum of the sample sub-image and the defect result to obtain the target sub-image classifier.

[0017] In one embodiment, after the step of dividing the sample spectral image into sub-images to obtain sample sub-images, the step further includes:

[0018] Determine the overlap area of ​​the sample sub-image and the defect;

[0019] Calculating the ratio between the area of ​​the overlapped region and the total area of ​​the sample sub-image;

[0020] When the ratio is greater than a preset ratio threshold, it is determined that the sample sub-image has a defect.

[0021] In one embodiment, after the step of dividing the sample spectral image into sub-images to obtain sample sub-images, the step further includes:

[0022] Search the defect database according to the sample sub-graph to obtain the sub-graph defect coordinates;

[0023] Determine the optimal characteristic band of the sample sub-image to obtain a fused sample sub-image;

[0024] The fused sample sub-image is input into an initial defect detector, and the initial defect detector is trained based on the sub-image defect coordinates to obtain the target defect detector.

[0025] In one embodiment, when the initial sub-image has defects, before the step of determining the optimal characteristic band of the initial sub-image to obtain the target sub-image, the step further includes:

[0026] Taking the target subgraph classifier as the particle fitness function, updating the particle velocity and particle position;

[0027] The updated optimal particle position is obtained by iterative optimization, and the band represented by the optimal particle position is used as the optimal selected band.

[0028] In one embodiment, the step of inputting the target sub-image into a target defect detector to obtain a defect detection result of the hyperspectral image includes:

[0029] Inputting the target sub-image into the target defect detector to extract multi-scale feature information of the target sub-image;

[0030] Based on the multi-scale feature information, detecting and obtaining defect categories and defect locations in the target sub-image;

[0031] The defect position is mapped to the hyperspectral image to obtain the three-dimensional coordinates of the defect.

[0032] In addition, to achieve the above-mentioned purpose, the present application also proposes a surface defect detection device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the surface defect detection method described above.

[0033] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the surface defect detection method described above are implemented.

[0034] One or more technical solutions proposed in this application are to obtain a hyperspectral image to be detected, divide the hyperspectral image into sub-images, and obtain multiple initial sub-images; determine the average spectrum of each of the initial sub-images, input the average spectrum of the initial sub-images into a target sub-image classifier, judge whether the initial sub-image has defects, divide the original image into initial sub-images, and use the average spectrum data to judge whether the initial sub-image contains defects in advance, which can reduce the detection of the background image and improve the detection efficiency; in the case where the initial sub-image has defects, determine the optimal feature band of the initial sub-image to obtain the target sub-image, the optimal feature band takes into account the needs of a specific application or task, and the obtained target sub-image can meet the specific requirements of the detection; input the target sub-image into a target defect detector to obtain the defect detection result of the hyperspectral image, and the defect detection result can be used to know the location of the defect in the hyperspectral image and the type of the defect. After the sub-image division, since there are a large number of initial sub-images without defects, the target sub-image with defects is screened out by the average spectrum, and then the target sub-image with the optimal feature band is detected to obtain a specific defect detection result, which can greatly reduce the number of sub-images to be detected and improve the detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0036] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0037] Figure 1 A schematic diagram of a process flow provided for the first embodiment of the surface defect detection method of the present application;

[0038] Figure 2 A schematic diagram of the sub-graph division involved in the surface defect detection method of the present application;

[0039] Figure 3 A schematic diagram of the structure of an initial defect detector involved in the surface defect detection method of the present application;

[0040] Figure 4 This is a schematic diagram of the structure of the PAFPN module involved in the surface defect detection method of this application;

[0041] Figure 5 A schematic diagram of a brief process of the training phase of a surface defect detection method provided in this application;

[0042] Figure 6 A schematic diagram of a simple process of the detection phase of a surface defect detection method provided in this application;

[0043] Figure 7 Schematic diagram of the device structure of the hardware operating environment involved in the surface defect detection method in the embodiment of the present application.

[0044] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0045] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0046] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0047] A surface defect detection method based on deep learning is provided in the related art. After acquiring the original high-resolution image, it is necessary to divide the extracted large target image into several uniform small target images, and then perform defect detection on the several divided small target images, and finally summarize the detection results of each small target and output them as the detection results of the original image. When performing the detection, the original image needs to be divided and all the divided images need to be detected. If the size of the segmentation is too large, the detection result of the image will be poor. If the size of the segmentation is too small, it will take a lot of detection time for one original image.

[0048] The related art also provides a method for hyperspectral image detection based on deep learning. After collecting hyperspectral data, the maximum difference method is used to select the bands with the most obvious contrast between damaged and undamaged images, and finally the high-resolution images composed of the selected bands are annotated and the model is trained, and finally the detection results are output through the Faster-RCNN network. After the spectral data is feature selected using the maximum difference method, the bands composed of the feature bands are annotated, and the image composed of the feature bands is directly detected using Faster-RCNN. The maximum difference method only considers a single statistic and does not consider the correlation between different bands. On the other hand, the human eye has limited recognition ability for non-RGB (Red Green Blue) images. Directly annotating the feature bands will affect the accuracy of the annotation, and the subjective comparison of the human eye with and without damaged band images is relatively subjective. The error is large. In addition, most of the current deep learning models for visual detection require a small input image size, and the detection hardware requirements for large-size images are high and time-consuming.

[0049] Based on this, the present application embodiment provides a surface defect detection method, referring to Figure 1 , Figure 1 This is a schematic flow chart of the first embodiment of the surface defect detection method of the present application.

[0050] In this embodiment, the surface defect detection method includes steps S10 to S40:

[0051] Step S10, obtaining a hyperspectral image to be detected, dividing the hyperspectral image into sub-images, and obtaining a plurality of initial sub-images;

[0052] The surface defect detection method of this embodiment is applied in the scenario where the input image is a hyperspectral image. A hyperspectral image refers to an image with a spectral resolution of 10 -2Spectral images within the range of the order of λ. Hyperspectral images contain wavelength information and can reflect differences in the physical structure of the surface or interior of an object. Hyperspectral images can be acquired using a hyperspectral camera. The hyperspectral image to be detected refers to an image containing the surface spectral information of the defect to be detected. A hyperspectral camera can be used to image the surface of the industrial product to be detected to obtain a hyperspectral image to be detected.

[0053] Sub-image division refers to the process of dividing the original image into multiple local images, and the local images formed after the division are sub-images. In this embodiment, the hyperspectral image is sub-imaged to obtain multiple initial sub-images. Since the hyperspectral image contains three-dimensional data, the hyperspectral image can be first separated into two-dimensional information, and then sub-image division is performed on the two-dimensional basis to obtain the initial sub-image.

[0054] In a feasible implementation manner, before step S10, steps S01 to S03 are also included:

[0055] Step S01, obtaining a sample spectral image, and separating a color channel image from the sample spectral image;

[0056] The sample spectral image refers to a spectral image used as sample data for model training. In this embodiment, both the target sub-image classifier and the target defect detector can be trained using the relevant data of the sample spectral image. The color channel image refers to an image that reflects the color of the object imaging in two dimensions, which can be an RGB channel image.

[0057] Step S02, marking defects in the color channel image;

[0058] It is understandable that the sample spectral image acquired by the hyperspectral camera contains three-dimensional data, so it is more difficult to directly annotate the hyperspectral image, while it is easier to annotate the color channel image. After obtaining the color channel image, you can use annotation software to annotate defects. The annotation software that can be used includes LabelMe, LabelImg, etc.

[0059] Step S03: extract the defect data of the marked position, and establish a defect database based on the defect data.

[0060] The marked position refers to the position of the marked defect, which can be the coordinate position in the color channel image. The coordinate position in the color channel image can be further mapped to the sample spectrum image to obtain the three-dimensional position in the sample spectrum image. The defect data can include defect type, defect location and spectral reflectance information. The above defect data can be recorded and stored in the defect database.

[0061] Step S20, determining the average spectrum of each of the initial sub-images, inputting the average spectrum of the initial sub-images into a target sub-image classifier, and judging whether the initial sub-images have defects;

[0062] The average spectrum of the initial sub-graph refers to the average of the spectral data of all the initial sub-graphs. The target sub-graph classifier refers to a trained model for classifying sub-graphs. The classification referred to here can be regarded as the classification of whether the initial sub-graph has defects, including two categories: defects exist and defects do not exist. The target sub-graph classifier can adopt different models as long as it can achieve the purpose of distinguishing whether there are defects. This embodiment does not impose specific restrictions on this. In a feasible implementation, the target sub-graph classifier adopts an SVM (Support Vector Machine) model. The result output by the target sub-graph classifier is used as the basis for whether the initial sub-graph has defects. If the currently input initial sub-graph does not have defects, the judgment of other initial sub-graphs can continue.

[0063] Step S30, when the initial sub-image has defects, determining the optimal characteristic band of the initial sub-image to obtain a target sub-image;

[0064] If it is determined that the initial sub-image has defects, the information of the defect category and the location of the defect is further obtained later. The data volume of hyperspectral images is huge, and the redundancy between the bands is high. Due to the existence of the Hughes effect, the hyperspectral images can be processed by dimensionality reduction. The optimal feature band refers to the band that meets the application scenario of this embodiment and can reflect the nature of the defect. It can be understood that the defect data stored in the defect database includes reflectance information, and the appropriate optimal feature band can be selected in combination with the data in the defect database. After determining the optimal feature band, fusion is performed to obtain the target sub-image.

[0065] Step S40: input the target sub-image into a target defect detector to obtain a defect detection result of the hyperspectral image.

[0066] It is understandable that the target defect detector refers to a trained model for detecting defect information in a hyperspectral image. When it is determined that there are defects in the target sub-image, the defect detection result can describe the location information and category information of the defect. This embodiment does not specifically limit the model structure of the target defect detector, as long as it can achieve the purpose of detecting the location and category of the defect. In some feasible implementations, the target defect detector can adopt a target detection model, for example, FPN (Feature Pyramid Network) and Faster-RCNN (Faster Regions with Convolutional Neural Network).

[0067] In a feasible implementation, step S40 includes steps S41 to S43:

[0068] Step S41, inputting the target sub-image into the target defect detector to extract multi-scale feature information of the target sub-image;

[0069] This embodiment uses a target defect detector with a PAFPN (Path Aggregation Feature Pyramid Network) structure. The target subgraph is split into patches, which are passed through a Linear layer and a BatchNorm layer respectively. After adding the position code according to the position, the token is input into the transformer encoder structure to obtain the original Figure 4 The feature map is downsampled by 8 times, 16 times, and 32 times, respectively, and the feature information of different scales is obtained by analogy.

[0070] Step S42, based on the multi-scale feature information, detecting and obtaining the defect category and defect position in the target sub-image;

[0071] By inputting feature maps of different scales into the PAFPN architecture, we can fuse the low-level detail information and high-level semantic information of the image, predict the defect category and defect location in the target sub-image, and annotate them with annotation boxes or other forms.

[0072] Step S43, mapping the defect position to the hyperspectral image to obtain the three-dimensional coordinates of the defect.

[0073] It can be understood that the target sub-image can be obtained by dividing in a two-dimensional scale, and there is a mapping relationship between the coordinate position in the target sub-image and the three-dimensional coordinate in the hyperspectral image. Through this mapping relationship, the predicted defect position can be mapped to three dimensions to obtain the three-dimensional coordinates of the defect. The three-dimensional coordinates of the defect and the defect category are used as the defect detection results.

[0074] This embodiment provides a surface defect detection method, which obtains a hyperspectral image to be detected, divides the hyperspectral image into sub-images, and obtains multiple initial sub-images; determines the average spectrum of each of the initial sub-images, inputs the average spectrum of the initial sub-image into a target sub-image classifier, judges whether the initial sub-image has defects, divides the original image into initial sub-images, and uses the average spectrum data to judge whether the initial sub-image contains defects in advance, which can reduce the detection of the background image and improve the detection efficiency; when the initial sub-image has defects, determines the optimal feature band of the initial sub-image to obtain the target sub-image, the optimal feature band takes into account the needs of a specific application or task, and the obtained target sub-image can meet the specific requirements of the detection; inputs the target sub-image into a target defect detector to obtain the defect detection result of the hyperspectral image, and the defect detection result can be used to know the position of the defect in the hyperspectral image and the type of the defect. After the sub-image division, since there are a large number of initial sub-images without defects, the target sub-image with defects is screened out by the average spectrum, and then the target sub-image with the optimal feature band is detected to obtain a specific defect detection result, which can greatly reduce the number of sub-images to be detected and improve the detection efficiency.

[0075] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the first embodiment can be referred to the above description, and will not be repeated hereafter. On this basis, before step S20, the surface defect detection method further includes steps A10 to A20:

[0076] Step A10, dividing the sample spectral image into sub-images to obtain sample sub-images;

[0077] The sub-image division method of the sample spectral image can be the same as the division method of the hyperspectral image in the detection process. The sample sub-image is also divided from the sample spectral image in two dimensions.

[0078] In a feasible implementation manner, step A10 further includes steps A11 to A13:

[0079] Step A11, determining the overlapping area of ​​the sample sub-image and the defect;

[0080] It is understandable that when there are many sub-graphs, the range involved in a single defect may include multiple sample sub-graphs. The overlapping area refers to the area where the initial sub-graph overlaps with the defect, which can be regarded as the area of ​​the intersection range between the defect and the sample sub-graph.

[0081] Step A12, calculating the ratio between the area of ​​the overlapped region and the total area of ​​the sample sub-image;

[0082] Figure 2 A schematic diagram of a subgraph partition is shown in Figure 2 Take the RGB image shown in as an example, where S a represents the complete defect area, S bi represents the i-th subgraph, S abi It indicates the part where the defect overlaps with sub-image i, and β is a constant, which is generally 0.3. That is, if the defective part of a sub-image exceeds 30%, the sub-image is considered defective. The ratio can be calculated by referring to the following formula 1.

[0083] Formula 1: .

[0084] Step A13: when the ratio is greater than a preset ratio threshold, determine that the sample sub-image has a defect.

[0085] Referring to the above formula 1, Represents the preset ratio threshold, which is a constant that can be set according to actual conditions, for example, it can be set to 0.3. When the ratio is greater than 0.3, the portion of the sample sub-image containing defects exceeds 30% of the total area, and it can be determined that the sample sub-image has defects. When the ratio is less than or equal to the preset ratio threshold, it is determined that the sample sub-image does not have defects. In the process of training the target sub-image classifier, the defect result of the sample sub-image obtained by the above judgment method can be used as the basis for classification.

[0086] Step A20: determining the average spectrum of the sample sub-image, and training the initial sub-image classifier based on the average spectrum of the sample sub-image and the defect result to obtain the target sub-image classifier.

[0087] The average spectrum of all sample sub-images can be calculated based on the defect database, and the average spectrum data can be used to train the initial sub-image classifier SVM in combination with the defect results obtained in the above manner. The initial sub-image classifier can refer to the following formula 2. The defect coordinates in the sample spectral image are obtained based on the established defect database, and the defect coordinates of the positive sample can be recalculated in combination with the defect results.

[0088] Formula 2: .

[0089] Among them, sign represents the sign function, x represents the average spectral data of the sub-image to be predicted, and x i represents the sub-graph average spectral data as support vector, y i Indicates whether the corresponding subgraph is defective, with a value of -1 or 1, α i It represents the non-negative weight factor of the corresponding subgraph, which is obtained by the training process. K is the kernel function, which is generally a radial basis kernel. b is the bias term, which is also obtained by the training process.

[0090] In a feasible implementation manner, before step S30, steps S31 to S32 are also included:

[0091] Step S31, using the target subgraph classifier as the particle fitness function, updating the particle velocity and particle position;

[0092] The amount of hyperspectral image data is very large, the redundancy between the bands is high, and due to the existence of the Hughes effect, the original image needs to be reduced in dimension. This embodiment uses a particle swarm optimization algorithm to select the optimal feature band. It can be understood that the band selection is regarded as a binary classification problem, and the final optimization target is an n-dimensional binary vector, where n represents the number of bands. The particle velocity can be updated with reference to the following formula 3, and the particle position can be updated with reference to the following formula 4.

[0093] Formula 3: .

[0094] Among them, v i,j represents the velocity of particle i in the j dimension, w is the inertia weight, c 1 、c 2 is the acceleration constant, r 1 、r 2 is a random number between 0 and 1, p i,j is the optimal position of particle i in dimension j, g j is the global best position.

[0095] Formula 4: .

[0096] Among them, x i,j Represents the position of particle i in the j dimension.

[0097] Step S32, iterative optimization is performed to obtain the updated optimal particle position, and the band represented by the optimal particle position is used as the optimal selected band.

[0098] Referring to the above steps, the best position of the example obtained after iterative optimization is the best selected band obtained by the particle swarm optimization algorithm. When performing defect detection, the method of using the human eye to judge whether the defect is obvious in different bands is too subjective, and methods such as the maximum difference method only consider the differences between bands without explicitly considering the needs of specific applications or tasks. Although some heuristic algorithms can achieve good results in band selection, they need to judge the quality of the band selection results during the iteration process, and additional training of other models will increase the system overhead. The method of using the particle swarm optimization algorithm for band selection proposed in this embodiment uses the SVM classifier as the fitness function, which not only gets rid of the subjectivity of human eye selection, but also comprehensively considers the specific requirements for identifying defects, and does not increase additional system overhead.

[0099] In a feasible implementation manner, step A10 further includes steps A14 to A16:

[0100] Step A14, searching the defect database according to the sample sub-graph to obtain the sub-graph defect coordinates;

[0101] The defect database stores data representing defect categories and defect locations, and the sub-image defect coordinates can be directly found.

[0102] Step A15, determining the optimal characteristic band of the sample sub-image to obtain a fused sample sub-image;

[0103] By referring to the process of band selection by the particle swarm optimization algorithm, the optimal characteristic band of the sample sub-graph can be calculated, and the sample sub-graphs are fused according to the optimal characteristic band to obtain a fused sample sub-graph.

[0104] Step A16: input the fused sample sub-image into the initial defect detector, train the initial defect detector based on the sub-image defect coordinates, and obtain the target defect detector.

[0105] Figure 3 is a schematic diagram of the structure of the initial defect detector, such as Figure 3 As shown in the figure, the overall progressive pyramid architecture is adopted, combined with the design idea of ​​vision transformer, the fusion sample sub-image generated by the best band is input, the sub-image is divided into 8*8 patches, and the Linear layer and BatchNorm layer are passed through respectively. After adding the position code according to the position, the token is input into the transformer encoder structure to obtain the original Figure 4 The feature map is downsampled by 8 times, 16 times, and 32 times, respectively. The feature map is input into the PAFPN module to fuse the low-level detail information and high-level semantic information of the image. The structure of the PAFPN module is as follows: Figure 4 As shown. Finally, four detection heads of different sizes are used to detect image defects and output the final detection results. During the training process, the model parameters in the initial defect detector are iteratively optimized, and the target defect detector is obtained after the training is completed.

[0106] In this embodiment, the sample spectral image is divided into sub-images, and the sub-image SVM classifier is trained using the average spectral data to judge in advance whether the sub-image contains defects, which greatly reduces the detection of the background image and improves the overall detection efficiency. The deep learning network combines the advantages of CNN inductive bias and transformer global attention, uses a pyramid structure to use transformer to generate multi-scale feature maps, and combines the PAFPN structure to detect defects of different scales.

[0107] Based on the above embodiments of the present application, in the third embodiment of the present application, the same or similar contents as those in the above embodiments can be referred to the above introduction, and will not be repeated in the following. On this basis, before step S10, the surface defect detection method further includes steps B10 to B20:

[0108] Step B10, obtaining an initial spectral reflection signal;

[0109] The initial spectral reflection signal refers to the spectral signal directly collected by the uncorrected device. It can be understood that the information carried by the spectral reflection signal is contained in the hyperspectral image.

[0110] Step B20, performing a black and white standard plate correction process on the initial spectral reflection signal to obtain a corrected target spectral reflection signal.

[0111] In order to reduce the noise and error introduced by the detection equipment, measurement environment and other factors, the reflectance spectrum is calibrated by black and white standard plates. Black and white standard plate calibration refers to the use of standard black and white plates to calibrate the reflection signal and correct the operation of spectral image acquisition equipment such as hyperspectral cameras.

[0112] The black and white standard plate calibration can be performed according to the following formula 5.

[0113] Formula 5: .

[0114] Among them, W b is the spectrum reflection signal of the ideal blackboard, W s is the spectral reflection signal of an ideal white plate, W o is the initial spectral reflection signal to be corrected, and W is the target spectral reflection signal after correction.

[0115] In this embodiment, spectral correction processing is performed before collecting the hyperspectral image, which can reduce the error of noise introduced by factors such as detection equipment and measurement environment, and improve imaging accuracy.

[0116] For example, to help understand the implementation process of the surface defect detection method obtained by combining this embodiment with the above embodiments, please refer to Figure 5 and Figure 6 , Figure 5 A brief flow chart of the training phase of the surface defect detection method is provided. Figure 6 A simple flow chart of the detection phase of a surface defect detection method is provided. Specifically, in the training phase, a hyperspectral image of an object is collected after pipe laying correction, and an RGB three-channel image is separated from the hyperspectral image. After defect annotation, the defect coordinates and band reflectance data are extracted, and a defect database is established based on this to store data. The hyperspectral image is used as the original image, and the original image is divided into sub-images. According to the defect database, the average spectrum data of the sub-image can be obtained to train the sub-image classifier. At the same time, the defect coordinates of the sub-image can also be obtained according to the defect database, so as to calculate the optimal band combination, and thus train the sub-image defect detector. In the detection phase, a hyperspectral image is collected using a debugged hyperspectral camera. The original image is divided into sub-images, the average spectrum of the sub-image is calculated, and the sub-image is input into the trained sub-image classifier to determine whether the sub-image has defects. If there are defects, defect detection is performed. If not, the next sub-image is judged. According to the optimal band combination obtained in the data preparation phase, the corresponding band of the sub-image is selected to obtain a new sub-image. The newly obtained sub-image is output to the trained defect detector, the defect is detected, and the coordinates and category of the defect are obtained. The defect coordinates obtained on the sub-image are mapped to the original image to obtain the final detection result.

[0117] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the surface defect detection method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0118] The present application provides a surface defect detection device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the surface defect detection method in the above-mentioned embodiment one.

[0119] Reference below Figure 7 , which shows a schematic diagram of the structure of a surface defect detection device suitable for implementing an embodiment of the present application. The surface defect detection device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7The surface defect detection device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0120] like Figure 7 As shown, the surface defect detection device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the surface defect detection device are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the surface defect detection device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a surface defect detection device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0121] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0122] The surface defect detection device provided by the present application adopts the surface defect detection method in the above embodiment, which can solve the technical problem of low detection efficiency of the existing method for defect detection on the surface of industrial products. Compared with the prior art, the beneficial effects of the surface defect detection device provided by the present application are the same as the beneficial effects of the surface defect detection method provided by the above embodiment, and the other technical features in the surface defect detection device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0123] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0124] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0125] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, wherein the computer-readable program instructions are used to execute the surface defect detection method in the above-mentioned embodiment.

[0126] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0127] The computer-readable storage medium may be included in the surface defect detection device; or may exist independently without being assembled into the surface defect detection device.

[0128] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the surface defect detection device, the surface defect detection device: obtains a hyperspectral image to be detected, divides the hyperspectral image into sub-images, and obtains multiple initial sub-images; determines the average spectrum of each of the initial sub-images, inputs the average spectrum of the initial sub-images into a target sub-image classifier, and determines whether the initial sub-image has defects; if the initial sub-image has defects, determines the optimal feature band of the initial sub-image to obtain a target sub-image; inputs the target sub-image into a target defect detector to obtain a defect detection result of the hyperspectral image.

[0129] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0130] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0131] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0132] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned surface defect detection method, and can solve the technical problem of low detection efficiency of existing methods for detecting defects on the surface of industrial products. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the surface defect detection method provided by the above-mentioned embodiment, and will not be described in detail here.

[0133] The present application also provides a computer program product, including a computer program, which implements the steps of the surface defect detection method as described above when executed by a processor.

[0134] The computer program product provided by the present application can solve the technical problem of low detection efficiency of existing methods for detecting defects on the surface of industrial products. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the surface defect detection method provided by the above embodiment, and will not be repeated here.

[0135] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A surface defect detection method, characterized in that: The surface defect detection method comprises: Acquire a sample spectral image, and separate a color channel image from the sample spectral image; marking defects in the color channel image; Extracting defect data of the marked positions, and establishing a defect database based on the defect data; Dividing the sample spectral image into sub-images to obtain sample sub-images; Determine the average spectrum of the sample sub-image, train an initial sub-image classifier based on the average spectrum of the sample sub-image and the defect result to obtain a target sub-image classifier, wherein the target sub-image classifier adopts a support vector machine model; Acquire a hyperspectral image to be detected, and divide the hyperspectral image into sub-images to obtain a plurality of initial sub-images; Determine an average spectrum of each of the initial sub-images, input the average spectrum of the initial sub-images into the target sub-image classifier, and judge whether the initial sub-images have defects; Taking the target subgraph classifier as the particle fitness function, updating the particle velocity and particle position; Iterative optimization is performed to obtain an updated optimal particle position, and the band represented by the optimal particle position is used as the optimal selected band; In the case where the initial sub-graph has defects, determining the optimal characteristic band of the initial sub-graph to obtain a target sub-graph, and the best selected band is used as a basis for determining the optimal characteristic band; The target sub-image is input into a target defect detector to obtain a defect detection result of the hyperspectral image.

2. The surface defect detection method according to claim 1, characterized in that: After the step of dividing the sample spectral image into sub-images to obtain sample sub-images, the following step further comprises: Determine the overlap area of ​​the sample sub-image and the defect; Calculating the ratio between the area of ​​the overlapped region and the total area of ​​the sample sub-image; When the ratio is greater than a preset ratio threshold, it is determined that the sample sub-image has a defect.

3. The surface defect detection method according to claim 1, characterized in that: After the step of dividing the sample spectral image into sub-images to obtain sample sub-images, the following step further comprises: Search the defect database according to the sample sub-graph to obtain the sub-graph defect coordinates; Determine the optimal characteristic band of the sample sub-image to obtain a fused sample sub-image; The fused sample sub-image is input into an initial defect detector, and the initial defect detector is trained based on the sub-image defect coordinates to obtain the target defect detector.

4. The surface defect detection method according to claim 1, characterized in that: The step of inputting the target sub-image into the target defect detector to obtain the defect detection result of the hyperspectral image comprises: Inputting the target sub-image into the target defect detector to extract multi-scale feature information of the target sub-image; Based on the multi-scale feature information, detecting and obtaining defect categories and defect locations in the target sub-image; The defect position is mapped to the hyperspectral image to obtain the three-dimensional coordinates of the defect.

5. The surface defect detection method according to any one of claims 1 to 4, characterized in that: Before the step of acquiring the hyperspectral image to be detected and dividing the hyperspectral image into sub-images to obtain a plurality of initial sub-images, the method further includes: Acquiring an initial spectral reflection signal; The initial spectral reflection signal is subjected to a black and white standard plate correction process to obtain a corrected target spectral reflection signal.

6. A surface defect detection device, characterized in that: The surface defect detection device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the surface defect detection method according to any one of claims 1 to 5.

7. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the surface defect detection method according to any one of claims 1 to 5 are implemented.

Images