Image defect detection method, detection device, detection equipment and storage medium

Abstract

Embodiments of the present application relate to an image defect detection method, a detection device, a detection equipment and a storage medium, comprising: acquiring an initial image on an object, and calling a target image corresponding to the type of the initial image; extracting a target feature point set in the target image, and extracting an initial feature point set in the initial image; matching the initial feature point set and the target feature point set to obtain a first image corresponding to the initial image; performing convolution network and deep learning processing on the first image to obtain a second image; and comparing the second image with the target image to obtain a detection result of the initial image. Thus, the defect detection of the image can be completed, thereby achieving the effect of improving the defect detection precision.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of image processing, and particularly relate to an image defect detection method, a detection device, a detection equipment and a storage medium. BACKGROUND

[0002] With the continuous development of artificial intelligence technology, pattern recognition technology, AI technology and image processing technology are widely used in production and life. In the field of image processing technology, defect detection technology is increasingly valued by enterprises. Among them, the importance of trademarks to enterprise brands is increasingly prominent, and the correctness and beauty of trademarks determine the quality of products. The defect detection result of the trademark directly affects the production efficiency and production cost of the product.

[0003] At present, most of the defect detection of products is realized by manual re-trial and manual framing modeling. However, manual re-trial may have errors and missed detection, and manual modeling method is affected by human factors, which may cause the problem of reducing defect detection accuracy. SUMMARY

[0004] In view of this, in order to solve the above technical problem of low defect detection accuracy, embodiments of the present application provide an image defect detection method, a detection device, a detection equipment and a storage medium.

[0005] In a first aspect, the embodiments of the present application provide an image defect detection method, comprising:

[0006] obtaining an initial image on an object, and calling a target image corresponding to the type of the initial image;

[0007] extracting a target feature point set in the target image, and extracting an initial feature point set in the initial image;

[0008] matching the initial feature point set and the target feature point set to obtain a first image corresponding to the initial image;

[0009] performing convolution network and deep learning processing on the first image to obtain a second image;

[0010] comparing the second image with the target image to obtain a detection result of the initial image.

[0011] In one possible implementation, before the calling of the target image corresponding to the type of the initial image, the method further comprises:

[0012] determining a region of interest on the object, and performing binary processing on the image of the region of interest;

[0013] determining contour information of a corresponding maximum contour from the image after binarization processing;

[0014] determining a minimum circumscribed rectangle of the contour information, and taking a region corresponding to the minimum circumscribed rectangle as a target image;

[0015] determining a type of the target image according to the contour information;

[0016] storing the target image according to the type.

[0017] In one possible implementation, the extracting a target feature point set in the target image and extracting an initial feature point set in the initial image comprises:

[0018] obtaining first target edge information of the target image and first initial edge information of the initial image;

[0019] obtaining second target edge information of the first target edge information in a non-linear scale space and second initial edge information of the first initial edge information in the non-linear scale space;

[0020] obtaining a plurality of first target feature points at a specified position from the second target edge information and a plurality of first initial feature points at the specified position from the second initial edge information;

[0021] obtaining a plurality of second target feature points corresponding to the plurality of first target feature points and a plurality of second initial feature points corresponding to the plurality of first initial feature points through descriptor processing;

[0022] obtaining a target feature point set of the target image through metric and registration processing of the plurality of second target feature points and obtaining an initial feature point set of the initial image through metric and registration processing of the plurality of second initial feature points.

[0023] In one possible implementation, the performing convolution network and deep learning processing on the first image to obtain a second image comprises:

[0024] obtaining a first feature point set corresponding to the first image through convolution network processing on the first image and the target image;

[0025] obtaining a first group of loss value information corresponding to the first feature point set through affine transformation and displacement field processing on the first feature point set;

[0026] determining a second group of loss value information of the first feature point set through refined displacement field and deep learning processing;

[0027] The first set of loss value information and the second set of loss value information are compared to obtain a comparison result;

[0028] According to the comparison result, a second image corresponding to the first image is obtained.

[0029] In one possible implementation, the matching of the initial feature point set and the target feature point set to obtain the first image corresponding to the initial image includes:

[0030] The initial image and the target image are subjected to a random sample consensus algorithm matching process to obtain a preset feature point set and preset transformation matrix information corresponding to the initial image;

[0031] The preset feature point set is subjected to affine transformation processing by using the preset transformation matrix information to obtain the first image corresponding to the initial image.

[0032] In one possible implementation, the comparison of the second image and the target image to obtain a detection result of the initial image includes:

[0033] The second image and the target image are compared to obtain a defect area corresponding to the initial image;

[0034] According to the defect area, a detection result of the initial image is determined.

[0035] In one possible implementation, the comparison of the second image and the target image to obtain a defect area corresponding to the initial image includes:

[0036] According to the target image, the second image is subjected to a defect detection algorithm process to obtain a third feature point set;

[0037] A region corresponding to the third feature point set is taken as the defect area corresponding to the initial image.

[0038] In one possible implementation, the determination of the detection result of the initial image according to the defect area includes:

[0039] When the defect area is greater than or equal to an area threshold, the initial image is determined to be a defect image;

[0040] When the defect area is less than the area threshold, the initial image is determined to be a non-defect image.

[0041] In a second aspect, an embodiment of the present application provides a detection device, which includes:

[0042] An acquisition module is configured to acquire an initial image on an object and call a target image corresponding to a type of the initial image;

[0043] An extraction module is configured to extract a set of target feature points in the target image and extract a set of initial feature points in the initial image;

[0044] A matching module is configured to match the set of initial feature points and the set of target feature points to obtain a first image corresponding to the initial image;

[0045] A learning module is configured to perform convolution network and deep learning processing on the first image to obtain a second image;

[0046] A comparison module is configured to compare the second image and the target image to obtain a detection result of the initial image.

[0047] In a third aspect, an embodiment of the present application provides a detection device, including a processor and a memory, the processor is configured to execute an image defect detection program stored in the memory to implement the image defect detection method in any of the first aspect.

[0048] In a fourth aspect, an embodiment of the present application provides a storage medium, the storage medium stores one or more programs, the one or more programs can be executed by one or more processors to implement the image defect detection method in any of the first aspect.

[0049] The image defect detection method, the detection device, the detection equipment and the storage medium provided by the embodiment of the present application can obtain the reference image and the image to be detected by acquiring the initial image on the object and calling the target image corresponding to the type of the initial image. The image feature points corresponding to the target image and the initial image are obtained by extracting the set of target feature points in the target image and extracting the set of initial feature points in the initial image, so as to provide data for subsequent image comparison. The first image corresponding to the initial image is obtained by matching the set of initial feature points and the set of target feature points, and the first image is used to represent the preliminary processing image obtained by defect detection. The second image is obtained by performing convolution network and deep learning processing on the first image, and the second image has more accurate feature information. The detection result of the initial image is obtained by comparing the second image and the target image. The defect detection result is more accurate by adding the second image with more accurate feature information on the basis of the first image. The defect detection of the image can be completed by the present scheme, so that the effect of improving the defect detection precision is realized. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1A flowchart of an image defect detection method provided by an embodiment of the present application is shown in the figure.

[0051] Figure 2 A flowchart of another image defect detection method provided by an embodiment of the present application is shown in the figure.

[0052] Figure 3 A flowchart of still another image defect detection method provided by an embodiment of the present application is shown in the figure.

[0053] Figure 4 A structural diagram of a detection device provided by an embodiment of the present application is shown in the figure.

[0054] Figure 5 A structural diagram of a detection device provided by an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0055] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in connection with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0056] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in connection with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0057] Figure 1 A flowchart of an image defect detection method provided by an embodiment of the present application is shown in the figure. The embodiment of the present application is used to illustrate the specific implementation of the image defect detection method, but the specific implementation steps only represent one possible implementation, not all the implementations.

[0058] Defect detection generally refers to the detection of surface defects of an object. Surface defect detection is to detect defects such as spots, pits, scratches, color difference and defects on the surface of a workpiece by using advanced machine vision detection technology.

[0059] According to the provided figure, the image defect detection method provided by an embodiment of the present application comprises: Figure 1 According to the provided figure, the image defect detection method provided by an embodiment of the present application comprises:

[0060] S101, acquiring an initial image on an object, and calling a target image corresponding to the type of the initial image.

[0061] The image defect detection method provided by the application is applied to product defect detection. The object can be understood as a product produced, the initial image can be understood as a picture first collected from the product, and the image to be detected is an injection mark. The target image can be understood as a reference template image corresponding to the image to be detected, so as to facilitate comparison of the target image to determine the defect condition of the initial image.

[0062] Further, different types of target images are stored in the database corresponding to the detection device in advance according to different types. The initial image on the object is collected by the image collector, and the target image corresponding to the type of the initial image is called by the detection device. Thus, the initial image and the target image can be obtained.

[0063] S102, a target feature point set in the target image is extracted, and an initial feature point set in the initial image is extracted.

[0064] The feature index of the target image can include the target feature point set, which is used to detect the feature information of the target image. At the same time, the feature index of the initial image can be represented by the initial feature point set, which is used to detect the feature information of the initial image.

[0065] Further, the feature point information of the initial image is extracted by a related algorithm, and the plurality of feature point information corresponding to the initial image is combined into the initial feature point set, which is used to represent the image features of the initial image. Similarly, the feature point information of the target image is extracted by a related technical means, and the plurality of feature point information obtained is combined into the target feature point set corresponding to the target image, and stored in the database of the detection device, which is used to provide reference comparison of the corresponding feature information for the image to be detected.

[0066] Optionally, before the target feature point set and the initial feature point set are extracted, the target image and the initial image are preprocessed to obtain a target gray image corresponding to the target image and an initial gray image corresponding to the initial image, so as to provide clear image information for feature extraction of the target image and the initial image.

[0067] S103, the initial feature point set and the target feature point set are matched to obtain a first image corresponding to the initial image.

[0068] The first image can be understood as a coarse registration image obtained after matching, which is used to represent the feature matching effect of the initial image and the target image.

[0069] Further, after obtaining the target feature point set corresponding to the target image and the initial feature point set corresponding to the initial image, the corresponding feature point information in the target feature point set and the initial feature point set is matched and analyzed to obtain the first image after matching. The first image carries the image information of the preliminary registration, so that the corresponding relationship between the feature points of the initial image and the feature points of the target image is established.

[0070] S104, the first image is processed by a convolutional network and deep learning to obtain a second image.

[0071] The convolutional network is a processing method in image processing technology, which is used to obtain a robust feature point set. Deep learning can be understood as a model training, which is used to realize the continuous identification and improvement of image features. The second image can be understood as a fine registration image obtained by processing the first image by the convolutional network and deep learning.

[0072] Further, through the deep learning and convolutional network processing of the first image, the matching with the target image is continuously performed, and the model is formed on this basis. Through the training of the model, the feature point set of the initial image is improved, and the second image corresponding to the first image is obtained.

[0073] S105, the second image and the target image are compared to obtain a detection result of the initial image.

[0074] The detection result can be understood as a standard for judging the image defect detection.

[0075] Further, on the basis of obtaining the fine registration second image, the second image carrying the initial feature point set is compared with the target image carrying the target feature point set. The comparison is the information comparison between the feature points, and an comparison result is obtained. Through the comparison result, the defect detection result of the initial image can be obtained, and thus the defect detection of the image is completed, thereby realizing the effect of improving the defect detection precision and reducing the cost.

[0076] The embodiment of the present application provides an image defect detection method, which comprises the following steps: acquiring an initial image on an object, and calling a target image corresponding to the type of the initial image; extracting a target feature point set in the target image, and extracting an initial feature point set in the initial image; obtaining image feature points corresponding to the target image and the initial image, so as to provide data for subsequent image comparison; matching the initial feature point set and the target feature point set to obtain a first image; the first image is used to represent a preliminary processing image obtained through defect detection; performing convolution network and deep learning processing on the first image to obtain a second image; the second image with more accurate feature information is extracted on the basis of the first image; and the second image and the target image are compared to obtain a detection result of the initial image. Through the first image and the second image with more accurate feature information, the defect detection result is more accurate. Thus, the image defect detection can be completed, and the defect detection precision is improved.

[0077] Figure 2 Another flowchart of an image defect detection method is provided in the embodiment of the present application. The method is introduced on the basis of the above-mentioned embodiment. According to the content shown in the figure, the image defect detection method specifically comprises the following steps. Figure 2

[0078] S201, determine a region of interest on an object, and perform binaryzation processing on the image of the region of interest.

[0079] The image defect detection method provided by the embodiment of the present application is applied to trademark defect detection. The trademark can be understood as image information pasted on a product, or image information injection molded on a specific position of the product. The qualified rate and production efficiency of the product can be shown by detecting the defect condition of the trademark.

[0080] The object can be understood as a product with pasted or injection molded trademark information. The region of interest (ROI) can be understood as a region selected by software, which is used to ensure that the trademark is in the ROI of the image region. The binaryzation of the image can obtain a corresponding black and white image.

[0081] Further, the position of the trademark on the product is processed by software to obtain an image in the ROI region. The binaryzation processing is performed on the image in the ROI region to obtain a black and white image corresponding to the image in the ROI region.

[0082] Since there is a color difference between the product background and the trademark, the binaryzation processing is performed on the image, a global threshold segmentation method is used, and then corrosion is used to remove redundant noise points. The specific formula is as follows:

[0083] ​

[0084] wherein thresh represents a threshold value, I is an input image, r is a row number, c is a column number, and O is an output image.

[0085] Optionally, the image in the ROI region is pre-processed to obtain a corresponding gray image before obtaining the binary image.

[0086] Further, the color image is collected by an industrial camera, an image sensor or the like, and the color image is divided into R, G and B three primary colors to obtain a corresponding gray image by weighted average with different weights. According to the fact that the human eye is most sensitive to green and least sensitive to blue, a gray image is obtained by a psychological gray formula:

[0087] gray = 0.30 * R + 0.59 * G + 0.11 * B

[0088] wherein gray represents a pixel point of the gray image, R represents red, G represents green, and B represents blue.

[0089] S202, determining the contour information of the corresponding maximum contour from the binary processed image.

[0090] S203, determining the minimum circumscribed rectangle of the contour information, and taking the region corresponding to the minimum circumscribed rectangle as a target image.

[0091] wherein the maximum contour can be understood as the outer edge information of the binary image, and the minimum circumscribed rectangle of the image can be understood as the minimum range containing the position of the image.

[0092] Further, the point set of the contour is composed according to the edge of the binary image. The point set of the maximum contour of the image is found through the point set of the contour, so as to determine the position of the trademark.

[0093] Further, the minimum circumscribed rectangle corresponding to the image is determined according to the point set of the maximum contour of the image. The minimum side length corresponding to the minimum circumscribed rectangle of the image is obtained, and the core of the square is composed by one tenth of the length of the minimum side length. The image is dilated to prevent the loss of the trademark image. The image is cropped according to the region corresponding to the minimum circumscribed rectangle of the image to obtain a target image.

[0094] S204, determining the type of the target image according to the contour information. The target image is stored according to the type.

[0095] Further, according to different profile information obtained from the target image, the target image is classified into different categories. According to different categories, the target image is stored in the database of the detection device. When different products are detected, different target images can be automatically obtained according to different categories due to different detection positions. The detection efficiency of the product can be improved.

[0096] S205, obtaining an initial image on an object, and calling a target image corresponding to a type of the initial image.

[0097] Further, the category of the target image is determined according to the category of the initial image. The corresponding target image in the database is obtained by calling the initial image to be detected and the target image.

[0098] Optionally, after obtaining the target image and the initial image, the target image and the initial image are preprocessed. According to R, G, B three primary colors and human sensitivity to color, a target gray image and an initial gray image are obtained.

[0099] S206, obtaining first target edge information of the target image and first initial edge information of the initial image.

[0100] The first target edge information can be understood as the information of the edge image corresponding to the target image obtained by edge detection. The first initial edge information can be understood as the information of the edge image of the initial image obtained by edge detection.

[0101] In a possible example scenario, the target image and the initial image obtained after gray processing are subjected to feature extraction by using an improved feature extraction algorithm (Accelerated KAZE, AKAZE). The first target edge information and the first initial edge information are obtained by using a morphological gradient method to perform image edge enhancement processing on the target image and the initial image.

[0102] S207, obtaining second target edge information of the first target edge information in a nonlinear scale space and second initial edge information of the first initial edge information in the nonlinear scale space.

[0103] The second target edge information can be understood as being obtained by feature extraction based on the first target edge information. Similarly, the second initial edge information can be understood as being obtained by feature extraction based on the first initial edge information.

[0104] Further, the first target edge information is subjected to edge information extraction in the nonlinear scale space to obtain corresponding second target edge information. Similarly, the first initial edge information is subjected to edge information extraction to obtain corresponding second initial edge information.

[0105] In a possible example scenario, a nonlinear scale space is constructed by a nonlinear filtering function and a fast display diffusion algorithm. First target edge information is processed in the constructed nonlinear scale space to obtain corresponding second target edge information. And second initial edge information corresponding to the first initial edge image.

[0106] S208, a plurality of first target feature points at a specified position are obtained from the second target edge information, and a plurality of first initial feature points at the specified position are obtained from the second initial edge information.

[0107] The second target edge information is obtained by feature extraction to obtain the first target feature points. The first target feature points are used to represent the feature information of the target image. Similarly, the first initial feature points are obtained by feature extraction on the second initial edge information. The first initial feature points are used to represent the feature information of the initial image. The specified position can be understood as the position of the feature point information to be extracted in the target image and the initial image.

[0108] Further, in the nonlinear scale space, the second target edge information is subjected to feature extraction at the specified position to obtain a plurality of first target feature points corresponding to the target image. And the second initial edge information is subjected to feature extraction at the specified position to obtain a plurality of first initial feature points corresponding to the initial image.

[0109] In a possible example scenario, the Hessian matrix algorithm is used to obtain extreme points of local positions of the initial image and the target image through different scale normalization processing, which provides important data for feature point extraction of the image.

[0110] S209, through descriptor processing, a plurality of second target feature points corresponding to the plurality of first target feature points and a plurality of second initial feature points corresponding to the plurality of first initial feature points are obtained.

[0111] The descriptor can be understood as obtaining the feature information around the feature point, including position, scale and direction. So that it is not affected by the external environment (such as: light change, angle change, etc.), so as to improve the probability of correct matching of the feature point and increase the robustness of the image. Robustness refers to the strong algorithm, which is often used to describe the ability to face complex adaptive systems, mainly reflected in the learning model with high precision or effectiveness in machine learning, and has little influence on feature point configuration.

[0112] Further, the plurality of first target feature points at the specified position are subjected to descriptor processing to obtain a plurality of second target feature points. Similarly, the plurality of first initial feature points at the specified position are subjected to descriptor processing to obtain a plurality of second initial feature points.

[0113] In a possible implementation, M-SURF descriptors in a Speeded Up Robust Features (SURF) algorithm are used to describe the feature point information, so that more robust image feature point information is obtained, and the robustness of image matching is increased. SURF is a method for obtaining features with adjacent pixels by using a square, taking the center as a key point, and taking angle rotation as a main direction.

[0114] The M-SURF descriptor formula is as follows:

[0115] d v =(∑L x ,∑L y ,∑|L x |,∑|L y |)

[0116] wherein Lx and Ly are first-order derivatives of different sub-bands, v represents a feature point vector, x represents an x-axis direction, y represents a y-axis direction, L represents a length, and d represents a distance.

[0117] S210, the plurality of second target feature points are measured and registered to obtain a target feature point set of the target image, and the plurality of second initial feature points are measured and registered to obtain an initial feature point set of the initial image.

[0118] The measurement and registration can be understood as an image matching method, which is used to measure the similarity feature points between the initial image and the target image.

[0119] Further, the plurality of second initial feature points are obtained by the descriptor, and the measurement and registration are used to process, so as to obtain the initial feature point set corresponding to the initial image. Similarly, the measurement and registration are used to process, so as to obtain the target feature point set corresponding to the target image, and the target feature point set is stored in the database of the detection device.

[0120] In a possible implementation, the Hamming distance is used as a similarity measurement of image feature matching, the nearest neighbor matching of the feature points is performed, the best feature point information of the image is obtained, and thus the target feature point set and the initial feature point set are obtained. The Hamming formula is as follows:

[0121]

[0122] wherein i represents a code number, and x and y are two code words.

[0123] S211, the initial feature point set and the target feature point set are matched to obtain a first image corresponding to the initial image.

[0124] Further, according to the feature point extraction of the target image and the feature point extraction of the initial image, the obtained initial feature point set and the target feature point set are matched to obtain the first image.

[0125] Optionally, the first image can be but is not limited to a coarse registration image. Through the feature point matching of the target image and the initial image, a more robust coarse registration image corresponding to the feature point information is obtained.

[0126] S212, the first image is processed by a convolutional network and deep learning to obtain a second image.

[0127] Further, according to the Convolutional Neural Network (CNN) algorithm, a multi-scale feature point is generated to obtain a more matched feature point set, and then through deep learning, a model is formed through continuous model training. Through autonomous learning of the model, a set of network model parameters is obtained, and the second image corresponding to the initial image is formed through the network model parameters.

[0128] In a possible implementation, the feature point set of the first image is obtained through the CNN algorithm. Through continuous model training under the processing of deep learning, a model is formed. Through autonomous learning, a fine registration image under unsupervised registration is obtained, and the precision of trademark defect detection is improved and the detection time is shortened.

[0129] S213, according to the target image, the second image is processed by a defect detection algorithm to obtain a third feature point set.

[0130] The defect detection algorithm can be understood as a comparison method of a difference image, which is used to obtain a difference image corresponding to the second image.

[0131] Further, the second image is processed by the defect detection algorithm. The target image and the second image are compared to obtain the third feature point set.

[0132] In a possible implementation, the absolute difference method is used for the second image to obtain a difference image. The difference method formula is as follows:

[0133] D (r,c) = |I-M|

[0134] Wherein, I represents the second image of fine registration, M represents the target image, D represents the difference image, r represents the number of rows, and c represents the number of columns.

[0135] The 10*10 image of the difference value image edge is subjected to edge graph mean processing, and then the mean value is taken as a threshold to perform binaryzation processing on the difference value graph. Through edge extraction processing, the edge information of the target image is obtained, and then the difference value graph is subjected to image overlay to remove artifacts to obtain a defect detection difference value graph.

[0136] S214, the area corresponding to the third feature point set is taken as the defect area of the initial image.

[0137] Further, the area of the region where the third feature point set is located is subjected to edge processing, and the obtained area is taken as the defect area of the initial image.

[0138] Alternatively, the area of the region where the third feature point set is located can be, but is not limited to, the planar area in the region, or the sum of the number of pixel points in the region, all of which can be used as a way to obtain the defect area, and all of which belong to the protection scope of the present application.

[0139] In one possible implementation, the obtained defect detection difference value graph is subjected to erosion processing to remove isolated pixel points, and a Blob algorithm is used to process the image to obtain information such as the number, position, and area of the defects. The Sobel algorithm formula is as follows:

[0140]

[0141] Lateral:

[0142] G x =(x3+2x6+x9)-(x1+2x3+x5)

[0143] Longitudinal:

[0144] G y =(x7+2x8+x9)-(x1+2x2+x3)

[0145] G=|G x |+|G y |

[0146] Wherein, Gx represents the lateral gradient value, Gy represents the longitudinal gradient value, x represents the horizontal coordinate, y represents the vertical coordinate, and G represents the defect area.

[0147] S215, when the defect area is greater than or equal to the area threshold, the initial image is determined to be a defect image.

[0148] Wherein, the area threshold can be understood as a reference area set in advance, which is used to judge the detection result of the initial image.

[0149] Further, when the obtained initial image corresponds to a defect area greater than or equal to the area threshold, it can be determined that the initial image is a defect image due to the defect area being too large.

[0150] In a possible implementation, the area threshold is set to 5 cm 2 When it is detected that the initial image corresponds to a defect area of 8 cm 2 , since 8>5, it can be determined that the initial image is a defect image.

[0151] S216, when the defect area is less than the area threshold, determining that the initial image is a non-defect image.

[0152] Further, when the obtained initial image corresponds to a defect area less than the area threshold, it can be determined that the initial image is a non-defect image.

[0153] In a possible implementation, the area threshold is set to 10 cm 2 When it is detected that the initial image corresponds to a defect area of 8 cm 2 , since 8<10, it can be determined that the initial image is a non-defect image.

[0154] The image defect detection method provided by the embodiment of the present application stores the target image into the database by automatically acquiring the target image. The initial image on the object is acquired, and the target image corresponding to the type of the initial image is called. The target feature point set and the initial feature point set are obtained by performing feature extraction on the target image and the initial image. The first image is obtained by matching processing, and the second image is obtained by convolution network and deep learning processing. The defect area corresponding to the initial image is obtained by comparing the target image and the second image. The detection result of the initial image is determined by using the area threshold. The second image with more accurate feature information is added on the basis of the first image, so that the defect detection result is more accurate. Thus, the image defect detection is completed, and the effect of improving the defect detection accuracy is achieved.

[0155] Figure 3 The flowchart of another image defect detection method provided by the embodiment of the present application is shown. According to Figure 3 the provided diagram, the specific steps of the image defect detection method include:

[0156] S301, acquiring an initial image on an object, and calling a target image corresponding to the type of the initial image.

[0157] The image defect detection method provided by the embodiment of the present application is applied to the defect detection of a trademark. The trademark can be an image pasted on the surface of a product, or an image obtained by injection molding.

[0158] Further, different types of target images are stored in the database corresponding to the detection device in advance by different types. An initial image on the object is collected by the image collector, and the target image corresponding to the type of the initial image is called by the detection device. Thus, the initial image and the target image can be obtained.

[0159] S302, a target feature point set in the target image is extracted, and an initial feature point set in the initial image is extracted.

[0160] Further, the feature point information of the initial image is extracted to obtain the initial feature point set, which is used to represent the image features of the initial image. Similarly, the feature point information of the target image is extracted to obtain the target feature point set, and the target feature point set is stored in the database of the detection device, which is used to provide corresponding feature information for the to-be-detected image.

[0161] Optionally, before the target feature point set and the initial feature point set are extracted, the target image and the initial image are preprocessed to obtain a target gray image corresponding to the target image and an initial gray image corresponding to the initial image.

[0162] S303, the initial image and the target image are subjected to random sample consensus matching processing to obtain a preset feature point set corresponding to the initial image and preset transformation matrix information.

[0163] The random sample consensus algorithm (RANSAC) can be understood as a matching method for obtaining the feature points of the initial image. The preset feature point set can be understood as the best feature point set obtained by feature extraction. The preset transformation matrix can be understood as the feature point information used for feature registration.

[0164] Further, the initial feature point set and the target feature point set are subjected to RANSAC matching to obtain the preset feature point set and the preset transformation matrix information.

[0165] In a possible implementation, the initial image corresponds to a plurality of inlier point information obtained by the RANSAC method. The plurality of inlier point information constitutes a matching feature point set corresponding to the initial image.

[0166] S304, the preset transformation matrix information is used for affine transformation processing on the preset feature point set to obtain the first image corresponding to the initial image.

[0167] The affine transformation can be understood as an image processing method for linear transformation and translation processing of a planar image.

[0168] Further, the first image is subjected to affine transformation by presetting feature point information of the transformation matrix.

[0169] In a possible implementation, the initial image is subjected to affine transformation by feature point information in the transformation matrix to obtain a coarse registration image.

[0170] S305, a first feature point set corresponding to the first image is obtained by performing convolution network processing on the first image and the target image.

[0171] The convolution network is used to obtain a robust feature point set.

[0172] Further, the first feature point set is obtained by performing convolution network processing on the first image.

[0173] S306, a first group of loss value information corresponding to the first feature point set is obtained by performing affine transformation and displacement field processing on the first feature point set.

[0174] The displacement field can be understood as a method for optimizing feature point information, and the loss value can be understood as feature point information discarded after displacement field processing.

[0175] Further, the first image is subjected to distortion and translation by performing affine transformation and displacement field processing on the first feature point set of the first image, so that a first group of loss value information is obtained, and a more matched first image is obtained.

[0176] The second image can be understood as a fine registration image obtained by performing convolution network and deep learning processing on the first image.

[0177] Further, by performing deep learning and convolution network processing on the first image, the first image is constantly matched with the target image, and model training is constantly performed on this basis to form a model. Through autonomous learning of the model, the feature point set of the initial image is improved, a group of network model parameters are obtained, and a second image corresponding to the first image is formed by the network model parameters.

[0178] S307, a second group of loss value information of the first feature point set is determined by refining the displacement field and deep learning processing.

[0179] The refined displacement field can be understood as extracting more matched feature point information on the basis of the displacement field. The deep learning can be understood as obtaining more feature point information of the initial image by establishing a model and constantly training.

[0180] Further, by refining the displacement field and deep learning, the number of inliers in model training is increased, and by dynamically increasing the inliers, more feature points are allowed to cause transformation, to determine the transformation of the overall image, optimize other feature points, and further improve the registration accuracy.

[0181] S308, comparing the first set of loss value information and the second set of loss value information to obtain a comparison result.

[0182] S309, obtaining the second image corresponding to the first image according to the comparison result.

[0183] Further, by comparing the first loss value information and the second loss value information, the minimum loss value information is obtained. According to the minimum loss value information, the second image of fine registration can be obtained.

[0184] S310, comparing the second image and the target image to obtain a defect area corresponding to the initial image.

[0185] Further, by comparing the feature point set in the target image with the feature point set corresponding to the fine registration of the second image, a difference value image after comparison can be obtained. By calculating the area of the difference value image, the defect area corresponding to the initial image is obtained.

[0186] S311, determining the detection result of the initial image according to the defect area.

[0187] Further, according to the size of the defect area, the defect detection result of the initial image can be determined.

[0188] The image defect detection method provided by the embodiment of the application comprises the following steps: obtaining an initial image on an object, and calling a target image corresponding to the type of the initial image; extracting a target feature point set in the target image, and extracting an initial feature point set in the initial image. By extracting features from the target image and the initial image and performing matching processing, a first image of coarse registration is obtained. By convolution network and deep learning processing, a second image of fine registration is obtained. By comparing the second image with the target image, a defect area is obtained. By analyzing the defect area, the defect detection result of the initial image is determined. By increasing the second image with more accurate feature information on the basis of the first image, the defect detection result is more accurate, the image defect detection is completed, and the effect of improving the trademark defect detection precision is realized.

[0189] Figure 4 The structure diagram of the detection device provided by the embodiment of the application is shown in the figure. Figure 4 According to the figure provided, the detection device comprises:

[0190] The acquisition module 41 is configured to acquire an initial image on an object, and call a target image corresponding to a type of the initial image.

[0191] The extraction module 42 is configured to extract a target feature point set in the target image, and extract an initial feature point set in the initial image.

[0192] The matching module 43 is configured to match the initial feature point set and the target feature point set to obtain a first image corresponding to the initial image.

[0193] The learning module 44 is configured to perform convolution network and deep learning processing on the first image to obtain a second image.

[0194] The comparison module 45 is configured to compare the second image with the target image to obtain a detection result of the initial image.

[0195] The detection device provided in the embodiment can be the detection device shown in the Figure 4 , and can perform all steps of the image defect detection method in the Figures 1-3 , and thus realize the technical effects of the image defect detection method shown in the Figures 1-3 . For details, refer to the related description, which will not be repeated here for brevity. Figures 1-3

[0196] Figure 5 The detection device provided in the embodiment is a structure schematic diagram of a detection device, Figure 5 The detection device 500 shown in the embodiment includes at least one processor 501, a memory 502, at least one network interface 504 and other user interfaces 503. Each component in the detection device 500 is coupled together through a bus system 505. It can be understood that the bus system 505 is used to realize the connection and communication between the components. The bus system 505 includes not only a data bus, but also a power bus, a control bus and a status signal bus. However, for the purpose of clear illustration, all kinds of buses are marked as the bus system 505 in the Figure 5 .

[0197] The user interface 503 can include a display, a keyboard or a clicking device (for example, a mouse, a trackball, a touchpad or a touch screen, etc.).

[0198] ​It is to be appreciated that the memory 502 in the embodiments of the present application can be a volatile memory or a nonvolatile memory, or can include both volatile and nonvolatile memory. Among them, the nonvolatile memory can be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a Random Access Memory (RAM) used as an external cache. By way of example, and not limitation, many forms of RAM can be used, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), Synch link DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The memory 502 described herein is intended to include, without being limited to, these and any other suitable types of memory.

[0199] In some embodiments, the memory 502 stores the following elements, executable units or data structures, or a subset of them, or an extended set of them: an operating system 5021 and an application program 5022.

[0200] Among them, the operating system 5021 contains various system programs, such as framework layer, core library layer, driver layer, etc., for implementing various basic services and processing hardware-based tasks. The application program 5022 contains various application programs, such as Media Player, Browser, etc., for implementing various application services. The program for implementing the method of the embodiments of the present application can be contained in the application program 5022.

[0201] In the embodiments of the present application, by calling the program or instruction stored in the memory 502, specifically, the program or instruction stored in the application program 5022, the processor 501 is used to execute the method steps provided by each method embodiment, for example, including:

[0202] An initial image on an acquisition object is acquired, and a target image corresponding to a type of the initial image is called. A target feature point set in the target image is extracted, and an initial feature point set in the initial image is extracted. The initial feature point set and the target feature point set are matched to obtain a first image corresponding to the initial image. The first image is processed by a convolution network and deep learning to obtain a second image. The second image and the target image are compared to obtain a detection result of the initial image.

[0203] The method disclosed in the embodiments of the present application can be applied to the processor 501 or implemented by the processor 501. The processor 501 can be an integrated circuit chip having a signal processing capability. In the implementation process, the steps of the above method can be completed by hardware integrated logic circuits or software form instructions in the processor 501. The processor 501 can be a general processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The disclosed methods, steps and logic block diagrams in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software units in the decoding processor for execution. The software unit can be located in a random access memory, a flash memory, a read only memory, a programmable read only memory, an electrically erasable programmable memory, a register or other mature storage medium in the art. The storage medium is located in the memory 502, and the processor 501 reads the information in the memory 502, and combines the hardware to complete the steps of the above method.

[0204] It can be understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing units can be implemented within one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSP Devices, DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), general purpose processors, controllers, micro-controllers, microprocessors, other electronic units designed to perform the functions described herein, or a combination thereof.

[0205] For software implementation, the techniques described herein can be implemented with a processing unit that executes program components or modules. The software codes can be stored in memory and executed by a processor. The memory can be implemented within the processor or external to the processor.

[0206] The detection device provided by the embodiment can be a detection device as shown in Figure 5 , can perform all steps of the image defect detection method as shown in Figures 1-3 , and thus achieve the technical effects of the image defect detection method as shown in Figures 1-3 . For details, please refer to the relevant description, which will not be repeated here for brevity. Figures 1-3

[0207] The embodiment of the application further provides a storage medium (computer readable storage medium). The storage medium stores one or more programs. The storage medium can include a volatile memory, such as a random access memory; the storage medium can also include a non-volatile memory, such as a read-only memory, a flash memory, a hard disk, or a solid state disk; the storage medium can also include a combination of the above kinds of memories.

[0208] When the one or more programs stored in the storage medium can be executed by one or more processors to implement the image defect detection method described above.

[0209] The processor is configured to execute the detection program stored in the memory to implement the following steps of the image defect detection method executed on the side of the detection device:

[0210] ​An initial image on an acquisition object is acquired, and a target image corresponding to a type of the initial image is called. A target feature point set in the target image is extracted, and an initial feature point set in the initial image is extracted. The initial feature point set and the target feature point set are matched to obtain a first image corresponding to the initial image. The first image is processed by a convolution network and deep learning to obtain a second image. The second image and the target image are compared to obtain a detection result of the initial image.

[0211] Those skilled in the art should further appreciate that the units and algorithm steps of each example described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, various components and steps have been described above generally in terms of their functionality, without limitation. Whether such functionality is implemented in hardware or software depends on the particular application and design constraints imposed on the overall system. Skilled persons can implement the described functionality in varying ways for each particular application, but such implementation should not be construed to limit the scope of the present application.

[0212] The steps of methods or algorithms described in connection with the embodiments disclosed herein can be implemented in hardware, software executed by a processor, or a combination of both. The software modules can be stored in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0213] The above detailed description sets forth the purposes, technical solutions, and beneficial effects of the present application. It should be understood that the above detailed description is merely a specific implementation of the present application, and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An image defect detection method, characterized in that, include: Obtain the initial image on the object, and retrieve the target image corresponding to the type of the initial image; Extract the target feature point set from the target image, and extract the initial feature point set from the initial image; The initial feature point set and the target feature point set are matched to obtain the first image corresponding to the initial image; The method of processing the first image using convolutional networks and deep learning to obtain a second image includes: processing the first image and the target image using convolutional networks to obtain a first set of feature points corresponding to the first image; performing affine transformation and displacement field processing on the first set of feature points to obtain a first set of loss value information corresponding to the first set of feature points; determining a second set of loss value information for the first set of feature points by refining the displacement field and performing deep learning processing; comparing the first set of loss value information and the second set of loss value information to obtain a comparison result; and obtaining the second image corresponding to the first image based on the comparison result. The second image and the target image are compared to obtain the detection result of the initial image.

2. The method according to claim 1, characterized in that, Before invoking the target image corresponding to the initial image type, the method further includes: Determine the region of interest on the object, and perform binarization processing on the image of the region of interest; Determine the contour information of the corresponding maximum contour from the binarized image; Determine the minimum bounding rectangle of the contour information, and use the region corresponding to the minimum bounding rectangle as the target image; The type of the target image is determined based on the contour information; The target image is stored according to the type described.

3. The method according to claim 1, characterized in that, The step of extracting the target feature point set from the target image and extracting the initial feature point set from the initial image includes: Obtain the first target edge information of the target image and the first initial edge information of the initial image; Acquire the second target edge information of the first target edge information in a nonlinear scale space, and the second initial edge information of the first initial edge information in a nonlinear scale space; Obtain multiple first target feature points at a specified location from the second target edge information, and obtain multiple first initial feature points at a specified location from the second initial edge information; Through descriptor processing, multiple second target feature points corresponding to multiple first target feature points and multiple second initial feature points corresponding to multiple first initial feature points are obtained. The target feature point set of the target image is obtained by measuring and registering multiple second target feature points, and the initial feature point set of the initial image is obtained by measuring and registering multiple second initial feature points.

4. The method according to claim 1, characterized in that, The step of matching the initial feature point set and the target feature point set to obtain the first image corresponding to the initial image includes: Random sampling consistency matching is performed on the initial image and the target image to obtain a preset feature point set and preset transformation matrix information corresponding to the initial image; Using the preset transformation matrix information, an affine transformation is performed on the preset feature point set to obtain the first image corresponding to the initial image.

5. The method according to claim 1, characterized in that, The step of comparing the second image and the target image to obtain the detection result of the initial image includes: The second image and the target image are compared to obtain the defect area corresponding to the initial image; The detection result of the initial image is determined based on the defect area.

6. The method according to claim 5, characterized in that, The step of comparing the second image and the target image to obtain the defect area corresponding to the initial image includes: Based on the target image, the second image is processed by a defect detection algorithm to obtain a third set of feature points; The region corresponding to the third set of feature points is taken as the defect area corresponding to the initial image.

7. The method according to claim 5, characterized in that, The step of obtaining the detection result of the initial image based on the defect area includes: When the defect area is greater than or equal to the area threshold, the initial image is determined to be a defect image; When the defect area is less than the area threshold, the initial image is determined to be a non-defect image.

8. An image defect detection device, characterized in that, include: The acquisition module is used to acquire an initial image on an object and to call a target image corresponding to the type of the initial image; The extraction module is used to extract the target feature point set in the target image and the initial feature point set in the initial image; A matching module is used to match the initial feature point set and the target feature point set to obtain a first image corresponding to the initial image; The learning module is used to perform convolutional network and deep learning processing on the first image to obtain a second image, including: processing the first image and the target image through a convolutional network to obtain a first set of feature points corresponding to the first image; performing affine transformation and displacement field processing on the first set of feature points to obtain a first set of loss value information corresponding to the first set of feature points; determining a second set of loss value information for the first set of feature points by refining the displacement field and performing deep learning processing; comparing the first set of loss value information and the second set of loss value information to obtain a comparison result; and obtaining the second image corresponding to the first image based on the comparison result. The comparison module is used to compare the second image and the target image to obtain the detection result of the initial image.

9. A testing device, characterized in that, include: A processor and a memory, the processor being configured to execute an image defect detection program stored in the memory to implement the image defect detection method according to any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium stores one or more programs, which can be executed by one or more processors to implement the image defect detection method according to any one of claims 1 to 7.

Images