Imaging Detection Method and System for Micro Defects and Defective Parts

Abstract

The present invention provides an image detection method and system for detecting minute defects and defective parts. In the object image recognition step, an object image is input into an image recognition model and a recognition result is output. When the recognition result is the first image, it is determined that the recognition result does not conform to the object image. When the recognition result is the second image, the following steps are executed. In the first feature extraction step, object feature values are extracted. In the second feature extraction step, a calibration image is input into the image recognition model and calibration feature values are extracted. In the feature comparison step, the object feature values are compared based on the calibration feature values to generate a similarity probability. In the recognition result judgment step, it is judged whether the recognition result conforms to the object image based on a threshold value. When the similarity probability is greater than or equal to the threshold value, it is determined that the recognition result conforms to the object image. Thereby, the accuracy rate of the recognition result is improved.

Description

Technical Field

[0001] The present invention relates to an image processing technology, and particularly to an image detection method and system for detecting minute defects and misassembled parts. Background Art

[0002] The existing Automated Optical Inspection (AOI) machines are very sensitive in detecting minute defects and misassembled parts, which easily cause misjudgment of products and make AOI alarms more frequent. Therefore, it is still necessary for in-factory personnel to conduct secondary confirmation. With the rise of Artificial Intelligence (AI), it is hoped to apply Deep Learning to the mechanism of detecting minute defects and misassembled parts to reduce the number of times for in-factory personnel to conduct secondary confirmation. However, in the case of minute defects or diverse back patterns of components, the AI model still cannot achieve a high recognition accuracy rate. In addition, when the AI model detects a defect image or a back pattern image that has never appeared before, the AI model may provide incorrect recognition results.

[0003] In view of this, how to develop an image detection method and system that can accurately identify object images is the goal and direction that relevant industries must strive to research and break through. Summary of the Invention

[0004] Therefore, the purpose of the present invention is to provide an image detection method and system for detecting minute defects and misassembled parts, which identify object images through an image recognition model, and verify the recognition results to determine whether the recognition results conform to the object images, thereby improving the accuracy rate of the recognition results and simultaneously reducing the missed detection rate.

[0005] According to an embodiment of the present invention, an image detection method for detecting minute defects and wrong parts is provided, which includes an object image recognition step. In the object image recognition step, an arithmetic processing unit is driven to input an object image into an image recognition model, so that the image recognition model performs image recognition on the object image and outputs a recognition result. The image recognition model is established by training a plurality of first images and a plurality of second images, and the object image corresponds to one of the second images. When the recognition result is one of the first images, the arithmetic processing unit determines that the recognition result does not conform to the object image. When the recognition result is one of the second images, the arithmetic processing unit executes a recognition result verification step, and the recognition result verification step includes a first feature extraction step, a second feature extraction step, a feature comparison step, and a recognition result determination step. In the first feature extraction step, the arithmetic processing unit is driven to extract an object feature value corresponding to the object image through the image recognition model. In the second feature extraction step, the arithmetic processing unit is driven to input a calibration image into the image recognition model according to the recognition result, and extract a calibration feature value corresponding to the calibration image through the image recognition model. The feature comparison step is to drive the arithmetic processing unit to execute a feature comparison software module. The feature comparison software module performs a similarity comparison on the object feature value based on the calibration feature value to generate a similarity probability. The recognition result determination step is to drive the arithmetic processing unit to determine whether the recognition result conforms to the object image according to the similarity probability and a threshold value. When the similarity probability is greater than or equal to the threshold value, the arithmetic processing unit determines that the recognition result conforms to the object image. When the similarity probability is less than the threshold value, the arithmetic processing unit determines that the recognition result does not conform to the object image.

[0006] According to another embodiment of the present invention, an image detection system for detecting minute defects and misassembled parts is provided, which includes a storage unit and an arithmetic processing unit. The storage unit stores an image recognition model, a plurality of first images, a plurality of second images, and a feature comparison software module. The arithmetic processing unit is connected to the storage unit and is configured to perform an object image recognition step. In the object image recognition step, an object image is input into the image recognition model, so that the image recognition model performs image recognition on the object image and outputs a recognition result. The image recognition model is established by training the plurality of first images and the plurality of second images, and the object image corresponds to one of the second images. When the recognition result is one of the first images, the arithmetic processing unit determines that the recognition result does not match the object image. When the recognition result is one of the second images, the arithmetic processing unit is configured to perform a recognition result verification step, and the recognition result verification step includes a first feature extraction step, a second feature extraction step, a feature comparison step, and a recognition result determination step. In the first feature extraction step, an object feature value corresponding to the object image is extracted through the image recognition model. In the second feature extraction step, a calibration image is input into the image recognition model according to the recognition result, and a calibration feature value corresponding to the calibration image is extracted through the image recognition model. The feature comparison step is to execute the feature comparison software module. The feature comparison software module performs a similarity comparison on the object feature value based on the calibration feature value to generate a similarity probability. The recognition result determination step determines whether the recognition result matches the object image based on the similarity probability and a threshold value. When the similarity probability is greater than or equal to the threshold value, the arithmetic processing unit determines that the recognition result matches the object image. When the similarity probability is less than the threshold value, the arithmetic processing unit determines that the recognition result does not match the object image.

[0007] Thus, the image detection method and system for detecting minute defects and misassembled parts of the present invention identify the object image through the image recognition model. In the case where the recognition result is a second image, the recognition result verification step is still used to re-determine whether the recognition result matches the object image by using similarity comparison, thereby improving the accuracy of the recognition result and simultaneously reducing the missed detection rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 is a schematic flowchart of an image detection method for detecting minute defects and misassembled parts according to the first embodiment of the present invention;

[0009] Figure 2 is shown Figure 1 a schematic diagram of the recognition result verification step of the image detection method for detecting minute defects and misassembled parts;

[0010] Figure 3 is a schematic diagram showing that an object image of the present invention is input into an image recognition model to generate a recognition result corresponding to a first image or a second image;

[0011] Figure 4 It is a schematic diagram showing the similarity probability of multiple object images based on a calibration image of the present invention; and

[0012] Figure 5 It is a block schematic diagram of an image detection system for minute defects and wrong parts according to a second embodiment of the present invention.

[0013] Among them, the reference numerals are explained as follows:

[0014] 100: Image detection method for minute defects and wrong parts

[0015] 110, 110a, 110b, 110c: Object images

[0016] 111: Object feature value

[0017] 120, 211: Image recognition model

[0018] 121: Convolutional layer

[0019] 122: First fully connected layer

[0020] 123: Second fully connected layer

[0021] 124: Activation function

[0022] 125, 212: Feature comparison software module

[0023] 130: Recognition result

[0024] 140, 213: Image database

[0025] 141, 2131: First image

[0026] 142, 2132: Second image

[0027] 150: Calibration image

[0028] 151: Calibration feature value

[0029] 160, 160a, 160b, 160c: Similarity probability

[0030] 170, 214: Threshold value

[0031] 200: Image detection system for minute defects and wrong parts

[0032] 210: Storage unit

[0033] 220: Arithmetic processing unit

[0034] S01: Object image recognition step

[0035] S02: Identification Result Verification Step

[0036] S021: First Feature Extraction Step

[0037] S022: Second Feature Extraction Step

[0038] S023: Feature Comparison Step

[0039] S024: Identification Result Judgment Step Detailed Implementation Manner

[0040] Please refer to Figure 1 , Figure 2 and Figure 3 together, where Figure 1 is a schematic flowchart of an image detection method 100 for minute defects and wrong parts according to the first embodiment of the present invention; Figure 2 is a schematic diagram showing Figure 1 the identification result verification step S02 of the image detection method 100 for minute defects and wrong parts; and Figure 3 is a schematic diagram showing the identification result 130 corresponding to the first image 141 or the second image 142 generated when the object image 110 of the present invention is input into the image identification model 120. As shown in the figure, the image detection method 100 for minute defects and wrong parts includes an object image identification step S01. The object image identification step S01 is to drive the arithmetic processing unit to input the object image 110 into the image identification model 120. The image identification model 120 performs image identification on the object image 110 and outputs the identification result 130. It should be noted that the image identification model 120 is established by the arithmetic processing unit based on a deep learning algorithm by training a plurality of first images 141 and a plurality of second images 142 stored in the image database 140, and the deep learning algorithm can be a convolutional neural network (CNN), a VGG network, a lightweight neural network (Mobilenet), a deep residual network (ResNet), or an algorithm for implementing image identification, but the present invention is not limited thereto. In addition, the aforementioned plurality of first images 141 used for training the image identification model 120 are different from the plurality of second images 142, and the object image 110 corresponds to one of the second images 142. When the identification result 130 is one of the first images 141, the arithmetic processing unit determines that the identification result 130 does not conform to the object image 110.

[0041] Specifically, the multiple first images 141 of the first embodiment may respectively be back pattern images of multiple series of chips manufactured by a first manufacturer, the multiple second images 142 may respectively be back pattern images of multiple series of chips manufactured by a second manufacturer, and the object image 110 is a back pattern image of any series of chips manufactured by the second manufacturer. In other embodiments, the object image, each first image, and each second image may be images of electronic components, integrated circuits (ICs), or RLC circuits composed of resistors (R), inductors (L), and capacitors (C), and the present invention is not limited thereto.

[0042] When the first image 141 appears in the output recognition result 130, the arithmetic processing unit can determine that the recognition result 130 is incorrect. For example, in Figure 3 the back pattern of the first image 141 shows T250Vz6C, which represents the model Vz6C in the T250 series of chips manufactured by the first manufacturer. The back pattern of the second image 142 shows T250Dz5P, which represents the model Dz5P in the T250 series of chips manufactured by the second manufacturer. The back pattern of the object image 110 shows T250Dz5Q. When the recognition result 130 output by the image recognition model 120 is the first image 141, the arithmetic processing unit can directly determine that the recognition result 130 does not match the object image 110. Thus, the image detection method 100 for micro defects and wrong parts of the present invention identifies the object image 110 through the image recognition model 120, and can determine whether the recognition result 130 matches the object image 110 according to whether the recognition result 130 is the first image 141 or the second image 142, thereby improving the recognition accuracy.

[0043] In addition, when the recognition result 130 output by the image recognition model 120 is the second image 142, the operation processing unit executes the recognition result verification step S02. The recognition result verification step S02 includes a first feature extraction step S021, a second feature extraction step S022, a feature comparison step S023, and a recognition result determination step S024. The first feature extraction step S021 is to drive the operation processing unit to extract the object feature value 111 corresponding to the object image 110 through the image recognition model 120. Specifically, the image recognition model 120 may include a convolutional layer 121, a first fully connected layer 122, a second fully connected layer 123, and an activation function 124. After receiving the object image 110, the convolutional layer 121 performs a convolutional operation on the object image 110 according to the convolutional kernel to generate a feature map. Then, the first fully connected layer 122 and the second fully connected layer 123 perform classification processing and connection on the feature map using the weight matrix and the bias vector, where the operation processing unit extracts the object feature value 111 from the first fully connected layer 122. Finally, a regression operation is performed through the activation function 124 (for example: Softmax function) to generate the recognition result 130. In other embodiments, the image recognition model may further include multiple convolutional layers and multiple pooling layers. The pooling layer is sandwiched between consecutive convolutional layers and is used to compress data and parameters to reduce overfitting.

[0044] The second feature extraction step S022 is to drive the operation processing unit to input the calibration image 150 to the image recognition model 120 according to the recognition result 130, and extract the calibration feature value 151 corresponding to the calibration image 150 through the first fully connected layer 122 in the image recognition model 120. Specifically, in the second feature extraction step S022, the operation processing unit selects the second image 142 that is the same as the recognition result 130 from the multiple second images 142 according to the recognition result 130 as the calibration image 150. In short, the operation processing unit mainly finds the same image as the calibration image 150 from the second images 142 through the image shown by the recognition result 130, and then extracts the calibration feature value 151 as the input value required for comparison in the subsequent feature comparison step S023.

[0045] The feature comparison step S023 is to drive the computing processing unit to execute the feature comparison software module 125. The feature comparison software module 125 performs a similarity comparison on the object feature value 111 according to the calibration feature value 151 to generate a similarity probability 160. Specifically, in the feature comparison step S023, the feature comparison software module 125 compares the calibration feature value 151 with the object feature value 111 according to the Euclidean distance algorithm to generate the similarity probability 160. In other embodiments, the feature comparison software module may also use any one of Manhattan distance, Chebyshev distance, Minkowski distance, Mahalanobis distance and Hamming distance to determine the difference between the calibration feature value and the object feature value to generate the similarity probability.

[0046] The recognition result determination step S024 is to drive the processing unit to determine whether the recognition result 130 is consistent with the object image 110 according to the similarity probability 160 and the threshold 170. When the similarity probability 160 is greater than or equal to the threshold 170, the processing unit determines that the recognition result 130 is consistent with the object image 110. When the similarity probability 160 is less than the threshold 170, the processing unit determines that the recognition result 130 is inconsistent with the object image 110.

[0047] Please also read Figure 1 , Figure 2 and Figure 4 ,in Figure 4 1 is a schematic diagram illustrating similarity probabilities 160a, 160b, 160c of a plurality of object images 110a, 110b, 110c based on a calibration image 150 of the present invention. As shown in the figure, the processing unit inputs the object images 110a, 110b, 110c to the image recognition model 120 respectively to generate the same recognition result 130, and the processing unit selects the calibration image 150 according to the recognition result 130, and then performs similarity comparison with the object images 110a, 110b, 110c respectively to generate similarity probabilities 160a, 160b, 160c. In detail, in the recognition result judgment step S024, the processing unit compares the similarity probabilities 160 with each other using a pre-set threshold 170.

[0048] For example, the value of threshold 170 is 0.91, the value of similarity probability 160a is 0.96, the value of similarity probability 160b is 0.75, and the value of similarity probability 160c is 0.88. Since similarity probability 160a is greater than threshold 170 (0.96 > 0.91), the arithmetic processing unit determines that the recognition result 130 corresponding to object image 110a conforms to object image 110a; in other words, the recognition result 130 corresponding to object image 110a has high accuracy. On the other hand, as Figure 4 can be seen, the back pattern of calibration image 150 (i.e., one of the second images 142) shows T250Dz5P. The back pattern of object image 110b shows T250Dz5Q, and the value of its similarity probability 160b is less than threshold 170 (0.75 < 0.91), so the arithmetic processing unit determines that the recognition result 130 corresponding to object image 110b does not conform to object image 110b; obviously, object image 110b is a wrong part with respect to calibration image 150. Similarly, the back pattern of object image 110c shows T250Dz5P, and the value of its similarity probability 160c is less than threshold 170 (0.88 < 0.91), so the arithmetic processing unit determines that the recognition result 130 corresponding to object image 110c does not conform to object image 110c; obviously, pattern P in object image 110c (i.e., the pattern after the lower half of the English letter P is cut off) is a minor defect with respect to calibration image 150. Thus, the image detection method 100 for minor defects and wrong parts of the present invention still uses similarity comparison in the recognition result inspection step S02 to re-determine whether the recognition result 130 conforms to object images 110b and 110c for the wrong object image 110b and the object image 110c with minor defects, thereby reducing incorrect component recognition and reducing the missed detection rate for minor defects and wrong parts.

[0049] Please refer to Figure 1 and Figure 5 , where Figure 5FIG. 0 is a block diagram showing an image detection system 200 for detecting minute defects and misaligned parts according to a second embodiment of the present invention. The image detection system 200 for detecting minute defects and misaligned parts is configured to identify an object image 110 and detect the identification result 130 to determine whether the identification result 130 conforms to the object image 110. As shown in the figure, the image detection system 200 for detecting minute defects and misaligned parts includes a storage unit 210 and an arithmetic processing unit 220. The storage unit 210 stores an image recognition model 211, a feature comparison software module 212, an image database 213, and a threshold value 214, where the image database 213 may include a plurality of first images 2131 and a plurality of second images 2132. The arithmetic processing unit 220 is electrically connected to the storage unit 210 and is configured to implement an image detection method 100 for detecting minute defects and misaligned parts. The arithmetic processing unit 220 may be a Digital Signal Processor (DSP), a Micro Processing Unit (MPU), a Central Processing Unit (CPU), or other electronic processors, but the present invention is not limited thereto.

[0050] Although the present invention has been disclosed above in the form of embodiments, it is not intended to limit the present invention. Any person skilled in the art can make various modifications and refinements without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be determined by the appended claims.

Claims

1. An image detection method for minute defects and wrong parts, characterized in that, It includes the following steps: An object image recognition step, which drives an arithmetic processing unit to input an object image into an image recognition model, so that the image recognition model performs image recognition on the object image and outputs a recognition result. The image recognition model is established by training a plurality of first images and a plurality of second images, and the object image corresponds to one of the second images; Wherein, when the recognition result is one of the first images, the arithmetic processing unit determines that the recognition result does not match the object image; Wherein, when the recognition result is one of the second images, the arithmetic processing unit executes a recognition result verification step, and the recognition result verification step includes: A first feature extraction step, which drives the arithmetic processing unit to extract an object feature value corresponding to the object image through the image recognition model; A second feature extraction step, which drives the arithmetic processing unit to input a calibration image into the image recognition model according to the recognition result, and extract a calibration feature value corresponding to the calibration image through the image recognition model; A feature comparison step, which drives the arithmetic processing unit to execute a feature comparison software module, wherein the feature comparison software module compares the object feature value with the calibration feature value according to the calibration feature value to generate a similarity probability; and A recognition result judgment step, which drives the arithmetic processing unit to judge whether the recognition result matches the object image according to the similarity probability and a threshold value; Wherein, when the similarity probability is greater than or equal to the threshold value, the arithmetic processing unit determines that the recognition result matches the object image; Wherein, when the similarity probability is less than the threshold value, the arithmetic processing unit determines that the recognition result does not match the object image.

2. The image detection method for minute defects and wrong parts as described in claim 1, characterized in that, The arithmetic processing unit establishes the image recognition model by training the plurality of first images and the plurality of second images according to a deep learning algorithm, and the plurality of first images are different from the plurality of second images.

3. The image detection method for minute defects and wrong parts according to claim 2, characterized in that, The image recognition model includes at least one convolutional layer, a first fully connected layer, a second fully connected layer and an activation function, and the arithmetic processing unit extracts the object feature value and the calibration feature value from the first fully connected layer.

4. The image detection method for minute defects and wrong parts as described in claim 1, characterized in that, In the second feature extraction step, the arithmetic processing unit selects one of the second images as the calibration image from the plurality of second images according to the recognition result.

5. The image detection method for minute defects and wrong parts according to claim 1, characterized in that, In the feature comparison step, the feature comparison software module compares the calibration feature value with the object feature value according to an Euclidean distance algorithm to generate the similarity probability.

6. An image detection system for detecting minute defects and misassembled parts, characterized in that, It includes: A storage unit, which stores an image recognition model, a plurality of first images, a plurality of second images and a feature comparison software module; and An arithmetic processing unit, connected to the storage unit and configured to implement an object image recognition step, wherein the object image recognition step is to input an object image into the image recognition model, so that the image recognition model performs image recognition on the object image and outputs a recognition result. The image recognition model is established by training the plurality of first images and the plurality of second images, and the object image corresponds to one of the second images; Wherein, when the recognition result is one of the first images, the arithmetic processing unit determines that the recognition result does not match the object image; Wherein, when the recognition result is one of the second images, the arithmetic processing unit is configured to perform a recognition result verification step, and the recognition result verification step includes: A first feature extraction step of extracting an object feature value corresponding to the object image through the image recognition model; A second feature extraction step of inputting a calibration image into the image recognition model according to the recognition result, and extracting a calibration feature value corresponding to the calibration image through the image recognition model; A feature comparison step of executing the feature comparison software module, wherein the feature comparison software module compares the object feature value with the calibration feature value based on the calibration feature value to generate a similarity probability; and A recognition result determination step of determining whether the recognition result matches the object image based on the similarity probability and a threshold; Wherein, when the similarity probability is greater than or equal to the threshold, the arithmetic processing unit determines that the recognition result matches the object image; Wherein, when the similarity probability is less than the threshold, the arithmetic processing unit determines that the recognition result does not match the object image.

7. The image detection system for minute defects and wrong parts according to claim 6, wherein The arithmetic processing unit trains the plurality of first images and the plurality of second images according to a deep learning algorithm to establish the image recognition model, and the plurality of first images are different from the plurality of second images.

8. The image detection system for minute defects and wrong parts according to claim 7, characterized in that, The image recognition model includes at least one convolutional layer, a first fully connected layer, a second fully connected layer, and an activation function, and the arithmetic processing unit extracts the object feature value and the calibration feature value from the first fully connected layer.

9. The image detection system for minute defects and wrong parts according to claim 6, wherein In the second feature extraction step, the arithmetic processing unit selects one of the second images as the calibration image from the plurality of second images according to the recognition result.

10. The image detection system for minute defects and wrong parts according to claim 6, characterized in that, In the feature comparison step, the feature comparison software module generates the similarity probability by comparing the calibration feature value with the object feature value based on an Euclidean distance algorithm.

Images