Part appearance detection method and device based on neural network fitting and storage medium

Abstract

The invention relates to a product appearance detection technology, discloses a part appearance detection method and device based on neural network fitting and a storage medium, and solves the problem that appearance detection cannot be efficiently and accurately carried out on a product with a special-shaped complex structure in the prior art. According to the scheme, firstly, appearance image data, including image data with normal appearance and image data with abnormal appearance, of a part sample are obtained; according to the appearance image data of the part sample, marking the contained target features; then, according to the appearance image data of the part sample and the marked target features, a feature extraction model is obtained through neural network fitting; calculating output features based on the feature extraction model according to appearance image data of a to-be-detected part; and finally, according to the output characteristics, obtaining an appearance detection result of the to-be-detected part. The method is suitable for appearance detection of products with special-shaped complex structures.

Description

Technical Field

[0001] The present invention relates to product appearance detection technology, and in particular to a component appearance detection method, device and storage medium based on neural network fitting. Background Art

[0002] With the development of industrial automation technology, the integration of many tiny parts is getting higher and higher. Minor defects in appearance will greatly reduce their important performance and effective service life. Therefore, appearance inspection of tiny parts has become an indispensable process link. Many manufacturers have invested a lot of money to improve product quality to the greatest extent and prevent defective products from entering the market.

[0003] Traditional appearance inspection is mainly done manually, mostly with the aid of fluorescent lamps, microscopes, and other auxiliary methods, using the human eye for inspection and judgment. Due to the strong subjectivity of manual inspection, there are usually problems such as low inspection consistency, inconsistent standards, and low efficiency. Therefore, more and more manufacturers have introduced automated inspection equipment and used machine vision technology to replace manual inspection of product appearance. The current automated appearance inspection methods are mainly based on traditional image processing and deep learning methods. Deep learning requires a large amount of data and requires high-configuration hardware equipment, and has the disadvantages of long cycle and low cost performance; traditional image processing methods are mainly aimed at some product scenarios with regular appearance shapes, such as circular and rectangular appearances, and do not have the ability to detect complex and irregular appearances.

[0004] However, for special industry product scenarios such as high-speed connectors and ceramic substrates, they often have special and complex structures and require high detection accuracy, fast speed, and low cost. For such scenarios, most current automated detection equipment is still in the immature stage and cannot completely replace manual labor. More often, machine detection is used, combined with manual sampling or re-inspection. Summary of the invention

[0005] The technical problem to be solved by the present invention is to propose a method, device and storage medium for component appearance detection based on neural network fitting, so as to solve the problem in the prior art that appearance detection cannot be performed efficiently and accurately for products with irregular and complex structures.

[0006] The technical solution adopted by the present invention to solve the above technical problems is:

[0007] In one aspect, the present invention provides a method for detecting the appearance of a component based on neural network fitting, comprising the following steps:

[0008] Acquire appearance image data of component samples, including image data of component samples with normal appearance and image data of component samples with abnormal appearance;

[0009] According to the appearance image data of the component sample, mark the target features contained therein;

[0010] According to the appearance image data of the component sample and the annotated target features, a feature extraction model is obtained through neural network fitting;

[0011] According to the appearance image data of the component to be inspected, output features are calculated based on the feature extraction model;

[0012] According to the output features, the appearance inspection result of the component to be inspected is obtained.

[0013] Furthermore, the obtaining of appearance image data of a component sample includes:

[0014] According to the appearance quality standard of the component samples, multiple component samples including those with normal appearance and those with abnormal appearance are selected;

[0015] A camera is used to shoot multiple selected component samples to obtain their appearance image data.

[0016] Furthermore, the step of labeling target features contained in the appearance image data of the component sample includes:

[0017] Marking a first target feature based on image data of a component sample with a normal appearance; the first target feature is an edge contour of each module of the component sample;

[0018] According to the image data of the component sample with abnormal appearance, a second target feature is marked; the second target feature is the edge contour of each module of the component sample excluding the abnormal area.

[0019] Furthermore, the marking method includes: automatic marking by program or manual marking or semi-automatic marking combining program and manual marking.

[0020] Furthermore, the feature extraction model is obtained by fitting the appearance image data of the component sample and the annotated target features through a neural network, including:

[0021] The image data of the component sample with normal appearance is used as input information, the first target feature is used as supervision information, and a first feature extraction model is obtained by neural network fitting;

[0022] The image data of the parts sample with abnormal appearance is used as input information, and the second target feature is used as supervision information, and the second feature extraction model is obtained through neural network fitting.

[0023] Furthermore, the step of calculating the output features based on the feature extraction model according to the appearance image data of the component to be inspected includes:

[0024] Inputting the appearance image data of the component to be inspected into a first feature extraction model to obtain a first output feature;

[0025] The appearance image data of the component to be inspected is input into the second feature extraction model to obtain a second output feature.

[0026] Furthermore, obtaining the appearance inspection result of the component to be inspected according to the output feature includes:

[0027] Calculating similarity using the first output feature and the second output feature to obtain a similarity measurement value;

[0028] The judgment is made based on the similarity measurement value and the preset conditions to obtain the appearance inspection result of the parts to be inspected.

[0029] Furthermore, the appearance detection result includes NG or OK category attributes; if the appearance detection result is NG, it also includes specific contour information or mask area information.

[0030] In a second aspect, the present invention further provides a component appearance detection device based on neural network fitting, comprising:

[0031] A data acquisition module, used to acquire appearance image data of component samples, including image data of normal appearance and image data of abnormal appearance;

[0032] A labeling module is used to label the target features contained in the component sample according to the appearance image data thereof;

[0033] A fitting module is used to obtain a feature extraction model through neural network fitting based on the appearance image data of the component sample and the annotated target features;

[0034] A feature acquisition module, used to calculate output features based on the feature extraction model according to the appearance image data of the component to be inspected;

[0035] The judgment module is used to obtain the appearance inspection result of the component to be inspected according to the output characteristics.

[0036] In a third aspect, the present invention further provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-mentioned method for component appearance detection based on neural network fitting.

[0037] The beneficial effects of the present invention are:

[0038] The present invention combines a neural network and an image processing method to obtain a feature extraction model, thereby calculating output features based on the feature extraction model according to the appearance image data of the parts to be detected, and obtaining the appearance detection results of the parts to be detected according to the output features. Since the neural network can fit the complex nonlinear associations between many features in the image data, the scheme of the present invention can efficiently and accurately detect whether the appearance of products with special-shaped and complex structures is abnormal. At the same time, detailed information such as the location and area of ​​the abnormal area can be output, so as to realize parameterization and configurability of the detection standard, and effectively improve the flexibility of the detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a flow chart of a component appearance detection method based on neural network fitting in Example 1 of the present invention;

[0040] Figure 2 This is a structural block diagram of a component appearance detection device based on neural network fitting in Example 2 of the present invention. DETAILED DESCRIPTION

[0041] The present invention aims to provide a method, device and storage medium for component appearance detection based on neural network fitting, so as to solve the problem in the prior art that products with irregular and complex structures cannot be efficiently and accurately detected. The scheme of the present invention combines neural networks and image processing methods, and by marking the target features of sample appearance image data, the powerful fitting ability of neural networks is used to fit the complex nonlinear associations between the numerous features in the image data, thereby obtaining a feature extraction model, and according to the appearance image data of the component to be detected, the output features are calculated based on the feature extraction model; finally, according to the output features, the appearance detection results of the component to be detected are obtained.

[0042] The scheme of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0043] Example 1

[0044] This embodiment takes a high-speed connector component product as an example and provides a component appearance detection method based on neural network fitting. The process is shown in FIG. Figure 1 , including the following implementation steps:

[0045] S1. Obtaining appearance image data of component samples, including image data of normal appearance and image data of abnormal appearance;

[0046] In this step, according to the appearance quality standard of high-speed connector components, multiple high-speed connector component product samples with normal appearance and abnormal appearance are selected. For the sake of data distribution uniformity, the ratio of the number of selected component product samples with normal appearance and abnormal appearance is about 1:1; then, according to the appearance size, CCD working distance and specific detection accuracy requirements of the selected specifications of the high-speed connector component product samples, the camera hardware is selected; the selected component product samples are imaged using the selected camera to obtain corresponding image data.

[0047] S2. Annotate the target features contained in the appearance image data of the component sample;

[0048] In this step, based on the image data of the high-speed connector component sample with normal appearance, the first target feature is marked; the first target feature is the edge contour of each module of the high-speed connector component sample; based on the image data of the high-speed connector component sample with abnormal appearance, the second target feature is marked; the second target feature is the edge contour of each module of the component excluding the abnormal area.

[0049] Among them, the marking method includes: automatic marking by program or manual marking or semi-automatic marking combining program and manual.

[0050] S3, obtaining a feature extraction model through neural network fitting based on the appearance image data of the component sample and the annotated target features;

[0051] In this step, the image data of the high-speed connector component sample with normal appearance is used as input information, the first target feature is used as supervision information, and the first feature extraction model is obtained by neural network fitting; the image data of the high-speed connector component sample with abnormal appearance is used as input information, the second target feature is used as supervision information, and the second feature extraction model is obtained by neural network fitting.

[0052] S4, calculating output features based on the feature extraction model according to the appearance image data of the component to be inspected;

[0053] In this step, a camera is used to capture the appearance image data of the high-speed connector component to be inspected, and then the appearance image data of the high-speed connector component to be inspected is input into a first feature extraction model to obtain a first output feature; the appearance image data of the high-speed connector component to be inspected is input into a second feature extraction model to obtain a second output feature.

[0054] S5. Obtaining appearance inspection results of the components to be inspected according to the output features;

[0055] In this step, similarity is calculated using the first output feature and the second output feature to obtain a similarity measurement value; a judgment is made based on the similarity measurement value and preset conditions to obtain an appearance detection result; the appearance detection result includes NG or OK category attributes; if the appearance detection result is NG, it also includes specific contour information or mask area information.

[0056] Based on the above description of this embodiment, it is possible to perform efficient and accurate appearance inspection of high-speed connector components with irregular and complex structures. At the same time, detailed information such as the location and area of ​​the abnormal area can be output, making the inspection standards parameterized and configurable, effectively improving the flexibility of inspection.

[0057] Example 2

[0058] This embodiment takes a high-speed connector component product as an example and provides a component appearance detection device based on neural network fitting. Figure 2 ,include:

[0059] A data acquisition module, used to acquire appearance image data of a sample of high-speed connector components, including image data of normal appearance and image data of abnormal appearance;

[0060] A labeling module, used for labeling target features contained in the sample of high-speed connector parts according to the appearance image data thereof;

[0061] A fitting module is used to obtain a feature extraction model through neural network fitting based on the appearance image data of the high-speed connector component sample and the annotated target features;

[0062] A feature acquisition module, used to calculate output features based on the feature extraction model according to the appearance image data of the high-speed connector component to be detected;

[0063] The judgment module is used to obtain the appearance inspection result of the high-speed connector component to be inspected according to the output characteristics.

[0064] Since the functional modules in the component appearance detection device based on neural network fitting in this embodiment correspond to the corresponding steps of the component appearance detection method based on neural network fitting in Example 1, the more specific functional implementation of each functional module refers to the description of Example 1 and will not be repeated here.

[0065] Finally, it should be noted that the above embodiments are only preferred implementations and are not intended to limit the present invention. It should be pointed out that for those skilled in the art, several modifications, equivalent replacements, improvements, etc. can be made without departing from the scope of the present invention and the scope of protection of the claims, and all of these should be included in the protection scope of the present invention.

Claims

1. A component appearance detection method based on neural network fitting, characterized in that: The following steps are involved: Acquire appearance image data of component samples, including image data of component samples with normal appearance and image data of component samples with abnormal appearance; According to the appearance image data of the component sample, mark the target features contained therein; According to the appearance image data of the component sample and the annotated target features, a feature extraction model is obtained through neural network fitting; According to the appearance image data of the component to be inspected, output features are calculated based on the feature extraction model; According to the output features, the appearance inspection result of the component to be inspected is obtained.

2. The component appearance detection method based on neural network fitting according to claim 1, characterized in that: The step of obtaining appearance image data of a component sample includes: According to the appearance quality standard of the component samples, multiple component samples including those with normal appearance and those with abnormal appearance are selected; A camera is used to shoot multiple selected component samples to obtain their appearance image data.

3. The component appearance detection method based on neural network fitting according to claim 1, characterized in that: The step of marking target features contained in the appearance image data of the component sample includes: Marking a first target feature based on image data of a component sample with a normal appearance; the first target feature is an edge contour of each module of the component sample; According to the image data of the component sample with abnormal appearance, a second target feature is marked; the second target feature is the edge contour of each module of the component sample excluding the abnormal area.

4. The component appearance detection method based on neural network fitting according to claim 1, characterized in that: The marking method includes: automatic marking by program or manual marking or semi-automatic marking combining program and manual marking.

5. The component appearance detection method based on neural network fitting as claimed in claim 3, characterized in that: The feature extraction model is obtained by fitting the appearance image data of the component sample and the marked target features through a neural network, including: The image data of the component sample with normal appearance is used as input information, the first target feature is used as supervision information, and a first feature extraction model is obtained by neural network fitting; The image data of the parts sample with abnormal appearance is used as input information, and the second target feature is used as supervision information, and the second feature extraction model is obtained through neural network fitting.

6. The component appearance detection method based on neural network fitting according to claim 5, characterized in that: The step of calculating output features based on the feature extraction model according to the appearance image data of the component to be inspected includes: Inputting the appearance image data of the component to be inspected into a first feature extraction model to obtain a first output feature; The appearance image data of the component to be inspected is input into the second feature extraction model to obtain a second output feature.

7. The component appearance detection method based on neural network fitting according to claim 6, characterized in that: The step of obtaining the appearance inspection result of the component to be inspected according to the output feature comprises: Calculating similarity using the first output feature and the second output feature to obtain a similarity measurement value; The judgment is made based on the similarity measurement value and the preset conditions to obtain the appearance inspection result of the parts to be inspected.

8. The component appearance detection method based on neural network fitting according to any one of claims 1 to 7, characterized in that: The appearance detection result includes NG or OK category attributes; if the appearance detection result is NG, it also includes specific contour information or mask area information.

9. A component appearance detection device based on neural network fitting, characterized in that: include: A data acquisition module, used to acquire appearance image data of component samples, including image data of normal appearance and image data of abnormal appearance; A labeling module is used to label the target features contained in the component sample according to the appearance image data thereof; A fitting module is used to obtain a feature extraction model through neural network fitting based on the appearance image data of the component sample and the annotated target features; A feature acquisition module, used to calculate output features based on the feature extraction model according to the appearance image data of the component to be inspected; The judgment module is used to obtain the appearance inspection result of the component to be inspected according to the output characteristics.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the component appearance detection method based on neural network fitting as described in any one of claims 1 to 8 are implemented.

Images