A deep learning-based point defect feature restoration method

Abstract

The application provides a point defect feature restoration method based on deep learning, which captures the morphology of the same type of point defects in different areas of the screen through optical shooting and camera imaging, obtains a plurality of defect small pictures through cutting, establishes an image pair dataset of the same type of point defects in the central area and the non-central area according to the defect small pictures, obtains a network model with a restoration function through a deep learning network training, maps the point defects in the non-central area into the morphology and gray scale features of the defects in the central area based on the network model, and then classifies the restored images. The method solves the classification difficulty problem caused by the large difference in defect morphology, makes up for the deficiency of the conventional correction algorithm, improves the reliability of defect morphology restoration, and thus improves the accuracy and efficiency of defect classification.

Description

Technical Field

[0001] This invention belongs to the field of near-eye display defect detection, specifically involving a point defect feature restoration method based on deep learning. Background Technology

[0002] The rapid development of virtual reality technology has spawned a series of new display devices, such as VR glasses. The product quality of VR glasses directly affects imaging effects and user experience. Currently, defect detection for VR glasses mainly focuses on point defects, including bright and dark spots. It is necessary to detect point defects, differentiate their brightness levels, and then categorize them according to the customer's quality plan. (See attached diagram in the instruction manual.) Figure 2 As shown in the diagram, a VR screen is typically divided into multiple areas, including a central area and non-central areas (including intermediate transition areas and edge areas). Due to factors such as image capture and the curved surface structure of the product, defects in the central area appear brighter and more realistically shaped, while defects in the non-central areas appear darker and more blurred, as shown in the attached diagram in the instruction manual. Figure 3 As shown, the significant differences in defect morphology across different regions have greatly complicated defect classification. Summary of the Invention

[0003] To address the problems of existing technologies, this invention proposes a method for point defect feature restoration based on the characteristics of the pix2pix model in deep learning. This method restores point defects in non-central areas of a VR screen to the shape and grayscale of point defects in the central area, achieving a unified defect feature effect. This invention captures the shape of the same type of point defect in different areas of the screen through optical imaging and camera capture. Based on the defect thumbnails, it establishes image pairs of the same type of point defect in the central and non-central areas, trains a network model with restoration capabilities, and maps point defects in non-central areas to the shape and grayscale features that the defect should have in the central area, before proceeding with subsequent tasks. The technical solution adopted by this invention to solve the problems existing in the prior art is as follows:

[0004] A method for restoring point defect features based on deep learning, specifically including the following steps:

[0005] Step 1: Set up the optical detection environment and configure appropriate imaging parameters;

[0006] The requirements for setting up the optical detection environment in step 1 are to adjust the focus to make the image of the central area clear, and the gray level of the central area image is 130±10.

[0007] Step 2: Use a black and white camera to capture an image, aiming the lens at the center of the product to ensure a complete and clear image;

[0008] Step 3: Extract defects from the image obtained in Step 2, and crop each region containing a defect to form several small defect images;

[0009] Step 4: Select a set of defect images near the center point from all the defect images obtained in Step 3;

[0010] The set of defect thumbnails near the center point selected in step 4 should contain all types of defects extracted.

[0011] Step 5: Combine the small defect images of non-central areas and the small defect images near the central point according to the same type to form several image pairs;

[0012] In step 5, the image stitching principle is to pair all defect sample points in the non-central area and near the central point with the corresponding positions of the image in the design screen.

[0013] Step 6: Feed the stitched images of the same type into the pix2pix neural network for training. After training, a network model with feature restoration function is obtained.

[0014] Step 7: Based on the network model obtained in Step 6, restore any small defect image to the greatest extent possible to restore its shape when it is in the center position.

[0015] Step 8: Classify the defects in the restored image.

[0016] In step 8, the defect types are divided according to specific classification requirements. If they need to be divided into two categories, large and small, the area after point restoration is classified by size. If they need to be divided into four categories, large, small, bright, and dark, they are classified according to the area and grayscale after point restoration.

[0017] The present invention has the following advantages:

[0018] 1) The method of this invention solves the problem of classification difficulty caused by large differences in defect morphology, makes up for the shortcomings of conventional correction algorithms, improves the reliability of defect morphology restoration, and thus improves the accuracy and efficiency of defect classification.

[0019] 2) The method of the present invention can greatly reduce the impact caused by image acquisition and product surface characteristics;

[0020] 3) The image restored by the method of the present invention is helpful for subsequent tasks, thereby improving the accuracy and precision of the image. Attached Figure Description

[0021] Figure 1 This is a flowchart of the method of the present invention;

[0022] Figure 2A diagram illustrating the area division of a VR screen;

[0023] Figure 3 A comparison chart of the states of the same type of point at different positions on the screen;

[0024] Figure 4 These are small defect images obtained by taking pictures in different areas during the sampling stage using a black and white camera in this embodiment of the invention.

[0025] Figure 5 These are small diagrams of eight types of defects near the center point in this embodiment of the invention;

[0026] Figure 6 In this embodiment of the invention, the image is a pair formed by stitching together the small image of the defect at the center point and the small images of the same type of defect in the non-center area;

[0027] Figure 7 In this embodiment of the invention, a deep learning model is used to restore the image of a point defect in a non-central region to the morphological image of the point defect in the central region. Detailed Implementation

[0028] The technical solution of the present invention will be further described in detail below through embodiments and in conjunction with the accompanying drawings, such as... Figure 1 As shown, a point defect feature reconstruction method based on deep learning includes the following steps:

[0029] Step 1: Set up the optical detection environment and configure appropriate imaging parameters, with imaging standards and image clarity as the principles, and the gray level of the central area image being 130±10.

[0030] Step 2: Use a black and white camera with a fixed magnification lens to capture an image. Point the lens at the center of the product and take a picture to ensure that the image is complete and clear.

[0031] Step 3: Extract defects from the image obtained in Step 2, detect false point defects using a detection algorithm, and crop each defect region to form several smaller defect images of the same size. Due to factors such as lighting and the curvature of the product surface, the shape and grayscale of defect points in different regions vary, such as... Figure 4 The diagram shows the morphology of the defect points in different areas of the product.

[0032] Step 4: In the central area of ​​the product, defect features are clearest and most obvious. Due to the curved surface of the product, the further away from the center, the greater the feature deformation deviation. False point defects in the central area are closest to the ideal state and better reflect defect features compared to other areas. From all the defect images obtained in Step 3, select a set of defect images near the center point. This set of defect images should contain one image of each type of defect, such as... Figure 5 As shown, a small image of a defect near the center point is displayed;

[0033] Step 5: Combine the small defect images of non-central areas and those near the center point according to the same type to form several image pairs; for example, using... Figure 5 Using this set of defect images as a baseline, similar defect images of the same type are selected from the remaining defect images in the non-central areas and stitched together to form several image pairs, such as... Figure 6 As shown;

[0034] Step 6: Create a dataset from the stitched images of the same type and feed it into the pix2pix deep learning neural network for model training. After training, a network model with feature reconstruction capabilities is obtained. pix2pix is ​​a network with image translation capabilities, employing a cGAN network structure, including a generator and a discriminator. The generator's function is to generate a target image from a noisy image, while the discriminator's function is to determine whether the generated image is realistic enough. During model training, the generator and discriminator continuously interact, with the two networks training alternately until an equilibrium is reached. The ultimate function of pix2pix is ​​to achieve mutual conversion between different image formats.

[0035] Step 7: As Figure 7 As shown, the network model generated based on the dataset obtained in step 6 can map point defects in non-central areas to the shape of the point defect in the central area, thereby achieving the purpose of defect shape restoration.

[0036] Step 8: Classify the defects in the restored image. Multiple categories can be defined according to customer needs. For example, if the customer only needs to classify into two categories (size and light), then the restored area of ​​the points will be classified by size. If the customer needs to classify into four categories (large, small, bright, and dark), then the restored area and grayscale of the points will be classified into four categories.

[0037] The scope of protection of this invention is not limited to the embodiments described above. Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its scope and spirit. If these modifications and variations fall within the scope of the claims of this invention and their equivalents, then the intent of this invention also includes these modifications and variations.

Claims

1. A point defect feature reconstruction method based on deep learning, characterized in that, Specifically, the steps include the following: Step 1: Set up the optical detection environment and configure appropriate imaging parameters; Step 2: Use a camera to capture an image, aiming the lens at the center of the product to ensure the image is complete and clear; Step 3: Extract defects from the image obtained in Step 2, and crop each region containing a defect to form several small defect images; Step 4: Select a set of defect images near the center point from all the defect images obtained in Step 3; Step 5: Combine the small defect images of non-central areas and the small defect images near the central point according to the same type to form several image pairs; Step 6: Feed the stitched images of the same type into the pix2pix neural network for training. After training, a network model with feature restoration function is obtained. Step 7: Based on the network model obtained in Step 6, restore any small defect image to restore its shape when it is in the center position. Step 8: Classify the defects in the restored image.

2. The point defect feature restoration method based on deep learning as described in claim 1, characterized in that: The requirement for setting up the optical detection environment in step 1 is to adjust the focus to make the image of the central area clear.

3. The point defect feature restoration method based on deep learning as described in claim 1, characterized in that: In step 1, the appropriate image acquisition parameters are to set the grayscale of the central region image to 130±10.

4. The point defect feature restoration method based on deep learning as described in claim 1, characterized in that: The camera used for image acquisition in step 2 is a black and white camera.

5. The point defect feature restoration method based on deep learning as described in claim 1, characterized in that: The defect miniatures obtained in step 3 have the same shape and size, including all types of defect miniatures in the central and non-central regions.

6. The point defect feature restoration method based on deep learning as described in claim 1, characterized in that: The set of defect thumbnails near the center point selected in step 4 should include one thumbnail of each type of defect extracted from that area.

7. The point defect feature restoration method based on deep learning as described in claim 1, characterized in that: In step 5, the image stitching principle is to pair all types of defect sample points in the non-central area and near the central point with the corresponding positions of the image in the design screen.

8. The point defect feature restoration method based on deep learning as described in claim 1, characterized in that: In step 6, the deep learning neural network pix2pix is ​​a network with image translation capabilities. It adopts a cGAN network structure and includes a generator and a discriminator. The generator generates a target image from a noisy image, and the discriminator judges whether the generated target image is realistic enough. Based on the pix2pix network, defect images of different regions are converted into each other. In step 6, the same type of image pairs stitched together in step 5 are made into a dataset and fed into pix2pix for model training to obtain a feature restoration network model generated from the dataset.

9. The point defect feature restoration method based on deep learning as described in claim 1, characterized in that: In step 8, the defect types are divided according to specific classification requirements. When it is necessary to divide them into two categories, large and small, the area after the points are restored is classified by size.

10. The point defect feature restoration method based on deep learning as described in claim 1, characterized in that: In step 8, the defect types are divided according to specific classification requirements. When it is necessary to divide them into four categories: large, small, bright, and dark, the four categories are classified according to the area and grayscale of the point after restoration.

Images