Image acquisition system and method for detecting weak defects of microlens array for holographic display

Abstract

The present invention relates to the technical field of defect detection and identification, and more specifically, it relates to an image acquisition system and a method for detecting weak defects of a microlens array for holographic display. The key points of the technical solution are that it includes a microscopic imaging component for capturing sample images; an image processing component electrically connected to the microscopic imaging component for processing the sample image; and a motion control component for carrying the sample and driving the sample to move. The present invention can realize full automation of quality inspection of microlens arrays for holographic display, improve the efficiency of inspection, and avoid the highly subjective judgment during manual inspection, reduce the error of inspection, and reduce the missed detection rate and false detection rate.

Description

Technical Field

[0001] The present invention relates to the technical field of defect detection and identification, and more specifically, to an image acquisition system and a method for detecting weak defects of a microlens array for holographic display. Background Art

[0002] The use of microlens arrays for holographic display can greatly reduce the area of ​​reflected light in the future, improve image quality and reduce image distortion. As an important optical function carrier, the quality inspection of microlens arrays is of great significance.

[0003] Generally speaking, there are three main difficulties in the quality inspection of microlens arrays for holographic displays: the uneven distribution of light has a great impact on the detection of microlens arrays, resulting in local differences in the image; the defects in the microlens arrays are irregular in micro size and shape, and the grayscale value of the defects in the collected image is close to the background grayscale, making it difficult to detect; the background texture of the collected image is complex and often crosses defects, making detection more difficult. So far, the inspection of weak defects in such microlens arrays, including classification and recording, has been done manually, resulting in the disadvantages of time-consuming, highly subjective, and eye damage caused by long-term work. Summary of the invention

[0004] In view of the deficiencies in the prior art, the object of the present invention is to provide an image acquisition system and a method for detecting faint defects of a microlens array for holographic display, which have the advantages of reducing detection errors and improving detection effects.

[0005] The above technical objectives of the present invention are achieved through the following technical solutions: an image acquisition system, comprising:

[0006] A microscopic imaging component for taking images of samples;

[0007] An image processing component, electrically connected to the microscopic imaging component, for processing the sample image;

[0008] The mobile control component is used to carry the sample and drive the sample to move.

[0009] In one embodiment, the microscopic imaging assembly comprises:

[0010] A CCD camera, electrically connected to the image processing component, for taking an image of the sample;

[0011] A light source for emitting light;

[0012] A spectrometer, disposed in front of the light, for processing the light emitted by the light source;

[0013] An objective lens is arranged between the light source and the beam splitter.

[0014] Two fixed-focus lenses are respectively arranged between the CCD camera and the spectrometer, and between the sample and the spectrometer.

[0015] In one embodiment, the image processing component includes:

[0016] An image acquisition card, electrically connected to the CCD camera, and used for storing the image;

[0017] A computer is electrically connected to the image acquisition card and is used for processing the image.

[0018] In one embodiment, the movement control component includes:

[0019] A carrier, used for receiving the sample;

[0020] A moving device, the output end of which is transmission-connected to the bearing member, and is used to drive the bearing member to move;

[0021] The controller is electrically connected to the mobile device and is used to control the moving direction of the mobile device.

[0022] A method for detecting weak defects of a microlens array for holographic display comprises the following steps:

[0023] S1, using the image acquisition system to capture an image of the sample, and processing the image;

[0024] S2. Establish an ISE-YOLO network structure model for defect classification based on the ISE attention mechanism;

[0025] S3. placing a target object on the carrier to detect whether the target object has defects.

[0026] In one embodiment, the step S1 specifically includes the following steps:

[0027] S11, marking defects of the image;

[0028] S12, expanding the image data with defects marked, and dividing it into a training data set and a verification data set.

[0029] In one embodiment, the defects include scratches, broken edges, broken spots, bubbles, film defects, glue bubbles, debonding and demolding.

[0030] In one embodiment, the step S2 comprises the following steps:

[0031] S21, adjusting the image size;

[0032] S22, set CSPDarkNet53 as the backbone network, enhance the feature fusion of the model through the CSP residual structure, and extract the information of the shallow feature map;

[0033] S23, the neck network Neck adopts the form of feature pyramid PANet;

[0034] The output end of the neck network Neck is provided with and Three dimensions.

[0035] In one embodiment, the step S2 further includes introducing an ISE attention mechanism;

[0036] Dimension The input feature layer X of is passed through the average pooling layer, and the mathematical expression is:

[0037]

[0038] Compressed into The excitation factor is used to establish an explicit mutual dependency between the excitation factor channels through two fully connected layers, and the weights are calculated. The mathematical expression is:

[0039] F ex (F sq ,ω)=σ(ω 1 δω 2 F sq )

[0040] In the above formula, σ represents the sigmoid activation function, ω 1 and ω 2 They are represented as two different fully connected layers, and δ is the ReLU activation function;

[0041] The weights are scaled to the input feature layer of the corresponding channel, and the scaling mathematical expression is:

[0042] F scale (X,F ex )=X·F ex

[0043] The input feature layer is further extracted through a 3×3 convolution and stacked with the output features obtained by the above formula in the channel direction. The semantic features are fused with a 1×1 convolution and the number of output channels is adjusted to be consistent with the input channels.

[0044] In one embodiment, the step S2 further includes the following steps:

[0045] S24, training the ISE-YOLO network structure model;

[0046] The input resolution is adjusted to 480×480 and trained using the stochastic gradient descent SGD optimizer.

[0047] The above-mentioned image acquisition system and method for detecting weak defects of a microlens array for holographic display have the following beneficial effects:

[0048] It can realize full automation of the quality inspection of microlens arrays for holographic display, improve the efficiency of inspection, avoid the highly subjective judgment during manual inspection, reduce the error of inspection, and reduce the missed detection rate and false detection rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 Schematic diagram of the structure distribution of the image acquisition system in this embodiment;

[0050] Figure 2 Schematic diagram of the composition of the image acquisition system in this embodiment;

[0051] Figure 3 is a schematic flow chart of a method for detecting weak defects of a microlens array in this embodiment;

[0052] Figure 4 Schematic diagram of the structure of the network model of ISE-YOLO in this embodiment;

[0053] Figure 5 It is a schematic diagram of the architecture of the ISE attention mechanism in this embodiment.

[0054] In the figure: 1. Microscope imaging component; 11. CCD camera; 12. Light source; 13. Spectrum splitter; 14. Objective lens; 15. Fixed-focus lens; 2. Image processing component; 21. Image acquisition card; 22. Computer; 3. Mobile control component; 31. Carrier; 32. Mobile device; 33. Controller; 4. Sample. DETAILED DESCRIPTION

[0055] The present invention is described in detail below in conjunction with the accompanying drawings and embodiments.

[0056] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0057] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, unless otherwise clearly and specifically defined.

[0058] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral one; it can be a mechanical connection, an electrical connection, or communication with each other; it can be a direct connection, or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0059] In the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, a first feature being "above", "above" or "above" a second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being "below", "below" or "below" a second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.

[0060] An image acquisition system, such as Figure 1 As shown, it includes a microscopic imaging component 1 for capturing an image of a sample 4, an image processing component 2 for processing the image of the sample 4, the image processing component 2 being electrically connected to the microscopic imaging component 1, and a movement control component 3 for carrying the sample 4 and driving the sample 4 to move.

[0061] The sample 4 is placed on the mobile control component 3, and the mobile control component 3 can drive the sample 4 to move in the longitudinal and transverse directions. At the same time, the microscope imaging component 1 takes a picture of the sample 4 and sends the taken picture of the sample 4 to the image processing component 2. The image processing component 2 processes the received sample 4 component to facilitate the next step of operation.

[0062] Specifically, Figure 2As shown, the microscope imaging assembly 1 includes a CCD camera 11, a light source 12, a beam splitter 13, an objective lens 14 and two fixed-focus lenses 15, wherein the camera head of the CCD camera 11 is facing the sample 4, and the light emitted by the light source 12 generates a parallel light beam through the objective lens 14 during propagation, and the parallel light beam is refracted by the beam splitter 13, and the refracted light beam passes through the fixed-focus lens 15 between the beam splitter 13 and the sample 4 and is focused on the sample 4. The sample 4 reflects the light beam irradiated to itself again, and the reflected light beam passes through the fixed-focus lens 15 between the beam splitter 13 and the sample 4 and the fixed-focus lens 15 between the beam splitter 13 and the CCD camera 11 in sequence, and is then received by the CCD camera 11.

[0063] The direction of the CCD camera 11 is perpendicular to the direction of the light reflected by the light source 12, so as to facilitate the adjustment of the emission and reception of light;

[0064] The resolution of the CCD camera 11 is 640×480 pixels, the magnification of the objective lens 14 is 30×, and the actual size of a single pixel is 4.375 μm, which can reduce the computational burden of the image processing component 2;

[0065] The light source 12 adopts a white LED, which has the advantages of being less susceptible to external interference and having low energy consumption.

[0066] Specifically, the image processing component 2 includes an image acquisition card 21 and a computer 22. The image acquisition card 21 is electrically connected to the CCD camera 11 and is used to store images captured by the CCD camera 11. The computer 22 is electrically connected to the image acquisition card 21 and is used to process the images.

[0067] The image acquisition card 21 can convert the analog signal collected by the sensor into a digital signal.

[0068] Specifically, the mobile control component 3 includes a carrier 31, a mobile device 32 and a controller 33. The carrier 31 adopts a platform or a platform structure. The sample 4 can be placed on the surface of the carrier 31. The output end of the mobile device 32 is connected to the carrier 31 in a transmission manner. The sample 4 is moved by driving the carrier 31 to move. The sample 4 can be moved in the longitudinal and transverse directions respectively. The controller 33 serves as the control end of the mobile device 32 and is electrically connected to the mobile device 32. At the same time, the controller 33 is electrically connected to the computer 22 in the image processing component 2, so that the moving position control of the mobile sample 4 is more precise.

[0069] like Figure 3 As shown, a method for detecting weak defects of a microlens array for holographic display comprises the following steps:

[0070] S1, using an image acquisition system to capture an image of sample 4, and processing the image;

[0071] Specifically, S11, the defects of the image are marked, and the defects include scratches, broken edges, broken points, bubbles, film defects, glue bubbles, debonding and demolding, as well as edge information of the sample 4 collected to cooperate with the mobile device 32 to automatically capture images. 898 of the 1059 images collected are used as training images and 161 are used as verification images.

[0072] S12. Expand the image data with marked defects and divide them into training data set and verification data set. Use the open source tool Label Img to mark the training images and verification images respectively to obtain data. Then, process the above data by horizontal random flipping, random brightness adjustment, random image cropping and fixed scaling, increase background complexity, etc. to obtain enhanced data. The enhanced data is divided into training data set and verification data set for network training.

[0073] like Figure 4 As shown in S2, an ISE-YOLO network structure model for defect classification is established based on the ISE attention mechanism;

[0074] S21, adjusting the image size;

[0075] That is, the image size of the enhanced dataset is adjusted to an input image of 480×480×3, and then the channel and size are changed to 240×240×48 through the Focus operation to reduce the inference burden.

[0076] S22, set up to use CSPDarkNet53 as the backbone network, enhance the feature fusion of the model through the CSP residual structure, and extract the information of the shallow feature map;

[0077] S23, the neck network Neck adopts the form of feature pyramid PANet;

[0078] Among them, the output end of the neck network Neck is provided with and Three dimensions.

[0079] Further, such as Figure 5 As shown, the ISE attention mechanism is introduced;

[0080] Dimension The input feature layer X of is passed through the average pooling layer, and the mathematical expression is:

[0081]

[0082] Compressed into The excitation factor is used to establish an explicit mutual dependency between the excitation factor channels through two fully connected layers, and the weights are calculated. The mathematical expression is:

[0083] Fex (F sq ,ω)=σ(ω 1 δω 2 F sq )

[0084] In the above formula, σ represents the sigmoid activation function, ω 1 and ω 2 They are represented as two different fully connected layers, and δ is the ReLU activation function;

[0085] The weights are scaled to the input feature layer of the corresponding channel. The scaling mathematical expression is:

[0086] F scale (X,F ex )=X·F ex

[0087] The input feature layer is further extracted through a 3×3 convolution and stacked with the output features obtained by the above formula in the channel direction. The semantic features are fused with a 1×1 convolution and the number of output channels is adjusted to be consistent with the input channels.

[0088] S24, training ISE-YOLO network structure model;

[0089] The input resolution is adjusted to 480×480, and the stochastic gradient descent SGD optimizer is used for training, with momentum and weight decay set to 0.9 and 0.0005 respectively. The batch size is set to 8, the epoch is 500, and the learning rate is a monotonically decreasing sequence from 0.01 to 0.002;

[0090] Furthermore, the ISE-YOLO network structure model was verified by uniformly adjusting the input resolution to 480×480, setting Conf_thres to 0.6, and lou_thres to 0.01.

[0091] Preferably, the PloyLoss function is used to calculate the classification loss feedback network. The PloyLoss function is based on the Taylor expansion of FocalLoss and is decomposed into a series of weighted polynomial bases. Its mathematical expression is:

[0092]

[0093] Here, P t is the network’s predicted probability of the true target, L FL and γ represent the FocalLoss loss function and its coefficient factor, ε n are the hyperparameter coefficients of the polynomial.

[0094] The present invention focuses on Lploy-N The first term, L ploy-1 , truncate all high-order polynomial terms and experimentally determine γ and ε 1 The two hyperparameters are adjusted to the optimal solution, and the final category loss formula is:

[0095] LOSS cls =L FL (P t , 1.1)+(1-P t ) 2.1 .

[0096] S3, placing the target object on the carrier 31, and detecting whether the target object has defects.

[0097] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A method for detecting weak defects of a microlens array for holographic display based on an image acquisition system, The image acquisition system comprises: A microscopic imaging component for taking images of samples; An image processing component, electrically connected to the microscopic imaging component, for processing the sample image; A mobile control component, used for carrying the sample and driving the sample to move; The microscopic imaging assembly comprises: A CCD camera, electrically connected to the image processing component, for taking an image of the sample; A light source for emitting light; A spectrometer, disposed in front of the light, for processing the light emitted by the light source; An objective lens is arranged between the light source and the beam splitter; Two fixed-focus lenses, respectively arranged between the CCD camera and the spectrometer, and between the sample and the spectrometer; Characterized in that the detection method comprises the following steps: S1, using the image acquisition system to capture an image of the sample, and processing the image; S2. Establish an ISE-YOLO network structure model for defect classification based on the ISE attention mechanism; S3, placing the target object on the carrier, and detecting whether the target object has defects; The step S1 specifically includes the following steps: S11, marking defects of the image; S12, expanding the image data with defects marked, and dividing it into a training data set and a verification data set; Step S2 includes the following steps: S21, adjusting the size of the image; S22, set CSPDarkNet53 as the backbone network, enhance the feature fusion of the model through the CSP residual structure, and extract the information of the shallow feature map; S23, the neck network Neck adopts the form of feature pyramid PANet; The output end of the neck network Neck is provided with and Three dimensions; The step S23 also includes introducing an ISE attention mechanism; Dimension The input feature layer X of is passed through the average pooling layer, and the mathematical expression is: Compressed into The excitation factor is used to establish an explicit mutual dependency between the excitation factor channels through two fully connected layers, and the weights are calculated. The mathematical expression is: F ex (F sq ,ω)=σ(ω1δω2F sq ) In the above formula, σ represents the sigmoid activation function, ω1 and ω2 represent two different fully connected layers, and δ is the ReLU activation function; The weights are scaled to the input feature layer of the corresponding channel, and the scaling mathematical expression is: F scale (X,F ex )=X·F ex The input feature layer is further subjected to a 3×3 convolution to extract features and is stacked with the output features of the output end in the channel direction, and a 1×1 convolution is used to fuse the semantic features and adjust the number of output channels to be consistent with the input channels; the step S2 further includes the following steps: S24, training the ISE-YOLO network structure model; The PloyLoss function based on the Taylor expansion of FocalLoss is used to calculate the classification loss feedback network, which is decomposed into a series of weighted polynomial bases, and all high-order polynomial terms are truncated. The two hyperparameters are adjusted to the optimal solution through experiments. The final category loss formula is: LOSS cls =L FL (P t ,1.1)+(1-P t ) 2.1 , P t is the network’s predicted probability of the true target, L FL Represents the FocalLoss loss function.

2. The method for detecting weak defects according to claim 1, characterized in that: The image processing component comprises: An image acquisition card, electrically connected to the CCD camera, and used for storing the image; A computer is electrically connected to the image acquisition card and is used for processing the image.

3. The method for detecting weak defects according to claim 1, characterized in that: The mobile control component includes: A carrier, used for receiving the sample; A moving device, the output end of which is transmission-connected to the bearing member, and is used to drive the bearing member to move; The controller is electrically connected to the mobile device and is used to control the moving direction of the mobile device.

4. The method for detecting weak defects according to claim 1, characterized in that: The defects include scratches, broken edges, broken spots, bubbles, film shortage, glue bubbles, debonding and demoulding.

Images