Glass screen tiny scratch detection method and system based on small sample deep learning

Abstract

The invention provides a glass screen tiny scratch detection method and system based on small sample deep learning, and aims to solve the difficulties and defects of tiny scratch defect detection in the current glass screen production and processing technology. Firstly, a small sample deep learning model is designed, and efficient training in a small sample environment is achieved through the steps of small scratch sample image training model, model fine tuning and the like. And then, identifying the image acquired by the image acquisition device by using the trained deep learning model, comprehensively judging whether the image identification glass screen under the three-color light has a tiny scratch defect or not, and displaying an identification result through a display device. According to the method, a three-color light source assistance and high-resolution image acquisition technology is combined with a small sample deep learning model, so that the problem of small scratch detection of the glass screen is solved, and high-accuracy defect detection under the small sample condition is realized through technologies such as data enhancement, pre-training and model fine adjustment.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent manufacturing technology, and specifically relates to a glass screen micro-scratch detection system based on small sample deep learning. Background Art

[0002] In the mobile phone processing technology, the mobile phone glass screen is not only the display layer, but also an important carrier of the touch layer. The quality of the mobile phone glass screen directly affects the user's visual and tactile experience. Although tiny scratches are difficult to detect with the naked eye, they may directly affect the overall quality and durability of the product. Therefore, micro-scratch detection technology is particularly important in the manufacturing process of high-quality glass screens. Traditional scratch detection methods usually rely on manual visual inspection, which is not only inefficient, but also easily affected by subjective factors and cannot meet the requirements of high-quality and high-precision detection. With the rapid development of artificial intelligence and deep learning, small sample deep learning has become a technical method that is expected to be applied in the field of scratch detection. In addition, the detection accuracy of mobile phone glass screens is usually required to be within tens of microns, which requires the detection equipment to have high resolution and high precision. In order to solve this problem, the present invention proposes a glass screen micro-scratch detection system based on small sample deep learning. A small amount of learning samples can be effectively used to complete the training of the deep learning model, and then complete the micro-scratch detection with high accuracy. Summary of the invention

[0003] In view of the difficulties and shortcomings in the detection of tiny scratch defects in the current glass screen production and processing technology, the present invention proposes a method and system for detecting tiny scratches on glass screens based on small sample deep learning.

[0004] In order to achieve the above objectives, the present invention provides a method for detecting tiny scratches on a glass screen based on small sample deep learning, comprising the following steps: Step S1, using an image acquisition device to sequentially acquire glass screen images in batches under three separate color lights of red, green and blue, obtaining three glass screen images in each batch, and sending the three glass screen images acquired in each batch to a recognition server; Step S2, pre-collecting a certain amount of glass screen images with tiny scratches and without scratches and making sample patterns, and the recognition server training a small sample deep learning model according to the sample patterns to obtain a trained small sample deep learning model; Step S3, the recognition server receives the three glass screen images collected by the image acquisition device in each batch, and performs image recognition operations on the three glass screen images collected in each batch through the trained small sample depth model to generate scratch defect recognition results; Step S4, receiving the scratch defect recognition result sent by the recognition server through the display device, and displaying the received scratch defect recognition result.

[0005] Furthermore, the step S2 specifically includes the following steps: Step S201, manually inspect and classify a certain amount of pre-collected glass screen images with tiny scratches and without scratches, sort out images including 70% with tiny scratches and 30% without scratches, and make a sample pattern; collect n sample images, where the value range of n is [10, 30]; make each sample image into a data packet Z, including the following two steps: Step A1, crop the sample image to obtain a standard sample image X of size 640x640, and use the labelme annotation tool to annotate it to obtain the annotated image P: ; in, is the element in the i-th row and j-th column of the labeled image P, To label the classification of the element in the i-th row and j-th column of the sample image, For the scratch area, This means that the pixel is a scratch; Indicates that this pixel is not a scratch.

[0006] Step A2: The labeled sample image X and the labeled image P are made into a data package Z, where Z={X, P}.

[0007] Step S202, building a small sample deep learning model; Step S203, using the data packet Z obtained in step S201 to train the small sample deep learning model, to obtain a trained small sample deep model.

[0008] Furthermore, the step S202 is specifically as follows: the small sample deep learning model includes a feature extraction module Net1, a feature three-pyramid network Net2, a feature fusion module Net3 and a segmentation output head module Net4; The feature extraction module Net1 includes a 6-layer network in series, and the structures are: the 1st, 3rd, and 5th layers are convolution layers, the number of convolution kernels are 4, 8, and 16 respectively, the convolution kernel size is 3x3, the step size is 1, and the filling method is , using the ReLU activation function; the 2nd, 4th, and 6th layers are pooling layers, using the maximum pooling method, the pooling kernel size is 2x2, the step size is 2, and the filling method is ; The Net1 input is a 640x640 sample image X, and the output is a 640x640x16 high-dimensional feature X1. ; The characteristic three-pyramid network Net2 is a parallel connection of three convolutional network branches Net21, Net22, and Net23. Each branch contains 10 layers of networks in series. The structures are: the 1st, 2nd, 3rd, 5th, 6th, 7th, and 9th layers are convolutional layers, the number of convolution kernels are 16, 16, 64, 64, 128, and 256 respectively, the size of the convolution kernel is 3x3, the step size is 1, and the filling method is , using the ReLU activation function; the 4th, 8th, and 10th layers are pooling layers, with a pooling kernel size of 2x2, a step size of 2, and a padding method of , using the maximum pooling method; The input of Net2 is the high-dimensional feature No. 1 X1, and the output is the high-dimensional feature No. 2 X2 of 80x80x256. ; in, For light color mask, is matrix multiplication; ; The feature fusion module Net3 is a 6-layer series network added at the end of the feature three-pyramid network structure. The structures are: the 1st, 3rd, and 5th layers are deconvolution layers, the number of convolution kernels are 128, 64, and 16 respectively, the convolution kernel size is 3x3, the step size is 1, and the filling method is , using the ReLU activation function; the 2nd, 4th, and 6th layers are anti-pooling layers, the pooling kernel size of the anti-pooling layer is 2x2, and the step size is 2; The Net3 input is the high-dimensional feature No. 2 X2, and the output is the high-dimensional feature No. 3 X3 of 640x640x16. ; The segmentation output head module Net4 contains 1 convolution layer, the number of convolution kernels is 1, the convolution kernel size is 1x1, the step size is 1, and the padding method is , use SoftMax activation function; The Net4 input is the No. 3 high-dimensional feature X3, and the output is a 640x640 single-channel result Y. .

[0009] Furthermore, step S203 specifically includes the following steps: Step B1, perform data enhancement processing on the data packets to obtain m enhanced data packets K, where m>>n, and the value range of m is [100, 1000]. 90% of the obtained m data packets K are used as training data K_train, and 10% are used as test data K_test. ; in, Indicates random cropping, flipping, rotation, and scaling of images. Indicates that the data set is divided proportionally, and the proportion is represented by the last two elements; Step B2, using the existing public data set to pre-train the small sample deep learning model to obtain model D; Step B3, use K_train to fine-tune model D. The loss function of the training process uses cross entropy loss. The specific calculation formula is: ; Among them, Loss_cls is the loss function; N is the number of pixels in the image, 2 is the number of categories; i and j are pixel coordinate indexes; is the one-hot encoding of the labeled image in the data packet flattened into a row, indicating that the i-th element belongs to the j-th category, There are no scratches. Indicates the presence of scratches. is the prediction result output by the deep learning model, , the value represents the probability of the existence or non-existence of scratches; log is the logarithmic function; learning rate The range is: ; The model D is regularized by the L2 regularization method to obtain a small sample deep learning model E. The L2 regularization method is: ; in, is the regularization factor, is the vectorized representation of all trainable parameters of model D; L is the loss function with regularization term; Step B4: Test the small sample deep learning model E on K_test. The evaluation indicators are global average accuracy OA and intersection over union (IoU). The calculation formula is as follows: ; Among them, TP is the number of pixels predicted to be scratches and actually are scratches, FN is the number of pixels that are not scratches but are not predicted, FP is the number of pixels that are scratches but are not predicted, and TN is the number of pixels that are not scratches but are predicted. Step B5: If the average test results of step B4, OA>0.99, IoU>0.95, the model training is considered to be completed, and a trained small sample deep learning model is obtained, then the subsequent stage is entered, otherwise return to step B3 for retraining.

[0010] Furthermore, in step B3, model D is fine-tuned, which specifically includes the following steps: Step B31, calculate all parameter gradients grad(w) of model D. The parameter gradients of all submodules of model D are included in grad(w). The parameter gradients of the three branches of the feature three-pyramid network are included in the parameter gradients of the feature three-pyramid: ; Among them, grad(w) represents the gradient of the function with respect to the variable w; Represents the partial derivative of the loss function L with respect to the parameter w; Step B32, update the parameters of each module: ; in, is the element-wise multiplication of matrices, It is a parameter matrix that can be manually adjusted according to the brightness of the light source, and the adjustment range is: Represents the sum of all elements.

[0011] It should be noted that, since red light has the longest wavelength and the smallest refractive index, the defect characteristics of the image collected when the light is red are the least obvious in terms of optics, and the image acquisition device has the smallest difference after being exposed to light, so the corresponding The smallest, similarly, blue light corresponds to maximum.

[0012] Furthermore, the step S3 specifically includes the following two steps: Step C1: send the glass screen images under red, green and blue light to the trained small sample deep learning model E in turn, perform image recognition operations, and record the output obtained each time as a 640x640 matrix .

[0013] Step C2, check the matrix one by one , generate the scratch defect recognition result Result, the calculation method of Result is: ; Among them, True and False are fields of the glass screen image scratch defect recognition result information output by the recognition server.

[0014] Furthermore, the specific method of step S4 is as follows: After receiving the glass screen image scratch defect recognition result information output by the recognition server, parse the scratch defect recognition result field Result in the information. If the field is "True", the screen displays red and prints the information "Scratch defect exists" on the screen; if the field is "False", the screen displays green and prints "No scratch defect" on the screen.

[0015] Furthermore, the present invention also discloses a glass screen micro-scratch detection system based on small sample deep learning, comprising an image acquisition device, wherein the image acquisition device sequentially acquires glass screen images under three separate color lights of red, green and blue in batches to obtain three glass screen images, and sends the three glass screen images acquired in each batch to a recognition server; the recognition server is used to identify micro-scratch defects in the image; A light source device, wherein there are three light source devices, each of which is installed directly above a plane parallel to the plane where the glass screen is located, and the image acquisition device is installed at a position in the same plane as the three light source devices; the light source device is used to assist in image acquisition; A display device receives the scratch defect recognition result sent by the recognition server and displays the received scratch defect recognition result.

[0016] Furthermore, the light source devices are: three red, green and blue LED light sources with wavelengths of 620-630 / 515-530 / 460-475 nm, color temperatures of 1000-2000 / 3000-5000 / 6500-9000 K, and brightness of 100-200 / 400-800 / 100-200 mcd, respectively. The three LED light sources are placed in an equilateral triangle, and the image acquisition device is placed at the center of gravity of the equilateral triangle. The line connecting the red LED light source and the image acquisition device should be consistent with the movement direction of the glass screen. The side length of the equilateral triangle is Range is ,in, is the height between the image acquisition device and the light source plane and the glass screen.

[0017] Furthermore, the image acquisition device is a microscope camera, and the lens of the microscope camera is a microscope objective lens, and its viewing angle Range is .

[0018] Beneficial Effects Compared with the prior art, the present invention has the following beneficial effects: The present invention provides a method for detecting tiny scratches on glass screens based on small sample deep learning. The method uses a small sample deep learning model combined with a three-color multi-band light imaging method to solve the problem of difficult manual detection and low accuracy of tiny scratches on glass screens. Through data enhancement, pre-training, model fine-tuning and other technologies, the model can also achieve high-accuracy defect detection in small sample conditions.

[0019] The present invention provides a glass screen micro-scratch detection system based on small sample deep learning. Through the synergy of image acquisition, three-color light source assistance, deep learning model and display device, efficient and accurate micro-scratch defect detection is achieved, further improving the detection efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A flowchart of a method for detecting tiny scratches on a glass screen based on small sample deep learning is provided for the present invention; Figure 2 This is a structural diagram of the small sample deep learning model proposed in the present invention; Figure 3 Schematic diagram of tiny scratches. DETAILED DESCRIPTION

[0021] In order to make the purpose and features of the present invention more obvious and easy to understand, the present technical solution is described in detail below through embodiments and in conjunction with the accompanying drawings.

[0022] like Figure 1 As shown, a method for detecting tiny scratches on a glass screen based on small sample deep learning includes the following steps: Step S1, collecting glass screen images in batches in sequence under three separate color lights of red, green and blue according to the image collection device, obtaining three glass screen images in each batch, and sending the collected three glass screen images in each batch to the recognition server; Step S2, pre-collecting a certain amount of glass screen images with tiny scratches and without scratches and making sample patterns, the recognition server trains a small sample deep learning model according to the sample patterns to obtain a trained small sample deep model. The specific steps include: Step S201, data collection and sample preparation, manually inspect and classify a certain amount of pre-collected glass screen images with tiny scratches and without scratches, sort out 70% of the images with tiny scratches and 30% of the images without scratches, and make sample patterns. These images will be used for training and verification of deep learning models; collect n sample images, where the value range of n is [10, 30]; each sample image will be made into a data package Z, including the following two steps: Step A1, crop the sample image to obtain a standard sample image X of size 640x640, and use the labelme annotation tool to annotate it to obtain the annotated image P: ; in, is the element in the i-th row and j-th column of the labeled image P, To label the classification of the element in the i-th row and j-th column of the sample image, For the scratch area, This means that the pixel is a scratch. Indicates that this pixel is not a scratch.

[0023] Step A2: The labeled sample image X and the labeled image P are made into a data package Z, where Z={X, P}.

[0024] Step S202: building a small sample deep learning model. The small sample deep learning model includes feature extraction module Net1, feature three-pyramid network Net2, feature fusion module Net3 and segmentation output head module Net4; The feature extraction module Net1 includes a 6-layer network in series. The 1st, 3rd and 5th layers are convolutional layers. The number of convolution kernels is 4, 8 and 16 respectively. The size of convolution kernel is 3x3, the step size is 1 and the filling method is , the activation function uses ReLU; the 2nd, 4th, and 6th layers are pooling layers, all of which use the maximum pooling method, the pooling kernel size is 2x2, the step size is 2, and the filling method is ; The Net1 input is a 640x640 sample image X, and the output is a 640x640x16 high-dimensional feature X1. ; The characteristic three-pyramid network Net2 is a parallel connection of three convolutional network branches Net21, Net22, and Net23. Each branch contains 10 layers of networks in series. The structures are: the 1st, 2nd, 3rd, 5th, 6th, 7th, and 9th layers are convolutional layers, the number of convolution kernels are 16, 16, 64, 64, 128, and 256 respectively, the size of the convolution kernel is 3x3, the step size is 1, and the filling method is , the activation function uses ReLU; the 4th, 8th, and 10th layers are pooling layers, the pooling kernel size is 2x2, the step size is 2, and the filling method is , both use the maximum pooling method; The input of Net2 is the high-dimensional feature No. 1 X1, and the output is the high-dimensional feature No. 2 X2 of 80x80x256. ; in, For light color mask, is matrix multiplication, ; The feature fusion module Net3 is a 6-layer series network added at the end of the feature three-pyramid structure, where the 1st, 3rd, and 5th layers are deconvolution layers, the number of convolution kernels are 128, 64, and 16 respectively, the size of the convolution kernel is 3x3, the step size is 1, and the filling method is , ReLU is used as the activation function; the 2nd, 4th and 6th layers are anti-pooling layers, the pooling kernel size of the anti-pooling layers is 2x2, and the step size is 2; The Net3 input is the high-dimensional feature No. 2 X2, and the output is the high-dimensional feature No. 3 X3 of 640x640x16. .

[0025] The segmentation output head module Net4 contains 1 convolution layer, the number of convolution kernels is 1, the convolution kernel size is 1x1, the step size is 1, and the padding method is , use SoftMax activation function; The Net4 input is the No. 3 high-dimensional feature X3, and the output is a 640x640 single-channel result Y. .

[0026] Step S203, using the data packet Z obtained in step S201 to perform small sample training of the deep learning model, specifically includes the following five steps: Step B1, data enhancement of data packet Z, including random cropping, flipping, rotation, and scaling; obtaining m enhanced data packets K, m>>n, and the value range of m is [100, 1000]; 90% of the obtained m data packets K are used as training data K_train, and 10% are used as test data K_test; ; in, Indicates random cropping, flipping, rotation, and scaling of images. Indicates that the data set is divided proportionally, and the proportion is represented by the last two elements; Step B2, pre-training the model using existing public data sets, the public data set packages include: NEUsurface defect database, DAGM 2007; after pre-training, model D is obtained; Step B3, use K_train to fine-tune model D. The loss function of the training process uses cross entropy loss. The specific calculation formula is: ; Where N is the number of pixels in the image, 2 is the number of categories; i and j are pixel coordinate indexes; is the one-hot encoding of the labeled image in the data packet flattened into a row, indicating that the i-th element belongs to the j-th category, There are no scratches. Indicates the presence of scratches. is the prediction result output by the deep learning model, , the value represents the probability of scratches or no scratches, log is the logarithmic function; the optimizer for the training process uses the Adam optimizer, and the learning rate The range is: ; However, since the process here is a small sample training, the model is prone to overfitting, so regularization processing is required. The regularization processing uses the L2 regularization method: ; in, is the regularization factor, is the vectorized representation of all trainable parameters of model D, L is the loss function with regularization term; after training in step B3, model E is obtained; the model parameter updating method includes the following steps: Step B31, calculate all parameter gradients grad(w) of model D, all submodule parameter gradients are included in grad(w), and the parameter gradients of the three branches of the feature three-pyramid network are included in the parameter gradient of the feature three-pyramid: ; Among them, grad(w) represents the gradient of the function with respect to the variable w; Represents the partial derivative of the loss function L with respect to the parameter w; Step B32, update the parameters of each module: ; in, is the element-wise multiplication of matrices, It is a parameter matrix that can be manually adjusted according to the brightness of the light source. Here it is set: Represents the sum of all elements.

[0027] Step B4: Test the trained model E on the test data in the m data packets K. The evaluation indicators are the global average accuracy OA and the intersection over union (IoU). The calculation formula is as follows: ; Among them, TP is the number of pixels predicted to be scratches and actually are scratches, FN is the number of pixels that are not scratches but are not predicted, FP is the number of pixels that are scratches but are not predicted, and TN is the number of pixels that are not scratches but are predicted. In step B5, if the average test results of step B4 are OA>0.99 and IoU>0.95, the model training is considered complete and the subsequent stage is entered, otherwise return to step B3 for retraining.

[0028] In step S3, the recognition server receives the glass screen image sent by the image acquisition device and performs an image recognition operation, which specifically includes the following two steps: Step C1: send the images under red, green and blue light to the trained small sample deep learning model E in turn, and record the output of each time as a 640x640 matrix .

[0029] Step C2: Check the three 640x640 matrices recorded in step C1 one by one , output the scratch defect recognition result Result, the calculation method of Result is: ; in, Represents the sum of all elements of the matrix; True and False are fields of the glass screen image scratch defect recognition result information output by the recognition server.

[0030] Step S4, the display device receives the scratch defect recognition result sent by the recognition server and displays the received scratch defect recognition result, specifically in the following manner: After receiving the information of the glass screen scratch recognition result output by the recognition server, parse the field Result of the scratch defect recognition result in the information. If the field is "True", the screen displays red and prints the information "Scratch defect exists" on the screen; if the field is "False", the screen displays green and prints "No scratch defect" on the screen.

[0031] So far, according to the method disclosed in the present invention, the working process of the present invention has been implemented once.

[0032] The present invention provides a glass screen micro-scratch detection system based on small sample deep learning, comprising an image acquisition device, wherein the image acquisition device sequentially acquires glass screen images under three separate color lights of red, green and blue in batches to obtain three glass screen images, and sends the three glass screen images acquired in each batch to a recognition server; the recognition server is used to identify micro-scratch defects in the image; A light source device, wherein there are three light source devices, each of which is installed directly above a plane parallel to the plane where the glass screen is located, and the image acquisition device is installed at a position in the same plane as the three light source devices; the light source device is used to assist in image acquisition; a display device, wherein the display device receives the scratch defect recognition result sent by the recognition server and displays the received scratch defect recognition result; The light source devices are: three red, green and blue LED light sources with wavelengths of 620-630 / 515-530 / 460-475 nm, color temperatures of 1000-2000 / 3000-5000 / 6500-9000 K, and brightness of 100-200 / 400-800 / 100-200 mcd, respectively. The three LED light sources are placed in an equilateral triangle, and the image acquisition device is placed at the center of gravity of the equilateral triangle. The line connecting the red LED light source and the image acquisition device should be consistent with the movement direction of the glass screen. The side length of the equilateral triangle is 1 / 4. Range is ,in, is the height between the image acquisition device and the light source plane and the glass screen.

[0033] The image acquisition device is a microscope camera, preferably a high-resolution microscope camera, and the lens of the microscope camera is a microscope objective lens, and its viewing angle Range is .

[0034] It should be noted that the three glass screen images are all single-channel grayscale images. The light source position setting, image acquisition device position setting and viewing angle range selection make the direct position of the three-color light source within the image acquisition range but within the viewing angle. Outside the range, because the three-color light sources have different wavelengths and different refractive indices on the glass screen, their response capabilities to different defects are different. Therefore, the model can sense defect information at different levels, which is more conducive to the model learning the tiny defects of the glass screen.

[0035] The above-mentioned specific implementation can be partially adjusted in different ways by those skilled in the art without departing from the principle and purpose of the present invention. The protection scope of the present invention shall be based on the claims and shall not be limited by the above-mentioned specific implementation. Each implementation scheme within its scope shall be subject to the constraints of the present invention.

Claims

1. A method for detecting tiny scratches on a glass screen based on small sample deep learning, characterized in that: The following steps are involved: Step S1, using an image acquisition device to sequentially acquire glass screen images in batches under three separate color lights of red, green and blue, obtaining three glass screen images in each batch, and sending the three glass screen images acquired in each batch to a recognition server; Step S2, pre-collecting a certain amount of glass screen images with tiny scratches and without scratches and making sample patterns, and the recognition server training a small sample deep learning model according to the sample patterns to obtain a trained small sample deep learning model; Step S3, the recognition server receives the three glass screen images collected by the image acquisition device in each batch, and performs image recognition operations on the three glass screen images collected in each batch through the trained small sample depth model to generate scratch defect recognition results; Step S4, receiving the scratch defect recognition result sent by the recognition server through the display device, and displaying the received scratch defect recognition result.

2. The method for detecting micro scratches on a glass screen based on small sample deep learning according to claim 1, characterized in that: The step S2 specifically includes the following steps: Step S201, manually inspect and classify a certain amount of pre-collected glass screen images with tiny scratches and without scratches, sort out images including 70% with tiny scratches and 30% without scratches, and make sample patterns; collect n sample images, where the value range of n is [10, 30]; make each sample image into a data packet Z, including the following two steps: Step A1, crop the sample image to obtain a standard sample image X of size 640x640, and use the labelme annotation tool to annotate it to obtain the annotated image P: ； in, is the element in the i-th row and j-th column of the labeled image P, To label the classification of the element in the i-th row and j-th column of the sample image, For the scratch area, This means that the pixel is a scratch; Indicates that this pixel is not a scratch; Step A2, making the labeled sample image X and the labeled image P into a data package Z, Z={X,P}; Step S202, building a small sample deep learning model; Step S203, using the data packet Z obtained in step S201 to train the small sample deep learning model, to obtain a trained small sample deep model.

3. The method for detecting micro scratches on a glass screen based on small sample deep learning according to claim 2, characterized in that: The step S202 is specifically as follows: The small sample deep learning model includes a feature extraction module Net1, a feature three-pyramid network Net2, a feature fusion module Net3 and a segmentation output head module Net4; The feature extraction module Net1 includes a 6-layer network in series, and the structures are: the 1st, 3rd, and 5th layers are convolutional layers, the number of convolution kernels are 4, 8, and 16 respectively, the convolution kernel size is 3x3, the step size is 1, and the filling method is , using the ReLU activation function; the 2nd, 4th, and 6th layers are pooling layers, using the maximum pooling method, the pooling kernel size is 2x2, the step size is 2, and the filling method is "same"; The Net1 input is a 640x640 sample image X, and the output is a 640x640x16 high-dimensional feature X1. ； The characteristic three-pyramid network Net2 is a parallel connection of three convolutional network branches Net21, Net22, and Net23. Each branch contains 10 layers of networks in series. The structures are: the 1st, 2nd, 3rd, 5th, 6th, 7th, and 9th layers are convolutional layers, the number of convolution kernels are 16, 16, 64, 64, 128, and 256 respectively, the size of the convolution kernel is 3x3, the step size is 1, and the filling method is , using the ReLU activation function; the 4th, 8th, and 10th layers are pooling layers, with a pooling kernel size of 2x2, a step size of 2, and a padding method of ;, using the maximum pooling method; The input of Net2 is the high-dimensional feature No. 1 X1, and the output is the high-dimensional feature No. 2 X2 of 80x80x256. ； in, For light color mask, is matrix multiplication; ； The feature fusion module Net3 is a 6-layer series network added at the end of the feature three-pyramid network structure, and the structures are: the 1st, 3rd, and 5th layers are deconvolution layers, the number of convolution kernels are 128, 64, and 16 respectively, the convolution kernel size is 3x3, the step size is 1, the filling mode is "same", and the ReLU activation function is used; the 2nd, 4th, and 6th layers are anti-pooling layers, and the pooling kernel size of the anti-pooling layer is 2x2, and the step size is 2; The Net3 input is the high-dimensional feature No. 2 X2, and the output is the high-dimensional feature No. 3 X3 of 640x640x16. ； The segmentation output head module Net4 contains 1 convolution layer, the number of convolution kernels is 1, the convolution kernel size is 1x1, the step size is 1, and the padding method is , use SoftMax activation function; The Net4 input is the No. 3 high-dimensional feature X3, and the output is a 640x640 single-channel result Y. 。 4. The method for detecting micro scratches on a glass screen based on small sample deep learning according to claim 3, characterized in that: The step S203 specifically includes the following steps: Step B1, perform data enhancement processing on the data packets to obtain m enhanced data packets K, where m>>n, and the value range of m is [100, 1000]. 90% of the obtained m data packets K are used as training data K_train, and 10% are used as test data K_test. ； in, Indicates random cropping, flipping, rotation, and scaling of images. Indicates that the data set is divided proportionally, and the proportion is represented by the last two elements; Step B2, using the existing public data set to pre-train the small sample deep learning model to obtain model D; Step B3, use K_train to fine-tune model D. The loss function of the training process uses cross entropy loss. The specific calculation formula is: ； Among them, Loss_cls is the loss function; N is the number of pixels in the image, 2 is the number of categories; i and j are pixel coordinate indexes; is the one-hot encoding of the labeled image in the data packet flattened into a row, indicating that the i-th element belongs to the j-th category, There are no scratches. Indicates the presence of scratches. is the prediction result output by the deep learning model, , the value represents the probability of the existence or non-existence of scratches; log is the logarithmic function; learning rate The range is: ； The model D is regularized by the L2 regularization method to obtain a small sample deep learning model E. The L2 regularization method is: ； in, is the regularization factor, is the vectorized representation of all trainable parameters of model D; L is the loss function with regularization term; Step B4: Test the small sample deep learning model E on K_test. The evaluation indicators are global average accuracy OA and intersection over union (IoU). The calculation formula is as follows: ； Among them, TP is the number of pixels predicted to be scratches and actually are scratches, is the number of pixels that are neither scratches nor predicted scratches, is the number of pixels that are actually scratches but predicted not to be scratches, is the number of pixels that are not scratches but are predicted to be scratches; Step B5: If the test results of step B4 are OA>0.99 and IoU>0.95, the model training is considered to be completed, and a trained small sample deep learning model is obtained, then the subsequent stage is entered, otherwise return to step B3 for retraining.

5. The method for detecting micro scratches on a glass screen based on small sample deep learning according to claim 4, characterized in that: In step B3, model D is fine-tuned, which specifically includes the following steps: Step B31, calculate all parameter gradients grad(w) of model D, the parameter gradients of all submodules of model D are included in grad(w), and the parameter gradients of the three branches of the feature three-pyramid network are included in the parameter gradients of the feature three-pyramid: ； Among them, grad(w) represents the gradient of the function with respect to the variable w; Represents the partial derivative of the loss function L with respect to the parameter w; Step B32, update the parameters of each module: ； in, is the element-wise multiplication of matrices, It is a parameter matrix that can be manually adjusted according to the brightness of the light source, and the adjustment range is: Represents the sum of all elements.

6. The method for detecting micro scratches on a glass screen based on small sample deep learning according to claim 5, characterized in that: The step S3 specifically includes the following two steps: Step C1: send the glass screen images under red, green and blue light to the trained small sample deep learning model E in turn, perform image recognition operations, and record the output obtained each time as a 640x640 matrix ; Step C2, check the matrix one by one , generate the scratch defect recognition result Result, the calculation method of Result is: ； Among them, True and False are fields of the glass screen image scratch defect recognition result information output by the recognition server.

7. The method for detecting micro scratches on a glass screen based on small sample deep learning according to claim 6, characterized in that: The specific method of step S4 is: After receiving the glass screen image scratch defect recognition result information output by the recognition server, parse the scratch defect recognition result field Result in the information; If the field is "True", the screen will be displayed in red and the message "Scratch defect exists" will be printed on the screen; If the field is "False", the screen appears green and "No Scratch Defects" is printed on the screen.

8. A glass screen micro-scratch detection system based on small sample deep learning, characterized in that: The system comprises an image acquisition device, which acquires glass screen images in batches under three separate colors of red, green and blue, obtains three glass screen images, and sends the three glass screen images acquired in each batch to a recognition server; the recognition server is used to identify tiny scratch defects in the image; A light source device, wherein there are three light source devices, each of which is installed directly above a plane parallel to the plane where the glass screen is located, and the image acquisition device is installed at a position in the same plane as the three light source devices; the light source device is used to assist in image acquisition; a display device, wherein the display device receives the scratch defect recognition result sent by the recognition server and displays the received scratch defect recognition result; A glass screen micro-scratch detection system based on small sample deep learning is implemented as described in any one of claims 1-7.

9. The glass screen micro-scratch detection system based on small sample deep learning according to claim 8, characterized in that: The light source devices are: three red, green and blue LED light sources with wavelengths of 620-630 / 515-530 / 460-475 nm, color temperatures of 1000-2000 / 3000-5000 / 6500-9000 K, and brightness of 100-200 / 400-800 / 100-200 mcd, respectively. The three LED light sources are placed in an equilateral triangle, and the image acquisition device is placed at the center of gravity of the equilateral triangle. The line connecting the red LED light source and the image acquisition device should be consistent with the movement direction of the glass screen. The side length of the equilateral triangle is 1 / 4. Range is ,in, is the height between the image acquisition device and the light source plane and the glass screen.

10. The glass screen micro-scratch detection system based on small sample deep learning according to claim 9, characterized in that: The image acquisition device is a microscope camera, and the lens of the microscope camera is a microscope objective lens, and its viewing angle Range is .

Images