Mobile phone glass cover plate visual inspection optimization method and system for edge device

Abstract

The invention relates to the technical field of mobile phone glass cover plate defect visual inspection, in particular to an edge device-oriented mobile phone glass cover plate visual inspection optimization method and system, and the method comprises the steps: building a student model according to the performance information of a target edge device, and determining a target distillation temperature according to the difference of a plurality of target defects; and performing knowledge distillation on a preset teacher model based on the target distillation temperature and training a student model to obtain a trained student model, and finally deploying the trained student model in the target edge performance equipment. Compared with the prior art, the method has the advantages that the student model conforming to the operational capability of the target edge equipment is established according to the performance information of the target edge equipment, then the student model is trained through the knowledge distillation technology to be close to the performance of the preset teacher model, and the temperature during knowledge distillation is controlled according to the difference of multiple target defects in the distillation process; the problem of how to deploy a deep learning machine vision detection model of a mobile phone glass cover plate to edge equipment is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of visual inspection of defects in mobile phone glass cover plates, and in particular to a method and system for optimizing visual inspection of mobile phone glass cover plates for edge devices. Background Art

[0002] With the development of deep learning technology, machine vision inspection has been widely used in the field of defect detection for mobile phone glass cover panels. Deep learning machine vision inspection technology primarily relies on the image recognition capabilities of neural networks. By training on a large number of defect and non-defect samples, the model learns to automatically distinguish and identify defect types. Currently, most defect detection systems send collected image data to a server for analysis and processing. While this approach achieves high detection accuracy, it also increases the difficulty of deploying mobile phone glass cover panel production lines because it requires a stable network connection and sufficient data transmission bandwidth, as well as the maintenance of expensive servers, which is costly.

[0003] Edge devices refer to devices or servers that process data close to the data source. These devices can be industrial gateways, smart cameras, or other computing-capable devices. Deploying deep learning models on edge devices can significantly reduce data transmission latency, reduce reliance on network bandwidth, and improve data security and privacy. This allows defect detection to be performed directly on the production line without sending data to a remote server, reducing the complexity and cost of production line deployment.

[0004] However, the computing power of edge devices is often inferior to that of servers, and the defect recognition model running on the server may not be able to run on edge devices. Therefore, people need a visual inspection optimization method for mobile phone glass cover plates for edge devices to realize deep learning machine vision inspection of edge devices. Summary of the Invention

[0005] Therefore, the present invention provides a mobile phone glass cover plate visual inspection optimization method and system for edge devices, which is used to solve the problem of how to deploy the deep learning machine vision inspection model of mobile phone glass cover plates on edge devices in the prior art.

[0006] The present invention provides a method for optimizing visual inspection of mobile phone glass cover plates for edge devices, comprising:

[0007] Obtain performance information of target edge devices and defect information of multiple target defects;

[0008] Based on the performance information, a student model is built;

[0009] Obtain a preset teacher model and preset training data, where both the preset teacher model and the student model are neural network models for detecting defects in mobile phone glass cover plates based on images;

[0010] determining a target distillation temperature based on differences in multiple target defects represented by multiple defect information;

[0011] Based on the target distillation temperature, the preset teacher model is subjected to knowledge distillation according to the preset training data and the student model is trained to obtain the trained student model;

[0012] Deploy the trained student model on the target edge performance device.

[0013] The present invention also provides a preferred solution: the defect information includes defect sample images, each defect sample image corresponds to a target defect; and the target distillation temperature is determined based on the differences between multiple target defects represented by the multiple defect information, including:

[0014] Analyze the similarity between each two defect sample images and obtain multiple similar feature values;

[0015] A target distillation temperature is obtained according to the distribution of the multiple similar eigenvalues, wherein the more similar target defects represented by the multiple similar eigenvalues, the lower the target distillation temperature.

[0016] The present invention also provides a preferred solution: analyzing the similarity of each pair of defect sample images to obtain multiple similarity feature values, including:

[0017] Obtain two target defect sample images, where the target defect sample images are the two defect sample images whose similarity is to be analyzed;

[0018] Convert the two target defect sample images into two image vectors respectively. Each element in the image vector represents a pixel value in the target defect sample image. Elements at the same position in the two image vectors correspond to the same pixel position.

[0019] Calculate the cosine similarity of the two image vectors and obtain the similarity feature values ​​of the two target defect sample images based on the cosine similarity.

[0020] The present invention also provides a preferred solution: the larger the value of the similarity feature value, the more similar the two corresponding defect sample images are; and the target distillation temperature is obtained according to the distribution of multiple similarity feature values, including:

[0021] Count the total number of types of multiple target defects;

[0022] Calculate the sum of multiple similar eigenvalues;

[0023] Calculate the statistical eigenvalues ​​of multiple similar eigenvalues, where the statistical eigenvalues ​​are used to represent the degree of dispersion of the multiple similar eigenvalues;

[0024] A target distillation temperature is obtained based on the total number of types of multiple target defects, the sum of multiple similar eigenvalues, and the statistical eigenvalues ​​of the multiple similar eigenvalues, wherein the target distillation temperature is inversely proportional to the total number of types of multiple target defects, inversely proportional to the sum of multiple similar eigenvalues, and directly proportional to the degree of discreteness represented by the statistical eigenvalues ​​of the multiple similar eigenvalues.

[0025] The present invention also provides a preferred solution: based on the target distillation temperature, knowledge distillation is performed on a preset teacher model according to preset training data and a student model is trained to obtain a trained student model, including:

[0026] Obtain a historical application model, which is a neural network model that has been actually used in the past to detect defects in mobile phone glass cover plates based on images;

[0027] Based on the target distillation temperature, a first reconstruction loss function is established between the preset teacher model and the student model;

[0028] Based on the target distillation temperature, a second reconstruction loss function is established between the historical application model and the student model;

[0029] Establish the supervised loss function of the student model;

[0030] Establishing a target loss function according to the first reconstruction loss function, the second reconstruction loss function and the supervision loss function;

[0031] Based on the target loss function, the student model is trained according to the preset training data to obtain the trained student model.

[0032] The present invention also provides a preferred solution: establishing a target loss function based on the first reconstruction loss function, the second reconstruction loss function and the supervision loss function, including:

[0033] The overlap in the types of defects that the statistical history application model and the student model need to identify;

[0034] According to the degree of overlap of categories, a weight adjustment parameter is determined. The weight adjustment coefficient is used to adjust the influence ratio of the second reconstruction loss function on the target loss function.

[0035] Establish the target loss function based on the preset weights and weight adjustment coefficients.

[0036] The present invention also provides a preferred solution: the target loss function is:

[0037]

[0038] Among them, L is the target loss function, is the first reconstruction loss function, is the second reconstruction loss function, L hard is the supervised loss function, α and β are different preset weights, ε is the weight adjustment parameter, the degree of overlap represented by the category overlap is inversely proportional to the weight adjustment parameter, and when the category overlap takes the maximum value, the weight adjustment parameter is zero.

[0039] The present invention also provides a preferred solution: establishing a student model based on performance information, including:

[0040] Based on the performance information, obtain the computing power level characteristic value of the target edge device;

[0041] According to the computing power level characteristic value, the number of fully connected layers in the student model and the number of neurons in each fully connected layer are obtained;

[0042] A student model is established based on the number of fully connected layers and the number of neurons in each fully connected layer.

[0043] The present invention also provides a preferred solution: according to the computing power level characteristic value, the number of fully connected layers in the student model and the number of neurons in each fully connected layer are obtained, including:

[0044] Obtain a preset computational complexity characteristic function, which is used to obtain a computational complexity characteristic value based on the number of fully connected layers and the number of neurons in each fully connected layer. The computational complexity characteristic value is used to characterize the computational complexity of the student model established based on the number of fully connected layers and the number of neurons;

[0045] Establishing a fitness function based on the difference between the computing power level characteristic value and the preset computing capacity characteristic function;

[0046] Taking the minimum difference between the eigenvalue of computing power level and the eigenvalue of computational volume as the optimization goal, the number of fully connected layers in the student model and the number of neurons in each fully connected layer are obtained based on the preset optimization algorithm.

[0047] The present invention also provides a mobile phone glass cover visual inspection optimization system for edge devices, comprising:

[0048] A data input module, used to obtain performance information of target edge devices and defect information of multiple target defects;

[0049] A model building module is used to build a student model based on the performance information;

[0050] A data preparation module is used to obtain a preset teacher model and preset training data, where both the preset teacher model and the student model are neural network models based on image detection of defects in mobile phone glass cover plates;

[0051] a distillation preparation module, configured to determine a target distillation temperature based on differences in multiple target defects represented by multiple defect information;

[0052] The model optimization module is used to perform knowledge distillation on the preset teacher model based on the target distillation temperature and the preset training data and train the student model to obtain the trained student model;

[0053] The model deployment module is used to deploy the trained student model on the target edge performance device.

[0054] The beneficial effects of adopting the above embodiment are:

[0055] The present invention provides a method and system for optimizing visual inspection of mobile phone glass cover plates for edge devices. The method first obtains performance information of the target edge device and defect information of multiple target defects, then establishes a student model based on the performance information, then obtains a preset teacher model and preset training data, and then determines a target distillation temperature based on the differences between multiple target defects represented by the multiple defect information. Based on the target distillation temperature, the preset teacher model is subjected to knowledge distillation and a student model is trained according to the preset training data to obtain a trained student model, and finally the trained student model is deployed in the target edge performance device. Compared with the prior art, the present invention establishes a student model that meets the computing power of the target edge device based on its performance information, and then trains the student model through knowledge distillation technology to make it close to the performance of the preset teacher model. The temperature during knowledge distillation is controlled according to the differences between multiple target defects during the distillation process, thereby achieving efficient and accurate optimization of the preset teacher model for the target edge device and actual production conditions, and solving the problem of how to deploy a deep learning machine vision inspection model for mobile phone glass cover plates on an edge device. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 A method flow chart of an embodiment of a method for optimizing visual inspection of mobile phone glass cover plates for edge devices provided by the present invention;

[0057] Figure 2 for Figure 1 Specific step diagram of step S104;

[0058] Figure 3 for Figure 1 Specific step diagram of step S105;

[0059] Figure 4 A schematic diagram of the knowledge distillation principle of an embodiment of the mobile phone glass cover plate visual inspection optimization method for edge devices provided by the present invention;

[0060] Figure 5This is a system structure diagram of an embodiment of a mobile phone glass cover visual inspection optimization system for edge devices provided by the present invention. DETAILED DESCRIPTION

[0061] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0062] Combine Figure 1 As shown, a specific embodiment of the present invention discloses a mobile phone glass cover visual inspection optimization method for edge devices, including:

[0063] S101, obtaining performance information of a target edge device and defect information of multiple target defects;

[0064] S102. Establish a student model based on the performance information;

[0065] S103, obtaining a preset teacher model and preset training data, wherein the preset teacher model and the student model are both neural network models for detecting defects in mobile phone glass cover plates based on images;

[0066] S104, determining a target distillation temperature based on differences in multiple target defects represented by multiple defect information;

[0067] S105. Based on the target distillation temperature, perform knowledge distillation on the preset teacher model according to the preset training data and train the student model to obtain a trained student model;

[0068] S106. Deploy the trained student model on the target edge performance device.

[0069] Compared with the existing technology, the present invention establishes a student model that meets the computing capabilities of the target edge device based on its performance information, and then trains the student model through knowledge distillation technology to make it close to the performance of the preset teacher model. During the distillation process, the temperature of knowledge distillation is controlled according to the differences in multiple target defects, thereby achieving efficient and accurate optimization of the preset teacher model for the target edge device and actual production conditions, and solving the problem of how to deploy the deep learning machine vision detection model of mobile phone glass cover on the edge device.

[0070] The target edge device in the above process refers to the device on which the deep learning machine vision inspection model is deployed. It can be any device with computing power, such as a newly deployed computer or smart camera in the mobile phone glass cover production line. The target defect is the defect on the mobile phone glass cover that the target edge device needs to detect, such as scratches, pits, stains, and chipped edges. Obviously, different edge devices may need to detect different defects depending on the actual situation.

[0071] Knowledge distillation is a model compression technique that allows a small, efficient model (the student model) to learn the knowledge of a large, complex model (the teacher model). This approach can improve the performance of the student model to approach that of the teacher model while maintaining computational efficiency.

[0072] The teacher model is a large, well-trained neural network. This model is typically trained on a large amount of data to achieve high accuracy. The teacher model serves as a source of "knowledge," meaning its output (including the predicted probability distribution) contains rich information that helps the student model learn the complex characteristics of the data. In this invention, the pre-set teacher model is a pre-trained model that accurately and comprehensively identifies defects in mobile phone glass cover plates.

[0073] The student model is a neural network with a simpler structure and fewer parameters. Its purpose is to learn from the teacher model so that it can achieve the same performance as the teacher model while maintaining a smaller model size. The student model is typically trained on the same dataset, but uses the teacher model's output as additional guidance.

[0074] Obviously, in the present invention, the specific structure of the student model needs to be determined based on the hardware capabilities of the target edge device and the actual work requirements. In practice, the structure of the student model can be determined by any existing method as needed. For example, the structure of the student model can be determined manually based on experience based on factors such as the hardware parameters of the target edge device, the process location, and the number of tasks during actual operation. Alternatively, a more complex method can be used to prune the preset teacher model based on the above factors to obtain the structure of the student model.

[0075] The present invention provides a preferred method. In a preferred embodiment, the above step S102, establishing a student model based on performance information, specifically includes:

[0076] Based on the performance information, obtain the computing power level characteristic value of the target edge device;

[0077] According to the computing power level characteristic value, the number of fully connected layers in the student model and the number of neurons in each fully connected layer are obtained;

[0078] A student model is established based on the number of fully connected layers and the number of neurons in each fully connected layer.

[0079] In practice, the YOLO model is generally used for defect detection. When optimizing the YOLO model to adapt to devices with different computing power, adjusting the fully connected layer is the most effective method. This is because the fully connected layer contains a large number of parameters, which accounts for the majority of the model parameters. By reducing the number of neurons in these layers, the computational complexity and parameter volume of the model can be significantly reduced, thereby reducing the demand for computing power. In addition, the adjustment of the fully connected layer has little effect on the feature extraction ability of the model, which means that it can be optimized without sacrificing accuracy and without changing the model input size. Optimizing the fully connected layer can also speed up the inference speed of the model on resource-constrained devices, and it is relatively simple to implement, without requiring complex modifications to the convolutional layers. Therefore, adjusting the fully connected layer is the preferred strategy for optimizing preset teacher models (especially the YOLO model) on different devices.

[0080] In this embodiment, the operation of the target edge device can be quantitatively represented by a computing power level characteristic value (for example, a performance score value is calculated by any existing method based on the processor frequency, memory size and other parameters of the target edge device, as a computing power level characteristic value). In this way, the scale of the fully connected layer can be adjusted based on the computing power level characteristic value. For example, a value that is less than the number of fully connected layers in the preset teacher model is arbitrarily taken as the number of fully connected layers, and a linear transformation is performed on the computing power level characteristic value to obtain a maximum value of the total number of neurons that the target edge device can theoretically bear, and then this maximum value is evenly distributed to these fully connected layers to obtain the number of neurons in each fully connected layer.

[0081] Clearly, the above process requires determining both the number of fully connected layers and the number of neurons, effectively creating a multi-objective optimization problem. Therefore, employing optimization algorithms (such as genetic algorithms and particle swarm optimization) can more effectively address the student model's structural determination. While the previously described method only yields an equal number of neurons in each fully connected layer, optimization algorithms offer a more flexible approach to achieving structures with varying numbers of neurons per fully connected layer.

[0082] Specifically, in a preferred embodiment, the above step of obtaining the number of fully connected layers and the number of neurons in each fully connected layer in the student model according to the computing power level characteristic value specifically includes:

[0083] Obtain a preset computational complexity characteristic function, which is used to obtain a computational complexity characteristic value based on the number of fully connected layers and the number of neurons in each fully connected layer. The computational complexity characteristic value is used to characterize the computational complexity of the student model established based on the number of fully connected layers and the number of neurons;

[0084] Establishing a fitness function based on the difference between the computing power level characteristic value and the preset computing capacity characteristic function;

[0085] Taking the minimum difference between the eigenvalue of computing power level and the eigenvalue of computational volume as the optimization goal, the number of fully connected layers in the student model and the number of neurons in each fully connected layer are obtained based on the preset optimization algorithm.

[0086] The preset computational characteristic function in the above process can be flexibly determined according to the specific situation. For example, the total number of parameters of the fully connected layer part in the current learning model can be calculated based on the number of fully connected layers and the number of neurons in each fully connected layer, and then a relatively simple linear change is made to it as the computational complexity of the current learning model, that is, the computational characteristic value. In this way, by using the difference between the computing power level characteristic value and the preset computational characteristic function as the fitness function, the structure of the student model that best meets the computing power of the target edge device can be obtained, thereby maximizing the performance of the target edge device. It can be understood that the optimization algorithm is a prior art. Based on the content described above, how to encode the number of fully connected layers and the number of neurons in each fully connected layer, as well as the specific type of the preset optimization algorithm, the specific optimization process of the preset optimization algorithm are all understandable and conceived by those skilled in the art, so this article will not explain it in detail.

[0087] Furthermore, the distillation temperature is an important hyperparameter in the knowledge distillation process, which is used to adjust the "softness" of the soft targets output by the teacher model. In knowledge distillation, the output of the teacher model (usually a probability distribution) is used as the target for training the student model, which is called a soft target. The temperature parameter affects the refinement of the knowledge learned by the student model by adjusting the smoothness of the probability distribution of the soft target. Generally speaking, a lower temperature will make the soft target closer to the hard target (that is, the theoretically true and accurate recognition result), which may cause the student model to focus too much on the prediction details of the teacher model, including noise. A higher temperature will make the soft target smoother, which helps the student model learn more general features instead of relying too much on the specific predictions of the teacher model.

[0088] It is worth emphasizing that, unlike conventional knowledge distillation technology, in the present invention, the distillation temperature is determined according to the degree of difference of multiple target defects. This can make the learning of the student model in the present invention more targeted, and can more accurately extract useful data from the preset teacher model, making training more efficient and accurate.

[0089] For example, if the actual defects that the student model needs to identify are black spots, pits, and stains, and the binary images of these three defects are relatively similar, the difference in the probability values ​​corresponding to the three defects in the soft target output by the preset teacher model may be small. In this case, a lower distillation temperature can be used for distillation to amplify the difference in the probability values ​​corresponding to the three defects in the soft target, so that the student model is more inclined to learn the ability to analyze details in the preset teacher model. Conversely, if the actual defects that the student model needs to identify are black spots, orange spots, and scratches, and the binary images of these three defects are quite different, the difference in the probability values ​​corresponding to the three defects in the soft target output by the preset teacher model may be obvious enough. In this case, a higher distillation temperature can be used for distillation to make the student model more inclined to comprehensively learn the most generalized abilities of the teacher model.

[0090] Specifically, combined Figure 2 As shown, in a preferred embodiment, the defect information includes defect sample images, each defect sample image corresponds to a target defect. On this basis, the above step S104, determining the target distillation temperature according to the differences of the multiple target defects represented by the multiple defect information, specifically includes:

[0091] S201, analyzing the similarity of each pair of defect sample images to obtain multiple similarity feature values;

[0092] S202 . Obtain a target distillation temperature based on the distribution of the multiple similar eigenvalues, wherein the more similar target defects represented by the multiple similar eigenvalues, the lower the target distillation temperature.

[0093] In the above process, similar feature values ​​are used to represent the difference between two defect sample images to quantify the difference between the two target defects. At the same time, similar feature values ​​of each two target defects are calculated pairwise. In this way, the differences between all target defects can be quantified as a whole based on multiple similar feature values, so that this embodiment can adapt to the situations of more target defects.

[0094] In the above process, the similarity between each two defect sample images can be analyzed using any existing method. For example, the two defect sample images can be subtracted and superimposed, and then the resulting image can be analyzed. If the values ​​of most pixels in the resulting image are very small, the two images can be considered very similar. Alternatively, other existing methods such as the Structural Similarity Index (SSIM), Peak Signal-to-Noise Ratio (PSNR), and histogram comparison can be used.

[0095] The present invention provides a preferred method for analyzing the similarity between each two defect sample images. Specifically, in a preferred embodiment, the above step S201, analyzing the similarity between each two defect sample images, and obtaining multiple similarity feature values, specifically includes:

[0096] Obtain two target defect sample images, where the target defect sample images are the two defect sample images whose similarity is to be analyzed;

[0097] Convert the two target defect sample images into two image vectors respectively. Each element in the image vector represents a pixel value in the target defect sample image. Elements at the same position in the two image vectors correspond to the same pixel position.

[0098] Calculate the cosine similarity of the two image vectors and obtain the similarity feature values ​​of the two target defect sample images based on the cosine similarity.

[0099] This embodiment uses cosine similarity to measure the difference between two target defect sample images. Its advantages are: by converting the image into a vector, cosine similarity can retain the spatial position information of the pixel, which helps the computer understand the local pattern and structure in the image. At the same time, the calculation of cosine similarity is relatively simple. For large-scale image data sets, its computational efficiency is high because it mainly involves dot product and modulus calculation. In addition, cosine similarity is particularly suitable for high-dimensional data and sparse data. For example, when facing the feature vector of the image, it will not be affected by the dimensionality curse. When facing defect sample images with most pixel values ​​​​of 0, cosine similarity can provide better performance than distance-based methods. The above advantages make cosine phase velocity particularly suitable for the application scenarios of the present invention.

[0100] Similarly, in addition to analyzing the similarity between each pair of defect sample images, the distribution of multiple similar feature values ​​can also be analyzed using any existing method. For example, assuming that larger similar feature values ​​represent more similarity between the two images, the proportion of similar feature values ​​exceeding a certain threshold can be directly counted as the distribution of the multiple similar feature values. The larger the proportion, the more similar target defects there are.

[0101] The present invention also provides a preferred method for analyzing the distribution of multiple similar feature values. Specifically, in a preferred embodiment, the larger the value of the similar feature value, the more similar the two corresponding defect sample images are. On this basis, the above step S202, obtaining the target distillation temperature according to the distribution of the multiple similar feature values, specifically includes:

[0102] Count the total number of types of multiple target defects;

[0103] Calculate the sum of multiple similar eigenvalues;

[0104] Calculate the statistical eigenvalues ​​of multiple similar eigenvalues, where the statistical eigenvalues ​​are used to represent the degree of dispersion of the multiple similar eigenvalues;

[0105] A target distillation temperature is obtained based on the total number of types of multiple target defects, the sum of multiple similar eigenvalues, and the statistical eigenvalues ​​of the multiple similar eigenvalues, wherein the target distillation temperature is inversely proportional to the total number of types of multiple target defects, inversely proportional to the sum of multiple similar eigenvalues, and directly proportional to the degree of discreteness represented by the statistical eigenvalues ​​of the multiple similar eigenvalues.

[0106] The significance of the above process lies in analyzing the distribution of multiple similar eigenvalues ​​across three dimensions: the total number of target defect types, the sum of multiple similar eigenvalues, and the statistical eigenvalues ​​of these similar eigenvalues, to obtain a more scientific and reasonable target distillation temperature. The greater the total number of target defect types, the more content the student model learns. In this case, the target distillation temperature should be reduced to increase the differentiation of the probability values ​​corresponding to different defects in the soft target. The sum of multiple similar eigenvalues ​​is used to characterize the degree of similarity among multiple target defects as a whole, while the statistical eigenvalues ​​of multiple similar eigenvalues ​​are used to further analyze the similarity between multiple similar eigenvalues, that is, to analyze the similarity between the similarities of different defects. The statistical eigenvalues ​​can be represented in any manner depending on the specific situation. For example, the variance of multiple similar eigenvalues ​​can be directly used as the statistical eigenvalue, or multiple statistical features such as the mean, median, and standard deviation of multiple similar eigenvalues ​​can be calculated and weighted summed to obtain the statistical eigenvalue.

[0107] A simple example of calculating the target distillation temperature is as follows:

[0108]

[0109] Among them, S represents the target distillation temperature, n represents the total number of types of multiple target defects, sum represents the sum of multiple similar eigenvalues, sta represents the statistical eigenvalue of multiple similar eigenvalues, and w1, w2 and w3 are different preset influence weights.

[0110] Further, combined Figure 3 As shown, in a preferred embodiment, the above step S105, based on the target distillation temperature, performs knowledge distillation on the preset teacher model according to the preset training data and trains the student model to obtain the trained student model, specifically includes:

[0111] S301, obtaining a historical application model, where the historical application model is a neural network model that has been actually applied in the past for detecting defects in mobile phone glass cover plates based on images;

[0112] S302: Based on the target distillation temperature, establish a first reconstruction loss function between the preset teacher model and the student model;

[0113] S303. Establishing a second reconstruction loss function between the historical application model and the student model based on the target distillation temperature;

[0114] S304, establishing a supervised loss function for the student model;

[0115] S305: Establish a target loss function based on the first reconstruction loss function, the second reconstruction loss function, and the supervision loss function;

[0116] S306. Based on the target loss function, the student model is trained according to the preset training data to obtain a trained student model.

[0117] This embodiment is mainly used for the deployment of models in target edge devices when the products of a production line have changed and only the target edge device has been updated in the entire production line, while the overall production line remains unchanged. In the above process, the historical application model is a deep learning machine vision detection model that has been put into practical application before the preset teacher model is re-optimized and deployed on the target edge device. For example, if the factory equipment is upgraded in practice and a new target edge device is purchased to replace the original edge device, the defect detection model running in the original edge device is the historical application model.

[0118] Based on the existing knowledge distillation technology, this embodiment further adds the learning of historical application models in the distillation process, so that the student model can not only learn the knowledge of the preset teacher model, but also learn relevant knowledge in the historical application model that conforms to the actual situation (such as the impact of lighting conditions in the factory on the input image and output results, and the knowledge that certain specific defects are more likely to occur due to the design of the process flow, the factory environment, or the characteristics of the raw materials), making the learning model more accurate and practical.

[0119] In the prior art, the reconstruction loss function is used to measure the difference between the output of the student model and the soft target of the teacher model at a set distillation temperature, and the supervision loss function is used to measure the difference between the output of the student model and the true label at a normal temperature (i.e., the temperature value is 1). Of these, the true label, needless to say, has the same definition as any existing deep learning model, and is a vector or numerical value representing the correct output result. The soft target is a probability distribution obtained based on the preset teacher model, which is generally obtained at a certain distillation temperature. The prior art uses the reconstruction loss function and the supervision loss function to jointly train the student model to achieve the effect of learning from the teacher model.

[0120] In this embodiment, the first reconstruction loss function is the reconstruction loss function in the prior art. The definitions of the first reconstruction loss function and the supervision loss function in the present invention are the same as those in the knowledge distillation technology in the prior art, so this article will not explain them in detail. The second reconstruction loss function is an improvement of the present invention, which is used to incorporate historical application models into learning. Its concept is actually similar to the first reconstruction loss function. For example, an example of a second reconstruction loss function using the cross entropy function as a loss measurement method is as follows:

[0121]

[0122] Where T is the target distillation temperature, Represents the second reconstruction function, which can be understood as the loss of the soft target output of the learning model compared to the output of the historical application model under the condition of the target distillation temperature. i represents the serial number corresponding to the target defect. represents the value corresponding to the i-th target defect in the soft target output by the historical application model at the target distillation temperature (because the historical application model and the preset teacher model also play a teaching role, the probability distribution obtained according to the historical application model at a certain distillation temperature is also referred to as the soft target in this embodiment). It represents the value corresponding to the i-th target defect in the results output by the student model at the target distillation temperature, and log is the logarithmic sign.

[0123] Furthermore, the soft labels output by the historical application model are specifically:

[0124]

[0125] Among them, k also represents the serial number corresponding to the target defect, v k In the historical application model, the output before normalization (generally the output of the previous layer of the last softmax layer) corresponds to the logical value of the k-th target defect, and exp() represents the natural exponential function. The distillation based on the target distillation temperature is reflected in the above formula. When the target distillation temperature is 1, the above formula is a conventional softmax function, which plays a role in normalization. Adding the target distillation temperature is equivalent to further amplifying the difference between different logical values ​​during normalization, increasing the amount of information contained in the soft target, which is similar to "distillation" in chemistry. The principle of adding the historical application model to train the student model in this embodiment is as follows: Figure 4 shown.

[0126] Similarly, the historical application model may also have inconsistent target defects that need to be detected with the student model. Therefore, in a preferred embodiment, the above step S305, based on the first reconstruction loss function, the second reconstruction loss function and the supervision loss function, establishes the target loss function, specifically including:

[0127] The overlap in the types of defects that the statistical history application model and the student model need to identify;

[0128] According to the degree of overlap of categories, a weight adjustment parameter is determined. The weight adjustment coefficient is used to adjust the influence ratio of the second reconstruction loss function on the target loss function.

[0129] Establish the target loss function based on the preset weights and weight adjustment coefficients.

[0130] The significance of this embodiment is that the degree of overlap in the types of defects that the historical application model and the student model need to identify is used to adjust the proportion of the influence of the second reconstruction loss function on the target loss function, thereby obtaining a more accurate recognition result. For example, to take an extreme example, if the target defects that the historical application model and the student model need to detect are completely inconsistent, that is, the type overlap is 0. Then it can be considered that the historical application model has no reference value. At this time, the preset weight corresponding to the second reconstruction loss function should be adjusted to 0 to avoid meaningless calculations. The weight adjustment coefficient in this embodiment is used to adjust the preset weight corresponding to the second reconstruction loss function, which can be linearly related to the type overlap.

[0131] Specifically, in a preferred embodiment, the objective loss function is:

[0132]

[0133] Among them, L is the target loss function, is the first reconstruction loss function, is the second reconstruction loss function, L hard is the supervised loss function, α and β are different preset weights, ε is the weight adjustment parameter, the degree of overlap represented by the category overlap is inversely proportional to the weight adjustment parameter, and when the category overlap takes the maximum value, the weight adjustment parameter is zero.

[0134] The significance of the above formula is that when the historical application model and the student model have exactly the same target defects to be detected, the first reconstruction loss function and the second reconstruction loss function should have the same impact on the target loss function. That is, the student model should learn the same proportion of content from the historical application model and the preset teacher model. In this case, the weight adjustment parameter can be 0. However, as the overlap of categories decreases and the weight adjustment parameter increases, the impact of the first reconstruction loss function on the target loss function will gradually become greater than that of the second reconstruction loss function, and the student model will focus more on learning from the preset teacher model.

[0135] It is understandable that, except for the above-mentioned contents, the other steps in the knowledge distillation process of the present invention are the same as those in the prior art and will not be described in detail herein.

[0136] Combine Figure 5 As shown, the present invention also provides a mobile phone glass cover visual inspection optimization system for edge devices, comprising:

[0137] A data input module 510 is used to obtain performance information of a target edge device and defect information of multiple target defects;

[0138] a model building module 520 for building a student model based on the performance information;

[0139] A data preparation module 530 is used to obtain a preset teacher model and preset training data, wherein the preset teacher model and the student model are both neural network models for detecting defects in mobile phone glass cover plates based on images;

[0140] a distillation preparation module 540 for determining a target distillation temperature based on differences in multiple target defects represented by multiple defect information;

[0141] A model optimization module 550 is configured to perform knowledge distillation on a preset teacher model based on a target distillation temperature and preset training data, and train a student model to obtain a trained student model;

[0142] The model deployment module 560 is used to deploy the trained student model in the target edge performance device.

[0143] It should be noted here that the corresponding system provided in the above embodiments can implement the technical solutions described in the above method embodiments. The specific implementation principles of the above modules or units can be found in the corresponding contents in the above method embodiments, which will not be repeated here.

[0144] The present invention provides a method and system for optimizing visual inspection of mobile phone glass cover plates for edge devices. The method first obtains performance information of the target edge device and defect information of multiple target defects, then establishes a student model based on the performance information, then obtains a preset teacher model and preset training data, and then determines a target distillation temperature based on the differences between multiple target defects represented by the multiple defect information. Based on the target distillation temperature, the preset teacher model is subjected to knowledge distillation and a student model is trained according to the preset training data to obtain a trained student model, and finally the trained student model is deployed in the target edge performance device. Compared with the prior art, the present invention establishes a student model that meets the computing power of the target edge device based on its performance information, and then trains the student model through knowledge distillation technology to make it close to the performance of the preset teacher model. The temperature during knowledge distillation is controlled according to the differences between multiple target defects during the distillation process, thereby achieving efficient and accurate optimization of the preset teacher model for the target edge device and actual production conditions, and solving the problem of how to deploy a deep learning machine vision inspection model for mobile phone glass cover plates on an edge device.

[0145] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referenced to each other.

[0146] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A visual inspection optimization method for mobile phone glass cover plates for edge devices, characterized in that: include: Obtain performance information of target edge devices and defect information of multiple target defects; Based on the performance information, a student model is built; Obtain a preset teacher model and preset training data, wherein the preset teacher model and the student model are both neural network models for detecting defects in mobile phone glass cover plates based on images; Determining a target distillation temperature according to differences in a plurality of target defects represented by a plurality of defect information; Based on the target distillation temperature, perform knowledge distillation on the preset teacher model according to the preset training data and train the student model to obtain the trained student model; Deploy the trained student model on the target edge performance device.

2. The method for optimizing visual inspection of mobile phone glass cover plates for edge devices according to claim 1, characterized in that: The defect information includes defect sample images, each of which corresponds to a target defect; The target distillation temperature is determined based on the differences of multiple target defects characterized by multiple defect information, including: Analyze the similarity of each two defect sample images in pairs to obtain multiple similar feature values; According to the distribution of the multiple similar characteristic values, the target distillation temperature is obtained, wherein the more similar target defects represented by the multiple similar characteristic values, the lower the target distillation temperature.

3. The method for optimizing visual inspection of mobile phone glass cover plates for edge devices according to claim 2, characterized in that: Analyze the similarity of each pair of defect sample images and obtain multiple similar feature values, including: Obtain two target defect sample images, where the target defect sample images are the two defect sample images whose similarity is to be analyzed; The two target defect sample images are converted into two image vectors respectively. Each element in the image vector is a pixel value in the target defect sample image. The elements at the same position in the two image vectors correspond to the same pixel position. Calculate the cosine similarity of the two image vectors, and obtain the similar feature values ​​of the two target defect sample images based on the cosine similarity.

4. The method for optimizing visual inspection of mobile phone glass cover plates for edge devices according to claim 2, characterized in that: The larger the similarity feature value is, the more similar the two corresponding defect sample images are; According to the distribution of multiple similar characteristic values, the target distillation temperature is obtained, including: Count the total number of types of multiple target defects; Calculate the sum of multiple similar eigenvalues; Calculate the statistical eigenvalues ​​of multiple similar eigenvalues, where the statistical eigenvalues ​​are used to represent the degree of dispersion of the multiple similar eigenvalues; A target distillation temperature is obtained based on the total number of types of multiple target defects, the sum of multiple similar eigenvalues, and the statistical eigenvalues ​​of multiple similar eigenvalues, wherein the target distillation temperature is inversely proportional to the total number of types of multiple target defects, inversely proportional to the sum of multiple similar eigenvalues, and directly proportional to the degree of discreteness represented by the statistical eigenvalues ​​of multiple similar eigenvalues.

5. The method for optimizing visual inspection of mobile phone glass cover plates for edge devices according to claim 1, characterized in that: Based on the target distillation temperature, the preset teacher model is subjected to knowledge distillation according to the preset training data and the student model is trained to obtain the trained student model, including: Obtain a historical application model, which is a neural network model that has been actually used in the past to detect defects in mobile phone glass cover plates based on images; Based on the target distillation temperature, a first reconstruction loss function between a preset teacher model and a student model is established; Based on the target distillation temperature, a second reconstruction loss function between the historical application model and the student model is established; Establish the supervised loss function of the student model; Establishing a target loss function according to the first reconstruction loss function, the second reconstruction loss function and the supervision loss function; Based on the target loss function, the student model is trained according to the preset training data to obtain the trained student model.

6. The method for optimizing visual inspection of mobile phone glass cover plates for edge devices according to claim 5, characterized in that: According to the first reconstruction loss function, the second reconstruction loss function and the supervision loss function, a target loss function is established, including: The overlap between the types of defects that the statistical history application model and the student model need to identify; According to the category overlap, a weight adjustment parameter is determined, and the weight adjustment coefficient is used to adjust the influence ratio of the second reconstruction loss function on the target loss function; Establish the target loss function based on the preset weights and weight adjustment coefficients.

7. The method for optimizing visual inspection of mobile phone glass cover plates for edge devices according to claim 6, characterized in that: The objective loss function is: Among them, L is the target loss function, is the first reconstruction loss function, is the second reconstruction loss function, L hard is the supervised loss function, α and β are different preset weights, ε is the weight adjustment parameter, the degree of overlap represented by the category overlap is inversely proportional to the weight adjustment parameter, and when the category overlap takes the maximum value, the weight adjustment parameter is zero.

8. The method for optimizing visual inspection of mobile phone glass cover plates for edge devices according to claim 1, characterized in that: Based on the performance information, a student model is built, including: According to the performance information, the characteristic value of the computing power level of the target edge device is obtained; According to the computing power level characteristic value, the number of fully connected layers in the student model and the number of neurons in each fully connected layer are obtained; A student model is established based on the number of fully connected layers and the number of neurons in each fully connected layer.

9. The method for optimizing visual inspection of mobile phone glass cover plates for edge devices according to claim 8, characterized in that: According to the computing power level characteristic value, the number of fully connected layers in the student model and the number of neurons in each fully connected layer are obtained, including: Obtain a preset computational characteristic function, where the computational characteristic function is used to obtain a computational characteristic value according to the number of fully connected layers and the number of neurons in each fully connected layer, and the computational characteristic value is used to characterize the computational complexity of the student model established based on the number of fully connected layers and the number of neurons; Establishing a fitness function based on the difference between the characteristic value of the computing power level and the preset computing amount characteristic function; Taking the minimization of the difference between the eigenvalue of computing power level and the eigenvalue of computational volume as the optimization goal, the number of fully connected layers in the student model and the number of neurons in each fully connected layer are obtained based on the preset optimization algorithm.

10. A mobile phone glass cover visual inspection optimization system for edge devices, characterized in that: include: A data input module, used to obtain performance information of a target edge device and defect information of multiple target defects; A model building module, used to build a student model based on the performance information; A data preparation module is used to obtain a preset teacher model and preset training data, wherein the preset teacher model and the student model are both neural network models for detecting defects in mobile phone glass cover plates based on images; A distillation preparation module, used to determine a target distillation temperature according to differences of multiple target defects represented by multiple defect information; A model optimization module is used to perform knowledge distillation on a preset teacher model based on a target distillation temperature and preset training data and train a student model to obtain a trained student model; The model deployment module is used to deploy the trained student model in the target edge performance device.

Images