Training of an image quality detection model, image quality detection method and related apparatus

Abstract

Embodiments of the present application disclose a training method of an image quality detection model, an image quality detection method and related devices, which are applied to an artificial intelligence scene. The training method is to input a first training image into a preset encoder to obtain a first latent variable, the first training image having preset label data indicating that it is a defective image; to obtain a feature disturbance vector by disturbing the first latent variable; to input the feature disturbance vector into a preset decoder to reconstruct a first reconstructed image; to train a preset encoder and a preset fully connected layer to obtain a target encoder and a target fully connected layer by using the first latent variable, the first reconstructed image and the preset label data; and to use the target encoder and the target fully connected layer as the image quality detection model. It can be seen that the model is more fully trained, and the generalization performance of the model can be ensured and the image quality detection effect of the model can be improved under the condition of using a small amount of training images with label data.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to the training of an image quality detection model, an image quality detection method, and related apparatus. Background Technology

[0002] Product defect quality inspection refers to the quality inspection of products during the manufacturing process. With the rise and development of artificial intelligence technology, image quality inspection methods based on artificial intelligence technology refer to inputting an image of a product and outputting label data to determine whether the image is a defective image, thereby determining the quality inspection result of the image.

[0003] In related technologies, image quality detection methods based on convolutional neural networks are commonly used. Specifically, this involves extracting features from product images using the convolutional neural network structure in the image quality detection model, and then performing quality detection through the fully connected layer in the image quality detection model to detect the label data of the image and determine the quality detection result of the image.

[0004] However, the above methods require a large number of labeled training images to train the model, but it is difficult to collect a large number of labeled defect images as training images in product defect quality inspection scenarios. Therefore, using a small number of labeled defect images for training can easily lead to model overfitting and poor generalization performance, resulting in poor image quality detection performance of the model. Summary of the Invention

[0005] To address the aforementioned technical issues, this application provides a training method for an image quality detection model, an image quality detection method, and related apparatus, enabling the image quality detection model to be trained more thoroughly, ensuring the model's generalization performance, and thereby improving the model's image quality detection effect.

[0006] The embodiments of this application disclose the following technical solutions:

[0007] On the one hand, this application provides a training method for an image quality detection model, the method comprising:

[0008] The first training image is subjected to feature extraction by a preset encoder to obtain a first latent variable; the first training image has preset label data indicating that the first training image is a defective image;

[0009] The first latent variable is perturbed to obtain a feature perturbation vector;

[0010] The feature perturbation vector is reconstructed using a preset decoder to obtain a first reconstructed image;

[0011] Based on the first latent variable, the first reconstructed image, and the preset label data, the preset encoder and the preset fully connected layer are trained to obtain the target encoder and the target fully connected layer.

[0012] The target encoder and the target fully connected layer are defined as the image quality detection model.

[0013] On the other hand, this application provides an image quality detection method, the method comprising:

[0014] Acquire the image of the product to be inspected;

[0015] The fourth latent variable is obtained by extracting features from the image to be detected using the target encoder in the image quality detection model.

[0016] The fourth latent variable is quality-detected by the target fully connected layer in the image quality detection model to obtain the target detection data of the image to be detected.

[0017] The quality detection result of the image to be detected is determined based on the target detection data;

[0018] The image quality detection model is obtained by performing the training method for the image quality detection model described above.

[0019] On the other hand, this application provides a training device for an image quality detection model, the device comprising: a first feature extraction unit, a feature perturbation unit, a reconstruction unit, a training unit, and a first determination unit;

[0020] The first feature extraction unit is used to extract features from the first training image through a preset encoder to obtain a first latent variable; the first training image has preset label data indicating that the first training image is a defective image;

[0021] The feature perturbation unit is used to perturb the first latent variable to obtain a feature perturbation vector;

[0022] The reconstruction unit is used to reconstruct the feature perturbation vector through a preset decoder to obtain a first reconstructed image;

[0023] The training unit is used to train the preset encoder and the preset fully connected layer based on the first latent variable, the first reconstructed image and the preset label data, so as to obtain the target encoder and the target fully connected layer.

[0024] The first determining unit is used to determine the target encoder and the target fully connected layer as an image quality detection model.

[0025] As one possible implementation, the training unit is specifically used for:

[0026] The first reconstructed image is used to extract features through the preset encoder to obtain the second latent variable;

[0027] The first latent variable and the second latent variable are respectively subjected to quality detection through the preset fully connected layer to obtain the first detection data of the first training image and the second detection data of the first reconstructed image.

[0028] Based on the first detection data, the second detection data, the preset label data, and the first loss function, train the parameters of the preset encoder and the parameters of the preset fully connected layer;

[0029] The pre-set encoder that has been trained is determined as the target encoder, and the pre-set fully connected layer that has been trained is determined as the target fully connected layer.

[0030] As one possible implementation, the feature perturbation unit is specifically used for:

[0031] The first latent variable is subjected to multiple perturbations with different features to obtain multiple perturbation vectors with different features;

[0032] The step of reconstructing the feature perturbation vector using a preset decoder to obtain a first reconstructed image includes:

[0033] The preset decoder is used to reconstruct the multiple different feature perturbation vectors respectively to obtain multiple different first reconstructed images;

[0034] The step of training the preset encoder and preset fully connected layer based on the first latent variable, the first reconstructed image, and the preset label data to obtain the target encoder and target fully connected layer includes:

[0035] Based on the first latent variable, the multiple different first reconstructed images, and the preset label data, the preset encoder and the preset fully connected layer are trained to obtain the target encoder and the target fully connected layer.

[0036] As one possible implementation, the device further includes: a first enhancement unit;

[0037] The first enhancement unit is used to perform image enhancement on the first training image to obtain a first enhanced image of the first training image;

[0038] The first feature extraction unit is further configured to extract features from the first enhanced image using the preset encoder to obtain the first latent variable.

[0039] As one possible implementation, the first loss function includes a cross-entropy loss function or a relative entropy loss function.

[0040] As one possible implementation, the preset label data also indicates that the first training image is a defect image of a preset defect type.

[0041] As one possible implementation, the device further includes: a second enhancement unit;

[0042] The second enhancement unit is used to perform image enhancement on the second training image to obtain a second enhanced image of the second training image;

[0043] The first feature extraction unit is also used to extract features from the second enhanced image through the initial encoder to obtain the third latent variable;

[0044] The reconstruction unit is also used to perform reconstruction processing on the third latent variable through the initial decoder to obtain a second reconstructed image;

[0045] The training unit is also used to train the parameters of the initial encoder and the parameters of the initial decoder based on the second reconstructed image, the second enhanced image, and the second loss function;

[0046] The first determining unit is further configured to determine the initial encoder that has completed training as the preset encoder and the initial decoder that has completed training as the preset decoder.

[0047] As one possible implementation, the image enhancement processing method includes at least one of scaling, rotation, flipping, translation, color transformation, contrast transformation, and noise addition.

[0048] As one possible implementation, the second loss function includes a mean squared error loss function.

[0049] On the other hand, this application provides an image quality detection device, the device comprising: an acquisition unit, a second feature extraction unit, a detection unit, and a second determination unit;

[0050] The acquisition unit is used to acquire the image of the product to be inspected;

[0051] The second feature extraction unit is used to extract features from the image to be detected through the target encoder in the image quality detection model to obtain the fourth latent variable;

[0052] The detection unit is used to perform quality detection on the fourth latent variable through the target fully connected layer in the image quality detection model to obtain the target detection data of the image to be detected.

[0053] The second determining unit is used to determine the quality detection result of the image to be detected based on the target detection data;

[0054] The image quality detection model is obtained by performing the training method for the image quality detection model described above.

[0055] On the other hand, this application provides a computer device, which includes a processor and a memory:

[0056] The memory is used to store program code and transmit the program code to the processor;

[0057] The processor is used to execute the training method or image quality detection method of the image quality detection model described above, according to the instructions in the program code.

[0058] On the other hand, embodiments of this application provide a computer-readable storage medium for storing a computer program, which, when executed by a processor, performs the training method for the image quality detection model or the image quality detection method described above.

[0059] On the other hand, embodiments of this application provide a computer program product, which includes a computer program or instructions; when the computer program or instructions are executed by a processor, the training method of the image quality detection model or the image quality detection method described above are executed.

[0060] As can be seen from the above technical solution, the first training image is input into a preset encoder for feature extraction to obtain the first latent variable. The first training image has preset label data indicating that it is a defective image. Feature perturbation of the first latent variable yields a feature perturbation vector, enabling data augmentation of the first latent variable at the feature space level during model training, covering various nonlinear combinations of data augmentation. The feature perturbation vector is input into a preset decoder for reconstruction processing to obtain the first reconstructed image, thereby reconstructing and expanding a large number of new defective images with preset label data as new training images. Using the first latent variable, the first reconstructed image, and the preset label data, a preset encoder and a preset fully connected layer are trained to obtain the target encoder and the target fully connected layer. The target encoder and the target fully connected layer serve as the image quality detection model. This method trains the preset encoder and the preset fully connected layer using the first training image and the new training images, allowing the target encoder and the target fully connected layer to be trained more fully, thus obtaining the image quality detection model. It ensures the model's generalization performance using a small number of training images with labeled data, thereby improving the model's image quality detection effect. Attached Figure Description

[0061] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0062] Figure 1 A schematic diagram illustrating an application scenario of a training method for an image quality detection model provided in this application embodiment;

[0063] Figure 2 A flowchart illustrating a training method for an image quality detection model provided in this application embodiment;

[0064] Figure 3 A schematic diagram of normal and defective images provided for embodiments of this application;

[0065] Figure 4 A schematic diagram of a training framework for a preset encoder and a preset decoder provided in an embodiment of this application;

[0066] Figure 5 A schematic diagram of a training framework for a target encoder and a target fully connected layer provided in an embodiment of this application;

[0067] Figure 6 A flowchart of an image quality detection method provided in an embodiment of this application;

[0068] Figure 7 This is a schematic diagram of target detection data for an image to be detected provided in an embodiment of this application;

[0069] Figure 8 A schematic diagram illustrating the framework of an image quality detection method provided in an embodiment of this application;

[0070] Figure 9 A structural diagram of a training device for an image quality detection model provided in an embodiment of this application;

[0071] Figure 10 A structural diagram of an image quality detection device provided in an embodiment of this application;

[0072] Figure 11 This application provides a schematic diagram of the structure of a server according to an embodiment of the present application.

[0073] Figure 12 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Detailed Implementation

[0074] The embodiments of this application will now be described with reference to the accompanying drawings.

[0075] Currently, image quality inspection methods based on convolutional neural networks are generally used to inspect products during the manufacturing process. Specifically, the product image is input into the convolutional neural network structure of the image quality inspection model for feature extraction, and then input into the fully connected layer of the image quality inspection model for quality inspection. The detected label data is then used to determine the image quality inspection result.

[0076] However, research has shown that the above methods require a large number of labeled training images to train the model. However, it is difficult to collect a large number of labeled defect images as training images in product defect inspection scenarios. Therefore, using a small number of labeled defect images for training can easily lead to model overfitting and poor generalization performance, resulting in poor image quality detection performance of the model.

[0077] In view of this, this application proposes a training method and related apparatus for an image quality detection model, an image quality detection method, and a training method. The method involves inputting a first training image into a preset encoder for feature extraction to obtain a first latent variable. The first training image has preset label data indicating that it is a defective image. Feature perturbation of the first latent variable yields a feature perturbation vector, enabling data augmentation of the first latent variable at the feature space level during model training, covering various nonlinear combinations of data augmentation. The feature perturbation vector is input into a preset decoder for reconstruction processing to obtain a first reconstructed image, thereby reconstructing and expanding a large number of new defective images with preset label data as new training images. Using the first latent variable, the first reconstructed image, and the preset label data, a preset encoder and a preset fully connected layer are trained to obtain a target encoder and a target fully connected layer. The target encoder and the target fully connected layer serve as the image quality detection model. This method trains the preset encoder and the preset fully connected layer using the first training image and the new training images, allowing the target encoder and the target fully connected layer to be trained more fully, thus obtaining an image quality detection model. It ensures the model's generalization performance using a small number of training images with labeled data, thereby improving the model's image quality detection effect.

[0078] To facilitate understanding of the technical solutions of this application, the training method of the image quality detection model and the image quality detection method provided in the embodiments of this application will be introduced below in conjunction with practical application scenarios.

[0079] See Figure 1 This figure is a schematic diagram illustrating an application scenario of a training method for an image quality detection model provided in an embodiment of this application. Figure 1In the application scenario shown, the application scenario includes a terminal device 101 and a server 102. The terminal device 101 collects a first training image and a second training image and sends them to the server 102.

[0080] Server 102 extracts features from the first training image using a preset encoder to obtain a first latent variable; the first training image has preset label data indicating that the first training image is a defective image. As an example, the first training image is X1, the second training image is X2, the initial encoder is E1, and the initial decoder is D1. Server 102 first trains E1 and D1 using the second enhanced image of X2, A(X2), to obtain the preset encoder E2 and the preset decoder D2; Server 102 then inputs X1 into E2 for feature extraction to obtain the first latent variable Z1, where X1 has preset label data y indicating that X1 is a defective image.

[0081] Server 102 performs feature perturbation on the first latent variable to obtain a feature perturbation vector. As an example, based on the above example, server 102 can perform feature perturbation on Z1 to obtain a feature perturbation vector Z'.

[0082] Server 102 reconstructs the feature perturbation vector using a preset decoder to obtain a first reconstructed image. As an example, based on the above example, server 102 can input Z' into D2 for reconstruction to obtain a first reconstructed image R(Z'), where R(Z') also has preset label data y indicating that R(Z') is a defective image.

[0083] Server 102 trains a preset encoder and a preset fully connected layer based on the first latent variable, the first reconstructed image, and preset label data to obtain the target encoder and the target fully connected layer. As an example, the preset fully connected layer is FC1. Based on the above example, server 102 can train E2 and FC1 using Z1, R(Z'), and y to obtain the target encoder as E3 and the target fully connected layer as FC2.

[0084] Server 102 determines the target encoder and the target fully connected layer as the image quality detection model. As an example, based on the above example, server 102 determines E3 and FC2 as the image quality detection model.

[0085] Therefore, the method involves inputting the first training image into a preset encoder for feature extraction to obtain the first latent variable. The first training image has preset label data indicating that it is a defective image. Feature perturbation of the first latent variable yields a feature perturbation vector, enabling data augmentation of the first latent variable at the feature space level during model training, covering various nonlinear combinations of data augmentation. The feature perturbation vector is then input into a preset decoder for reconstruction to obtain the first reconstructed image, thereby reconstructing and expanding a large number of new defective images with preset label data as new training images. Using the first latent variable, the first reconstructed image, and the preset label data, a preset encoder and a preset fully connected layer are trained to obtain the target encoder and target fully connected layer. The target encoder and target fully connected layer serve as the image quality detection model. This method trains the preset encoder and preset fully connected layer using the first training image and new training images, allowing for more thorough training of the target encoder and target fully connected layer, thus obtaining the image quality detection model. It ensures the model's generalization performance while using a small number of training images with labeled data, thereby improving the model's image quality detection effect.

[0086] The image quality detection model training method and image quality detection method provided in this application can be applied to processing devices with data processing capabilities, such as servers and terminal devices. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services, but is not limited to these. Terminal devices include, but are not limited to, mobile phones, tablets, computers, smart cameras, smart voice interaction devices, smart home appliances, vehicle terminals, and aircraft, but are not limited to these. The terminal devices and servers can be directly or indirectly connected via wired or wireless communication, which is not restricted in this application.

[0087] The training method for the image quality detection model provided in this application involves Artificial Intelligence (AI) and cloud technology. Artificial intelligence is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results. In other words, artificial intelligence is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. Artificial intelligence also studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making functions.

[0088] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, as well as machine learning / deep learning, autonomous driving, and intelligent transportation.

[0089] The training method for the image quality detection model provided in this application mainly involves broad areas such as computer vision technology and machine learning / deep learning. Computer vision (CV) is a scientific discipline that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes for target recognition, tracking, and measurement, and further performs image processing to make the computer-processed images more suitable for human observation or transmission to instruments for detection. As a scientific discipline, computer vision researches related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multi-dimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, autonomous driving, intelligent transportation, and common recognition technologies such as facial recognition.

[0090] Machine learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learn-by-doing.

[0091] Cloud technology refers to a hosting technology that unifies hardware, software, and network resources within a wide area network (WAN) or local area network (LAN) to achieve data computation, storage, processing, and sharing. It encompasses network technology, information technology, integration technology, management platform technology, and application technology based on cloud computing business models. These technologies can form resource pools, allowing for flexible and convenient on-demand use. Cloud computing technology will become a crucial support. Backend services of technical network systems require substantial computing and storage resources, such as video websites, image websites, and many portal websites. With the rapid development and application of the internet industry, every item may have its own identification mark in the future, requiring transmission to backend systems for logical processing. Data at different levels will be processed separately, and various industry data will require robust system support, which can only be achieved through cloud computing.

[0092] The training method and image quality detection method of the image quality detection model provided in this application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, vehicle scenarios, intelligent transportation, and assisted driving.

[0093] Next, taking the aforementioned server as a training device for the image quality detection model, the training method of the image quality detection model provided in this application embodiment will be described in detail.

[0094] See Figure 2 This figure is a flowchart of a training method for an image quality detection model provided in an embodiment of this application. Figure 2 As shown, the training method for this image quality detection model includes the following steps:

[0095] S201: The first training image is subjected to feature extraction by a preset encoder to obtain the first latent variable; the first training image has preset label data indicating that the first training image is a defective image.

[0096] In related technologies, image quality detection methods based on convolutional neural networks are commonly used to inspect products during the manufacturing process. This involves inputting the product image into a convolutional neural network structure within the image quality detection model for feature extraction, then inputting it into a fully connected layer for quality detection to determine the image's label data and thus the quality detection result. However, research has shown that training the model using this method requires a large number of labeled training images. In product defect inspection scenarios, it is easy to collect a large number of normal images with labeled data as training images, but difficult to collect a large number of defective images with labeled data. Therefore, training with a small number of defective images with labeled data easily leads to overfitting and poor generalization performance, resulting in poor image quality detection performance.

[0097] See Figure 3 , Figure 3 These are schematic diagrams illustrating normal and defective images provided in embodiments of this application. Figure 3 (a) represents a normal image. Figure 3 Image (b) shows a defect image with minor defects. Figure 3 (c) represents a defect image with severe defects.

[0098] In this embodiment of the application, in order to solve the above-mentioned problems, for a small number of defective images with tagged data, for example, Figure 3 (b) or Figure 3 In the model training phase, the latent variables obtained by the model through feature extraction are first augmented at the feature space level. Then, a large number of defect images with labeled data are obtained by image reconstruction of the data-augmented latent variables as training images. This expands the small number of defect images with labeled data so that the model can be trained more fully in the future, so as to ensure the generalization performance of the model and thus improve the image quality detection effect of the model.

[0099] Based on this, it is first necessary to obtain a preset encoder with rich feature extraction capabilities and a preset decoder with rich image reconstruction capabilities. This requires training the initial encoder and initial decoder using a second augmented image of the second training image to obtain the preset encoder and preset decoder. The specific training method is as follows: First, image augmentation is performed on the second training image to obtain an augmented image of the second training image, which serves as the second augmented image. Second, the second augmented image is input into the initial encoder for feature extraction to obtain the latent variables of the second augmented image, which serve as the third latent variables. These third latent variables represent the semantic features and augmentation features from the second training image to the second augmented image. Then, the third latent variables are input into the initial decoder for reconstruction processing to obtain the reconstructed image of the second augmented image, which serves as the second reconstructed image. Finally, using the second reconstructed image and the second augmented image, combined with the loss functions corresponding to the initial encoder and initial decoder (i.e., the second loss function), the loss between the second reconstructed image and the second augmented image is calculated to train the parameters of the initial encoder and the initial decoder. Training is complete when the preset number of training iterations is reached or the loss no longer decreases. The trained initial encoder is then used as the preset encoder, and the trained initial decoder is used as the preset decoder. Therefore, in one possible implementation of this application embodiment, the training steps of the preset encoder and preset decoder may include, for example, the following S1-S5 (not shown in the figure):

[0100] S1: Perform image enhancement on the second training image to obtain the second enhanced image of the second training image.

[0101] The image enhancement processing methods can include, for example, one or a nonlinear combination of scaling, rotation, flipping, translation, color transformation, contrast transformation, and noise addition. Therefore, in one possible implementation of this application embodiment, the image enhancement processing methods include at least one of scaling, rotation, flipping, translation, color transformation, contrast transformation, and noise addition.

[0102] The image enhancement methods described above utilize human experience, which can also be understood as prior experience. By adding information or transforming data in the original image through certain means, the image selectively highlights the features of interest or suppresses (masks) certain unwanted features, so that the image matches the visual response characteristics. That is, the useful information in the image is enhanced to improve the visual effect of the image. It should be noted that image enhancement is a data enhancement performed at the image level.

[0103] S2: Extract features from the second enhanced image using the initial encoder to obtain the third latent variable.

[0104] Considering the current maturity of convolutional neural network (CNN) structures, to improve the feature extraction capability of the initial encoder, the initial encoder can, for example, adopt a CNN structure. Specifically, the initial encoder is used to perform convolution operations, pooling operations, nonlinear operations, etc., to extract features from the second enhanced image to obtain a third latent variable; the third latent variable can, for example, be an N-dimensional vector. Of course, the embodiments of this application do not limit the specific structure of the initial encoder, and the initial encoder can also adopt other structures different from the CNN structure.

[0105] S3: The third latent variable is reconstructed using the initial decoder to obtain the second reconstructed image.

[0106] Considering the current maturity of convolutional neural network (CNN) structures, to improve the image reconstruction capability of the initial decoder, the initial decoder can, for example, adopt a CNN structure. Specifically, the initial decoder is used to perform convolution operations, upsampling operations, nonlinear operations, etc., to reconstruct the second enhanced image and obtain the second reconstructed image. Of course, the embodiments of this application do not limit the specific structure of the initial decoder, and the initial decoder can also adopt other structures different from the CNN structure.

[0107] The reconstruction process refers to an iterative process of evolution using a third latent variable representing the semantic features and enhancement features of the second training image to the second enhanced image, and attempts to reconstruct the second enhanced image through evolution.

[0108] S4: Train the parameters of the initial encoder and the initial decoder based on the second reconstructed image, the second enhanced image, and the second loss function.

[0109] The second loss function can be, for example, a mean squared error loss function used to calculate the difference between the second reconstructed image and the second enhanced image as the loss. Therefore, in one possible implementation of this application, the second loss function includes a mean squared error loss function. Of course, this application does not limit the specific content of the second loss function; it may also include other loss functions different from the mean squared error loss function used to calculate the loss between the second reconstructed image and the second enhanced image.

[0110] S5: Determine the initial encoder that has been trained as the preset encoder, and the initial decoder that has been trained as the preset decoder.

[0111] As an example, see Figure 4 , Figure 4 This is a schematic diagram of a training framework for a preset encoder and a preset decoder provided in an embodiment of this application. The second training image is X2, the initial encoder is E1, the initial decoder is D1, and image enhancement is performed on X2 to obtain a second enhanced image A(X2). A(X2) is input into E1 for feature extraction to obtain a third latent variable Z3, i.e., Z3 = f E1 (A(X2);θ E1 ), where f E1 (·;θ E1 ) represents the convolution operation, pooling operation, non-linear operation, etc. of E1, θ E1 The parameter represents E1; inputting Z3 into D1 for reconstruction processing yields the second reconstructed image R(Z3), i.e., R(Z3) = f D1 (Z3;θ D1 ), where f D1 (·;θ D1 ) represents the convolution operation, upsampling operation, nonlinear operation, etc. of D1, θ D1 Let L represent the parameters of D1. Using R(Z3) and A(X2), the second loss function corresponding to E1 and D1 is L. rec Calculate the loss between R(Z3) and A(X2), i.e., L rec =‖A(X2)-R(Z3)‖2, where ‖·‖2 represents the calculation of the L2 norm, which is the calculation of the mean squared error, in order to train θ E1 and θ D1 The trained E1 is used as the preset encoder, which is E2, and the trained D1 is used as the preset decoder, which is D2.

[0112] Based on the above description, after obtaining the preset encoder and preset decoder, firstly, the first training image with preset label data representing the first training image as a defective image needs to be input into the preset encoder for feature extraction to obtain the latent variable of the first training image as the first latent variable. The first latent variable represents the semantic feature of the first training image, which can be an N-dimensional vector.

[0113] As an example, the first training image is X1, and X1 has a preset label data y that indicates that X1 is a defective image; based on the above example, X1 with y is input into E2 for feature extraction to obtain the first latent variable Z1.

[0114] Furthermore, in this embodiment, considering that there is more than one type of defect in a defective image—for example, the defect type of a defective image can be divided into minor defects or severe defects—the preset label data, in addition to indicating that the first training image is a defective image, can also indicate that the first training image is a defective image of a preset defect type. Therefore, in one possible implementation of this embodiment, the preset label data further indicates that the first training image is a defective image of a preset defect type.

[0115] As an example, y=1 represents a defect image with severe defects in the first training image, and y=2 represents a defect image with mild defects in the first training image.

[0116] S202: Perform feature perturbation on the first latent variable to obtain the feature perturbation vector.

[0117] In this embodiment, after obtaining the latent variables of the first training image in S201, i.e., the first latent variables, data augmentation needs to be performed on the first latent variables at the feature space level, that is, data augmentation is performed at the feature vector level. Feature perturbation is used to perturb the first latent variables, resulting in the data-augmented first latent variables, which serve as the feature perturbation vector. This feature perturbation vector encompasses various nonlinear combinations of data augmentation; that is, the feature perturbation vector not only represents the semantic features of the first training image but also the perturbation features of the first training image. This method is equivalent to augmenting and expanding the latent variables of the first training image, achieving augmentation and expansion of the latent variables of a small number of defective images, so that the first training image can be reconstructed and expanded subsequently through image reconstruction, achieving reconstruction and expansion of a small number of defective images. As an example, based on the above example, feature perturbation is performed on Z1 to obtain the feature perturbation vector Z'; where the feature perturbation method can be, for example, adding a ~N(0,1) standard Gaussian noise near Z1 to obtain Z'.

[0118] Furthermore, in this embodiment, considering further enhancement and expansion of the latent variables of the first training image, before obtaining the latent variables of the first training image by feature extraction from the first training image through a preset encoder, image enhancement can be performed on the first training image to obtain an enhanced image of the first training image, which is used as the first enhanced image to replace the first training image. Then, feature extraction is performed on the first enhanced image through the preset encoder to obtain the first latent variables representing the semantic features and enhancement features of the first enhanced image. Based on this, the feature perturbation vector obtained by perturbing the first latent variable represents the semantic features, enhancement features, and perturbation features from the first training image to the first enhanced image, thereby enabling further enhancement and expansion of the latent variables of the first training image. Therefore, in a possible implementation of this embodiment, the method may further include S6 (not shown in the figure): performing image enhancement on the first training image to obtain the first enhanced image of the first training image; correspondingly, S201 may include, for example, performing feature extraction on the first enhanced image through a preset encoder to obtain the first latent variables.

[0119] As an example, based on the above example, image enhancement is performed on X1 to obtain the first enhanced image of X1, which is A(X1). The A(X1) with y is input into E2 for feature extraction to obtain Z1.

[0120] Similarly, in one possible implementation of the embodiments of this application, the image enhancement processing method includes at least one of scaling, rotation, flipping, translation, color transformation, contrast transformation, and noise addition.

[0121] S203: The feature perturbation vector is reconstructed using a preset decoder to obtain the first reconstructed image.

[0122] In this embodiment, after obtaining the first latent variable after data augmentation in step S202 as a feature perturbation vector, the feature perturbation vector needs to be input into a preset decoder for reconstruction processing to obtain a reconstructed image of the feature perturbation vector corresponding to the first training image, which is then used as the first reconstructed image. This method reconstructs and expands the first training image through image reconstruction, thereby expanding and reconstructing a small number of defective images. This allows for more thorough training of the model subsequently, ensuring the model's generalization performance and improving the model's image quality detection effect.

[0123] The reconstruction process refers to an iterative process of evolution using the first latent variable representing the semantic features, enhancement features, and perturbation features of the first training image, and attempts to reconstruct the first training image through evolution.

[0124] As an example, based on the above example, Z' is input into D2 for reconstruction processing to obtain the first reconstructed image R(Z').

[0125] S204: Based on the first latent variable, the first reconstructed image, and the preset label data, train the preset encoder and the preset fully connected layer to obtain the target encoder and the target fully connected layer.

[0126] In this embodiment of the application, after executing S204 to obtain the reconstructed image of the first training image, the first reconstructed image also has preset label data that matches the first training image. The first reconstructed image needs to be used as a new training image. The preset encoder with rich feature extraction capability and the preset fully connected layer for quality detection are trained more fully in combination with the first training image to ensure the generalization performance of the target encoder and the target fully connected layer after training is completed, thereby improving the image quality detection effect of the target encoder and the target fully connected layer.

[0127] In the specific implementation of S204, firstly, the first reconstructed image with the same preset label data is input into the preset encoder for feature extraction to obtain the latent variable of the first reconstructed image, which serves as the second latent variable. This second latent variable represents the semantic features of the first reconstructed image. Secondly, the first latent variable and the second latent variable are respectively input into the preset fully connected layer for quality detection to obtain detection data on whether the first training image is a defective image, which serves as the first detection data, and detection data on whether the first reconstructed image is a defective image, which serves as the second detection data. Then, using the first detection data, the second detection data, and the preset label data, combined with the loss function corresponding to the preset encoder and the preset fully connected layer, i.e., the first loss function, the loss between the first detection data and the preset label data, and the loss between the second detection data and the preset label data are calculated to train the parameters of the preset encoder and the parameters of the preset fully connected layer. The training is considered complete when the training reaches a preset number of times or the loss no longer decreases. Thus, the trained preset encoder is used as the target encoder, and the trained preset fully connected layer is used as the target fully connected layer. Therefore, in one possible implementation of this application embodiment, S204 may include, for example, the following S2041-S2044 (not shown in the figure):

[0128] S2041: Extract features from the first reconstructed image using a preset encoder to obtain the second latent variable.

[0129] The second latent variable can be, for example, an N-dimensional vector.

[0130] S2042: Perform quality checks on the first latent variable and the second latent variable respectively through a preset fully connected layer to obtain the first detection data of the first training image and the second detection data of the first reconstructed image.

[0131] The first detection data may be, for example, the probability that the first training image is a defective image, and the second detection data may be, for example, the probability that the first reconstructed image is a defective image.

[0132] S2043: Based on the first detection data, the second detection data, the preset label data, and the first loss function, train the parameters of the preset encoder and the parameters of the preset fully connected layer.

[0133] The second loss function can be, for example, a cross-entropy loss function used to measure the difference between the first detection data and the preset label data as the loss, or a relative entropy loss function used to measure the difference between the first detection data and the preset label data as the loss. Therefore, in one possible implementation of this application embodiment, the first loss function includes a cross-entropy loss function or a relative entropy loss function. Of course, this application embodiment does not limit the specific content of the second loss function, and the second loss function may also include other loss functions different from the cross-entropy loss function and the relative entropy loss function, used to calculate the loss between the first detection data and the preset label data, and the loss between the second detection data and the preset label data.

[0134] S2044: The pre-set encoder that has been trained is identified as the target encoder, and the pre-set fully connected layer that has been trained is identified as the target fully connected layer.

[0135] As an example, see Figure 5 , Figure 5 This is a schematic diagram of a training framework for a target encoder and a target fully connected layer provided in an embodiment of this application. The preset fully connected layer is FC1. Based on the above example, feature extraction is performed on the input E2 with X1 having y to obtain Z1, i.e., Z1 = f E2 (X1;θ E2 ), where f E2 (·;θ E2 ) represents convolution, pooling, and nonlinear operations in E2, θ E2 Let E2 be the parameter; Z1 is perturbed to obtain the feature perturbation vector Z'; Z' is input into D2 for reconstruction to obtain R(Z'), that is, R(Z') = f D2 (Z';θ) D2 ), where f D2 (·;θ D2 ) represents the convolution operation, upsampling operation, nonlinear operation, etc. of D2, θ D2 Let f represent the parameters of D2. Inputting R(Z') with the same y value into E2 and performing feature extraction yields the second latent variable Z2, i.e., Z2 = f E2 (R(Z'); θ E2Input Z1 and Z2 into FC1 for quality detection to obtain the first detection data as p1 and the second detection data as p2, that is, p1 = f FC1 (Z1;θ FC1 p2 = f FC1 (Z2;θ FC1 ), where f FC1 (·;θ FC1 ) indicates quality inspection operations of FC1, etc., θ FC1 The parameters of FC1 are represented by p1, p2, and y; the first loss function corresponding to E2 and FC1 is L. sup Calculate the loss between p1 and y, and the loss between p2 and y, i.e., L sup =CE((p1, p2), y), where CE(·) represents the cross-entropy loss function, used to train θ E2 and θ FC1 The trained E2 is used as the target encoder, which is E3. The trained FC1 is used as the target fully connected layer, which is FC2.

[0136] Furthermore, in this embodiment of the application, when executing S202, the first latent variable is perturbed once using a feature perturbation method, which is equivalent to enhancing and expanding the latent variable of the first training image once, thereby enhancing and expanding the latent variable of a small number of defective images once. In order to further enhance and expand the latent variable of the first training image, when executing S202, the first latent variable can be perturbed multiple times using a feature perturbation method with different features to obtain multiple first latent variables with different data enhancements, which serve as multiple different feature perturbation vectors. Based on this, during S203, multiple different feature perturbation vectors are input into a preset decoder for reconstruction processing to obtain a reconstructed image of each feature perturbation vector in the multiple different feature perturbation vectors corresponding to the first training image, which serves as multiple different first reconstructed images. This method achieves further reconstruction and expansion of a small number of defective images. During S204, the preset encoder and preset fully connected layer can be trained using the first latent variable, multiple different first reconstructed images, and preset label data to obtain the target encoder and target fully connected layer. This method achieves further and more thorough training of the target encoder and target fully connected layer to further ensure the generalization performance of the target encoder and target fully connected layer, thereby further improving the image quality detection effect of the target encoder and target fully connected layer.

[0137] Therefore, in one possible implementation of this application embodiment, S202 may include, for example, performing multiple different feature perturbations on the first latent variable to obtain multiple different feature perturbation vectors; correspondingly, S203 may include, for example, performing reconstruction processing on the multiple different feature perturbation vectors respectively through a preset decoder to obtain multiple different first reconstructed images; S204 may include, for example, training a preset encoder and a preset fully connected layer based on the first latent variable, multiple different first reconstructed images and preset label data to obtain a target encoder and a target fully connected layer.

[0138] S205: The target encoder and the target fully connected layer are defined as the image quality detection model.

[0139] In this embodiment of the application, after executing S204 to obtain the target encoder and the target fully connected layer, since the target encoder and the target fully connected layer are trained more fully to ensure their generalization performance and image quality detection effect, the target encoder and the target fully connected layer need to be used as the image quality detection model. That is, the image quality detection model is trained more fully to ensure the generalization performance of the model, thereby improving the image quality detection effect of the model.

[0140] As an example, based on the above example, E3 and FC2 are used as image quality detection models.

[0141] The training method for the image quality detection model provided in the above embodiments involves inputting a first training image into a preset encoder for feature extraction to obtain a first latent variable. The first training image has preset label data representing a defective image. Feature perturbation of the first latent variable yields a feature perturbation vector, enabling data augmentation of the first latent variable at the feature space level during model training, covering various nonlinear combinations of data augmentation. The feature perturbation vector is input into a preset decoder for reconstruction processing to obtain a first reconstructed image, thereby reconstructing and expanding a large number of new defective images with preset label data as new training images. Using the first latent variable, the first reconstructed image, and the preset label data, a preset encoder and a preset fully connected layer are trained to obtain a target encoder and a target fully connected layer. The target encoder and the target fully connected layer serve as the image quality detection model. This method trains the preset encoder and the preset fully connected layer using the first training image and new training images, allowing the target encoder and the target fully connected layer to be trained more fully, thus obtaining an image quality detection model. It ensures the model's generalization performance using a small number of training images with labeled data, thereby improving the model's image quality detection effect.

[0142] Based on the training method of the above image quality detection model, the trained image quality detection model is more fully trained, which can ensure the generalization performance of the model and thus improve the image quality detection effect. Therefore, when the product to be detected has quality detection requirements, firstly, the image of the product to be detected can be obtained as the image to be detected; secondly, the image to be detected is input into the target encoder in the image quality detection model for feature extraction to obtain the latent variable of the image to be detected, which is used as the fourth latent variable. This fourth latent variable represents the semantic features of the image to be detected; then, the fourth latent variable is input into the target fully connected layer in the image quality detection model for quality detection to obtain detection data indicating whether the image to be detected is a defective image, which is used as the target detection data; finally, the quality detection result of whether the image to be detected is a defective image can be determined by the target detection data.

[0143] Next, taking the aforementioned server as an image quality detection device, we will provide a detailed description of the image quality detection method provided in the embodiments of this application.

[0144] See Figure 6 This figure is a flowchart of an image quality detection method provided in an embodiment of this application. Figure 6 As shown, the image quality detection method includes the following steps:

[0145] S601: Obtain the image of the product to be inspected.

[0146] As an example, the image to be detected is X3.

[0147] S602: The fourth latent variable is obtained by extracting features from the image to be detected through the target encoder in the image quality detection model.

[0148] As an example, based on the above example, the fourth latent variable Z4 is obtained by extracting features from E3 in the input image quality detection model X3; where Z4 can be, for example, an N-dimensional vector.

[0149] S603: The fourth latent variable is detected by the target fully connected layer in the image quality detection model to obtain the target detection data of the image to be detected.

[0150] As an example, based on the above example, Z4 is input into the FC2 of the image quality detection model to perform quality detection and obtain target detection data, which is the probability p that the image to be detected is a defective image.

[0151] As an example, see Figure 7 This figure is a schematic diagram of target detection data for an image to be detected provided in an embodiment of this application. Figure 7In the middle (a), it indicates that the target detection data of image 1 is 0.05. Figure 7 In the middle (b), it means that the target detection data of the image to be detected 1 is 0.95.

[0152] S604: Determine the quality detection result of the image to be detected based on the target detection data.

[0153] In the specific implementation of S604, the target detection data can be compared with the preset threshold data. If the target detection data is greater than the preset threshold data, the quality detection result of the image to be detected indicates that the image to be detected is a defective image; if the target detection data is less than or equal to the preset threshold data, the quality detection result of the image to be detected indicates that the image to be detected is a normal image.

[0154] As an example, see Figure 8 The figure is a schematic diagram of the framework of an image quality detection method provided in an embodiment of this application. The preset threshold data is 0.5. Based on the above example, X3 is input into E3 in the image quality detection model for feature extraction to obtain Z4, Z4 = f E3 (X3;θ E3 ), where f E3 (·;θ E3 ) represents convolution, pooling, and nonlinear operations in E3, θ E3 This represents the parameters of E2; Z4 is input into FC2 in the image quality detection model to perform quality detection and obtain p, that is, p = f FC2 (Z4;θ FC2 ), where f FC2 (·;θ FC2 ) indicates quality inspection operations such as FC2, θ FC2 The parameter represents FC2; when p is greater than 0.5, the quality inspection result of X3 indicates that X3 is a defective image; when p is less than or equal to 0.5, the quality inspection result of X3 indicates that X3 is a normal image.

[0155] The image quality detection method provided in the above embodiments acquires an image of the product to be detected; secondly, the image to be detected is input into the target encoder in the image quality detection model for feature extraction to obtain a fourth latent variable; then, the fourth latent variable is input into the target fully connected layer in the image quality detection model for quality detection to obtain target detection data indicating whether the image to be detected is a defective image; finally, the quality detection result of whether the image to be detected is a defective image can be determined through the target detection data. It can be seen that this method, with more thorough training of the image quality detection model, can ensure the model's generalization performance using a small number of labeled training images, thereby improving the model's image quality detection effect. Furthermore, by using the image quality detection model to perform image quality detection on the image of the product to be detected, the obtained quality detection result of the image to be detected is more accurate, thus making the quality detection of the product to be detected more accurate and providing reliable technical support for the quality detection of the product to be detected.

[0156] In addition to the training method for the image quality detection model described above, this application also provides a training device for the image quality detection model. The training device for the image quality detection model provided in this application will be described in detail below.

[0157] See Figure 9 , Figure 9 This is a structural diagram of a training device for an image quality detection model provided in an embodiment of this application. Figure 9 As shown, the training device 900 for the image quality detection model includes: a first feature extraction unit 901, a feature perturbation unit 902, a reconstruction unit 903, a training unit 904, and a first determination unit 905.

[0158] The first feature extraction unit 901 is used to extract features from the first training image through a preset encoder to obtain a first latent variable; the first training image has preset label data indicating that the first training image is a defective image;

[0159] The feature perturbation unit 902 is used to perturb the first latent variable to obtain the feature perturbation vector;

[0160] The reconstruction unit 903 is used to reconstruct the feature perturbation vector through a preset decoder to obtain the first reconstructed image;

[0161] Training unit 904 is used to train a preset encoder and a preset fully connected layer based on the first latent variable, the first reconstructed image and preset label data, so as to obtain a target encoder and a target fully connected layer.

[0162] The first determining unit 905 is used to determine the target encoder and the target fully connected layer as the image quality detection model.

[0163] As one possible implementation, training unit 904 is specifically used for:

[0164] The second latent variable is obtained by extracting features from the first reconstructed image using a preset encoder;

[0165] By performing quality checks on the first and second latent variables through a preset fully connected layer, the first detection data of the first training image and the second detection data of the first reconstructed image are obtained.

[0166] Based on the first detection data, the second detection data, the preset label data, and the first loss function, train the parameters of the preset encoder and the parameters of the preset fully connected layer;

[0167] The pre-set encoder that has been trained is designated as the target encoder, and the pre-set fully connected layer that has been trained is designated as the target fully connected layer.

[0168] As one possible implementation, the feature perturbation unit 902 is specifically used for:

[0169] The first latent variable is perturbed multiple times with different features to obtain multiple different feature perturbation vectors;

[0170] The feature perturbation vector is reconstructed using a preset decoder to obtain a first reconstructed image, including:

[0171] Multiple different feature perturbation vectors are reconstructed by a preset decoder to obtain multiple different first reconstructed images;

[0172] Based on the first latent variable, the first reconstructed image, and the preset label data, the preset encoder and preset fully connected layer are trained to obtain the target encoder and target fully connected layer, including:

[0173] Based on the first latent variable, multiple different first reconstructed images, and preset label data, the preset encoder and preset fully connected layer are trained to obtain the target encoder and target fully connected layer.

[0174] As one possible implementation, the device further includes: a first enhancement unit;

[0175] The first enhancement unit is used to perform image enhancement on the first training image to obtain a first enhanced image of the first training image;

[0176] The first feature extraction unit 901 is also used to extract features from the first enhanced image through a preset encoder to obtain the first latent variable.

[0177] As one possible implementation, the first loss function includes either the cross-entropy loss function or the relative entropy loss function.

[0178] As one possible implementation, the preset label data also indicates that the first training image is a defect image of a preset defect type.

[0179] As one possible implementation, the device further includes: a second enhancement unit;

[0180] The second enhancement unit is used to enhance the second training image to obtain the second enhanced image;

[0181] The first feature extraction unit 901 is also used to extract features from the second enhanced image through the initial encoder to obtain the third latent variable;

[0182] The reconstruction unit 903 is also used to reconstruct the third latent variable through the initial decoder to obtain the second reconstructed image;

[0183] Training unit 904 is also used to train the parameters of the initial encoder and the parameters of the initial decoder based on the second reconstructed image, the second enhanced image and the second loss function;

[0184] The first determining unit 905 is further configured to determine the initial encoder that has been trained as the preset encoder and the initial decoder that has been trained as the preset decoder.

[0185] As a possible implementation method, image enhancement processing methods include at least one of scaling, rotation, flipping, translation, color transformation, contrast transformation, and noise addition.

[0186] As one possible implementation, the second loss function includes the mean squared error loss function.

[0187] The training apparatus for the image quality detection model provided in the above embodiments extracts features from a first training image using a preset encoder to obtain a first latent variable. The first training image has preset label data indicating that it is a defective image. The first latent variable is perturbed to obtain a feature perturbation vector, enabling data augmentation of the first latent variable at the feature space level during model training, covering various nonlinear combinations of data augmentation. The feature perturbation vector is input into a preset decoder for reconstruction processing to obtain a first reconstructed image, thereby reconstructing and expanding a large number of new defective images with preset label data as new training images. Using the first latent variable, the first reconstructed image, and the preset label data, a preset encoder and a preset fully connected layer are trained to obtain a target encoder and a target fully connected layer. The target encoder and the target fully connected layer are used as the image quality detection model. This method trains the preset encoder and the preset fully connected layer using the first training image and the new training images, enabling the target encoder and the target fully connected layer to be trained more fully, thus obtaining an image quality detection model. It ensures the model's generalization performance using a small number of training images with labeled data, thereby improving the model's image quality detection effect.

[0188] In addition to the image quality detection method described above, this application also provides an image quality detection device. The image quality detection device provided in this application will be described in detail below.

[0189] See Figure 10 , Figure 10 This is a structural diagram of an image quality detection device provided in an embodiment of this application. Figure 10 As shown, the image quality detection device 1000 includes: an acquisition unit 1001, a second feature extraction unit 1002, a detection unit 1003, and a second determination unit 1004;

[0190] The acquisition unit 1001 is used to acquire the image of the product to be inspected.

[0191] The second feature extraction unit 1002 is used to extract features from the image to be detected through the target encoder in the image quality detection model to obtain the fourth latent variable.

[0192] Detection unit 1003 is used to perform quality detection on the fourth latent variable through the target fully connected layer in the image quality detection model to obtain target detection data of the image to be detected;

[0193] The second determining unit 1004 is used to determine the quality detection result of the image to be detected based on the target detection data;

[0194] The image quality detection model is obtained by training the image quality detection model according to the above embodiments.

[0195] The image quality detection device provided in the above embodiment acquires an image of the product to be detected; secondly, it inputs the image to be detected into the target encoder in the image quality detection model for feature extraction to obtain a fourth latent variable; then, it inputs the fourth latent variable into the target fully connected layer in the image quality detection model for quality detection to obtain target detection data indicating whether the image to be detected is a defective image; finally, the quality detection result of whether the image to be detected is a defective image can be determined through the target detection data. It can be seen that this method, with more thorough training of the image quality detection model, can ensure the model's generalization performance using a small number of labeled training images, thereby improving the model's image quality detection effect. Furthermore, by using the image quality detection model to perform image quality detection on the image of the product to be detected, the obtained quality detection result of the image to be detected is more accurate, thus making the quality detection of the product to be detected more accurate and providing reliable technical support for the quality detection of the product to be detected.

[0196] In relation to the image quality detection model training method and image quality detection method described above, this application embodiment also provides a training device or image quality detection device for the image quality detection model, so that the above-mentioned image quality detection model training method and image quality detection method can be implemented and applied in practice. The computer device provided in this application embodiment will be introduced from the perspective of hardware physicalization below.

[0197] See Figure 11 , Figure 11 This is a schematic diagram of a server structure provided in an embodiment of this application. The server 1100 can vary significantly due to different configurations or performance, and may include one or more Central Processing Units (CPUs) 1122 (e.g., one or more processors) and a memory 1132, and one or more storage media 1130 (e.g., one or more mass storage devices) for storing application programs 1142 or data 1144. The memory 1132 and storage media 1130 can be temporary or persistent storage. The program stored in the storage media 1130 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the server. Furthermore, the CPU 1122 may be configured to communicate with the storage media 1130 and execute the series of instruction operations in the storage media 1130 on the server 1100.

[0198] Server 1100 may also include one or more power supplies 1126, one or more wired or wireless network interfaces 1150, one or more input / output interfaces 1158, and / or one or more operating systems 1141, such as Windows Server. TM Mac OS X TM Unix TM Linux TM FreeBSD TM etc.

[0199] The steps performed by the server in the above embodiments can be based on this Figure 11 The server structure shown.

[0200] CPU 1122 is used to perform the following steps:

[0201] The first training image is subjected to feature extraction by a preset encoder to obtain the first latent variable; the first training image has preset label data indicating that the first training image is a defective image;

[0202] The first latent variable is perturbed to obtain the feature perturbation vector;

[0203] The first reconstructed image is obtained by reconstructing the feature perturbation vector through a preset decoder;

[0204] Based on the first latent variable, the first reconstructed image, and the preset label data, the preset encoder and the preset fully connected layer are trained to obtain the target encoder and the target fully connected layer.

[0205] The target encoder and the target fully connected layer are defined as the image quality detection model.

[0206] CPU 1122 is used to perform the following steps:

[0207] Acquire the image of the product to be inspected;

[0208] The fourth latent variable is obtained by extracting features from the image to be detected using the target encoder in the image quality detection model.

[0209] The fourth latent variable is detected by the target fully connected layer in the image quality detection model to obtain the target detection data of the image to be detected.

[0210] Determine the quality detection result of the image to be detected based on the target detection data;

[0211] The image quality detection model is obtained by training the image quality detection model according to the above embodiments.

[0212] Optionally, the CPU 1122 may also execute the method steps of the training method of the image quality detection model or any specific implementation of the image quality detection method in the embodiments of this application.

[0213] See Figure 12 , Figure 12 This is a schematic diagram of a terminal device provided in an embodiment of this application. For ease of explanation, only the parts related to the embodiment of this application are shown; for specific technical details not disclosed, please refer to the method section of the embodiment of this application. The terminal device can be any terminal device including mobile phones, tablets, PDAs, etc. Taking a mobile phone as an example:

[0214] Figure 12 This diagram illustrates a partial structure of a mobile phone related to the terminal device provided in the embodiments of this application. (Reference) Figure 12 The mobile phone includes components such as a radio frequency (RF) circuit 1210, a memory 1220, an input unit 1230, a display unit 1240, a sensor 1250, an audio circuit 1260, a Wi-Fi module 1270, a processor 1280, and a power supply 1290. Those skilled in the art will understand that... Figure 12 The mobile phone structure shown does not constitute a limitation on the mobile phone and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0215] The following is combined Figure 12 A detailed introduction to each component of a mobile phone:

[0216] RF circuit 1210 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and processes it with processor 1280; additionally, it transmits uplink data to the base station. Typically, RF circuit 1210 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier (LNA), a duplexer, etc. Furthermore, RF circuit 1210 can also communicate wirelessly with networks and other devices. The aforementioned wireless communication can use any communication standard or protocol, including but not limited to Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, and Short Messaging Service (SMS).

[0217] The memory 1220 can be used to store software programs and modules. The processor 1280 executes the software programs and modules stored in the memory 1220 to realize various functions and data processing of the mobile phone. The memory 1220 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory 1220 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0218] The input unit 1230 can be used to receive input numerical or character information, and to generate key signal inputs related to user settings and function control of the mobile phone. Specifically, the input unit 1230 may include a touch panel 1231 and other input devices 1232. The touch panel 1231, also known as a touch screen, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel 1231), and drive the corresponding connection devices according to a pre-set program. Optionally, the touch panel 1231 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 1280, and can also receive and execute commands sent by the processor 1280. In addition, the touch panel 1231 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 1231, the input unit 1230 may also include other input devices 1232. Specifically, other input devices 1232 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0219] The display unit 1240 can be used to display information input by the user or information provided to the user, as well as various menus of the mobile phone. The display unit 1240 may include a display panel 1241, which may optionally be configured as a Liquid Crystal Display (LCD), Organic Light-Emitting Diode (OLED), or similar display panel. Furthermore, a touch panel 1231 may cover the display panel 1241. When the touch panel 1231 detects a touch operation on or near it, it transmits the information to the processor 1280 to determine the type of touch event. Subsequently, the processor 1280 provides corresponding visual output on the display panel 1241 according to the type of touch event. Although in Figure 12 In this embodiment, the touch panel 1231 and the display panel 1241 are two separate components to realize the input and output functions of the mobile phone. However, in some embodiments, the touch panel 1231 and the display panel 1241 can be integrated to realize the input and output functions of the mobile phone.

[0220] The mobile phone may also include at least one sensor 1250, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 1241 according to the ambient light level, and the proximity sensor can turn off the display panel 1241 and / or the backlight when the phone is moved to the ear. As a type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometer, taps), etc. Other sensors that may be configured in the mobile phone, such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0221] Audio circuit 1260, speaker 1261, and microphone 1262 provide an audio interface between the user and the mobile phone. Audio circuit 1260 converts received audio data into electrical signals and transmits them to speaker 1261, where speaker 1261 converts them into sound signals for output. On the other hand, microphone 1262 converts collected sound signals into electrical signals, which are received by audio circuit 1260, converted into audio data, and then processed by processor 1280 before being transmitted via RF circuit 1210 to, for example, another mobile phone, or the audio data can be output to memory 1220 for further processing.

[0222] WiFi is a short-range wireless transmission technology. Through the WiFi module 1270, mobile phones can help users send and receive emails, browse web pages, and access streaming media, providing users with wireless broadband internet access. Although Figure 12 WiFi module 1270 is shown, but it is understood that it is not an essential component of a mobile phone and can be omitted as needed without changing the essence of the invention.

[0223] The processor 1280 is the control center of the mobile phone, connecting various parts of the phone through various interfaces and lines. It performs various functions and processes data by running or executing software programs and / or modules stored in the memory 1220 and calling data stored in the memory 1220, thereby controlling the phone as a whole. Optionally, the processor 1280 may include one or more processing units; preferably, the processor 1280 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 1280.

[0224] The mobile phone also includes a power supply 1290 (such as a battery) that supplies power to various components. Preferably, the power supply can be logically connected to the processor 1280 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.

[0225] Although not shown, mobile phones may also include a camera, Bluetooth module, etc., which will not be described in detail here.

[0226] In this embodiment of the application, the memory 1220 included in the mobile phone can store program code and transmit the program code to the processor.

[0227] The processor 1280 included in the mobile phone can execute the training method or image quality detection method of the image quality detection model provided in the various optional implementations of the above aspects according to the instructions in the program code.

[0228] This application also provides a computer-readable storage medium for storing a computer program for executing the training method or image quality detection method of the image quality detection model provided in the above embodiments.

[0229] This application also provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the training method or image quality detection method for the image quality detection model provided in the various optional implementations of the above aspects.

[0230] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium can be at least one of the following media: read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, etc., and other media capable of storing program code.

[0231] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the solution in this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0232] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A training method for an image quality detection model, characterized in that, The method includes: The first training image is subjected to feature extraction by a preset encoder to obtain a first latent variable; the first training image has preset label data indicating that the first training image is a defective image; The first latent variable is perturbed to obtain a feature perturbation vector; the feature perturbation is used to enhance and expand the first latent variable. The feature perturbation vector is reconstructed using a preset decoder to obtain a first reconstructed image; Based on the first latent variable, the first reconstructed image, and the preset label data, the preset encoder and the preset fully connected layer are trained to obtain the target encoder and the target fully connected layer. This includes: extracting features from the first reconstructed image using the preset encoder to obtain a second latent variable; performing quality checks on the first latent variable and the second latent variable using the preset fully connected layer to obtain first detection data for the first training image and second detection data for the first reconstructed image; calculating the loss between the first detection data and the preset label data, and the loss between the second detection data and the preset label data, based on the first detection data, the second detection data, and the preset label data, combined with a first loss function corresponding to the preset encoder and the preset fully connected layer, to train the parameters of the preset encoder and the preset fully connected layer; and determining the trained preset encoder as the target encoder and the trained preset fully connected layer as the target fully connected layer. The target encoder and the target fully connected layer are defined as the image quality detection model.

2. The method according to claim 1, characterized in that, The step of perturbing the first latent variable to obtain a feature perturbation vector includes: The first latent variable is subjected to multiple perturbations with different features to obtain multiple perturbation vectors with different features; The step of reconstructing the feature perturbation vector using a preset decoder to obtain a first reconstructed image includes: The preset decoder is used to reconstruct the multiple different feature perturbation vectors respectively to obtain multiple different first reconstructed images; The step of training the preset encoder and preset fully connected layer based on the first latent variable, the first reconstructed image, and the preset label data to obtain the target encoder and target fully connected layer includes: Based on the first latent variable, the multiple different first reconstructed images, and the preset label data, the preset encoder and the preset fully connected layer are trained to obtain the target encoder and the target fully connected layer.

3. The method according to claim 1, characterized in that, The method further includes: Image enhancement is performed on the first training image to obtain a first enhanced image of the first training image; The step of extracting features from the first training image using a preset encoder to obtain the first latent variable includes: The first latent variable is obtained by extracting features from the first enhanced image using the preset encoder.

4. The method according to claim 1, characterized in that, The first loss function includes either the cross-entropy loss function or the relative entropy loss function.

5. The method according to any one of claims 1-3, characterized in that, The preset label data also indicates that the first training image is a defect image of a preset defect type.

6. The method according to claim 1, characterized in that, The training steps for the preset encoder and the preset decoder include: Image enhancement is performed on the second training image to obtain a second enhanced image of the second training image; The third latent variable is obtained by extracting features from the second enhanced image using the initial encoder; The third latent variable is reconstructed using the initial decoder to obtain the second reconstructed image; Based on the second reconstructed image, the second enhanced image, and the second loss function, train the parameters of the initial encoder and the parameters of the initial decoder; The initial encoder that has completed training is determined as the preset encoder, and the initial decoder that has completed training is determined as the preset decoder.

7. The method according to claim 3 or 6, characterized in that, The image enhancement processing methods include at least one of scaling, rotation, flipping, translation, color transformation, contrast transformation, and noise addition.

8. The method according to claim 6, characterized in that, The second loss function includes the mean squared error loss function.

9. An image quality detection method, characterized in that, The method includes: Acquire the image of the product to be inspected; The fourth latent variable is obtained by extracting features from the image to be detected using the target encoder in the image quality detection model. The fourth latent variable is quality-detected by the target fully connected layer in the image quality detection model to obtain the target detection data of the image to be detected. The quality detection result of the image to be detected is determined based on the target detection data; The image quality detection model is obtained by performing the training method of the image quality detection model according to any one of claims 1-8.

10. A training device for an image quality detection model, characterized in that, The device includes: a first feature extraction unit, a feature perturbation unit, a reconstruction unit, a training unit, and a first determination unit; The first feature extraction unit is used to extract features from the first training image through a preset encoder to obtain a first latent variable; the first training image has preset label data indicating that the first training image is a defective image; The feature perturbation unit is used to perturb the first latent variable to obtain a feature perturbation vector; the feature perturbation is used to enhance and expand the first latent variable. The reconstruction unit is used to reconstruct the feature perturbation vector through a preset decoder to obtain a first reconstructed image; The training unit is configured to train the preset encoder and the preset fully connected layer based on the first latent variable, the first reconstructed image, and the preset label data to obtain the target encoder and the target fully connected layer. Specifically, the training unit is configured to: extract features from the first reconstructed image using the preset encoder to obtain the second latent variable; perform quality checks on the first latent variable and the second latent variable using the preset fully connected layer to obtain first detection data for the first training image and second detection data for the first reconstructed image; calculate the loss between the first detection data and the preset label data, and the loss between the second detection data and the preset label data, based on the first detection data, the second detection data, and the preset label data, combined with the first loss function corresponding to the preset encoder and the preset fully connected layer, to train the parameters of the preset encoder and the preset fully connected layer; and determine the trained preset encoder as the target encoder and the trained preset fully connected layer as the target fully connected layer. The first determining unit is used to determine the target encoder and the target fully connected layer as an image quality detection model.

11. The apparatus according to claim 10, characterized in that, The feature perturbation unit is specifically used to perform multiple different feature perturbations on the first latent variable to obtain multiple different feature perturbation vectors. The reconstruction unit is specifically used to perform reconstruction processing on the multiple different feature perturbation vectors through the preset decoder to obtain multiple different first reconstructed images; The training unit is further configured to train the preset encoder and the preset fully connected layer based on the first latent variable, the multiple different first reconstructed images and the preset label data, so as to obtain the target encoder and the target fully connected layer.

12. The apparatus according to claim 10, characterized in that, The device further includes: a first enhancement unit; The first enhancement unit is used to perform image enhancement on the first training image to obtain a first enhanced image of the first training image; The first feature extraction unit is specifically used to extract features from the first enhanced image through the preset encoder to obtain the first latent variable.

13. The apparatus according to claim 10, characterized in that, The device further includes: a second enhancement unit; The second enhancement unit is used to perform image enhancement on the second training image to obtain a second enhanced image of the second training image; The first feature extraction unit is further configured to extract features from the second enhanced image through the initial encoder to obtain a third latent variable; The reconstruction unit is also used to perform reconstruction processing on the third latent variable through the initial decoder to obtain a second reconstructed image; The training unit is further configured to train the parameters of the initial encoder and the parameters of the initial decoder based on the second reconstructed image, the second enhanced image, and the second loss function. The first determining unit is further configured to determine the initial encoder that has completed training as the preset encoder and the initial decoder that has completed training as the preset decoder.

14. An image quality detection device, characterized in that, The device includes: an acquisition unit, a second feature extraction unit, a detection unit, and a second determination unit; The acquisition unit is used to acquire the image of the product to be inspected; The second feature extraction unit is used to extract features from the image to be detected through the target encoder in the image quality detection model to obtain the fourth latent variable; The detection unit is used to perform quality detection on the fourth latent variable through the target fully connected layer in the image quality detection model to obtain the target detection data of the image to be detected. The second determining unit is used to determine the quality detection result of the image to be detected based on the target detection data; The image quality detection model is obtained by performing the training method of the image quality detection model according to any one of claims 1-8.

15. A computer device, characterized in that, The computer device includes a processor and memory: The memory is used to store program code and transmit the program code to the processor; The processor is configured to execute the training method of the image quality detection model according to any one of claims 1-8, or the image quality detection method according to claim 9, according to the instructions in the program code.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program, which, when executed by a processor, performs the training method of the image quality detection model according to any one of claims 1-8, or the image quality detection method according to claim 9.

17. A computer program product, characterized in that, Includes a computer program or instructions; when the computer program or instructions are executed by a processor, the training method of the image quality detection model according to any one of claims 1-8, or the image quality detection method according to claim 9, is performed.

Images