Continuous learning method and device for picture quality inspection, equipment and medium

Abstract

The invention discloses a continuous learning method and device for picture quality inspection, equipment and a medium, and the method comprises the steps: constructing a cue word pool, carrying out the similarity calculation of picture quality inspection data and a cue vector in the cue word pool through employing a pre-training visual converter model, and constructing an input vector of the pre-training visual converter model, furthermore, on the basis of the loss function, the prompt vector in the prompt language pool, the correspondingly associated key vector and the parameters of the classifier for picture quality inspection inference are optimized and updated, picture quality inspection data for training do not need to be stored while continuous learning is met, a task identifier does not need to be accessed during inference, and the learning efficiency is improved. The technical problems that in an existing method, a machine learning model is adopted for picture quality inspection, the storage space is limited, and task identifiers are relied on are solved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a continuous learning method, device, equipment and medium for image quality inspection. Background Art

[0002] In the insurance customer service quality inspection project, it is very important to ensure that the images sent out meet the regulations and requirements. Traditional image quality inspection methods mainly rely on manual review and rule engines, but these methods have some obvious limitations. Manual review is not only inefficient, but also may have inconsistent judgment criteria due to subjectivity. Although rule engines can automate part of the process, they often lack flexibility and are difficult to adapt to the ever-changing forms of violations.

[0003] With the advancement of artificial intelligence technology, especially the development of machine learning models, image quality inspection has begun to move towards automation and intelligence. Machine learning models can identify illegal content in images by learning a large amount of data, greatly improving the efficiency and accuracy of quality inspection. However, these models also face challenges in practical applications, especially in continuous learning.

[0004] Storage space limitations: Some continual learning methods use a memory buffer to store samples of previously learned tasks so that they can reuse this data when learning new tasks later. Although this method helps to keep the memory of old tasks, it requires a lot of storage space and has limited scalability when the number of tasks increases.

[0005] Task Identifier Dependence: Other approaches rely on knowing the identifier of the current task at test time in order to retrieve knowledge of previously learned tasks. This may be impractical in real applications, where the task identifier may be unknown or the system needs to be able to automatically recognize and adapt to new tasks as they arise. Summary of the invention

[0006] The present application provides a continuous learning method, apparatus, device and medium for image quality inspection, which solves the technical problems of storage space limitation and reliance on task identifiers in existing methods for image quality inspection using machine learning models.

[0007] In view of this, the first aspect of the present application provides a continuous learning method for image quality inspection, the method comprising:

[0008] Constructing a prompt pool containing key-value pairs, wherein the key-value pairs are composed of a plurality of prompt vectors and key vectors respectively associated with the prompt vectors;

[0009] Filtering out the first several key vectors from the prompt pool in descending order of similarity, and combining the prompt vectors corresponding to the key vectors into a first vector group;

[0010] Obtaining a first embedding vector of the training image quality inspection data through an embedding vector module of a pre-trained visual transformer model;

[0011] After concatenating the first vector group with the first embedding vector, the concatenated vectors are used as input vectors of the pre-trained visual transformer model to obtain a first encoding vector output by a self-attention encoding module of the pre-trained visual transformer model;

[0012] The prompt vector, the key vector and the quality inspection parameters of the image quality inspection inference classifier are optimized and updated through a loss function composed of the first encoding vector and the first vector group.

[0013] Optionally, the training image quality inspection data is pre-acquired, and the specific acquisition steps include:

[0014] Acquire training image quality inspection data, wherein the training image quality inspection data includes image quality inspection data of several different quality inspection types;

[0015] According to the quality inspection type, the training image quality inspection data is split and integrated to form several image quality inspection data groups.

[0016] Optionally, the sorting in descending order of similarity to select the first several key vectors from the prompt pool, and combining the prompt vectors corresponding to the key vectors into a first vector group is specifically:

[0017] Taking any image quality inspection data in the training image quality inspection data as input of the preset query mapping function;

[0018] Based on a preset similarity function, the similarity between the image quality inspection data and each of the key vectors is calculated by minimizing the sum of similarities between the key vector corresponding to each of the prompt vectors and the output of the query mapping function;

[0019] Filter out the first several key vectors by sorting in descending order of similarity;

[0020] The prompt vectors corresponding to the key vectors are combined into a first vector group.

[0021] Optionally, the optimizing and updating of the prompt vector, the corresponding associated key vector, and the quality inspection parameters of the image quality inspection inference classifier by using the loss function formed by the first encoding vector and the first vector group is specifically as follows:

[0022] Calculating a first loss value of an image quality inspection inference classifier according to the first encoding vector and the corresponding quality inspection label of the training image quality inspection data;

[0023] Based on the weighting coefficient, combined with the average of the sum of similarities between the key vectors corresponding to the prompt vectors in the first vector group and the output of the query mapping function, a second loss value is obtained by weighted calculation;

[0024] With the purpose of minimizing the sum of the first loss value and the second loss value, the prompt vector in the prompt pool, the corresponding associated key vector, and the quality inspection parameters of the image quality inspection classifier are optimized and updated.

[0025] Optionally, after optimizing and updating the prompt vector, the corresponding key vector, and the quality inspection parameter of the image quality inspection inference classifier by using the loss function composed of the first encoding vector and the first vector group, the method further includes:

[0026] Obtain quality inspection data of the image to be predicted;

[0027] Based on the quality inspection data of the image to be predicted, the first several key vectors are selected from the prompt pool in descending order of similarity, and the prompt vectors corresponding to the key vectors are combined into a second vector group;

[0028] Obtaining a second embedding vector of the to-be-predicted image quality inspection data through an embedding vector module of the pre-trained visual transformer model;

[0029] After concatenating the second vector group with the second embedding vector, the concatenated vectors are used as input vectors of the pre-trained visual transformer model to obtain a second encoding vector output by the self-attention encoding module of the pre-trained visual transformer model;

[0030] The second encoding vector is used as the input of the picture quality inspection inference classifier to obtain the quality inspection label of the to-be-predicted picture quality inspection data output by the picture quality inspection inference classifier.

[0031] Optionally, after obtaining the quality inspection label of the to-be-predicted image quality inspection data output by the image quality inspection inference classifier, the method further includes:

[0032] Updating the training image quality inspection data by using the to-be-predicted image quality inspection data and its quality inspection label;

[0033] Return and re-execute the optimization update of the prompt vector, the corresponding associated key vector, and the quality inspection parameters of the image quality inspection inference classifier.

[0034] Optionally, after obtaining the quality inspection label of the to-be-predicted image quality inspection data output by the image quality inspection inference classifier, the method further includes:

[0035] Obtaining user feedback on the quality inspection label of the quality inspection data of the image to be predicted;

[0036] Determining whether to update the quality inspection label according to the feedback information;

[0037] If the quality inspection label is updated, the feedback information is associated with the quality inspection labels before and after the prediction image quality inspection data is updated to form new training image quality inspection data.

[0038] A second aspect of the present application provides a continuous learning device for image quality inspection, the device comprising:

[0039] A construction unit, used to construct a prompt pool containing key-value pairs, wherein the key-value pairs are composed of a plurality of prompt vectors and key vectors respectively associated with the prompt vectors;

[0040] A first processing unit, configured to filter out first several key vectors from the prompt word pool in descending order of similarity, and combine prompt vectors corresponding to the key vectors into a first vector group;

[0041] A second processing unit, configured to obtain a first embedding vector of the training image quality inspection data through an embedding vector module of a pre-trained visual transformer model;

[0042] a third processing unit, configured to concatenate the first vector group and the first embedding vector as an input vector of the pre-trained visual transformer model, and obtain a first encoding vector output by a self-attention encoding module of the pre-trained visual transformer model;

[0043] An updating unit is used to optimize and update the quality inspection parameters of the prompt vector, the key vector and the image quality inspection inference classifier through a loss function composed of the first encoding vector and the first vector group.

[0044] A third aspect of the present application provides a continuous learning device for image quality inspection, the device comprising a processor and a memory:

[0045] The memory is used to store program code and transmit the program code to the processor;

[0046] The processor is used to execute the steps of the method for continuous learning for image quality inspection as described in the first aspect above according to the instructions in the program code.

[0047] A fourth aspect of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store program code, and the program code is used to execute the method described in the first aspect above.

[0048] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0049] In the present application, a continuous learning method, device, equipment and medium for image quality inspection are provided. By constructing a prompt word pool, a pre-trained visual transformer model is used to calculate the similarity between the image quality inspection data and the prompt vector in the prompt word pool, thereby constructing the input vector of the pre-trained visual transformer model, and further based on the loss function, the prompt vector in the prompt word pool and the corresponding associated key vector, as well as the parameters of the classifier used for image quality inspection inference are optimized and updated. While meeting the requirements of continuous learning, there is no need to store the image quality inspection data used for training, and there is no need to access the task identifier during inference, which solves the technical problems of storage space limitation and dependence on task identifier in the existing method of using machine learning models for image quality inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A method flow chart of a continuous learning method for image quality inspection in an embodiment of the present application;

[0051] Figure 2 Schematic diagram of the structure of a continuous learning device for image quality inspection in an embodiment of the present application;

[0052] Figure 3 Schematic diagram of the structure of the continuous learning device used for image quality inspection in the embodiment of the present application. DETAILED DESCRIPTION

[0053] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0054] The present application designs a continuous learning method, device, equipment and medium for image quality inspection, which solves the technical problems of storage space limitation and dependence on task identifiers in existing methods that use machine learning models for image quality inspection.

[0055] For easier understanding, see Figure 1 , Figure 1 This is a method flow chart of the continuous learning method for image quality inspection in the embodiment of the present application, such as Figure 1 As shown, specifically:

[0056] Before executing the continuous learning method for image quality inspection of the present application, it is first necessary to obtain training image quality inspection data;

[0057] Specifically:

[0058] Obtain training image quality inspection data, which includes image quality inspection data of several different quality inspection types;

[0059] According to the quality inspection type, the training image quality inspection data is split and integrated to form several image quality inspection data groups;

[0060] It should be noted that the image datasets collected for training models, such as false propaganda, exaggerated descriptions, etc., contain the images themselves and the corresponding quality inspection labels.

[0061] In order to train the model more effectively, the training image data needs to be classified according to the quality inspection type. This means grouping all the image data according to the type of quality inspection problem they represent, with each group focusing on a specific type of quality inspection.

[0062] Each image quality inspection data set contains all the image data of a specific quality inspection type. The purpose of this is to allow the model to learn specific features and patterns for each quality inspection type.

[0063] The image quality inspection data in each data group consists of the following parts:

[0064] Quality inspection type: identifies the type of quality inspection problem the image belongs to.

[0065] Total data volume: The number of images included in the data group of this quality inspection type.

[0066] Position information: The index or position of each image in the dataset, which helps the model locate and reference it during processing.

[0067] Quality inspection label: A label associated with each image that indicates the quality status or type of problem with the image.

[0068] Each image dataset can be viewed as a structured dataset that contains all the necessary information related to a specific quality inspection type. This structured approach helps the model learn and identify the characteristics of different quality inspection problems more systematically.

[0069] Using these organized datasets, the model can be trained using supervised learning. During the training process, the model learns how to predict the quality inspection label of the input image based on its features.

[0070] The training goal of the model is to minimize the difference between the predicted labels and the actual labels, which is usually achieved through optimization algorithms (such as gradient descent) and loss functions (such as cross entropy loss).

[0071] The generalization ability of the model:

[0072] By training on multiple quality inspection types, the model can not only identify specific problems, but also improve its generalization ability, that is, it can accurately perform quality inspection when faced with new and unseen image data.

[0073] This series of image quality inspection data is expressed as:

[0074] D={D1,D2,…,D t};

[0075] Among them, the t-th quality inspection point represents Among them, n t represents the number of quality inspection images of the tth category, represents the i-th image of the t-th category quality inspection, Represents the quality inspection label of the i-th image of the t-th category quality inspection.

[0076] Refinement: The dataset may contain images of defects of different types (such as scratches, stains, blur, etc.).

[0077] For example, the data set contains 1,000 pictures, each of which has a corresponding quality inspection label, such as "qualified" or "unqualified".

[0078] S101, constructing a prompt pool containing key-value pairs, where the key-value pairs are composed of a number of prompt vectors and key vectors respectively associated with the prompt vectors;

[0079] It should be noted that, assuming there are M prompt vectors in the pool, the prompt pool P is expressed as:

[0080] P={P1,P2,…,P M};

[0081] Associating each prompt vector as a value vector to a learnable key vector, we can get M key-value pairs, represented as follows:

[0082] {(K1,P1),(K2,P2),…,(K M ,P M )};

[0083] in That is, the key vector K i It is a D k dimensional real vector, represents the set of real numbers, D k is the dimension of the key vector, which is learnable.

[0084] In machine learning and deep learning, "key vector" usually refers to a vector used together with a hint vector (or query vector) to calculate similarity or distance to find the most relevant items in a retrieval task or similarity matching task. The key vector is learnable, which usually means that the key vector is not a fixed value set in advance, but is automatically adjusted and optimized by a learning algorithm during the model training process.

[0085] For example, the prompt pool may contain 100 prompt vectors, each of which is associated with a key vector. These vectors are pre-trained and can represent different image features.

[0086] S102, sorting in descending order of similarity to select the first several key vectors from the prompt pool, and combining the prompt vectors corresponding to the key vectors into a first vector group;

[0087] Specifically:

[0088] Taking any image quality inspection data in the training image quality inspection data as input of the preset query mapping function;

[0089] Based on the preset similarity function, the similarity between the key vector corresponding to each prompt vector and the output of the query mapping function is minimized to calculate the similarity between the image quality inspection data and each key vector;

[0090] Sort by similarity in descending order to filter out the first several key vectors;

[0091] The prompt vectors corresponding to the key vectors are combined into a first vector group.

[0092] It should be noted that, based on the training data, the prompt vectors that are most similar to the query mapping function output are found to form the first vector group.

[0093] The first vector group is a set of prompt vectors corresponding to key vectors related to the quality inspection data of the training images. These prompt vectors help the model understand and process specific features or information in the image by associating with the key vectors.

[0094] Refinement: Use the similarity function to calculate the similarity between each hint vector and the output of the query mapping function, and select the top N hint vectors in descending order of similarity.

[0095] For example: if N=5, then the 5 prompt vectors with the highest similarity will be selected.

[0096] During training, an instance query x is used to dynamically select the N prompt vectors in the prompt pool that are most relevant to the current input, expressed as follows:

[0097]

[0098] The final selected N top value vectors are expressed as sim(·,·) represents the similarity function, q(·) represents the query mapping function, is the key vector in the prompt word pool, P s The prompt vector representing the filtered prompt word pool.

[0099] Commonly used similarity functions include but are not limited to: cosine similarity function, Euclidean distance function, Manhattan distance function, Gerard similarity coefficient, Hamming distance function, Mahalanobis distance function, etc. The following is an introduction to the application scenarios and advantages and disadvantages of each similarity function:

[0100] Cosine similarity function:

[0101] Application scenarios: Commonly used in text analysis, recommendation systems, and any scenario that requires measuring angular similarity. Cosine similarity measures the similarity of two vectors in direction, regardless of their magnitude.

[0102] Pros: Not sensitive to length or scale, only direction.

[0103] Disadvantage: Undefined if vector length is zero.

[0104] Euclidean distance function:

[0105] Application scenarios: Applicable to scenarios where the straight-line distance between two points in space needs to be measured, such as geographic coordinates, physical distance, or points in multidimensional space.

[0106] Advantages: Intuitive, easy to understand and calculate.

[0107] Disadvantages: sensitive to the size of the data and may be affected by high dimensionality (curse of dimensionality).

[0108] Manhattan distance function:

[0109] Application scenarios: Suitable for distance calculation of grid-like structures, such as the distance of city blocks, or when each dimension of the data points is independent and evenly distributed.

[0110] Advantages: Simple calculation, suitable for grid path planning.

[0111] Disadvantages: May not be intuitive compared to Euclidean distance.

[0112] Gerard similarity coefficient:

[0113] Application scenarios: Often used to measure the similarity between sets, such as measuring the similarity of user interests in a recommendation system.

[0114] Advantages: The value range is between 0 and 1, which is easy to interpret.

[0115] Disadvantages: May not be sensitive enough to imbalanced datasets.

[0116] Hamming distance function:

[0117] Application scenarios: Used to measure the difference between two strings of equal length, commonly used in text similarity calculation, error detection and DNA sequence comparison.

[0118] Advantages: Simple, fast and easy to calculate.

[0119] Disadvantages: It only considers whether the positions are the same, but not the similarity of the characters themselves.

[0120] Mahalanobis distance function:

[0121] Application scenarios: Applicable to distance calculation when data has different variances or correlations, such as measuring the risk of an investment portfolio in financial data analysis.

[0122] Advantages: It takes into account the covariance of the data and can handle the heteroskedasticity of the data.

[0123] Disadvantages: The computational complexity is high and the covariance matrix needs to be estimated.

[0124] When choosing a similarity function or a distance function, you need to consider the characteristics of the data, the requirements of the application scenario, and the limitations of computing resources. For example, for text data, you may prefer to use cosine similarity, while for geographic coordinate data, you may be more suitable to use Euclidean distance. The correct choice of similarity or distance function is crucial to improving the performance and accuracy of the model.

[0125] Query Mapping Function plays an important role in machine learning and especially in applications of continuous learning, transfer learning or similarity search. It is usually used to map input data (such as pictures, text, etc.) to a low-dimensional feature space for further processing, such as similarity comparison, clustering or classification.

[0126] S103, obtaining a first embedding vector of the training image quality inspection data through an embedding vector module of a pre-trained visual transformer model;

[0127] It should be noted that the pre-trained visual transformer model is used to extract the feature representation of the training image. Specifically, the embedding vector can be obtained by the embedding vector module of the pre-trained visual transformer model, that is, represents the self-embedding vector module of the pre-trained model ViT, x e Represents the embedding vector of the input vector x.

[0128] The first embedding vector is obtained by the embedding vector module of the pre-trained visual transformer model, which represents the feature representation of the training image quality inspection data. The embedding vector module is usually used to convert the original image data (such as pixel values) into a numerical vector form that can be processed by the model.

[0129] Refinement: The embedding vector module converts the image into a fixed-dimensional vector that contains the key features of the image.

[0130] For example: For each image, the model outputs a 512-dimensional embedding vector.

[0131] The pre-trained Vision Transformer model (ViT) is a model that applies the Transformer architecture widely used in natural language processing to computer vision tasks. ViT works by splitting the image into fixed-size patches, then linearly embedding these patches into a series of vectors, and then inputting these vector sequences into a standard Transformer encoder.

[0132] S104, concatenating the first vector group and the first embedding vector as the input vector of the pre-trained visual transformer model, and obtaining a first encoding vector output by the self-attention encoding module of the pre-trained visual transformer model;

[0133] It should be noted that the first vector group is combined with the first embedding vector and input into the visual transformer model to obtain the encoded representation.

[0134] Concatenating the first vector group with the first embedding vector means merging these vectors into a single input vector. This concatenation operation allows the model to simultaneously receive the direct feature representation of the image (through the embedding vector) and auxiliary information related to the specific task (through the hint vector).

[0135] The self-attention mechanism is a technique that allows a model to dynamically adjust its focus based on the importance of different parts of the data when processing sequence data. In the visual transformer model, the self-attention encoding module generates an encoding vector that captures the important features of the input data by calculating the interactions between the input vectors.

[0136] The first encoding vector is the output of the self-attention encoding module, which represents the high-level feature representation of the input data (i.e. the concatenated first vector group and the first embedding vector). This encoding vector will be used in subsequent processing steps such as classification, regression or other tasks.

[0137] Refinement: The encoding vector is the model's internal representation of the input data, which is used for subsequent classification and optimization. Specifically:

[0138] For example, the encoded vector may have 1024 dimensions, containing the combined features of the image and the hint vector.

[0139] Add the selected N prompt vectors to the input vector to obtain:

[0140]

[0141] Among them, x e Represents the embedding vector of the input vector x.

[0142] Represents the self-attention encoding module of the pre-trained model ViT, through A first encoding vector may be obtained.

[0143] S105, optimizing and updating the quality inspection parameters of the prompt vector, the key vector, and the image quality inspection inference classifier through a loss function composed of the first encoding vector and the first vector group;

[0144] Specifically:

[0145] Calculate a first loss value of the image quality inspection inference classifier according to the first encoding vector and the quality inspection label of the corresponding training image quality inspection data;

[0146] Based on the weighting coefficient, a second loss value is obtained by weighted calculation in combination with an average of the sum of similarities between the key vector corresponding to each prompt vector in the first vector group and the output of the query mapping function;

[0147] With the purpose of minimizing the sum of the first loss value and the second loss value, the prompt vectors in the prompt pool, the corresponding associated key vectors, and the quality inspection parameters of the image quality inspection classifier are optimized and updated.

[0148] It should be noted that the model parameters are optimized through the loss function to improve the accuracy of image quality detection.

[0149] The loss function consists of:

[0150] First loss value: This is calculated based on the difference between the model's output (the first encoded vector) and the actual quality inspection label. This loss value measures the accuracy of the model's quality inspection of the training image in its current state. Usually, this loss value is calculated using a loss function such as cross entropy loss or mean square error.

[0151] Second loss value: This loss value is calculated based on the similarity between the prompt vector and the key vector. Specifically, it calculates the mean of the sum of the similarities between the key vector corresponding to each prompt vector in the first vector group and the output of the query mapping function. This loss value helps to optimize the key vector so that it is better aligned with the relevant prompt vector, thereby improving the model's ability to capture image features.

[0152] The weighting coefficient is used to balance the weights of the first loss value and the second loss value in the total loss function. This balance is important because it allows the model to optimize the relationship between the key vector and the prompt vector while pursuing accurate prediction of the quality inspection label to better understand and process the image data.

[0153] Minimize total loss: The training goal of the model is to minimize the weighted sum of the first loss value and the second loss value. This is usually achieved through gradient descent or other optimization algorithms, which adjust the model parameters (including the prompt vector, key vector, and the quality parameters of the classifier) ​​to reduce the loss.

[0154] Parameter update: In each iteration, the model parameters are updated according to the gradient of the loss function. This process is repeated until the model achieves satisfactory performance on the training data or reaches a predetermined number of iterations.

[0155] Through this optimization process, the model is able to more accurately predict the quality of the image, while better understanding and processing the features in the image. This not only improves the performance of the model on the training data, but also helps the model perform better on new, unseen data, that is, improves the generalization ability of the model.

[0156] Refinement: The loss function combines the output of the classifier and the similarity of the prompt vector, aiming to minimize the error between the two.

[0157] For example, if the classifier predicts that a certain image is "qualified", but it is actually "unqualified", then the loss function will calculate this error and adjust the model parameters during training to reduce this error.

[0158] The loss function is as follows:

[0159]

[0160] Among them, φ represents the classifier g φ (·) parameters, that is, the parameters that need to be trained; P and K represent the “prompt vector” and “key vector” of the prompt word pool respectively; represents the self-attention encoding module of the pre-trained model ViT, which outputs an encoding vector; sim(·,·) represents the similarity function, and the cosine similarity is generally used by default; x prepresents the input image data; y represents the quality inspection label or the expected output; λ is the weight coefficient of the regularization term, which is used to control the influence of the similarity term in the total loss; q(x) is the query mapping function, is the key vector in the prompt word pool.

[0161] Furthermore, after optimizing and updating the prompt vectors and the corresponding key vectors in the prompt pool and the parameters of the classifier used for image quality inspection inference through the loss function composed of the first encoding vector and the first vector group, it also includes:

[0162] Obtain quality inspection data of the image to be predicted;

[0163] Based on the quality inspection data of the image to be predicted, the first several key vectors are selected from the prompt pool in descending order of similarity, and the prompt vectors corresponding to the key vectors are combined into a second vector group;

[0164] Obtaining a second embedding vector of the to-be-predicted image quality inspection data through an embedding vector module of the pre-trained visual transformer model;

[0165] After concatenating the second vector group with the second embedding vector, the concatenated vectors are used as the input vectors of the pre-trained visual transformer model to obtain the second encoding vector output by the self-attention encoding module of the pre-trained visual transformer model;

[0166] The second encoded vector is used as the input of the image quality inspection inference classifier to obtain the quality inspection label of the to-be-predicted image quality inspection data output by the image quality inspection inference classifier.

[0167] It should be noted that after the model training is completed, the model is used to perform quality detection on new image data, and the model is further optimized based on the results.

[0168] Refinement: This involves acquiring new image data to be predicted, using the model to make predictions, and feeding the predictions back into the training process.

[0169] For example: For 100 new images to be predicted, the model will output their quality labels and further train the model based on these labels.

[0170] Furthermore, after obtaining the quality inspection label of the to-be-predicted image quality inspection data output by the image quality inspection inference classifier, the following steps are also included:

[0171] Update the training image quality inspection data through the quality inspection data of the image to be predicted and its quality inspection labels;

[0172] Return to re-execute the optimization update of the prompt vector, the corresponding associated key vector, and the quality inspection parameters of the image quality inspection inference classifier.

[0173] It should be noted that after the model performs quality inspection on the predicted images and generates quality inspection labels, these images and their quality inspection labels are added to the training data set. The entire process of this application is an iterative process. Through continuous training and optimization, the model can better learn and adapt to new image data and improve its performance in image quality detection tasks.

[0174] Furthermore, after obtaining the quality inspection label of the to-be-predicted image quality inspection data output by the image quality inspection inference classifier, the following steps are also included:

[0175] Obtain user feedback on the quality inspection labels of the predicted image quality inspection data;

[0176] Determine whether to update the quality inspection label according to the feedback information;

[0177] If the quality inspection label is updated, the feedback information is associated with the quality inspection labels before and after the predicted image quality inspection data is updated to form new training image quality inspection data;

[0178] Otherwise, the training image quality inspection data is updated using the quality inspection data of the image to be predicted and its quality inspection label.

[0179] It should be noted that obtaining user feedback: after the model generates quality inspection labels, collect user feedback on these labels. This can be a direct evaluation of the accuracy of the labels, or additional observations by users on the quality of the images.

[0180] Determine whether to update labels: Based on user feedback, decide whether to update the quality inspection labels generated by the model. This step is critical because it involves the accuracy of model predictions and user satisfaction.

[0181] Update QA labels: If you decide to update the QA labels, incorporate user feedback and the updated QA labels into the training data. This helps the model learn more accurate QA standards.

[0182] Do not update the quality inspection labels: If the user's feedback does not lead to an update of the quality inspection labels, the image to be predicted and its quality inspection label are still added to the training data. This is done to maintain the diversity and integrity of the data even in the absence of direct feedback.

[0183] Retraining: Retrain the model using the updated training data. This includes optimizing the parameters of the prompt vector, key vector, and image quality inspection inference classifier.

[0184] Optimization and update: Through continuous iterative training, the model can better adapt to new image data and improve its performance in image quality detection tasks.

[0185] The key to this process is that it is a dynamic, iterative process that allows the model to continuously learn from user feedback, thereby improving the accuracy and reliability of its predictions. In this way, the model can adapt to the ever-changing data environment and user needs, ensuring its effectiveness and efficiency in real-world applications.

[0186] See also Figure 2 , Figure 2 Schematic diagram of the structure of the continuous learning device for image quality inspection in the embodiment of the present application, such as Figure 2 As shown, specifically:

[0187] A construction unit 201 is used to construct a prompt pool containing key-value pairs, wherein the key-value pairs are composed of a plurality of prompt vectors and key vectors respectively associated with the prompt vectors;

[0188] A first processing unit 202 is used to filter out the first several key vectors from the prompt pool in descending order of similarity, and combine the prompt vectors corresponding to the key vectors into a first vector group;

[0189] A second processing unit 203, configured to obtain a first embedding vector of the training image quality inspection data through an embedding vector module of a pre-trained visual transformer model;

[0190] A third processing unit 204 is configured to concatenate the first vector group and the first embedding vector as an input vector of the pre-trained visual transformer model, and obtain a first encoding vector output by a self-attention encoding module of the pre-trained visual transformer model;

[0191] The updating unit 205 is used to optimize and update the quality inspection parameters of the prompt vector, the key vector and the image quality inspection inference classifier through a loss function composed of the first encoding vector and the first vector group.

[0192] Another embodiment of the present invention provides a continuous learning device for image quality inspection, such as Figure 3 As shown, the device 10 includes:

[0193] One or more processors 110 and memory 120, Figure 3 A processor 110 is used as an example for description. The processor 110 and the memory 120 may be connected via a bus or other means. Figure 3 The example of connecting through bus is taken in the following.

[0194] The processor 110 is used to complete various control logics of the device 10, and it can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a single-chip microcomputer, an ARM (Acorn RISCMachine) or other programmable logic device, a discrete gate or transistor logic, a discrete hardware component, or any combination of these components. In addition, the processor 110 can also be any traditional processor, microprocessor or state machine. The processor 110 can also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors combined with a DSP and / or any other such configuration.

[0195] The memory 120 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions corresponding to the method for constructing a multilingual phoneme representation model in the embodiment of the present invention. The processor 110 executes various functional applications and data processing of the device 10 by running the non-volatile software programs, instructions and units stored in the memory 120, that is, implements the method for constructing a multilingual phoneme representation model in the above method embodiment.

[0196] The memory 120 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function; the data storage area may store data created according to the use of the device 10, etc. In addition, the memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 120 may optionally include a memory remotely arranged relative to the processor 110, and these remote memories may be connected to the device 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0197] One or more units are stored in the memory 120, and when executed by one or more processors 110, the following steps are implemented:

[0198] Construct a prompt pool containing key-value pairs, where the key-value pairs consist of a number of prompt vectors and key vectors respectively associated with the prompt vectors;

[0199] Filter out the first several key vectors from the prompt pool in descending order of similarity, and combine the prompt vectors corresponding to the key vectors into a first vector group;

[0200] Obtaining a first embedding vector of the training image quality inspection data through an embedding vector module of a pre-trained visual transformer model;

[0201] After concatenating the first vector group with the first embedding vector, the concatenated vectors are used as input vectors of the pre-trained visual transformer model to obtain a first encoding vector output by a self-attention encoding module of the pre-trained visual transformer model;

[0202] The prompt vector, the key vector, and the quality inspection parameters of the image quality inspection inference classifier are optimized and updated through the loss function composed of the first encoding vector and the first vector group.

[0203] An embodiment of the present invention provides a non-volatile computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by one or more processors, any implementation of the continuous learning method for image quality inspection described in the above embodiments is implemented.

[0204] As an example, non-volatile storage media can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) as external cache memory. By way of illustration and not limitation, RAM can be obtained in many forms such as synchronous RAM (SRAM), dynamic RAM, (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and direct Rambus RAM (DRRAM). The disclosed memory components or memories of the operating environment described herein are intended to include one or more of these and / or any other suitable types of memory.

[0205] In the embodiments of the present application, a continuous learning method, apparatus, device and medium for image quality inspection are provided. A prompt word pool is constructed, and a pre-trained visual transformer model is used to calculate the similarity between image quality inspection data and prompt vectors in the prompt word pool, thereby constructing an input vector of the pre-trained visual transformer model. The prompt vectors in the prompt word pool and the corresponding associated key vectors, as well as the parameters of the classifier used for image quality inspection inference are further optimized and updated based on the loss function. While meeting the requirements of continuous learning, there is no need to store image quality inspection data used for training, and there is no need to access task identifiers during inference. This solves the technical problems of storage space limitations and reliance on task identifiers in existing methods for image quality inspection using machine learning models.

[0206] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0207] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0208] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0209] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0210] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0211] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0212] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (full name in English: Read-Only Memory, English abbreviation: ROM), random access memory (full name in English: Random Access Memory, English abbreviation: RAM), disk or optical disk and other media that can store program codes.

[0213] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

[0214] It should be noted that if software tools or components other than those of our company appear in the embodiments of the present application, they are only used for illustrative purposes and do not represent actual use.

Claims

1. A continuous learning method for image quality inspection, characterized in that: include: Constructing a prompt pool containing key-value pairs, wherein the key-value pairs are composed of a plurality of prompt vectors and key vectors respectively associated with the prompt vectors; Filtering out the first several key vectors from the prompt pool in descending order of similarity, and combining the prompt vectors corresponding to the key vectors into a first vector group; Obtaining a first embedding vector of the training image quality inspection data through an embedding vector module of a pre-trained visual transformer model; After concatenating the first vector group with the first embedding vector, the concatenated vectors are used as input vectors of the pre-trained visual transformer model to obtain a first encoding vector output by a self-attention encoding module of the pre-trained visual transformer model; The prompt vector, the key vector and the quality inspection parameters of the image quality inspection inference classifier are optimized and updated through a loss function composed of the first encoding vector and the first vector group.

2. The continuous learning method for image quality inspection according to claim 1, characterized in that: The training image quality inspection data is obtained in advance, and the specific acquisition steps include: Acquire training image quality inspection data, wherein the training image quality inspection data includes image quality inspection data of several different quality inspection types; According to the quality inspection type, the training image quality inspection data is split and integrated to form several image quality inspection data groups.

3. The continuous learning method for image quality inspection according to claim 1, characterized in that: The method of sorting in descending order of similarity to select the first several key vectors from the prompt pool, and combining the prompt vectors corresponding to the key vectors into a first vector group is specifically as follows: Taking any image quality inspection data in the training image quality inspection data as input of the preset query mapping function; Based on a preset similarity function, the similarity between the image quality inspection data and each of the key vectors is calculated by minimizing the sum of similarities between the key vector corresponding to each of the prompt vectors and the output of the query mapping function; Filter out the first several key vectors by sorting in descending order of similarity; The prompt vectors corresponding to the key vectors are combined into a first vector group.

4. The continuous learning method for image quality inspection according to claim 3, characterized in that: The optimization and updating of the prompt vector, the corresponding key vector, and the quality inspection parameters of the image quality inspection inference classifier by using the loss function composed of the first encoding vector and the first vector group is specifically as follows: Calculating a first loss value of an image quality inspection inference classifier according to the first encoding vector and the corresponding quality inspection label of the training image quality inspection data; Based on the weighting coefficient, combined with the average of the sum of similarities between the key vectors corresponding to the prompt vectors in the first vector group and the output of the query mapping function, a second loss value is obtained by weighted calculation; With the purpose of minimizing the sum of the first loss value and the second loss value, the prompt vector in the prompt pool, the corresponding associated key vector, and the quality inspection parameters of the image quality inspection classifier are optimized and updated.

5. The continuous learning method for image quality inspection according to claim 1, characterized in that: After optimizing and updating the prompt vector, the corresponding key vector, and the quality inspection parameter of the image quality inspection inference classifier by the loss function composed of the first encoding vector and the first vector group, the method further includes: Obtain quality inspection data of the image to be predicted; Based on the quality inspection data of the image to be predicted, the first several key vectors are selected from the prompt pool in descending order of similarity, and the prompt vectors corresponding to the key vectors are combined into a second vector group; Obtaining a second embedding vector of the to-be-predicted image quality inspection data through an embedding vector module of the pre-trained visual transformer model; After concatenating the second vector group with the second embedding vector, the concatenated vectors are used as input vectors of the pre-trained visual transformer model to obtain a second encoding vector output by the self-attention encoding module of the pre-trained visual transformer model; The second encoding vector is used as the input of the picture quality inspection inference classifier to obtain the quality inspection label of the to-be-predicted picture quality inspection data output by the picture quality inspection inference classifier.

6. The continuous learning method for image quality inspection according to claim 5, characterized in that: After obtaining the quality inspection label of the to-be-predicted image quality inspection data output by the image quality inspection inference classifier, the method further includes: Updating the training image quality inspection data by using the to-be-predicted image quality inspection data and its quality inspection label; Return and re-execute the optimization update of the prompt vector, the corresponding associated key vector, and the quality inspection parameters of the image quality inspection inference classifier.

7. The continuous learning method for image quality inspection according to claim 5, characterized in that: After obtaining the quality inspection label of the to-be-predicted image quality inspection data output by the image quality inspection inference classifier, the method further includes: Obtaining user feedback on the quality inspection label of the quality inspection data of the image to be predicted; Determining whether to update the quality inspection label according to the feedback information; If the quality inspection label is updated, the feedback information is associated with the quality inspection labels before and after the prediction image quality inspection data is updated to form new training image quality inspection data.

8. A continuous learning device for image quality inspection, characterized in that: include: A construction unit, used to construct a prompt pool containing key-value pairs, wherein the key-value pairs are composed of a plurality of prompt vectors and key vectors respectively associated with the prompt vectors; A first processing unit, configured to filter out first several key vectors from the prompt word pool in descending order of similarity, and combine prompt vectors corresponding to the key vectors into a first vector group; A second processing unit, configured to obtain a first embedding vector of the training image quality inspection data through an embedding vector module of a pre-trained visual transformer model; a third processing unit, configured to concatenate the first vector group and the first embedding vector as an input vector of the pre-trained visual transformer model, and obtain a first encoding vector output by a self-attention encoding module of the pre-trained visual transformer model; An updating unit is used to optimize and update the quality inspection parameters of the prompt vector, the key vector and the image quality inspection inference classifier through a loss function composed of the first encoding vector and the first vector group.

9. A continuous learning device for image quality inspection, characterized in that: The device comprises a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the continuous learning method for image quality inspection according to any one of claims 1-7 according to the instructions in the program code.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store program code, and the program code is used to execute the continuous learning method for image quality inspection according to any one of claims 1-7.

Images