An image quality evaluation model training method, device, equipment and medium
By batch training the image quality assessment model and setting independent model parameters and loss functions, the problem of insufficient generalization ability of existing models is solved, and high-accuracy image quality assessment is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NINGBO XINLIANXIN MEDICAL TECH CO LTD
- Filing Date
- 2022-07-26
- Publication Date
- 2026-06-26
Smart Images
Figure CN115187569B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image quality assessment technology, and more specifically, to an image quality assessment model training method, apparatus, device, and medium. Background Technology
[0002] Image quality refers to people's subjective evaluation of the visual experience of an image. It is generally considered to refer to the degree of error that the tested image (i.e., the target image) produces relative to the standard image (i.e., the original image) in the human visual system. Image quality can be further divided into image fidelity and image intelligibility. Image fidelity describes the degree of deviation between the processed image and the original image; while image intelligibility indicates the degree to which humans or machines can extract relevant feature information from the image.
[0003] With the development of the internet and information technology, image quality assessment has become an important aspect of various computer vision and image processing applications, such as image recognition, transmission, and enhancement. Existing image quality assessment models, during continuous learning, tend to forget previously learned content after multiple training sessions, resulting in insufficient generalization ability, low accuracy, and an inability to meet current high requirements for image quality. Summary of the Invention
[0004] In view of this, the purpose of this application is to provide an image quality assessment model training method, apparatus, device and medium, wherein the image quality assessment model trained by the method of this application has strong generalization ability and high accuracy.
[0005] In a first aspect, embodiments of this application provide a method for training an image quality assessment model, the method comprising:
[0006] The collected candidate sample images containing the target object are divided into a training sample set and a test sample set; wherein, the training sample set includes a random number of batches of meta-training sets; each batch of the meta-training set corresponds to independent model parameters;
[0007] Construct an initial evaluation model; the initial evaluation model includes initial model parameters, which are used to predict the quality of the corresponding image and obtain a quality prediction score;
[0008] Replace the initial model parameters of the initial evaluation model with the independent model parameters corresponding to the meta-training set of any target batch, and train the initial evaluation model using the meta-training set of the target batch to obtain the adjusted process model parameters corresponding to the meta-training set of each batch.
[0009] Based on the weights matched to the process model parameters, the target model parameters are calculated. When the preset convergence conditions are met, the target evaluation model containing the target model parameters is verified using a test sample set. After the preset training requirements are met, the training is completed.
[0010] In some technical solutions of this application, the aforementioned candidate sample images also correspond to subjective opinion scores, and the independent model parameters include independent quality prediction head parameters; the method further includes:
[0011] The collected candidate sample images containing the target object and the subjective opinion score of each candidate sample image are divided into the training sample set and the test sample set;
[0012] Construct an initial evaluation model that includes initial feature extractor parameters and initial quality prediction head parameters, and a loss function for the initial evaluation model with respect to subjective opinion score and quality prediction score;
[0013] Based on the meta-training sets of each batch, the independent quality prediction head parameters and loss function of the meta-training sets of that batch, the initial evaluation model containing the initial feature extractor parameters is trained respectively to obtain the adjusted process quality prediction head parameters corresponding to the meta-training sets of each batch and the target feature extractor parameters obtained by training the meta-training sets of all batches.
[0014] Based on the weights matched by each of the process quality prediction head parameters, the target quality prediction head parameters are calculated, and thus a target evaluation model containing target feature extractor parameters and target quality prediction head parameters is obtained.
[0015] In some technical solutions of this application, the training sample set, test sample set, and meta-training set are obtained in the following ways:
[0016] Based on the attribute type of the target object, the candidate sample images and their subjective opinion scores are grouped to obtain multiple candidate sample sets;
[0017] A portion of the candidate sample set is used as the training sample set, and the remaining portion is used as the test sample set.
[0018] Based on the clarity of the target object contained in each sample image in the training sample set, the training sample set is divided into multiple groups of meta-training sets.
[0019] In some technical solutions of this application, the loss function is the sum of the absolute values of the differences between the subjective opinion score and the quality prediction score.
[0020] In some technical solutions of this application, the process quality prediction header parameters are obtained in the following ways:
[0021] The process evaluation model is obtained by replacing the initial quality prediction head parameter in the initial evaluation model with any target-independent quality prediction head parameter.
[0022] The meta-training set corresponding to the target batch of the independent quality prediction head parameters is input into the process evaluation model to obtain the quality prediction score of the target batch meta-training set output by the process evaluation model.
[0023] The process quality prediction head parameters are obtained based on the quality prediction score and loss function of the target batch training set.
[0024] In some technical solutions of this application, the process quality prediction header parameters are obtained based on the quality prediction score and loss function of the target batch training set, including:
[0025] The quality prediction loss value of the target batch meta-training set is calculated based on the quality prediction score of the target batch meta-training set and the loss function.
[0026] The target independent quality prediction head parameters are updated by weighting the quality prediction loss value of the target batch training set to obtain the process quality prediction head parameters.
[0027] In some technical solutions of this application, the independent quality prediction header parameters of the meta-training set of the target batch are set in the following manner:
[0028] If the target batch is the first batch, the independent quality prediction head parameters of the meta-training set of the target batch are preset;
[0029] If the target batch is not the first batch, the independent quality prediction head parameters of the meta-training set of the target batch are set according to the independent quality prediction head parameters of the meta-training set of the previous batch.
[0030] Secondly, embodiments of this application provide an image quality assessment model training apparatus, the apparatus comprising:
[0031] The classification module is used to divide the collected candidate sample images containing the target object into a training sample set and a test sample set; wherein, the training sample set includes a random number of batches of meta-training sets; each batch of the meta-training set corresponds to independent model parameters;
[0032] A construction module is used to build an initial evaluation model; the initial evaluation model contains initial model parameters, which are used to predict the quality of the corresponding image and obtain a quality prediction score;
[0033] The training module is used to replace the initial model parameters of the initial evaluation model with the independent model parameters corresponding to the meta-training set of any target batch, and to train the initial evaluation model using the meta-training set of the target batch to obtain the adjusted process model parameters corresponding to the meta-training set of each batch.
[0034] The completion module is used to calculate the target model parameters based on the weights matched by the parameters of each process model, and to verify the target evaluation model containing the target model parameters using a test sample set when the preset convergence conditions are met. After the preset training requirements are met, the training is completed.
[0035] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the image quality assessment model training method described above.
[0036] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when run by a processor, executes the steps of the image quality assessment model training method described above.
[0037] The technical solutions provided by the embodiments of this application may include the following beneficial effects:
[0038] This application's method includes dividing multiple candidate sample images containing target objects into a training sample set and a test sample set; wherein, the training sample set includes a random number of batches of meta-training sets; each batch of the meta-training set corresponds to independent model parameters; constructing an initial evaluation model; the initial evaluation model includes initial model parameters used to predict the quality of corresponding images to obtain a quality prediction score; replacing the initial model parameters of the initial evaluation model with the independent model parameters corresponding to any batch of the target meta-training set, and training the initial evaluation model using the target batch of the meta-training set to obtain adjusted process model parameters corresponding to each batch of the meta-training set; calculating target model parameters according to the weights matched by each process model parameter; and verifying the target evaluation model containing the target model parameters using a test sample set when a preset convergence condition is met, completing the training after meeting the preset training requirements. The image quality evaluation model trained by this application's method has good memory of all training samples, does not forget the features of previously trained samples during continuous training, has strong generalization ability, and high accuracy.
[0039] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description
[0040] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0041] Figure 1 A flowchart illustrating an image quality assessment model training method provided in an embodiment of this application is shown.
[0042] Figure 2 This illustration shows a schematic diagram of a segmentation method for a candidate sample image provided in an embodiment of this application;
[0043] Figure 3 This illustration shows a schematic diagram of an image quality assessment model training device provided in an embodiment of this application;
[0044] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation
[0045] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.
[0046] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.
[0047] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.
[0048] Image quality refers to people's subjective evaluation of the visual experience of an image. It is generally considered to refer to the degree of error that the tested image (i.e., the target image) produces relative to the standard image (i.e., the original image) in the human visual system. Image quality can be further divided into image fidelity and image intelligibility. Image fidelity describes the degree of deviation between the processed image and the original image; while image intelligibility indicates the degree to which humans or machines can extract relevant feature information from the image.
[0049] With the development of the internet and information technology, image quality assessment has become an important aspect of various computer vision and image processing applications, such as image recognition, transmission, and enhancement. Existing image quality assessment models, during continuous learning, tend to forget previously learned content after multiple training sessions, resulting in insufficient generalization ability, low accuracy, and an inability to meet current high requirements for image quality.
[0050] Based on this, embodiments of this application provide a method, apparatus, device, and medium for training an image quality assessment model, which are described below through embodiments.
[0051] Figure 1 The diagram illustrates a flowchart of an image quality assessment model training method provided in an embodiment of this application, wherein the method includes steps S101-S104; specifically:
[0052] S101. The collected multiple candidate sample images containing the target object are divided into a training sample set and a test sample set; wherein, the training sample set includes a random number of batches of meta-training sets; each batch of the meta-training set corresponds to independent model parameters;
[0053] S102. Construct an initial evaluation model; the initial evaluation model includes initial model parameters, which are used to predict the quality of the corresponding image and obtain a quality prediction score;
[0054] S103. Replace the initial model parameters of the initial evaluation model with the independent model parameters corresponding to the meta-training set of any target batch, and train the initial evaluation model using the meta-training set of the target batch to obtain the adjusted process model parameters corresponding to the meta-training set of each batch.
[0055] S104. Calculate the target model parameters based on the weights matched by the parameters of each process model. When the preset convergence condition is met, use the test sample set to verify the target evaluation model containing the target model parameters. After the preset training requirements are met, the training is completed.
[0056] The image quality assessment model trained by the method of this application has a good memory of all training samples. It will not forget the features of the samples trained earlier during continuous training, and has strong generalization ability and high accuracy.
[0057] The following describes some embodiments of this application in detail. Unless otherwise specified, the following embodiments and features can be combined with each other.
[0058] S101. The collected multiple candidate sample images containing the target object are divided into a training sample set and a test sample set; wherein, the training sample set includes a random number of batches of meta-training sets; each batch of the meta-training set corresponds to independent model parameters.
[0059] This application involves capturing multiple candidate sample images of a target object, including specific human bodies, objects, etc. For example, the target object may be a lesion, meaning it is a diseased part of a human or animal body. Furthermore, the candidate sample images in this application are natural images obtained directly from capturing the target object. Natural images represent images that have not undergone any processing after being captured.
[0060] To train the subsequent model, this application divides the multiple candidate sample images into a training sample set and a test sample set. Both the training sample set and the test sample set contain multiple candidate sample images.
[0061] After obtaining the candidate sample images, this application also assigns a subjective opinion score to each candidate sample image. This subjective opinion score is obtained through human evaluation. Specifically, multiple expert members may each give their own evaluation score for the quality of the candidate sample image. Then, the subjective opinion score of the candidate sample image is calculated by combining the evaluation scores of all expert members. Since the candidate sample images in this application also have a subjective opinion evaluation score, this application also assigns the corresponding subjective opinion score to the candidate sample image when dividing it into candidate sample images. That is, the training sample set and test sample set in this application not only include candidate sample images but also the corresponding subjective opinion scores.
[0062] To improve training effectiveness, this application further divides the training sample set into a meta-training set. When dividing the meta-training set, this application randomly extracts samples from the training sample set without replacement. Specifically, a random number of candidate sample images and their corresponding subjective opinion scores are selected from the training sample set to form a batch of meta-training sets, thus obtaining the random number of batches of the meta-training set.
[0063] In this embodiment of the application, as an optional embodiment, the process of segmenting the selected sample image is as follows: Figure 2 As shown, steps S201-S203 are included:
[0064] S201. Group the candidate sample images and their subjective opinion scores according to the attribute type of the target object to obtain multiple candidate sample sets;
[0065] S202. Use a portion of the sample set in the candidate sample set as the training sample set, and the remaining portion of the sample set as the test sample set;
[0066] S203. Based on the clarity of the target object contained in each sample image in the training sample set, the training sample set is divided into multiple groups of meta-training sets.
[0067] This application groups images based on the attribute type of the target object. When the target object is a lesion, candidate sample images containing the same type of lesion and their subjective opinion scores are grouped together. For example, based on the location of the lesion infection, candidate sample images of infected arms and infected legs are divided into two groups. After obtaining multiple candidate sample sets, a portion of the candidate sample sets is selected as the training sample set according to training needs, and the remainder is the test sample set. In practice, the number of training sample sets is generally greater than the number of test sample sets. After obtaining the training sample set, in order to further improve the accuracy of the model, this application further distinguishes the training sample set. Based on the clarity of the target object region contained in each sample image in the training sample set, the training sample set is divided into multiple meta-training sets. For example, multiple sample images of infected legs, with the clarity of the infected area divided into Level 1 clarity, Level 2 clarity, and Level 3 clarity, are then divided into a three-meta-training set.
[0068] In this embodiment of the application, as an optional implementation, N synthetically distorted images and the subjective opinion score of each image are obtained, and the N images are divided into T groups D = {D1, D2, ..., D...} according to the lesion type. t ,…,D T}, and combine the M groups of images in D and their corresponding subjective opinion scores to form the training sample set D. tr The remaining images in D and their corresponding subjective opinion scores form the test sample set D. te Then, based on the clarity of the P types of lesion regions in the images, the training sample set D is divided. tr Divided into Π group training set Among them, D t Let represent the t-th image in the meta-learning process, M≥T / 2, T≥3, Π=M×P, P>3.
[0069] S102. Construct an initial evaluation model; the initial evaluation model is used to predict the quality of the corresponding image and obtain a quality prediction score.
[0070] An initial evaluation model is constructed, and the target evaluation model is obtained by training this initial evaluation model using the meta-training sets from each batch mentioned above. The initial evaluation model in this application includes an image feature extraction subnetwork and an image quality prediction subnetwork. The specific construction process is as follows:
[0071] Construct a no-reference image quality assessment network model Y, which includes a sequentially connected image feature extraction subnetwork R and an image quality prediction subnetwork M. The image feature extraction subnetwork R contains multiple sequentially connected residual units. Each residual unit includes multiple convolutional layers, multiple activation layers, and a batch normalization layer. The sum of the input of the residual unit and the output of the last convolutional layer in the residual unit is used as the input of the last activation layer. M includes multiple fully connected layers arranged in parallel.
[0072] The image feature extraction subnetwork R contains four residual units: Residual Unit 1, Residual Unit 2, Residual Unit 3, and Residual Unit 4. Each residual unit consists of one convolutional layer, one batch normalization layer, one activation layer, one convolutional layer, and one ReLU activation layer connected in series. The sum of the input of each residual unit and the output of the last convolutional layer in that residual unit is used as the input of the last ReLU activation layer. The number of convolutional kernels in the first and second convolutional layers of each residual unit are set to (32, 64), (128, 256), (512, 256), and (128, 64), respectively. The kernel size in both the first and second convolutional layers of each residual unit is set to (1×1, 3×3). The stride of the first and second convolutional layers of each residual unit is set to 1 and 2, respectively.
[0073] The image quality prediction subnetwork M contains 5 fully connected layers, and its specific structure is as follows: global pooling layer → 1st fully connected layer → 2nd fully connected layer → 3rd fully connected layer → 4th fully connected layer → 5th fully connected layer, with the lengths of the serially connected layers being 112, 56, 28, 14 and 1, respectively.
[0074] The initial evaluation model in this application includes an image feature extraction subnetwork and an image quality prediction subnetwork, thus encompassing initial feature extractor parameters and initial quality prediction head parameters. To enhance the generalization ability of the target evaluation model trained on the initial evaluation model, this application sets independent model parameters for each batch of meta-training sets. Training with different independent model parameters for different batches of meta-training sets improves the model's generalization ability. If all batches of meta-training sets are trained using the initial model parameters of the initial evaluation model, the features of later input meta-training sets will overwhelm the features of earlier input meta-training sets during training.
[0075] In setting the independent model parameters, this application sets independent quality prediction head parameters for the linear layer image quality prediction subnetwork, but not for the image feature extraction subnetwork. In other words, the independent model parameters in this application include independent quality prediction head parameters.
[0076] Set the independent quality prediction header parameters for the meta-training set of the target batch in the following manner:
[0077] If the target batch is the first batch, the independent quality prediction head parameters of the meta-training set of the target batch are preset;
[0078] If the target batch is not the first batch, the independent quality prediction head parameters of the meta-training set of the target batch are set according to the independent quality prediction head parameters of the meta-training set of the previous batch.
[0079] To ensure the effectiveness of model training, this application also constructs a loss function for the initial evaluation model corresponding to the subjective opinion score and the quality prediction score, wherein the loss function is the sum of the absolute values of the differences between the subjective opinion score and the quality prediction score.
[0080] In this embodiment of the application, as an optional embodiment, the loss function L of the no-reference image quality assessment network model Y is defined as follows:
[0081]
[0082] Where L represents the quality prediction loss function, Q i Let Q represent the subjective opinion score corresponding to the i-th training sample. pred,i Let represent the quality prediction score corresponding to the i-th training sample in the training set, and Σ represent the summation operation.
[0083] S103. Replace the initial model parameters of the initial evaluation model with the independent model parameters corresponding to the meta-training set of any target batch, and train the initial evaluation model using the meta-training set of the target batch to obtain the adjusted process model parameters corresponding to the meta-training set of each batch.
[0084] After obtaining the batch meta-training sets and their independent model parameters, this application trains the initial evaluation model using the batch meta-training sets. Specifically, any batch is selected as the target batch from multiple batch meta-training sets. The independent model parameters corresponding to the target batch meta-training set replace the initial model parameters contained in the initial evaluation model. Then, the initial evaluation model containing the independent model parameters is trained using the target batch meta-training set, resulting in the process model parameters after training the initial evaluation model with the batch meta-training sets.
[0085] Furthermore, based on the meta-training sets of each batch, the independent quality prediction head parameters and loss functions of the meta-training sets of that batch, the initial evaluation model containing the initial feature extractor parameters is trained respectively to obtain the adjusted process quality prediction head parameters corresponding to each batch of meta-training sets and the target feature extractor parameters obtained from training all batches of meta-training sets.
[0086] The initial evaluation model of this application mainly includes initial feature extractor parameters and initial quality prediction head parameters. The training methods for these two parameters differ. For the initial feature extractor parameters, this application trains them using all batches of meta-training sets to obtain the trained target feature extractor parameters. For the initial quality prediction head parameters, to ensure the final model has strong generalization ability, this application does not directly train the initial quality prediction head parameters included in the initial evaluation model. Instead, it sets independent quality prediction head parameters for different batches of meta-training sets and trains them to obtain the trained target quality prediction head parameters.
[0087] The specific training process is as follows: First, replace the initial quality prediction head parameter in the initial evaluation model with any target independent quality prediction head parameter to obtain the process evaluation model. Second, input the target batch meta-training set corresponding to the target independent quality prediction head parameter into the process evaluation model to obtain the quality prediction score of the target batch meta-training set output by the process evaluation model. Third, calculate the quality prediction loss value of the target batch meta-training set based on the quality prediction score of the target batch meta-training set and the loss function. Fourth, update the weights of the target independent quality prediction head parameter based on the quality prediction loss value of the target batch meta-training set to obtain the process quality prediction head parameter.
[0088] This application sets independent quality prediction head parameters for different batches of meta-training sets and uses these independent quality prediction head parameters to replace the initial quality prediction head parameters in the initial evaluation model to construct the process evaluation model. Different batches of meta-training sets are input into the process evaluation model to obtain the quality prediction scores for each batch of meta-training sets output by the process model. These quality prediction scores are then substituted into the loss function to calculate the quality prediction loss value for each batch of meta-training sets. By calculating the partial derivative of the quality prediction loss value with respect to the independent quality prediction head parameters, and then backpropagating using gradient descent and a pre-defined optimization function, the weights of the independent quality prediction head parameters are updated to obtain the process quality prediction head parameters.
[0089] S104. Calculate the target model parameters based on the weights matched by the parameters of each process model. When the preset convergence condition is met, use the test sample set to verify the target evaluation model containing the target model parameters. After the preset training requirements are met, the training is completed.
[0090] After obtaining the process model parameters for each batch of meta-training sets, corresponding weights are assigned to each process model parameter. The target model parameters are obtained by summing the values of each process model parameter and its corresponding weight. The target evaluation model is then obtained by replacing the initial model parameters in the initial evaluation model with the target model parameters.
[0091] The training process is complete when the target evaluation model reaches the preset convergence condition. After one training cycle, the target evaluation model is validated using a test sample set. Sample images from the test sample set are input into the target evaluation model to obtain the quality prediction score of the test sample set output by the model. The quality prediction score and the subjective opinion score of the test sample set are then substituted into the loss function to calculate the loss value of the test sample set. If the loss value is within a preset range, training is complete; otherwise, training is repeated.
[0092] Furthermore, after obtaining the process quality prediction head parameters for each batch of meta-training sets, weights are assigned to the process quality prediction head parameters for each batch. Based on each process quality prediction head parameter and its weight, the target quality prediction head parameters are calculated. The target feature extractor parameters and target quality prediction head parameters are then used to replace the initial feature extraction parameters and initial quality prediction head parameters in the initial evaluation model to obtain the target evaluation model.
[0093] In this embodiment of the application, as an optional embodiment, the meta-learning-based no-reference image quality assessment network model Y is iteratively trained:
[0094] (3a) Initialize the number of iterations to k, the maximum number of iterations to K, K>20, and the current no-reference image quality assessment network model is Y. t Y t The parameter Φ t For including feature extractor parameters and quality prediction head parameter θ t And let t = 1;
[0095] (3b) Let k = 1;
[0096] (3c) will be trained from the Π group training set The training set of N subsets is randomly selected without replacement, and independent quality prediction head parameters are set for each subset. 1≤n≤N, let The initial value is θ t At this point, we obtain the network model Y corresponding to the nth element training set. n,t The parameter Φ n,t ,
[0097] (3d) Using Φ n,t To obtain the image quality prediction score for the nth group, a loss function L is used. The first step involves obtaining the loss value L for each quality prediction score and the corresponding subjective opinion. n,t Then calculate L n,t For Φ n,t partial derivatives Then, based on the gradient descent method and combined with the Adam optimization function, backpropagation is performed on Φ. n,t The updated result is obtained by updating the weights.
[0098] (3e) Determine if n = N is true. If yes, perform a weighted average. Obtain Φ t Update results Otherwise, n = n + 1, and proceed to step (3d);
[0099] (3e) Determine whether k = K holds true. If so, obtain the trained meta-learning-based no-reference image quality assessment network model Y. t * Otherwise, k = k + 1, and proceed to step (3c);
[0100] (4) Obtain the quality assessment results of the no-reference image:
[0101] Test sample set D te Y, a trained meta-learning-based no-reference image quality assessment network model t *The input is used for forward inference to obtain the quality prediction score for each test sample.
[0102] Figure 3 This illustration shows a schematic diagram of an image quality assessment model training device provided in an embodiment of this application. The device includes:
[0103] The classification module is used to divide the collected candidate sample images containing the target object into a training sample set and a test sample set; wherein, the training sample set includes a random number of batches of meta-training sets; each batch of the meta-training set corresponds to independent model parameters;
[0104] A construction module is used to build an initial evaluation model; the initial evaluation model contains initial model parameters, which are used to predict the quality of the corresponding image and obtain a quality prediction score;
[0105] The training module is used to replace the initial model parameters of the initial evaluation model with the independent model parameters corresponding to the meta-training set of any target batch, and to train the initial evaluation model using the meta-training set of the target batch to obtain the adjusted process model parameters corresponding to the meta-training set of each batch.
[0106] The completion module is used to calculate the target model parameters based on the weights matched by the parameters of each process model, and to verify the target evaluation model containing the target model parameters using a test sample set when the preset convergence conditions are met. After the preset training requirements are met, the training is completed.
[0107] The candidate sample images also correspond to subjective opinion scores, and the independent model parameters include independent quality prediction head parameters;
[0108] The classification module is used to divide the collected multiple candidate sample images containing the target object and the subjective opinion score of each candidate sample image into the training sample set and the test sample set;
[0109] A construction module is used to construct an initial evaluation model containing initial feature extractor parameters and initial quality prediction head parameters, and a loss function corresponding to the initial evaluation model with respect to subjective opinion scores and quality prediction scores.
[0110] The training module is used to train the initial evaluation model containing the initial feature extractor parameters based on the meta-training sets of each batch, the independent quality prediction head parameters of the meta-training sets of that batch, and the loss function, respectively, to obtain the adjusted process quality prediction head parameters corresponding to each batch of meta-training sets and the target feature extractor parameters obtained by training all batches of meta-training sets.
[0111] The completion module is used to calculate the target quality prediction head parameters based on the weights matched by each of the process quality prediction head parameters, and then obtain a target evaluation model that includes target feature extractor parameters and target quality prediction head parameters.
[0112] The training sample set, test sample set, and meta-training set are obtained in the following ways:
[0113] Based on the attribute type of the target object, the candidate sample images and their subjective opinion scores are grouped to obtain multiple candidate sample sets;
[0114] A portion of the candidate sample set is used as the training sample set, and the remaining portion is used as the test sample set.
[0115] Based on the clarity of the target object contained in each sample image in the training sample set, the training sample set is divided into multiple groups of meta-training sets.
[0116] The loss function is the sum of the absolute values of the differences between the subjective opinion score and the quality prediction score.
[0117] The process quality prediction header parameters are obtained in the following manner:
[0118] The process evaluation model is obtained by replacing the initial quality prediction head parameter in the initial evaluation model with any target-independent quality prediction head parameter.
[0119] The meta-training set corresponding to the target batch of the independent quality prediction head parameters is input into the process evaluation model to obtain the quality prediction score of the target batch meta-training set output by the process evaluation model.
[0120] The process quality prediction head parameters are obtained based on the quality prediction score and loss function of the target batch training set.
[0121] The process quality prediction header parameters are obtained based on the quality prediction score and loss function of the target batch training set, including:
[0122] The quality prediction loss value of the target batch meta-training set is calculated based on the quality prediction score of the target batch meta-training set and the loss function.
[0123] The target independent quality prediction head parameters are updated by weighting the quality prediction loss value of the target batch training set to obtain the process quality prediction head parameters.
[0124] Set the independent quality prediction header parameters for the meta-training set of the target batch in the following manner:
[0125] If the target batch is the first batch, the independent quality prediction head parameters of the meta-training set of the target batch are preset;
[0126] If the target batch is not the first batch, the independent quality prediction head parameters of the meta-training set of the target batch are set according to the independent quality prediction head parameters of the meta-training set of the previous batch.
[0127] like Figure 4 As shown, this application provides an electronic device for executing the image quality assessment model training method of this application. The device includes a memory, a processor, a bus, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the image quality assessment model training method described above.
[0128] Specifically, the aforementioned memory and processor can be general-purpose memory and processor, without any specific limitations. When the processor runs the computer program stored in the memory, it can execute the aforementioned image quality assessment model training method.
[0129] Corresponding to the image quality assessment model training method in this application, this application embodiment also provides a computer-readable storage medium storing a computer program, which is executed by a processor to perform the steps of the above-described image quality assessment model training method.
[0130] Specifically, the storage medium can be a general-purpose storage medium, such as a removable disk or hard disk. When the computer program on the storage medium is run, it can execute the image quality assessment model training method described above.
[0131] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. The system embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and there may be other division methods in actual implementation. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the coupling or direct coupling or communication connection shown or discussed may be through some communication interface; the indirect coupling or communication connection between systems or units may be electrical, mechanical, or other forms.
[0132] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0133] In addition, the functional units in the embodiments provided in this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.
[0134] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0135] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.
[0136] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application. All should be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims.
Claims
1. A method for training an image quality assessment model, characterized in that, The method includes: The collected candidate sample images containing the target object are divided into a training sample set and a test sample set; wherein, the training sample set includes a random number of batches of meta-training sets; each batch of the meta-training set corresponds to independent model parameters; wherein, the independent model parameters include independent quality prediction head parameters; the independent quality prediction head parameters are preset or set according to the independent quality prediction head parameters of the previous batch of the meta-training set; Construct an initial evaluation model; the initial evaluation model includes initial feature extractor parameters and initial model parameters, which are used to predict the quality of the corresponding image and obtain a quality prediction score; Replace the initial model parameters of the initial evaluation model with the independent model parameters corresponding to the meta-training set of any target batch to form a process evaluation model; and train the process evaluation model using the meta-training set of the target batch to obtain the adjusted process model parameters corresponding to each batch of meta-training sets and the target feature extractor parameters obtained by training all batches of meta-training sets. The target model parameters are obtained by weighted summation based on the weights matched for each process model parameter. When the preset convergence condition is met, the target evaluation model containing the target model parameters is verified using a test sample set. After the preset training requirements are met, the training is completed.
2. The method according to claim 1, characterized in that, The candidate sample images also correspond to subjective opinion scores; the method further includes: The collected candidate sample images containing the target object and the subjective opinion score of each candidate sample image are divided into the training sample set and the test sample set; Construct an initial evaluation model that includes initial feature extractor parameters and initial quality prediction head parameters, and a loss function for the initial evaluation model with respect to subjective opinion score and quality prediction score; Based on the meta-training sets of each batch, the independent quality prediction head parameters and loss function of the meta-training sets of that batch, the initial evaluation model containing the initial feature extractor parameters is trained respectively to obtain the adjusted process quality prediction head parameters corresponding to the meta-training sets of each batch and the target feature extractor parameters obtained by training the meta-training sets of all batches. Based on the weights matched by each of the process quality prediction head parameters, the target quality prediction head parameters are calculated, and thus a target evaluation model containing target feature extractor parameters and target quality prediction head parameters is obtained.
3. The method according to claim 2, characterized in that, The training sample set, test sample set, and meta-training set are obtained in the following ways: Based on the attribute type of the target object, the candidate sample images and their subjective opinion scores are grouped to obtain multiple candidate sample sets; A portion of the candidate sample set is used as the training sample set, and the remaining portion is used as the test sample set. Based on the clarity of the target object contained in each sample image in the training sample set, the training sample set is divided into multiple groups of meta-training sets.
4. The method according to claim 2, characterized in that, The loss function is the sum of the absolute values of the differences between the subjective opinion score and the quality prediction score.
5. The method according to claim 2, characterized in that, The process quality prediction header parameters are obtained in the following manner: The process evaluation model is obtained by replacing the initial quality prediction head parameter in the initial evaluation model with any target-independent quality prediction head parameter. The meta-training set corresponding to the target batch of the independent quality prediction head parameters is input into the process evaluation model to obtain the quality prediction score of the target batch meta-training set output by the process evaluation model. The process quality prediction head parameters are obtained based on the quality prediction score and loss function of the target batch training set.
6. The method according to claim 5, characterized in that, The process quality prediction header parameters are obtained based on the quality prediction score and loss function of the target batch meta-training set, including: The quality prediction loss value of the target batch meta-training set is calculated based on the quality prediction score of the target batch meta-training set and the loss function. The target independent quality prediction head parameters are updated by weighting the quality prediction loss value of the target batch training set to obtain the process quality prediction head parameters.
7. The method according to claim 2, characterized in that, Set the independent quality prediction header parameters for the meta-training set of the target batch in the following manner: If the target batch is the first batch, the independent quality prediction head parameters of the meta-training set of the target batch are preset; If the target batch is not the first batch, the independent quality prediction head parameters of the meta-training set of the target batch are set according to the independent quality prediction head parameters of the meta-training set of the previous batch.
8. An image quality assessment model training device, characterized in that, The device includes: A classification module is used to divide multiple candidate sample images containing target objects into a training sample set and a test sample set; wherein, the training sample set includes a random number of batches of meta-training sets; each batch of the meta-training set corresponds to independent model parameters; wherein, the independent model parameters include independent quality prediction head parameters; the independent quality prediction head parameters are preset or set according to the independent quality prediction head parameters of the previous batch of the meta-training set; A construction module is used to build an initial evaluation model; the initial evaluation model includes initial feature extractor parameters and initial model parameters, which are used to predict the quality of the corresponding image and obtain a quality prediction score; The training module is used to replace the initial model parameters of the initial evaluation model with the independent model parameters corresponding to the meta-training set of any target batch to form a process evaluation model; and to train the process evaluation model using the meta-training set of the target batch to obtain the adjusted process model parameters corresponding to each batch of meta-training sets and the target feature extractor parameters obtained by training all batches of meta-training sets. The completion module is used to obtain the target model parameters by weighted summation based on the weights matched by the parameters of each process model, and to verify the target evaluation model containing the target model parameters using a test sample set when the preset convergence condition is met. After the preset training requirements are met, the training is completed.
9. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the steps of the image quality assessment model training method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the image quality assessment model training method as described in any one of claims 1 to 7.