A method and apparatus for training an image quality prediction model

Abstract

This application relates to the field of image recognition technology, and provides a method and apparatus for training an image quality prediction model. The method includes: acquiring a training sample set of target images; acquiring quality score enhancement labels for the target images; inputting the target image and the corresponding quality score enhancement labels into an image quality prediction model to obtain an image quality prediction result; the image quality prediction model includes a backbone network and a quality classification head; iteratively updating the parameters of the image quality prediction model according to a target loss function until a preset iteration termination condition is reached, thereby obtaining the trained image quality prediction model. This application generates image quality score labels through data augmentation, which can provide a more reasonable definition of image quality in complex scenes, and transforms the quality score prediction problem into a classification problem, thus helping to improve the accuracy and pass rate of downstream tasks such as image recognition.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, and in particular to a method and apparatus for training an image quality prediction model. Background Technology

[0002] In liveness detection or other face authentication tasks, to ensure the accuracy of subsequent liveness detection / face recognition, enhance security performance, and allow users to perform quality compliance assessments using multiple networks, multiple network models are not conducive to edge deployment, leading to problems such as numerous models and slow speed, thus affecting response speed and user experience. At the same time, the small number of various non-compliant image samples also poses challenges to the stable convergence and accuracy of image quality score prediction.

[0003] Therefore, how to design an image quality prediction model that can comprehensively determine image quality and save inference time at the edge to ensure the accuracy of subsequent image recognition applications and improve user experience is a technical problem that needs to be solved. Summary of the Invention

[0004] In view of this, embodiments of this application provide an image quality prediction model training method, apparatus, electronic device, and computer-readable storage medium to solve the problem of low accuracy in image quality prediction in the prior art.

[0005] A first aspect of this application provides a method for training an image quality prediction model, comprising:

[0006] Obtain the target image training sample set;

[0007] Obtain the quality score enhancement label of the target image;

[0008] The target image and the corresponding quality score enhancement label are input into the image quality prediction model to obtain the image quality prediction result; the image quality prediction model includes a backbone network and a quality classification head.

[0009] Based on the target loss function, the parameters of the image quality prediction model are iteratively updated until a preset iteration termination condition is reached, thereby obtaining the trained image quality prediction model.

[0010] A second aspect of this application provides an image quality prediction model training apparatus, the apparatus comprising:

[0011] The training set acquisition module is configured to acquire a training sample set of the target image;

[0012] The quality score enhancement label acquisition module is configured to acquire the quality score enhancement label of the target image;

[0013] The image quality prediction result output module is configured to input the target image and the corresponding quality score enhancement label into the image quality prediction model to obtain the image quality prediction result; the image quality prediction model includes a backbone network and a quality classification unit.

[0014] The image quality prediction model training module is configured to iteratively update the parameters of the image quality prediction model according to the target loss function until a preset iteration termination condition is reached, thereby obtaining the trained image quality prediction model.

[0015] A third aspect of this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described in the first aspect.

[0016] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0017] The beneficial effects of this application's embodiments compared to existing technologies include at least the following: This application's embodiments acquire a training sample set of target images; acquire quality score enhancement labels for the target images; input the target image and the corresponding quality score enhancement labels into an image quality prediction model to obtain image quality prediction results; the image quality prediction model includes a backbone network and a quality classification head; and iteratively update the parameters of the image quality prediction model according to the target loss function until a preset iteration termination condition is reached, thereby obtaining the trained image quality prediction model. This application generates image quality score labels through data augmentation, which can provide a more reasonable definition of image quality in complex scenes, and transforms the quality score prediction problem into a classification problem, which helps improve the accuracy and pass rate of downstream tasks such as image recognition. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is one of the flowcharts illustrating the image quality prediction model training method provided in this application embodiment;

[0020] Figure 2 This is the second flowchart illustrating the image quality prediction model training method provided in this application embodiment;

[0021] Figure 3 This is the third flowchart illustrating the image quality prediction model training method provided in this application embodiment;

[0022] Figure 4 This is the third flowchart illustrating the image quality prediction model training method provided in this application embodiment;

[0023] Figure 5 This is a schematic diagram of the structure of the image quality prediction model training device provided in the embodiments of this application;

[0024] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0025] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0026] The following describes in detail, with reference to the accompanying drawings, an image quality prediction model training method, apparatus, electronic device, and storage medium according to embodiments of this application.

[0027] As described in the background section, for input to an image recognition system, the quality of the image itself—such as whether it is blurry, its illumination, contrast, and sharpness—as well as the quality of the target to be recognized in the image, such as a face, including whether there is a face, whether key facial features are obscured, whether it is a frontal or side view, and motion blur, all affect the subsequent recognition accuracy. Therefore, it is necessary to control various non-compliant image inputs. However, image quality is a difficult problem to define; what constitutes a high-quality image and what constitutes a low-quality image remains an open question in the industry. Some traditional methods tend to define image-level features, such as illumination symmetry, blur, sharpness, and noise, to define image quality. This method is highly interpretable and directly related to image quality from a human's intuitive perspective. However, currently, end-to-end deep learning networks are more popular in image target recognition, especially in the field of face quality recognition. The focus of this work is also on defining quality labels, but this part of the work has poor interpretability and relies heavily on the richness of images within the class.

[0028] Therefore, with the widespread application of image information technology, image quality assessment has become a broad and fundamental issue. Due to the unparalleled advantages of image information compared to other types of information, the rational processing of image information has become an indispensable tool in various fields. During the acquisition, processing, transmission, and recording of images, imperfections in imaging systems, processing methods, transmission media, and recording equipment, coupled with factors such as object movement and noise pollution, inevitably lead to some image distortion and degradation. This poses significant difficulties for understanding the objective world and researching and solving problems. For example, in image recognition, the quality of the acquired image directly affects the accuracy and reliability of the recognition results. Furthermore, systems such as remote conferencing and video-on-demand are affected by transmission errors and network latency, requiring real-time online image quality monitoring so that service providers can dynamically adjust source location strategies to meet service quality requirements. In military applications, the effectiveness of battlefield surveillance and strike assessment also depends on the quality of images or videos acquired by aerial photography equipment such as drones. Therefore, the rational assessment of image quality has significant application value. From the perspective of whether or not human participation is involved, image quality evaluation methods can be divided into two branches: subjective evaluation and objective evaluation. Subjective evaluation uses humans as observers to evaluate images subjectively, striving to accurately reflect human visual perception; objective evaluation methods use a mathematical model to reflect the subjective perception of the human eye and provide results based on numerical calculations.

[0029] Image quality assessment includes subjective and objective evaluation. Subjective image quality assessment involves only qualitative evaluations made by humans. It uses humans as observers to subjectively assess the quality of an image. The selection of observers generally considers either untrained "laymen" or trained "experts." This method is based on statistical significance; to ensure that subjective image assessments are statistically meaningful, a sufficient number of observers should participate, including both absolute and relative evaluations. For objective image quality assessment, the basic goal is to design a computational model that can accurately and automatically perceive image quality. Its ultimate goal is to use computers to replace the human visual system in viewing and perceiving images. Internationally, objective image quality assessment typically evaluates the consistency between quantitative image quality values ​​obtained through computational models and human subjective observations, testing the performance of multiple factors affecting image quality.

[0030] Most image quality prediction methods are supervised, meaning they explicitly identify the cause of poor image quality. This type of method is called attribution classification, where the cause is clearly known. Common attributions for low-quality images include: large facial pose angles (e.g., profile images are considered lower quality than frontal images); blurry images; and occlusions on the face / image (wearing masks, sunglasses, etc.). Conversely, clear, frontal images with unobstructed faces are considered high quality. Non-attribution classification refers to obtaining a quality score for an image through certain algorithms, but it's difficult to explain the specific factors that caused the high / low quality score. This score is often related to the performance of the face recognition model. In other words, most non-attribution algorithms consider images that can be easily recognized by the face recognition model to have good quality.

[0031] Meanwhile, there are also image quality prediction methods based on probabilistic models. These methods first establish a statistical probability model between image features and image quality, often using a multivariate Gaussian distribution to describe the probability distribution. For the image to be evaluated, features are extracted, and the maximum a posteriori probability of image quality is calculated based on the probabilistic model, or the image quality is predicted and estimated based on the degree of matching with the probabilistic model. Recently, there have also been image quality prediction methods based on neural networks. These methods first extract certain image transform domain or spatial features, then train a neural network regression analysis model based on known quality data to predict image quality from image features. For example, convolutional neural networks (CNNs) can be used to integrate feature extraction and regression analysis into the same network, or deep learning algorithms with multi-layered network structures can be used for image quality evaluation.

[0032] Therefore, to ensure the accuracy of subsequent liveness authentication / face recognition in liveness authentication or other face authentication tasks, enhance security performance, and allow users to perform quality compliance judgments separately across multiple networks, overcoming the problem of multiple network models being unfavorable for edge deployment and leading to slow speeds due to numerous models, this paper proposes an image quality prediction algorithm based on unsupervised anisotropic cascaded label generation and smoothed dynamic loss. This single algorithm addresses multiple quality non-compliance issues. Unsupervised anisotropic cascaded label generation technology solves the problem of limited and difficult-to-collect images with various quality non-compliance criteria, while smoothed dynamic loss ensures stable convergence and accuracy of quality score prediction. A single model performs comprehensive judgments, saving inference time at the edge and significantly reducing user waiting time. Furthermore, by controlling non-compliant images, the pass rate and accuracy of subsequent face recognition are guaranteed, improving the user experience.

[0033] like Figure 1 The diagram shown is a flowchart of an image quality prediction model training method according to this application. The method includes:

[0034] S101: Obtain the target image training sample set.

[0035] S102: Obtain the quality score enhancement label of the target image above.

[0036] S103: Input the above target image and the corresponding quality score enhancement label into the image quality prediction model to obtain the image quality prediction result; the above image quality prediction model includes a backbone network and a quality classification head.

[0037] S104: Based on the target loss function, iteratively update the parameters of the above image quality prediction model until the preset iteration termination condition is reached, and obtain the trained image quality prediction model.

[0038] In some embodiments, the quality score enhancement label of the target image is obtained, such as... Figure 2 As shown, it includes:

[0039] S211: Obtain the original quality score label of the target image above.

[0040] S212: Perform data enhancement on the target image according to the preset data enhancement method to obtain the corresponding data enhancement intensity; the data enhancement method includes one or more of Gaussian blur, random erasure and / or noise addition.

[0041] S213: Based on the original quality score label and the data enhancement intensity, the quality score enhancement label is obtained as follows:

[0042] ,

[0043] In the formula, To enhance the strength of the above data, For the above original quality score labels, This is the default value.

[0044] Specifically, Gaussian blur is implemented using functions such as OpenCV's Gaussian blur function. The degree of blur is controlled by adjusting the size and standard deviation of the Gaussian kernel; a larger kernel and a larger standard deviation result in a higher degree of blur. We define the blur level as ten levels from 1 to 10, i.e., the blur enhancement level is [1, 2, 3... 10]. Random erasure uses four parameters to control the erasure level: the lower and upper bounds of the erase area ratio, and the lower and upper bounds of the aspect ratio of the erased rectangle. We also define the erasure level as ten levels from 1 to 10 based on the erased area, i.e., the blur enhancement level is [1, 2, 3... 10]. There are two ways to add noise: salt-and-pepper noise and Gaussian noise. One noise enhancement method is selected randomly. If salt-and-pepper noise is added, the noise level is controlled by the noise ratio; simply put, pixel values ​​greater than the threshold are set to 255, and those less than the threshold are set to 0. If Gaussian noise is added, the noise level is controlled by adjusting the mean and variance. We define the noise level as ten levels from 1 to 10, meaning the blur enhancement level is [1, 2, 3... 10]. It should be noted that data augmentation methods can be used simultaneously or separately. This embodiment selects them simultaneously according to a certain probability, and the order of data augmentation is not distinguished. However, it should be understood that regardless of the data augmentation method used, the goal is to obtain an enhanced quality score; therefore, all are within the protection scope of this application's technical solution. Meanwhile, In one embodiment of this application, the value is 15.

[0045] Specifically, during training, for each image sample in each batch, a data augmentation method is randomly selected with a probability of 0.5, and the corresponding quality score label is calculated. The enhanced images and quality labels are then used in training. It's important to note that "batch" refers to an iteration during network training, indicating the total number of images used in that batch.

[0046] In some embodiments, the original quality score label of the target image is obtained, such as... Figure 3 As shown, it includes:

[0047] S311: Extract image features from all target images in the above training sample set.

[0048] S312: Based on the above image features, construct positive sample pairs and negative sample pairs, and calculate the feature similarity between the positive sample pairs and the negative sample pairs, including cosine similarity.

[0049] S313: Calculate the distance between the above feature similarities and convert the distance to a preset numerical range as the original quality score label. The above distance includes the Wasserstein distance.

[0050] It should be noted that the above steps achieve unsupervised, anisotropic cascaded face quality label generation. This means that data augmentation addresses the difficulty of collecting and labeling substandard images. Furthermore, by incorporating manually designed quality-influencing factors such as occlusion, blurring, and noise, a cascade of attributional and non-attributional factors is achieved. This ensures that the quality score label considers both face recognition performance and objective influencing factors, enabling a more reasonable definition of face image quality in complex scenes.

[0051] In some embodiments, the target image and the corresponding quality score enhancement label are input into the image quality prediction model to obtain the image quality prediction result, such as... Figure 4 As shown, it includes:

[0052] S411: Input the target image into the backbone network of the image quality prediction model to obtain the first feature vector.

[0053] S412: Input the first feature vector into the quality classification head to obtain a second feature vector of a preset dimension; the preset dimension corresponds to the number of quality score values; the quality classification head includes a flattening layer, a PReLU layer and a fully connected layer.

[0054] S413: Perform Softmax calculation on the second feature vector mentioned above to obtain the corresponding quality score. Predicted probability value .

[0055] S414: Based on each quality score and its corresponding predicted probability value, the predicted image quality score is obtained as follows:

[0056] ,

[0057] In the formula The quantity is the quality score.

[0058] Specifically, MobileetV3 is used as the backbone network, and the input is a resolution of The image output is Feature map Then, the feature map The input to the classification header first includes the feature map. Flatten A total of 6912-dimensional feature vectors are generated, then activated by a PreLU layer, followed by random dropout with a probability of 0.2, and then processed by a layer with a dimension of... The fully connected layer (FC) yields a feature vector of dimension 101. , After the softmax operation, 101 probability values ​​were obtained. Finally, the probability value Multiply the result by the corresponding label and sum the results to obtain the final predicted score. .

[0059] In some embodiments, inputting the target image and the corresponding quality score enhancement label into an image quality prediction model to obtain an image quality prediction result further includes: determining a first loss function and / or a second loss function based on the predicted image quality score and the quality score enhancement label; wherein the first loss function is:

[0060] ,

[0061] In the formula, The number of image samples in the training sample set mentioned above. .

[0062] Specifically, since the original cross-entropy loss function is expressed as In the formula, This indicates the number of categories: 101 (i.e., scores from 0 to 100). Indicates the number of samples; This is the true category label; if it matches the score, it is 1; otherwise, it is 0. Let be the predicted probability value. However, to ensure that the loss function penalizes different prediction errors with varying degrees, a smooth dynamic cross-entropy loss, or the first loss function, is defined. This loss function means that if the label is 65 points and the prediction is 63 points, the penalty is 0.2; if the prediction is 75 points, the penalty is 1.1. The closer the distance, the lower the penalty; the farther the distance, the higher the penalty.

[0063] In some embodiments, the second loss function described above is:

[0064] .

[0065] In some embodiments, inputting the target image and the corresponding quality score enhancement label into an image quality prediction model to obtain an image quality prediction result further includes: determining a third loss function based on the predicted image quality score.

[0066] ,

[0067] In the formula, This indicates the quality score corresponding to the respective category.

[0068] In some embodiments, the target loss function is determined based on the first loss function, the second loss function, and the third loss function as follows:

[0069] ,

[0070] In the formula, , and These are hyperparameters used to adjust the weights of the first loss function, the second loss function, and the third loss function, respectively. Specifically, in one embodiment of this application, Set to 0.8, Set to 0.2, Set to 0.02.

[0071] It should be noted that the above steps achieve a smooth dynamic classification loss quality score prediction technique. The quality score prediction problem is transformed into a classification problem, employing a "soft label" approach: predictions close to the true score are penalized with low weight, while predictions far from the true score are penalized with high weight. This allows the network to adaptively focus on learning from inaccurate predictions.

[0072] This application's embodiments involve: acquiring a training sample set of target images; obtaining quality score enhancement labels for the target images; inputting the target image and the corresponding quality score enhancement labels into an image quality prediction model to obtain image quality prediction results; the image quality prediction model includes a backbone network and a quality classification head; iteratively updating the parameters of the image quality prediction model according to the target loss function until a preset iteration termination condition is reached, thus obtaining the trained image quality prediction model. This application generates image quality score labels through data augmentation, which can provide a more reasonable definition of image quality in complex scenes, and transforms the quality score prediction problem into a classification problem, helping to improve the accuracy and pass rate of downstream tasks such as image recognition.

[0073] All of the above-mentioned optional technical solutions can be combined in any way to form the optional embodiments of this application, and will not be described in detail here.

[0074] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the system embodiments of this application, please refer to the method embodiments of this application.

[0075] Figure 5 This is a schematic diagram of an image quality prediction model training device provided in an embodiment of this application. Figure 5 As shown, the image quality prediction model training device includes:

[0076] The training set acquisition module 501 is capable of acquiring a training sample set of target images;

[0077] The quality score enhancement label acquisition module 502 is capable of acquiring the quality score enhancement label of the target image mentioned above.

[0078] The image quality prediction result output module 503 can input the above-mentioned target image and the corresponding quality score enhancement label into the image quality prediction model to obtain the image quality prediction result; the above-mentioned image quality prediction model includes a backbone network and a quality classification unit.

[0079] The image quality prediction model training module 504 can iteratively update the parameters of the image quality prediction model according to the target loss function until a preset iteration termination condition is reached, thereby obtaining the trained image quality prediction model.

[0080] It should be understood that the image quality prediction model training apparatus of one embodiment of this specification can also perform... Figures 1 to 4 A method for training an image quality prediction model is described, and the image quality prediction model training device is implemented in... Figures 1 to 4 The functions of the examples shown will not be repeated here. Furthermore, the sequence numbers of the steps in the above embodiments do not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0081] Figure 6 This is a schematic diagram of the electronic device 6 provided in an embodiment of this application. Figure 6 As shown, the electronic device 6 of this embodiment includes a processor 601, a memory 602, and a computer program 603 stored in the memory 602 and executable on the processor 601. When the processor 601 executes the computer program 603, it implements the steps in the various method embodiments described above. Alternatively, when the processor 601 executes the computer program 603, it implements the functions of each module / unit in the various device embodiments described above.

[0082] Electronic device 6 can be a desktop computer, laptop, handheld computer, cloud server, or other electronic device. Electronic device 6 may include, but is not limited to, processor 601 and memory 602. Those skilled in the art will understand that... Figure 6 This is merely an example of electronic device 6 and does not constitute a limitation on electronic device 6. It may include more or fewer components than shown, or different components.

[0083] The memory 602 can be an internal storage unit of the electronic device 6, such as a hard disk or RAM of the electronic device 6. The memory 602 can also be an external storage device of the electronic device 6, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc., equipped on the electronic device 6. The memory 602 can also include both internal and external storage units of the electronic device 6. The memory 602 is used to store computer programs and other programs and data required by the electronic device.

[0084] Processor 601 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 601 reads the corresponding computer program from non-volatile memory into memory and then executes it, forming a shared resource access control device at the logical level. The processor executes the program stored in memory and specifically performs the following operations:

[0085] Obtain the target image training sample set;

[0086] Obtain the quality score enhancement labels for the target images mentioned above;

[0087] The target image and the corresponding quality score enhancement label are input into the image quality prediction model to obtain the image quality prediction result; the image quality prediction model includes a backbone network and a quality classification head.

[0088] Based on the objective loss function, the parameters of the above image quality prediction model are iteratively updated until the preset iteration termination condition is reached, thus obtaining the trained image quality prediction model.

[0089] The above is as described in this instruction manual. Figures 1 to 4The image quality prediction model training method disclosed in the illustrated embodiment can be applied to, or implemented by, processor 601. Processor 601 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor described above can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this specification. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this specification can be directly implemented by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0090] Of course, in addition to the software implementation, the electronic devices in the embodiments of this specification do not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0091] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0092] If an integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program may include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium may include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in a computer-readable medium can be appropriately added to or subtracted according to the requirements of legislation and patent practice in a jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0093] This specification also provides an embodiment of a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by a portable electronic device including multiple applications, enable the portable electronic device to perform... Figures 1 to 4 The image quality prediction model training method shown in the embodiment is specifically used to perform the following methods:

[0094] Obtain the target image training sample set;

[0095] Obtain the quality score enhancement labels for the target images mentioned above;

[0096] The target image and the corresponding quality score enhancement label are input into the image quality prediction model to obtain the image quality prediction result; the image quality prediction model includes a backbone network and a quality classification head.

[0097] Based on the objective loss function, the parameters of the above image quality prediction model are iteratively updated until the preset iteration termination condition is reached, thus obtaining the trained image quality prediction model.

[0098] In summary, the above description is merely a preferred embodiment of this specification and is not intended to limit the scope of protection of this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of protection of this specification.

[0099] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0100] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0101] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0102] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0103] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for training an image quality prediction model, characterized in that, include: Obtain the target image training sample set; Obtain the quality score enhancement label of the target image; The target image and the corresponding quality score enhancement label are input into the image quality prediction model to obtain the image quality prediction result; the image quality prediction model includes a backbone network and a quality classification head. Based on the target loss function, the parameters of the image quality prediction model are iteratively updated until a preset iteration termination condition is reached, thereby obtaining the trained image quality prediction model. Obtaining the quality score enhancement label of the target image includes: Obtain the original quality score label of the target image; The target image is augmented according to a preset data augmentation method to obtain a corresponding data augmentation intensity; the data augmentation method includes one or more of Gaussian blur, random erasure, and / or noise addition; Based on the original quality score label and the data augmentation intensity, the quality score enhancement label is obtained as follows: ， In the formula, To enhance the strength of the data, For the original quality score label, This is the default value.

2. The method according to claim 1, characterized in that, Obtaining the original quality score label of the target image includes: Extract image features from all target images in the training sample set; Based on the image features, positive sample pairs and negative sample pairs are constructed, and the feature similarity between the positive sample pairs and the negative sample pairs is calculated, including cosine similarity. Calculate the distance between the feature similarities and convert the distance to a preset numerical range as the original quality score label. The distance includes the Wasserstein distance.

3. The method according to claim 1, characterized in that, The target image and the corresponding quality score enhancement label are input into the image quality prediction model to obtain the image quality prediction result, including: The target image is input into the backbone network of the image quality prediction model to obtain the first feature vector; The first feature vector is input into the quality classification head to obtain a second feature vector of a preset dimension; the preset dimension corresponds to the number of quality score values; the quality classification head includes a flattening layer, a PReLU layer, and a fully connected layer. The second feature vector is then subjected to Softmax calculation to obtain the corresponding quality score. Predicted probability value ; Based on each quality score and its corresponding predicted probability value, the predicted image quality score is obtained as follows: ， In the formula The quantity is the quality score.

4. The method according to claim 3, characterized in that, The process of inputting the target image and the corresponding quality score enhancement label into an image quality prediction model to obtain an image quality prediction result further includes: determining a first loss function and / or a second loss function based on the predicted image quality score and the quality score enhancement label; wherein the first loss function is: ， In the formula, The number of image samples in the training sample set. ; and / or, The second loss function is: 。 5. The method according to claim 4, characterized in that, The target image and the corresponding quality score enhancement label are input into the image quality prediction model to obtain the image quality prediction result. The method further includes: determining a third loss function based on the predicted image quality score. ， In the formula, This indicates the quality score corresponding to the respective category.

6. The method according to claim 5, characterized in that, Based on the first loss function, the second loss function, and the third loss function, the target loss function is determined as follows: ， In the formula, , and These are hyperparameters used to adjust the weights of the first loss function, the second loss function, and the third loss function, respectively.

7. A training device for a target image quality prediction model, characterized in that, The device includes: The training set acquisition module is configured to acquire a training sample set of the target image; The quality score enhancement label acquisition module is configured to acquire the quality score enhancement label of the target image; The image quality prediction result output module is configured to input the target image and the corresponding quality score enhancement label into the image quality prediction model to obtain the image quality prediction result; the image quality prediction model includes a backbone network and a quality classification unit. The image quality prediction model training module is configured to iteratively update the parameters of the image quality prediction model according to the target loss function until a preset iteration termination condition is reached, thereby obtaining the trained image quality prediction model. The quality score enhancement label acquisition module is specifically configured to: obtain the original quality score label of the target image; perform data enhancement on the target image according to a preset data enhancement method to obtain a corresponding data enhancement intensity; the data enhancement method includes one or more of Gaussian blur, random erasure, and / or noise addition; and obtain the quality score enhancement label based on the original quality score label and the data enhancement intensity. ， In the formula, To enhance the strength of the data, For the original quality score label, This is the default value.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 6.

Images