Training method of image quality evaluation model, image quality evaluation method and device

Abstract

The present disclosure relates to the technical field of image processing, and provides a training method of an image quality evaluation model, an image quality evaluation method and device. The training method comprises: obtaining a first training image and label information; segmenting the first training image to determine a foreground region and a background region; changing first pixel information in the background region to obtain a second training image; changing second pixel information in the foreground region to obtain a third training image; inputting the first training image, the second training image and the third training image into a preset evaluation model to obtain a predicted quality score and a feature weight map; the feature weight map is determined by a spatial attention network in the preset evaluation model; based on the second pixel information, the predicted quality score, the feature weight map and the label information, adjusting model parameters of the preset evaluation model to obtain an image quality evaluation model meeting a preset training end condition, which can accurately evaluate the quality of a target object.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of image processing, and particularly relates to a training method of an image quality evaluation model, an image quality evaluation method and device. BACKGROUND

[0002] With the development of science and technology, it is easier to obtain images, and the number of images is increasing, which makes it essential to evaluate the quality of images. By evaluating the quality of images, low-quality image data can be filtered out, and the efficiency and accuracy of subsequent image processing can be improved. At present, when evaluating the quality of images, a quality score reflecting the overall image quality is determined according to the blur, brightness, exposure, sharpness and other characteristics of the image, and then an end-to-end training is performed according to the quality score to obtain an image quality evaluation model. The image quality evaluation model is difficult to accurately evaluate the quality of target objects in images. SUMMARY

[0003] Therefore, the embodiments of the present disclosure provide a training method of an image quality evaluation model, an image quality evaluation method and device to solve the problem that the image quality evaluation model in the prior art is difficult to accurately evaluate the quality of target objects in images.

[0004] In a first aspect, the embodiments of the present disclosure provide a training method of an image quality evaluation model, comprising:

[0005] obtaining a first training image containing a target object and label information corresponding to the first training image;

[0006] segmenting the first training image to determine a foreground region and a background region corresponding to the target object;

[0007] changing first pixel information in the background region to obtain a second training image, and changing second pixel information in the foreground region to obtain a third training image;

[0008] inputting the first training image, the second training image and the third training image into a preset evaluation model to obtain a predicted quality score and a feature weight map; wherein the feature weight map is determined by a spatial attention network in the preset evaluation model;

[0009] adjusting model parameters of the preset evaluation model based on the second pixel information, the predicted quality score, the feature weight map and the label information to obtain an image quality evaluation model meeting a preset training end condition.

[0010] In a second aspect, the embodiments of the present disclosure provide an image quality evaluation method, comprising:

[0011] obtaining an image to be evaluated; wherein the image to be evaluated contains a target object;

[0012] inputting the image to be evaluated into the image quality evaluation model to obtain a quality evaluation result;

[0013] The image quality evaluation model is trained by using the method.

[0014] In a third aspect, the present disclosure provides a training device of an image quality evaluation model, comprising:

[0015] The data acquisition module is configured to acquire a first training image containing a target object and label information corresponding to the first training image.

[0016] The segmentation processing module is configured to segment the first training image to determine a foreground region and a background region corresponding to the target object.

[0017] The change processing module is configured to change first pixel information in the background region to obtain a second training image, and change second pixel information in the foreground region to obtain a third training image.

[0018] The input processing module is configured to input the first training image, the second training image and the third training image into a preset evaluation model to obtain a predicted quality score and a feature weight map, wherein the feature weight map is determined by a spatial attention network in the preset evaluation model.

[0019] The parameter adjustment module is configured to adjust model parameters of the preset evaluation model based on the second pixel information, the predicted quality score, the feature weight map and the label information to obtain an image quality evaluation model meeting a preset training end condition.

[0020] In a fourth aspect, the present disclosure provides an image quality evaluation device, comprising:

[0021] The image acquisition module is configured to acquire an image to be evaluated, wherein the image to be evaluated contains a target object.

[0022] The image evaluation module is configured to input the image to be evaluated into an image quality evaluation model to obtain a quality evaluation result.

[0023] The image quality evaluation model is trained by using the method.

[0024] In a fifth aspect, the present disclosure provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0025] In a sixth aspect, the present disclosure provides a computer readable storage medium, which stores a computer program, wherein the computer program is executed by a processor to implement the steps of the above method.

[0026] The beneficial effects of the embodiments of the present disclosure compared with the prior art are: by acquiring a first training image containing a target object and label information corresponding to the first training image; then segmenting the first training image to determine a foreground region and a background region corresponding to the target object, and changing the first training image in the background region to obtain a second training image, and changing the second pixel information in the foreground region to obtain a third training image. The first training image, the second training image and the third training image are input into a preset evaluation model to obtain a prediction quality score and a feature weight map, wherein the feature weight map is determined by a spatial attention network in the preset evaluation model. Further, the model parameters of the preset model are adjusted according to the second pixel information, the preset quality score, the feature weight map and the label information to obtain an image quality evaluation model that meets a preset training end condition. In the technical solutions provided in the present disclosure, by segmenting the first training image to obtain the foreground region and the background region, and changing the first pixel information in the foreground region and the second pixel information in the background region, the influence of the changed foreground image and the changed background image on the prediction quality score is introduced into the process of adjusting the model parameters, which can effectively decouple the target object quality and the overall image quality in the image, remove the interference of the image itself quality and the background quality, and enable the determined image quality evaluation model to accurately evaluate the target object quality. BRIEF DESCRIPTION OF DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative labor.

[0028] Figure 1 is a flowchart of a training method of an image quality evaluation model provided by an embodiment of the present disclosure;

[0029] Figure 2 is a flowchart of an image quality evaluation method provided by an embodiment of the present disclosure;

[0030] Figure 3 is a structural schematic diagram of a training device of an image quality evaluation model provided by an embodiment of the present disclosure;

[0031] Figure 4 is a structural schematic diagram of an image quality evaluation device provided by an embodiment of the present disclosure;

[0032] Figure 5 is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0033] In the following description, for purposes of explanation and not limitation, specific details are set forth such as particular architectures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known methods, devices, circuits, and methods are omitted so as not to obscure the description of the present disclosure.

[0034] Figure 1 is a flow diagram of a training method of an image quality evaluation model provided by an embodiment of the present disclosure. Figure 1 The training method of the image quality evaluation model can be executed by a server, and the method comprises:

[0035] S101, a first training image containing a target object and label information corresponding to the first training image are obtained.

[0036] S102, the first training image is segmented to determine the foreground region and the background region corresponding to the target object.

[0037] S103, the first pixel information in the background region is changed to obtain a second training image; the second pixel information in the foreground region is changed to obtain a third training image.

[0038] S104, the first training image, the second training image and the third training image are input into a preset evaluation model to obtain a predicted quality score and a feature weight map; wherein the feature weight map is determined by a spatial attention network in the preset evaluation model.

[0039] S105, based on the second pixel information, the predicted quality score, the feature weight map and the label information, the model parameters of the preset evaluation model are adjusted to obtain an image quality evaluation model that meets a preset training end condition.

[0040] Specifically, the target object is a research object in the first training image. For example, the target object can be a face, a vehicle or an animal, etc. When the target object is a face, the first training image is a face image. When the target object is a vehicle, the first training image is a vehicle image. When the target object is an animal, the first training image is an animal image.

[0041] Further, the label information can be real annotation data or calculated pseudo label. For example, a quality score pseudo label of the first training image is calculated by using a SDD (Single Shot MultiBox Detector) algorithm, the first training image is classified, and a distribution distance result of intra-class samples and inter-class samples is taken as the quality score pseudo label of the first training image. For the ith first training image x i , the quality score pseudo label thereof is denoted as y i .

[0042] Further, the first training image is segmented, for example, the first training image is segmented into a foreground region and a background region by introducing an image segmentation algorithm PP-LiteSeg, the foreground region contains an object of interest of a user, i.e., a target object, and the background region has no effect on the target object. For example, the foreground region, i.e., a face region, and the background region, i.e., a non-face region, are determined by segmenting a face image.

[0043] Further, the first pixel information in the background region is changed to obtain a second training image, and the foreground region in the second training image does not change. The first pixel information is information corresponding to part of pixel points randomly selected in the background region. The first pixel information can be replaced by a random value, or the pixel value of the pixel point of the first pixel information can be modified to a set value.

[0044] In a possible implementation, the background region is divided into background regions, and the division manner can be equal in size or different in size. Random selection is performed in the background regions, the number of randomly selected background regions can be at least one, the pixel points in the selected background regions correspond to the first pixel information, and the selected background regions are filled with random values to replace the original pixel points to obtain the second training image.

[0045] Further, the second pixel information in the foreground region is changed to obtain a third training image, and the background region in the third training image does not change. The second pixel information is information corresponding to part of pixel points randomly selected in the foreground region. The second pixel information can be replaced by a random value, or the pixel value of the pixel point of the second pixel information can be modified to a set value.

[0046] In a possible implementation, the foreground region is divided into foreground regions, and the division manner can be equal in size or different in size. Random selection is performed in the foreground regions, the number of randomly selected foreground regions can be at least one, the pixel points in the selected foreground regions correspond to the second pixel information, and the selected foreground regions are filled with random values to replace the original pixel points to obtain the third training image.

[0047] The original image, i.e., the first training image, the changed image, i.e., the second training image, and the third training image are input into a preset evaluation model to obtain a predicted quality score and a feature weight map, wherein the feature weight map is determined by a spatial attention network in the preset evaluation model, and the feature weight map can be used to indicate the importance of the changed first pixel information and the second pixel information.

[0048] In a possible implementation, the preset evaluation model is built in advance, and the preset evaluation model includes a feature extraction network, a prediction output network, and a spatial attention network. The number of the feature extraction networks can be multiple, and the connection mode of the feature extraction networks is in series. The prediction output network is connected after all the feature extraction networks, performs quality regression according to the features extracted by the feature extraction networks, and outputs a predicted quality score. The spatial attention network is connected with one feature extraction network, learns the features extracted by the feature extraction network connected therewith, and obtains a feature weight map. The introduction of the spatial attention network can make the model pay more attention to important areas to regress the result.

[0049] Further, a constrained loss function is constructed according to the second pixel information, the predicted quality score, the feature weight map, and label information. The model parameters of the preset evaluation model are adjusted according to the loss function, and an image quality evaluation model meeting a preset training end condition is obtained.

[0050] The predicted quality score is a predicted value of the preset evaluation model, and the label information can be regarded as a labeled value of the training data. Therefore, the predicted quality score and the label information are used to construct the loss function. The first pixel information corresponds to a background region, and the change of the background region should have a relatively small impact on the original image, i.e., the first training image. The quality score of the second training image should be close to the quality score of the first training image. The second pixel information corresponds to a foreground region, and the change of the foreground region should have a relatively large impact on the original image, i.e., the first training image. That is, the predicted quality score should pay more attention to the foreground region, and the change of the background should not cause a large fluctuation of the predicted quality score. Therefore, the second pixel information is introduced into the construction of the loss function. The feature weight map can indicate the importance of the replaced second pixel information, and therefore the feature weight map is introduced into the construction of the loss function. The loss function is accurately constructed according to the second pixel information, the predicted quality score, the feature weight map, and the label information, and the image quality evaluation model obtained by using the loss function can accurately evaluate the target object in an image.

[0051] It's important to note that target object quality (target object image quality) and image quality (overall image quality) are two separate issues. For example, in face recognition tasks, an image with a blurred background but a relatively straight face angle and no obvious occlusion is beneficial for face recognition and should be considered a high-quality face image. Conversely, a high-resolution profile image should be considered a low-quality face image, which contradicts the conclusion about overall image quality. Therefore, an image quality assessment model trained based on overall image quality may not accurately evaluate the target object quality. In this embodiment, an image segmentation algorithm separates the foreground and background, modifies each separately, and calculates the regression quality score for each. This effectively decouples target object image quality from overall image quality, removing interference from the image's inherent quality and background quality, and thus more accurately estimates the target object quality.

[0052] According to the technical solution provided in this disclosure, a first training image containing the target object and corresponding label information are obtained; then, the first training image is segmented to determine the foreground and background regions corresponding to the target object, and the first training image in the background region is modified to obtain a second training image, and the second pixel information in the foreground region is modified to obtain a third training image. The first, second, and third training images are input into a preset evaluation model to obtain a predicted quality score and a feature weight map, wherein the feature weight map is determined by the spatial attention network in the preset evaluation model. Furthermore, the model parameters of the preset model are adjusted based on the second pixel information, the preset quality score, the feature weight map, and the label information to obtain an image quality evaluation model that meets the preset training termination conditions. In the technical solution provided in this disclosure, the first training image is segmented to obtain a foreground region and a background region. The first pixel information in the foreground region and the second pixel information in the background region are changed. The influence of changing the foreground image and changing the background image on the predicted quality score is introduced into the process of adjusting the model parameters. This can effectively decouple the quality of the target object in the image from the overall image quality, remove the interference of the image quality itself and the background quality, and enable the determined image mass spectrometry evaluation model to accurately evaluate the quality of the target object.

[0053] In some embodiments, the predicted quality score includes a first predicted score corresponding to a first training image, a second predicted score corresponding to a second training image, and a third predicted score corresponding to a third training image; step S105 includes:

[0054] S1051, based on the first predicted score and label information, determine the first loss; based on the second predicted score and label information, determine the second loss.

[0055] S1052, determine a first influence coefficient corresponding to the second pixel information based on the feature weight map; and determine a third loss based on the first influence coefficient, the third prediction score, and the label information.

[0056] S1053, adjust the model parameters of the preset evaluation model based on the first loss, the second loss, and the third loss, to obtain an image quality evaluation model meeting a preset training end condition.

[0057] Specifically, the first training image is an original image without any change, and thus the first loss can be determined based on the first prediction score corresponding to the first training image and the label information.

[0058] For example, the first training image x i is input into the preset evaluation model to obtain a first prediction score q i output by the preset evaluation model. i The mean square error can be used as a loss function L1 corresponding to the first loss for training constraint, and a formula of the loss function L1 is as follows:

[0059] L1 = (y i -q i ) 2 .

[0060] Further, the second training image is obtained by changing the background region, and the change of the background region has a small influence on the quality of the target object, and thus the second loss can be determined based on the second prediction score corresponding to the second training image and the label information.

[0061] For example, for the first training image x i , pixel points in a corresponding background region are divided into kxk sub-region blocks with possibly different sizes, and a certain sub-region block (i.e., a block with m x n pixel points) is randomly selected. The original pixel points in the sub-region block are replaced by random values to obtain a second training image x . is input into the preset evaluation model to obtain a second prediction score q i back , and a loss function L2 corresponding to the second loss is determined, and the loss function L2 is used for training constraint, and a formula of the loss function L2 is as follows:

[0062]

[0063] The loss function L2 means that the first training image x iThe prediction quality score of the second training image obtained after the random filling replacement of a part of the pixels in the background area should be very close to that of the first training image, that is, the prediction quality score should pay more attention to the foreground part, and the replacement of the pixels in the background should not cause a large fluctuation in the quality score.

[0064] Further, the weight information can be obtained from the feature weight map, the first influence coefficient corresponding to the second pixel information is determined according to the weight information, and the third loss is determined according to the first influence coefficient, the third prediction score and the label information.

[0065] For example, for the first training image x i , the pixels in the corresponding foreground area are divided into kxk sub-region blocks of possibly different sizes, a certain sub-region block (i.e. , a pixel point) is randomly selected, and the original pixel point is replaced with a random value to obtain a third training image is input into a preset evaluation model to obtain a third prediction score In the case where the determined first influence coefficient is a i , a loss function L3 corresponding to the third loss is determined, and the loss function L3 is used for training constraint, wherein the formula of the loss function L3 is as follows:

[0066]

[0067] , a i characteristic of the replaced foreground area (such as a face area), and theoretically, after replacing the pixel point in a certain sub-region block in the foreground area, the quality score of the image becomes (1-a i ) times the original image quality score. For example, if the foreground area is a face area, if the pixels in the eye part and other discriminative feature areas are replaced, it is considered that the quality score of the image should decrease significantly.

[0068] Further, a total loss is constructed according to the first loss, the second loss and the third loss, the model parameters of the preset evaluation model are adjusted by using the total loss, and an image quality evaluation model meeting a preset training end condition is obtained.

[0069] According to the technical scheme provided by the embodiments of the present disclosure, by accurately determining the first loss, the second loss and the third loss, the model parameters of the preset evaluation model are accurately adjusted, and the changes in the foreground area and the changes in the background area are introduced into the loss function, which is beneficial to decouple the target object quality and the overall image quality in the image, remove the interference of the image itself quality and the background quality, and more accurately evaluate the target object quality. ​

[0070] In some embodiments, the determining, in S1052, the first influence coefficient corresponding to the second pixel information based on the feature weight map comprises:

[0071] S10521, determining first dimension information corresponding to the feature weight map and second dimension information corresponding to the first training image.

[0072] S10522, determining, based on the first dimension information, the second dimension information and the feature weight map, a second influence coefficient corresponding to each pixel point in the first training image.

[0073] S10523, determining, based on the second pixel information and the second influence coefficient corresponding to each pixel point in the first training image, the first influence coefficient corresponding to the second pixel information.

[0074] Specifically, the first dimension information is used to indicate the size of the feature weight map, and the second dimension information is used to indicate the size of the first training image. The first dimension information and the second dimension information are compared to expand the second dimension information to the first dimension information, so that the second influence coefficient corresponding to each pixel point in the first training image can be obtained according to the feature weight map, and the second influence coefficient is used to indicate the influence degree of the pixel point. The target pixel point included in the second pixel information is determined, the sum of the second influence coefficients corresponding to the target pixel point is determined, and the sum of the second influence coefficients corresponding to the target pixel point is determined as the first influence coefficient corresponding to the second pixel information, and the first influence coefficient is used to indicate the influence degree of the second pixel information.

[0075] For example, the resolution of the first training image is s i 112x112. The preset evaluation model uses a residual neural network (IResNet18) as a backbone network, and is divided into a total of 4 stages, one stage being one feature extraction network, and the depth of each stage being (2, 2, 2, 2) respectively. The input is an image with a resolution of (3, 112, 112), and the output is a feature map of (512, 7, 7). Then a quality score regression head, i.e. a prediction output network, is connected. First, a convolution with a convolution kernel of 3x3 and a channel number of 256 is performed, then the features are flattened into a feature vector of 256x7x7 dimensions, followed by a PReLU activation, followed by a dropout layer with a drop probability of 0.5, followed by a fully connected layer with a dimension of (256x7x7, 1), and then connected with a sigmoid operation, and the output is a predicted quality score.

[0076] In the second stage of the backbone network, a spatial attention mechanism, namely the spatial attention network, is added. After obtaining the (128, 28, 28) feature map, global max pooling and average pooling are performed on the pixel values ​​at the same position on different feature maps in the channel dimension to obtain two spatial attention feature maps. Based on the two spatial attention feature maps, a feature weight map is obtained. That is, the two spatial attention feature maps are concatenated to obtain a feature map with dimensions of (2, 28, 28). Then, a 7x7 convolution kernel is used to convolve the obtained feature map, followed by a softmax operation, to obtain a spatial matrix of (1, 28, 28), which is the feature weight map.

[0077] The second stage of the backbone network learns a (1, 28, 28) spatial attention feature weight map, specifying the weights for each point (e.g., z). j The weight value represents the sum of the weights of the 4x4 grid with that point as the top-left corner. In other words, the weight of each pixel in that 4x4 grid is [value missing]. The weighted graph of (28, 28) (corresponding to the first dimension information) is expanded to (112, 112) (corresponding to the second dimension information), denoted as A. i Meanwhile, for the first training image x i The foreground region's pixels are divided into kxk sub-regions of varying sizes, and one of these sub-regions is randomly selected (i.e., The third training image is obtained by replacing the original pixels with random values ​​(pixels). The attention weight of this sub-region is the sum of the weights of all pixels in that sub-region, denoted as a. i .

[0078] According to the technical solution provided in this disclosure, after determining the first dimension information and the second dimension information, the feature weight map is expanded to obtain the second influence coefficient corresponding to each pixel in the first training image, thereby accurately determining the first influence coefficient.

[0079] In some embodiments, the method further includes:

[0080] S1054, determine the third influence coefficients corresponding to the second and third losses respectively;

[0081] Accordingly, S1053 includes: adjusting the model parameters of the preset evaluation model based on the first loss, the second loss, the third loss, and the third influence coefficient to obtain an image quality evaluation model that meets the preset training termination conditions.

[0082] Specifically, the second loss and the third loss correspond to different third influence coefficients respectively, the third influence coefficients are used to indicate the influence degree corresponding to the loss, so as to accurately construct the total loss by considering the third influence coefficients when adjusting the model parameters, which is beneficial to obtain an image quality evaluation model with better performance.

[0083] In some embodiments, S1054 includes:

[0084] S10541, determining ratio information of a sum of pixel points in the foreground region to a sum of pixel points in the first training image.

[0085] S10542, determining the ratio information as a third influence coefficient corresponding to the third loss, and determining a difference result of a preset value and the ratio information as a third influence coefficient corresponding to the second loss.

[0086] Specifically, the pixel points in the foreground region are counted to determine the sum of the pixel points in the foreground region. The pixel points in the first training image are counted to determine the sum of the pixel points in the first training image, and further determine the ratio information of the sum of the pixel points in the foreground region to the sum of the pixel points in the first training image. The ratio information is taken as the third influence coefficient corresponding to the third loss, a preset value such as 1 is determined in advance, and a difference result of the preset value and the ratio information is determined as the third influence coefficient corresponding to the second loss. Because the third influence coefficient is determined according to the foreground region and the first training image, the model parameters of the preset evaluation model can be adjusted according to the second pixel information, the predicted quality score, the feature weight map, the label information, the foreground region, and the first training image, to obtain an image quality evaluation model meeting a preset training end condition. The preset training end condition can be set in advance, such as model convergence or training times reaching a set number of times.

[0087] For example, the step of determining the first loss is mode 1, the steps of determining the second loss and the third loss are collectively referred to as mode 2, and three loss functions are used to supervise the training process. The first loss, the second loss, and the third loss are used to determine the total loss, which is:

[0088]

[0089] wherein M represents that there are M samples in the training set, s i represents the sum of the pixel points in the first training image, represents the sum of the pixel points in the foreground region, represents the third influence coefficient of the third loss, represents the third influence coefficient of the second loss. The meaning of the total loss is that if the first training image x iIf the input is trained in the preset evaluation model (mode = 1), the mean square error is used as the loss function constraint; if the second training image and the third training image are input into the preset evaluation model, the foreground area and the background area are respectively used as weights, which means that as the proportion of the foreground area increases, the replacement of the foreground area should cause greater changes in the loss function, and vice versa. If the proportion of the background area is small, the loss change caused by the replacement of the background area is also small. According to the technical scheme provided by the embodiments of the present disclosure, the foreground area and the background area of the first training image are separated by image segmentation, and are respectively randomly filled. The importance of the replaced area is used to regress the change of the quality score. The more important the replaced area is, the more obvious the expected quality score decreases. The less important the replaced area is, the less the expected quality score changes. The image quality and the target object quality are decoupled to a certain extent, so that the predicted quality score of the preset evaluation model tends to represent the target object quality rather than the image quality of the background and other interference information. This helps to control the input image quality of the target object recognition task.

[0090] According to the technical scheme provided by the embodiments of the present disclosure, the foreground area and the background area of the first training image are separated by image segmentation, and are respectively randomly filled. The importance of the replaced area is used to regress the change of the quality score. The more important the replaced area is, the more obvious the expected quality score decreases. The less important the replaced area is, the less the expected quality score changes. The image quality and the target object quality are decoupled to a certain extent, so that the predicted quality score of the preset evaluation model tends to represent the target object quality rather than the image quality of the background and other interference information. This helps to control the input image quality of the target object recognition task.

[0091] In some embodiments, S104 includes:

[0092] S1041, determining a first input probability corresponding to the first training image.

[0093] S1042, determining a second input probability corresponding to the second training image and the third training image.

[0094] S1043, inputting the first training image into the preset evaluation model with the first input probability, and inputting the second training image and the third training image into the preset evaluation model with the second input probability, to obtain the predicted quality score and the feature weight map.

[0095] Specifically, a first input probability p corresponding to the first training image is determined, p e (0, 1), a second input probability 1-p corresponding to the second training image and the third training image is determined, the first training image is input into the preset evaluation model with the probability p, and the second training image and the third training image are input into the preset evaluation model with 1-p, to obtain the predicted quality score and the feature weight map.

[0096] According to the technical scheme provided by the embodiments of the present disclosure, the input of the first training image, the second training image and the third training image is controlled according to the first input probability and the second input probability, and the proportion of different images is grasped, which is conducive to obtaining an image quality evaluation model with good performance.

[0097] ​All the optional technical solutions described above can be combined to form optional embodiments of the present application, which will not be described again here.

[0098] Figure 2 is a flowchart of an image quality evaluation method provided by an embodiment of the present disclosure. Figure 2 The image quality evaluation method can be executed by a server, and the method comprises:

[0099] S201, obtaining an image to be evaluated; wherein the image to be evaluated contains a target object;

[0100] S202, inputting the image to be evaluated into an image quality evaluation model to obtain a quality evaluation result;

[0101] The image quality evaluation model is obtained by training the method described above.

[0102] According to the technical solution provided by the embodiment of the present disclosure, after obtaining the image quality evaluation model by the training method of the image quality evaluation model described above, the image quality evaluation model is used for image quality evaluation, that is, the image to be evaluated containing the target object is input into the image quality evaluation model, and the quality of the target object can be accurately evaluated.

[0103] The following is an apparatus embodiment of the present disclosure, which can be used to execute the method embodiment of the present disclosure. For details not disclosed in the apparatus embodiment of the present disclosure, please refer to the method embodiment of the present disclosure.

[0104] Figure 3 is a schematic diagram of a training apparatus of an image quality evaluation model provided by an embodiment of the present disclosure.

[0105] As shown in Figure 3 , the training apparatus of the image quality evaluation model comprises:

[0106] The data acquisition module 301 is configured to acquire a first training image containing a target object and label information corresponding to the first training image.

[0107] The segmentation processing module 302 is configured to segment the first training image to determine a foreground region and a background region corresponding to the target object.

[0108] The change processing module 303 is configured to change first pixel information in the background region to obtain a second training image, and change second pixel information in the foreground region to obtain a third training image.

[0109] The input processing module 304 is configured to input the first training image, the second training image and the third training image into a preset evaluation model to obtain a predicted quality score and a feature weight map; wherein the feature weight map is determined by a spatial attention network in the preset evaluation model.

[0110] The parameter adjustment module 305 is configured to adjust model parameters of the preset evaluation model based on the second pixel information, the prediction quality score, the feature weight map, and the label information, to obtain an image quality evaluation model that meets a preset training end condition.

[0111] According to the technical scheme provided in the embodiments of the present disclosure, the first training image containing the target object and the label information corresponding to the first training image are obtained. Then, the first training image is segmented to determine the foreground region and the background region corresponding to the target object, and the first training image in the background region is changed to obtain the second training image, and the second pixel information in the foreground region is changed to obtain the third training image. The first training image, the second training image, and the third training image are input into the preset evaluation model to obtain the prediction quality score and the feature weight map, wherein the feature weight map is determined by the spatial attention network in the preset evaluation model. Further, the model parameters of the preset model are adjusted based on the second pixel information, the preset quality score, the feature weight map, and the label information, to obtain an image quality evaluation model that meets a preset training end condition. In the technical scheme provided in the present disclosure, the first training image is segmented to obtain the foreground region and the background region, and the first pixel information in the foreground region and the second pixel information in the background region are changed, the influence of the changed foreground image and the changed background image on the prediction quality score is introduced into the process of adjusting the model parameters, which can effectively decouple the target object quality and the overall image quality in the image, remove the interference of the image itself quality and the background quality, and make the determined image quality evaluation model accurately evaluate the target object quality.

[0112] In some embodiments, the prediction quality score includes a first prediction score corresponding to the first training image, a second prediction score corresponding to the second training image, and a third prediction score corresponding to the third training image; and the parameter adjustment module includes:

[0113] The first loss determination unit is configured to determine a first loss based on the first prediction score and the label information, and determine a second loss based on the second prediction score and the label information.

[0114] The second loss determination unit is configured to determine a first influence coefficient corresponding to the second pixel information based on the feature weight map, and determine a third loss based on the first influence coefficient, the third prediction score, and the label information.

[0115] The parameter adjustment unit is configured to adjust the model parameters of the preset evaluation model based on the first loss, the second loss, and the third loss, to obtain an image quality evaluation model that meets a preset training end condition.

[0116] In some embodiments, the second loss determination unit includes:

[0117] The dimension determination subunit is used to determine the first dimension information corresponding to the feature weight map and the second dimension information corresponding to the first training image.

[0118] The first determining subunit is used to determine the second influence coefficient corresponding to each pixel in the first training image based on the first dimension information, the second dimension information and the feature weight map;

[0119] The second determining subunit is used to determine the first influence coefficient corresponding to the second pixel information based on the second pixel information and the second influence coefficient corresponding to each pixel in the first training image.

[0120] In some embodiments, the apparatus further includes:

[0121] The coefficient determination module is used to determine the third influence coefficients corresponding to the second and third losses, respectively.

[0122] The parameter adjustment module is further used to adjust the model parameters of the preset evaluation model based on the first loss, the second loss, the third loss, and the third influence coefficient, so as to obtain an image quality evaluation model that meets the preset training termination conditions.

[0123] In some embodiments, the parameter adjustment module includes:

[0124] The ratio information determination unit is used to determine the ratio information of the sum of the number of pixels in the foreground region to the sum of the number of pixels in the first training image;

[0125] The influence coefficient determination unit is used to determine the ratio information as the third influence coefficient corresponding to the third loss; and to determine the difference between the preset value and the ratio information as the third influence coefficient corresponding to the second loss.

[0126] In some embodiments, the input processing module includes:

[0127] The first probability determination unit is used to determine the first input probability corresponding to the first training image;

[0128] The second probability determination unit is used to determine the second input probability corresponding to the second training image and the third training image;

[0129] The input processing unit is used to input the first training image into the preset evaluation model with a first input probability, and input the second training image and the third training image into the preset evaluation model with a second input probability to obtain the prediction quality score and feature weight map.

[0130] Figure 4 This is a schematic diagram of the image quality assessment apparatus provided in an embodiment of this disclosure. Figure 4 As shown, the image quality assessment device includes:

[0131] The image acquisition module 401 is configured to acquire an image to be evaluated; wherein the image to be evaluated contains a target object.

[0132] The image evaluation module 402 is configured to input the image to be evaluated into an image quality evaluation model to obtain a quality evaluation result.

[0133] The image quality evaluation model is obtained by the training method described above.

[0134] According to the technical scheme provided by the embodiments of the present disclosure, after the image quality evaluation model is obtained by the training method of the image quality evaluation model described above, the image quality evaluation model is used for image quality evaluation, that is, the image to be evaluated containing the target object is input into the image quality evaluation model, and the quality of the target object can be accurately evaluated.

[0135] It should be understood that the size of the serial number of each step in the above embodiments does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present disclosure.

[0136] Figure 5 is a schematic diagram of an electronic device 5 provided by the embodiments of the present disclosure. As shown in the figure, the electronic device 5 of this embodiment includes a processor 501, a memory 502, and a computer program 503 stored in the memory 502 and executable on the processor 501. The processor 501 implements the steps in each of the above method embodiments when executing the computer program 503. Alternatively, the processor 501 implements the functions of each module / unit in each of the above device embodiments when executing the computer program 503. Figure 5

[0137] The electronic device 5 can be a desktop computer, a notebook computer, a palm computer, a cloud server, and the like. The electronic device 5 can include but is not limited to the processor 501 and the memory 502. Those skilled in the art can understand that the electronic device 5 can include more or fewer components or different components than those shown. Figure 5 The electronic device 5 is only an example of the electronic device 5 and does not constitute a limitation on the electronic device 5, which can include more or fewer components or different components than those shown.

[0138] ​The processor 501 can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0139] The memory 502 can be an internal storage unit of the electronic device 5, for example, a hard disk or a memory of the electronic device 5. The memory 502 can also be an external storage device of the electronic device 5, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 5. The memory 502 can also include both the internal storage unit and the external storage device of the electronic device 5. The memory 502 is used to store computer programs and other programs and data required by the electronic device.

[0140] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional units and modules is exemplified, and in actual application, the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0141] The integrated modules / units, if implemented in the form of software functional units and sold or used as independent products, can be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by instructing related hardware through a computer program, and the computer program can be stored in a computer readable storage medium. When the computer program is executed by a processor, the steps of the above-mentioned various method embodiments can be implemented. The computer program can include computer program code, which can be in the form of source code, object code, executable files or some intermediate forms, etc. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electric carrier signal and telecommunication signal.

[0142] The above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than limit them; although the present disclosure has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure.

Claims

1. A training method for an image quality assessment model, characterized in that, include: Obtain the first training image containing the target object and the label information corresponding to the first training image; The first training image is segmented to determine the foreground and background regions corresponding to the target object; By changing the first pixel information in the background region, a second training image is obtained; By changing the second pixel information in the foreground region, a third training image is obtained; The first training image, the second training image, and the third training image are input into a preset evaluation model to obtain a predicted quality score and a feature weight map; wherein, the feature weight map is determined by the spatial attention network in the preset evaluation model; Based on the second pixel information, the predicted quality score, the feature weight map, and the label information, the model parameters of the preset evaluation model are adjusted to obtain an image quality evaluation model that meets the preset training termination conditions. The predicted quality score includes a first predicted score corresponding to the first training image, a second predicted score corresponding to the second training image, and a third predicted score corresponding to the third training image; the step of obtaining an image quality assessment model that meets the preset training termination conditions includes: Based on the first predicted score and the label information, a first loss is determined; based on the second predicted score and the label information, a second loss is determined. Based on the feature weight map, a first influence coefficient corresponding to the second pixel information is determined; based on the first influence coefficient, the third prediction score, and the label information, a third loss is determined. Based on the first loss, the second loss, and the third loss, the model parameters of the preset evaluation model are adjusted to obtain an image quality evaluation model that meets the preset training termination conditions.

2. The method according to claim 1, characterized in that, The step of determining the first influence coefficient corresponding to the second pixel information based on the feature weight map includes: Determine the first dimension information corresponding to the feature weight map and the second dimension information corresponding to the first training image; Based on the first dimension information, the second dimension information, and the feature weight map, a second influence coefficient corresponding to each pixel in the first training image is determined. Based on the second pixel information and the second influence coefficient corresponding to each pixel in the first training image, the first influence coefficient corresponding to the second pixel information is determined.

3. The method according to claim 1, characterized in that, The method further includes: Determine the third influence coefficients corresponding to the second loss and the third loss, respectively; The step of adjusting the model parameters of the preset evaluation model based on the first loss, the second loss, and the third loss to obtain an image quality evaluation model that meets the preset training termination conditions includes: Based on the first loss, the second loss, the third loss, and the third influence coefficient, the model parameters of the preset evaluation model are adjusted to obtain an image quality evaluation model that meets the preset training termination conditions.

4. The method according to claim 3, characterized in that, The determination of the third influence coefficients corresponding to the second loss and the third loss respectively includes: Determine the ratio of the sum of the number of pixels in the foreground region to the sum of the number of pixels in the first training image; The ratio information is determined as the third influence coefficient corresponding to the third loss; the difference between the preset value and the ratio information is determined as the third influence coefficient corresponding to the second loss.

5. The method according to any one of claims 1-4, characterized in that, The step of inputting the first training image, the second training image, and the third training image into a preset evaluation model to obtain a predicted quality score and a feature weight map includes: Determine the first input probability corresponding to the first training image; Determine the second input probability corresponding to the second training image and the third training image; The first training image is input into a preset evaluation model with the first input probability, and the second training image and the third training image are input into the preset evaluation model with the second input probability to obtain the predicted quality score and the feature weight map.

6. An image quality assessment method, characterized in that, include: Obtain the image to be evaluated; wherein the image to be evaluated contains the target object; The image to be evaluated is input into the image quality assessment model to obtain the quality assessment result; The image quality assessment model is trained using the method described in any one of claims 1-5.

7. A training device for an image quality assessment model, characterized in that, include: The data acquisition module is used to acquire a first training image containing the target object and the label information corresponding to the first training image; The segmentation processing module is used to segment the first training image and determine the foreground region and background region corresponding to the target object; The processing module is modified to change the first pixel information in the background region to obtain a second training image; By changing the second pixel information in the foreground region, a third training image is obtained; The input processing module is used to input the first training image, the second training image, and the third training image into a preset evaluation model to obtain a predicted quality score and a feature weight map; wherein the feature weight map is determined by the spatial attention network in the preset evaluation model; A parameter adjustment module is used to adjust the model parameters of the preset evaluation model based on the second pixel information, the predicted quality score, the feature weight map, and the label information to obtain an image quality evaluation model that meets the preset training termination conditions. The predicted quality score includes a first prediction score corresponding to the first training image, a second prediction score corresponding to the second training image, and a third prediction score corresponding to the third training image. The step of obtaining an image quality evaluation model that meets the preset training termination conditions includes: determining a first loss based on the first prediction score and the label information; determining a second loss based on the second prediction score and the label information; determining a first influence coefficient corresponding to the second pixel information based on the feature weight map; determining a third loss based on the first influence coefficient, the third prediction score, and the label information; and adjusting the model parameters of the preset evaluation model based on the first loss, the second loss, and the third loss to obtain an image quality evaluation model that meets the preset training termination conditions.

8. An image quality assessment device, characterized in that, include: An image acquisition module is configured to acquire an image to be evaluated; wherein the image to be evaluated contains a target object; The image evaluation module is configured to input the image to be evaluated into the image quality evaluation model to obtain the quality evaluation result; The image quality assessment model is trained using the method described in any one of claims 1-5.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 6.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 6.

Images