Quality evaluation method, device, electronic device and storage medium

Through the multi-stage quality evaluation method, the combined use of coarse-grained and fine-grained models is used to solve the problem of low accuracy in the quality evaluation of multimedia data in the prior art, and high-precision evaluation of image or video quality is achieved.

CN112950567BActive Publication Date: 2024-11-19BEIJING KINGSOFT CLOUD NETWORK TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110211837.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-02-25
Publication Date
2024-11-19
Estimated Expiration
2041-02-25

AI Technical Summary

Technical Problem

The prior art has low accuracy in the quality evaluation of multimedia data such as images or videos, especially data with similar quality cannot be accurately measured, resulting in a decline in user visual experience.

Method used

Using a multi-stage strategy, first using the first quality evaluation model of coarse-grained size for preliminary evaluation, and then based on the prior knowledge, the second quality evaluation model of fine-grained size for fine adjustment to achieve quality evaluation from coarse to fine.

Benefits of technology

It improves the accuracy of multimedia data quality evaluation, can accurately measure the quality of images or videos, and improves the user's visual experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112950567B_ABST
    Figure CN112950567B_ABST
Patent Text Reader

Abstract

The quality evaluation method, device, electronic device and storage medium provided in the present application respectively use a first quality evaluation model and a second quality evaluation model to evaluate the quality of the object to be measured, thereby realizing a quality evaluation scheme that uses a multi-stage (two-stage) strategy to perform quality prediction from coarse to fine on the object to be measured, which can effectively solve the quality evaluation problem of multimedia data such as images or videos. In addition, since the scheme first uses the first quality evaluation model to perform coarse-grained quality evaluation on the object to be measured, and then further uses the coarse-grained quality evaluation result as prior knowledge and uses the second quality evaluation model to fine-tune the quality evaluation (ultimately obtaining a fine-grained quality evaluation result of the object to be measured), it can achieve high-precision quality evaluation of the object to be measured.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of multimedia data quality detection, and in particular to a quality evaluation method, device, electronic device and storage medium. Background Art

[0002] As an important medium for visualization and information interaction, multimedia such as digital images or videos have been widely developed in industries such as video conferencing and television broadcasting. Moreover, with the continuous improvement of the Internet and multimedia technologies, people's demand for various multimedia data is also increasing.

[0003] However, in the collection, encoding, and transmission of multimedia data such as images or videos, multimedia data generally faces a certain degree of quality loss. These quality losses usually lead to a significant decrease in the quality of multimedia data such as videos or images. Low-quality images or videos will seriously reduce the visual perception of the human eye. Therefore, in order to improve the user's visual experience, it is of great significance to provide a solution that can effectively evaluate the quality of multimedia data such as images or videos. Summary of the invention

[0004] In view of this, the present application provides a quality evaluation method, device, electronic device and storage medium, which predicts the quality of objects such as multimedia data by adopting a multi-stage strategy, thereby realizing a coarse-to-fine, high-precision quality evaluation scheme suitable for objects such as multimedia data.

[0005] The specific technical solutions are as follows:

[0006] A quality evaluation method, comprising:

[0007] Get the target object to be evaluated;

[0008] Using a first quality evaluation model to perform quality evaluation on the target object to obtain a first evaluation result;

[0009] Using a second quality evaluation model, performing a quality evaluation on the target object according to the first evaluation result to obtain a second evaluation result;

[0010] The second quality assessment model is trained based on samples of the first quality assessment model and the first evaluation results of the samples by the first quality assessment model, so that the quality assessment fine granularity corresponding to the second evaluation results of the second quality assessment model is higher than the quality assessment fine granularity corresponding to the first evaluation results of the first quality assessment model.

[0011] Optionally, the first quality assessment model includes a first feature extraction layer, a first generalization processing layer and a softmax layer;

[0012] The using the first quality evaluation model to perform quality evaluation on the target object to obtain a first evaluation result includes:

[0013] Inputting the target object into the first feature extraction layer, and performing feature extraction processing on the target object using the first feature extraction layer to obtain a first feature;

[0014] Refining the first feature using the first generalization processing layer to obtain a second feature;

[0015] The softmax layer is used to map the second feature into confidences corresponding to the first granularity scores in the sample label space; the first evaluation result includes: the target first granularity score with the highest confidence among the first granularity scores.

[0016] Optionally, the second quality assessment model includes a second feature extraction layer, a second generalization processing layer and a fully connected layer;

[0017] The using the second quality evaluation model to perform quality evaluation on the target object according to the first evaluation result to obtain a second evaluation result includes:

[0018] Inputting the target object into the second feature extraction layer, and performing feature extraction processing on the target object using the second feature extraction layer to obtain a third feature;

[0019] Refining the third feature using the second generalization processing layer to obtain a fourth feature;

[0020] Inputting the fourth feature and the first evaluation result into the fully connected layer, and mapping the fourth feature and the first evaluation result to a sample labeling space by using the fully connected layer to obtain a second granularity score corresponding to the target object in the sample labeling space;

[0021] Among them, the second evaluation result includes the second granularity score; the second granularity score is composed of the target first granularity score and a fine-tuning score, and the fine-tuning score is a non-negative value that is less than the absolute value of the difference between two adjacent first granularity scores.

[0022] Optional:

[0023] The number of the first generalization processing layer is one;

[0024] The number of the second generalization processing layers is multiple.

[0025] Optionally, the second quality evaluation model is: a model obtained by training one of the two model branches of the twin network model;

[0026] The second quality evaluation model is obtained by training the one branch using each sample in the sample set of the first quality evaluation model and a first evaluation result of each sample by the first quality evaluation model;

[0027] The twin network model is obtained by sorting and training a twin structure network determined based on a predetermined classification network using each sample pair in the sample set of the first quality evaluation model and a first evaluation result of the first quality evaluation model on each sample in the sample pair;

[0028] The sample pair is a sample pair consisting of any two samples in the sample set.

[0029] Optionally, the above method, before obtaining the target object to be evaluated, further includes:

[0030] Constructing the first quality evaluation model;

[0031] The second quality assessment model is constructed based on the sample set of the first quality assessment model and the first assessment result of the first quality assessment model on each sample in the sample set.

[0032] Optionally, the constructing the first quality evaluation model, based on the sample set of the first quality evaluation model and a first evaluation result of the first quality evaluation model on each sample in the sample set, constructing the second quality evaluation model includes:

[0033] Based on the sample set and the quality evaluation label annotated for each sample in the sample set, training the model network determined based on the first classification network to obtain the first quality evaluation model;

[0034] Based on each sample pair in the sample set, the quality evaluation label annotated for each sample in the sample pair, and the first evaluation result of the first quality evaluation model on each sample in the sample pair, the twin structure network determined based on the second classification network is sorted and trained to obtain a twin network model;

[0035] Based on the sample set, the quality evaluation label annotated for each sample in the sample set, and the first evaluation result of each sample in the sample set by the first quality evaluation model, training a branch in the twin network model to obtain the second quality evaluation model;

[0036] The sample pair is a sample pair consisting of any two samples in the sample set.

[0037] Optionally, the quality assessment label marked for the samples in the sample set is a second granularity score;

[0038] The first evaluation result includes a first granularity score; the evaluation result obtained by performing quality evaluation on the sample by any branch in the twin network model includes a second granularity score; the second evaluation result includes a second granularity score;

[0039] The quality evaluation fine granularity corresponding to the second granularity score is higher than the quality evaluation fine granularity corresponding to the first granularity score.

[0040] A quality evaluation device, comprising:

[0041] An acquisition unit, used for acquiring a target object to be evaluated;

[0042] A first evaluation unit, configured to perform quality evaluation on the target object using a first quality evaluation model to obtain a first evaluation result;

[0043] A second evaluation unit, configured to use a second quality evaluation model to perform quality evaluation on the target object according to the first evaluation result to obtain a second evaluation result;

[0044] The second quality assessment model is trained based on samples of the first quality assessment model and the first evaluation results of the samples by the first quality assessment model, so that the quality assessment fine granularity corresponding to the second evaluation results of the second quality assessment model is higher than the quality assessment fine granularity corresponding to the first evaluation results of the first quality assessment model.

[0045] An electronic device, comprising:

[0046] Memory, used to store computer instruction sets;

[0047] A processor is used to implement the quality evaluation method as described in any one of the above items by executing an instruction set stored in the memory.

[0048] A computer-readable storage medium stores a computer instruction set, and when the computer instruction set is executed by a processor, the quality evaluation method as described in any one of the above items is implemented.

[0049] According to the above scheme, the quality evaluation method, device, electronic device and storage medium provided by the present application, by respectively using the first quality evaluation model and the second quality evaluation model to evaluate the quality of the object to be measured, realizes a quality evaluation scheme that uses a multi-stage (two-stage) strategy to perform quality prediction from coarse to fine on the object to be measured, which can effectively solve the quality evaluation problem of multimedia data such as images or videos. In addition, since the scheme first uses the first quality evaluation model to perform coarse-grained quality evaluation on the object to be measured, and then further uses the coarse-grained quality evaluation result as prior knowledge, and uses the second quality evaluation model to fine-tune the quality evaluation (ultimately obtaining a fine-grained quality evaluation result of the object to be measured), it can achieve high-precision quality evaluation of the object to be measured. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0051] Figure 1 It is a flow chart of the quality evaluation method provided in the embodiment of the present application;

[0052] Figure 2 is a schematic diagram of the model structure of the first quality evaluation model provided in an embodiment of the present application;

[0053] Figure 3 is a schematic diagram of a process of performing quality evaluation on a target object using a first quality evaluation model provided in an embodiment of the present application;

[0054] Figure 4 is a schematic diagram of the model structure of the second quality evaluation model provided in the embodiment of the present application;

[0055] Figure 5 is a flow chart of using the second quality evaluation model to evaluate the quality of a target object provided in an embodiment of the present application;

[0056] Figure 6 is another flow chart of the quality evaluation method provided in the embodiment of the present application;

[0057] Figure 7 It is a flowchart of constructing a multi-stage quality evaluation model provided in an embodiment of the present application;

[0058] Figure 8 It is a schematic diagram of the model structure of the twin network model provided in the embodiment of the present application;

[0059] Fig. 9 It is a structural schematic diagram of a quality evaluation device provided in an embodiment of the present application;

[0060] Fig.10 is another structural schematic diagram of the quality evaluation device provided in an embodiment of the present application;

[0061] Fig.11 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0062] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0063] In order to improve the user's visual experience, it is of great significance to provide a solution that can effectively evaluate the quality of multimedia data such as images or videos. The inventor has found through research that traditional technologies generally directly obtain the quality score of videos or images through a single-stage task processing method. The accuracy of the quality evaluation results of this method is not high, especially for two videos or images with similar quality, it is often impossible to accurately measure the quality of the two videos or images.

[0064] To this end, the present application discloses a quality evaluation method, device, electronic device and storage medium. By adopting a multi-stage strategy to predict the quality of objects such as multimedia data, a coarse-to-fine, high-precision quality evaluation scheme suitable for objects such as multimedia data is implemented, so as to at least improve the accuracy of the quality evaluation results of traditional technologies for multimedia data such as images and videos to a certain extent.

[0065] See also Figure 1 , which is a flow chart of a quality evaluation method provided in an optional embodiment of the present application. The method can be applied to but not limited to terminal devices with data processing functions, such as mobile phones, tablet computers, personal PCs (such as notebooks, all-in-one computers, desktop computers), or corresponding physical machines such as private cloud / public cloud platforms, local area network / Internet servers, etc. with data processing functions.

[0066] like Figure 1 As shown, in this embodiment, the above-mentioned quality evaluation method specifically includes:

[0067] Step 101: Obtain the target object to be evaluated.

[0068] The target object to be evaluated may be, but is not limited to, multimedia data such as images or videos. When the target object is a video, it may be a complete video, or any video segment obtained by intercepting (or slicing) the complete video.

[0069] Typically, the method of the present application can be applied to filtering scenarios of multimedia data such as images / videos. In this scenario, the quality of uploaded images or videos is evaluated by using the method of the present application, and data that does not meet the quality requirements is filtered out to improve the user's visual perception. Correspondingly, in this scenario, the target object to be evaluated can be individual pictures or videos / video clips uploaded to the network (such as uploaded to a video client backend server).

[0070] Step 102: Use a first quality evaluation model to perform quality evaluation on the target object to obtain a first evaluation result.

[0071] The first quality evaluation model is the model used in the first stage of the multi-stage quality prediction of this application. The model is a coarse-grained quality evaluation model and can be pre-constructed based on a predetermined classification network. The classification network it is based on can be but is not limited to convolutional neural networks such as ResNet50 and Vgg16.

[0072] The model structure of the first quality assessment model is as follows: Figure 2 As shown, it includes the first feature extraction layer ( Figure 1 denoted as CNN Features), the first generalization processing layer ( Figure 2 , represented as FC Block) and softmax layer; wherein, the number of the first generalization processing layer can be one or more.

[0073] In implementation, the number of first generalization processing layers can be set according to actual needs on the basis of balancing model efficiency and effect. For a coarse-grained first quality evaluation model, it does not require fine-grained learning of the underlying rules between object features and object quality, and an increase in the number of first generalization processing layers will not bring about a significant improvement in model effect. In view of this, in implementation, it is preferred to set the number of first generalization processing layers in the first quality evaluation model to one.

[0074] based on Figure 2 The model structure of the first quality assessment model is shown in Figure 3 , the first evaluation result can be obtained by performing the following quality evaluation processing on the target object using the first quality evaluation model:

[0075] Step 301: input a target object into a first feature extraction layer, and use the first feature extraction layer to perform feature extraction processing on the target object to obtain a first feature;

[0076] Specifically, a target object such as an image or video can be input into the first feature extraction layer of the first quality assessment model. The first feature extraction layer can be specifically implemented as including one or more convolutional layers for extracting the underlying features of the target object. Taking the target object as an image or video as an example, the extracted underlying features may include but are not limited to the underlying features of the image frames in the image or video in terms of color, brightness, edges, corners and / or texture.

[0077] If the target object input to the first quality assessment model is a video, the frame-level features of at least part of the video frames (e.g., all the video frames of the video) are averaged to obtain the underlying features of the video. For ease of description, in this embodiment, the underlying features extracted by the first feature extraction layer are referred to as first features.

[0078] Step 302: Refine the first feature using the first generalization processing layer to obtain the second feature;

[0079] The first generalization processing layer receives the above-mentioned underlying features (first features) output by the first feature extraction layer, and performs further feature refinement processing on the received underlying features to prevent overfitting of the first quality evaluation model and improve the generalization ability of the first quality evaluation model.

[0080] See also Figure 2 In this embodiment, the first generalization processing layer consists of a fully connected layer ( Figure 2 FC), ReLU activation layer ( Figure 2 ReLU in ), normalization layer ( Figure 2 The first feature extraction layer is composed of a first feature extraction layer, a second feature extraction layer, and a second feature extraction layer. The first feature extraction layer is composed of a first feature extraction layer, a second feature extraction layer, and a second feature extraction layer. The first feature extraction layer is composed of a second ...

[0081] Step 303: Use a softmax layer to map the second feature into confidences corresponding to each first granularity score in the sample label space; the first evaluation result includes: the target first granularity score with the highest confidence among the first granularity scores.

[0082] The softmax layer receives the second feature output by the first generalization processing layer, and maps the second feature to the confidence corresponding to each first granularity score in the sample label space. The target first granularity score with the highest confidence is the first evaluation result obtained by the target object after being processed by the first quality evaluation model.

[0083] Among them, the first granularity score is a coarse-grained score in the sample labeling space. Assuming that the range of the subjective annotation score value of the sample in the sample labeling space is 1-5, and the annotated score value is specifically a floating point type, for example, 1.0, 1.1, 1.7, 3.5, 4.9, etc., then the first granularity scores (coarse-grained scores) corresponding to the sample labeling space can be specifically: 1, 2, 3, 4. From a logical perspective, the first granularity scores 1, 2, 3, 4 correspond to 1-2, 2-3, 3-4, 4-5 in the subjective annotation score values ​​1-5, that is, if the target first granularity score with the highest confidence output by the first quality assessment model is 1, it means that the actual quality score of the target object is between 1-2, and may be any score value between 1.0, 1.1...1.9.

[0084] It should be noted that the above-mentioned sample labeling space and its corresponding first granularity scores (coarse-grained scores) are only exemplary illustrations of the present application. In actual applications, other implementation forms may also be possible. For example, the sample labeling space is a natural number of 1-100, and the first granularity scores corresponding to the sample labeling space are: 0, 1, 2...9, or 0, 10, 20...90, etc., and logically correspond to actual labeling score values ​​of 1-9, 10-19, 20-29...90-99, respectively, without limitation here.

[0085] Step 103: Use a second quality evaluation model to perform quality evaluation on the target object according to the first evaluation result to obtain a second evaluation result.

[0086] Among them, the second quality evaluation model is trained based on the samples of the first quality evaluation model and the first evaluation results of the first quality evaluation model on its samples, so that the quality evaluation fine granularity corresponding to the second evaluation results of the second quality evaluation model is higher than the quality evaluation fine granularity corresponding to the first evaluation results of the first quality evaluation model.

[0087] The second quality evaluation model is the model used in the second stage of the multi-stage quality prediction of the present application. The model is a fine-grained quality evaluation model and can also be pre-constructed based on a predetermined classification network. The classification network on which it is based can be but is not limited to convolutional neural networks such as ResNet50 and Vgg16. In addition, the classification network on which the second quality evaluation model is based and the classification network on which the first quality evaluation model is based can be the same or different. For example, both quality evaluation models are based on ResNet50, or one of the quality evaluation models is based on ResNet50, and the other is based on Vgg16, etc.

[0088] See also Figure 4 , shows the model structure of the second quality assessment model, including the second feature extraction layer ( Figure 4 denoted as CNN Features), the second generalization processing layer ( Figure 4 FC Block) and the fully connected layer ( Figure 4 FC in the figure).

[0089] Similarly, in implementation, the number of second generalization processing layers can be set according to actual needs on the basis of balancing the efficiency and effect of the model. For a fine-grained second quality evaluation model, in view of the need to learn the potential rules between its fine-grained object (image or video, etc.) features and object quality, it is preferred that the number of second generalization processing layers in the second quality evaluation model can be set to multiple.

[0090] based on Figure 4 The model structure of the second quality assessment model is shown in Figure 5 , the second evaluation result can be obtained by performing the following quality evaluation processing on the target object using the second quality evaluation model:

[0091] Step 501: Input the target object into the second feature extraction layer, and use the second feature extraction layer to perform feature extraction processing on the target object to obtain a third feature.

[0092] Similar to the function of the first feature extraction layer in the first quality assessment model, the second feature extraction layer in the second quality assessment model can also be implemented to include one or more convolutional layers for extracting underlying features of the target object, such as underlying features of color, brightness, edges, corners and / or texture of image frames in images or videos, etc. The underlying features extracted by the second feature extraction layer are described as third features.

[0093] If the target object input to the second quality assessment model is a video, the frame-level features of at least some video frames (eg, all video frames) of the video are also averaged to obtain the underlying features of the video.

[0094] Step 502: Refine the third feature using the second generalization processing layer to obtain a fourth feature;

[0095] Similarly, the second generalization processing layer may also include a fully connected layer (FC), a ReLU activation layer (ReLU), a normalization layer (Batch Norm) and Dropout, which is used to further refine the underlying features (third features) of the target object extracted by the second feature extraction layer through the various layers included to obtain the fourth feature, so as to prevent the second quality evaluation model from overfitting and improve the generalization ability of the second quality evaluation model.

[0096] The difference from the first quality evaluation model is that the first quality evaluation model learns the rules between object (image or video, etc.) features and coarse-grained object quality during the model training stage, while the second quality evaluation model learns the rules between object (image or video, etc.) features and fine-grained object quality during the model training stage. This is reflected in the model itself in the differences in the model's network parameters and / or the specific processing processes of each functional layer in the model (such as convolution-based feature extraction, feature refinement, etc.).

[0097] Step 503: Input the fourth feature and the first evaluation result into a fully connected layer, and use the fully connected layer to map the fourth feature and the first evaluation result to the sample labeling space to obtain a second granularity score corresponding to the target object in the sample labeling space.

[0098] The second evaluation result output by the second quality evaluation model for performing quality evaluation on the target object includes the second granularity score.

[0099] Specifically, the fully connected layer receives the fourth feature output by the second generalization processing layer, and, as Figure 4 As shown, the fully connected layer also takes the first evaluation result output by the first quality evaluation model as input (such as Figure 4 , MOS1 is input to FC). After that, the two inputs are fused, wherein the vectors corresponding to the two inputs are fused (concat), and the fusion result is mapped to the sample label space to obtain the second granularity score corresponding to the target object in the sample label space.

[0100] The quality evaluation fine granularity corresponding to the second granularity score is higher than the quality evaluation fine granularity corresponding to the first granularity score, and the second granularity score and the subjective annotation score value of the sample in the sample labeling space belong to the same fine granularity. Still using the above example, assuming that the target first granularity score with the highest confidence output by the first quality evaluation model is 1, the second granularity score output by the second quality evaluation model can be specifically 1.7.

[0101] In the present application, the first evaluation result of the first quality evaluation model is used together with the target object to be evaluated as the input of the second quality evaluation model, and the second quality evaluation model performs quality evaluation on the target object in combination with the first evaluation result. The essential purpose is to provide the first evaluation result of the first quality evaluation model as prior knowledge to the second quality evaluation model, so that the second quality evaluation model does not need to perform coarse-grained quality evaluation on the target object, but only fine-tunes on the basis of the coarse-grained quality evaluation result obtained a priori (the target first granularity score with the highest confidence), and obtains the corresponding matching fine-grained quality evaluation result (the second granularity score in the second evaluation result). Therefore, in essence, the second granularity score in the second evaluation result is composed of the target first granularity score with the highest confidence in the first evaluation result and a fine-tuning score, and the fine-tuning score is a non-negative value that is less than the absolute value of the difference between two adjacent first granularity scores.

[0102] It is easy to understand that during the model training stage, the second quality evaluation model also uses the coarse-grained quality evaluation results output by the first quality evaluation model as prior knowledge, and only learns the potential rules between object (image or video, etc.) features and object quality in a targeted and fine-grained manner. Accordingly, the model has the function of fine-tuning the quality score based on the input a priori coarse-grained quality evaluation results, and the coarse-grained "potential rules between object features and object quality" are obtained through learning by the first quality evaluation model.

[0103] According to the above scheme, it can be seen that the quality evaluation method of this embodiment, by respectively using the first quality evaluation model and the second quality evaluation model to evaluate the quality of the object to be measured, realizes a quality evaluation scheme that uses a multi-stage (two-stage) strategy to perform quality prediction from coarse to fine on the object to be measured, which can effectively solve the quality evaluation problem of multimedia data such as images or videos. In addition, since the scheme first uses the first quality evaluation model to perform coarse-grained quality evaluation on the object to be measured, and then further uses the coarse-grained quality evaluation result as prior knowledge, and uses the second quality evaluation model to fine-tune the quality evaluation (ultimately obtaining a fine-grained quality evaluation result of the object to be measured), it can achieve high-precision quality evaluation of the object to be measured.

[0104] Traditional technology directly obtains the quality score of a video or image through a single-stage task processing method, and the accuracy of its quality evaluation result is not high. After research, the inventor found that the fundamental reason is that the quality evaluation task is essentially a regression task, that is, a certain video or image data is input, and the refined quality result of the data (such as a refined quality score) is regressed through the model. The higher the accuracy required by the regression task, the greater its difficulty. In other words, it is difficult for a model in a single-stage task processing to learn the potential law between the object features such as images / videos and the quality evaluation scores of images / videos in the full sample labeling space with high accuracy. This invention Please use a multi-stage (two-stage) strategy to predict the quality of the target object from coarse to fine, so that one model can be used to learn the coarse-grained "potential rules between object features and object quality" (to preliminarily determine which coarse-grained score it belongs to based on the coarse-grained object features), and another model can be used to use the coarse-grained quality evaluation results as prior knowledge to specifically learn the fine-grained "potential rules between object features and object quality" (to further determine which fine-grained score it belongs to based on the coarse-grained score based on the fine-grained object features). This solves this problem and improves the quality evaluation accuracy of images or videos.

[0105] In addition, the single-stage task processing method of traditional technology often cannot accurately measure the quality of two videos or images with similar quality.

[0106] To further solve this problem, optionally, in an embodiment of the present application, when constructing the second quality evaluation model, the "sample pairs" in the sample set of the first quality evaluation model and the first evaluation results of the first quality evaluation model for each sample in the sample pair are first used to sort and train the twin structure network designed based on a predetermined classification network to obtain a twin network model.

[0107] The predetermined classification network based on which the twin network model is constructed may be, but is not limited to, a convolutional neural network such as ResNet50 or Vgg16, and the classification network based on which the first quality assessment model is constructed may be the same as or different from the classification network based on which the first quality assessment model is constructed. The two branches in the twin network model share network parameters, and based on sorting training, each branch in the twin network model can effectively measure the quality of two videos or images of similar quality.

[0108] On this basis, one of the branches is selected, and the branch is further trained using the samples in the sample set of the first quality evaluation model and the first evaluation results of each sample by the first quality evaluation model. After this training, the selected branch can further use the coarse-grained quality evaluation results as prior knowledge to learn the fine-grained "potential laws between object features and object quality", and finally obtain the second quality evaluation model.

[0109] Next, the construction process of each stage model involved in this application will be described in detail in an embodiment below.

[0110] In this embodiment, the second quality assessment model has both the function of measuring the quality of two objects such as videos or images with similar quality, and the function of fine-grained quality assessment based on the coarse-grained quality assessment results as prior results. While improving the quality assessment accuracy of objects such as images or videos, it can further achieve the purpose of accurately measuring the quality of two objects such as videos or images with similar quality.

[0111] The implementation of the method of the present application requires the pre-construction of the first quality evaluation model and the second quality evaluation model. Therefore, in an optional embodiment, Figure 6 The quality evaluation method shown in FIG. 1 may further include, before step 101, the following steps:

[0112] Step 601: construct a first quality evaluation model; and construct a second quality evaluation model based on a sample set of the first quality evaluation model and a first evaluation result of the first quality evaluation model on each sample in the sample set.

[0113] See also Figure 7 The process of constructing the first quality evaluation model and the second quality evaluation model includes:

[0114] Step 701: Based on a sample set and a quality evaluation label annotated for each sample in the sample set, a twin structure network determined based on a first classification network is trained to obtain the first quality evaluation model.

[0115] For the construction of the first quality evaluation model, it is first necessary to construct a sample space and a sample labeling space, wherein a series of images or videos can be selected as a sample set, and each sample is labeled for quality. In this embodiment, it is assumed that the range of the subjective labeling score value when the sample set is labeled for quality is 1-5, wherein the labeled score value is a floating point type, for example, 1.0, 1.1, 1.7, 3.5, 4.9, etc. Each subjective labeling score value in the range of 1-5 constitutes a sample labeling space, and the subjective labeling score values ​​of the sample labeling space are divided into four groups, namely 1-2, 2-3, 3-4, and 4-5, and the subjective labeling score values ​​of the videos or images of the corresponding score segments are mapped to four coarse-grained scores of 1, 2, 3, and 4 (i.e., the first granularity score mentioned above).

[0116] Afterwards, each sample is input into the network structure designed based on the selected first classification network such as ResNet50 or Vgg16 for model training. The designed model network structure is as follows: Figure 2As shown, it includes a first feature extraction layer, a first generalization processing layer and a softmax layer, wherein the first feature extraction layer can be implemented as including one or more convolutional layers, the number of the first generalization processing layer can be one or more, preferably one, and the first generalization processing layer includes a fully connected layer, a ReLU activation layer, a normalization layer and a Dropout layer. When inputting a sample, the sample is specifically input into the first feature extraction layer of the network.

[0117] During the model training process, the coarse-grained scores (such as 1, 2, 3, and 4 above) obtained by mapping the subjective annotation score values ​​of the videos or images of the corresponding score segments are used as the basis to measure the model loss and the model parameters and model optimization based on this, until the model converges or reaches the preset number of iterations. The trained coarse-grained quality evaluation network can be obtained and used as the first quality evaluation model of the present application. The functions of each layer in the model network structure can be specifically combined with the description of the above embodiments, and will not be repeated here.

[0118] Step 702: Based on each sample pair in the sample set, the quality evaluation label annotated for each sample in the sample pair, and the first evaluation result of the first quality evaluation model on each sample in the sample pair, the twin structure network determined based on the second classification network is sorted and trained to obtain a twin network model;

[0119] Optionally, in order to effectively measure the quality of two videos or images of similar quality to be measured, for the construction of the second quality evaluation model, this embodiment first trains a network model with a twin network structure, and then further selects a branch of the twin network model to train the second quality evaluation model.

[0120] The training of the twin network model uses each pair of samples in the sample set of the first quality evaluation model as training samples, and uses the ranking of the two subjective annotation scores corresponding to each sample pair in the sample label space of the first quality evaluation model (reflecting the quality of the two samples) as labels. In implementation, preferably, in order to obtain a better model effect, the sample set should contain a sufficient number of pairs of samples with similar quality.

[0121] Afterwards, the two samples in each sample pair are input one by one into the two branches of the twin network structure designed based on the selected second classification network such as ResNet50 or Vgg16. The network structure of each branch is as follows: Figure 8As shown, it includes a second feature extraction layer, a second generalization processing layer and a fully connected layer. Among them, the second feature extraction layer can be implemented as including one or more convolutional layers, the number of the second generalization processing layer can be one or more, preferably multiple, and the second generalization processing layer includes a fully connected layer, a ReLU activation layer, a normalization layer and a Dropout layer. When inputting a sample, the sample is specifically input into the second feature extraction layer of the corresponding branch.

[0122] The two branches of the twin network model share network parameters during training, and the loss function adopts the rankingloss function.

[0123] In addition, during the model training process, the first evaluation result of the first quality evaluation model for the sample input in the branch (the target first granularity score with the highest confidence) is added to the input of each branch's fully connected layer, and is fused with the features output by the functional layer before the fully connected layer in the branch structure, and then the model is trained in the fully connected layer. After the two samples in the sample pair are input one-to-one into the two branches of the twin network model, each branch outputs the quality evaluation value of the input sample of the corresponding branch, and calculates the corresponding loss value through the rankingloss loss function, and then feeds back to the model to update the network parameters until the model converges or reaches the preset number of iterations, and a trained twin network model is obtained accordingly.

[0124] Compared with the first quality evaluation model mentioned above, the twin network model has a finer granularity in quality evaluation of objects such as images or videos. During the training process, the quality evaluation values ​​output by each branch of the twin network model are fine-grained quality scores, such as 1.2, 1.3, etc.

[0125] It should be noted that during the training phase of the twin network model, the model learns the object features of two samples of similar quality in the sample pair and the potential rules between the quality of the two samples. During the training process, the rankingloss loss function is used to measure the difference between the quality ranking of the two samples reflected by the output results of the two branches and the quality ranking of the two samples reflected by the subjective annotation scores in the sample labeling space. The model loss and the model parameter adjustment and model optimization based on this are finally obtained, and two model branches with shared network parameters can be obtained. That is, during the training process, the measurement of model loss does not consider the difference between the output scores of the model branches and the annotation scores, but only considers the difference between the ranking results. For example, if the annotation scores of the two samples in the sample pair are 1.1 and 1.2 respectively, and the scores output by the two branches of the model are 1.2 and 1.3, and the ranking results of the two are the same, then the model output is considered to be consistent with the annotation results and there is no loss.

[0126] Step 703: Based on the sample set, the quality evaluation label annotated for each sample in the sample set, and the first evaluation result of the first quality evaluation model on each sample in the sample set, train a branch in the twin network model to obtain the second quality evaluation model.

[0127] After completing the training of the above-mentioned twin network model, select a branch (any branch) therefrom for further model training to construct a second quality evaluation model.

[0128] The sample set based on which the model is trained is still the sample set used for training the first quality evaluation model. The sample labeling space is the labeling space corresponding to the above-mentioned subjective labeling score value range, such as the fine quality scores of various floating-point types in the range of 1-5. The model structure follows the structure of the model branch in the twin network model, as shown in the following example. Figure 4 As shown, it includes a second feature extraction layer, a second generalization processing layer and a fully connected layer.

[0129] During the training process, each sample in the sample set is input into the selected branch, specifically into the second feature extraction layer of the selected branch, and after feature extraction by the second feature extraction layer and feature refinement by the second generalization processing layer, the result is output to the fully connected layer. In addition, the first evaluation result of the first quality evaluation model on the input sample in the branch (the target first granularity score with the highest confidence, such as Figure 4 The MOS1 input to the FC in the branch structure is merged with the features output by the functional layer before the fully connected layer in the branch structure, and then the model is trained in the fully connected layer. By calculating the loss between the model output and the subjective annotation score (the fine-grained score of the original annotation, i.e., the second granularity score mentioned above, such as 1.1, 1.7, 3.5, 4.9, etc.), continuous iteration and training are performed to achieve model fine-tuning, so that the prediction results of the model continue to approach the fine-grained subjective annotation score until the model converges or reaches the preset number of iterations, and the trained fine-grained quality evaluation network is obtained accordingly, i.e., the second quality evaluation model mentioned above.

[0130] The model loss can be specifically measured by L1 loss or L2 loss, where L1 loss refers to the mean absolute error (MAE), which is a loss function used for regression models; L2 loss refers to the mean-square error (MSE), which is another loss function used for regression models.

[0131] The training in this stage enables the model to have the function of effectively measuring the quality of two objects such as videos or images with similar quality. It uses the coarse-grained quality evaluation results output by the first quality evaluation model as prior knowledge to further learn the underlying rules between fine-grained object features and object quality, so that the model output is closer to the actual annotation score (for example, for a trained twin network model, it can effectively correctly rank the quality of two objects with similar quality, but may predict an object with an annotation quality score of 1.1 as a quality score of 1.2 or 1.3. Based on the training in this stage, this problem of the twin network model's branches can be further corrected, so that its output is closer to the actual annotation result).

[0132] During the model usage phase, objects such as videos or images are input into the trained first quality evaluation model to obtain a coarse-grained quality evaluation result (the first evaluation result above). The object is then further input into the trained second quality evaluation model, and the coarse-grained quality evaluation result of the first quality evaluation model is added as input to the fully connected layer of the second quality evaluation model. Finally, the second quality evaluation model outputs a fine-grained quality evaluation result of the object (the second evaluation result above). The specific quality evaluation process can be referred to the description of the corresponding embodiment above, which will not be repeated here.

[0133] Based on the model building process provided in this embodiment, it is possible to implement a coarse-to-fine quality evaluation of the image or video to be tested based on a multi-stage evaluation strategy. Compared with the single-stage quality evaluation method of traditional technology, the quality evaluation result is more accurate and can effectively measure the quality of two videos or images with similar quality.

[0134] Corresponding to the above quality evaluation method, the present application embodiment also provides a quality evaluation device, such as Fig. 9 The structural schematic diagram of the device shown may include:

[0135] An acquisition unit 901 is used to acquire a target object to be evaluated;

[0136] A first evaluation unit 902 is used to perform quality evaluation on the target object using a first quality evaluation model to obtain a first evaluation result;

[0137] A second evaluation unit 903 is used to use a second quality evaluation model to perform quality evaluation on the target object according to the first evaluation result to obtain a second evaluation result;

[0138] Among them, the second quality evaluation model is trained based on the samples of the first quality evaluation model and the first evaluation results of the first quality evaluation model on its samples, so that the quality evaluation fine granularity corresponding to the second evaluation results of the second quality evaluation model is higher than the quality evaluation fine granularity corresponding to the first evaluation results of the first quality evaluation model.

[0139] In an optional implementation of the embodiment of the present application, the first quality assessment model includes a first feature extraction layer, a first generalization processing layer and a softmax layer;

[0140] The first evaluation unit 902 is specifically used for:

[0141] Inputting the target object into the first feature extraction layer, and performing feature extraction processing on the target object using the first feature extraction layer to obtain a first feature;

[0142] Refining the first feature using the first generalization processing layer to obtain a second feature;

[0143] The softmax layer is used to map the second feature into confidences corresponding to the first granularity scores in the sample label space; the first evaluation result includes: the target first granularity score with the highest confidence among the first granularity scores.

[0144] In an optional implementation of the embodiment of the present application, the second quality assessment model includes a second feature extraction layer, a second generalization processing layer and a fully connected layer;

[0145] The second evaluation unit 903 is specifically used for:

[0146] Inputting the target object into the second feature extraction layer, and performing feature extraction processing on the target object using the second feature extraction layer to obtain a third feature;

[0147] Refining the third feature using the second generalization processing layer to obtain a fourth feature;

[0148] Inputting the fourth feature and the first evaluation result into the fully connected layer, and mapping the fourth feature and the first evaluation result to a sample labeling space by using the fully connected layer to obtain a second granularity score corresponding to the target object in the sample labeling space;

[0149] Among them, the second evaluation result includes the second granularity score; the second granularity score is composed of the target first granularity score and a fine-tuning score, and the fine-tuning score is a non-negative value that is less than the absolute value of the difference between two adjacent first granularity scores.

[0150] In an optional implementation of the embodiment of the present application:

[0151] The number of the first generalization processing layer is one;

[0152] The number of the second generalization processing layers is plural.

[0153] In an optional implementation of the embodiment of the present application, the second quality evaluation model is: a model obtained by training one of the two model branches of the twin network model;

[0154] A second quality evaluation model is obtained by training the one branch using each sample in the sample set of the first quality evaluation model and a first evaluation result of each sample by the first quality evaluation model;

[0155] A twin network model is obtained by sorting and training a twin structure network determined based on a predetermined classification network by using each sample pair in the sample set of the first quality evaluation model and a first evaluation result of the first quality evaluation model on each sample in the sample pair;

[0156] The sample pair is a sample pair consisting of any two samples in the sample set.

[0157] In an optional implementation of the present application, see Fig.10 The quality evaluation device of the present application may further include:

[0158] The model building unit 904 is used to: build the first quality evaluation model before obtaining the target object to be evaluated; and build the second quality evaluation model based on the sample set of the first quality evaluation model and the first evaluation result of the first quality evaluation model on each sample in the sample set.

[0159] In an optional implementation of the embodiment of the present application, the model building unit 904 is specifically used to:

[0160] Based on the sample set and the quality evaluation label annotated for each sample in the sample set, training the model network determined based on the first classification network to obtain the first quality evaluation model;

[0161] Based on each sample pair in the sample set, the quality evaluation label annotated for each sample in the sample pair, and the first evaluation result of the first quality evaluation model on each sample in the sample pair, the twin structure network determined based on the second classification network is sorted and trained to obtain a twin network model;

[0162] Based on the sample set, the quality evaluation label annotated for each sample in the sample set, and the first evaluation result of each sample in the sample set by the first quality evaluation model, training a branch in the twin network model to obtain the second quality evaluation model;

[0163] The sample pair is a sample pair consisting of any two samples in the sample set.

[0164] In an optional implementation of the embodiment of the present application, the quality evaluation label marked for the samples in the sample set is a second granularity score;

[0165] The first evaluation result includes a first granularity score; the evaluation result obtained by performing quality evaluation on the sample by any branch in the twin network model includes a second granularity score; the second evaluation result includes a second granularity score;

[0166] The quality evaluation fine granularity corresponding to the second granularity score is higher than the quality evaluation fine granularity corresponding to the first granularity score.

[0167] As for the quality evaluation device disclosed in the embodiment of the present application, since it corresponds to the quality evaluation method disclosed in the above method embodiment, the description is relatively simple. For relevant similarities, please refer to the description of the corresponding method embodiment above, which will not be described in detail here.

[0168] An embodiment of the present application also discloses an electronic device, which may be, but is not limited to, a terminal device such as a mobile phone, a tablet computer, a personal PC (such as a notebook, an all-in-one computer, a desktop computer) with data processing capabilities, or a physical machine corresponding to a private cloud / public cloud platform, a local area network / Internet server, etc. with data processing capabilities.

[0169] like Fig.11 The structural schematic diagram of the electronic device shown at least includes:

[0170] Memory 1101, used for storing computer instruction sets;

[0171] The computer instruction set may be implemented in the form of a computer program.

[0172] The memory 1101 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0173] The processor 1102 is used to implement the quality evaluation method of any of the above method embodiments by executing an instruction set stored in the memory.

[0174] Among them, the processor 1102 can be a central processing unit (CPU), an application-specific integrated circuit (ASIC), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, etc.

[0175] In addition, the electronic device may also include components such as a communication interface and a communication bus. The memory, the processor and the communication interface communicate with each other through the communication bus.

[0176] The communication interface is used for communication between electronic devices and other devices. The communication bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc.

[0177] In this embodiment, when the processor in the electronic device executes the computer instruction set stored in the memory, it respectively uses the first quality evaluation model and the second quality evaluation model to evaluate the quality of the object to be measured, thereby realizing a quality evaluation scheme that uses a multi-stage (two-stage) strategy to perform coarse-to-fine quality prediction on the object to be measured, which can effectively solve the quality evaluation problem of multimedia data such as images or videos. In addition, since the scheme first uses the first quality evaluation model to perform coarse-grained quality evaluation on the object to be measured, and then further uses the coarse-grained quality evaluation result as prior knowledge, and uses the second quality evaluation model to fine-tune the quality evaluation (ultimately obtaining a fine-grained quality evaluation result of the object to be measured), it can achieve high-precision quality evaluation of the object to be measured.

[0178] In addition, an embodiment of the present application further discloses a computer-readable storage medium, in which a computer instruction set is stored. When the computer instruction set is executed by a processor, a quality evaluation method disclosed in any of the above method embodiments is implemented.

[0179] Specifically, when the computer instruction set in the computer-readable storage medium of this embodiment is executed by the processor, the quality of the object to be measured is evaluated by using the first quality evaluation model and the second quality evaluation model respectively, thereby realizing a quality evaluation scheme that uses a multi-stage (two-stage) strategy to perform coarse-to-fine quality prediction on the object to be measured, which can effectively solve the quality evaluation problem of multimedia data such as images or videos. In addition, since the scheme first uses the first quality evaluation model to perform coarse-grained quality evaluation on the object to be measured, and then further uses the coarse-grained quality evaluation result as prior knowledge and uses the second quality evaluation model to fine-tune the quality evaluation (ultimately obtaining a fine-grained quality evaluation result of the object to be measured), it can achieve high-precision quality evaluation of the object to be measured.

[0180] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0181] For the convenience of description, the above system or device is described by dividing it into various modules or units according to its functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0182] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly contributed to the prior art in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0183] Finally, it should be noted that, in this article, relational terms such as first, second, third and fourth are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0184] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A quality evaluation method, characterized in that: include: Get the target object to be evaluated; The target object to be evaluated is multimedia data; Using a first quality evaluation model to perform quality evaluation on the target object to obtain a first evaluation result; Using a second quality evaluation model, performing a quality evaluation on the target object according to the first evaluation result to obtain a second evaluation result; The second quality assessment model is trained based on the sample of the first quality assessment model and the first assessment result of the sample by the first quality assessment model, so that the quality assessment fine granularity corresponding to the second assessment result of the second quality assessment model is higher than the quality assessment fine granularity corresponding to the first assessment result of the first quality assessment model; Wherein, the first quality assessment model includes a first feature extraction layer, a first generalization processing layer and a softmax layer; The using the first quality evaluation model to perform quality evaluation on the target object to obtain a first evaluation result includes: Inputting the target object into the first feature extraction layer, and performing feature extraction processing on the target object using the first feature extraction layer to obtain a first feature; Refining the first feature using the first generalization processing layer to obtain a second feature; The second feature is mapped into confidences corresponding to the first granularity scores in the sample label space by using the softmax layer; the first evaluation result includes: the target first granularity score with the highest confidence among the first granularity scores; The second quality assessment model includes a second feature extraction layer, a second generalization processing layer and a fully connected layer; The using the second quality evaluation model to perform quality evaluation on the target object according to the first evaluation result to obtain a second evaluation result includes: Inputting the target object into the second feature extraction layer, and performing feature extraction processing on the target object using the second feature extraction layer to obtain a third feature; Refining the third feature using the second generalization processing layer to obtain a fourth feature; Inputting the fourth feature and the first evaluation result into the fully connected layer, and mapping the fourth feature and the first evaluation result to a sample labeling space by using the fully connected layer to obtain a second granularity score corresponding to the target object in the sample labeling space; Among them, the second evaluation result includes the second granularity score; the second granularity score is composed of the target first granularity score and a fine-tuning score, and the fine-tuning score is a non-negative value that is less than the absolute value of the difference between two adjacent first granularity scores.

2. The method according to claim 1, characterized in that: The number of the first generalization processing layer is one; The number of the second generalization processing layers is multiple.

3. The method according to claim 1, characterized in that The second quality evaluation model is: a model obtained by training one of the two model branches of the twin network model; The second quality evaluation model is obtained by training the one branch using each sample in the sample set of the first quality evaluation model and a first evaluation result of each sample by the first quality evaluation model; The twin network model is obtained by sorting and training a twin structure network determined based on a predetermined classification network using each sample pair in the sample set of the first quality evaluation model and a first evaluation result of the first quality evaluation model on each sample in the sample pair; The sample pair is a sample pair consisting of any two samples in the sample set.

4. The method according to claim 1, characterized in that: Before obtaining the target object to be evaluated, the method further includes: Constructing the first quality evaluation model; The second quality assessment model is constructed based on the sample set of the first quality assessment model and the first assessment result of the first quality assessment model on each sample in the sample set.

5. The method according to claim 4, characterized in that The constructing of the first quality evaluation model, based on the sample set of the first quality evaluation model and the first evaluation result of the first quality evaluation model on each sample in the sample set, constructing the second quality evaluation model, comprises: Based on the sample set and the quality evaluation label annotated for each sample in the sample set, training the model network determined based on the first classification network to obtain the first quality evaluation model; Based on each sample pair in the sample set, the quality evaluation label annotated for each sample in the sample pair, and the first evaluation result of the first quality evaluation model on each sample in the sample pair, the twin structure network determined based on the second classification network is sorted and trained to obtain a twin network model; Based on the sample set, the quality evaluation label annotated for each sample in the sample set, and the first evaluation result of each sample in the sample set by the first quality evaluation model, training a branch in the twin network model to obtain the second quality evaluation model; The sample pair is a sample pair consisting of any two samples in the sample set.

6. The method according to claim 5, characterized in that The quality assessment labels marked for the samples in the sample set are second granularity scores; The first evaluation result includes a first granularity score; the evaluation result obtained by performing quality evaluation on the sample by any branch in the twin network model includes a second granularity score; the second evaluation result includes a second granularity score; The quality evaluation fine granularity corresponding to the second granularity score is higher than the quality evaluation fine granularity corresponding to the first granularity score.

7. A quality evaluation device, characterized in that: include: An acquisition unit, used for acquiring a target object to be evaluated; The target object to be evaluated is multimedia data; A first evaluation unit, configured to perform quality evaluation on the target object using a first quality evaluation model to obtain a first evaluation result; A second evaluation unit, configured to use a second quality evaluation model to perform quality evaluation on the target object according to the first evaluation result to obtain a second evaluation result; The second quality assessment model is trained based on the sample of the first quality assessment model and the first assessment result of the sample by the first quality assessment model, so that the quality assessment fine granularity corresponding to the second assessment result of the second quality assessment model is higher than the quality assessment fine granularity corresponding to the first assessment result of the first quality assessment model; Wherein, the first quality assessment model includes a first feature extraction layer, a first generalization processing layer and a softmax layer; The using the first quality evaluation model to perform quality evaluation on the target object to obtain a first evaluation result includes: Inputting the target object into the first feature extraction layer, and performing feature extraction processing on the target object using the first feature extraction layer to obtain a first feature; Refining the first feature using the first generalization processing layer to obtain a second feature; The second feature is mapped into confidences corresponding to the first granularity scores in the sample label space by using the softmax layer; the first evaluation result includes: the target first granularity score with the highest confidence among the first granularity scores; The second quality assessment model includes a second feature extraction layer, a second generalization processing layer and a fully connected layer; The second evaluation unit uses a second quality evaluation model to perform quality evaluation on the target object according to the first evaluation result to obtain a second evaluation result, including: Inputting the target object into the second feature extraction layer, and performing feature extraction processing on the target object using the second feature extraction layer to obtain a third feature; Refining the third feature using the second generalization processing layer to obtain a fourth feature; Inputting the fourth feature and the first evaluation result into the fully connected layer, and mapping the fourth feature and the first evaluation result to a sample labeling space by using the fully connected layer to obtain a second granularity score corresponding to the target object in the sample labeling space; Among them, the second evaluation result includes the second granularity score; the second granularity score is composed of the target first granularity score and a fine-tuning score, and the fine-tuning score is a non-negative value that is less than the absolute value of the difference between two adjacent first granularity scores.

8. An electronic device, characterized in that: include: Memory, used to store computer instruction sets; A processor, configured to implement the quality evaluation method according to any one of claims 1 to 6 by executing an instruction set stored in the memory.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer instruction set, and when the computer instruction set is executed by a processor, the quality evaluation method according to any one of claims 1 to 6 is implemented.