Automatic video quality assessment

Abstract

The invention proposes a method (PM) for predicting an objective quality score (VQSo) of an image or video (VID). The method comprises: obtaining (E10) at least one selected feature (FEATS) associated with the image or video; selecting (E20) a set of parameters (MODi) as a function of the at least one selected feature (FEAT S ); and determining (E30) the objective quality score (VQSo) of the image or video (VID) as a function of at least one qualified feature (FEATQ) of the image or video (VID) by using a learning-based prediction method (LBPM) configured by the set of parameters (MODi). A training method is used to determine different sets of parameters of a learning-based prediction method as a function of a set of images or videos. The method comprises a phase of training the learning-based prediction method only by images or videos of a subset of said set, said subset comprising only images or videos classified into a category selected as a function of the at least one selected feature, thereby learning a set of parameters of the learning-based prediction method.

Description

Technical Field

[0001] This invention generally relates to the field of video communication.

[0002] More specifically, this invention proposes systems and methods for improving the performance of image and video quality assessment. Background Technology

[0003] The context of this disclosure is the automated evaluation of video quality using so-called objective quality metrics. This disclosure includes quality evaluation of all types of video, particularly game and cloud gaming video content, 2D and immersive video content.

[0004] For example, in cloud gaming applications, gamers can access games using client devices, with the game scenes generated by a remote server. The player's action and control information, obtained from the input device, is captured and transmitted from the client to the server. The client is typically a personal computer or mobile phone.

[0005] On the server side, received action and control information can influence the generation of game scenes. By analyzing the generated game scenes and / or the user's action and control information, the game engine renders new scenes.

[0006] The corresponding raw video rendered by the game engine is encoded, and the bitstream is sent back to the client. On the client side, the bitstream is decoded, and game images are displayed.

[0007] Objective quality assessment refers to evaluating the perceived quality of an image or video in order to predict the subjective quality perceived by a human observer in an automatic manner (typically through average opinion scores).

[0008] In cloud gaming environments, having reliable, objective metrics is a key differentiator for service providers. When used in the cloud, this metric allows for adjustments to the characteristics of the game engine and video encoder to ensure sufficient quality video streams are delivered to users. When used on the client side, the metric provides important information about the quality perceived by the user. When sent back to the server via a feedback channel, the metric can influence the behavior of the game engine and encoder.

[0009] Therefore, not only in the context of cloud gaming, automated video quality assessment is important at all stages of the video creation, transmission, and rendering pipeline, because non-automatic methods (such as subjective assessments by human observers) are either unrealistic or too expensive.

[0010] Recently, automatic video quality assessment has utilized a new class of learning-based algorithms, such as ITU-T recommendations P.1203.1 and P.1204.3 (objective bitstream-based metrics based on low-complexity learning), CNN-based metrics NDNetGaming, and Deep Bilinear Convolutional Neural Network (DBCNN).

[0011] The results showed that the quality scores predicted using these methods were not satisfactory. Summary of the Invention

[0012] The present invention aims to overcome at least some of the above-mentioned disadvantages.

[0013] As detailed below, this disclosure relates to learning-based objective image or video quality metrics. This disclosure allows for category selection based on features, and the application of different network architectures, models, or hyperparameter sets based on the selected category. Switching from one model to another is performed at the image level, block level, etc. The features can be read from the bitstream or metadata, derived from parsed syntax elements, or derived from pixels. The models can be further derived from or interpolated to each other to obtain finer granularity.

[0014] According to this disclosure, objective assessment can be performed immediately after encoding and before transmission, thereby affecting the encoder itself, or performed in the cloud or on the client, to obtain accurate information about the quality of the video service or game service received by the client.

[0015] As detailed below, this disclosure relates to (i) methods and systems for predicting objective quality scores for images or videos, and (ii) methods and systems for determining a set of parameters for a learning-based prediction method based on a set of images or videos.

[0016] The video used in this invention can be:

[0017] - Source video captured by a camera or source video generated by a computer; or

[0018] - A decoded video obtained by decoding the video stream, and the decoded video contains the decoded pixels of the video.

[0019] Therefore, according to a first aspect, the present invention also relates to a method (hereinafter referred to as the "prediction method") for predicting an objective quality score of an image or video, the method comprising:

[0020] - Obtain at least one feature associated with the image or video, the at least one feature being referred to as a selection feature;

[0021] - Select a parameter set from multiple parameter sets based on the at least one selection feature;

[0022] - An objective quality score of a video is determined by using a learning-based prediction method based on at least one feature, referred to as a qualifying feature, associated with the image or video, wherein the learning-based prediction method is configured by a selected set of parameters.

[0023] Relatedly, the present invention relates to a system for predicting objective quality scores of images or videos (hereinafter referred to as the "prediction system"), the system comprising:

[0024] - A module for obtaining at least one feature associated with the image or video, the at least one feature being referred to as a selection feature;

[0025] - A module for selecting a parameter set from multiple parameter sets based on the at least one selection feature; and

[0026] - A learning-based prediction method is configured to determine an objective quality score of the image or video based on at least one feature, referred to as a qualifying feature, associated with the image or video, wherein the learning-based prediction method is configured by a selected set of parameters.

[0027] When the video is obtained by decoding the video stream, the video-associated features can be syntax elements parsed directly from the video stream (i.e., read from the video stream). For example, the video-associated features can be profiles used by the encoder (random access / low latency / ...).

[0028] When the video is obtained by decoding the video bitstream, the video-associated features can be values ​​calculated based on syntax elements parsed from the bitstream. For example, an encoding mode (intra-frame encoding mode, inter-frame encoding mode, or skip encoding mode) is obtained for each block, and the features associated with the video bitstream are associated with a percentage of these encoding modes.

[0029] When the video is obtained by decoding the video stream, the features associated with the video can be elements obtained by decoding the video stream, such as pixels or motion vectors.

[0030] When the video is obtained by decoding the video stream, the features associated with the video can be values ​​calculated based on elements obtained by decoding the video stream. For example, the feature could be an indicator indicating whether a given image region is a contour region, and the contour is obtained by calculating decoded pixels. Alternatively, the feature could also be obtained based on a histogram calculated for the decoded pixels. For example, the feature could be global motion calculated for the image based on motion vectors.

[0031] When the video is a source video, the features associated with the video can be values ​​calculated based on elements of the source video, such as values ​​calculated based on pixels or motion vectors of the source video. For example, the feature could be an indicator indicating whether a given image region is a contour region, and the contour is calculated based on pixels of the source video. Alternatively, the feature could also be obtained from a histogram calculated for pixels of the source video. For example, the feature could be global motion calculated for the image based on motion vectors.

[0032] In embodiments of the prediction method, if the parameter set is not selected based on the at least one selection feature, a default parameter set is selected. This is particularly likely to occur in the following situations:

[0033] -If the prediction system fails to obtain the selected features; or

[0034] -If the prediction system fails to select the parameter set based on the selection features.

[0035] Furthermore, according to a second aspect, the present invention proposes a method (hereinafter referred to as the "training method") for determining multiple parameter sets of a learning-based prediction method based on a set of images or videos, the method comprising the following steps:

[0036] - Obtain at least one feature associated with each image or video, referred to as a selection feature; and

[0037] - Each image or video is categorized into a category selected based on the selection features, and a subset is determined for each category, wherein the subset includes only images or videos of that category.

[0038] For each subset of at least two said subsets, the method further includes the following stages:

[0039] - The learning-based prediction method is trained using only a subset of images or videos to learn the parameter set of the learning-based prediction method, thereby minimizing the error between the following two:

[0040] (i) the objective quality of the images or videos of the subset, wherein, when the parameter set is configured, the objective quality is calculated by the learning-based prediction method based on at least one feature, referred to as a qualifying feature, associated with these images or videos; and

[0041] (ii) The expected quality associated with the image or video.

[0042] Relatedly, the present invention relates to a system (hereinafter referred to as a "training system") for determining multiple parameter sets of a learning-based objective quality prediction method based on a set of images or videos, the system comprising:

[0043] - A module configured to obtain at least one feature associated with each image or video, the at least one feature associated with each image or video being referred to as a selection feature;

[0044] - Configured to categorize each image or video into a category selected based on the selection features, and to determine a subset for each category, wherein the subset includes only videos of that category; and

[0045] - A learning-based prediction method is configured to: learn a set of parameters for the learning-based prediction method when trained only from a subset of images or videos, wherein the set of parameters minimizes the error between the following two:

[0046] (i) the objective quality scores of the images or videos of the subset, wherein, when the parameter set is configured, the objective quality scores are calculated by the learning-based prediction method based on at least one feature, referred to as a qualifying feature, associated with these images or videos; and

[0047] (ii) The expected quality associated with the image or video.

[0048] To overcome the limitations of existing technologies, this disclosure proposes to combine machine learning techniques with traditional algorithmic methods.

[0049] While in the prior art, the model evolves and converges to a single model during the training step, this disclosure also applies to multiple models, but during prediction, the final model used at a specific granularity can evolve over time based on several existing or trained or derived models by automatically selecting from several models according to several features.

[0050] The method disclosed herein improves the quality of predictions, thereby improving the quality of the obtained quality scores. Another effect of this disclosure is reduced complexity. For example, in a cloud gaming environment, the following two assumptions can be made:

[0051] - A first category defined by a first feature set (e.g., a game similar to chess) can generate accurate quality scores using a simple learning-based prediction method, such as a simple linear regression model based on a single feature;

[0052] - The second category, defined by a second feature set (e.g., flight simulation), only produces accurate quality scores when using complex deep networks involving multiple layers and several features.

[0053] Choosing a model (or more generally, a set of parameters for a learning-based prediction method) based on the type of game (or more generally, based on features associated with an image or video) typically results in lower complexity.

[0054] The method of this invention provides better objective scores than existing learning-based methods, which learn a general model. This general model can provide scores for any input content. The general model is learned on a training set consisting of various content and incorporates all features of the content.

[0055] In embodiments of the training and prediction methods, either the selection feature or the qualifying feature is extracted from the video or from metadata associated with the video, or either the selection feature or the qualifying feature is calculated based on features extracted from the video.

[0056] In this embodiment, any one of the selection feature or qualifying feature is one of the following:

[0057] - Syntax elements, wherein the syntax elements are extracted from the video stream when the video is obtained by decoding the video stream; or

[0058] - When the video is a source video or a decoded video, the value is calculated based on the pixel values ​​of the image or video.

[0059] Recently, automatic quality assessment of videos has utilized new classes of learning-based algorithms, including machine learning, neural networks, convolutional neural networks, etc. The proposed disclosures cover all types of learning-based prediction methods: full-reference, no-reference, pixel-based, bitstream-based, etc.

[0060] In an embodiment, the learning-based prediction method includes a function and a parameter set. The parameter set includes the coefficients of the function. For example:

[0061] -The video is a video stream generated by an encoder;

[0062] - Learning-based prediction methods conform to the types recommended by ITU-T P.1203:

[0063] MOS q = q1 + q2 * exp(q3 * quant)

[0064] - The parameter set includes three coefficients q1, q2, and q3 of the learning-based prediction method; and

[0065] The term "quant" refers to the quantization step of the encoder, i.e., the qualified features of the video in this invention.

[0066] In another embodiment, the learning-based prediction method is a neural network, and the parameter set includes a model of the neural network. For example, the neural network may be of the random forest type or the support vector regression type.

[0067] In another embodiment, the learning-based prediction method implements at least a first neural network and a second neural network of different types, wherein:

[0068] A parameter set includes the model of the first neural network; and

[0069] Another set of parameters includes the model of the second neural network.

[0070] In another embodiment, the learning-based prediction method includes a neural network, the parameter set including parameters related to the architecture of the neural network.

[0071] In one embodiment, the training method includes sending information to a prediction system for deriving a new parameter set from at least one parameter set.

[0072] Accordingly, in this embodiment, the prediction method includes: receiving information for deriving the new parameter set from the at least one parameter set.

[0073] In an embodiment, the prediction method includes: deriving a new parameter set from at least one basic parameter set in a plurality of parameter sets, wherein:

[0074] For a given set of values ​​for the at least one selection feature, a new set of parameters should be selected to replace the at least one set of basic parameters; and

[0075] The parameters of the new parameter set are calculated based on the parameters of the at least one basic parameter set.

[0076] In an embodiment, the parameters of the new parameter set are weighted based on the distance between the at least one selection feature and the application constraints of the at least one basic parameter set.

[0077] The present invention also relates to a computer program including instructions configured to implement the steps of the above-described training method when the at least one computer program is executed by a computer.

[0078] The present invention also relates to a computer program including instructions configured to implement the steps of the above-described prediction method when the at least one computer program is executed by a computer.

[0079] These programs can use any programming language and can be in source code form, object code form, or intermediate code form between source code and object code, such as partially compiled form, or any other desired form.

[0080] The present invention also relates to a readable medium comprising at least one computer program from a collection of computer programs.

[0081] The recording medium can be any entity or device capable of storing programs. For example, the carrier device can include a storage device. The storage device is, for example, a ROM, and the carrier device is, for example, a CD-ROM or a microelectronic circuit ROM, or a magnetic recording device, such as a hard disk.

[0082] On the other hand, the recording medium can be a transmissible medium, such as electrical or optical signals, which can be carried by cable or optical fiber, radio, or other means. The program according to the invention can be downloaded, in particular, over an Internet-type network.

[0083] Optionally, the recording medium may be an integrated circuit with a program that is adapted to execute or be used to execute the disclosed method. Attached Figure Description

[0084] Other features and advantages of the invention will become apparent from the following description with reference to the accompanying drawings, which illustrate exemplary embodiments of the invention without any limiting features. In the drawings:

[0085] - Figure 1 This describes a prediction system according to an embodiment of the present invention;

[0086] - Figure 2 A flowchart illustrating the main steps of the prediction method according to an embodiment of the present invention;

[0087] - Figure 3 This describes a training system according to an embodiment of the present invention;

[0088] - Figure 4 A flowchart illustrating the main steps of a training method according to an embodiment of the present invention;

[0089] - Figure 5This describes a prediction system in a video game environment according to an embodiment of the present invention;

[0090] Figure 6 illustrates two methods for deriving parameter sets in a prediction system according to embodiments of the present invention;

[0091] - Figure 7 The hardware architecture of a prediction system according to an embodiment of the present invention is shown; and

[0092] - Figure 8 The hardware architecture of a training system according to an embodiment of the present invention is shown. Detailed Implementation

[0093] Figure 1 This refers to a prediction system (PS) according to an embodiment of the present invention.

[0094] In this disclosure, a reference VID may represent an image or a video.

[0095] The purpose of the prediction system PS is to determine the objective quality score VQSo of an image or video VID. The video VID can be:

[0096] - Source video SV captured by a camera or source video SV generated by a computer; or

[0097] - A decoded video DV containing decoded pixels of a video, wherein the pixels are obtained by decoding the video stream using a decoder DEC.

[0098] The image or video VID can be associated with the video-related metadata MTD.

[0099] The prediction system PS includes a learning-based prediction method (LBPM). The prediction method is configured to: predict based on at least one feature FEAT associated with the image or video VID. Q To determine the objective quality score VQSo of an image or video VID, wherein the feature FEAT Q These are referred to as qualified characteristics in the following text.

[0100] The learning-based prediction method LBPM can be derived from the parameter set MOD. i Configuration, wherein the parameter set MOD i It is a MOD of multiple parameter sets from the parameter database (DBP). i MOD j Selected from [the specified elements]. FEAT can be based on at least one feature associated with the VID of the image or video.s Determine the parameter set MOD i , wherein the feature FEAT s These are referred to as selection features in the following text.

[0101] The learning-based prediction method LBPM can be of different types.

[0102] In an embodiment, the learning-based prediction method LBPM may include a function and a parameter set. The parameter set MOD i LBPM, a learning-based prediction method used to configure the coefficients of the function.

[0103] For example, if the video VID is decoded based on the video bitstream generated by the encoder, the learning-based prediction method LBPM can conform to the type recommended by ITU-T P.1203:

[0104] MOS q = q1 + q2 * exp(q3 * quant)

[0105] Each parameter set MOD i This corresponds to a different set of (q1, q2, q3) parameters. The variable "quant" is the quantization step size of the encoder, i.e., the qualified feature FEAT associated with the video VID in this invention. Q .

[0106] In another embodiment, the learning-based prediction method LBPM used to predict objective quality is a neural network, such as a random forest type neural network or a support vector regression (SVR) type neural network, and different parameter sets MOD i Different models corresponding to neural networks.

[0107] In another embodiment, the learning-based prediction method LBPM implements different types of neural networks and different parameter sets MOD. i This can correspond to different models of neural networks. For example:

[0108] - The first parameter set MOD1 can be the first model (or kernel) of an SVR type network;

[0109] - The second parameter set MOD2 can be the second model (or kernel) of an SVR-type neural network; and

[0110] - The third parameter set MOD3 can be a set of coefficients or weights for a logistic regression model.

[0111] Therefore, according to this embodiment, the learning-based prediction method LBPM can be configured to implement different networks with different models or different coefficients.

[0112] In another embodiment, the learning-based prediction method LBPM can be a neural network, which can be configured according to different parameter sets MOD. i MOD j Configure the architecture of the neural network. For example, the number of layers may vary, and therefore the complexity of the network will also vary.

[0113] The prediction system PS includes at least one selected feature FEAT for obtaining the VID of an image or video. S The module; and at least one qualified feature FEAT for obtaining the VID of an image or video. Q The module.

[0114] In this embodiment, the same module ME10 of the prediction system PS is configured to: obtain at least one selection feature FEAT associated with the image or video VID during the same operation. S and at least one qualifying feature FEAT associated with the image or video VID Q Or, during different operations, obtain at least one selection feature FEAT associated with the image or video VID. S and at least one defining feature FEAT Q .

[0115] In an embodiment, at least one feature associated with an image or video VID obtained by module ME10 can be used for the following two purposes:

[0116] (i) FEAT as a selection feature S To select a parameter set to configure the learning-based prediction method LBPM for the image or video VID, and

[0117] (ii) FEAT as a qualifying characteristic Q The learning-based prediction method LBPM uses the qualified feature FEAT. Q To determine the objective quality score VQSo for the video image or VID.

[0118] The generic symbol FEAT used below is used to specify the characteristics of the video VID obtained by the ME10 module, and this characteristic can be used as the qualified characteristic FEAT of the video. Q and / or the video's selection feature FEAT S .

[0119] In this embodiment, the feature FEAT is the bit rate of the video VID.

[0120] In this embodiment, the feature FEAT is a syntax element parsed from the video stream VBS.

[0121] In an embodiment, the feature FEAT includes syntax elements parsed from the video bitstream VBS, for example:

[0122] - The type of codec used to generate the video stream VBS;

[0123] -Video stream VBS profiles (Baseline Profile (BP), Main Profile (MP), Extended Profile (XP), High-resolution HiP, High-resolution 10-bit Hi10P, High-resolution 4:2:2, High-resolution 4:4:4, etc.);

[0124] - The encoding structure of the video stream VBS (e.g., MPEG-4).

[0125] - Video stream encoding mode (intra-frame encoding, inter-frame encoding, skip encoding, merge encoding, etc.).

[0126] In an embodiment, the feature FEAT includes syntax elements corresponding to pixel residual information or motion vector residual information.

[0127] In an embodiment, the feature FEAT includes the pixel values ​​of the source video SV.

[0128] In an embodiment, the feature FEAT includes the value of the decoded pixel or the value of the decoded motion vector of the decoded video DV.

[0129] In an embodiment, the feature FEAT is metadata MTD associated with the video VID, such as the identifier of the video (e.g., the identifier of a video game), the identifier of the category of the video VID (e.g., the category of a video game), or any information not included in the bitstream VBS or decoded video DV.

[0130] In this embodiment, the feature FEAT is a metadata MTD associated with the video VID, which includes quality information obtained from another metric and calculated in a previous stage.

[0131] In an embodiment, the feature FEAT is calculated based on features extracted from the image or video VID.

[0132] For example, the feature FEAT is a percentage of the contour in an image or video, where the percentage is obtained by edge calculation based on source pixels or decoded pixels, for example, using a Canny filter for edge calculation.

[0133] For example, the feature FEAT can be obtained by calculating a histogram based on the decoded pixels, or by calculating a mixture between the number of intra-frame coding modes and the bit rate of the video VID.

[0134] The prediction system PS includes module M20. Module M20 is used to: select at least one FEAT feature of the video VID obtained by module ME10. S Select parameter set MOD from multiple parameter sets MODi and MODj. i .

[0135] When the parameter set MOD is configured i At that time, the learning-based prediction method LBPM can use the qualified features FEAT obtained for the image or video. Q To determine the objective quality score VQSo for an image or video VID.

[0136] Figure 2 This is a flowchart illustrating the main steps of the prediction method PM according to an embodiment of the present invention. This method can be performed to predict the objective quality score VQSo of an image or video VID.

[0137] The prediction method includes step E10: obtaining at least one selected feature FEAT of the image or video VID. S .

[0138] The prediction method includes step E20: based on the at least one selection feature FEAT S MOD from multiple parameter sets i MOD j Select parameter set MOD i .

[0139] The prediction method includes step E30: using a learning-based prediction method LBPM, based on at least one qualified feature FEAT associated with the image or video VID. Q To determine the objective quality score VQSo of an image or video VID, wherein the learning-based prediction method LBPM is derived from the parameter set MOD. i Configuration.

[0140] In an embodiment, the at least one qualifying feature FEAT used to determine the objective quality score VQSo is Q It can be used to select the parameter set MOD i The at least one selection feature FEAT S .

[0141] In another embodiment, the at least one qualifying feature FEAT used to determine the objective quality score VQSoQ Unlike the parameter set MOD used for selection i The at least one selection feature FEAT S .

[0142] exist Figure 2 In one embodiment, at least one selection feature FEAT associated with the image or video VID is obtained. S At that time, the at least one qualified feature FEAT can be obtained during step E10. Q Alternatively, the at least one qualifying feature FEAT Q and the at least one selection feature FEAT S It can be obtained in different steps.

[0143] exist Figure 2 In this embodiment, if the prediction system PS fails to select the parameter set in step E20, then the parameter database DB is used. P Default parameter set MOD D To configure the learning-based prediction method LBPM.

[0144] Figure 3 This refers to the training system TS according to an embodiment of the present invention.

[0145] The training system TS is designed to determine Figure 1 The prediction system uses multiple parameter sets MOD i MOD j .

[0146] The training system is used to determine different sets of parameters for a learning-based prediction method based on a set of images or videos. The training system performs the following phases: training the learning-based prediction method using only the subset of images or videos, thereby learning the parameter set of the learning-based prediction method. The subset includes only images or videos categorized into categories selected based on at least one selection feature.

[0147] exist Figure 3 In one embodiment, the training system TS includes a database DB. V The database DB V This includes a set of image or video VIDs, each of which is associated with a desired quality MOS.

[0148] The training system TS includes module MF10. Module MF10 is configured to obtain data from the database DB. V At least one selected feature FEAT associated with each image or video VID. S .

[0149] In this embodiment, the module MF10 is similar to the reference module. Figure 3 The prediction system PS described herein is module ME10.

[0150] In an embodiment, for a given image or video VID, at least one selection feature FEAT associated with the image or video is obtained by module ME10 of the prediction system PS. S FEAT, associated with the image or video, obtained by module MF10 of the training system TS. S They are the same.

[0151] In this embodiment, the training system TS includes a module MF20. The module MF20 is configured to: store the database DB... V Each image or video VID is categorized according to at least one selected feature FEAT. S Selected category C i The module MF20 constitutes multiple subsets SE. i , where each subset SE i Only include the same category C i Images or videos.

[0152] The training system TS includes a learning-based prediction method LBPM. When training the learning-based prediction method LBPM using an image or video set, the training system TS is configured to learn a parameter set for the learning-based prediction method LBPM, which minimizes the error between:

[0153] (i) the objective quality score VQSo of these images or videos, wherein the training system TS is based on at least one feature associated with these images or videos (i.e., qualifying feature FEAT). Q The objective quality score VQSo is calculated; and

[0154] (ii) Expected quality MOS associated with these images or videos.

[0155] The training system TS includes: a qualified feature FEAT configured to obtain the video VID. Q The module. In Figure 3 In one embodiment, the module MF10 of the training system PS is configured to obtain at least one selection feature FEAT of the video VID. S and at least one qualifying feature FEAT of the video VID Q .

[0156] In the embodiments, for category C iFor each subset of images or videos, the learning-based prediction method LBPM is trained independently, based on at least one selected feature FEAT obtained for each video. S To categorize these images or videos. For each subset SE i (Or, for each category C) i When using a subset SE i When training the learning-based prediction method LBPM, the learning parameter set MOD is used. i .

[0157] like Figure 3 As shown, for subset SE i For each image or video in the dataset, at least one qualifying feature of the image or video will be FEAT. Q The expected quality (MOS) is provided to the learning-based prediction method LBPM. Training the learning-based prediction method LBPM determines the parameter set MOD of the LBPM. i The parameter set MOD i Able to use the entire subset SE i To minimize the error between the objective quality score VQSo of these images or videos and the expected quality MOS associated with these images or videos, wherein, based on at least one feature (i.e., the qualifying feature FEAT) Q The objective quality score VQSo is calculated using this method.

[0158] For each subset SE i All obtained parameter set MOD i .

[0159] These parameter sets MOD i The parameters can be stored in the prediction system's parameter database DB. P middle.

[0160] Figure 4 A flowchart illustrating the main steps of the training method TM according to an embodiment of the present invention.

[0161] The training method TM includes step F10: obtaining at least one selected feature FEAT associated with each image or video VID based on the image or video set. S .

[0162] The training method TM includes step F20: classifying each image or video into category C. i In, wherein, based on at least one selection feature FEAT obtained for the image or video. S To select the category C i The original set of images or videos can be divided into multiple subsets, each subset SEi Only category C is included. i Images or videos.

[0163] For each subset SE in the multiple subsets i The training method TM includes the following phase F40: training only through a subset SE i The image or video is used to train the learning-based prediction method LBPM to learn the parameter set MOD of the learning-based prediction method LBPM. i This minimizes the error between the following two:

[0164] (i) The objective quality score VQSo of the images or videos of the subset, wherein, when the parameter set is configured, the objective quality score VQSo is determined by the learning-based prediction method LBPM based on at least one qualified feature FEAT associated with these images or videos. Q The objective quality score VQSo was calculated; and

[0165] (ii) Expected quality MOS associated with the image or video.

[0166] The following are three non-limiting examples of embodiments of the present invention.

[0167] First example of an embodiment of the present invention

[0168] This invention can be applied in gaming environments.

[0169] Figure 5 The image shows a cloud-gaming service provider (CGSP) and a mobile phone (MP).

[0170] The mobile phone MP is a system for predicting an objective score VQSo according to an embodiment of the present invention. The mobile phone MP implements the prediction method PM according to an embodiment of the present invention.

[0171] In this scenario, players use their mobile phones (MPs) to play games via cloud gaming services provided by cloud gaming service provider CGS. The mobile phone receives a High Efficiency Video Coding (HEVC) bitstream (VBS) and an additional stream (MTD) containing metadata about the game's game category (GC).

[0172] The mobile phone MP includes an HEVC decoder to obtain the decoded video VID. The HEVC decoder includes an HEVC parser (HEVCP).

[0173] The mobile phone MP also includes a metadata parser (MTDP).

[0174] The HEVC parser HEVCP provides the number of bits (nbpf) used for each frame of the video VID.

[0175] The metadata parser MTDP obtains the game's category (gc) from the metadata.

[0176] In this invention, the game category (gc) and the number of bits used by the frame (nbpf) are two selection features of the video VID (FeAT). S And these are also two qualifying features of video VID: FEAT Q .

[0177] In this invention, the combination of these parsers constitutes module M10, which is used to obtain at least one selection feature FEAT associated with the video VID. S and at least one qualifying feature FEAT associated with the video. Q .

[0178] The mobile phone MP includes a module ME20. For each video frame, the module ME20 is used to select a parameter set MOD from four parameter sets MOD1-MOD4 based on two selection features gc and nbpf. i The four parameter sets MOD1-MOD4 are stored in the parameter database BD. P middle.

[0179] When a parameter set MOD is configured i At that time, the bit-stream based learning-based prediction method (LBPM) of the mobile phone MP uses gc and nbpf features as two qualified features of the video frame FEAT. Q To provide the objective quality score VQSo for the frame.

[0180] In this embodiment, the mobile phone MP sends the objective quality score (VQSo) for each frame back to the cloud gaming service provider CGSP for further analysis.

[0181] In a first example of an embodiment of the present invention, it is assumed that the cloud gaming service provider CGSP has implemented the training method according to the embodiment of the present invention, thereby designing a bitstream-based learning-based prediction method LBPM and four parameter sets MOD1-MOD4.

[0182] To this end, videos of different games have been encoded at different bit rates to form a first video set, and each video has been associated with subjective quality, or more precisely, with the average opinion score (expected quality in this invention).

[0183] Based on two selection features corresponding to the game category (GC) and video bitrate, respectively, FEAT S Each video has been categorized into category C, chosen from four categories C1-C4. i In the middle. Four subsets of the video have been obtained, of which each subset SE i Only the given category C is included. i The video.

[0184] For each of the four subsets SE1-SE4, SE i The subset SE has already been used. i The expected quality (MOS) associated with the video is used to train the bitstream-based learning-based prediction method LBPM, thereby learning the parameter set MOD. i The parameter set MOD i This minimizes the error between the expected quality MOS and the objective quality score of these videos, wherein the bitstream-based learning-based prediction method LBPM is used to predict the video's FEAT score based on at least one qualifying feature. Q To calculate the objective quality score for these videos.

[0185] For example, the four parameter sets used in the training method and the prediction method in the first embodiment can be:

[0186] -MOD1: Parameter set for LBPM, a learning-based prediction method based on bitstream, for the game category GC "Shooter and Simulation" and bitrates below 10Mbps (nbpf);

[0187] -MOD2: A parameter set for LBPM, a learning-based prediction method based on bitstream, for the game category GC "Shooter and Simulation" and bitrates above 10Mbps (nbpf).

[0188] -MOD3: A parameter set for LBPM, a learning-based prediction method based on bitstream, for game categories other than "shooter and simulation" (GC) and bitrates below 15Mbps (NbpF).

[0189] –MOD4: Parameter set for LBPM, a learning-based prediction method based on bitstream. Used for game categories other than "Shooter and Simulation" (GC) and bitrates above 15 Mbps (Nbpf).

[0190] Second example of the embodiments of the present invention

[0191] This invention can be used in 2D video environments.

[0192] In another embodiment of the invention, a user is watching a basketball game on an ultra-high definition television (UHD TV).

[0193] Typically, sports images consist of natural content captured by a 2D camera and overlays such as text or graphics (so-called screen content).

[0194] The television receives a multi-functional video coding (VVC) bitstream (VBS) containing 2D UHD content, along with an additional stream of metadata (MTD) of each block of video content (natural content or screen content).

[0195] In this embodiment, the TV has an embedded VVC parser and a VVC decoder. The VVC decoder provides decoded pixels corresponding to each image.

[0196] The MTD parser provides a selection feature FEAT for each block of the decoded frame. S The selection feature FEAT S Indicates whether the content is natural content or screen content.

[0197] The television includes features for FEAT based on content category. S Select one of the following modules:

[0198] - The first SVR model MOD1 for the Natural Content category; or

[0199] - The second SVR model MOD2 for screen content categories.

[0200] An objective quality score for the block is calculated using a learning-based prediction method (LBPM) of type SVR. The SVR type is configured to evaluate the block based on at least one qualifying feature (FEAT). Q (e.g., the pixels of a block) to select the first SVR model or the second SVR model.

[0201] In this embodiment, the television sends back the objective quality score (VQSo) for each frame to the television broadcast for further analysis. The objective quality score (VQSo) for each frame is obtained by averaging the scores calculated for each block of the frame.

[0202] In the second example, the standardization committee could design a learning-based prediction method of the SVR type to assess the quality of customers’ perception of 2D content.

[0203] The committee used a collection of UHD videos encoded at different bit rates. Based on experiments, the committee decided to define two categories, CE1 and CE2, according to the nature of the content.

[0204] Based on the features read from the block-level metadata, create two subsets, SE1 and SE2, each containing blocks of each type.

[0205] During training, the SVR-based prediction method independently learns its model on each subset of SE1 and SE2 to create two distinct models, MOD1 and MOD2. This learning can proceed due to the expected quality corresponding to the objective scores. These objective scores are provided by a complex, complete reference metric applied to different blocks.

[0206] Third example of the embodiments of the present invention

[0207] This invention can be used in immersive video environments.

[0208] In another embodiment of the present invention, a user watches immersive video on a head-mounted display (HMD).

[0209] The HMD receives HEVC bitstream VBS containing immersive content.

[0210] The HMD is connected to a personal computer (PC) that embeds a learning-based prediction method (LBPM) to analyze the content the user is viewing. In this embodiment, the LBPM can be configured to implement a deep neural network with 50 layers, a deep neural network with 100 layers, or an SVR model.

[0211] The PC includes an HEVC decoder and an HEVC parser.

[0212] The HEVC decoder provides decoded pixels corresponding to each image.

[0213] The HEVC parser obtains the selection feature FEAT. S The selection feature FEAT S Indicate whether the content is 360° panoramic, 180° panoramic, or perspective.

[0214] In this example, the PC includes module ME20. Module ME20 is used to select feature FEAT. S Select parameter set MOD from the following parameter set. i :

[0215] MOD1: The parameter set for a 50-layer deep neural network used for 360-degree panoramic content categories;

[0216] MOD2: The parameter set for a 100-layer deep neural network used for 180° panoramic content categories;

[0217] MOD3: The parameter set for SVR models in the perspective category.

[0218] Processing and displaying the 360-degree panoramic content is both complex and time-consuming. In this embodiment, as a trade-off, a deep learning method with a reasonable number of layers is used to maintain real-time capabilities.

[0219] Processing the 180° panoramic content is complex but relatively time-efficient. Deep learning methods with more layers can be used to process the 180° panoramic content.

[0220] The perspective content is more complex to process because it corresponds to multiple 2D camera captures. In this embodiment, a simple SVR is used.

[0221] The learning-based prediction method LBPM is used to predict based on at least one qualified feature FEAT associated with the video VID. Q To calculate an objective quality score, the objective quality score is sent back to the service provider for further analysis.

[0222] In this third example, immersive video service providers can design learning-based prediction methods to assess the quality of customers' perception of immersive video content. Collections of immersive videos encoded at different bitrates can be used.

[0223] In this example, based on some experiments, three categories are defined according to the properties of the content. These three categories allow for a good trade-off between the time required to process the content and the time allocated to computation for learning-based prediction methods.

[0224] Based on the selection features (the nature of the content) read from the bitstream, the video collection is divided into three subsets, each containing only one type of video.

[0225] For each category, the following different algorithms are used:

[0226] - A deep neural network with 50 layers for 360-degree panoramic content categories;

[0227] - A deep neural network with 100 layers for 180° panoramic content categories;

[0228] - SVR models for the perspective category.

[0229] Each algorithm learns its model independently on each subset to create different MOD models. i The MOD i This is the output of the training process. Due to the qualified features of the video, FEAT... Q And the mean opinion scores (MOS) obtained from subjective tests associated with these videos, thus enabling this learning to take place.

[0230] In each of the embodiments described above, multiple parameter sets MOD are considered. i and MOD j And each parameter set corresponds to the FEAT selected based on the chosen feature. S The given category obtained.

[0231] The following will describe how, according to some specific embodiments of the present invention, MOD can be derived from an existing model. i and MOD j The additional intermediate parameter set MOD is derived or interpolated from the middle. k .

[0232] A key aspect of this interpolation method is simulating a larger set of parameters (MOD). k This results in a better VQSo score due to the finer granularity of the model.

[0233] In both methods shown with reference to Figure 6, the derivation occurs during the prediction phase.

[0234] Assuming there are two parameter sets MOD1 and MOD2, when the selection feature FEAT S When the value is within the first range [min, F], the first parameter set MOD1 is selected, and the selection feature FEAT is selected. S When the value is within the second range [F,max], select the second parameter set MOD2.

[0235] It should be noted that, for the general symbol p i，k Given: k = 1, ..., N, and N is the parameter set MOD i The number of parameters in the middle.

[0236] Figure 6A This indicates that the selection feature FEAT is based on the aforementioned selection feature. S The value of parameter p selected from the parameter set s,k The value of p, where p s,k The value is determined as follows:

[0237] -If FEAT S Located within [min, F]: p s,k =p 1,k ;

[0238] -If FEAT S Located within [F,max]: p s,k =p 2,k .

[0239] Assuming that the selection feature FEAT is taken into account in the prediction system or the prediction method S Located within the range [B1, B2], and taking into account when selecting feature FEAT S When the parameter sets MOD1 and MOD2 are within the range [B1, B2], they are not precise. Therefore, the learning-based prediction method LBPM should apply the parameter set MOD derived from the first parameter set MOD1 and the second parameter set MOD2. k .

[0240] exist Figure 6B In the example, the exported parameter set MOD k The parameter p in r,k It is the average of the parameters in the first parameter set and the parameters in the second parameter set. Therefore, according to the selection feature FEAT S The value of the parameter p of the selected model s,k The value is determined as follows:

[0241] -If FEAT S Located within [min, B1[: p s,k =p 1,k ;

[0242] -If FEAT S Located within [B1, B2]: p s,k =p r,k =(p 1,k +p 2,k ) / 2;

[0243] -If FEAT S Located within [B2,max]: p s,k =p 2,k .

[0244] exist Figure 6C In the example, based on the selection feature FEAT S The application of the first parameter set MOD1 and the second parameter set MOD2 restricts the distance between B1 and B2, and the resulting parameter set MOD... k The parameter p r,k Weighting is applied. Therefore, based on the selection feature FEAT... S The value of the selected model parameter p s,k The value is determined as follows:

[0245] -If FEAT S Located within [min, B1[: p s,k =p 1,k ;

[0246] -If FEAT S Located within [B1, B2]: p s,k =p r,k =(p 1,k (B2-FEAT S )+p 2,k (FEAT S -B1)) / (B2-B1);

[0247] -If FEAT S Located within [B2,max]: p s,k =p 2,k .

[0248] In another embodiment, the new parameter set MOD is performed by the training system TS. k The derivation shows that the training system provides the prediction system PS with methods on how to compute the new parameter set.

[0249] As mentioned earlier, during step F20, which classifies each video VID into a category, the number of categories NC is considered, and a learning-based prediction method LBPM is trained independently from the videos of the NC subset of the videos, thereby learning the NC parameter set.

[0250] In this embodiment, a relatively large number of categories NC is selected, but not all parameter sets learned for the NC subset are provided to the prediction system.

[0251] Conversely, in this embodiment, the training system TS performs the following steps:

[0252] (i) Calculate a function to derive at least one NC parameter set (referred to as the "new parameter set") from at least one basic parameter set in the NC parameter set, and

[0253] (ii) Provide the prediction system PS with at least one basic parameter set and the function, such that the prediction system can use the function to derive a new parameter set from the basic parameter set.

[0254] For example, return to reference Figure 6C For example, the training system can determine the derivation function p. r,k =(p 1,k (B2-FEAT S )+p 2,k (FEAT SThe constraints B1 and B2 used in -B1)) / (B2-B1) are provided to the prediction system.

[0255] Figure 7 The hardware architecture of a prediction system PS according to an embodiment of the present invention is shown. The prediction system PS can be in the form of a PC, or, for example, a smartphone.

[0256] The prediction system PS specifically includes processor 1 P Random Access Memory 3 P Read-only memory 2 P Non-volatile flash memory 4 P , and a communication device not shown.

[0257] The read-only memory 2 P The recording medium constitutes the present invention and can be processed by processor 1 P Reading. Recording the computer program PG according to the invention on the recording medium. P .

[0258] Computer program PG P The functional (software) modules of the prediction system PS are defined.

[0259] In this embodiment, these functional modules include:

[0260] - Module ME10, used to obtain at least one selectable feature FEAT of the video VID. S and at least one qualifying feature FEAT Q ;

[0261] - Module ME20, for using the at least one selection feature FEAT S To select multiple parameter sets MOD i and MOD j Parameter set MOD i ;as well as

[0262] - The learning-based prediction method LBPM is configured to: predict based on the at least one qualifying feature FEAT Q To determine the objective quality score VQSo of the video VID, wherein the learning-based prediction method LBPM is derived from the parameter set MOD. i Configuration.

[0263] In particular, these functional modules may also include video decoders.

[0264] Figure 8 The hardware architecture of a training system TS according to an embodiment of the present invention is shown. For example, the training system TS may be in the form of a server.

[0265] Specifically, the prediction system TS includes a processor 1 T Random Access Memory 3 T Read-only memory 2 T Non-volatile flash memory 4 T And a communication device not shown.

[0266] The read-only memory 2 T The recording medium constitutes the present invention and can be processed by processor 1 T Read. The computer program PG according to the invention is recorded on the recording medium. T .

[0267] The computer program PG T Define the functional (software) modules of the training system TS.

[0268] In this embodiment, these functional modules include:

[0269] - Module MF10, used to obtain at least one selected feature FEAT of the video VID. S and at least one qualifying feature FEAT Q ;

[0270] Module MF20 is configured to classify video VIDs according to at least one selection feature FEAT. S Selected category C i In, and for each of the categories C i Determine the subset SE i , wherein the subset SE i Only category C is included. i Videos; and

[0271] - The learning-based prediction method LBPM is configured to: when only one subset SE is used for prediction... i During video training, the parameter set MOD of the learning-based prediction method LBPM is learned. i The parameter set MOD i Minimize the error between the following two:

[0272] (i) The objective quality score VQSo of the videos of the subset SEi, wherein the parameter set MOD is configured. i At that time, the learning-based prediction method LBPM is used to predict the video based on at least one qualifying feature FEAT. Q The objective quality score VQSo was calculated; and

[0273] (ii) Expected quality MOS associated with the video.

Claims

1. A method for predicting the objective quality score (VQSo) of an image or video (VID), characterized in that, The method includes: - Obtain at least one feature associated with the image or video (FEAT) S At least one of these features is called a selection feature (FEAT). S ); -Based on at least one of the selection features (FEAT) S ), from multiple parameter sets (MOD i MOD j Select parameter set (MOD) i );as well as - By using a learning-based prediction method (LBPM), based on at least one feature (FEAT) associated with the image or video (VID), known as a qualifying feature. Q The learning-based prediction method (LBPM) determines the objective quality score (VQSo) of the image or video (VID) using the parameter set (MOD). i Configuration; -From the multiple parameter sets (MOD) i MOD j At least one basic parameter set (MOD) in ) i MOD j ), export new parameter set (MOD) k ); Among them, for the at least one selection feature (FEAT) S Given several determined values, select the new parameter set (MOD). k ) to replace the at least one basic parameter set (MOD) i MOD j The new parameter set (MOD) k ) parameters (p r, k The selection is determined in the following manner: based on at least one selection feature (FEAT). S ) and the multiple parameter sets (MOD) i MOD j At least one basic parameter set (MOD) in ) i MOD j The distance between the upper and lower limits of the application range (B1, B2) for the at least one basic parameter set (MOD) i MOD j The parameter (p) in ) 1, k , p 2, k The new parameter set (MOD) is obtained by weighting the parameters. k ) parameters (p r, k ).

2. The method according to claim 1, characterized in that, include: Receive information, the information being used to receive information from the at least one parameter set (MOD) i MOD j Export the new parameter set (MOD) k ).

3. The method according to claim 1, characterized in that, If based on at least one of the selection features (FEAT) S If no parameter set can be selected, the default parameter set (MOD) will be selected. D ).

4. A multiple parameter set (MOD) for determining a learning-based prediction method based on an image or video (VID) set. i MOD j The method (TM) is characterized by, Includes the following steps: - Obtain at least one feature (FEAT) associated with each of the images or videos (VIDs). S At least one of these features is called a selection feature (FEAT). S );as well as - Classify each of the images or videos (VIDs) according to at least one of the selected features (FEAT). S Among the selected categories, and for each of the categories (C) i Determine the subset (SE) i The subset (SE) i Only the category (C) is included. i Images or videos; For at least two subsets (SE) i SE j Each subset (SE) in ) i The method further includes the following stages: - Only through one of the aforementioned subsets (SE) i The learning-based prediction method (LBPM) is trained using images or videos to learn the parameter set (MOD) of the learning-based prediction method (LBPM). i ), thereby minimizing the error between the following two: (i) The objective quality score (VQSo) of the images or videos of the subset, wherein, when the parameter set (MOD) is configured... i When the learning-based prediction method (LBPM) is used, based on the qualified features (FEAT) associated with the image or video, the prediction is performed. Q At least one feature of FEAT Q The objective quality score (VQSo) is calculated; and (ii) Desired quality (MOS) associated with the image or video; - Send information to the system that predicts the objective quality score (VQSo) of an image or video (VID), the information being used to derive from the plurality of parameter sets (MOD). i MOD j At least one basic parameter set (MOD) in ) i MOD j Export a new parameter set (MOD) k The information, wherein, for the at least one selection feature (FEAT) S Given several determined values, select the new parameter set (MOD). k ) to replace the at least one basic parameter set (MOD) i MOD j The new parameter set (MOD) k ) parameters (p r, k The selection is determined in the following manner: based on at least one selection feature (FEAT). S ) and the multiple parameter sets (MOD) i MOD j At least one basic parameter set (MOD) in ) i MOD j The distance between the upper and lower limits of the application range (B1, B2) for the at least one basic parameter set (MOD) i MOD j The parameter (p) in ) 1, k , p 2, k The new parameter set (MOD) is obtained by weighting the parameters. k ) parameters (p r, k ).

5. The method according to claim 4, characterized in that, The video (VID) is either the source video (SV) or the decoded video (DV) obtained by decoding the video stream (VBS).

6. The method according to claim 4, characterized in that, Extract the selection feature (FEAT) from the image or video (VID) or from the metadata (MTD) associated with the image or video. S ) or the qualified feature (FEAT) Q Either one of the following; or, the selection feature (FEAT) is calculated based on features extracted from the image or video. S ) or the qualified feature (FEAT) Q Any one of them.

7. The method according to claim 4, characterized in that, The selection feature (FEAT) S ) or the qualified feature (FEAT) Q Any one of them can be: - Syntax elements, wherein, when the video is obtained by decoding the video stream (VBS), the syntax elements are directly parsed from the video stream (VBS); or - Values ​​calculated based on the syntax elements parsed from the video stream (VBS); or - Elements obtained by decoding the video stream (VBS); or - The value calculated based on the elements obtained from decoding the video stream (VBS); or - When the video is source video (SV) or decoded video (DV), the value is calculated based on the pixel values ​​of the image or the video (VID).

8. The method according to claim 4, characterized in that, The learning-based prediction method (LBPM) includes a function, and the parameter set (MOD) i This includes the coefficients of the function.

9. The method according to claim 4, characterized in that, The learning-based prediction method (LBPM) includes a neural network, and the parameter set (MOD) i (This includes the model of the neural network.) 10. The method according to claim 4, characterized in that, The learning-based prediction method (LBPM) implements at least different types of first and second neural networks, wherein: - One of the parameter sets (MOD) i ) including the model of the first neural network; and -Another set of parameters (MOD) j (This includes the model of the second neural network.) 11. The method according to claim 4, characterized in that, The learning-based prediction method (LBPM) includes a neural network, and the parameter set (MOD) i This includes parameters related to the architecture of the neural network.

12. A system for predicting the objective quality score (VQSo) of an image or video (VID), characterized in that, include: - Module (ME10) for obtaining at least one feature (FEAT) associated with the image or video (VID). S At least one of these features is called a selection feature (FEAT). S ); - Module (ME20) for using the at least one selection feature (FEAT) S From multiple parameter sets (MOD) i MOD j Select parameter set (MOD) i ); - A learning-based prediction method (LBPM) is configured to: predict based on at least one feature (FEAT) associated with the image or video (VID). Q The learning-based prediction method (LBPM) determines the objective quality score (VQSo) of the image or video (VID) using the parameter set (MOD). i Configuration; - The module (ME20) is also used to obtain data from the plurality of parameter sets (MOD). i MOD j At least one basic parameter set (MOD) in ) i MOD j ), export new parameter set (MOD) k ), wherein, for the at least one selection feature (FEAT) S Given several determined values, select the new parameter set (MOD). k ) to replace the at least one basic parameter set (MOD) i MOD j The new parameter set (MOD) k ) parameters (p r, k The selection is determined in the following manner: based on at least one selection feature (FEAT). S ) and the multiple parameter sets (MOD) i MOD j At least one basic parameter set (MOD) in ) i MOD j The distance between the upper and lower limits of the application range (B1, B2) for the at least one basic parameter set (MOD) i MOD j The parameter (p) in ) 1, k , p 2, k The new parameter set (MOD) is obtained by weighting the parameters. k ) parameters (p r, k ).

13. A multiple parameter set (MOD) for determining a learning-based prediction method based on a set of images or videos (VIDs). i MOD j The system, characterized in that, include: - Module (MF10) is configured to obtain at least one feature (FEAT) associated with each of the images or videos (VIDs). S At least one of these features is called a selection feature (FEAT). S ); - Module (MF20) is configured to classify each of the images or videos (VIDs) into a category selected based on the selection features, and for each of the categories (C... i Determine the subset (SE) i The subset (SE) i Only the category (C) is included. i Images or videos; - A learning-based prediction method (LBPM) is configured to: learn a parameter set (MODi) of the learning-based prediction method (LBPM) when the LBPM is trained only on a subset (SEi) of images or videos, thereby minimizing the error between the following two: (i) The objective quality score (VQSo) of the images or videos of the subset (SEi), wherein, when the parameter set (MODi) is configured, the objective quality score is determined by the learning-based prediction method (LBPM) based on at least one feature (FEAT) associated with the image or video, referred to as a qualified feature. Q The objective quality score (VQSo) of the images or videos in the subset (SEi) is obtained by calculating the features of the subset; and (ii) Desired quality (MOS) associated with the image or video; The system is also configured to send information to a system predicting the objective quality score (VQSo) of an image or video (VID), the information being used to select from the plurality of parameter sets (MODs). i MOD j At least one basic parameter set (MOD) in ) i MOD j Export a new parameter set (MOD) k The information, wherein, for the at least one selection feature (FEAT) S Given several determined values, select the new parameter set (MOD). k ) to replace the at least one basic parameter set (MOD) i MOD j The new parameter set (MOD) k ) parameters (p r, k The selection is determined in the following manner: based on at least one selection feature (FEAT). S ) and the multiple parameter sets (MOD) i MOD j At least one basic parameter set (MOD) in ) i MOD j The distance between the upper and lower limits of the application range (B1, B2) for the at least one basic parameter set (MOD) i MOD j The parameter (p) in ) 1, k , p 2, k The new parameter set (MOD) is obtained by weighting the parameters. k ) parameters (p r, k ).

14. A computer-readable storage medium, characterized in that, It includes computer program instructions that, when run on a computer, cause the computer to perform the method described in any one of claims 1-11.

15. A computer program product, the computer program product comprising instructions, characterized in that, When the instructions are executed by a computer, they implement the method described in any one of claims 1-3.

16. A computer program product, the computer program product comprising instructions, characterized in that, When the instructions are executed by a computer, they implement the method described in any one of claims 4-11.

Images