Method for evaluating image video quality based on approximation value and related device

Abstract

The application provides a method for evaluating image / video quality based on an approximation value and a related device, and relates to the technical field of computer vision (image) or machine learning of artificial intelligence, and the method comprises the following steps: obtaining a sample to be evaluated; based on the parameters of the sample to be evaluated, a first model on a line is used to calculate a first approximation value approximating a subjective true value of the sample to be evaluated; the first model is a model obtained based on an offline second model, the second model is a model obtained by taking k samples and the subjective true values of the k samples as a training set, the first model is a model obtained by fitting the parameters of the k samples by taking the approximation values of the k samples obtained by using the second model as a reference, and k>0; and the image / video quality is evaluated based on the first approximation value. The image / video quality can be evaluated based on the approximation value approximating the subjective true value in real time without increasing the hardware cost of the server side and ensuring the evaluation accuracy.

Description

Technical Field

[0001] This application relates to the technical fields of artificial intelligence, such as computer vision (image) or machine learning, and more specifically, to methods and related apparatus for evaluating image and video quality based on approximation values. Background Technology

[0002] The quality of images / videos can generally be calculated using algorithmic models to determine the quality indicators of videos / images.

[0003] Typically, subjective scoring models and calculated values ​​can reflect the objective and subjective quality of a video, but their computational complexity and accuracy are not ideal. For example, one could use encoder quantization parameters (QP), peak signal-to-noise ratio (PSNR), and structural similarity (SSIM) to infer the objective quality score of a game video. While this method is not highly complex, its correlation with subjective quality evaluation is low, which has limitations, and it also introduces some latency. Another approach is to use models such as Video Multimethod Assessment Fusion (VAMF) to obtain the objective quality score of a game video. This method is more accurate than PSNR, but its computational complexity is much higher, making real-time calculation impossible for high frame rate, high-resolution videos.

[0004] In addition, there are many types of cloud games, and the massive amount of video data requires real-time acquisition of subjective scores for the videos, but there are no models or solutions in the relevant technologies for real-time feedback of subjective quality.

[0005] Therefore, there is an urgent need to provide a method that can subjectively assess image and video quality based on real-time feedback. Summary of the Invention

[0006] This application provides a method and related apparatus for evaluating image and video quality based on approximation values, which can evaluate image and video quality based on approximation values ​​that approximate the subjective truth in real time without increasing server-side hardware costs or ensuring evaluation accuracy.

[0007] On the one hand, this application provides a method for evaluating image and video quality based on approximation values, including:

[0008] Obtain the sample to be evaluated, which includes the video or image to be evaluated.

[0009] Based on the parameters of the sample to be evaluated, the first approximation value that approximates the subjective truth value of the sample to be evaluated is calculated using the first online model.

[0010] The first model is a model obtained based on the offline second model. The second model is a model obtained by using k samples and the subjective truth values ​​of the k samples as a training set. The subjective truth values ​​of the k samples are obtained by subjective scoring. The first model is a model obtained by fitting the parameters of the k samples using the approximation values ​​of the k samples obtained by the second model as a reference, where k > 0.

[0011] The quality of the video or image to be evaluated is assessed based on this first approximation value.

[0012] On the other hand, this application provides an apparatus for evaluating image and video quality based on approximation values, comprising:

[0013] The acquisition unit is used to acquire the sample to be evaluated, which includes the video or image to be evaluated.

[0014] The calculation unit is used to calculate a first approximation value that approximates the subjective truth value of the sample to be evaluated based on the parameters of the sample to be evaluated using the first online model.

[0015] The first model is a model obtained based on the offline second model. The second model is a model obtained by using k samples and the subjective truth values ​​of the k samples as a training set. The subjective truth values ​​of the k samples are obtained by subjective scoring. The first model is a model obtained by fitting the parameters of the k samples using the approximation values ​​of the k samples obtained by the second model as a reference, where k > 0.

[0016] An evaluation unit is used to evaluate the quality of the video or image to be evaluated based on the first approximation value.

[0017] On the other hand, this application provides a method for training a first model, including:

[0018] Obtain k samples, and the subjective truth value of these k samples is obtained by subjective scoring;

[0019] Using the k samples and their subjective truth values ​​as the training set, a second model is obtained;

[0020] Using these k samples as input, the second model is used to obtain approximate values ​​of the subjective truth for these k samples;

[0021] Using the approximation values ​​of these k samples as a reference, the parameters of these k samples are fitted to obtain the first model.

[0022] On the other hand, this application provides a training apparatus for a first model, comprising:

[0023] The first acquisition unit is used to acquire k samples, and the subjective truth values ​​of the k samples are obtained by subjective scoring.

[0024] The first training unit is used to obtain the second model by using the k samples and the subjective truth values ​​of the k samples as the training set;

[0025] The second acquisition unit is used to take the k samples as input and use the second model to obtain the approximation value of the subjective truth of the k samples.

[0026] The second training unit is used to fit the parameters of the k samples to obtain the first model by using the approximation values ​​of the k samples as a reference.

[0027] On the other hand, this application provides an electronic device, including:

[0028] Processor, adapted to implement computer instructions; and,

[0029] A computer-readable storage medium storing computer instructions adapted for loading by a processor and executing the above-described method for evaluating image / video quality based on approximation values ​​or the training method of the first model described above.

[0030] On the other hand, embodiments of this application provide a computer-readable storage medium storing computer instructions. When these computer instructions are read and executed by a processor of a computer device, the computer device performs the aforementioned method for evaluating image and video quality based on approximation values ​​or the aforementioned training method for the first model.

[0031] On the other hand, embodiments of this application provide a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the aforementioned method for evaluating image / video quality based on approximation values ​​or the aforementioned training method for the first model.

[0032] In this embodiment, based on the parameters of the sample to be evaluated, a first approximation value that approximates the subjective truth value of the sample to be evaluated is calculated using an online first model; then, the quality of the video or image to be evaluated is assessed based on the first approximation value. On one hand, the online first model can calculate and feedback the first approximation value in real time, and thus, the image and video quality can be assessed based on the real-time feedback approximation value that approximates the subjective truth value. On the other hand, the first model is constructed as a model obtained based on an offline second model, and the second model is constructed as a model obtained using k samples and the subjective truth values ​​of the k samples as a training set. The subjective truth values ​​of the k samples are obtained by subjective scoring. The first model is a model obtained by fitting the parameters of the k samples using the approximation values ​​obtained by the second model as a reference, where k > 0; equivalently, the first model is obtained by fitting the trained second model, which can ensure the evaluation accuracy of the first model without increasing the server-side hardware cost.

[0033] In summary, the method provided in this application can evaluate image and video quality based on approximation values ​​that approximate the subjective truth based on real-time feedback without increasing server-side hardware costs or ensuring evaluation accuracy. Attached Figure Description

[0034] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a schematic diagram of the interface of the subjective scoring platform provided in the embodiments of this application.

[0036] Figure 2 This is a schematic flowchart illustrating the method for training a quality evaluation model based on randomly selected samples, as provided in the embodiments of this application.

[0037] Figure 3 This is a schematic flowchart illustrating the method for evaluating image and video quality based on approximation values ​​provided in the embodiments of this application.

[0038] Figure 4 This is a schematic block diagram illustrating the working principle of the first model provided in the embodiments of this application.

[0039] Figure 5 This is a schematic block diagram illustrating the training and evaluation principles of the first model provided in the embodiments of this application.

[0040] Figure 6 This is a schematic block diagram illustrating the optimization principle of the first model provided in the embodiments of this application.

[0041] Figure 7 This is a schematic block diagram of a service system including a first model provided in an embodiment of this application.

[0042] Figure 8 This is a schematic flowchart of the training method for the first model provided in the embodiments of this application.

[0043] Figure 9 This is a schematic block diagram of an apparatus for evaluating image and video quality based on approximation values, provided in an embodiment of this application.

[0044] Figure 10 This is a schematic block diagram of the training device for the first model provided in the embodiments of this application.

[0045] Figure 11 This is a schematic block diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0046] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0047] The solutions provided in this application may involve artificial intelligence technology.

[0048] Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that utilize digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce new intelligent machines that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making capabilities.

[0049] It should be understood that artificial intelligence (AI) technology is a comprehensive discipline involving a wide range of fields, encompassing both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0050] With the research and advancement of artificial intelligence (AI) technology, AI is being studied and applied in various fields, such as smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, autonomous driving, drones, robots, smart healthcare, and smart customer service. It is believed that with the development of technology, AI will be applied in more fields and play an increasingly important role.

[0051] This application's embodiments may relate to Computer Vision (CV) technology within artificial intelligence. Computer vision is a science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes in recognizing, tracking, and measuring targets, and further performs image processing to transform the computer-processed images into those more suitable for human observation or transmission to instruments for detection. As a scientific discipline, computer vision researches related theories and technologies, attempting to establish artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), and other technologies, as well as common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0052] This application's embodiments may also relate to Machine Learning (ML) in artificial intelligence technology. ML is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, and many other disciplines. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning.

[0053] To facilitate understanding of the proposed solution, the relevant terms used in this application will be explained below.

[0054] (1) Image and video quality assessment: This refers to the perception, measurement, and evaluation of distortion in images or video frames through subjective and objective methods. Subjective scoring methods generally use the mean opinion score (MOS) or the difference mean opinion score (DMOS) to represent the quality. Subjective scoring methods can also be called subjective rating methods. Objective scoring methods generally use algorithmic models to calculate the quality indicators of videos / images. Objective scoring methods can also be called methods that use quality assessment models to output quality scores.

[0055] (2) Subjective scoring platform: A labeling platform that performs subjective scoring on images and videos. Subjective scoring refers to the evaluation / labeling of a picture or a video based on its quality or aesthetics.

[0056] Figure 1 This is a schematic diagram of the interface of the subjective scoring platform provided in the embodiments of this application.

[0057] like Figure 1 As shown, the interface of a subjective scoring platform can include the video to be scored and scoring options. Specifically, a five-point scoring system can be used, corresponding to five levels: Excellent, Good, Average, Poor, and Very Poor. A sample is typically scored by multiple reviewers, who select a score within that level. The final quality score for that sample is obtained by averaging the scores from all reviewers. Other subjective scoring methods exist, such as paired comparison, where reviewers are shown two images or videos and asked to choose which is better.

[0058] (3) Active learning: Subjective scoring platforms need to select a portion of samples from a massive image and video library for evaluators to perform subjective scoring. Obtaining subjective scores through evaluators' subjective scoring of samples can be called obtaining subjective scores of samples by subjective scoring. Samples labeled with subjective scores can be called labeled samples. Active learning can proactively select samples that the current model considers most difficult to distinguish or have the most information through specific selection strategies for evaluators to score. In this way, the number of samples that need to be labeled can be effectively reduced while ensuring model performance.

[0059] (4) Passive learning: Generally, the model is trained using randomly selected samples.

[0060] Figure 2 This is a schematic flowchart illustrating the method for training a quality evaluation model based on randomly selected samples, as provided in the embodiments of this application.

[0061] like Figure 2 As shown, the training process of the quality assessment model is a "waterfall" algorithm development process. Several (n) samples are randomly selected from a massive database and placed on a subjective scoring platform for evaluation by evaluators (evaluators 1 to t). After scoring, the model is trained. Regarding this method of training the quality assessment model, the random sample selection method easily leads to the selection of many worthless samples, especially in massive image and video databases where there is a lot of similar and redundant data. Moreover, the number of selected samples needs to be predetermined, making it difficult to control. Furthermore, completely isolating subjective scoring from model training in this "waterfall" development process means that if the quality of the subjectively scored dataset is found to be low, re-scoring is required, which is time-consuming, labor-intensive, and has a very low error tolerance.

[0062] (5) Mean opinion score (MOS), which is the final quality score of a sample mentioned above. The specific value of this score can be obtained from the average of the scores of all evaluators. For example, the subjective truth value involved in this application can be the MOS.

[0063] (6) Fitting: Fitting is the process of connecting a series of points on a plane with a smooth curve. Because there are countless possible curves, there are various fitting methods. The fitted curve can generally be represented by a function, and different functions have different fitting names. Commonly used fitting methods include least squares curve fitting. If the function to be determined is linear, it is called linear fitting or linear regression (mainly in statistics); otherwise, it is called nonlinear fitting or nonlinear regression. The expression can also be a piecewise function, in which case it is called spline fitting. For example, the first model involved in this application can be a model obtained through fitting. The prediction result of this first model can be an approximation of the subjective truth.

[0064] (7) Training: A model can be obtained by training the image / video dataset after subjective scoring. For example, the second model involved in this application can be a model obtained through training. The prediction result of the second model can be an approximation of the subjective true value.

[0065] (8) Image Quality Assessment (IQA): This is one of the fundamental techniques in image processing. It mainly involves analyzing the characteristics of an image to assess its quality (degree of image distortion). Image quality assessment plays a crucial role in image processing systems, particularly in algorithm analysis and comparison, and system performance evaluation. In recent years, with the extensive research in the field of digital images, research on image quality assessment has received increasing attention from researchers, leading to the proposal and refinement of many image quality assessment indicators and methods.

[0066] (9) Video quality assessment (VQA): It is one of the basic techniques in video processing. It mainly involves analyzing the characteristics of the video and then evaluating the quality of the video (objective video quality).

[0067] Typically, subjective scoring models and calculated values ​​can reflect the objective and subjective quality of a video, but their computational complexity and accuracy are not ideal. For example, one could use encoder quantization parameters (QP), peak signal-to-noise ratio (PSNR), and structural similarity (SSIM) to infer the objective quality score of a game video. While this method is not highly complex, its correlation with subjective quality evaluation is low, which has limitations, and it also introduces some latency. Another approach is to use models such as Video Multimethod Assessment Fusion (VAMF) to obtain the objective quality score of a game video. This method is more accurate than PSNR, but its computational complexity is much higher, making real-time calculation impossible for high frame rate, high-resolution videos.

[0068] In addition, there are many types of cloud games, and the massive amount of video data requires real-time acquisition of subjective scores for the videos, but there are no models or solutions in the relevant technologies for real-time feedback of subjective quality.

[0069] Therefore, this application provides a method for evaluating image and video quality based on approximation values, which can evaluate image and video quality based on approximation values ​​that approximate the subjective truth in real time without increasing server-side hardware costs or ensuring evaluation accuracy.

[0070] Figure 3 This is a schematic flowchart of a method 100 for evaluating image and video quality based on approximation values ​​provided in this application embodiment. It should be noted that the solution provided in this application embodiment can be executed by any electronic device with data processing capabilities. For example, the electronic device can be implemented as a server. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, and big data and artificial intelligence platforms. The server can be directly or indirectly connected via wired or wireless communication, and this application does not impose any limitations. For ease of description, a service system is used as an example below.

[0071] like Figure 3As shown, the method 100 may include some or all of the following:

[0072] S110, Obtain the sample to be evaluated, which includes the video or image to be evaluated;

[0073] S120, Based on the parameters of the sample to be evaluated, calculate the first approximation value that approximates the subjective truth value of the sample to be evaluated using the first online model;

[0074] The first model is a model obtained based on the offline second model. The second model is a model obtained by using k samples and the subjective truth values ​​of the k samples as a training set. The subjective truth values ​​of the k samples are obtained by subjective scoring. The first model is a model obtained by fitting the parameters of the k samples using the approximation values ​​of the k samples obtained by the second model as a reference, where k > 0.

[0075] S130, evaluate the quality of the video or image to be evaluated based on the first approximation value.

[0076] In cloud gaming, the game runs on the server side, and rendered game video footage is captured via screen capture, compressed using a video encoder, and transmitted to the user client over the network. Based on this, the user client can evaluate image and video quality using an approximation value, which approximates the subjective truth value based on real-time feedback.

[0077] In this embodiment, based on the parameters of the sample to be evaluated, a first approximation value that approximates the subjective truth value of the sample to be evaluated is calculated using an online first model; then, the quality of the video or image to be evaluated is assessed based on the first approximation value. On one hand, the online first model can calculate and feedback the first approximation value in real time, and thus, the image and video quality can be assessed based on the real-time feedback approximation value that approximates the subjective truth value. On the other hand, the first model is constructed as a model obtained based on an offline second model, and the second model is constructed as a model obtained using k samples and the subjective truth values ​​of the k samples as a training set. The subjective truth values ​​of the k samples are obtained by subjective scoring. The first model is a model obtained by fitting the parameters of the k samples using the approximation values ​​obtained by the second model as a reference, where k > 0; equivalently, the first model is obtained by fitting the trained second model, which can ensure the evaluation accuracy of the first model without increasing the server-side hardware cost.

[0078] In summary, the method provided in this application can evaluate image and video quality based on approximation values ​​that approximate the subjective truth in real-time feedback, without increasing server-side hardware costs or ensuring evaluation accuracy. In other words, the first model can calculate an approximation value that approximates the subjective truth of the sample to be evaluated online, and then use this approximation value to evaluate the quality of cloud gaming videos. Essentially, it can solve the problem of real-time evaluation of video subjective quality during online gaming. For example, it can be used for online monitoring of the overall quality of cloud gaming dashboards.

[0079] It should be noted that, in this embodiment of the application, the user can utilize a cloud gaming video quality evaluation scheme closely aligned with the application scenario on the user client to obtain subjective ground truth values ​​through subjective scoring, which will then serve as training samples for the second model. For example, it can employ... Figure 1 The interface of the subjective scoring platform shown is another example, which can be adopted. Figure 2 The process shown is based on training a second model using randomly selected samples, but this application is not limited thereto.

[0080] In some embodiments, the parameters of the sample to be evaluated include at least one of the following: feedback parameters of the network module, setting parameters of the cloud gaming module, and calculation parameters of the encoding / decoding module. In one implementation, the type of parameters of the sample to be evaluated can be defined based on the classification of Software Development Kits (SDKs). For example, SDKs include, but are not limited to, network SDKs, cloud gaming SDKs, and encoding / decoding SDKs.

[0081] Figure 4 This is a schematic block diagram illustrating the working principle of the first model provided in the embodiments of this application.

[0082] like Figure 4 As shown, the first model utilizes parameters 1 to n of the input sample to be evaluated (e.g., parameters are selected based on a scenario, which includes, but is not limited to, encoding parameters related to the sample to be evaluated, such as parameters related to the encoded frame and bitstream, such as quantization parameters (QP), MV, frame rate, frame length, frame complexity parameters, and frame type). The frame complexity parameter can be, for example, the sum of absolute transform differences (SATD). The frame rate can refer to the number of images played per second, such as 24 frames per second (24 images per second), 60 frames per second (60 images per second), and so on. The frame length can refer to the length of the data frame. By calculating the input parameters 1 to n, a first approximation value that approximates the subjective truth value of the sample to be evaluated can be obtained.

[0083] In some embodiments, prior to S120, the method 100 may further include:

[0084] Based on the parameters of the k samples, the first model is used to calculate the approximation value of the k samples. Based on the approximation value of the k samples obtained using the second model and the approximation value of the k samples calculated using the first model, the first model is evaluated. For each of the k samples, if the difference between the approximation value calculated using the first model and the approximation value obtained using the second model is less than or equal to a first preset threshold, the evaluation result of the first model is determined to be positive; otherwise, the evaluation result of the first model is determined to be negative. For example, for any sample among the k samples, if the difference between the approximation value calculated using the first model and the approximation value obtained using the second model is greater than the first preset threshold, the evaluation result of the first model is determined to be negative.

[0085] In short, the performance or accuracy of the first model can be evaluated using the approximation values ​​obtained from the second model.

[0086] In some embodiments, prior to S120, the method 100 may further include:

[0087] Based on the subjective true values ​​of the k samples and the approximation values ​​of the k samples calculated using the first model, the first model is evaluated. For each of the k samples, if the difference between the approximation value calculated using the first model and the subjective true value is less than or equal to a second preset threshold, the evaluation result of the first model is determined to be positive; otherwise, the evaluation result of the first model is determined to be negative. For example, for any one of the k samples, if the difference between the approximation value calculated using the first model and the subjective true value is greater than the second preset threshold, the evaluation result of the first model is determined to be negative.

[0088] In short, subjective truth values ​​can be used to evaluate the performance or accuracy of the first model.

[0089] Figure 5 This is a schematic block diagram illustrating the training and evaluation principles of the first model provided in the embodiments of this application.

[0090] like Figure 5 As shown, the fitting process for the first model can be as follows: Figure 5The section indicated by the dashed single arrow involves two aspects: firstly, obtaining the subjective ground truth value A of the image frames in the decoded sequence set through subjective scoring, so that the obtained subjective ground truth value A can be used as the training set to train the second model; secondly, obtaining the first model based on the second model and the parameter set to be input. The optimization process for the first model can be represented by the solid double arrow. On one hand, the first model can be evaluated based on the subjective approximation value B output by the second model and the subjective approximation value A output by the first model; on the other hand, the first model can also be evaluated based on the subjective ground truth value A and the subjective approximation value A output by the first model.

[0091] The training of the second model may include the following steps:

[0092] (1) Obtain the decoded sequence set.

[0093] Because cloud games are diverse, varying in complexity, and offer different subjective experiences, this application categorizes cloud game videos by scene. This allows for the training of a second sub-model within a second model and a first sub-model within a first model for each scene. Alternatively, the scene-classified cloud game videos can be further segmented based on game context, allowing for the training of a second sub-model within a second model and a first model within a first model for each game context within a specific scene. This application does not limit the specific classification of scenes and game contexts. For example, a scene can be the type of device used to play the video or image to be evaluated. A game context can be a scene from a game, such as a battle scene or a non-battle scene. For a particular category of video after classification, it can be encoded and compressed according to an encoding configuration scheme (adapted to the characteristics of cloud games, such as frame rate, bit rate, and resolution) to obtain an encoded and compressed sequence set. This encoded and compressed sequence set undergoes lossy compression compared to the source sequence set, resulting in a loss of detail in the video image quality. The encoded and compressed sequence set is then decoded to obtain a decoded sequence set. Optionally, the decoded sequence set can be the k samples mentioned above.

[0094] (2) Obtain the subjective true value A of the image frames in the decoded sequence set by subjective scoring.

[0095] Because cloud gaming scenarios differ from real-time communication subjective video scenarios—for example, cloud gaming scenarios can involve mobile devices, fixed PCs, and TV terminals—and because gamers' experience requirements differ from those of real-time communication users, this application utilizes a standard system for subjective evaluation of cloud gaming videos to obtain the subjective truth value A of image frames in the decoded sequence set through subjective scoring. For example, the encoded and compressed sequence set is decoded and played by a third party, and then human reviewers (i.e., evaluators) obtain the subjective truth value A of the image frames in the decoded sequence set through subjective scoring based on the standard system for subjective evaluation of cloud gaming videos.

[0096] (3) Train the second model based on the subjective true value A of the image frames in the decoded sequence set.

[0097] For the encoded compressed sequence set, a decoded sequence set is created through a third-party decoder. This decoded sequence set can then be used as the training set for a second model. The trained second model is then used to obtain the subjective approximation value B (which infinitely approaches the subjective true value A) of the image frames in the decoded sequence set. This application does not place much emphasis on the performance complexity of the second model, only requiring high accuracy; that is, the second model can be a very complex model with extremely high accuracy.

[0098] (4) The first model is obtained based on the second model.

[0099] After obtaining the subjective approximation value B of the image frames in the decoded sequence set using the trained second model, a fitting calculation can be performed based on the subjective approximation value B and the parameters of the image frames in the decoded sequence set to obtain the first model. Fitting involves connecting a series of points on a plane with a smooth curve. Because there are countless possible curves, there are various fitting methods. The fitted curve can generally be represented by a function, and different fitting names are given depending on the function. Commonly used fitting methods include least squares curve fitting. If the function to be determined is linear, it is called linear fitting or linear regression (mainly in statistics); otherwise, it is called nonlinear fitting or nonlinear regression. The expression can also be a piecewise function, in which case it is called spline fitting. For example, the first model involved in this application can be a model obtained through fitting. The prediction result of the first model can be an approximation value that approximates the subjective truth.

[0100] In some embodiments, the method 100 may further include:

[0101] If the evaluation result of the first model is positive, then the first model is integrated into the service system; if the evaluation result of the first model is negative, then the first model is refitted until the evaluation result of the first model is positive.

[0102] In other words, if the first model achieves the expected evaluation results, it can be integrated into the service system; if the first model does not achieve the expected results, it needs to be refitted until it achieves the expected results.

[0103] In some embodiments, the method 100 may further include:

[0104] The statistical module reports the first approximation value to the second model; the second model is used to obtain the second approximation value of the sample to be evaluated; based on the first and second approximation values, it is determined whether the first module should be optimized; if the difference between the first and second approximation values ​​is greater than a third preset threshold, the first model is optimized using the parameters of the sample to be evaluated and the second approximation value; if the difference between the first and second approximation values ​​is less than or equal to the third preset threshold, it is determined that the first module does not need to be optimized.

[0105] In short, the first approximation value can be used to optimize the first model in order to improve the accuracy of the first model.

[0106] In some embodiments, before determining whether to optimize the first module based on the first approximation value and the second approximation value, the subjective true value of the sample to be evaluated is obtained; if the difference between the second approximation value and the subjective true value of the sample to be evaluated is greater than a fourth preset threshold, the second model is optimized using the sample to be evaluated and the subjective true value of the sample to be evaluated.

[0107] In short, the first approximation value can be used to optimize the second model, thereby improving the accuracy of the second model.

[0108] Figure 6 This is a schematic block diagram illustrating the optimization principle of the first model provided in the embodiments of this application.

[0109] like Figure 6 As shown, a first model is first obtained to calculate the approximation value to the subjective truth value, and then the first model is placed into the cloud gaming server system. Based on this, the acquisition frame (i.e., the sample to be evaluated) is first acquired and then encoded to obtain the encoded frame; then, through the parameters and bitstream information brought by the encoded frame, the parameters of the sample to be evaluated that need to be input into the first model are obtained. After the parameters of the sample to be evaluated are input into the first model, the first model calculates based on the parameters of the sample to be evaluated and obtains the first approximation value of the sample to be evaluated; at this time, the first approximation value can be fed back to the platform statistics module through the data statistics module so that the platform statistics module can determine whether to optimize the first model or the second model based on the first approximation value.

[0110] Figure 7This is a schematic block diagram of a service system including a first model provided in an embodiment of this application.

[0111] like Figure 7 As shown, the fitted first model is integrated into the encoding / decoding module of the service system. Input parameters, such as parameters P1, P2, and P3, can be obtained from the service system. Parameter P1 represents the feedback parameter of the network module, parameter P2 represents the setting parameter of the cloud gaming module, and parameter P3 represents the calculation parameter of the encoding / decoding module. Then, the first model calculates a first approximation value for the sample to be evaluated based on parameters P1, P2, and P3. Optionally, this first approximation value can be reported to the platform statistics module of the service system through the data statistics module, so that the platform statistics module can determine whether the second model and the first model need to be optimized based on the first approximation value. For example, the platform statistics module can perform statistical classification and determine whether the first sub-model or the second sub-model corresponding to the classification needs to be optimized. Of course, the first model can also be optimized based on the optimized second model after optimizing the second model. This embodiment does not specifically limit this; by optimizing the first model, the accuracy of the approximation value can be improved.

[0112] In some embodiments, the first model includes multiple first sub-models corresponding to multiple scenarios, the second model includes multiple second sub-models corresponding to the multiple scenarios, the multiple first sub-models are models obtained based on the multiple second sub-models, and the multiple scenarios include the first scenario where the sample to be evaluated is located; based on this, S120 may include: determining the first sub-model corresponding to the first scenario; and calculating the first approximation value based on the parameters of the sample to be evaluated and using the first sub-model corresponding to the first scenario.

[0113] In other words, the parameters of samples from different scenarios can be fitted to different first sub-models, and different second sub-models can be obtained through training based on samples from different scenarios. This can improve the accuracy of both the first and second sub-models.

[0114] In some embodiments, the plurality of scenarios includes the type of device used to play the video or image to be evaluated.

[0115] Of course, in other alternative embodiments of this application, the multiple scenarios may also include the application or playback method to which the video or image to be evaluated belongs, and this application does not specifically limit this.

[0116] In some embodiments, the first model is a half-reference model or a no-reference model, and the second model is a full-reference model or a no-reference model. The half-reference model refers to a model obtained by referring to some parameters in the image frame before compression and the image frame after compression. The no-reference model refers to a model obtained by referring only to the image frame after compression. The full-reference model refers to a model obtained by referring to the image frame before compression and the image frame after compression.

[0117] Of course, the semi-reference model can also refer to a model obtained by referring to some parameters in the image frame before encoding and the image frame after encoding. The no-reference model can also refer to a model obtained by referring only to the image frame after encoding. The full-reference model can also refer to a model obtained by referring to the image frame before encoding and the image frame after encoding. This application does not make any specific limitations on this.

[0118] In summary, this application provides a method for evaluating image and video quality based on approximation values, which can assess the subjective quality of video in real time. Optionally, information such as cloud game types and platforms can be categorized, and for a given category, the subjective ground truth value A of k samples can be obtained. Then, an offline second model is used to obtain an approximation value B (very close to A) of the k samples that approximates the subjective ground truth value A. The obtained approximation value B is then used as a reference ground truth value, and multiple parameters from the input k samples are used to fit a first model (used to calculate the approximation value C of the subjective ground truth value A, and capable of quickly calculating the approximation value C). Based on this, the approximation value of the subjective ground truth value of real-time cloud game video can be obtained online through this first model. Optionally, for subsequent additions of games, the online first model can be optimized based on the updated second model.

[0119] Figure 8 This is a schematic flowchart of the training method 200 for the first model provided in this application embodiment. It should be noted that the solution provided in this application embodiment can be executed by any electronic device with data processing capabilities. For example, the electronic device can be implemented as a server. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, and big data and artificial intelligence platforms. The server can be directly or indirectly connected via wired or wireless communication, and this application does not impose any limitations on this.

[0120] like Figure 8 As shown, the method 200 may include:

[0121] S210, obtain k samples, and the subjective truth values ​​of these k samples are obtained by subjective scoring;

[0122] S220, using the k samples and the subjective truth values ​​of the k samples as the training set, the second model is obtained;

[0123] S230, using the k samples as input, the second model is used to obtain the approximation value of the subjective truth for the k samples;

[0124] S240, using the approximation values ​​of the k samples as a reference, the parameters of the k samples are fitted to obtain the first model.

[0125] It should be noted that the method 200 may also include the schemes related to the evaluation and optimization of the first model in the method 100. In other words, the schemes related to the evaluation and optimization of the first model in the method 200 can refer to the corresponding schemes in the method 100. To avoid repetition, they will not be described again here.

[0126] The preferred embodiments of this application have been described in detail above with reference to the accompanying drawings. However, this application is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this application, various simple modifications can be made to the technical solutions of this application, and these simple modifications all fall within the protection scope of this application. For example, the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, this application will not describe the various possible combinations separately. Furthermore, various different embodiments of this application can also be arbitrarily combined, as long as they do not violate the spirit of this application, they should also be considered as the content disclosed in this application.

[0127] It should also be understood that, in the various method embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. Furthermore, in the embodiments of this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. Specifically, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0128] The methods provided in the embodiments of this application have been described above. The apparatus provided in the embodiments of this application will be described below.

[0129] Figure 9 This is a schematic block diagram of an apparatus 300 for evaluating image and video quality based on approximation values, provided in an embodiment of this application.

[0130] The acquisition unit 310 is used to acquire a sample to be evaluated, which includes a video or an image to be evaluated.

[0131] The calculation unit 320 is used to calculate a first approximation value that approximates the subjective truth value of the sample to be evaluated based on the parameters of the sample to be evaluated using the first online model.

[0132] The first model is a model obtained based on the offline second model. The second model is a model obtained by using k samples and the subjective truth values ​​of the k samples as a training set. The subjective truth values ​​of the k samples are obtained by subjective scoring. The first model is a model obtained by fitting the parameters of the k samples using the approximation values ​​of the k samples obtained by the second model as a reference, where k > 0.

[0133] Evaluation unit 330 is used to evaluate the quality of the video or image to be evaluated based on the first approximation value.

[0134] In some embodiments, the parameters of the sample to be evaluated include at least one of the following parameters: feedback parameters of the network module, setting parameters of the cloud gaming module, and calculation parameters of the encoding / decoding module.

[0135] In some embodiments, before the calculation unit 320 calculates a first approximation value that approximates the subjective truth value of the sample to be evaluated using an online first model based on the parameters of the sample to be evaluated, the evaluation unit 330 is further configured to:

[0136] Based on the parameters of the k samples, the approximation values ​​of the k samples are calculated using the first model;

[0137] The first model is evaluated based on the approximation values ​​of the k samples obtained using the second model and the approximation values ​​of the k samples calculated using the first model.

[0138] For each of the k samples, if the difference between the approximation value calculated using the first model and the approximation value obtained using the second model is less than or equal to a first preset threshold, then the evaluation result of the first model is determined to be positive; otherwise, the evaluation result of the first model is determined to be negative.

[0139] In some embodiments, before the calculation unit 320 calculates a first approximation value that approximates the subjective truth value of the sample to be evaluated using an online first model based on the parameters of the sample to be evaluated, the evaluation unit 330 is further configured to:

[0140] The first model is evaluated based on the subjective truth values ​​of the k samples and the approximation values ​​of the k samples calculated using the first model.

[0141] For each of the k samples, if the difference between the approximation value calculated using the first model and the subjective true value is less than or equal to the second preset threshold, then the evaluation result of the first model is determined to be positive; otherwise, the evaluation result of the first model is determined to be negative.

[0142] In some embodiments, the evaluation unit 330 is further configured to:

[0143] If the evaluation result of the first model is positive, then the first model is integrated into the service system; if the evaluation result of the first model is negative, then the first model is refitted until the evaluation result of the first model is positive.

[0144] In some embodiments, the evaluation unit 330 is further configured to:

[0145] The statistical module reports the first approximation value to the second model.

[0146] The second model is used to obtain the second approximation value of the sample to be evaluated;

[0147] Based on the first approximation value and the second approximation value, determine whether to optimize the first module;

[0148] If the difference between the first approximation value and the second approximation value is greater than the third preset threshold, then the first model is optimized using the parameters of the sample to be evaluated and the second approximation value; if the difference between the first approximation value and the second approximation value is less than or equal to the third preset threshold, then it is determined that the first module does not need to be optimized.

[0149] In some embodiments, before determining whether to optimize the first module based on the first approximation value and the second approximation value, the evaluation unit 330 is further configured to:

[0150] Obtain the subjective true value of the sample to be evaluated;

[0151] If the difference between the second approximation value and the subjective true value of the sample to be evaluated is greater than the fourth preset threshold, then the second model is optimized using the sample to be evaluated and the subjective true value of the sample to be evaluated.

[0152] In some embodiments, the first model includes multiple first sub-models corresponding to multiple scenarios, the second model includes multiple second sub-models corresponding to the multiple scenarios, the multiple first sub-models are models obtained based on the multiple second sub-models, and the multiple scenarios include the first scenario where the sample to be evaluated is located; wherein, the computing unit 320 is specifically used for:

[0153] Determine the first sub-model corresponding to the first scene;

[0154] Based on the parameters of the sample to be evaluated, the first approximation value is calculated using the first sub-model corresponding to the first scenario.

[0155] In some embodiments, the plurality of scenarios includes the type of device used to play the video or image to be evaluated.

[0156] In some embodiments, the first model is a half-reference model or a no-reference model, and the second model is a full-reference model or a no-reference model. The half-reference model refers to a model obtained by referring to some parameters in the image frame before compression and the image frame after compression. The no-reference model refers to a model obtained by referring only to the image frame after compression. The full-reference model refers to a model obtained by referring to the image frame before compression and the image frame after compression.

[0157] Figure 10 This is a schematic block diagram of the training device 400 for the first model provided in the embodiments of this application.

[0158] like Figure 10 As shown, the device 400 may include:

[0159] The first acquisition unit 410 is used to acquire k samples, and the subjective truth values ​​of the k samples are acquired by subjective scoring.

[0160] The first training unit 420 is used to obtain the second model by using the k samples and the subjective truth values ​​of the k samples as the training set;

[0161] The second acquisition unit 430 is used to obtain the approximation value of the subjective truth of the k samples by using the second model with the k samples as input.

[0162] The second training unit 440 is used to fit the parameters of the k samples to obtain the first model by using the approximation values ​​of the k samples as a reference.

[0163] It should be understood that the device embodiments and method embodiments can correspond to each other, and similar descriptions can be referred to the method embodiments. To avoid repetition, further details are omitted here. Specifically, the device 300 can correspond to the corresponding subject in executing the method 100 of the embodiments of this application, and each unit in the device 300 is used to implement the corresponding process in the method 100. Similarly, the device 400 can correspond to the corresponding subject in executing the method 200 of the embodiments of this application, and the units in the device 400 can be used to implement the process in the method 200. For the sake of brevity, further details are omitted here.

[0164] It should also be understood that the various units in the video processing apparatus involved in the embodiments of this application can be individually or entirely merged into one or more other units, or some of the units can be further divided into multiple functionally smaller units. This can achieve the same operation without affecting the technical effect of the embodiments of this application. The above-mentioned units are based on logical function division. In practical applications, the function of one unit can also be implemented by multiple units, or the function of multiple units can be implemented by one unit. In other embodiments of this application, the apparatus 300 or the apparatus 400 may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented by multiple units working together. According to another embodiment of this application, the apparatus 300 or the apparatus 400 involved in the embodiments of this application can be constructed by running a computer program (including program code) capable of executing the steps involved in the corresponding method on a general-purpose computing device including processing elements and storage elements such as a central processing unit (CPU), random access storage medium (RAM), and read-only storage medium (ROM), and implementing the method for evaluating image and video quality based on approximation values ​​or the training method of the first model provided in the embodiments of this application. Computer programs can be recorded on, for example, a computer-readable storage medium, loaded into an electronic device via the computer-readable storage medium, and run therein to implement the corresponding methods of the embodiments of this application.

[0165] In other words, the units mentioned above can be implemented in hardware, in software instructions, or in a combination of hardware and software. Specifically, the steps of the method embodiments in this application can be completed by the integrated logic circuits in the processor's hardware and / or by software instructions. The steps of the method disclosed in the embodiments of this application can be directly manifested as being executed by a hardware decoding processor, or executed by a combination of hardware and software in the decoding processor. Optionally, the software can reside in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. This storage medium is located in memory, and the processor reads the information in the memory and completes the steps in the above method embodiments in conjunction with its hardware.

[0166] Figure 11 This is a schematic structural diagram of the electronic device 500 provided in the embodiments of this application.

[0167] like Figure 11As shown, the electronic device 500 includes at least a processor 510 and a computer-readable storage medium 520. The processor 510 and the computer-readable storage medium 520 can be connected via a bus or other means. The computer-readable storage medium 520 stores a computer program 521, which includes computer instructions. The processor 510 executes the computer instructions stored in the computer-readable storage medium 520. The processor 510 is the computing and control core of the electronic device 500, and is suitable for implementing one or more computer instructions, specifically for loading and executing one or more computer instructions to achieve a corresponding method flow or function.

[0168] As an example, processor 510 may also be referred to as a central processing unit (CPU). Processor 510 may include, but is not limited to: general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0169] As an example, the computer-readable storage medium 520 may be a high-speed RAM memory or a non-volatile memory, such as at least one disk storage device; optionally, it may also be at least one computer-readable storage medium located remotely from the aforementioned processor 510. Specifically, the computer-readable storage medium 520 includes, but is not limited to, volatile memory and / or non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory may be random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0170] In one implementation, the electronic device 500 can be Figure 9 The apparatus 300 shown is for evaluating image and video quality based on approximation values; the computer-readable storage medium 520 stores computer instructions; the processor 510 loads and executes the computer instructions stored in the computer-readable storage medium 520 to implement... Figure 3 The corresponding steps in the method embodiment shown are as follows; in specific implementation, the computer instructions in the computer-readable storage medium 520 are loaded and executed by the processor 510, and will not be described again here to avoid repetition.

[0171] In one implementation, the electronic device 500 can be Figure 10 The training device 400 for the first model shown; the computer-readable storage medium 520 stores computer instructions; the processor 510 loads and executes the computer instructions stored in the computer-readable storage medium 520 to achieve... Figure 8 The corresponding steps in the method embodiment shown are as follows; in specific implementation, the computer instructions in the computer-readable storage medium 520 are loaded and executed by the processor 510, and will not be described again here to avoid repetition.

[0172] According to another aspect of this application, embodiments of this application also provide a computer-readable storage medium (Memory), which is a memory device in an electronic device 500 for storing programs and data. For example, a computer-readable storage medium 520. It is understood that the computer-readable storage medium 520 here may include both the built-in storage medium in the electronic device 500 and extended storage media supported by the electronic device 500. The computer-readable storage medium provides storage space that stores the operating system of the electronic device 500. Furthermore, the storage space also stores one or more computer instructions suitable for loading and execution by a processor 510, which may be one or more computer programs 521 (including program code).

[0173] According to another aspect of this application, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. For example, computer program 521. In this case, electronic device 500 may be a computer, and processor 510 reads the computer instructions from computer-readable storage medium 520, executes the computer instructions, causing the computer to perform the method for evaluating image / video quality based on approximation values ​​or the training method of the first model provided in the various alternative embodiments described above.

[0174] In other words, when implemented using software, it can be implemented entirely or partially in the form of a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes of the embodiments of this application are run or the functions of the embodiments of this application are implemented. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0175] Those skilled in the art will recognize that the units and process steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0176] Finally, it should be noted that the above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for evaluating image and video quality based on approximation values, characterized in that, include: Obtain the sample to be evaluated, which includes the video or image to be evaluated. Based on the parameters of the sample to be evaluated, a first approximation value that approximates the subjective truth value of the sample to be evaluated is calculated using the first online model. The parameters of the sample to be evaluated include at least one of the following parameters: feedback parameters of the network module, setting parameters of the cloud gaming module, and calculation parameters of the encoding and decoding module. Wherein, the first model is a model obtained based on the offline second model, the second model is a model obtained by using k samples and the subjective truth values ​​of the k samples as a training set, the subjective truth values ​​of the k samples are obtained by subjective scoring, the first model is a model obtained by fitting and calculating the parameters of the k samples using the approximation values ​​of the k samples obtained by the second model as a reference, k>0, the k samples are a decoded sequence set, the decoded sequence set is obtained by encoding and compressing a type of video according to the encoding configuration scheme, and then decoding the encoded and compressed sequence set, the subjective truth values ​​of the k samples are the subjective truth values ​​of the image frames in the decoded sequence set, and the prediction result of the second model is the approximation value of the subjective truth value; The quality of the video or image to be evaluated is assessed based on the first approximation value.

2. The method according to claim 1, characterized in that, Before calculating the first approximation value of the subjective truth value of the sample to be evaluated using the first online model based on the parameters of the sample to be evaluated, the method further includes: Based on the parameters of the k samples, the approximation values ​​of the k samples are calculated using the first model; The first model is evaluated based on the approximation values ​​of the k samples obtained using the second model and the approximation values ​​of the k samples calculated using the first model. For each of the k samples, if the difference between the approximation value calculated using the first model and the approximation value obtained using the second model is less than or equal to a first preset threshold, then the evaluation result of the first model is determined to be positive; otherwise, the evaluation result of the first model is determined to be negative.

3. The method according to claim 1, characterized in that, Before calculating the first approximation value of the subjective truth value of the sample to be evaluated using the first online model based on the parameters of the sample to be evaluated, the method further includes: The first model is evaluated based on the subjective truth values ​​of the k samples and the approximation values ​​of the k samples calculated using the first model. For each of the k samples, if the difference between the approximation value calculated using the first model and the subjective true value is less than or equal to a second preset threshold, then the evaluation result of the first model is determined to be positive; otherwise, the evaluation result of the first model is determined to be negative.

4. The method according to claim 2 or 3, characterized in that, The method further includes: If the evaluation result of the first model is positive, the first model is integrated into the service system; if the evaluation result of the first model is negative, the first model is refitted until the evaluation result of the first model is positive.

5. The method according to claim 1, characterized in that, The method further includes: The statistical module reports the first approximation value to the second model; The second model is used to obtain the second approximation value of the sample to be evaluated; Based on the first approximation value and the second approximation value, determine whether to optimize the first model; If the difference between the first approximation value and the second approximation value is greater than a third preset threshold, then the first model is optimized using the parameters of the sample to be evaluated and the second approximation value; if the difference between the first approximation value and the second approximation value is less than or equal to the third preset threshold, then it is determined that the first model does not need to be optimized.

6. The method according to claim 5, characterized in that, Before determining whether to optimize the first model based on the first approximation value and the second approximation value, the method further includes: Obtain the subjective true value of the sample to be evaluated; If the difference between the second approximation value and the subjective true value of the sample to be evaluated is greater than the fourth preset threshold, then the second model is optimized using the sample to be evaluated and the subjective true value of the sample to be evaluated.

7. The method according to claim 1, characterized in that, The first model includes multiple first sub-models corresponding to multiple scenarios, the second model includes multiple second sub-models corresponding to the multiple scenarios, the multiple first sub-models are models obtained based on the multiple second sub-models, and the multiple scenarios include the first scenario where the sample to be evaluated is located; The step of calculating a first approximation value that approximates the subjective truth value of the sample to be evaluated using an online first model based on the parameters of the sample to be evaluated includes: Determine the first sub-model corresponding to the first scenario; Based on the parameters of the sample to be evaluated, the first approximation value is calculated using the first sub-model corresponding to the first scenario.

8. The method according to claim 7, characterized in that, The multiple scenarios include the types of devices used to play the video or image to be evaluated.

9. The method according to any one of claims 1 to 3, characterized in that, The first model is a semi-reference model or a no-reference model, and the second model is a full-reference model or a no-reference model. The semi-reference model refers to a model obtained by referring to some parameters in the image frame before compression and the image frame after compression. The no-reference model refers to a model obtained by referring only to the image frame after compression. The full-reference model refers to a model obtained by referring to the image frame before compression and the image frame after compression.

10. An apparatus for evaluating image and video quality based on approximation values, characterized in that, include: An acquisition unit is used to acquire a sample to be evaluated, which includes a video or an image to be evaluated. The calculation unit is used to calculate a first approximation value that approximates the subjective truth value of the sample to be evaluated based on the parameters of the sample to be evaluated using a first online model. The parameters of the sample to be evaluated include at least one of the following parameters: feedback parameters of the network module, setting parameters of the cloud gaming module, and calculation parameters of the encoding and decoding module. Wherein, the first model is a model obtained based on the offline second model, the second model is a model obtained by using k samples and the subjective truth values ​​of the k samples as a training set, the subjective truth values ​​of the k samples are obtained by subjective scoring, the first model is a model obtained by fitting the parameters of the k samples with the approximation values ​​of the k samples obtained by the second model as a reference, k>0, the k samples are a decoded sequence set, the decoded sequence set is obtained by encoding and compressing a class of videos according to the encoding configuration scheme, and then decoding the encoded and compressed sequence set, the subjective truth values ​​of the k samples are the subjective truth values ​​of the image frames in the decoded sequence set, and the prediction result of the second model is the approximation value of the subjective truth value; An evaluation unit is used to evaluate the quality of the video or image to be evaluated based on the first approximation value.

11. A training method for a first model, characterized in that, include: k samples are obtained, and the subjective truth value of the k samples is obtained by subjective scoring. The k samples are a decoded sequence set. The decoded sequence set is obtained by encoding and compressing a type of video according to the encoding configuration scheme, and then decoding the encoded and compressed sequence set. The subjective truth value of the k samples is the subjective truth value of the image frames in the decoded sequence set. Using the k samples and the subjective truth values ​​of the k samples as the training set, a second model is obtained, and the prediction result of the second model is an approximation value that approximates the subjective truth value. Using the k samples as input, the second model is used to obtain approximation values ​​of the subjective truth for the k samples; Using the approximation values ​​of the k samples as a reference, the parameters of the k samples are fitted and calculated to obtain the first model. The first model is used to calculate a first approximation value that approximates the subjective truth value of the sample to be evaluated based on the parameters of the sample to be evaluated. The sample to be evaluated includes a video or an image to be evaluated. The parameters of the sample to be evaluated include at least one of the following parameters: feedback parameters of the network module, setting parameters of the cloud gaming module, and calculation parameters of the encoding and decoding module.

12. A training device for a first model, characterized in that, include: The first acquisition unit is used to acquire k samples, the subjective truth value of the k samples is acquired by subjective scoring, the k samples are a decoded sequence set, the decoded sequence set is obtained by encoding and compressing a type of video according to the encoding configuration scheme, and then decoding the encoded and compressed sequence set, the subjective truth value of the k samples is the subjective truth value of the image frame in the decoded sequence set; The first training unit is used to obtain a second model by using the k samples and the subjective truth values ​​of the k samples as a training set, and the prediction result of the second model is an approximation value that approximates the subjective truth value. The second acquisition unit is used to take the k samples as input and use the second model to obtain the approximation value of the subjective truth of the k samples; The second training unit is used to use the approximation values ​​of the k samples as a reference to fit and calculate the parameters of the k samples to obtain the first model. The first model is used to calculate a first approximation value that approximates the subjective truth value of the sample to be evaluated based on the parameters of the sample to be evaluated. The sample to be evaluated includes a video or an image to be evaluated. The parameters of the sample to be evaluated include at least one of the following parameters: feedback parameters of the network module, setting parameters of the cloud gaming module, and calculation parameters of the encoding and decoding module.

13. An electronic device, characterized in that, include: A processor, adapted to execute computer programs; A computer-readable storage medium storing a computer program that, when executed by the processor, implements the method for evaluating image / video quality based on approximation values ​​as described in any one of claims 1 to 9, or the training method for the first model as described in claim 11.

14. A computer-readable storage medium, characterized in that, Used to store computer programs that cause a computer to perform a method for evaluating image / video quality based on approximation values ​​as claimed in any one of claims 1 to 9, or a method for training a first model as claimed in claim 11.

Images