Video quality evaluation method and device, electronic equipment and storage medium

Abstract

The invention discloses a video quality evaluation method and device, electronic equipment and a storage medium, and is used for solving the technical problems that a current video quality evaluation method depends on an external truth value tag, the generalization ability is weak, and the interpretability is poor. The method comprises the following steps: acquiring a to-be-evaluated video; inputting a to-be-evaluated video into a generative world model pre-trained based on self-supervised learning, and generating a prediction error through forward propagation; and inputting the prediction error into a quality evaluator which performs parameter optimization in combination with an AI reward model, performing multi-dimensional quality evaluation on the prediction error through the quality evaluator, and outputting a comprehensive quality evaluation report of the to-be-evaluated video.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and signal processing technology, and in particular to a video quality assessment method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the widespread application of video compression, enhancement, and restoration technologies, the demand for video quality assessment is becoming increasingly urgent. Traditional video quality assessment methods typically rely on objective metrics with a reference (i.e., comparison with the original video). However, in practical applications, original lossless video is often unavailable, especially in areas such as video streaming, surveillance systems, and video restoration. Therefore, No-Reference Video Quality Assessment (NR-VQA) has gradually become a research hotspot due to its unique advantage of not requiring a reference video. The core objective of NR-VQA is to quantitatively assess video quality by analyzing the video content itself, without relying on the original lossless video as a reference.

[0003] Traditional methods rely on handcrafted features, resulting in poor generalization ability. Supervised learning-based methods heavily depend on large-scale human subjective rating datasets, which are not only costly and prone to subjective evaluations, but also struggle to quantify the multiple physical dimensions of video quality. Some recent methods have attempted to use reinforcement learning (RL) to fit a set of objective, non-differentiable engineering metrics. However, this still limits the model's optimization objective to a predefined, potentially incomplete set of metrics.

[0004] In summary, the main technical problems with the current technology are as follows:

[0005] Limitations of the evaluation paradigm: Current methods are inherently "discriminative," meaning they learn a mapping function from "video features" to "quality scores." This paradigm lacks a deep understanding of the intrinsic generation patterns of high-quality videos, leading to poor performance when faced with unseen types of distortion.

[0006] Dependence on "true value" labels: Whether it's subjective human scoring or objective engineering metrics, traditional techniques require an externally defined "correct answer" as a supervisory signal. This limits the learning ceiling of the model and prevents it from achieving true autonomous learning and evolution.

[0007] Lack of deep interpretability: While outputting one or more quality score vectors may have some effect, they do not provide fundamental diagnostic information and have poor interpretability.

[0008] Fixed optimization objectives: Current methods anchor optimization objectives to a fixed set of indicators, making it difficult for models to adapt to new quality dimensions or evaluation standards, and lacking flexibility and adaptability. Summary of the Invention

[0009] This invention provides a video quality assessment method, apparatus, electronic device, and storage medium to solve or partially solve the technical problems of current video quality assessment methods, such as reliance on external ground truth labels, weak generalization ability, and poor interpretability.

[0010] This invention provides a video quality assessment method, the method comprising:

[0011] Obtain the video to be evaluated;

[0012] The video to be evaluated is input into a generative world model pre-trained based on self-supervised learning, and a prediction error is generated through forward propagation.

[0013] The prediction error is input into a quality evaluator that combines an AI reward model for parameter optimization. The quality evaluator performs a multi-dimensional quality assessment of the prediction error and outputs a comprehensive quality assessment report of the video to be evaluated.

[0014] Optionally, the training process of the generative world model includes:

[0015] Construct a generative spatiotemporal world model framework;

[0016] An unlabeled historical video corpus is obtained, and based on the historical video corpus, the reconstruction error between real frames and predicted frames is minimized as the optimization objective. The prediction error of the video is used as the model output, and the generative spatiotemporal world model framework is trained through self-supervised learning to obtain a trained generative world model.

[0017] Optionally, the generative spatiotemporal world model framework adopts a hierarchical structure based on a combination of a state-space model and a convolutional encoder; the generative spatiotemporal world model framework includes an input preprocessing layer, a spatial encoding layer, a temporal state-space modeling layer, a prediction head, and an error reconstruction module.

[0018] Optionally, the quality evaluator is built based on a lightweight diagnostic model framework; the training process of the quality evaluator includes:

[0019] The prediction error data output by the generative world model during training is used as the model input to train the quality evaluator for multi-dimensional quality evaluation and generate a multi-dimensional quality evaluation report.

[0020] The quality assessment report is scored using an AI reward model based on constitutional rules, and a reward score is output.

[0021] Based on the reward scores, a reinforcement learning algorithm is used to optimize the parameters of the quality evaluator.

[0022] By combining the reward feedback of the AI ​​reward model based on constitutional rules, and through continuous iterative optimization of the quality evaluator, the final quality evaluator is obtained.

[0023] Optionally, the quality assessment report includes quantitative scores for multiple dimensions, as well as a video diagnostic text description in natural language; the multiple dimensions include video fidelity, smoothness, and texture quality; the video diagnostic text description is generated based on a preset text rule template or a lightweight natural language generation network.

[0024] Optionally, the training process of the AI ​​reward model based on constitutional rules includes:

[0025] Build an AI reward model;

[0026] The AI ​​reward model is trained by using the quality assessment report and the corresponding video clip as model input, the reward score as model output, and the preset video quality assessment constitutional rules.

[0027] By combining the optimization feedback from the quality evaluator and continuously iterating and optimizing the AI ​​reward model, the final AI reward model is obtained.

[0028] Optionally, the AI ​​reward model includes an input encoding layer, a cross-modal fusion layer, a constitutional rule item mapping layer, and a score calculation layer; the step of using the quality assessment report and the corresponding input video segment as model input, and the reward score as model output, combined with preset video quality assessment constitutional rules, to train the AI ​​reward model for reward scoring includes:

[0029] The quality assessment report and the corresponding input video segment are input into the input coding layer. Spatial features are extracted from the input video segment through the input coding layer to obtain temporal features. At the same time, the quality assessment report is converted into text features.

[0030] The cross-modal fusion layer performs cross-modal attention learning on the temporal features and the text features to learn the correspondence between the quality assessment report and the true quality deviation of the input video segment, forming a joint feature representation;

[0031] The joint feature representation is input into the constitutional rule item mapping layer, and multi-dimensional mapping is performed in combination with the preset video quality assessment constitutional rules to obtain the rule mapping output values ​​of each dimension of the constitutional rules; the multi-dimensional mapping includes prediction fidelity mapping, error information entropy mapping, temporal consistency mapping and frequency domain health mapping;

[0032] The final reward score is obtained by performing a weighted linear combination of the output values ​​of each rule mapping by the scoring calculation layer.

[0033] The present invention also provides a video quality assessment device, comprising:

[0034] The video acquisition unit is used to acquire the video to be evaluated.

[0035] The prediction error generation unit is used to input the video to be evaluated into a generative world model pre-trained based on self-supervised learning, and generate prediction error through forward propagation.

[0036] A multi-dimensional quality assessment unit is used to input the prediction error into a quality evaluator that combines an AI reward model for parameter optimization. The quality evaluator performs a multi-dimensional quality assessment on the prediction error and outputs a comprehensive quality assessment report of the video to be evaluated.

[0037] The present invention also provides an electronic device, the device comprising a processor and a memory:

[0038] The memory is used to store program code and transmit the program code to the processor;

[0039] The processor is configured to execute the video quality assessment method as described above, according to the instructions in the program code.

[0040] The present invention also provides a computer-readable storage medium for storing program code for performing the video quality assessment method as described in any of the preceding claims.

[0041] As can be seen from the above technical solutions, the present invention has the following advantages:

[0042] This paper presents a video quality assessment method. First, the video to be assessed is input into a generative world model pre-trained based on self-supervised learning, generating prediction errors through forward propagation. Then, the prediction errors are input into a quality evaluator with parameters optimized using an AI reward model. The quality evaluator performs a multi-dimensional quality assessment of the prediction errors and outputs a comprehensive quality assessment report for the video. Thus, by using a generative world model pre-trained based on self-supervised learning that understands the "physical laws" of natural videos, it can generate prediction errors with greater objectivity without any manual annotation. Next, a quality evaluator trained using reinforcement learning combined with a constitutional AI reward model based on "first principles" performs a multi-dimensional quality assessment of the prediction errors, quickly outputting a comprehensive quality assessment report. By implementing this technical solution, the assessment system can autonomously and interpretably diagnose the degree of deviation between the video signal and its inherent ideal model without any manual labels or predefined indicators, thereby achieving a fundamental quantification of video quality. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a schematic diagram of a multi-dimensional quality score generation process;

[0045] Figure 2 This is a schematic diagram of a process for updating a quality evaluator based on a constitutional AI reward model and RLAIF.

[0046] Figure 3 This is an example diagram illustrating a real-world application scenario for video quality assessment.

[0047] Figure 4 A flowchart illustrating the steps of a video quality assessment method;

[0048] Figure 5 A schematic diagram illustrating the overall process of a video quality assessment method;

[0049] Figure 6 This is a structural block diagram of a video quality assessment device. Detailed Implementation

[0050] This invention provides a video quality assessment method, apparatus, electronic device, and storage medium to solve or partially solve the technical problems of current video quality assessment methods, such as reliance on external ground truth labels, weak generalization ability, and poor interpretability.

[0051] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0052] To enable those skilled in the art to better understand the technical solutions provided in the embodiments of the present invention, some of the technical features involved in the solutions are briefly described first:

[0053] VQA (Video Quality Assessment): A technology designed to automatically quantify the subjective or objective perceived quality of video.

[0054] Generative World Model: A type of deep learning model. It primarily constructs an internal, predictable, and simulable representation of the world by learning the dynamic changes in the environment. In the video domain, generative world models learn the "physical" laws governing natural video signals.

[0055] RLAIF (Reinforcement Learning from AI Feedback): A training paradigm in which the policy of a reinforcement learning agent is optimized through feedback signals provided by an AI reward model, rather than relying on human annotations or a fixed reward function.

[0056] Constitutional AI: A method for training AI models (especially reward models). It avoids human bias by having the model self-optimize and align itself by following a clear set of fundamental principle-based "constitutional" rules.

[0057] State Space Model: An advanced deep learning architecture for processing sequential data (such as video) that can efficiently capture long-range spatiotemporal dependencies.

[0058] As an example, traditional methods for video quality assessment rely on handcrafted features, resulting in poor generalization ability. Supervised learning-based methods heavily depend on large-scale human subjective rating datasets, which are not only costly and prone to subjective evaluations, but also difficult to quantify the multiple physical dimensions of video quality. Some recent methods have attempted to use reinforcement learning (RL) to fit a set of objective, non-differentiable engineering metrics. However, this still limits the model's optimization objective to a predefined, potentially incomplete set of metrics.

[0059] In summary, the main technical problems with the current technology are as follows:

[0060] Limitations of the evaluation paradigm: Current methods are inherently "discriminative," meaning they learn a mapping function from "video features" to "quality scores." This paradigm lacks a deep understanding of the intrinsic generation patterns of high-quality videos, leading to poor performance when faced with unseen types of distortion.

[0061] Dependence on "true value" labels: Whether it's subjective human scoring or objective engineering metrics, traditional techniques require an externally defined "correct answer" as a supervisory signal. This limits the learning ceiling of the model and prevents it from achieving true autonomous learning and evolution.

[0062] Lack of deep interpretability: While outputting one or more quality score vectors may have some effect, they do not provide fundamental diagnostic information and have poor interpretability.

[0063] Fixed optimization objectives: Current methods anchor optimization objectives to a fixed set of indicators, making it difficult for models to adapt to new quality dimensions or evaluation standards, and lacking flexibility and adaptability.

[0064] Therefore, one of the core inventive points of this invention is to provide a novel video quality assessment method based on self-supervised learning, generative modeling, and artificial intelligence feedback, for achieving automated, multi-dimensional, and interpretable diagnosis and assessment of video quality. Specifically, it trains a generative world model capable of understanding the "physical laws" of natural videos, and combines a constitutional AI reward model based on "first principles" with a quality evaluator trained using reinforcement learning based on the RLAIF framework. This allows the assessment system to autonomously and interpretably diagnose the degree of deviation between the video signal and its inherent ideal model without any manual labels or predefined metrics, thereby achieving a fundamental quantification of video quality.

[0065] This invention provides a video quality assessment system architecture. This architecture mainly comprises three core modules: a generative spatiotemporal world model, a constitutional AI reward model, and a quality evaluator. Its training and inference process adopts the innovative three-stage paradigm shown below.

[0066] Phase 1: Self-supervised pre-training of generative world models

[0067] The goal of this stage is to enable the model to learn the inherent spatiotemporal dynamics of high-quality videos. Prediction error is used as the core basis for quality deviation.

[0068] The first step is model building. We begin by constructing a generative spatiotemporal world model, for example, using a cutting-edge Mamba architecture or a spatiotemporal Vision Transformer. This model is designed to predict the content of the next frame F(t+1) given a series of historical video frames F(tn), ..., F(t).

[0069] Secondly, self-supervised learning is employed. The generative spatiotemporal world model is trained using a massive, unlabeled, high-quality video corpus. The optimization objective of the model is to minimize the reconstruction error between its predicted frame F_pred(t+1) and the real frame F_real(t+1).

[0070] Through the process described above, the world model can learn the underlying "physical laws" of natural video signals. These laws include the continuity of object motion, changes in lighting, and texture details. This forms a powerful internal representation of how an "ideal video" should evolve.

[0071] Phase One thus provides a video quality assessment method based on a generative world model. Through self-supervised learning, a spatiotemporal world model is trained to learn the dynamic generation patterns of high-quality videos, and the model's prediction error on the input video is used as the fundamental basis for measuring quality deviation in subsequent quality assessments. Phase One establishes a novel generative paradigm for video quality assessment, basing assessment standards on an understanding of the inherent patterns in the data, fundamentally improving the objectivity and robustness of the assessment.

[0072] Figure 1 A schematic diagram illustrating the generation process of a multi-dimensional quality score is shown. Among them, Figure 1 The processing flow before multi-dimensional quality scoring corresponds to Stage 1. It primarily illustrates the construction and self-supervised pre-training process of the generative world model. The prediction error output by the generative world model will serve as the fundamental basis for subsequent multi-dimensional quality score evaluation.

[0073] Phase Two: Training an AI Reward Model Based on the "Constitution"

[0074] This stage is one of the core innovations of this invention. Its aim is to construct an objective quality judge that requires no human data. Objective scoring is achieved by training an AI reward model to follow constitutional rules.

[0075] First, the definition of the "constitution" is provided. In this embodiment of the invention, instead of using traditional methods of human scoring for video quality assessment, a "VQA constitution" based on the first principles of signal processing and visual perception is defined. This constitution consists of a series of computable and quantifiable rules. For example:

[0076] Constitutional Rule 1: Prediction Fidelity. This rule requires that high-quality video should closely match the predictions of the world model. The lower the energy of the prediction error (F_real - F_pred), the higher the quality.

[0077] Constitutional Rule 2: Information Complexity. This rule requires that the prediction error signal itself should exhibit high entropy, white noise-like characteristics. If structured, low-entropy patterns (such as large areas of blur or repetitive block artifacts) appear in the error signal, it indicates a quality defect.

[0078] Constitutional Rule 3: Timing Consistency. This rule requires that the variation in prediction error between consecutive frames should be smooth. Drastic, non-physical jumps in the error signal usually correspond to frame loss, jitter, or coding artifacts.

[0079] Constitutional Rule 4: Frequency Domain Health. This rule requires that the Fourier transform spectrum of the prediction error should not contain peaks at specific frequencies. In practice, these peaks are often directly related to periodic artifacts introduced by compression algorithms, such as the Discrete Cosine Transform Block Effect (DCT).

[0080] It should be noted that the above four constitutional rules are only one optional embodiment and are not exhaustive. In practical applications, other rules based on signal statistical characteristics, visual perception mechanisms, or frequency domain distribution patterns can be flexibly added or replaced according to different needs of video quality assessment and model iteration processes. This embodiment of the invention does not limit the number of rules, nor does it limit the specific type of rules, but emphasizes the scalability and adaptability of the mechanism.

[0081] Furthermore, regarding the conditions for fulfilling the rules and the hierarchical evaluation, these rules are not required to be absolutely satisfied simultaneously. Instead, they serve as a set of weighted feature indicators used to collectively constitute the evaluation system for video quality. In practical implementation, the following strategies can be freely and flexibly selected based on actual needs:

[0082] When multiple rules are met and the deviation is low, the video is rated as high quality;

[0083] When some rules are abnormal or deviate significantly, they are judged as medium or low quality.

[0084] Different rules can be assigned different weights, and the contribution of each rule to the final evaluation is hierarchical;

[0085] The weights can be automatically learned and dynamically adjusted by the AI ​​reward model during training, rather than being fixed.

[0086] Therefore, the value of Phase Two of this invention lies in proposing a quality assessment mechanism based on constitutional principles. Its essence is to achieve an objective, transparent, and interpretable quality assessment process by relying on a set of calculable, scalable, and weighted internal quality rules, rather than external manual scoring.

[0087] The second step is training the reward model. An AI reward model is trained. The input to this model is a video and a quality assessment report generated by a quality evaluator, and the output is a reward score (scalar reward value). In this embodiment of the invention, the goal of training the AI ​​reward model is to make it award higher rewards to quality assessment reports that better conform to the aforementioned "VQA constitution." For example, by comparing two different assessment reports, the reward model is trained to favor quality assessment reports that indicate low prediction fidelity and low error information complexity.

[0088] Phase Two proposes a constitutional AI reward model designed for VQA tasks. This reward model does not rely on human scoring for training; instead, it self-aligns and optimizes by following a set of constitutional rules based on first principles of signal processing and visual perception (such as prediction fidelity, information complexity, and temporal consistency). This invention, for the first time, applies the concept of Constitutional AI to the field of video quality assessment, creating a completely autonomous, objective, and scalable "quality judge," thus solving the problem of dependence on human data with subjective opinions.

[0089] Phase 3: Fine-tuning of the RLAIF-based quality evaluator

[0090] This stage primarily utilizes a pre-trained AI reward model to guide and optimize the final quality evaluator. Specifically, based on the RLAIF framework, the quality evaluator is optimized using feedback signals from the AI ​​reward model. During inference, the system receives the video to be evaluated, calculates the prediction error using a world model, and then the evaluator outputs a comprehensive quality assessment report containing multi-dimensional quantitative scores and natural language diagnostics.

[0091] The first step is strategy exploration. For an input video, the quality evaluator (which can be the world model itself or a lighter diagnostic model) generates a multi-dimensional quality assessment report. This report includes not only quantified scores across multiple dimensions (such as fidelity, smoothness, and texture quality) but also a diagnostic description in natural language. For example, "Between 0:15 and 0:20 seconds, prediction fidelity significantly decreased due to motion blur; blocky patterns associated with H.264 compression were detected in the background region."

[0092] The lighter diagnostic model proposed in this invention can be preferably implemented using the following specific structure. This model mainly consists of the following components:

[0093] Feature extraction layer. A shallow CNN (Convolutional Neural Network) or a lightweight MobileNet / ConvNeXt-Tiny is used to extract spatiotemporal local features from the video error map and inter-frame difference map.

[0094] Temporal modeling layer. Employs 2-4 layers of lightweight Transformer or state-space models (such as small-scale Mambablocks) to model the continuity of errors in the temporal dimension.

[0095] Rule-based feature mapping layer. Features are mapped to the following dimensions: prediction bias energy, error information entropy, temporal smoothness, and spectral anomaly intensity.

[0096] Multi-task output header. Primarily used to output multi-dimensional scoring vectors (e.g., [fidelity, fluency, texture quality, overall quality]), and latent semantic embeddings for subsequent text diagnostics.

[0097] Among them, the parameter scale of the above-mentioned diagnostic model is significantly smaller than that of the generative world model in stage one, making it suitable for quickly outputting quality assessment results during the inference stage.

[0098] Furthermore, natural language description generation can employ the following mechanisms.

[0099] During the training phase, the quality evaluator outputs vector-based quality features and generates an initial text description. This process can be achieved in two ways: First, by using a rule template (e.g., "Phenomenon X occurs during time period T"). Second, by using a lightweight natural language generation network (Small Language Decoder). The Small Language Decoder employs 2-4 layers of Transformer Decoder, taking the quality feature embedding as input and outputting a natural language text description.

[0100] Understandably, lightweight natural language generation networks are not large-scale language models, but rather small text generators integrated with evaluation tasks. By incorporating reward feedback, the AI ​​reward model provides feedback on the plausibility of quality defects described in the text, making it more accurately reflect true distortion features. Therefore, this invention can explicitly provide a natural language output mechanism, rather than simply offering numerical scoring.

[0101] Secondly, there is AI feedback. The quality assessment report generated by the quality assessor is submitted to the constitutional AI reward model, which assigns a reward score according to the "VQA Constitution".

[0102] Finally, there is policy optimization. In this embodiment of the invention, advanced reinforcement learning algorithms, such as PPO (Proximal Policy Optimization) or GRPO (Generalized Reinforcement Policy Optimization), are employed to guide the parameter updates of the quality evaluator. Specifically, PPO or GRPO is used as a trigger to optimize the parameters of the quality evaluator based on the reward score feedback signal from the AI ​​reward model. This process iterates continuously, enabling the evaluation reports generated by the quality evaluator to increasingly accurately reflect the degree of deviation of the video signal from the "ideal" world model, thereby gaining higher recognition from the AI ​​reward model.

[0103] Figure 2 A schematic diagram illustrates a process for updating a quality evaluator based on a constitutional AI reward model and RLAIF. Among other things, Figure 2 This corresponds to Phases Two and Three. It primarily demonstrates the construction of a constitutional AI reward model, the RLAIF training process, and the strategy optimization path. Among them, Figure 2 The small diagram on the left shows the SFT (Supervised Fine-Tuning) process for the quality evaluator model (a multimodal large model). Its purpose is to initialize and fine-tune the quality evaluator before RLAIF training, enabling it to have basic quality scoring and description generation capabilities.

[0104] The key points of this implementation method include: training with existing natural video data; loss function including prediction error approximation loss and rule consistency loss; and fine-tuning results serving as the initial strategy before reinforcement learning.

[0105] Figure 2The small diagram on the right illustrates the policy optimization process using reinforcement learning combined with AI rewards. PPO or GRPO can be used as the policy optimization algorithm. In some embodiments, PPO is preferred due to its maturity and stability. GRPO, as an alternative algorithm, can achieve the same optimization effect.

[0106] The quality evaluator model used in Phase 3 can be implemented using currently available deep learning model architectures (such as the Visual Transformer, the Spatio-Temporal Spatial Transformer Model, or a multimodal encoder). Its basic network structure follows existing publicly available information, and no original improvements are claimed to be made to the underlying structure and network topology. It should be noted that the innovation of this invention does not lie in redesigning the underlying architecture of the model, but rather in using the RLAIF framework to enable the parameters of the quality evaluator to adaptively optimize under the feedback of constitutional AI rewards, thereby making its output video quality diagnostic results more consistent with the rules of the "VQA Constitution".

[0107] Thus, Phase 3 provides a reinforcement learning training framework based on AI reward feedback (RLAIF). By utilizing feedback signals generated by a constitutional AI reward model, it guides and optimizes the strategy of a quality evaluator, ensuring that its generated quality assessment reports (including quantitative scores and natural language diagnostics) are highly consistent with "constitutional" principles. By constructing a closed-loop, automated optimization process, the quality evaluator can learn to perform complex, multi-dimensional, and interpretable quality diagnostics without explicit supervision, achieving autonomous evolution of assessment capabilities.

[0108] On the other hand, Phase Three also provides a system capable of outputting a comprehensive quality report that includes quantitative scores and natural language diagnostics. Compared to traditional technologies, this embodiment of the invention achieves diagnostic value far exceeding that of traditional VQA systems. The generated natural language diagnostic report greatly enhances the interpretability and usability of the evaluation results, directly serving engineers in troubleshooting and system optimization.

[0109] After training the relevant model, it can be applied to the actual video quality assessment inference stage. Specifically, after training is complete, the system receives the video to be evaluated, which is then forward-propagated through a generative world model to calculate the prediction error. Subsequently, the quality evaluator directly analyzes the prediction error signal and instantly outputs a comprehensive quality assessment report containing multi-dimensional quantification scores and natural language diagnostic text.

[0110] For example, Figure 3The diagram illustrates a practical application scenario for video quality assessment. It demonstrates the deployment method and operation process of the technical solution provided by this invention in a real business system (such as a surveillance video quality assessment system), including input video, feature decoding, multimodal quality assessment, and final multi-dimensional quality score output.

[0111] This invention trains a generative world model using self-supervised learning to learn the dynamic generation patterns of high-quality videos, and uses the model's prediction error on the input video as the fundamental criterion for measuring its quality deviation. Simultaneously, by establishing a novel generative paradigm for video quality assessment, the evaluation criteria are based on an understanding of the inherent patterns in the data, fundamentally improving the objectivity and robustness of the assessment. The method provided in this invention overcomes the limitations of traditional methods, deeply integrating large-scale multimodal models with reinforcement learning techniques to form a practical video quality assessment solution. This satisfies both the need for high-precision quality diagnosis and adapts to large-scale industrial video processing scenarios, meeting the business upgrade needs in the video technology field.

[0112] Based on the content introduced above, refer to Figure 4 The diagram illustrates a flowchart of a video quality assessment method provided by an embodiment of the present invention, which may specifically include the following steps:

[0113] Step 401: Obtain the video to be evaluated;

[0114] In practical applications, when video quality assessment is required, it is necessary to first obtain the video to be assessed.

[0115] Step 402: Input the video to be evaluated into a generative world model pre-trained based on self-supervised learning, and generate prediction error through forward propagation;

[0116] The video to be evaluated is then input into a generative world model pre-trained based on self-supervised learning, and prediction errors are generated through forward propagation.

[0117] In some embodiments, in conjunction with the foregoing discussion, the training process of the generative world model may include: constructing a generative spatiotemporal world model framework; obtaining an unlabeled historical video corpus, and based on the historical video corpus, with the optimization objective of minimizing the reconstruction error between real frames and predicted frames, using the prediction error of the video as the model output, training the generative spatiotemporal world model framework through self-supervised learning to obtain a trained generative world model.

[0118] In some embodiments, the generative spatiotemporal world model framework adopts a hierarchical structure based on a combination of a state-space model (Mamba) and a convolutional encoder. The generative spatiotemporal world model framework mainly includes an input preprocessing layer, a spatial encoding layer, a temporal state-space modeling layer, a prediction head, and an error reconstruction module. Specifically:

[0119] Input preprocessing layer: The input historical video frame sequence F(tn), \dots, F(t) is uniformly resampled to a fixed resolution (e.g., 224×224) and frame rate. Each frame is normalized and data augmented (e.g., random cropping, flipping, etc.) to improve the world model's ability to fit the natural video distribution.

[0120] Spatial encoding layer: A CNN encoder consisting of 3-4 convolutional layers is used to extract spatial features for each frame. Each convolutional layer is followed by batch normalization and a non-linear activation function (such as GELU), with the number of channels increasing layer by layer (e.g., from 64→128→256). Each frame is encoded as a feature map of size (H', W', C), and flattened in the spatial dimension to obtain a patch sequence of length H' × W'.

[0121] Spatiotemporal state space modeling layer (Mamba main body): Concatenates patch sequences from different time steps along the temporal dimension, introducing explicit temporal position encoding. Stacked L-layer state space blocks (e.g., L=8~16), each layer including a linear projection layer, selection gate structure, state space evolution module, and residual connection and normalization layer. Compared to standard Mamba, this embodiment adds a cross-frame residual channel (used for differential operations on the hidden states of adjacent time steps) to each state space block, and fuses it back to the main branch through a lightweight MLP (Multi-Layer Perceptron) to enhance sensitivity to subtle temporal distortions.

[0122] Prediction Head: The hidden state at the final time t is used as a condition and input into an upsampled decoder (the decoder consists of several layers of transposed convolution or upsampled + convolution) to gradually restore the prediction frame \hat{F}(t+1) to the same resolution as the input frame. During the decoding process, skip connections are introduced to concatenate the intermediate features of the spatial coding layer with the decoded features of the corresponding scale to improve the reconstruction capability of texture details.

[0123] Error Reconstruction Module: Calculates the pixel-level error map E = F_{\text{real}}(t+1) - \hat{F}(t+1) between the predicted frame and the ground truth frame. The error map is further encoded using a shallow CNN, outputting multi-scale error features for subsequent "constitutional" feature calculations. These features include: local energy distribution, temporal difference, and frequency domain statistics.

[0124] The key parameters of the above framework are set as follows: the length of the input time window n is preferably 4 to 8 frames; the number of state space layers L is preferably 8 to 16 layers to balance modeling capability and inference overhead; and the number of feature channels C is preferably 256 to 512 to ensure the ability to represent complex motion and texture patterns.

[0125] Through the above structural design, compared with the traditional Mamba / spatiotemporal Vision Transformer architecture, this embodiment of the invention introduces a cross-frame residual channel to enhance sensitivity to non-stationary temporal distortion. Simultaneously, by explicitly designing error reconstruction and feature extraction branches, subsequent "constitutional principle" calculations and quality diagnostics are facilitated, making it more suitable for the task scenario of video quality assessment based on prediction errors.

[0126] The spatiotemporal Vision Transformer architecture mentioned in this invention can be divided into the following parts:

[0127] Input Layer: The input is video data (continuous RGB frames). Each frame is segmented into fixed-size image patches through the Patch Embedding layer and mapped to an embedding vector. Temporal information is encoded using either Temporal Embedding or Position Embedding.

[0128] Spatio-Temporal Feature Extraction: The model introduces two methods to handle spatio-temporal information. The first is 3D convolution, which uses 3D convolutional layers to extract spatio-temporal features in the initial stage. The second is a hierarchical attention mechanism, which uses spatial attention and temporal attention to process spatial and temporal information respectively. For example, the model first extracts preliminary spatio-temporal features through 3D convolution, and then inputs them into the Transformer layer.

[0129] Transformer layers (Transformer Blocks): These are used to capture global spatiotemporal relationships. Each Transformer layer includes a Multi-Head Self-Attention Mechanism (MHSA), a Feed-Forward Network (FFN), Layer Normalization, and Residual Connections. The MHSA captures spatial and temporal dependencies; the FFN performs non-linear transformations to enhance feature representation; and layer normalization and residual connections stabilize training.

[0130] Spatio-Temporal Attention Mechanisms: To better capture spatio-temporal relationships in videos, a dedicated spatio-temporal attention module can be designed. For example, spatial attention focuses on the spatial features of each frame, while temporal attention focuses on the temporal dependencies between different frames. These attention modules can process spatial and temporal information in parallel or sequentially.

[0131] Classification Layer: The final feature vectors are classified through a fully connected layer (FC layer). Furthermore, temporal pooling can be introduced to further compress the temporal dimension information.

[0132] The spatiotemporal Vision Transformer can employ sequential, parallel, or cross-layer connections. Sequential connections involve first extracting preliminary features from video data using 3D convolutions, then inputting the data into the Transformer layer for global relational modeling. Parallel connections allow spatial and temporal attention modules to process features in parallel within the Transformer layer, with the results combined through a fusion layer. Cross-layer connections allow features from different layers to interact via residual connections or skip connections.

[0133] Step 403: Input the prediction error into a quality evaluator that combines an AI reward model for parameter optimization. The quality evaluator performs a multi-dimensional quality assessment of the prediction error and outputs a comprehensive quality assessment report of the video to be evaluated.

[0134] Finally, the prediction error is input into a quality evaluator that combines an AI reward model for parameter optimization. The quality evaluator performs a multi-dimensional quality assessment of the prediction error and outputs a comprehensive quality assessment report of the video to be evaluated.

[0135] In some embodiments, the quality evaluator can be built based on a lightweight diagnostic model framework. The training process of the quality evaluator may include the following steps S01 to S04:

[0136] Step S01: Use the prediction error data output by the generative world model during training as the model input to train the quality evaluator for multi-dimensional quality evaluation and generate a multi-dimensional quality evaluation report.

[0137] Step S02: Use an AI reward model based on constitutional rules to score the quality assessment report and output the reward score;

[0138] Step S03: Based on the reward score, use a reinforcement learning algorithm to optimize the parameters of the quality evaluator;

[0139] Step S04: Combining the reward feedback of the AI ​​reward model based on constitutional rules, the final quality evaluator is obtained through continuous iterative optimization of the quality evaluator.

[0140] Based on the preceding discussion, the quality assessment report includes quantitative scores across multiple dimensions, as well as a video diagnostic text description in natural language. These dimensions include video fidelity, smoothness, and texture quality; the video diagnostic text description can be generated based on a pre-defined text rule template or a lightweight natural language generation network.

[0141] In some embodiments, the training process of an AI reward model based on constitutional rules may include the following steps S11 to S13:

[0142] Step S11: Construct an AI reward model;

[0143] Step S12: Using the quality assessment report and the corresponding input video segment as the model input, and the reward score as the model output, the AI ​​reward model is trained by combining the preset video quality assessment constitutional rules.

[0144] Step S13: Combining the optimization feedback from the quality evaluator, the final AI reward model is obtained through continuous iterative optimization of the AI ​​reward model.

[0145] Furthermore, the AI ​​reward model may include an input encoding layer, a cross-modal fusion layer, a constitutional rule item mapping layer, and a score calculation layer. Therefore, the implementation process in step S12, which uses the quality assessment report and its corresponding input video segment as model input, and the reward score as model output, combined with preset video quality assessment constitutional rules, to train the AI ​​reward model for reward scoring, may include the following steps S21 to S24:

[0146] Step S21: Input the quality assessment report and the corresponding input video segment into the input coding layer. Extract spatial features from the input video segment through the input coding layer to obtain temporal features, and convert the quality assessment report into text features.

[0147] For the processing flow of the input encoding layer, the input video segment is first processed by a CNN encoder consistent with the generative world model to extract spatial features and obtain a temporal feature sequence (video error features).

[0148] The natural language quality assessment report generated by the quality evaluator is converted into text features represented by vectors through a text encoding model (such as Transformer / BERT (Bidirectional Encoder Representations from Transformers) / LLM embedding (Large Language Model Embedding Model)).

[0149] Step S22: Perform cross-modal attention learning on temporal features and text features through a cross-modal fusion layer to learn the correspondence between the quality assessment report and the actual quality deviation of the input video segment, forming a joint feature representation;

[0150] The cross-modal fusion layer preferably employs a multimodal cross-attention structure. Text features are used as the query, and video error features are used as the key / value, enabling the model to learn the correspondence between the quality assessment report and the actual video quality deviation.

[0151] The outputs of each attention head are normalized and stabilized using a residual connection and a LayerNorm. Video error features and text features are concatenated in the feature space or fused across modalities to form a joint feature representation.

[0152] Step S23: Input the joint feature representation into the constitutional rule item mapping layer, and perform multi-dimensional mapping in combination with the preset video quality assessment constitutional rules to obtain the rule mapping output values ​​of each dimension of the constitutional rules; the multi-dimensional mapping includes prediction fidelity mapping, error information entropy mapping, temporal consistency mapping and frequency domain health mapping;

[0153] The rule item mapping layer is a rule feature explicit mapping module used to map various rules in the "VQA Constitution". Among them, mapping dimension 1 corresponds to prediction fidelity; mapping dimension 2 corresponds to error information entropy; mapping dimension 3 corresponds to temporal consistency; and mapping dimension 4 corresponds to frequency domain health.

[0154] Each mapping branch contains a shallow MLP (1-2 fully connected layers + GELU activation function) for extracting rule-sensitive features.

[0155] Step S24: The final reward score is obtained by performing a weighted linear combination of the output values ​​of each rule mapping through the score calculation layer.

[0156] In the scoring layer, all rule mapping outputs are weighted linearly combined to obtain the final reward score.

[0157] The weight vector can be fixed or learned automatically during training. The score calculation layer ultimately outputs a scalar reward value to guide subsequent RLAIF training.

[0158] As an example, some key parameter settings during the training process of an AI reward model can be: the video encoding feature dimension is preferably 256~512; the text encoding dimension is preferably 768~1024; the number of cross-modal attention heads is preferably 4~8; the MLP hidden layer size is preferably 256~512; and the output reward value is a single-dimensional floating-point scalar.

[0159] This structural design ensures that the descriptions in the quality assessment report accurately correspond to actual video distortion patterns. Simultaneously, constitutional rules are physically interpretable through an explicit mapping model layer. Furthermore, the AI ​​reward model does not rely on manual labeling but rather self-monitors based on rule fulfillment.

[0160] This invention provides a novel video quality assessment method based on self-supervised learning, generative modeling, and artificial intelligence feedback. First, a generative world model capable of understanding the "physical laws" of natural videos is trained. Then, a constitutional AI reward model based on "first principles" is used, combined with a quality evaluator trained using reinforcement learning based on the RLAIF framework. This allows the evaluation system to autonomously and interpretably diagnose the degree of deviation between the video signal and its inherent ideal model without any human labels or predefined metrics, thereby achieving a fundamental quantification of video quality.

[0161] Implementing the technical solution provided by this invention can bring about the following technical effects:

[0162] Paradigm innovation with strong objectivity: Shifting from discriminative to generative models, evaluation criteria originate from learning the inherent patterns of video data, rather than external labels. The AI ​​feedback mechanism, based on a "constitution," completely eliminates reliance on any form of manually labeled data (subjective or objective), exhibiting greater objectivity and autonomy.

[0163] Deep interpretability: It can generate natural language diagnostic reports that not only indicate the quality but also explain "why" the video is bad, pinpointing the specific time and artifact type, providing a fundamental basis for troubleshooting.

[0164] Robustness to unknown distortions: Since the model learns the "first principles" of the video rather than features of a specific type of distortion, it has a stronger ability to generalize and detect novel and hybrid distortions that have not appeared in the training set.

[0165] The model can evolve autonomously: the entire training framework is self-consistent and automated. As higher-quality video data is continuously used to pre-train the world model, and as the "VQA constitution" is continuously improved, the entire evaluation system can continuously and autonomously improve its performance ceiling without additional manual annotation costs.

[0166] The technical solutions provided in this invention are applicable to the automated, multi-dimensional, and interpretable diagnosis and evaluation of various video qualities in video scenarios. For example, they can be applied to image quality inspection, helping users automate video quality monitoring and reducing manual inspection costs. Furthermore, this invention abandons the traditional discriminative evaluation paradigm, eliminating the reliance on manual annotation in traditional video quality evaluation. It possesses strong objectivity, in-depth interpretability, and robustness to unknown distortions, and can be widely applied to quality evaluation scenarios such as video surveillance and streaming media.

[0167] For better explanation, refer to Figure 5 This diagram illustrates the overall flow of a video quality assessment method provided by an embodiment of the present invention. It should be noted that this embodiment only provides a brief description of the general process of video quality assessment. The specific implementation process of each step can be understood by referring to the relevant content in the foregoing embodiments, and will not be elaborated upon here. It is understood that the present invention does not impose any limitations on this.

[0168] Step 501: Obtain an unlabeled historical video corpus, and based on the historical video corpus, with the optimization objective of minimizing the reconstruction error between real frames and predicted frames, use the prediction error of the video as the model output, train the generative spatiotemporal world model framework through self-supervised learning, and obtain the trained generative world model.

[0169] Step 502: Use the prediction error data output by the generative world model during training as the model input to train the quality evaluator for multi-dimensional quality evaluation and generate a multi-dimensional quality evaluation report.

[0170] Step 503: Use an AI reward model based on constitutional rules to score the quality assessment report, output the reward score, and use a reinforcement learning algorithm to optimize the parameters of the quality assessor based on the reward score;

[0171] Step 504: Combining the reward feedback of the AI ​​reward model based on constitutional rules, and through continuous iterative optimization of the quality evaluator, a well-trained quality evaluator is obtained;

[0172] Step 505: Obtain the video to be evaluated and input it into the trained generative world model to generate prediction error through forward propagation;

[0173] Step 506: Input the prediction error into the trained quality evaluator, perform multi-dimensional quality evaluation on the prediction error through the quality evaluator, and output a comprehensive quality evaluation report of the video to be evaluated.

[0174] Reference Figure 6 The diagram illustrates a structural block diagram of a video quality assessment device provided in an embodiment of the present invention, which may specifically include:

[0175] The video acquisition unit 601 is used to acquire the video to be evaluated.

[0176] The prediction error generation unit 602 is used to input the video to be evaluated into a generative world model pre-trained based on self-supervised learning, and generate prediction error through forward propagation.

[0177] The multi-dimensional quality assessment unit 603 is used to input the prediction error into a quality evaluator that combines an AI reward model for parameter optimization, and to perform multi-dimensional quality assessment on the prediction error through the quality evaluator, and output a comprehensive quality assessment report of the video to be evaluated.

[0178] In one alternative embodiment, the apparatus further includes a generative world model training unit; the generative world model training unit is specifically used for:

[0179] Construct a generative spatiotemporal world model framework;

[0180] An unlabeled historical video corpus is obtained, and based on the historical video corpus, the reconstruction error between real frames and predicted frames is minimized as the optimization objective. The prediction error of the video is used as the model output, and the generative spatiotemporal world model framework is trained through self-supervised learning to obtain a trained generative world model.

[0181] In one optional embodiment, the generative spatiotemporal world model framework adopts a hierarchical structure based on a combination of a state-space model and a convolutional encoder; the generative spatiotemporal world model framework includes an input preprocessing layer, a spatial encoding layer, a temporal state-space modeling layer, a prediction head, and an error reconstruction module.

[0182] In one optional embodiment, the quality evaluator is built based on a lightweight diagnostic model framework; the device further includes a quality evaluator training unit; the quality evaluator training unit specifically includes:

[0183] The multi-dimensional quality assessment training unit is used to take the prediction error data output by the generative world model during the training process as the model input, train the quality evaluator for multi-dimensional quality assessment, and generate a multi-dimensional quality assessment report.

[0184] The reward scoring unit is used to score the quality assessment report using an AI reward model based on constitutional rules and output a reward score.

[0185] The parameter optimization unit is used to optimize the parameters of the quality evaluator using a reinforcement learning algorithm based on the reward score.

[0186] The quality evaluator acquisition unit is used to combine the reward feedback of the AI ​​reward model based on constitutional rules and obtain the final quality evaluator through continuous iterative optimization.

[0187] In one optional embodiment, the quality assessment report includes quantitative scores for multiple dimensions, as well as a video diagnostic text description in natural language; the multiple dimensions include video fidelity, smoothness, and texture quality; the video diagnostic text description is generated based on a preset text rule template or a lightweight natural language generation network.

[0188] In one optional embodiment, the reward scoring unit includes an AI reward model training unit; the AI ​​reward model training unit specifically includes:

[0189] AI reward model building unit, used to build an AI reward model;

[0190] The reward scoring training unit is used to train the AI ​​reward model by taking the quality assessment report and the corresponding input video segment as model input, the reward score as model output, and combining the preset video quality assessment constitutional rules.

[0191] The AI ​​reward model acquisition unit is used to combine the optimization feedback of the quality evaluator and obtain the final AI reward model through continuous iterative optimization of the AI ​​reward model.

[0192] In one optional embodiment, the AI ​​reward model includes an input encoding layer, a cross-modal fusion layer, a constitutional rule item mapping layer, and a score calculation layer; the reward scoring training unit includes:

[0193] An input processing unit is configured to input the quality assessment report and the corresponding input video segment into the input coding layer, extract spatial features from the input video segment through the input coding layer to obtain temporal features, and convert the quality assessment report into text features.

[0194] The cross-modal fusion unit is used to perform cross-modal attention learning on the temporal features and the text features through the cross-modal fusion layer, so as to learn the correspondence between the quality assessment report and the true quality deviation of the input video segment and form a joint feature representation;

[0195] The constitutional rule mapping unit is used to input the joint feature representation into the constitutional rule item mapping layer, and perform multi-dimensional mapping in combination with the preset video quality assessment constitutional rules to obtain the rule mapping output values ​​of each dimension of the constitutional rules; the multi-dimensional mapping includes prediction fidelity mapping, error information entropy mapping, temporal consistency mapping and frequency domain health mapping.

[0196] The scoring calculation unit is used to obtain the final reward score by performing a weighted linear combination of the output values ​​of each rule mapping based on the scoring calculation layer.

[0197] As the device embodiment is basically similar to the method embodiment, it is described in a relatively simple way. For relevant details, please refer to the description of the method embodiment above.

[0198] This invention also provides an electronic device, which includes a processor and a memory:

[0199] The memory is used to store program code and transfer the program code to the processor;

[0200] The processor is used to execute the video quality assessment method of any embodiment of the present invention according to the instructions in the program code.

[0201] This invention also provides a computer-readable storage medium for storing program code for executing the video quality assessment method of any embodiment of this invention.

[0202] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0203] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this invention are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0204] In the embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or units through some interfaces, and may be electrical, mechanical, or other forms.

[0205] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0206] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0207] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0208] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A video quality assessment method, characterized in that, include: Obtain the video to be evaluated; The video to be evaluated is input into a generative world model pre-trained based on self-supervised learning, and a prediction error is generated through forward propagation. The prediction error is input into a quality evaluator that combines an AI reward model for parameter optimization. The quality evaluator performs a multi-dimensional quality assessment of the prediction error and outputs a comprehensive quality assessment report of the video to be evaluated.

2. The video quality assessment method according to claim 1, characterized in that, The training process of the generative world model includes: Construct a generative spatiotemporal world model framework; An unlabeled historical video corpus is obtained, and based on the historical video corpus, the reconstruction error between real frames and predicted frames is minimized as the optimization objective. The prediction error of the video is used as the model output, and the generative spatiotemporal world model framework is trained through self-supervised learning to obtain a trained generative world model.

3. The video quality assessment method according to claim 2, characterized in that, The generative spatiotemporal world model framework adopts a hierarchical structure based on a combination of a state-space model and a convolutional encoder; the generative spatiotemporal world model framework includes an input preprocessing layer, a spatial encoding layer, a temporal state-space modeling layer, a prediction head, and an error reconstruction module.

4. The video quality assessment method according to claim 1, characterized in that, The quality evaluator is built based on a lightweight diagnostic model framework; the training process of the quality evaluator includes: The prediction error data output by the generative world model during training is used as the model input to train the quality evaluator for multi-dimensional quality evaluation and generate a multi-dimensional quality evaluation report. The quality assessment report is scored using an AI reward model based on constitutional rules, and a reward score is output. Based on the reward scores, a reinforcement learning algorithm is used to optimize the parameters of the quality evaluator. By combining the reward feedback of the AI ​​reward model based on constitutional rules, and through continuous iterative optimization of the quality evaluator, the final quality evaluator is obtained.

5. The video quality assessment method according to claim 4, characterized in that, The quality assessment report includes quantitative scores across multiple dimensions, as well as a video diagnostic text description in natural language. The multiple dimensions include video fidelity, smoothness, and texture quality. The video diagnostic text description is generated based on a preset text rule template or a lightweight natural language generation network.

6. The video quality assessment method according to claim 4 or 5, characterized in that, The training process of the AI ​​reward model based on constitutional rules includes: Build an AI reward model; The AI ​​reward model is trained by using the quality assessment report and the corresponding video clip as model input, the reward score as model output, and the preset video quality assessment constitutional rules. By combining the optimization feedback from the quality evaluator and continuously iterating and optimizing the AI ​​reward model, the final AI reward model is obtained.

7. The video quality assessment method according to claim 6, characterized in that, The AI ​​reward model includes an input encoding layer, a cross-modal fusion layer, a constitutional rule item mapping layer, and a score calculation layer. The process involves using the quality assessment report and the corresponding video segment as model input, and the reward score as model output, combined with preset video quality assessment constitutional rules, to train the AI ​​reward model for reward scoring, including: The quality assessment report and the corresponding input video segment are input into the input coding layer. Spatial features are extracted from the input video segment through the input coding layer to obtain temporal features. At the same time, the quality assessment report is converted into text features. The cross-modal fusion layer performs cross-modal attention learning on the temporal features and the text features to learn the correspondence between the quality assessment report and the true quality deviation of the input video segment, forming a joint feature representation; The joint feature representation is input into the constitutional rule item mapping layer, and multi-dimensional mapping is performed in combination with the preset video quality assessment constitutional rules to obtain the rule mapping output values ​​of each dimension of the constitutional rules; the multi-dimensional mapping includes prediction fidelity mapping, error information entropy mapping, temporal consistency mapping and frequency domain health mapping; The final reward score is obtained by performing a weighted linear combination of the output values ​​of each rule mapping by the scoring calculation layer.

8. A video quality assessment device, characterized in that, include: The video acquisition unit is used to acquire the video to be evaluated. The prediction error generation unit is used to input the video to be evaluated into a generative world model pre-trained based on self-supervised learning, and generate prediction error through forward propagation. A multi-dimensional quality assessment unit is used to input the prediction error into a quality evaluator that combines an AI reward model for parameter optimization. The quality evaluator performs a multi-dimensional quality assessment on the prediction error and outputs a comprehensive quality assessment report of the video to be evaluated.

9. An electronic device, characterized in that, The device includes a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is configured to execute the video quality assessment method according to any one of claims 1-7 according to the instructions in the program code.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store program code for performing the video quality assessment method according to any one of claims 1-7.

Images