A video encoding method, device, storage medium and electronic device

Abstract

The specification discloses a video encoding method, device, storage medium and electronic equipment, wherein the method comprises: determining a target video frame to be processed, obtaining reference decoding features corresponding to reference video frames and reference frame context information, the reference frames being previous video encoding frames of the target video frame, performing frame motion prediction processing based on the reference decoding features and the reference video frames to obtain predicted video frame features, performing context feature extraction based on the predicted video frame features to obtain predicted frame context information for the target video frame, performing context quality optimization based on the predicted frame context information, the reference frame context information and the reference decoding features to obtain enhanced context information, and performing frame reconstruction processing on the target video frame based on the enhanced context information to obtain a target video encoding frame.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a video encoding method, apparatus, storage medium, and electronic device. Background Technology

[0002] In recent years, with the rapid development of computer technology, video communication has become deeply integrated into people's daily work and life, becoming an indispensable tool. Whether in the business field or in various scenarios such as education and entertainment, the application of video communication is particularly important. For example, video conferencing is changing traditional office models, enabling companies to overcome geographical limitations and achieve efficient remote collaboration; distance education provides learners with more flexible learning methods, allowing them to access high-quality educational resources regardless of their location; and live streaming platforms, through their instant interactivity and broad audience base, have become a new medium for people to obtain information, enjoy entertainment, and even conduct business promotion.

[0003] At the same time, the development of video communication media has been accompanied by the widespread application of ultra-high-definition video content. This ultra-high-definition video content not only brings clearer and more vivid picture effects, but also enhances the communication experience, making information transmission more intuitive and efficient. Summary of the Invention

[0004] This specification provides a video encoding method, apparatus, storage medium, and electronic device, the technical solution of which is as follows: Firstly, this specification provides a video encoding method, the method comprising: The target video frame to be processed is determined, and the reference decoding features and reference frame context information corresponding to the reference video frame are obtained. The reference frame is the previous video encoded frame of the target video frame. Based on the reference decoding features and the reference video frame, frame motion prediction processing is performed to obtain predicted video frame features, and based on the predicted video frame features, context feature extraction is performed to obtain predicted frame context information for the target video frame. Enhanced context information is obtained by performing context quality optimization based on the predicted frame context information, the reference frame context information, and the reference decoding features; The target video frame is reconstructed based on the enhanced context information to obtain the target video encoded frame.

[0005] Secondly, this specification provides a video encoding apparatus, the apparatus comprising: The data determination module is used to determine the target video frame to be processed, and to obtain the reference decoding features and reference frame context information corresponding to the reference video frame, wherein the reference frame is the previous video encoded frame of the target video frame. The context prediction module is used to perform frame motion prediction processing based on the reference decoding features and the reference video frame to obtain predicted video frame features, and to extract context features based on the predicted video frame features to obtain predicted frame context information for the target video frame. The context optimization module is used to perform context quality optimization based on the predicted frame context information, the reference frame context information, and the reference decoding features to obtain enhanced context information; The video frame reconstruction module is used to perform frame reconstruction processing on the target video frame based on the enhanced context information to obtain the target video encoded frame.

[0006] Thirdly, this specification provides a computer storage medium storing at least one instruction adapted for loading by a processor and executing method steps of one or more embodiments of this specification.

[0007] Fourthly, this specification provides a computer program product storing at least one instruction adapted to be loaded by a processor and to execute the method steps of one or more embodiments of this specification.

[0008] Fifthly, this specification provides an electronic device that may include: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to execute the method steps of one or more embodiments of this specification.

[0009] The beneficial effects of the technical solutions provided in some embodiments of this specification include at least the following: In one or more embodiments of this specification, the electronic device determines the target video frame to be processed, obtains the reference decoding features and reference frame context information corresponding to the previous reference video frame, performs frame motion prediction processing based on the reference decoding features and the reference video frame to obtain predicted video frame features, extracts context features based on the predicted video frame features to obtain predicted frame context information for the target video frame, performs context quality optimization based on the predicted frame context information, reference frame context information and reference decoding features to obtain enhanced context information, and then performs frame reconstruction processing on the target video frame to obtain the target video encoded frame. By mining the spatiotemporal correlation between consecutive video frames, and utilizing the decoding features and contextual information of the previous reference frame to accurately optimize the predicted context of the current target frame, errors in frame motion prediction are effectively reduced, ensuring the accuracy of the predicted video frame context. Context feature extraction and quality optimization further enhance the feature representation capability of the target video frame, avoiding information loss and noise interference. Furthermore, fusion optimization based on multi-source contextual information improves the quality perception capability of the target video frame, while significantly reducing the encoding overhead of redundant data. Ultimately, the target video encoded frame generated through frame reconstruction processing maintains high visual quality and inter-frame consistency, while also achieving high efficiency in video compression, meeting the practical needs of high-quality video transmission and storage. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in this specification or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a scene diagram of a video encoding system provided in this manual; Figure 2 This is a flowchart illustrating a video encoding method provided in this manual; Figure 3 This is an end-to-end video coding framework based on spatiotemporal adaptive context optimization, which is involved in the video coding method provided in this specification. Figure 4 This is a flowchart illustrating a context optimization method provided in this specification; Figure 5 This is a schematic diagram of a processing scenario provided by the context optimization module in this manual; Figure 6 This is a flowchart illustrating a quality perception process provided in this manual; Figure 7This is a schematic diagram of a quality sensing unit performing quality sensing processing, as provided in this manual. Figure 8 This is a flowchart illustrating a consistency and collaborative processing method provided in this manual. Figure 9 This is a schematic diagram illustrating a scenario in which a consistency collaboration module performs context content consistency optimization processing, as described in this specification. Figure 10 This is a schematic diagram of the device structure of a video encoding device provided in this specification; Figure 11 This is a schematic diagram of the structure of an electronic device provided in this specification. Detailed Implementation

[0012] The technical solutions in this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0013] In the description of this specification, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. In the description of this specification, it should be noted that, unless otherwise expressly specified and limited, "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices. Those skilled in the art can understand the specific meaning of the above terms in this specification based on the specific circumstances. Furthermore, in the description of this specification, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.

[0014] In related technologies, the widespread adoption of ultra-high-definition video has brought enormous challenges to storage and transmission, especially given the massive data volume of high-resolution video, which makes existing storage devices and network bandwidth insufficient to fully meet the demands. Therefore, more efficient video encoding methods have become a pressing issue that needs to be addressed.

[0015] The bottlenecks in ultra-high-definition video storage and transmission are: 1) Huge data volume: Videos with resolutions of 4K and above occupy a large amount of storage space, while high frame rates (such as 60 frames per second and above) further increase the data volume. 2) Bandwidth limitations make real-time transmission of high-definition video difficult, especially in low-latency scenarios (such as video conferencing).

[0016] Limitations of traditional compression techniques: While rule-based traditional video coding standards (such as H.264, H.265, AV1, etc.) have significantly improved compression ratios, they struggle to further explore the complex nonlinear relationships between data. The trade-off between coding efficiency and image quality gradually pushes traditional methods towards their optimization limits.

[0017] Therefore, this specification proposes a video coding method to improve coding efficiency on an end-to-end video coding framework, thereby significantly reducing the bit rate required for video coding based on a certain resolution video coding mode (such as 2K resolution, 4K resolution). By utilizing spatiotemporal correlation to optimize context information, coding efficiency is improved by reducing the difference between context information and the information to be encoded.

[0018] The present specification will now be described in detail with reference to specific embodiments.

[0019] Please see Figure 1 This is a scene diagram of a video encoding system provided in this specification. Figure 1 As shown, the video encoding system may include at least a client cluster and a service platform 100.

[0020] The client cluster may include at least one client, such as Figure 1 As shown, it specifically includes client 1 corresponding to user 1, client 2 corresponding to user 2, ..., client n corresponding to user n, where n is an integer greater than 0.

[0021] Each client in a client cluster can be an electronic device with communication capabilities, including but not limited to: wearable devices, handheld devices, personal computers, tablets, in-vehicle devices, smartphones, computing devices, or other processing devices connected to a wireless modem. Electronic devices may have different names in different networks, such as: user equipment, access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, electronic device, wireless communication device, user agent or user device, cellular phone, cordless phone, personal digital assistant (PDA), and electronic devices in 5G networks or future evolved networks.

[0022] The service platform 100 can be a standalone server device, such as a rack-mount, blade, tower, or cabinet-type server device, or a workstation, mainframe, or other hardware device with strong computing power; or it can be a server cluster composed of multiple servers. The servers in the service cluster can be composed in a symmetrical manner, wherein each server is functionally and hierarchically equivalent in the transaction chain, and each server can provide services independently. The independent provision of services can be understood as not requiring the assistance of other servers.

[0023] In one or more embodiments of this specification, the service platform 100 can establish a communication connection with at least one client in the client cluster, and complete the data interaction during the video encoding process based on the communication connection, such as online transaction data interaction. For example, the service platform 100 can obtain the target video encoding frame based on the video encoding method of this specification and can realize the transmission of video frames to the client. It should be noted that the service platform 100 establishes a communication connection with at least one client in the client cluster via a network for interactive communication. This network can be a wireless network or a wired network. Wireless networks include, but are not limited to, cellular networks, wireless LANs, infrared networks, or Bluetooth networks. Wired networks include, but are not limited to, Ethernet, universal serial bus (USB), or controller area networks. In one or more embodiments of the specification, technologies and / or formats including Hyper Text Markup Language (HTML), Extensible Markup Language (XML), etc., are used to represent data exchanged over the network (such as target compressed packets). Furthermore, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), and Internet Protocol Security (IPsec) can be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can be used to replace or supplement the aforementioned data communication technologies.

[0024] The video encoding system embodiments provided in this specification and the video encoding methods described in one or more embodiments belong to the same concept. The execution entity corresponding to the video encoding method involved in one or more embodiments of this specification can be the aforementioned service platform 100; the execution entity corresponding to the video encoding method involved in one or more embodiments of this specification can also be the electronic device corresponding to the client, specifically determined based on the actual application environment. The implementation process of the video encoding system embodiments can be detailed in the following method embodiments, and will not be repeated here.

[0025] based on Figure 1 The following is a detailed description of the video encoding method provided by one or more embodiments of this specification, as illustrated in the scene diagram.

[0026] Please see Figure 2 This document provides a flowchart illustrating a video encoding method according to one or more embodiments. This method can be implemented using a computer program and can run on a video encoding device based on the von Neumann architecture. The computer program can be integrated into an application or run as a standalone utility application. The video encoding device can be an electronic device.

[0027] Specifically, the video encoding method includes: S102: Determine the target video frame to be processed, and obtain the reference decoding features and reference frame context information corresponding to the reference video frame, wherein the reference frame is the previous video encoded frame of the target video frame; Target video frame: The original video frame that needs to be encoded, which can be denoted as x. t .

[0028] Reference video frame: A video frame that has already been encoded and precedes the target video frame (i.e., the previous or preceding video frame), used as a basis for prediction during the encoding process, and can be denoted as... .

[0029] Reference decoding feature: The decoding feature representation of the reference video frame stored in the decoding buffer after decoding, which can be denoted as f. t-1 .

[0030] Reference frame context information: Semantic information describing the overall content of the reference frame. It is the context information of the previous frame obtained by extracting context features from the reference video frame (e.g., using a (spatiotemporally adaptive) context optimization module, or using a machine learning model from the reference decoding features). The reference frame context information includes multi-scale context feature representations at different spatial resolutions. Assume the reference frame context features represented by the spatial resolution of the i-th layer are as follows: For example, if spatial resolution is divided into three levels—low, medium, and high (which can be considered as layer 1, layer 2, and layer 3)—the sampling parameters of the spatial resolution corresponding to the reference frame context features represented by different spatial resolutions are different. The reference frame context features contained in the reference frame context information can be represented as follows: : : Low-resolution reference frame context features at low spatial resolution (e.g., global semantic dimension features). : Contextual features of a medium-resolution reference frame at a medium spatial resolution (e.g., semantic dimension features of a local region). : High-resolution reference frame context features at higher spatial resolution (e.g., edge detail dimension features) Indicatively, determine the current target video frame x to be processed. t Obtain the reference video frame f from the decoding buffer. t-1 and reference frame context information ; S104: Perform frame motion prediction processing based on the reference decoding features and the reference video frame to obtain predicted video frame features, and extract context features based on the predicted video frame features to obtain predicted frame context information for the target video frame; For illustrative purposes, please refer to Figure 3 , Figure 3 This is an end-to-end video coding framework based on spatiotemporal adaptive context optimization involved in the video coding method of this specification. The end-to-end video coding framework consists of a motion coding part, a context optimization part, and a conditional coding part. Frame motion prediction processing: This process generates predicted video frame features by utilizing the motion relationship (such as optical flow or motion vectors) between the reference frame and the target frame. Figure 3 The processing procedure of the motion coding part.

[0031] Predicted video frame features: The predicted features of the target frame represent the preliminary estimation results of the target frame.

[0032] Predicted frame context information: Contextual semantic information extracted from the features of the predicted video frame, used for context optimization. It should be noted that the predicted frame context information is not based on the current target video frame, but is obtained based on the previous reference video frame through the motion coding part.

[0033] In one feasible implementation, the frame motion prediction processing based on the reference decoding features and the reference video frame to obtain the predicted video frame features can be performed in the following manner: A2: Based on the reference video frame, perform frame image motion estimation processing to obtain the motion vector field; A4: Motion encoding is performed based on the motion vector field to obtain motion representation information, and motion decoding is performed on the motion representation information to obtain motion vectors; A6: Based on the motion vector and the reference decoding features, motion compensation processing is performed to obtain the predicted video frame features.

[0034] For details, please refer to Figure 3 In the motion coding section, motion estimation (Optical Flow) uses neural networks or optical flow estimation algorithms to perform motion estimation on the current target video frame and the previous reference video frame. Based on the reference video frame, it uses neural networks or optical flow estimation algorithms to perform frame image motion estimation processing to obtain the motion vector field v. t Motion vector field v t It describes the horizontal and vertical displacement of pixels between frames.

[0035] Figure 3 Motion encoding process: For the motion vector field v t Encoding and compression are performed to generate compact motion representation information. For example, a variational autoencoder (VAE) or a compressed neural network can be used to perform motion encoding processing on the motion vector field to obtain motion representation information (usually in the form of representation vectors).

[0036] Figure 3 Motion decoding process: Motion decoding is performed on the compressed motion representation information. Motion decoding is the process of recovering approximate motion vectors. (This can be called optical flow).

[0037] Figure 3 The motion compensation (warping) process in the middle involves extracting the decoding features of the previous frame, i.e., the reference decoding features f, from the decoding buffer. t 1. Utilize the decoded motion vectors For the reference decoding feature f t 1. Perform a warp operation for motion compensation (i.e., re-predict and reposition the pixels from the previous frame to their current position through a warp operation), generating predicted video frame features f for the current target video frame. t,pred .

[0038] Motion coding output: Generates predicted video frame features f t,pred This is used by the context optimization and conditional coding sections to predict video frame features f. t,pred Multi-resolution prediction frame context features can be extracted by sampling at different spatial resolutions using features extraction networks. All the predicted frame context features constitute the predicted frame context information.

[0039] S106: Context quality optimization is performed based on the predicted frame context information, the reference frame context information, and the reference decoding features to obtain enhanced context information; Enhanced contextual information: Contextual information for the target video frame after context quality optimization; Please refer to the following in this instruction manual: Figure 3 The (spatiotemporal adaptive) context optimization module in the context optimization part is built based on a machine learning model. The (spatiotemporal adaptive) context optimization module takes the predicted frame context information, the reference frame context information, and the reference decoding features as inputs to the module. The context optimization module performs context quality optimization to obtain enhanced context information. The context optimization module mitigates bit waste in conditional coding caused by inaccurate context estimation. By using the predicted context as input, it improves context quality by ensuring better consistency with the decoded features, thereby increasing coding efficiency. In some embodiments, the context optimization module consists of multiple quality-aware units and a consistency coordination module, arranged in a hierarchical stacked structure, enabling it to simultaneously process context information at different spatial resolutions. Furthermore, the context optimization module models the spatial quality mapping knowledge between the reference frame context information and the reference decoded features in the previous frame, and learns this spatial quality mapping knowledge to transfer it to the predicted frame context information of the current frame. Context information at different resolutions predicted in the temporal domain is exchanged through spatial upsampling and downsampling, and processed separately by the quality-aware units and the consistency coordination module. This results in enhanced context information obtained through context quality optimization based on the predicted frame context information, the reference frame context information, and the reference decoded features. The optimized enhanced context information aligns with the content of the reconstructed features, thereby improving coding performance and reducing the bitrate overhead of video coding during conditional coding.

[0040] S108: Based on the enhanced context information, perform frame reconstruction processing on the target video frame to obtain the target video encoded frame.

[0041] In one feasible implementation, the frame reconstruction process based on the enhanced context information to obtain the target video encoded frame can be performed as follows: The target frame reconstruction features are obtained by the conditional coding module based on the enhanced context information and the target video frame. The target video frame is then reconstructed by the conditional coding module based on the target frame reconstruction features to obtain the target video encoded frame.

[0042] The conditional coding module can be referenced. Figure 3 The conditional coding section is a module based on specific enhanced context information input to guide feature encoding and decoding and frame reconstruction of the target video frame. Based on the enhanced context information of the target video frame, it efficiently encodes the difference (residual) between the predicted information and the actual information, and finally generates the target video coded frame.

[0043] Furthermore, the following methods can be used: S2. The conditional encoder performs context feature information transfer on the target video frame based on the enhanced context information to obtain feature compression representation information; Conditional encoder: A module that dynamically adjusts the encoding process based on input conditions (such as enhanced context information), responsible for generating efficient and compact feature representations. Its inputs are enhanced context information and the target video frame; such as... Figure 3 As shown, the enhanced context information is combined with the target video frame, and a feature representation adapted to the target frame is generated through a conditional encoder. The information transfer includes spatiotemporal correlation alignment and efficient compressed representation of cross-frame information, and the compressed feature representation information is obtained and output. Feature compression representation information: The compressed feature representation vector retains the important spatiotemporal characteristics of the target video frame, while reducing redundant information to facilitate transmission or storage.

[0044] In a schematic manner, a conditional encoder receives initial features of the target frame and enhanced contextual information. The conditional encoder uses the optimized enhanced contextual information as conditional information and employs techniques such as convolutional neural networks to extract features from the current target video frame. The extracted features include, but are not limited to, edge, texture, and motion vector features. The target frame features are aligned and fused with the enhanced contextual information to ensure spatiotemporal consistency. An attention mechanism can be used to weightedly fuse the features. Then, the target frame features after fusing contextual information are compressed to generate a more efficient compressed representation. For example, through dimensionality reduction and attention mechanisms, the optimized enhanced contextual information is used as conditional information to map the high-dimensional features of the target video frame to a low-dimensional compressed representation. The compressed representation retains key details while discarding redundant information to reduce storage and transmission costs, ultimately yielding the compressed representation. S4. Obtain the target frame reconstruction features by performing frame feature decoding based on the enhanced context information and the feature compression representation information using the conditional decoder; Conditional decoder: Based on specific conditions (such as enhanced contextual information and feature compression representation), it decodes low-dimensional features and gradually recovers high-dimensional spatial features to obtain the reconstructed features of the target frame.

[0045] Target frame reconstruction features: The intermediate feature representation after reconstruction contains important and structural information of the target frame, and can be denoted as f.t ; In a schematic manner, the feature compression representation information is initially decoded through a conditional decoder to restore part of the spatial resolution. Enhanced contextual information is used to guide the decoding process of the conditional decoder to ensure inter-frame consistency. Then, the features of the target frame are decoded at multiple spatial resolutions. Features of target frames at different resolutions are fused to preserve both local details and global consistency. During the decoding process, texture, edge and other details in the dynamic conditional features, as well as compensation mechanisms such as residual information, can be introduced to repair blurred and lost areas. Finally, the reconstructed features of the target frame are output. S6. Based on the target frame reconstruction features, perform frame reconstruction processing on the target video frame to obtain the target video encoded frame.

[0046] Frame reconstruction processing: This can be understood as mapping the reconstructed features of the target frame to the pixel space and generating the complete target frame through a decoder.

[0047] Target video encoded frame: The final target video frame output by the decoder, which retains visual quality while meeting compression ratio requirements.

[0048] In one or more embodiments of this specification, the electronic device determines the target video frame to be processed, obtains the reference decoding features and reference frame context information corresponding to the previous reference video frame, performs frame motion prediction processing based on the reference decoding features and the reference video frame to obtain predicted video frame features, extracts context features based on the predicted video frame features to obtain predicted frame context information for the target video frame, performs context quality optimization based on the predicted frame context information, the reference frame context information, and the reference decoding features to obtain enhanced context information, and then performs frame reconstruction processing on the target video frame to obtain the target video encoded frame. By mining the spatiotemporal correlation between consecutive video frames, the decoding features and context information of the previous reference frame are used to accurately predict and optimize the target frame. Furthermore, frame motion prediction processing effectively reduces errors in motion estimation, ensuring the accuracy of predicted video frame features. Context feature extraction and quality optimization further enhance the feature representation capability of the target video frame, avoiding information loss and noise interference. Additionally, fusion enhancement optimization based on multi-source context information improves the quality perception capability of the target video frame, while significantly reducing the coding bit rate overhead of redundant data. Ultimately, the target video coded frame generated through frame reconstruction processing maintains high visual quality and inter-frame consistency, while also achieving high efficiency in video compression coding, meeting the practical needs of high-quality video transmission and storage.

[0049] Please see Figure 4 , Figure 4This is a flowchart illustrating a context optimization method proposed in one or more embodiments of this specification. Specifically, the process of performing context quality optimization based on the predicted frame context information, the reference frame context information, and the reference decoding features to obtain enhanced context information can be described as follows: Please refer to Figure 3 The (spatiotemporal adaptive) context optimization module in the context optimization part is built based on a machine learning model. The (spatiotemporal adaptive) context optimization module takes the predicted frame context information, the reference frame context information, and the reference decoding features as inputs to the module. The context optimization module performs context quality optimization to obtain enhanced context information. S2002: Use a context optimization module to determine multiple reference frame context features with different spatial resolutions corresponding to the reference frame context information, and determine multiple prediction frame context features with different spatial resolutions corresponding to the prediction frame context information. The context optimization module takes as input multiple reference frame context features at different spatial resolutions, multiple predicted frame context features at different spatial resolutions, and reference decoded features. Through upsampling or downsampling operations, the context information is divided into context features at different spatial resolution levels (e.g., high resolution, medium resolution, low resolution). Each spatial resolution focuses on different spatial scale characteristics.

[0050] Further, optional, such as Figure 5 As shown, Figure 5 This is a schematic diagram of a context optimization module's processing scenario. The context optimization module includes multiple quality-aware units and consistency coordination modules. This module consists of stacked quality-aware units and consistency coordination modules arranged in a hierarchical stacked structure, such as... Figure 5 As shown, this multi-scale hierarchical stacked structure enables the context optimization module to process context features at different spatial resolutions simultaneously, and facilitates information exchange between them through methods such as upsampling and downsampling. After residual connections, the enhanced context information is output, ensuring better alignment with the reconstructed features and further improving quality.

[0051] The reference decoding feature is denoted as f t-1 The reference decoding feature is denoted as f. t-1 Including feature representations at different spatial resolutions, assuming the reference decoded feature of the i-th layer spatial resolution representation is... ,like Figure 5 As shown, for example, taking the spatial resolution as divided into three levels—low, medium, and high (which can be considered as layer 1, layer 2, and layer 3)—the reference decoding features represented by the spatial resolution of different layers are as follows: : Low-resolution reference decoding features with low spatial resolution (e.g., global semantic dimension features). : Medium-resolution reference decoding features at medium spatial resolution (e.g., semantic dimension features of local regions) High-resolution reference decoding features at higher spatial resolution levels (e.g., edge detail dimension features). Furthermore, the reference frame context features contained in the reference frame context information can be represented as follows: : : Low-resolution reference frame context features at low spatial resolution (e.g., global semantic dimension features). : Contextual features of a medium-resolution reference frame at a medium spatial resolution (e.g., semantic dimension features of a local region). : High-resolution reference frame context features at higher spatial resolution (e.g., edge detail dimension features) Furthermore, the predicted frame context features contained in the predicted frame context information can be represented as follows: : : Low-resolution reference frame context features at low spatial resolution (e.g., global semantic dimension features). : Contextual features of a medium-resolution reference frame at a medium spatial resolution (e.g., semantic dimension features of a local region). : High-resolution reference frame context features at higher spatial resolution (e.g., edge detail dimension features) The context optimization module employs a multi-scale structure (high, medium, and low spatial resolution) Quality Aware Unit (QPU) and Coordinating Module (CSM), enabling them to simultaneously process context information (reference frame context features, predicted frame context features) and reference decoded features at different spatial resolutions. This effectively captures spatiotemporal correlations at different levels. Information exchange between contexts of different spatial resolutions is achieved through upsampling and downsampling, further optimizing the context. This allows low-resolution context information to be passed to high-resolution context, and vice versa. After context optimization, the optimized context is fused with the predicted context of the original input through residual connections. This connection method effectively prevents information loss, preserves useful original information, and enhances the final output quality. After optimization, the enhanced context is output, and the optimized context is better aligned with the reconstructed features of the current frame, further improving coding efficiency and reducing bit waste.

[0052] S2004: The context optimization module uses the reference decoding features, the reference frame context features of each spatial resolution, and the predicted frame context features to perform quality-aware processing on the context features and the decoding feature domain to obtain context-aware result information corresponding to multiple spatial resolutions; Quality-aware processing: This can be understood as the process of evaluating and adjusting the quality of context features, used to address the differences between context features and decoding features at different resolutions and improve consistency.

[0053] Schematic illustration: The quality-aware unit (QPU) processes contextual features at each spatial resolution (such as reference frame contextual features). and predicted frame context features ) and reference decoding features Interacting with the system, the system uses neural network operations such as self-attention and convolution to perceive the quality mapping relationship between the reference frame context and the decoded feature domain. The learned quality mapping relationship is then used to transfer and optimize the prediction frame context of the current frame, and used for temporal propagation, thereby improving the context quality and generating context-aware results.

[0054] S2006: The context optimization module performs context content consistency optimization on the context-aware result information of different spatial resolutions to obtain context refinement results of different spatial resolutions. The context refinement results are then used to enhance the context features of the predicted frame to obtain enhanced context information.

[0055] Contextual content consistency optimization: By using the Consistency Coordination Module (CSM) to utilize perception results from adjacent spatial scales, consistency between contextual perception results is ensured across multiple spatial resolution levels, resolving potential discrepancies in cross-resolution information interaction. For example... Figure 5 As shown, in the first-layer consensus coordination module (CSM), the (joint) context-aware result information... Compared to low-resolution estimation from deeper levels Cooperative processing is performed (if perceptual results exist at adjacent spatial scales) and is implemented through a two-branch structure. The global branch in the Coordinating Module (CSM) employs a multi-head self-attention mechanism to capture long-range correlations between feature blocks, while the local texture detail branch still models residual blocks. This approach addresses the issue of (joint) perceptual results... There is a lack of explicit correspondence between inter-domain relationships (feature consistency between different resolutions, such as low and high resolution) and intra-domain relationships (associations between feature blocks within the same resolution). The results from the two branches are then integrated to obtain the enhanced context. Similarly, the second-layer Consistency Coordination Module (CSM) further refines this using the same approach. To obtain the context refinement result The result of this context refinement Subsequently, it is combined with the predicted frame context features. By combining feature enhancements, the final enhanced context at different spatial resolution levels is generated. All enhanced contexts This constitutes enhanced contextual information and serves as prior knowledge for subsequent conditional encoding.

[0056] In this specification, the reference frame context features and predicted frame context features, as described above, achieve substantial quality optimization and consistency enhancement at the multi-resolution spatial level, effectively resolving the inconsistency issue in cross-resolution information interaction. With the help of the context optimization module, the context features not only enhance global semantic associations but also preserve the integrity of local details, thereby generating higher-quality enhanced context information. Ultimately, this optimized context information, as prior knowledge, can significantly improve the coding efficiency of the target frame, reduce redundant bit rate, and enhance reconstruction quality, meeting the practical needs of efficient video coding.

[0057] Optional, please see Figure 6 , Figure 6This is a flowchart illustrating a quality-aware processing procedure. The procedure involves using the reference decoding features, the reference frame context features for each spatial resolution, and the predicted frame context features through the context optimization module to perform quality-aware processing on the context features and the decoding feature domain, obtaining context-aware result information corresponding to multiple spatial resolutions. The following implementation method can be referenced: S3002: Transmit the reference decoding features, the reference frame context features of each of the spatial resolutions, and the predicted frame context features to the quality sensing unit; Combination Figure 6 Please refer to the flowchart shown below. Figure 7 , Figure 7 This is a schematic diagram illustrating a scenario where a quality sensing unit performs quality sensing processing, as described in this specification. Figure 7 In this context, Layer-Norm represents the processing layer, Conv k3s1 represents the convolution operation, and Spatio-Norm represents the spatial processing operation corresponding to the spatial processing layer. S3004: The first global quality perception mechanism of the quality perception unit performs cross-attention processing on the reference frame context features and the reference decoding features to determine the semantic correlation of all reference frame context features and the reference decoding features, and uses the semantic correlation to perform global quality enhancement processing on the predicted frame context features to obtain a global context perception result for the predicted frame context features. The first global quality-aware mechanism: It employs cross-attention processing to calculate the semantic relevance between the reference frame context features and the reference decoded features. Based on this semantic relevance, it performs global quality enhancement on the predicted frame context features, generating a global context-aware result. Figure 7 In the first (first) global quality perception (mechanism part): Layer-Norm represents the regularization operation of layers with different spatial resolutions, Conv k3s1 represents the convolution operation, k3s1 represents the kernel size of the convolutional network corresponding to the convolution operation, Spatio-Norm represents the regularization operation corresponding to the spatial processing layer; reference decoding feature f t-1 The reference feature value is obtained after processing by Layer-Norm, Conv k3s1, and Spatio-Norm. The reference decoded feature f is then used. t-1 The reference decoding feature values ​​and reference frame context features are obtained after processing by Layer-Norm, Conv k3s1, and Spatio-Norm. The reference frame context features are obtained after processing by Layer-Norm, Conv k3s1, and Spatio-Norm. Then, the reference decoded features and the reference frame context features are multiplied (e.g., dot product, cross product) and processed by the "softmax" attention mechanism to obtain some semantic relevance. ; Targeting the context features of the predicted frame The results obtained from the first global quality perception mechanism after processing by Layer-Norm and Conv k3s1 are related to semantics. Enhanced by multiplication operations (such as dot product and cross product) and Conv k1s1, this process indicates that the global context-aware result can be obtained by referring to the second calculation formula; Cross-attention mechanism: Captures semantic relevance by matching reference decoded features with reference frame context features. For example, it identifies highly semantically relevant regions in the reference decoded features, thereby enhancing the quality of predicted features.

[0058] Global quality enhancement: By optimizing the weight allocation of cross-attention, the global information of the predicted frame context features is optimized to make it semantically consistent with the reference decoded features.

[0059] Based on the results of global context awareness, explicit optimization of global information is achieved, improving the semantic alignment of context features; exist Figure 5 The medium-scale quality sensing unit (QPU) adopts a multi-scale structure, and different spatial resolutions use corresponding QPUs. i ,like Figure 7 As shown, since quality differences are closely related to content, we adopt a dual-branch structure to perceive the global content (intensity, contrast, etc.) and local texture (contours, edges, etc.) represented by features, respectively. The predicted frame context features are calculated using the spatial resolution of the i-th layer. For example, when performing global quality perception, the reference frame context features are used. and reference decoding features The data is divided into non-overlapping blocks, and the semantic relevance between all reference frame context features and the reference decoded features is determined through cross-attention processing. Then, the semantic relevance is used to perform global quality enhancement processing on the predicted frame context features to obtain a global context-aware result for the predicted frame context features. ; Optionally, the semantic relevance of all reference frame context features and reference decoded features determined by performing cross-attention processing on the reference frame context features and reference decoded features through the first global quality perception mechanism of the quality perception unit can be performed in the following manner: Quality sensing unit (QPU) iThe system employs a dual-branch structure, namely a first global quality perception mechanism and a first local quality perception mechanism. The first global quality perception mechanism perceives the global content (intensity, contrast, etc.) represented by features. Layer For example, when performing global quality perception, the reference frame context features are used. and reference decoding features The data are divided into non-overlapping blocks, and the semantic relevance between them is calculated through cross-attention.

[0060] Specifically: through the Quality Perception Unit (QPU) i The first global quality-aware mechanism uses reference frame context features Cross-attention processing is performed on the reference decoded features to determine the reference frame context features. Corresponding first weight information and reference decoding features Corresponding second weight information Based on reference frame context features First weight information Reference decoding features Second weight information The semantic relevance corresponding to each spatial resolution is obtained using the first calculation formula. Then, semantic relevance is used. Propagated to time t via the time domain. Used to enhance To improve its quality, we need to focus on the details within the text, that is, to use semantic relevance. Global quality enhancement processing is performed on the predicted frame context features to obtain the predicted frame context features. Global context-aware results .

[0061] Furthermore, the first calculation formula satisfies the following formula:

[0062] in, Indicating semantic relevance, the This represents the first weight information, the The reference frame context features, the This represents the second weight information, the The reference decoding feature is represented by i, where i represents the layer number of the spatial resolution. This semantic relevance A macroscopic connection with spatially distant content was established, effectively representing the quality mapping relationship between context and decoding feature domain.

[0063] Optionally, the global quality enhancement processing of the predicted frame context features using the semantic relevance to obtain a global context-aware result for the predicted frame context features can be performed in the following manner: Global quality enhancement processing is performed on the predicted frame context features based on semantic relevance. The third weight information of the predicted frame context features is determined, and the comprehensive semantic relevance is obtained by solving the global context relevance based on the semantic relevance of all spatial resolutions. The global context perception result for the predicted frame context features is obtained by using the second calculation formula based on the third weight information, the predicted frame context features, the semantic relevance, and the comprehensive semantic relevance. Furthermore, the second calculation formula satisfies the following formula:

[0064] Among them, the The global context-aware result representing the predicted frame context features at the i-th spatial resolution, the This represents the third weight information, the The predicted frame context features are represented by the... This refers to the comprehensive semantic relevance; Specifically, semantic relevance based on all spatial resolutions Perform context-wide relevance calculation to obtain comprehensive semantic relevance. For context-based global dependency calculation, please refer to = x and y represent the horizontal and vertical coordinates of space, respectively. S3006: The reference decoding features, the reference frame context features, and the predicted frame context features are concatenated through the first local quality perception mechanism of the quality perception unit to obtain a comprehensive concatenated feature. A convolutional residual network processing operation is used to perform local quality enhancement on the comprehensive concatenated feature to obtain a local context perception result for the predicted frame context features. The first local quality-aware mechanism concatenates the reference decoding features, reference frame context features, and predicted frame context features to form a comprehensive concatenated feature. A convolutional residual network is then used to perform local quality enhancement on the comprehensive concatenated feature to generate a local context-aware result.

[0065] Convolutional residual network processing operations: Convolutional residual networks use residual blocks to capture local features (such as texture and edge details), while skip connections avoid gradient vanishing and preserve original detail information.

[0066] For local quality perception , and The data are concatenated and processed through a convolutional residual network to capture microscopic correlations, thereby improving the accuracy of local details in the predicted frame context features, such as compensating for missing object edges or texture features.

[0067] exist Figure 7 In the first (first) local quality perception (mechanism part): © indicates the splicing operation: referencing the decoded features The reference frame context features and the predicted frame context features The composite features are obtained by concatenating the features. Then, the composite features are processed by a convolutional residual network. The convolutional residual network consists of Conv k3s1, PReLU, Conv k3s1, PReLU and Conv k1s1 in the figure. The convolutional residual network adds the result of the composite features to the composite features to finally obtain the local context-aware result.

[0068] Layer-Norm represents the regularization operation for layers with different spatial resolutions; Conv k3s1 represents the convolution operation; k3s1 represents the kernel size of the convolutional network corresponding to the convolution operation; Spatio-Norm represents the regularization operation corresponding to the spatial processing layer; reference decoding feature f t-1 The reference feature value is obtained after processing by Layer-Norm, Conv k3s1, and Spatio-Norm. The reference decoded feature f is then used. t-1 The reference decoding feature values ​​and reference frame context features are obtained after processing by Layer-Norm, Conv k3s1, and Spatio-Norm. The reference frame context features are obtained after processing by Layer-Norm, Conv k3s1, and Spatio-Norm. Then, the reference decoded features and the reference frame context features are multiplied (e.g., dot product, cross product) and processed by the "softmax" attention mechanism to obtain some semantic relevance. ; Targeting the context features of the predicted frame The results obtained from the first global quality perception mechanism after processing by Layer-Norm and Conv k3s1 are related to semantics. Enhanced by multiplication operations (such as dot product and cross product) and Conv k1s1, this process indicates that the global context-aware result can be obtained by referring to the second calculation formula; Optionally, the local quality enhancement of the synthesized spliced ​​features obtained by performing the convolutional residual network processing operation to obtain the local context-aware result for the predicted frame context features can be performed in the following manner: Based on the third calculation formula, a convolutional residual network processing operation is used to perform local quality enhancement on the comprehensive splicing features to obtain a local context-aware result for the context features of the predicted frame. Furthermore, the third calculation formula satisfies the following formula:

[0069] Among them, the The local context-aware result representing the predicted frame context features at the i-th layer spatial resolution, the This represents the combined splicing features after splicing, where RB() represents the convolutional residual network processing operation; S3008: The quality perception unit performs result feature aggregation on the global context perception result and the local context perception result to obtain a quality perception feature aggregation result, or performs result feature aggregation on the global context perception result, the local context perception result, and the reference context perception result information to obtain a quality perception feature aggregation result. The quality perception feature aggregation result and the predicted frame context features are then combined to obtain context perception result information corresponding to each spatial resolution. The reference context perception result information is the context perception result information of the previous quality perception unit when the quality perception unit has a previous quality perception unit.

[0070] Result feature aggregation: Aggregate the global context-aware results and the local context-aware results to generate the final quality-aware feature aggregation result. If the current quality-aware unit has a previous quality-aware unit, it can be further optimized by combining the reference context-aware result information.

[0071] Feature aggregation methods can be simple fusion, such as weighted summation or concatenation. Alternatively, an attention mechanism can be used, assigning different weights to global and local results to ensure proper fusion in the final features.

[0072] A comprehensive operation is performed to integrate the quality-aware feature aggregation results and the prediction frame context features: the aggregation results and prediction frame context features are finally fused to generate context-aware result information for each spatial resolution. Based on global semantic consistency and local detail enhancement, high-quality multi-resolution context-aware results for the prediction frame are generated. If the current quality-aware unit has a previous quality-aware unit, the accumulated information further improves the robustness and accuracy of the results.

[0073] Next, the estimation results of the global context-aware result and the local context-aware result are combined using the aggregation module contained in the quality-aware unit. This aggregation module references the previous quality-aware unit (QPU). i-1(If available) high-resolution sensing results to produce more accurate estimates while ensuring consistency across spatial scales; such as Figure 7 As shown, the part of the quality perception unit other than global quality perception and local quality perception can be regarded as an aggregation module. The operation process of the aggregation module can be referred to the fourth calculation formula. First, the aggregation module merges the global context perception result and the local context perception result through the © splicing operation. The spliced ​​feature " "go through , The left branch of the aggregation module consists of Conv k3s1, PReLU, Conv k3s1, GeLU, multiplication operations ☉ (such as dot product), and Conv k1s1, and can be represented as follows: After feature aggregation, it can be represented as The quality-perceived feature aggregation results are obtained. The right branch of the aggregation module consists of splicing operations ©, Conv k1s1, PReLU, Conv k3s1, and PReLU; Furthermore, the current Quality Perception Unit (QPU) i through Then, it will determine whether a previous Quality Sensing Unit (QPU) exists. i-1 High-resolution perception results (which can be referred to as reference context perception result information); When i=1, there is no previous Quality Sensing Unit (QPU). i-1 The high-resolution sensing results are obtained through the quality sensing unit. First to Perform an identity operation followed by feature aggregation of the results (the feature aggregation process can be represented as follows). The quality-perceived feature aggregation result is obtained. The process of feature aggregation processing can be represented as follows:

[0074] When i>1, there exists a previous quality sensing unit (QPU). i-1 High-resolution perception results (QPU in some embodiments) i-1 The high-resolution sensing results can also be expressed as This is referred to as the reference context-aware result information, which will be compared with the previous quality sensing unit (QPU). i-1 The high-resolution perception results are stitched together, and the quality perception unit performs result feature aggregation on the global context perception results, the local context perception results, and the reference context perception results to obtain a quality perception feature aggregation result. This result feature aggregation process can be represented as follows: Optionally, the context-aware result information is calculated based on a fourth calculation formula, which includes: When i=1, there is no previous Quality Sensing Unit (QPU). i-1 The quality perception feature aggregation result is obtained by aggregating the global context perception result and the local context perception result through the quality perception unit, which is also the fourth calculation formula. ; When i>1, there exists a previous quality sensing unit (QPU). i-1 The quality perception feature aggregation result is obtained by aggregating the global context perception result, the local context perception result, and the reference context perception result information through the quality perception unit. This is also known as the fourth calculation formula. ; Among them, the The context-aware result information representing the context features of the predicted frame at the i-th spatial resolution, the This represents the operation of a multi-spatial-resolution context fusion network, the This indicates multi-scale context fusion network processing operations.

[0075] Understandable. For the quality sensing unit (QPU) 1 It is an identity operation, that is, using Can be regarded as QPU 2 QPU 3 A network convergence operation; Can be regarded as QPU 1 QPU 2 QPU 3 A network fusion operation, with no high-resolution reference available.

[0076] In this specification, the quality-aware unit implements dual quality enhancement, both globally and locally. On the one hand, it utilizes a cross-attention mechanism to capture the global semantic relationship between reference and predicted features, improving the global consistency of the predicted frame context and achieving global optimization. On the other hand, it uses a convolutional residual network to capture high-frequency details in local regions, compensating for the lack of texture and edge information in the predicted context. Furthermore, by combining global and local features, it generates a more complete context-aware result. The context-aware result information generated by the quality-aware unit at different resolution levels can accurately capture global relationships while preserving local details, thereby effectively improving the context quality in video coding, providing strong prior support for subsequent conditional coding, and reducing redundant bit rate.

[0077] Optional, please see Figure 8 , Figure 8 This is a flowchart illustrating a consistent collaborative processing approach. The process involves using the context optimization module to perform context content consistency optimization on the context-aware result information at different spatial resolutions, obtaining context refinement results at different spatial resolutions. These refinement results are then used to enhance the context features of the predicted frame, resulting in enhanced context information. The following method can be used as a reference: In some embodiments, each spatial resolution processing structure of the context optimization module has a quality-aware unit and two consistency coordination modules. Composition, such as Figure 5 As shown, the context optimization module has a three-layer spatial resolution processing structure (corresponding to the high spatial resolution processing structure, the medium spatial resolution processing structure, and the low spatial resolution processing structure, respectively). Each layer of spatial resolution processing structure consists of a quality sensing unit and two consistency coordination modules. S4002: The consistency coordination module performs content consistency processing on the context-aware result information and reference context-aware result information of different spatial resolutions to obtain the preliminary context consistency result for each spatial resolution; Taking the first-layer consistency and collaboration module as an example, in the perception result (In the process of representing the context-aware result information of the i-th layer spatial resolution), the first step is to use the low-resolution estimation from a deeper layer. (This represents the context-aware result information at the (i+1)th layer spatial resolution, also known as the reference context-aware result information) to maintain semantic content consistency. This process can be represented as...

[0078] in This indicates the initial context consistency result. This represents a network operation for standardizing multiple spatial resolutions. Please note that... right It is an identity operation because there is no low-resolution perceptual result to use as a reference. It can be regarded as an identity feature; The Consistency Coordination Module (CSM) leverages context-aware results from adjacent spatial scales to significantly improve context quality while maintaining content consistency across context-aware results at different resolutions. Within the CSM of the first-layer spatial resolution processing structure, the (joint) sensing results... Compared to low-resolution estimation from deeper levels Cooperative processing is performed (if available) and implemented via a dual-branch structure. Unlike the Quality-Aware Unit (QPU), the second global quality-aware mechanism in the Coherence and Coordination Module (CSM) employs a multi-head self-attention mechanism to capture correlations between blocks, while still modeling local texture details through residual blocks. This approach addresses the (joint) perception outcome. The problem lies in the lack of explicit correspondence between inter-domain and intra-domain relationships. The results from the two branches are then integrated to obtain the enhanced context. Similarly, the second-layer Consistency Coordination Module (CSM) further refines this using the same approach. After processing, the result is The refined results Subsequently with Combined, the final context is generated.

[0079] Combination Figure 8 Please refer to the flowchart shown below. Figure 9 , Figure 9 This is a schematic diagram illustrating a scenario where a consistency coordination module performs context content consistency optimization processing, as described in this specification. Figure 9 In the middle, the branch in the dashed box on the right is composed of splicing operations ©, Conv k1s1, PReLU, Conv k3s1, and PReLU, which can be called... This represents a network that fuses context-aware results from multiple spatial resolutions. Please note that... Figure 9 As shown, when i=3, right (The consistency and coordination module in the third-layer spatial resolution processing structure) is an identity operation that can be understood as... The output is consistent with the input, and no fusion processing is performed. At this time, = Because there are no low-resolution perception results available for reference. right The consistency and coordination module (in the third-layer spatial resolution processing structure) is a network fusion operation, which can be referred to in the above formula. ; Conv k1s1: Conv represents two-dimensional convolution, k1 represents the size of the convolution kernel as 1×1, and s1 represents the stride of the convolution as 1, meaning that the convolution operation moves 1 pixel at a time.

[0080] PReLU: Parametric Modified Linear Unit in related technologies, which enhances nonlinearity and prevents gradient vanishing. The learnable parameters of PReLU provide greater model adaptability. Conv k3s1: k3: indicates that the kernel size is 3×3, s1: indicates that the stride of the convolution is 3; Data stream processing: The input features are first concatenated (©), then processed by Conv k1s1: 1×1 convolution is used to compress or blend the channel dimensions while maintaining spatial resolution. Next, the PReLU activation function is applied: non-linearity is introduced to enhance feature representation. Then, Conv k3s1: 3×3 convolution is used to extract local features. Finally, the PReLU activation function is applied again: non-linearity is further enhanced, making the output features richer.

[0081] Furthermore, the initial context-consistent results will then be processed. After processing by Conv k3s1 and Layer-Norm, the initial context consistency results are fed into the second global quality awareness mechanism and the second local quality awareness mechanism, respectively. Conv k3s1 represents the convolution operation; k3s1 represents the kernel size and amplitude of the convolution network corresponding to the convolution operation. Layer-Norm represents the regularization operation for layers with different spatial resolutions; S4004: The second global quality perception mechanism of the consistency collaboration module determines the context content relevance based on the preliminary context consistency result, and uses the context content relevance to perform global context content quality collaboration processing on the preliminary context consistency result to obtain the global quality collaboration result; The second global quality-aware mechanism in the Consistency Coordination Module (CSM) employs a multi-head self-attention mechanism to evaluate the initial context-consistency results. Globally enhance the quality synergy within the system. When performing global quality synergy... The content is divided into non-overlapping blocks and its autocorrelation is calculated, represented by the fifth calculation formula, to capture the correlation between blocks; Multi-head self-attention mechanism, that is Figure 9 In the k-heads part, the multi-head attention mechanism performs initial contextualization of the input results. The model is divided into multiple subspaces, each of which independently calculates an attention score. These scores are then combined to capture information from different dimensions. This process typically involves dividing the query (Q), key (K), and value (V) into multiple heads, each calculating its own attention score. These results are then concatenated or weighted as the final output. Through multi-head attention, the model can understand input information from multiple perspectives and handle various relevances simultaneously in context optimization, thereby improving its ability to model spatiotemporal information.

[0082] The second global quality awareness mechanism: uses cross-attention processing to calculate the initial context consistency result. Contextual relevance Based on the relevance of the context content The preliminary context consistency result is then subjected to global context content quality co-processing to obtain the global quality co-processing result. .

[0083] exist Figure 9 Second (Global Quality Awareness) (Mechanism): Conv k3s1 represents the convolution operation; k3s1 represents the kernel size and amplitude of the convolutional network corresponding to the convolution operation; Spatio-Norm represents the regularization operation corresponding to the spatial processing layer; Preliminary Context Consistency Results After processing by Conv k3s1, the results are input into three branches, each involving Layer-Norm processing and multiplication operations. (e.g., cross product) The result of the Layer-Norm processing at the top is multiplied by the result of the Layer-Norm processing in the middle branch. The obtained data is then processed using the softmax activation function and a multi-head attention mechanism to determine certain weight coefficients (such as key weight coefficients and value weight coefficients) (refer to the fifth calculation formula) to obtain the contextual content relevance. This contextual content relevance is then compared with the preliminary contextualization result of the third branch input. Through multiplication operation Perform global contextual content quality co-processing with Conv k1s1 (refer to the sixth calculation formula) to obtain the global quality co-processing result. ; Optionally, determining context content relevance based on the preliminary context consistency result through the second global quality awareness mechanism of the consistency collaboration module includes: performing result block relevance parsing processing on the preliminary context consistency result through the second global quality awareness mechanism of the consistency collaboration module to obtain key weight coefficients and value weight coefficients for the preliminary context consistency result; and calculating context content relevance based on the key weight coefficients, value weight coefficients, and the preliminary context consistency result using a fifth calculation formula; the fifth calculation formula satisfies the following formula:

[0084] Among them, the The i-th layer context content relevance (relevance can be a relevance score) is represented by the following. The preliminary context consistency result representing the spatial resolution of the i-th layer, the The key weight coefficients representing the initial context consistency result, the The value weight coefficient represents the initial context consistency result; context content relevance is subsequently used to guide global quality collaboration, as can be found in the sixth calculation formula.

[0085] Optionally, the step of using the context content relevance to perform global context content quality collaborative processing on the preliminary context consistency result to obtain a global quality collaborative result includes: performing global quality enhancement processing on the preliminary context consistency result based on the context content relevance, determining a third weight coefficient for the context content relevance, and performing global context relevance calculation based on the context content relevance of all spatial resolutions to obtain a comprehensive content relevance; and using a sixth calculation formula based on the third weight coefficient, the context content relevance, the preliminary context consistency result, and the comprehensive content relevance to obtain a global quality collaborative result for each spatial resolution; the sixth calculation formula satisfies the following formula:

[0086] Among them, the Q2 represents the global quality coordination result of the spatial resolution of the i-th layer, and Q2 represents the third weight coefficient. S4006: The preliminary context consistency result is processed by residual network processing to obtain a local quality collaboration result by the second local quality perception mechanism of the consistency collaboration module; The second local quality perception mechanism is similar to the first local quality perception mechanism of the quality perception unit, except that the input data for the local quality perception mechanism is different. The input of the second local quality perception mechanism is the preliminary context consistency result, while the second global quality perception mechanism still models local texture details through residual blocks. The preliminary context consistency result is processed by the second local quality perception mechanism of the consistency coordination module. Local quality collaboration results are obtained by using residual network processing operations to perform local context content quality collaborative processing. exist Figure 9 Second (Mechanism) Local Quality Awareness: The system performs convolutional residual network (CRLF) processing on the initial context-consistent results. The CRLF consists of Conv k3s1, PReLU, Conv k3s1, PReLU, and Conv k1s1 (as shown in the diagram). The processing is performed, and the result is consistent with the initial contextualization result. The addition operation ultimately yields a local context-aware result.

[0087] Optionally, the local quality coordination result obtained by performing local context content quality coordination processing on the preliminary context consistency result using residual network processing can be achieved in the following manner: Based on the seventh calculation formula, residual network processing is used to perform local context consistency processing on the preliminary context consistency result to obtain the local quality collaboration result. The seventh calculation formula satisfies the following formula:

[0088] Among them, the The local quality co-processing result representing the spatial resolution of the i-th layer, the This represents the processing operation of the convolutional residual network; S4008: The consistency collaboration module performs result feature aggregation processing on the global quality collaboration result and the local quality collaboration result to obtain a consistency collaboration feature aggregation result. The consistency collaboration feature aggregation result and the context-aware result information are integrated to obtain the context refinement result corresponding to the spatial resolution. The context refinement result and the predicted frame context features are used to perform feature enhancement to obtain the enhanced context information corresponding to each spatial resolution.

[0089] Result feature aggregation: Aggregate the global quality collaboration results and the local quality collaboration results to generate a consistent collaboration feature aggregation result. If the current consistent collaboration module has a previous consistent collaboration module, it can be further optimized by combining the reference consistent collaboration feature aggregation result.

[0090] Feature aggregation methods can be simple fusion, such as weighted summation or concatenation. Alternatively, an attention mechanism can be used, assigning different weights to global and local results to ensure proper fusion in the final features.

[0091] Next, the aggregation module included in the consistency and coordination module merges the global quality coordination results and the estimated results of the local quality coordination results. For example... Figure 9 As shown, the part of the consistency and coordination module other than global quality awareness and local quality awareness can be regarded as the aggregation module. The operation process of the aggregation module can be referred to First, the aggregation module merges the local quality collaboration results and the local quality collaboration results through the © splicing operation. The spliced ​​feature is " "go through , The left branch of the aggregation module consists of Conv k3s1, PReLU, Conv k3s1, GeLU, multiplication operations ☉ (such as dot product), and Conv k1s1, and can be represented as follows: , and then with Perform addition.

[0092] When the consistency collaboration module is the first-layer consistency collaboration module, the consistency collaboration feature aggregation result and the context-aware result information are integrated to obtain the context refinement result corresponding to the spatial resolution. When the consistency coordination module is a second-layer consistency coordination module, the context refinement results and the predicted frame context features can be used. Feature enhancement is performed to obtain enhanced context information corresponding to each spatial resolution.

[0093] Schematic, the aggregation module then merges the global and local quality coordination results, and residual linking is used to control the stability of the consistency coordination process, yielding the final coordination result, which can be represented as follows:

[0094] in This represents the aggregated result of consistent collaborative features output. This indicates a multi-scale fusion (network) processing operation.

[0095] Similarly, the second-layer Consistency Coordination Module (CSM) further refines the process using the same approach. ,get The refined results Subsequently with Combined, the final enhanced context information is generated. And use it as a priori for conditional coding.

[0096] Optionally, the process of integrating the consistent collaborative feature aggregation result and the context-aware result to obtain the context refinement result corresponding to the spatial resolution, and using the context refinement result and the predicted frame context features to perform feature enhancement to obtain the enhanced context information corresponding to each spatial resolution, can refer to the following method: When the consistency collaboration module is the first-layer consistency collaboration module, the consistency collaboration feature aggregation result and the context-aware result information are integrated to obtain the context refinement result corresponding to the spatial resolution; When the consistency coordination module is the second-layer consistency coordination module, the context refinement result and the predicted frame context features can be used. Feature enhancement is performed to obtain enhanced context information corresponding to each spatial resolution.

[0097] In one or more embodiments of this specification, the Coherence and Synergy Module (CSM) achieves coordinated optimization of global semantic consistency and local detail fidelity in multi-resolution context features by combining global and local information through a dual-branch structure that coordinates global and local quality. The global branch utilizes a multi-head self-attention mechanism to capture long-distance dependencies, ensuring semantic relevance between feature blocks; the local branch enhances high-frequency details and local texture information through a convolutional residual network. By fusing global and local features and introducing residual connections, CSM effectively solves the cross-domain inconsistency problem between multi-resolution features. The generated optimized context features possess both global expressive power and preserve local details, providing high-quality prior knowledge for subsequent conditional coding, thereby improving the efficiency and reconstruction quality of video coding.

[0098] The following will combine Figure 10 This manual provides a detailed description of the video encoding device provided. It should be noted that... Figure 10 The video encoding device shown is used to perform the functions described in this specification. Figures 1-9 The methods of the embodiments shown are illustrated only in connection with this specification for ease of explanation. For specific technical details not disclosed, please refer to this specification. Figures 1-9 The example shown.

[0099] Please see Figure 10 This diagram illustrates the structure of the video encoding apparatus described in this specification. The video encoding apparatus 1 can be implemented as all or part of a user's electronic device through software, hardware, or a combination of both. According to some embodiments, the video encoding apparatus 1 includes a data determination module 11, a context prediction module 12, a context optimization module 13, and a video frame reconstruction module 14, specifically used for: The data determination module 11 is used to determine the target video frame to be processed, and to obtain the reference decoding features and reference frame context information corresponding to the reference video frame, wherein the reference frame is the previous video encoded frame of the target video frame. The context prediction module 12 is used to perform frame motion prediction processing based on the reference decoding features and the reference video frame to obtain predicted video frame features, and to extract context features based on the predicted video frame features to obtain predicted frame context information for the target video frame. Context optimization module 13 is used to perform context quality optimization based on the predicted frame context information, the reference frame context information, and the reference decoding features to obtain enhanced context information; The video frame reconstruction module 14 is used to perform frame reconstruction processing on the target video frame based on the enhanced context information to obtain the target video encoded frame.

[0100] Optionally, the context optimization module 13 is used to: A context optimization module is used to determine multiple reference frame context features with different spatial resolutions corresponding to the reference frame context information, and to determine multiple prediction frame context features with different spatial resolutions corresponding to the prediction frame context information. The context optimization module uses the reference decoding features, the reference frame context features of each spatial resolution, and the prediction frame context features to perform quality-aware processing on the context features and decoding feature domains to obtain context-aware result information corresponding to multiple spatial resolutions. The context optimization module then performs context content consistency optimization processing on the context-aware result information of each spatial resolution to obtain context refinement results for different spatial resolutions. Finally, the context refinement results are used to enhance the prediction frame context features to obtain enhanced context information.

[0101] Optionally, the context optimization module includes multiple quality sensing units. The step involves using the reference decoding features, the reference frame context features for each spatial resolution, and the predicted frame context features through the context optimization module to perform quality-aware processing on the context features and the decoding feature domain to obtain context-aware result information corresponding to multiple spatial resolutions, including: The reference decoding features, reference frame context features at each of the spatial resolutions, and the predicted frame context features are transmitted to the quality-aware unit. The first global quality-aware mechanism of the quality-aware unit performs cross-attention processing on the reference frame context features and the reference decoding features to determine the semantic relevance of all reference frame context features and the reference decoding features. The semantic relevance is then used to perform global quality enhancement processing on the predicted frame context features to obtain a global context-aware result for the predicted frame context features. The first local quality-aware mechanism of the quality-aware unit concatenates the reference decoding features, the reference frame context features, and the predicted frame context features to obtain a comprehensive concatenated feature. A convolutional residual network is then used to process the concatenated feature. Local quality enhancement is performed by combining splicing features to obtain a local context-aware result for the context features of the predicted frame. The quality-aware unit aggregates the global context-aware result and the local context-aware result to obtain a quality-aware feature aggregation result, or aggregates the global context-aware result, the local context-aware result, and the reference context-aware result information to obtain a quality-aware feature aggregation result. The quality-aware feature aggregation result and the context features of the predicted frame are combined to obtain context-aware result information corresponding to each spatial resolution. The reference context-aware result information is the context-aware result information of the previous quality-aware unit when the quality-aware unit has a previous quality-aware unit.

[0102] Optionally, the step of determining the semantic relevance of all reference frame context features and reference decoded features by performing cross-attention processing on the reference frame context features and reference decoded features through the first global quality perception mechanism of the quality perception unit includes: performing cross-attention processing on the reference frame context features and reference decoded features through the first global quality perception mechanism of the quality perception unit to determine the first weight information corresponding to the reference frame context features and the second weight information corresponding to the reference decoded features; and obtaining the semantic relevance corresponding to each spatial resolution based on the reference frame context features, the first weight information, the reference decoded features, and the second weight information using a first calculation formula, wherein the first calculation formula satisfies the following formula:

[0103] in, Indicating semantic relevance, the This represents the first weight information, the The reference frame context features, the This represents the second weight information, the The reference decoding feature is represented by i, where i represents the layer number of the spatial resolution. The step of performing global quality enhancement processing on the predicted frame context features using the semantic relevance to obtain a global context-aware result for the predicted frame context features includes: performing global quality enhancement processing on the predicted frame context features based on the semantic relevance, determining the third weight information of the predicted frame context features, and performing context global relevance calculation based on the semantic relevance of all spatial resolutions to obtain a comprehensive semantic relevance; and using a second calculation formula based on the third weight information, the predicted frame context features, the semantic relevance, and the comprehensive semantic relevance to obtain a global context-aware result for the predicted frame context features; the second calculation formula satisfies the following formula:

[0104] Among them, the The global context-aware result representing the predicted frame context features at the i-th spatial resolution, the This represents the third weight information, the The predicted frame context features are represented by the... This refers to the comprehensive semantic relevance; The step of using convolutional residual network processing operations to perform local quality enhancement on the synthesized spliced ​​features to obtain a local context-aware result for the predicted frame context features includes: using convolutional residual network processing operations based on a third calculation formula to perform local quality enhancement on the synthesized spliced ​​features to obtain a local context-aware result for the predicted frame context features, wherein the third calculation formula satisfies the following formula:

[0105] Among them, the The local context-aware result representing the predicted frame context features at the i-th layer spatial resolution, the This represents the combined splicing features after splicing, where RB() represents the convolutional residual network processing operation; The context-aware result information is calculated based on a fourth calculation formula, which includes: When i > 1, the fourth calculation formula is: ; When i=1, the fourth calculation formula is: ; Among them, the The context-aware result information representing the context features of the predicted frame at the i-th spatial resolution, the This represents the operation of a multi-spatial-resolution context fusion network, the This indicates multi-scale context fusion network processing operations.

[0106] Optionally, the context optimization module includes multiple consistency coordination modules. The context optimization module performs context content consistency optimization processing on the context-aware result information at different spatial resolutions to obtain context refinement results at different spatial resolutions. The context refinement results are then used to enhance the context features of the predicted frame to obtain enhanced context information, including: The consistency and coordination module performs content consistency processing on the context-aware result information and reference context-aware result information at different spatial resolutions to obtain preliminary context consistency results for each spatial resolution. The second global quality perception mechanism of the consistency and coordination module determines the context content relevance based on the preliminary context consistency results, and uses the context content relevance to perform global context content quality coordination processing on the preliminary context consistency results to obtain a global quality coordination result. The second local quality perception mechanism of the consistency and coordination module performs local context content quality coordination processing on the preliminary context consistency results using residual network processing operations to obtain a local quality coordination result. The consistency and coordination module performs result feature aggregation processing on the global quality coordination result and the local quality coordination result to obtain a consistency coordination feature aggregation result. The consistency coordination feature aggregation result and the context-aware result information are then integrated to obtain the context refinement result corresponding to the spatial resolution. The context refinement result and the predicted frame context features are used for feature enhancement to obtain enhanced context information corresponding to each spatial resolution.

[0107] Optionally, determining context content relevance based on the preliminary context consistency result through the second global quality awareness mechanism of the consistency collaboration module includes: performing result block relevance parsing processing on the preliminary context consistency result through the second global quality awareness mechanism of the consistency collaboration module to obtain key weight coefficients and value weight coefficients for the preliminary context consistency result; and calculating context content relevance based on the key weight coefficients, value weight coefficients, and the preliminary context consistency result using a fifth calculation formula; the fifth calculation formula satisfies the following formula:

[0108] Among them, the Indicates the relevance of the i-th layer of context content, the The preliminary context consistency result representing the spatial resolution of the i-th layer, the The key weight coefficients representing the initial context consistency result, the The value weight coefficient represents the initial context consistency result; The step of using the context content relevance to perform global context content quality collaborative processing on the preliminary context consistency result to obtain a global quality collaborative result includes: performing global quality enhancement processing on the preliminary context consistency result based on the context content relevance, determining the third weight coefficient of the context content relevance, and performing global context relevance calculation based on the context content relevance of all spatial resolutions to obtain a comprehensive content relevance; and using a sixth calculation formula based on the third weight coefficient, the context content relevance, the preliminary context consistency result, and the comprehensive content relevance to obtain the global quality collaborative result for each spatial resolution; the sixth calculation formula satisfies the following formula:

[0109] Among them, the Q2 represents the global quality coordination result of the spatial resolution of the i-th layer, and Q2 represents the third weight coefficient. The step of performing local context content quality co-processing on the preliminary context consistency result using residual network processing to obtain a local quality co-processing result includes: performing local context consistency processing on the preliminary context consistency result using residual network processing based on a seventh calculation formula to obtain a local quality co-processing result, wherein the seventh calculation formula satisfies the following formula:

[0110] Among them, the The local quality co-processing result representing the spatial resolution of the i-th layer, the This represents the processing operation of the convolutional residual network; The step of integrating the consistency collaboration feature aggregation result and the context-aware result information to obtain the context refinement result corresponding to the spatial resolution, and using the context refinement result and the predicted frame context features to perform feature enhancement to obtain the enhanced context information corresponding to each spatial resolution, includes: when the consistency collaboration module is a first-layer consistency collaboration module, integrating the consistency collaboration feature aggregation result and the context-aware result information to obtain the context refinement result corresponding to the spatial resolution; when the consistency collaboration module is a second-layer consistency collaboration module, using the context refinement result and the predicted frame context features to perform feature enhancement to obtain the enhanced context information corresponding to each spatial resolution.

[0111] Optionally, the video frame reconstruction module 14 is configured to: obtain target frame reconstruction features by performing feature reconstruction based on the enhanced context information and the target video frame through the conditional coding module, and obtain target video encoded frames by performing frame reconstruction processing on the target video frame based on the target frame reconstruction features through the conditional coding module.

[0112] Optionally, the video frame reconstruction module 14 is configured to: perform context feature information transfer on the target video frame based on the enhanced context information using the conditional encoder to obtain feature compression representation information; perform frame feature decoding based on the enhanced context information and the feature compression representation information using the conditional decoder to obtain target frame reconstruction features; and perform frame reconstruction processing on the target video frame based on the target frame reconstruction features to obtain target video encoded frames.

[0113] Optionally, the data determination module 11 is configured to: perform frame image motion estimation processing based on the reference video frame to obtain a motion vector field; perform motion encoding based on the motion vector field to obtain motion representation information; perform motion decoding on the motion representation information to obtain a motion vector; and perform motion compensation processing based on the motion vector and the reference decoded features to obtain predicted video frame features.

[0114] It should be noted that the video encoding apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when executing the video encoding method. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the video encoding apparatus and the video encoding method embodiments provided in the above embodiments belong to the same concept, and the implementation process is detailed in the method embodiments, which will not be repeated here.

[0115] The serial numbers in this specification are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0116] This specification also provides a computer storage medium capable of storing multiple instructions adapted to be loaded and executed by a processor as described above. Figures 1-9 The video encoding method described in the illustrated embodiment can be found in the following documentation for its specific execution process. Figures 1-9 The specific details of the illustrated embodiments will not be elaborated here.

[0117] This specification also provides a computer program product that stores at least one instruction, said at least one instruction being loaded and executed by the processor as described above. Figures 1-9 The video encoding method described in the illustrated embodiment can be found in the following documentation for its specific execution process. Figures 1-9 The specific details of the illustrated embodiments will not be elaborated here.

[0118] Please refer to Figure 11 This is a structural block diagram of an electronic device provided in an embodiment of this specification. The electronic device in this specification may include one or more of the following components: a processor 1010, a memory 1020, an input device 1030, an output device 1040, and a bus 1050. The processor 1010, memory 1020, input device 1030, and output device 1040 may be connected to each other via the bus 1050.

[0119] Processor 1010 may include one or more processing cores. Processor 1010 connects to various parts of the electronic device using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 1020, and by calling data stored in memory 1020. Optionally, processor 1010 may be implemented using at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). Processor 1010 may integrate one or more of a central processing unit (CPU), graphics processing unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into processor 1010 and may be implemented separately through a communication chip.

[0120] The memory 1020 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 1020 may include non-transitory computer-readable storage medium. The memory 1020 may be used to store instructions, programs, code, code sets, or instruction sets.

[0121] The input device 1030 is used to receive input instructions or data, and includes, but is not limited to, a keyboard, mouse, camera, microphone, or touch device. The output device 1040 is used to output instructions or data, and includes, but is not limited to, a display device and a speaker. In this embodiment, the input device 1030 can be a temperature sensor for acquiring the operating temperature of the electronic device. The output device 1040 can be a speaker for outputting audio signals.

[0122] In addition, those skilled in the art will understand that the structure of the electronic device shown in the above figures does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements. For example, the electronic device may also include radio frequency circuits, input units, sensors, audio circuits, wireless fidelity (WIFI) modules, power supplies, Bluetooth modules, etc., which will not be described in detail here.

[0123] In the embodiments of this specification, the executing entity for each step can be the electronic device described above. Optionally, the executing entity for each step can be the operating system of the electronic device. The operating system can be Android, iOS, or other operating systems; this specification does not limit this.

[0124] exist Figure 11 In the electronic device, the processor 1010 can be used to call a program stored in the memory 1020 and execute it to implement the video encoding method as described in the various method embodiments of this specification.

[0125] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory, or random access memory, etc.

[0126] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in the embodiments of this specification are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the object characteristics, interactive behavior characteristics, and user information involved in this specification were all obtained under full authorization.

[0127] The above-disclosed embodiments are merely preferred embodiments of this specification and should not be construed as limiting the scope of this specification. Therefore, any equivalent variations made in accordance with the claims of this specification shall still fall within the scope of this specification.

Claims

1. A video encoding method, the method comprising: The target video frame to be processed is determined, and the reference decoding features and reference frame context information corresponding to the reference video frame are obtained. The reference video frame is the previous video encoded frame of the target video frame. Based on the reference decoding features and the reference video frame, frame motion prediction processing is performed to obtain predicted video frame features, and based on the predicted video frame features, context feature extraction is performed to obtain predicted frame context information for the target video frame. Enhanced context information is obtained by performing context quality optimization based on the predicted frame context information, the reference frame context information, and the reference decoding features; Based on the enhanced context information, the target video frame is reconstructed to obtain the target video encoded frame. The enhanced context information is obtained by performing context quality optimization based on the predicted frame context information, the reference frame context information, and the reference decoding features, including: A context optimization module is used to determine multiple reference frame context features with different spatial resolutions corresponding to the reference frame context information, and to determine multiple prediction frame context features with different spatial resolutions corresponding to the prediction frame context information. The context optimization module uses the reference decoding features, the reference frame context features for each spatial resolution, and the predicted frame context features to perform quality-aware processing on the context features and the decoding feature domain to obtain context-aware result information corresponding to multiple spatial resolutions. The context optimization module performs context content consistency optimization on the context-aware result information at different spatial resolutions to obtain context refinement results at different spatial resolutions. The context refinement results are then used to enhance the context features of the predicted frame to obtain enhanced context information.

2. The method according to claim 1, wherein the context optimization module comprises multiple quality sensing units. The step involves using the reference decoding features, the reference frame context features for each spatial resolution, and the predicted frame context features through the context optimization module to perform quality-aware processing on the context features and the decoding feature domain to obtain context-aware result information corresponding to multiple spatial resolutions, including: The reference decoding features, the reference frame context features for each of the spatial resolutions, and the predicted frame context features are transmitted to the quality sensing unit. The first global quality perception mechanism of the quality perception unit performs cross-attention processing on the reference frame context features and the reference decoding features to determine the semantic correlation between the reference frame context features and the reference decoding features. The semantic correlation is then used to perform global quality enhancement processing on the predicted frame context features to obtain a global context perception result for the predicted frame context features. The reference decoding features, the reference frame context features, and the predicted frame context features are concatenated using the first local quality perception mechanism of the quality perception unit to obtain a comprehensive concatenated feature. A convolutional residual network is then used to perform local quality enhancement on the comprehensive concatenated feature to obtain a local context perception result for the predicted frame context features. The quality perception unit performs feature aggregation on the global context perception result and the local context perception result to obtain a quality perception feature aggregation result, or performs feature aggregation on the global context perception result, the local context perception result, and the reference context perception result information to obtain a quality perception feature aggregation result. The quality perception feature aggregation result and the predicted frame context features are then combined to obtain context perception result information corresponding to each spatial resolution. The reference context perception result information is the context perception result information of the previous quality perception unit when the quality perception unit has a previous quality perception unit.

3. The method according to claim 2, The step of determining the semantic relevance of the reference frame context features and the reference decoded features by performing cross-attention processing on the reference frame context features and the reference decoded features through the first global quality perception mechanism of the quality perception unit includes: The reference frame context features and the reference decoded features are cross-attention processed by the first global quality perception mechanism of the quality perception unit to determine the first weight information corresponding to the reference frame context features and the second weight information corresponding to the reference decoded features. Based on the reference frame context features, the first weight information, the reference decoded features, and the second weight information, the semantic relevance corresponding to each spatial resolution is obtained by a first calculation formula. The first calculation formula satisfies the following formula: in, Indicating semantic relevance, the This represents the first weight information, the The reference frame context features, the This represents the second weight information, the The reference decoding feature is represented by i, where i represents the layer number of the spatial resolution. The step of performing global quality enhancement processing on the predicted frame context features using the semantic relevance to obtain a global context-aware result for the predicted frame context features includes: performing global quality enhancement processing on the predicted frame context features based on the semantic relevance, determining the third weight information of the predicted frame context features, and performing context global relevance calculation based on the semantic relevance of all spatial resolutions to obtain a comprehensive semantic relevance; and using a second calculation formula based on the third weight information, the predicted frame context features, the semantic relevance, and the comprehensive semantic relevance to obtain a global context-aware result for the predicted frame context features; the second calculation formula satisfies the following formula: Among them, the The global context-aware result representing the predicted frame context features at the i-th spatial resolution, the This represents the third weight information, the The predicted frame context features are represented by the... This refers to the comprehensive semantic relevance; The step of using convolutional residual network processing operations to perform local quality enhancement on the synthesized spliced ​​features to obtain a local context-aware result for the predicted frame context features includes: using convolutional residual network processing operations based on a third calculation formula to perform local quality enhancement on the synthesized spliced ​​features to obtain a local context-aware result for the predicted frame context features, wherein the third calculation formula satisfies the following formula: Among them, the The local context-aware result representing the predicted frame context features at the i-th layer spatial resolution, the This represents the combined splicing features after splicing, where RB() represents the convolutional residual network processing operation; The context-aware result information is calculated based on a fourth calculation formula, which includes: When i > 1, the fourth calculation formula is: ; When i=1, the fourth calculation formula is: ; Among them, the The context-aware result information representing the context features of the predicted frame at the i-th spatial resolution, the This represents the operation of a multi-spatial-resolution context fusion network, the This indicates multi-scale context fusion network processing operations.

4. The method according to claim 1, wherein the context optimization module comprises multiple consistency coordination modules, and the step of performing context content consistency optimization processing on the context-aware result information of different spatial resolutions through the context optimization module to obtain context refinement results of different spatial resolutions, and using the context refinement results to perform feature enhancement on the predicted frame context features to obtain enhanced context information, includes: The consistency coordination module performs content consistency processing on the context-aware result information and reference context-aware result information at different spatial resolutions to obtain the preliminary context consistency result for each spatial resolution. The second global quality perception mechanism of the consistency collaboration module determines the context content relevance based on the preliminary context consistency result, and uses the context content relevance to perform global context content quality collaboration processing on the preliminary context consistency result to obtain the global quality collaboration result. The preliminary context consistency result is processed by residual network processing to obtain a local quality collaboration result by the second local quality perception mechanism of the consistency collaboration module. The consistency collaboration module performs result feature aggregation processing on the global quality collaboration result and the local quality collaboration result to obtain the consistency collaboration feature aggregation result. The consistency collaboration feature aggregation result and the context-aware result information are integrated to obtain the context refinement result corresponding to the spatial resolution. The context refinement result and the predicted frame context features are used to perform feature enhancement to obtain the enhanced context information corresponding to each spatial resolution.

5. The method according to claim 4, The second global quality awareness mechanism through the consistency collaboration module determines context content relevance based on the preliminary context consistency result, including: The second global quality perception mechanism of the consistency collaboration module performs result block relevance parsing on the preliminary context consistency result to obtain the key weight coefficient and value weight coefficient for the preliminary context consistency result. Based on the key weight coefficient, value weight coefficient and the preliminary context consistency result, the fifth calculation formula is used to calculate the context content relevance. The fifth calculation formula satisfies the following formula: Among them, the Indicates the relevance of the i-th layer of context content, the The preliminary context consistency result representing the spatial resolution of the i-th layer, the The key weight coefficients representing the initial context consistency result, the The value weight coefficient represents the initial context consistency result; The step of using the context content relevance to perform global context content quality collaborative processing on the preliminary context consistency result to obtain a global quality collaborative result includes: performing global quality enhancement processing on the preliminary context consistency result based on the context content relevance, determining the third weight coefficient of the context content relevance, and performing global context relevance calculation based on the context content relevance of all spatial resolutions to obtain a comprehensive content relevance; and using a sixth calculation formula based on the third weight coefficient, the context content relevance, the preliminary context consistency result, and the comprehensive content relevance to obtain the global quality collaborative result for each spatial resolution; the sixth calculation formula satisfies the following formula: Among them, the The global quality coordinating result represents the spatial resolution of the i-th layer. The Q2 represents the third weighting coefficient, indicating the overall content relevance. The step of performing local context content quality co-processing on the preliminary context consistency result using residual network processing to obtain a local quality co-processing result includes: performing local context consistency processing on the preliminary context consistency result using residual network processing based on a seventh calculation formula to obtain a local quality co-processing result, wherein the seventh calculation formula satisfies the following formula: Among them, the The local quality co-processing result representing the spatial resolution of the i-th layer, the This represents the convolutional residual network processing operation, the... This indicates the preliminary context consistency result; The process of integrating the consistent collaborative feature aggregation result and the context-aware result information to obtain the context refinement result corresponding to the spatial resolution, and using the context refinement result and the predicted frame context features to perform feature enhancement to obtain the enhanced context information corresponding to each spatial resolution, includes: When the consistency collaboration module is the first-layer consistency collaboration module, the consistency collaboration feature aggregation result and the context-aware result information are integrated to obtain the context refinement result corresponding to the spatial resolution; When the consistency coordination module is the second-layer consistency coordination module, the context refinement result and the predicted frame context features are used to perform feature enhancement to obtain the enhanced context information corresponding to each spatial resolution.

6. The method according to claim 1, wherein performing frame reconstruction processing on the target video frame based on the enhanced context information to obtain the target video encoded frame comprises: The target frame reconstruction features are obtained by the conditional coding module based on the enhanced context information and the target video frame. The target video frame is then reconstructed by the conditional coding module based on the target frame reconstruction features to obtain the target video encoded frame.

7. The method according to claim 6, wherein the conditional coding module includes a conditional encoder and a conditional decoder, and the step of obtaining target frame reconstruction features by performing feature reconstruction based on the enhanced context information and the target video frame through the conditional coding module, and obtaining target video coded frames by performing frame reconstruction processing on the target video frame based on the target frame reconstruction features through the conditional coding module, includes: The conditional encoder performs context feature information transfer on the target video frame based on the enhanced context information to obtain feature compression representation information; The target frame reconstruction features are obtained by the conditional decoder performing frame feature decoding based on the enhanced context information and the feature compression representation information. The target video frame is reconstructed based on the target frame reconstruction features to obtain the target video encoded frame.

8. The method according to claim 1, wherein the step of performing frame motion prediction processing based on the reference decoding features and the reference video frame to obtain predicted video frame features includes: The motion vector field is obtained by performing frame image motion estimation processing based on the reference video frame. Motion representation information is obtained by motion encoding based on the motion vector field, and motion representation information is obtained by motion decoding to obtain motion vectors; Motion compensation processing is performed based on the motion vectors and the reference decoding features to obtain the predicted video frame features.

9. A video encoding apparatus, the apparatus comprising: The data determination module is used to determine the target video frame to be processed, and to obtain the reference decoding features and reference frame context information corresponding to the reference video frame, wherein the reference video frame is the previous video encoded frame of the target video frame. The context prediction module is used to perform frame motion prediction processing based on the reference decoding features and the reference video frame to obtain predicted video frame features, and to extract context features based on the predicted video frame features to obtain predicted frame context information for the target video frame. The context optimization module is used to perform context quality optimization based on the predicted frame context information, the reference frame context information, and the reference decoding features to obtain enhanced context information; A video frame reconstruction module is used to perform frame reconstruction processing on the target video frame based on the enhanced context information to obtain a target video encoded frame; The context optimization module is used for: A context optimization module is used to determine multiple reference frame context features with different spatial resolutions corresponding to the reference frame context information, and to determine multiple prediction frame context features with different spatial resolutions corresponding to the prediction frame context information. The context optimization module uses the reference decoding features, the reference frame context features of each spatial resolution, and the prediction frame context features to perform quality-aware processing on the context features and decoding feature domains to obtain context-aware result information corresponding to multiple spatial resolutions. The context optimization module then performs context content consistency optimization processing on the context-aware result information of each spatial resolution to obtain context refinement results for different spatial resolutions. Finally, the context refinement results are used to enhance the prediction frame context features to obtain enhanced context information.

10. A computer storage medium storing a plurality of instructions adapted for loading by a processor and executing the method steps of any one of claims 1 to 8.

11. A computer program product storing at least one instruction, said at least one instruction being loaded by a processor and executing the method steps of any one of claims 1 to 8.

12. An electronic device, comprising: A processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and executed the method steps as claimed in any one of claims 1 to 8.

Images