An artificial intelligence-based dynamic video resolution adaptive method

Abstract

The application discloses a kind of dynamic video resolution self-adaptive methods based on artificial intelligence, including step one: the playing state data in the process of video playing is collected;Step two: the playing state data is preprocessed, obtains standard video frame tensor and standard environment characteristic vector;Step three: standard video frame tensor is generated visual content representation vector through improved MobileViT network;Step four: visual content representation vector and standard environment characteristic vector are spliced and fused;Step five: resolution self-adaptive prediction is carried out, obtains initial predicted resolution level, resolution adjustment amplitude, resolution confidence and upgrade trend parameter;Step six: resolution upgrade strategy, resolution downgrade strategy and smooth transition strategy are executed, and target resolution level is obtained;Step seven: the video frame rendered using target resolution level is output.The application improves the picture quality stability, playing fluency and adaptive intelligence in the process of video playing.

Description

Technical Field

[0001] This invention relates to the field of video processing and streaming media transmission technology, and in particular to a dynamic video resolution adaptive method based on artificial intelligence. Background Technology

[0002] With the rapid popularization of mobile internet, multi-terminal video playback platforms, and ultra-high-definition video content, optimization technologies for streaming media playback experience under complex network conditions have received widespread attention in the industry. Existing video players and content delivery networks generally employ fixed-resolution rendering or adaptive bitrate control based on bandwidth thresholds, switching resolutions by monitoring cache usage, instantaneous bandwidth, or download speed. In practical applications, these methods commonly suffer from the following problems: Existing resolution adaptive strategies primarily rely on network throughput or buffer changes for decision-making, neglecting the complexity of video content textures, brightness variations, and dynamic motion information. This leads to significant differences in video quality under the same bandwidth conditions, easily resulting in excessive degradation or unnecessary upgrades. Traditional bitrate control methods based on rules or statistical models cannot handle scenarios involving multiple coupled factors such as network jitter, sudden packet loss, fluctuations in device computing power, and changes in ambient lighting. Their prediction accuracy is insufficient, making it difficult to ensure a balance between playback continuity and visual clarity. Existing image quality assessment methods based on convolutional neural networks mostly focus on static frame feature extraction, lacking the ability to model content changes across time steps. They cannot identify semantic-level video content features that affect clarity requirements, such as high-frequency texture areas, fast-moving objects, or low-brightness scenes, thus reducing the reliability and timeliness of resolution adaptive decisions. Resolution adjustment mechanisms generally suffer from slow response, frequent skipping, or inconsistent quality between frames. Traditional smoothing strategies lack constraints on prediction confidence and historical resolution trends, easily causing fluctuations in playback experience.

[0003] Therefore, how to provide an AI-based dynamic video resolution adaptation method is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] One objective of this invention is to propose an artificial intelligence-based dynamic video resolution adaptation method. This invention constructs an improved MobileViT network, incorporating color enhancement, brightness modeling, structural complexity analysis, and spatiotemporal attention modeling, to extract semantic-level visual content representation vectors reflecting video clarity requirements. A standard environment feature vector is constructed by combining network state, device computing power, and viewing environment during playback. The visual content representation vector and the standard environment feature vector are then concatenated and fused to obtain a video state feature vector. This video state feature vector is then used for resolution level prediction and dynamic strategy adjustment, performing resolution upgrades, downgrades, and smooth switching under multi-factor constraints. This invention possesses advantages such as strong real-time performance, high adaptability, and interpretable adjustment strategies, effectively improving video rendering stability and viewing experience under complex network conditions and multi-terminal playback environments.

[0005] According to an embodiment of the present invention, a dynamic video resolution adaptive method based on artificial intelligence includes the following steps: Step 1: Collect playback status data during video playback. The playback status data includes video frames, resolution parameters, network status parameters, device computing power parameters, and display environment parameters. Step 2: Align and normalize the video frames and calculate the brightness difference to obtain the standard video frame tensor; normalize the resolution parameters, network status parameters, device computing power parameters and display environment parameters to obtain the standard environment feature vector; Step 3: Input the standard video frame tensor into the improved MobileViT network to generate a visual content representation vector; the improved MobileViT network includes a color channel enhancement module, a luminance differential coding module, a structural complexity analysis module, and a spatiotemporal MobileViT Block feature coding module; Step 4: Concatenate the visual content representation vector with the standard environment feature vector along the feature dimension, and fuse them through a two-layer MLP structure to generate a video state feature vector; Step 5: Perform adaptive resolution prediction based on video state feature vectors to obtain the initial predicted resolution level, resolution adjustment range, resolution confidence, and upgrade trend parameters; Step 6: Based on the initial predicted resolution level, resolution adjustment range, resolution confidence and upgrade trend parameters, execute the resolution upgrade strategy, resolution downgrade strategy and smooth transition strategy to obtain the target resolution level, and update the resolution parameters of the video rendering module based on the target resolution level; Step 7: Output video frames rendered at the target resolution level and update the historical resolution level sequence.

[0006] Optionally, the network status parameters include network bandwidth, packet loss rate, and latency; the device computing power parameters include CPU utilization, GPU utilization, device temperature, and device power consumption; and the display environment parameters include ambient light intensity and user viewing distance.

[0007] Optionally, step two specifically includes: The video frame at the current time step is aligned to the set image size, and all pixel values ​​of the video frame are mapped to the [0,1] interval using the max-min normalization method to obtain the standard video frame tensor; the standard video frame tensor contains three channels, namely the R channel, the G channel, and the B channel; The resolution parameters, network status parameters, device computing power parameters, and display environment parameters of the current time step are normalized using the min-max normalization method and then arranged in order to form a standard environment feature vector.

[0008] Optionally, the color channel enhancement module performs color modeling and channel recalibration on the standard video frame tensor to generate a color-enhanced feature tensor, specifically: Based on the R, G, and B channels of the standard video frame tensor, the color difference between the R and G channels, the color difference between the G and B channels, and the color difference between the R and B channels are calculated respectively, and organized into color difference tensors according to the pixel positions. The color difference matrix and the standard video frame tensor are concatenated along the channel dimension to obtain the color video frame tensor. Perform a 1×1 convolution operation on the color video frame tensor to extract cross-channel fusion features, generate channel response video frame tensors, and perform global average pooling on each channel of the channel response video frame tensor to obtain channel statistical vectors. The channel statistical vector is input into two fully connected layers to perform feature dimension compression and restoration operations, generating a channel recalibration weight vector. The color enhancement feature tensor is obtained by performing element-wise multiplication of the channel recalibration weight vector and the channel response video frame tensor along the channel dimension.

[0009] Optionally, the luminance differential coding module performs luminance differential modeling on the standard video frame tensor to generate a luminance enhancement feature tensor, specifically: Obtain the R-channel, G-channel, and B-channel values ​​of each pixel in the standard video frame tensor, calculate the luminance channel value of each pixel using the luminance formula, and assemble all luminance channel values ​​into a luminance matrix according to the pixel position order. Calculate the difference between the two luminance matrices at the current time step and the previous time step to obtain the luminance difference matrix; The luminance difference matrix and the standard video frame tensor are concatenated along the channel dimension to generate the luminance fused video frame tensor. A 3×3 convolution and a 1×1 convolution are performed on the luminance fusion video frame tensor, respectively. The 3×3 convolution is used to extract local luminance change features, and the 1×1 convolution is used to perform channel mapping to generate a luminance enhancement feature tensor.

[0010] Optionally, the structural complexity analysis module includes a texture density extraction branch, an edge response extraction branch, and a frequency response extraction branch. These branches perform local texture gradient statistics, edge intensity operator convolution, and spectral energy modeling on the brightness matrix of the standard video frame tensor, respectively, and then concatenate them along the channel dimension to generate a structural fusion feature tensor. Specifically: Input the brightness matrix of the current time step into the texture density extraction branch, set a window area with each pixel as the center, calculate the average brightness within the window area, and calculate the square of the difference between the brightness value of each pixel and the average brightness to obtain the squared difference of each pixel within the window area. The local brightness variance value of the current pixel is obtained by summing the squared differences of all pixels in the window area and then dividing by the total number of pixels in the window area. The local brightness variance values ​​of all pixels are then organized into a texture density feature matrix according to the pixel position. The brightness matrix of the current time step is input into the edge response extraction branch, and the horizontal gradient matrix and vertical gradient matrix are generated by the horizontal Sobel convolution kernel and the vertical Sobel convolution kernel, respectively. The edge intensity matrix is ​​obtained by squaring, summing, and taking the square root of the values ​​at corresponding positions in the horizontal and vertical gradient matrices. The brightness matrix at the current time step is input into the frequency response extraction branch, and a frequency domain feature matrix is ​​generated by two-dimensional fast Fourier transform; the amplitude of the frequency domain feature matrix is ​​calculated to obtain the spectral energy matrix. The texture density feature matrix, edge response matrix, and spectral energy matrix are concatenated along the channel dimension to obtain the structural fusion feature tensor.

[0011] Optionally, the spatiotemporal MobileViT Block feature encoding module concatenates the color enhancement feature tensor, brightness enhancement feature tensor, and structural fusion feature tensor along the channel dimension to form a fused feature tensor. Then, through a time-shifting mechanism, multi-head self-attention operation, and global average pooling, spatiotemporal feature encoding is performed on the fused feature tensor to obtain a visual content representation vector, specifically: The fused feature tensor is divided into several non-overlapping image blocks according to a set size, and all the non-overlapping image blocks are expanded to obtain a two-dimensional token sequence; A time offset mechanism is introduced to time offset the two-dimensional token sequence to obtain a time-offset token sequence. The time offset mechanism is as follows: the channel dimension features of the two-dimensional token sequence are divided into three parts. The features of the first third of the channel dimension are shifted forward by one time step, the middle third remains unchanged, and the features of the last third of the channel dimension are shifted backward by one time step. Perform multi-head self-attention operation on the temporally offset token sequence to obtain the spatiotemporal attention feature sequence; The spatiotemporal attention feature sequence is structurally reconstructed, and features are fused and dimension compressed through 3×3 convolution and 1×1 convolution to generate a spatiotemporal encoded feature tensor; the spatiotemporal encoded feature tensor has the same structure as the fused feature tensor. Global average pooling is performed on the spatiotemporal coding feature tensor in both the time and spatial dimensions to obtain the visual content representation vector.

[0012] Optionally, step five specifically includes: The video state feature vector is input into four prediction sub-modules to obtain the initial predicted resolution level, resolution adjustment range, resolution confidence, and upgrade trend parameters. These four prediction sub-modules are independently trained sub-network structures, specifically: The video state feature vector is passed through a fully connected layer and the Softmax function to generate a resolution level probability vector, and the index corresponding to the maximum probability value in the resolution level probability vector is used as the initial predicted resolution level. A fully connected network and the ReLU activation function are used to transform the video state feature vector into non-negative values, which are then used as the resolution adjustment range. The video state feature vector is passed through a fully connected network and a sigmoid activation function to generate a resolution confidence score, which reflects the reliability of the current prediction result. A fully connected network and a Tanh activation function are used to transform the video state feature vector into an upscaling trend parameter, which reflects the preference for higher resolution in the current video state.

[0013] Optionally, step six specifically includes: The resolution upgrade strategy is as follows: Set the upscaling trend threshold and the maximum resolution level; If the upgrade trend parameter is greater than the upgrade trend threshold, and the initial predicted resolution level is lower than the maximum resolution level, then, without exceeding the maximum resolution level, the resolution adjustment range is multiplied by the set total number of resolution levels and rounded up to obtain the upgrade level number. The initial predicted resolution level is then added to the upgrade level number to obtain the updated resolution level. : ; in, Indicates the initial prediction resolution level. Indicates the resolution adjustment range. This indicates the total number of resolution levels set. The symbol represents the rounding up sign. Indicates the maximum resolution level. Describes the minimum value function; The resolution degradation strategy is specifically as follows: Set the downshift trend threshold and minimum resolution level; If the upscaling trend parameter is less than the downscaling trend threshold, and the initial predicted resolution level is higher than the minimum resolution level, then, provided it is not lower than the minimum resolution level, the downscaling level number is calculated based on the resolution confidence and resolution adjustment magnitude. The updated resolution level is then obtained by subtracting the downscaling level number from the initial predicted resolution level. : ; in, Indicates the minimum resolution level. Indicates resolution confidence. Represents the maximum value function; Set the historical sliding window range, obtain the historical resolution level sequence within the historical sliding window range, and calculate the historical average resolution level; A resolution change threshold is set. If the difference between the updated resolution level and the historical average resolution level is greater than the resolution change threshold, a smooth transition strategy is executed, specifically: The resolution confidence level is multiplied by the updated resolution level in a weighted manner. The resolution confidence level is subtracted from 1 and multiplied by the historical average resolution level in a weighted manner. The two are then added together to obtain the smoothed resolution level. Based on resolution upgrade strategy, resolution downgrade strategy and smooth transition strategy, the updated resolution level or smooth resolution level is determined as the target resolution level, and the target resolution level is set as the resolution parameter of the video rendering module.

[0014] The beneficial effects of this invention are: First, the improved MobileViT network effectively enhances its sensitivity to color distribution and illumination changes in video frames by introducing a color channel enhancement module and a luminance differential coding module, thereby strengthening its structural perception capabilities in low-light scenes and areas with abrupt changes in strong light. Combined with a structural complexity analysis module, it performs fine-grained spatial analysis of indicators such as edge intensity, texture density, and visual frequency distribution, significantly improving the model's ability to express the complexity of image content. Furthermore, by integrating a spatiotemporal MobileViT Block feature coding module and combining local convolution and global attention mechanisms, it achieves dynamic modeling and temporal context extraction of video frame sequences, effectively improving the network's accuracy in perceiving content clarity requirements under continuous frame changes.

[0015] Secondly, this invention constructs a resolution upgrade strategy, a resolution downgrade strategy, and a smooth transition strategy. The resolution upgrade strategy, based on the resolution adjustment magnitude and upgrade trend parameters, performs a step-by-step upgrade operation under the premise of sufficient bandwidth margin and computing power, ensuring that image details are fully displayed. The resolution downgrade strategy, in scenarios such as network congestion, degraded device performance, or worsened lighting conditions, actively downgrades the resolution based on the predicted resolution confidence level, resolution adjustment magnitude, and upgrade trend parameters to prevent frequent oscillating switching. The smooth transition strategy utilizes the fusion result of historical resolution level sequences and the current updated resolution level to dynamically adjust the resolution switching rate and magnitude, achieving visual consistency between consecutive frames and significantly alleviating visual problems such as frame skipping, tearing, and flickering.

[0016] Furthermore, the method of this invention demonstrates high adaptability and robustness under different terminal devices, heterogeneous network conditions, and complex playback environments. Through the synergistic effect of the improved MobileViT network and the three strategies, the system can proactively perceive the current video playback status and make content-driven resolution decisions, avoiding the limitations of traditional bandwidth- or cache-triggered switching methods in terms of perception accuracy and response stability. This ensures that high-quality video playback is maintained while effectively controlling loading pressure under bandwidth fluctuations.

[0017] In summary, this invention enables adaptive resolution switching in dynamic environments, improves the stability and visual continuity of video playback, enhances the system's generalization ability to diverse playback scenarios and improves user experience, and has significant practical application value and promotion potential. Attached Figure Description

[0018] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1This is a schematic diagram of a dynamic video resolution adaptive method based on artificial intelligence proposed in this invention; Figure 2 This is a flowchart of the improved MobileViT network structure in an artificial intelligence-based dynamic video resolution adaptive method proposed in this invention. Figure 3 This is a flowchart of the process for generating target resolution levels in a dynamic video resolution adaptive method based on artificial intelligence proposed in this invention. Detailed Implementation

[0019] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0020] refer to Figures 1-3 An AI-based dynamic video resolution adaptation method includes the following steps: Step 1: Collect playback status data during video playback. The playback status data includes video frames, resolution parameters, network status parameters, device computing power parameters, and display environment parameters. Step 2: Align and normalize the video frames and calculate the brightness difference to obtain the standard video frame tensor; normalize the resolution parameters, network status parameters, device computing power parameters and display environment parameters to obtain the standard environment feature vector; Step 3: Input the standard video frame tensor into the improved MobileViT network to generate a visual content representation vector; the improved MobileViT network includes a color channel enhancement module, a luminance differential coding module, a structural complexity analysis module, and a spatiotemporal MobileViT Block feature coding module; Step 4: Concatenate the visual content representation vector with the standard environment feature vector along the feature dimension, and fuse them through a two-layer MLP structure to generate a video state feature vector; Step 5: Perform adaptive resolution prediction based on video state feature vectors to obtain the initial predicted resolution level, resolution adjustment range, resolution confidence, and upgrade trend parameters; Step 6: Based on the initial predicted resolution level, resolution adjustment range, resolution confidence and upgrade trend parameters, execute the resolution upgrade strategy, resolution downgrade strategy and smooth transition strategy to obtain the target resolution level, and update the resolution parameters of the video rendering module based on the target resolution level; Step 7: Output video frames rendered at the target resolution level and update the historical resolution level sequence.

[0021] In this embodiment, the network status parameters include network bandwidth, packet loss rate, and latency; the device computing power parameters include CPU utilization, GPU utilization, device temperature, and device power consumption; and the display environment parameters include ambient light intensity and user viewing distance.

[0022] In this embodiment, step two specifically includes: The video frame at the current time step is aligned to the set image size, and all pixel values ​​of the video frame are mapped to the [0,1] interval using the max-min normalization method to obtain the standard video frame tensor; the standard video frame tensor contains three channels, namely the R channel, the G channel, and the B channel; The resolution parameters, network status parameters, device computing power parameters, and display environment parameters of the current time step are normalized using the min-max normalization method and then arranged in order to form a standard environment feature vector.

[0023] In this embodiment, the color channel enhancement module performs color modeling and channel recalibration on the standard video frame tensor to generate a color enhancement feature tensor, specifically as follows: Based on the R, G, and B channels of the standard video frame tensor, the color difference between the R and G channels, the color difference between the G and B channels, and the color difference between the R and B channels are calculated respectively, and organized into color difference tensors according to the pixel positions. The color difference matrix and the standard video frame tensor are concatenated along the channel dimension to obtain the color video frame tensor. Perform a 1×1 convolution operation on the color video frame tensor to extract cross-channel fusion features, generate channel response video frame tensors, and perform global average pooling on each channel of the channel response video frame tensor to obtain channel statistical vectors. The channel statistical vector is input into two fully connected layers to perform feature dimension compression and restoration operations, generating a channel recalibration weight vector. The color enhancement feature tensor is obtained by performing element-wise multiplication of the channel recalibration weight vector and the channel response video frame tensor along the channel dimension.

[0024] In this invention, a color channel enhancement module performs color modeling and channel recalibration operations on the tensor of standard video frames, fully exploring the color differences and correlations among the R, G, and B channels. This effectively enhances the improved MobileViT network's ability to recognize color details, edge variations, and tone shifts in video frames. The introduction of channel response extraction and channel recalibration mechanisms achieves significant enhancement of important color channels and suppresses interference from redundant channels, making the extracted visual content features more distinctive and discriminative. This provides more accurate color information support for resolution adaptive prediction, enhancing the generalization ability and robustness of this invention across various video scenarios.

[0025] In this embodiment, the luminance differential coding module performs luminance differential modeling on the standard video frame tensor to generate a luminance enhancement feature tensor, specifically as follows: Obtain the R-channel, G-channel, and B-channel values ​​of each pixel in the standard video frame tensor, calculate the luminance channel value of each pixel using the luminance formula, and assemble all luminance channel values ​​into a luminance matrix according to the pixel position order. Calculate the difference between the two luminance matrices at the current time step and the previous time step to obtain the luminance difference matrix; The luminance difference matrix and the standard video frame tensor are concatenated along the channel dimension to generate the luminance fused video frame tensor. A 3×3 convolution and a 1×1 convolution are performed on the luminance fusion video frame tensor, respectively. The 3×3 convolution is used to extract local luminance change features, and the 1×1 convolution is used to perform channel mapping to generate a luminance enhancement feature tensor.

[0026] In this invention, the brightness variation model of the standard video frame tensor is modeled by the brightness difference coding module, which can effectively extract the dynamic brightness information during video playback, improve the model's ability to perceive changes in illumination, scene switching and image complexity, thereby enhancing the temporal sensitivity and local response capability of visual content representation. This helps to more accurately identify the trend of image quality changes when predicting resolution adaptively, and achieve a more stable and clearer video playback effect.

[0027] In this embodiment, the structural complexity analysis module includes a texture density extraction branch, an edge response extraction branch, and a frequency response extraction branch. These branches perform local texture gradient statistics, edge intensity operator convolution, and spectral energy modeling on the brightness matrix of the standard video frame tensor, respectively, and then concatenate them along the channel dimension to generate a structural fusion feature tensor. Specifically: Input the brightness matrix of the current time step into the texture density extraction branch, set a window area with each pixel as the center, calculate the average brightness within the window area, and calculate the square of the difference between the brightness value of each pixel and the average brightness to obtain the squared difference of each pixel within the window area. The local brightness variance value of the current pixel is obtained by summing the squared differences of all pixels in the window area and then dividing by the total number of pixels in the window area. The local brightness variance values ​​of all pixels are then organized into a texture density feature matrix according to the pixel position. The brightness matrix of the current time step is input into the edge response extraction branch, and the horizontal gradient matrix and vertical gradient matrix are generated by the horizontal Sobel convolution kernel and the vertical Sobel convolution kernel, respectively. The edge intensity matrix is ​​obtained by squaring, summing, and taking the square root of the values ​​at corresponding positions in the horizontal and vertical gradient matrices. The brightness matrix at the current time step is input into the frequency response extraction branch, and a frequency domain feature matrix is ​​generated by two-dimensional fast Fourier transform; the amplitude of the frequency domain feature matrix is ​​calculated to obtain the spectral energy matrix. The texture density feature matrix, edge response matrix, and spectral energy matrix are concatenated along the channel dimension to obtain the structural fusion feature tensor.

[0028] In this invention, a structural complexity analysis module is used to model the luminance matrix in the tensor of a standard video frame in multiple dimensions, fusing three types of structural information: texture density, edge response, and spectral energy. This enables a comprehensive characterization of the detail levels, contour changes, and frequency components in the video frame. Texture complexity is measured by local luminance variance, the Sobel operator extracts significant edge contours, and Fourier transform captures frequency distribution features. This allows the structural fusion feature tensor to accurately reflect the richness of detail and texture complexity of different video frames. This module effectively improves the ability of the improved MobileViT network to identify structurally sensitive regions in complex scenes, enhancing the accuracy and intelligence of video quality control.

[0029] In this embodiment, the spatiotemporal MobileViT Block feature encoding module concatenates the color enhancement feature tensor, the brightness enhancement feature tensor, and the structural fusion feature tensor along the channel dimension to form a fused feature tensor. Then, through a time-shifting mechanism, multi-head self-attention operation, and global average pooling, spatiotemporal feature encoding is performed on the fused feature tensor to obtain a visual content representation vector, specifically: The fused feature tensor is divided into several non-overlapping image blocks according to a set size, and all the non-overlapping image blocks are expanded to obtain a two-dimensional token sequence; A time offset mechanism is introduced to time offset the two-dimensional token sequence to obtain a time-offset token sequence. The time offset mechanism is as follows: the channel dimension features of the two-dimensional token sequence are divided into three parts. The features of the first third of the channel dimension are shifted forward by one time step, the middle third remains unchanged, and the features of the last third of the channel dimension are shifted backward by one time step. Perform multi-head self-attention operation on the temporally offset token sequence to obtain the spatiotemporal attention feature sequence; The spatiotemporal attention feature sequence is structurally reconstructed, and features are fused and dimension compressed through 3×3 convolution and 1×1 convolution to generate a spatiotemporal encoded feature tensor; the spatiotemporal encoded feature tensor has the same structure as the fused feature tensor. Global average pooling is performed on the spatiotemporal coding feature tensor in both the time and spatial dimensions to obtain the visual content representation vector.

[0030] In this invention, the introduction of a time-off mechanism enhances the ability of the spatiotemporal MobileViT Block feature encoding module to model dynamic information between video frames, improving the perception accuracy and response robustness of the improved MobileViT network to temporal trends. By converting the fused feature tensor into a two-dimensional token sequence, dividing it into three sub-regions along the channel dimension, and performing forward offset, stationary offset, and backward offset operations respectively, guided fusion of contextual information across time steps is achieved. Compared to single-frame static modeling, this method can effectively capture fine-grained differences and continuous features between adjacent time steps, effectively improving the improved MobileViT network's ability to discriminate slow motion, occlusion switching, and scene abrupt changes. Segmented offsetting of the channel dimension maintains temporal consistency within local regions, avoiding feature distortion caused by overall offsetting, and improving the stability and representational capability of the encoding structure.

[0031] In this embodiment, step five specifically includes: The video state feature vector is input into four prediction sub-modules to obtain the initial predicted resolution level, resolution adjustment range, resolution confidence, and upgrade trend parameters. These four prediction sub-modules are independently trained sub-network structures, specifically: The video state feature vector is passed through a fully connected layer and the Softmax function to generate a resolution level probability vector, and the index corresponding to the maximum probability value in the resolution level probability vector is used as the initial predicted resolution level. The correspondence between the predicted resolution level and the actual resolution is as follows: Predicted resolution level 0 corresponds to a resolution of 426×240; predicted resolution level 1 corresponds to a resolution of 640×360; predicted resolution level 2 corresponds to a resolution of 854×480; predicted resolution level 3 corresponds to a resolution of 1280×720; predicted resolution level 4 corresponds to a resolution of 1920×1080; predicted resolution level 5 corresponds to a resolution of 2560×1440; and predicted resolution level 6 corresponds to a resolution of 3840×2160. A fully connected network and the ReLU activation function are used to transform the video state feature vector into non-negative values, which are then used as the resolution adjustment range. The video state feature vector is passed through a fully connected network and a sigmoid activation function to generate a resolution confidence score, which reflects the reliability of the current prediction result. A fully connected network and a Tanh activation function are used to transform the video state feature vector into an upscaling trend parameter, which reflects the preference for higher resolution in the current video state.

[0032] In this embodiment, step six specifically includes: The resolution upgrade strategy is as follows: Set the upscaling trend threshold and the maximum resolution level; If the upgrade trend parameter is greater than the upgrade trend threshold, and the initial predicted resolution level is lower than the maximum resolution level, then, without exceeding the maximum resolution level, the resolution adjustment range is multiplied by the set total number of resolution levels and rounded up to obtain the upgrade level number. The initial predicted resolution level is then added to the upgrade level number to obtain the updated resolution level. : ; in, Indicates the initial prediction resolution level. Indicates the resolution adjustment range. This indicates the total number of resolution levels set. The symbol represents the rounding up sign. Indicates the maximum resolution level. Describes the minimum value function; The resolution degradation strategy is specifically as follows: Set the downshift trend threshold and minimum resolution level; If the upscaling trend parameter is less than the downscaling trend threshold, and the initial predicted resolution level is higher than the minimum resolution level, then, provided it is not lower than the minimum resolution level, the downscaling level number is calculated based on the resolution confidence and resolution adjustment magnitude. The updated resolution level is then obtained by subtracting the downscaling level number from the initial predicted resolution level. : ; in, Indicates the minimum resolution level. Indicates resolution confidence. Represents the maximum value function; Set the historical sliding window range, obtain the historical resolution level sequence within the historical sliding window range, and calculate the historical average resolution level; A resolution change threshold is set. If the difference between the updated resolution level and the historical average resolution level is greater than the resolution change threshold, a smooth transition strategy is executed, specifically: The resolution confidence level is multiplied by the updated resolution level in a weighted manner. The resolution confidence level is subtracted from 1 and multiplied by the historical average resolution level in a weighted manner. The two are then added together to obtain the smoothed resolution level. Based on resolution upgrade strategy, resolution downgrade strategy and smooth transition strategy, the updated resolution level or smooth resolution level is determined as the target resolution level, and the target resolution level is set as the resolution parameter of the video rendering module.

[0033] In this invention, by adaptively adjusting the resolution level during the video rendering process, a dynamic balance management between user visual needs and system resource status can be achieved. Specifically, the resolution upgrade strategy ensures higher quality images are provided when visual attention increases or scene complexity increases, enhancing the visual experience; the resolution downgrade strategy ensures timely resolution reduction in situations of decreased attention or resource constraints, saving computing power and transmission overhead; and the smooth transition strategy dynamically weights resolution changes using resolution confidence, avoiding image instability and visual abruptness caused by frequent jumps, thus improving the stability, continuity, and user-perceived quality of the content rendering process.

[0034] Example 1 To verify the feasibility of this invention in practice, it was applied to a video content adaptive resolution adjustment module of a smart TV terminal. During implementation, users watch high-definition video content, such as movies, variety shows, or sports events, on their home TVs. However, due to limited device computing resources, frequent fluctuations in network bandwidth, and unstable video source quality, traditional fixed-resolution playback mechanisms often fail to balance visual experience and system performance. For example, when playing fast-moving scenes or complex scenes such as fireworks or rainy nights, a fixed high resolution can cause decoding stuttering and ghosting, while switching to a lower resolution significantly degrades image details and color reproduction, resulting in a decreased viewing experience.

[0035] The method of this invention can perform structural complexity analysis, spatiotemporal content modeling, and accuracy requirement assessment on each frame of video image in real time, thereby driving the video decoding module to perform adaptive resolution switching operations. Through the improved MobileViT network constructed in this invention, the currently playing video frame image stream is processed frame by frame, extracting image content feature vectors and inter-frame dynamic features. Combined with the structural complexity embedding vector generated by the structural complexity analysis module, the required resolution level of the current frame is estimated, and the decoding frame rate and target pixel density of the video playback engine are adjusted in real time.

[0036] In the experimental verification, a mainstream domestic smart TV model was selected, and multiple video content sources were collected, including action movie clips, static interviews, night aerial videos, and variety show stage clips, covering different lighting conditions, dynamic levels, and structural complexities. The method of this invention was compared with three mainstream video resolution control methods: Scheme A: fixed resolution playback; Scheme B: a simple dynamic adjustment method based on bitrate changes; and Scheme C: a complexity-aware method based on CNN modeling. Each method was run on the same device platform with the same bandwidth resource limits and playback duration. Comparison metrics included average playback frame rate, frame rate fluctuation amplitude, number of playback stutters, average user subjective rating (out of 5), average picture clarity index, and average playback power consumption. The experimental results are shown in Table 1. The average picture clarity index was measured using average PSNR, and the average playback power consumption was measured using power consumption per unit time.

[0037] Table 1. Comparison Results of the Invention Method and the Comparative Scheme in Video Adaptive Resolution Adjustment

[0038] As shown in Table 1, the method of this invention outperforms the comparative schemes in multiple comparative indicators. Regarding playback smoothness, the average playback frame rate of the method of this invention is 29.7 fps, significantly higher than the 23.1 fps of comparative scheme A, 27.4 fps of comparative scheme B, and 28.3 fps of comparative scheme C. Furthermore, the frame rate fluctuation is controlled within ±4.2%, far lower than the maximum fluctuation range of ±16.8% in the comparative schemes. This indicates that the method of this invention has stronger frame rate stability and timing consistency during video resolution adjustment, effectively avoiding playback stuttering or latency issues caused by resolution switching. In terms of the number of stutters, the method of this invention experiences only 0.3 stutters, significantly better than the 5.6 stutters of comparative scheme A, 2.8 stutters of comparative scheme B, and 1.2 stutters of comparative scheme C, further demonstrating its robustness under complex network conditions.

[0039] Secondly, regarding user experience perception, the subjective score of this invention reached 4.72 points, significantly higher than the scores of Comparison Solution A (3.21 points), Comparison Solution B (3.98 points), and Comparison Solution C (4.41 points), indicating that users generally believe the video quality provided by this invention is superior, and the viewing experience is more natural and smooth. Simultaneously, the average image clarity index output by the method of this invention reached 42.3dB, a significant improvement compared to existing solutions, indicating that it can better preserve image details and edge texture information during resolution adjustment, ensuring consistent visual quality.

[0040] In addition, in terms of system resource consumption, the method of the present invention maintains high image quality and smoothness while consuming an average of only 18.5W / h of playback power, which is lower than the comparative scheme. This effectively reduces the power consumption of mobile terminals or embedded devices and is suitable for low-power scenarios with high energy efficiency requirements.

[0041] The method of this invention effectively controls energy consumption during playback while improving video quality and stability, demonstrating excellent overall performance and showing good practical application prospects and promotional value.

[0042] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A dynamic video resolution adaptive method based on artificial intelligence, characterized in that, Includes the following steps: Step 1: Collect playback status data during video playback. The playback status data includes video frames, resolution parameters, network status parameters, device computing power parameters, and display environment parameters. Step 2: Align and normalize the video frames and calculate the brightness difference to obtain the standard video frame tensor; normalize the resolution parameters, network status parameters, device computing power parameters and display environment parameters to obtain the standard environment feature vector; Step 3: Input the standard video frame tensor into the improved MobileViT network to generate a visual content representation vector; the improved MobileViT network includes a color channel enhancement module, a luminance differential coding module, a structural complexity analysis module, and a spatiotemporal MobileViT Block feature coding module; Step 4: Concatenate the visual content representation vector with the standard environment feature vector along the feature dimension, and fuse them through a two-layer MLP structure to generate a video state feature vector; Step 5: Perform adaptive resolution prediction based on video state feature vectors to obtain the initial predicted resolution level, resolution adjustment range, resolution confidence, and upgrade trend parameters; Step 6: Based on the initial predicted resolution level, resolution adjustment range, resolution confidence and upgrade trend parameters, execute the resolution upgrade strategy, resolution downgrade strategy and smooth transition strategy to obtain the target resolution level, and update the resolution parameters of the video rendering module based on the target resolution level; Step 7: Output video frames rendered at the target resolution level and update the historical resolution level sequence.

2. The method for adaptive dynamic video resolution based on artificial intelligence according to claim 1, characterized in that, The network status parameters include network bandwidth, packet loss rate, and latency; the device computing power parameters include CPU utilization, GPU utilization, device temperature, and device power consumption; and the display environment parameters include ambient light intensity and user viewing distance.

3. The dynamic video resolution adaptive method based on artificial intelligence according to claim 1, characterized in that, Step two specifically includes: The video frame at the current time step is aligned to the set image size, and all pixel values ​​of the video frame are mapped to the [0,1] interval using the max-min normalization method to obtain the standard video frame tensor; the standard video frame tensor contains three channels, namely the R channel, the G channel, and the B channel; The resolution parameters, network status parameters, device computing power parameters, and display environment parameters of the current time step are normalized using the min-max normalization method and then arranged in order to form a standard environment feature vector.

4. The method for adaptive dynamic video resolution based on artificial intelligence according to claim 1, characterized in that, The color channel enhancement module performs color modeling and channel recalibration on the standard video frame tensor to generate a color-enhanced feature tensor, specifically: Based on the R, G, and B channels of the standard video frame tensor, the color difference between the R and G channels, the color difference between the G and B channels, and the color difference between the R and B channels are calculated respectively, and organized into color difference tensors according to the pixel positions. The color difference matrix and the standard video frame tensor are concatenated along the channel dimension to obtain the color video frame tensor. Perform a 1×1 convolution operation on the color video frame tensor to extract cross-channel fusion features, generate channel response video frame tensors, and perform global average pooling on each channel of the channel response video frame tensor to obtain channel statistical vectors. The channel statistical vector is input into two fully connected layers to perform feature dimension compression and restoration operations, generating a channel recalibration weight vector. The color enhancement feature tensor is obtained by performing element-wise multiplication of the channel recalibration weight vector and the channel response video frame tensor along the channel dimension.

5. The dynamic video resolution adaptive method based on artificial intelligence according to claim 1, characterized in that, The luminance differential encoding module performs luminance differential modeling on the standard video frame tensor to generate a luminance enhancement feature tensor, specifically: Obtain the R-channel, G-channel, and B-channel values ​​of each pixel in the standard video frame tensor, calculate the luminance channel value of each pixel using the luminance formula, and assemble all luminance channel values ​​into a luminance matrix according to the pixel position order. Calculate the difference between the two luminance matrices at the current time step and the previous time step to obtain the luminance difference matrix; The luminance difference matrix and the standard video frame tensor are concatenated along the channel dimension to generate the luminance fused video frame tensor. A 3×3 convolution and a 1×1 convolution are performed on the luminance fusion video frame tensor, respectively. The 3×3 convolution is used to extract local luminance change features, and the 1×1 convolution is used to perform channel mapping to generate a luminance enhancement feature tensor.

6. The dynamic video resolution adaptive method based on artificial intelligence according to claim 1, characterized in that, The structural complexity analysis module includes texture density extraction, edge response extraction, and frequency response extraction branches. These branches perform local texture gradient statistics, edge intensity operator convolution, and spectral energy modeling on the brightness matrix of the standard video frame tensor, respectively, and then concatenate them along the channel dimension to generate a structural fusion feature tensor. Specifically: Input the brightness matrix of the current time step into the texture density extraction branch, set a window area with each pixel as the center, calculate the average brightness within the window area, and calculate the square of the difference between the brightness value of each pixel and the average brightness to obtain the squared difference of each pixel within the window area. The local brightness variance value of the current pixel is obtained by summing the squared differences of all pixels in the window area and then dividing by the total number of pixels in the window area. The local brightness variance values ​​of all pixels are then organized into a texture density feature matrix according to the pixel position. The brightness matrix of the current time step is input into the edge response extraction branch, and the horizontal gradient matrix and vertical gradient matrix are generated by the horizontal Sobel convolution kernel and the vertical Sobel convolution kernel, respectively. The edge intensity matrix is ​​obtained by squaring, summing, and taking the square root of the values ​​at corresponding positions in the horizontal and vertical gradient matrices. The brightness matrix of the current time step is input into the frequency response extraction branch, and a frequency domain feature matrix is ​​generated by two-dimensional fast Fourier transform. Calculate the magnitude of the frequency domain characteristic matrix to obtain the spectral energy matrix; The texture density feature matrix, edge response matrix, and spectral energy matrix are concatenated along the channel dimension to obtain the structural fusion feature tensor.

7. The method for adaptive dynamic video resolution based on artificial intelligence according to claim 1, characterized in that, The spatiotemporal MobileViT Block feature encoding module concatenates the color enhancement feature tensor, brightness enhancement feature tensor, and structural fusion feature tensor along the channel dimension to form a fused feature tensor. Then, through a time-shifting mechanism, multi-head self-attention operation, and global average pooling, spatiotemporal feature encoding is performed on the fused feature tensor to obtain a visual content representation vector, specifically: The fused feature tensor is divided into several non-overlapping image blocks according to a set size, and all the non-overlapping image blocks are expanded to obtain a two-dimensional token sequence; A time offset mechanism is introduced to time offset the two-dimensional token sequence to obtain a time-offset token sequence. The time offset mechanism is as follows: the channel dimension features of the two-dimensional token sequence are divided into three parts. The features of the first third of the channel dimension are shifted forward by one time step, the middle third remains unchanged, and the features of the last third of the channel dimension are shifted backward by one time step. Perform multi-head self-attention operation on the temporally offset token sequence to obtain the spatiotemporal attention feature sequence; The spatiotemporal attention feature sequence is structurally reconstructed, and features are fused and dimension compressed through 3×3 convolution and 1×1 convolution to generate a spatiotemporal encoded feature tensor; the spatiotemporal encoded feature tensor has the same structure as the fused feature tensor. Global average pooling is performed on the spatiotemporal coding feature tensor in both the time and spatial dimensions to obtain the visual content representation vector.

8. The method for adaptive dynamic video resolution based on artificial intelligence according to claim 1, characterized in that, Step five specifically includes: The video state feature vector is input into four prediction sub-modules to obtain the initial predicted resolution level, resolution adjustment range, resolution confidence, and upgrade trend parameters. These four prediction sub-modules are independently trained sub-network structures, specifically: The video state feature vector is passed through a fully connected layer and the Softmax function to generate a resolution level probability vector, and the index corresponding to the maximum probability value in the resolution level probability vector is used as the initial predicted resolution level. A fully connected network and the ReLU activation function are used to transform the video state feature vector into non-negative values, which are then used as the resolution adjustment range. The video state feature vector is passed through a fully connected network and a sigmoid activation function to generate a resolution confidence score, which reflects the reliability of the current prediction result. A fully connected network and a Tanh activation function are used to transform the video state feature vector into an upscaling trend parameter, which reflects the preference for higher resolution in the current video state.

9. The dynamic video resolution adaptive method based on artificial intelligence according to claim 1, characterized in that, Step six specifically includes: The resolution upgrade strategy is as follows: Set the upscaling trend threshold and the maximum resolution level; If the upgrade trend parameter is greater than the upgrade trend threshold, and the initial predicted resolution level is lower than the maximum resolution level, then, without exceeding the maximum resolution level, the resolution adjustment range is multiplied by the set total number of resolution levels and rounded up to obtain the upgrade level number. The initial predicted resolution level is then added to the upgrade level number to obtain the updated resolution level. : ； in, Indicates the initial prediction resolution level. Indicates the resolution adjustment range. This indicates the total number of resolution levels set. The symbol represents the rounding up sign. Indicates the maximum resolution level. Describes the minimum value function; The resolution degradation strategy is specifically as follows: Set the downshift trend threshold and minimum resolution level; If the upscaling trend parameter is less than the downscaling trend threshold, and the initial predicted resolution level is higher than the minimum resolution level, then, provided it is not lower than the minimum resolution level, the downscaling level number is calculated based on the resolution confidence and resolution adjustment magnitude. The updated resolution level is then obtained by subtracting the downscaling level number from the initial predicted resolution level. : ； in, Indicates the minimum resolution level. Indicates resolution confidence. Represents the maximum value function; Set the historical sliding window range, obtain the historical resolution level sequence within the historical sliding window range, and calculate the historical average resolution level; A resolution change threshold is set. If the difference between the updated resolution level and the historical average resolution level is greater than the resolution change threshold, a smooth transition strategy is executed, specifically: The resolution confidence level is multiplied by the updated resolution level in a weighted manner. The resolution confidence level is subtracted from 1 and multiplied by the historical average resolution level in a weighted manner. The two are then added together to obtain the smoothed resolution level. Based on resolution upgrade strategy, resolution downgrade strategy and smooth transition strategy, the updated resolution level or smooth resolution level is determined as the target resolution level, and the target resolution level is set as the resolution parameter of the video rendering module.

Images