AI Computing Power Adaptive Allocation Method and System for Video Super-Resolution Reconstruction

By generating regional degradation description data and AI computing power levels, image blocks are cropped for reconstruction and local high-precision compensation is performed. This solves the problem of inaccurate computing power allocation in video super-resolution reconstruction, improves reconstruction quality and efficiency, and is suitable for real-time playback and edge devices.

CN123123571APending Publication Date: 2026-11-17LINBAIXIN SCI & TECH SECONDARY SPECIALIZED SCHOOL SHANTOU CITY +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611245893.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-17
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing video super-resolution reconstruction technologies lack a continuous basis for allocating computing power in accordance with the sources of video coding degradation, regional reconstruction risks, and partition reconstruction boundaries. This results in limited AI computing power being evenly distributed across the entire frame or areas with low returns, failing to effectively solve the problem of local reconstruction failures.

Method used

By acquiring the block-level quantization intensity, block-level motion vector, and block-level residual intensity of the video stream to be processed, regional degradation description data is generated. Based on this data, an AI computing power level is generated, and image blocks are reconstructed by cropping the outer boundary. Combined with local high-precision compensation processing after the initial reconstruction, abnormal regions are accurately identified and corrected.

Benefits of technology

It achieves accurate matching of computing power to the source of encoding degradation, reduces invalid consumption, improves the boundary texture and detail mutation of partition reconstruction, reduces inference latency and computing resource consumption, and is suitable for real-time playback and edge device deployment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123123571A_ABST
    Figure CN123123571A_ABST
Patent Text Reader

Abstract

This invention proposes an AI computing power adaptive allocation method and system for video super-resolution reconstruction. The method includes: acquiring block-level quantization intensity, block-level motion vector, and block-level residual intensity during the video bitstream decoding process, and organizing them into region degradation description data; generating an AI computing power level for the processing region based on this data; determining the outer boundary width based on the AI ​​computing power level to crop temporal image blocks, calling the corresponding preset AI super-resolution reconstruction path for reconstruction, and synthesizing the initial AI reconstructed video data; calculating the internal structural continuity and adjacent frame structural stability of the region based on the initial AI reconstructed video data, determining the local reprocessing region, performing local high-precision AI compensation, and synthesizing the target super-resolution video data. This invention combines encoded degradation information with computing power allocation and introduces a local verification compensation mechanism, effectively improving the reconstruction quality and temporal stability of video super-resolution while reducing computing resource consumption.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of video image processing and artificial intelligence technology, and in particular relates to an AI computing power adaptive allocation method and system for video super-resolution reconstruction. Background Technology

[0002] Video super-resolution reconstruction technology is mainly used to restore low-resolution videos to high-resolution videos, and it has high application value in scenarios such as digital media playback, cloud video enhancement, video conferencing, security monitoring, and mobile image enhancement. Existing engineering systems typically decode the input video frame by frame first, and then input the decoded image sequence into an AI super-resolution model or image enhancement module for unified processing. Some solutions combine inter-frame motion estimation, optical flow compensation, or reuse of historical frame features to improve temporal consistency. Other solutions select super-resolution models of different sizes based on device computing power, frame rate requirements, or image complexity to reduce inference overhead. Although the above solutions can improve video clarity to a certain extent, there is still a relatively concentrated engineering problem in actual deployment: the allocation of AI computing power required for video super-resolution lacks a continuous basis that matches the sources of video coding degradation, regional reconstruction risks, and partition reconstruction boundaries. This leads to limited AI computing power being easily distributed evenly across entire frames or areas with low returns, while areas that are truly prone to reconstruction failure do not receive sufficient processing in the initial reconstruction stage, nor do they receive targeted local correction after the initial reconstruction. Specifically, low-resolution videos exhibit significant regional degradation differences during encoding, transmission, and decoding. For instance, regions with high block-level quantization intensity are more prone to detail loss and block artifacts, while regions with low motion vector stability are more likely to correspond to motion boundaries, occlusion changes, or mixed local motion locations. Residual intensity concentrated near edges, text strokes, fine lines, or complex textures often indicates underprediction and high difficulty in local structure recovery. However, most existing super-resolution systems only re-evaluate reconstruction difficulty based on image texture, edges, or single-frame complexity after decoding, failing to fully utilize the quantization, motion, and residual information already formed in the bitstream or decoding stage, resulting in inaccurate AI computing power allocation before reconstruction. On the other hand, existing adaptive super-resolution schemes typically only select the model level or processing intensity before reconstruction. Once local anomalies such as text edge distortion, broken fine lines, motion boundary blurring, sudden changes in sharpness during regional stitching, or texture flickering in adjacent frames occur after the initial AI reconstruction, the system often has to directly output the initial reconstruction result or re-execute high-computation AI super-resolution processing on the entire frame. The former affects image quality stability, while the latter increases inference latency and computational resource consumption. For scenarios such as real-time playback, multi-channel video processing, and edge device deployment, simply increasing the model size or improving the reconstruction level of an entire frame does not conform to actual engineering constraints. Summary of the Invention

[0003] This invention discloses an AI computing power adaptive allocation method and system for video super-resolution reconstruction, in order to solve the problems mentioned in the background art.

[0004] To achieve the above objectives, a first aspect of the present invention provides an AI computing power adaptive allocation method for video super-resolution reconstruction, the method comprising:

[0005] Obtain the block-level quantization intensity, block-level motion vector, and block-level residual intensity of the video stream to be processed during the decoding process. Divide the video frames into processing regions according to the preset region size. Organize the block-level quantization intensity, the block-level motion vector, and the block-level residual intensity into region degradation description data.

[0006] The AI ​​computing power level of the processing area is generated based on the regional degradation description data.

[0007] The outer boundary width of the image patch is determined according to the AI ​​computing power level, and the temporal image patch of the processing area is cropped according to the outer boundary width; the temporal image patch is reconstructed by calling the corresponding preset AI super-resolution reconstruction path according to the AI ​​computing power level to obtain the region super-resolution result; the initial AI reconstructed video data is synthesized according to the region super-resolution result.

[0008] Based on the initial AI-reconstructed video data, the internal structural continuity of the processing area and the structural stability of adjacent frames are calculated; based on the internal structural continuity and the structural stability of adjacent frames, a local reprocessing area is determined; local high-precision AI compensation processing is performed on the local reprocessing area to obtain a local compensation result; the local compensation result is combined with the initial AI-reconstructed video data to obtain the target super-resolution video data.

[0009] Further, the step of organizing the block-level quantization intensity, the block-level motion vector, and the block-level residual intensity into region degradation description data includes:

[0010] Read the quantization parameters of the coded blocks within the processing area to obtain the region quantization description;

[0011] Read the motion vectors of the coded blocks within the processing area, determine the main motion vector, calculate the relative deviation of the motion vectors of the coded blocks relative to the main motion vector, and obtain the regional motion vector stability based on the relative deviation.

[0012] The distribution of residual coding blocks within the processing area is statistically analyzed to obtain a description of the regional residuals;

[0013] The region quantization description, the region motion vector stability, and the region residual description are combined into the region degradation description data.

[0014] Further, generating the AI ​​computing power level of the processing region based on the region degradation description data includes:

[0015] The region quantization description in the region degradation description data is converted into a compressed state, and the region residual description in the region degradation description data is converted into a residual state;

[0016] The basic computing power level of the processing region is determined based on the compression state, the residual state, and the regional motion vector stability in the regional degradation description data.

[0017] The AI ​​computing power level of the processing area is determined based on the base computing power level of the processing area and the base computing power level of adjacent processing areas.

[0018] Further, determining the AI ​​computing power level of the processing region based on the base computing power level of the processing region and the base computing power levels of adjacent processing regions includes:

[0019] Determine whether the basic computing power level of the processing area is a low computing power level;

[0020] If the base computing power level of the processing region is the low computing power level, and there is an adjacent processing region sharing an edge with the processing region, and the base computing power level of the adjacent processing region is the high computing power level, then the AI ​​computing power level of the processing region is set to the normal computing power level.

[0021] If the basic computing power level of the processing region is not the low computing power level, or there is no adjacent processing region sharing an edge with the processing region and having a high computing power level, then the basic computing power level of the processing region shall be used as the AI ​​computing power level of the processing region.

[0022] Further, determining the outer boundary width of image patch cropping based on the AI ​​computing power level includes:

[0023] Obtain the base expansion width and the expansion width of adjacent levels;

[0024] Obtain a neighbor protection flag, which is used to indicate whether there is an adjacent processing region that shares an edge with the processing region and has an AI computing power level higher than that of the processing region;

[0025] The outer expansion boundary width is calculated based on the basic outer expansion width, the adjacent level outer expansion width, the AI ​​computing power level, and the adjacent level protection flag.

[0026] Further, the step of reconstructing the temporal image block by calling the corresponding preset AI super-resolution reconstruction path according to the AI ​​computing power level includes:

[0027] If the AI ​​computing power level is low, the low-computation AI super-resolution reconstruction path is called to reconstruct the time-series image block;

[0028] If the AI ​​computing power level is a conventional computing power level, the conventional AI super-resolution reconstruction path is called to reconstruct the temporal image block;

[0029] If the AI ​​computing power level is high, the high-precision AI super-resolution reconstruction path is invoked to reconstruct the time-series image block;

[0030] The number of residual blocks in the low-computation AI super-resolution reconstruction path, the conventional AI super-resolution reconstruction path, and the high-precision AI super-resolution reconstruction path increases sequentially.

[0031] Further, the step of calculating the internal structural continuity of the processing region and the structural stability of adjacent frames based on the initial AI-reconstructed video data, and determining the local reprocessing region based on the internal structural continuity and the structural stability of adjacent frames, includes:

[0032] The region gradient description of the processing area is calculated using a fixed edge operator, and the region gradient description of the corresponding region in adjacent frames is also calculated.

[0033] Calculate the local temporal structure jump variable of the processing region based on the regional gradient description of the processing region and the regional gradient description of the corresponding region of the adjacent frame;

[0034] The internal structure of the processed area is reviewed to obtain the results of local structural anomaly detection.

[0035] If the local temporal structure jump variable reaches the preset structure jump judgment threshold, or if the local structure anomaly check result indicates the existence of a local structure anomaly, then the processing area is determined as the local reprocessing area.

[0036] Further, the step of performing local high-precision AI compensation processing on the locally reprocessed region to obtain a local compensation result includes:

[0037] The adjacent processing regions that are both identified as the local reprocessing regions are merged to obtain a continuous reprocessing block;

[0038] The continuous reprocessing block is input into the local high-precision AI compensation path to obtain the local compensation result;

[0039] The local high-precision AI compensation path includes a structural feature extraction part, an edge preservation part, and a temporal consistency part.

[0040] Further, the process of synthesizing the initial AI-reconstructed video data based on the region super-resolution results includes:

[0041] For the processing region within the same video frame, the central effective region in the region super-resolution result is taken as the final reconstruction result of the processing region;

[0042] For the overlapping regions of the processing regions adjacent to the processing region, the super-resolution results of the adjacent processing regions are weighted and transitioned according to the proximity of the pixels in the overlapping region to the central effective region, so as to obtain the initial AI reconstructed video data.

[0043] A second aspect of the invention provides an AI computing power adaptive allocation system for video super-resolution reconstruction, the system comprising:

[0044] The region degradation description generation module is used to obtain the block-level quantization intensity, block-level motion vector and block-level residual intensity of the video bitstream to be processed during the decoding process, divide the video frames into processing regions according to the preset region size, and organize the block-level quantization intensity, the block-level motion vector and the block-level residual intensity into region degradation description data.

[0045] The computing power level determination module is used to generate the AI ​​computing power level of the processing area based on the area degradation description data.

[0046] The partitioned AI super-resolution reconstruction module is used to determine the outer boundary width of the image block cropping according to the AI ​​computing power level, crop the temporal image block of the processing area according to the outer boundary width, call the corresponding preset AI super-resolution reconstruction path according to the AI ​​computing power level to reconstruct the temporal image block, and obtain the regional super-resolution result; and synthesize the initial AI reconstructed video data according to the regional super-resolution result.

[0047] The local compensation module is used to calculate the internal structural continuity of the processing area and the structural stability of adjacent frames based on the initial AI reconstructed video data; determine the local reprocessing area based on the internal structural continuity of the area and the structural stability of adjacent frames; perform local high-precision AI compensation processing on the local reprocessing area to obtain the local compensation result; and synthesize the local compensation result with the initial AI reconstructed video data to obtain the target super-resolution video data.

[0048] The beneficial technical effects of this invention are as follows: First, by fully utilizing block-level quantization, motion vectors, and residual information from the video decoding stage to construct a regional degradation description, the computing power allocation is precisely matched to the actual sources of encoding degradation and reconstruction risks, avoiding ineffective consumption of computing power in low-yield areas. Second, an adjacency protection mechanism is introduced in the computing power level determination, setting a transition level for low-computing-power areas surrounding high-computing-power areas, and dynamically adjusting the outward expansion boundary of image block cropping in conjunction with the computing power level, significantly improving the texture and detail abrupt changes at the partition reconstruction boundary and ensuring spatial stitching continuity. Finally, after the initial reconstruction, a local verification mechanism based on the continuity of the internal structure of the region and the structural stability of adjacent frames is added to accurately identify anomalies such as text adhesion, fine line breakage, and motion blur, performing local high-precision compensation only on abnormal areas, avoiding rework of the entire frame, improving the reliability of complex texture and motion boundary reconstruction while effectively reducing inference latency and computing resource consumption, meeting the needs of real-time playback and edge deployment. Attached Figure Description

[0049] The present invention will be further described with reference to the accompanying drawings, but the embodiments in the drawings do not constitute any limitation on the present invention. For those skilled in the art, other drawings can be obtained based on the following drawings without creative effort.

[0050] Figure 1 This is a flowchart of an AI computing power adaptive allocation method for video super-resolution reconstruction in an embodiment of the present invention.

[0051] Figure 2 This is a flowchart illustrating the generation process of regional degradation description data in an embodiment of the present invention.

[0052] Figure 3 This is a flowchart illustrating the AI ​​computing power level generation process in an embodiment of the present invention.

[0053] Figure 4 This is a flowchart of the partitioned AI super-resolution reconstruction process in an embodiment of the present invention.

[0054] Figure 5 This is a flowchart of partial verification and compensation in an embodiment of the present invention.

[0055] Figure 6 This is a framework diagram of an AI computing power adaptive allocation system for video super-resolution reconstruction in an embodiment of the present invention. Detailed Implementation

[0056] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0057] Figure 1 A flowchart illustrating an AI computing power adaptive allocation method for video super-resolution reconstruction according to some embodiments of this disclosure is shown. Figure 1 As shown, the method includes steps S101, S102, S103 and S104.

[0058] like Figure 2 As shown, in step S101, the block-level quantization intensity, block-level motion vector, and block-level residual intensity of the video stream to be processed during the decoding process are obtained. The video frames are divided into processing regions according to the preset region size. The block-level quantization intensity, the block-level motion vector, and the block-level residual intensity are organized into region degradation description data.

[0059] Specifically, the input for this step is the low-resolution video bitstream to be processed. . It can originate from local low-resolution video files, real-time streaming cache, video conferencing servers, surveillance recording platforms, edge computing nodes, or player decoding inputs; essentially, it is already encoded and compressed video stream data. The system receives... Subsequently, the software decoder, hardware decoding chip, or bitstream parsing module is invoked for decoding, and block-level coding information related to the difficulty of video super-resolution reconstruction is obtained synchronously during the decoding process. The obtained information includes block-level quantization intensity, block-level motion vector, and block-level residual intensity. Block-level quantization intensity is obtained from the bitstream syntax elements or the quantization parameters output by the decoder, and is used to indicate the degree to which the details of the coding block are weakened during compression; block-level motion vector is obtained from the inter-frame prediction decoding process, and is used to indicate the displacement relationship of the current coding block relative to the reference frame; block-level residual intensity is obtained from the dequantized residual signal, residual coefficient statistics, or the residual energy interface provided by the decoder, and is used to indicate image changes that are still not interpreted after motion prediction or intra-frame prediction. The above three types of information correspond to three types of reconstruction difficulties in video super-resolution engineering: texture loss caused by compression, boundary ghosting caused by motion inconsistency, and local detail anomalies caused by insufficient prediction. Therefore, their technical meanings are retained in this step as the basis data for subsequent regional-level AI computing power determination.

[0060] The system divides each frame into multiple processing regions according to preset region sizes. The size of this region should be consistent with the processing granularity of subsequent AI super-resolution reconstruction paths. For example, if the subsequent AI super-resolution path uses a fixed image patch as input, this step uses a region that is the same size as or slightly larger than that image patch as the processing region; if the subsequent AI super-resolution path supports whole-frame input and allows region-level computational power control, this step generates the processing region according to a regular grid. Each processing region... The system typically contains multiple coded blocks, and organizes the block-level information into a region-level degradation description. For quantization intensity, the system reads the quantization parameters of each coded block within the processing region and takes the median value representing the main compression state of the region, avoiding region judgment bias caused by a single abnormal coded block. For residual intensity, the system statistically analyzes the distribution of coded blocks with significant residuals within the processing region and retains the correspondence between the concentrated residual locations and the edges, text strokes, thin line structures, or complex textures within the region, ensuring that the region residual description reflects the location characteristics of local prediction deficiencies. For motion vectors, the system determines whether the motion direction and amplitude of multiple coded blocks within the same region are consistent, because areas prone to motion blur and boundary breaks in video super-resolution are usually motion-mixed regions, while stable motion regions with large simple displacements may not have the same reconstruction risk.

[0061] In order to obtain the processing area To ensure motion vector stability, the system obtains the motion vector of each coded block within a region from the decoder output and reads its horizontal and vertical components separately. Then, it determines the median of the horizontal and vertical components within the region and combines them to obtain the main motion vector for that region. Using the median reduces the impact of a few abnormal motion blocks on the main motion determination. For example, if most coded blocks in a region belong to a slowly moving background, and a few coded blocks fall on the edge of a rapidly swinging object, the median is closer to the main motion state of that region. Afterward, the system compares the deviation of each coded block's motion vector within the region from the main motion vector and organizes these deviations into a region motion deviation. . This represents the relative deviation result; a larger value indicates more inconsistent motion within that region. Processing area. The motion vector stability is obtained by the following formula:

[0062] ;

[0063] This formula originates from the monotonically decreasing reciprocal mapping in mathematics, and its original form is to represent the non-negative change quantity. Mapped to This is used to express the relationship that "the greater the deviation, the lower the stability". This application uses [the relevant term] to express this relationship. Replace with processing area Motion deviation The stability of the motion vector is obtained. When the motion vectors within the region are basically the same, Approaching low value, Approaching a highly stable state; when the differences in motion vectors within the region are significant. Increase Consequently, it decreases. Both sides of this equation are proportional values. Before substitution, the relative deviation of the motion vector has been transformed into a non-negative proportional deviation, which can be directly added to the constant term. Among these, Indicates the processing area The motion vector stability is calculated in this step based on the block-level motion vectors output by the decoder; This represents a processing region in the current frame, divided according to a preset region size. Its position is determined by the frame number and region coordinates. Indicates the processing area The relative deviation of the motion vectors of each coded block within the region from the main motion vectors of that region is obtained by organizing the block-level motion vectors output by the decoder.

[0064] In a typical set of regional processing procedures, a certain processing region It contains 16 coded blocks. The motion vectors output by the decoder show that 13 of the coded blocks have similar motion directions and amplitudes, while the other 3 are located at the edge of the object, with motion directions significantly different from the main body's motion. The system first determines the main motion vector for this region based on the motion vectors of the 16 coded blocks, and then calculates the deviation of each coded block relative to this main motion vector. If after processing... Substituting into the above equation, we get This indicates that the overall motion in this region is relatively stable, with only slight motion mixing; if another processing region is located near the outline of a fast-moving object, the motion direction of the coded blocks within the region is dispersed, and after processing, we get... ,but This indicates that the region has lower motion stability and is more suitable for inclusion in the candidate range of higher AI computing power levels. This calculation result corresponds to the actual reconstruction risk in video super-resolution, that is, the more inconsistent the motion vectors of a region, the more likely it is to form motion boundary ghosting, edge breakage, or texture abruptness in subsequent reconstruction.

[0065] After obtaining the regional motion vector stability The system then organizes each processed region into a single region degradation description record. This record includes the region location identifier, region quantization description, and region motion vector stability. And regional residual description. The regional location identifier is formed by the current frame number and regional coordinates, which is used to ensure that subsequent steps can accurately map the AI ​​computing power level to the same video region; the regional quantization description comes from the sorted result of the quantization intensity of the coded blocks in the region, which is used to indicate the risk of compression detail loss; regional motion vector stability. The motion consistency calculations described above are used to represent the risk of motion instability. The regional residual descriptions, derived from the organized block-level residual information within the region, are used to represent the risk of complex textures, fine-line structures, or under-prediction. For subtitle regions in online course videos, regional quantization descriptions may show significant loss of compressed details, while regional residual descriptions may show residuals concentrated at the edges of text strokes, indicating regional motion vector stability. It may be relatively high; for the outline region of moving objects in sports videos, the region quantization description may not be the highest, but the region motion vector stability is... The values ​​may be lower, and the regional residual descriptions are concentrated near the contour. These different types of regional degradation descriptions will trigger different AI computing power level determination rules in the next step.

[0066] The output of this step is region degradation description data. . It is a data set organized by video frames and processing regions, with each processing region corresponding to a degradation description record, derived from the input video bitstream. The encoded information generated during the decoding process is organized. Includes region location identification, region quantization description, and region motion vector stability. And the description of the regional residuals. This output will serve as direct input for the next step, which will be based on... Generate initial AI computing power allocation results for each processing region.

[0067] like Figure 3 As shown, in step S102, the AI ​​computing power level of the processing area is generated based on the area degradation description data.

[0068] Specifically, this step receives the region degradation description data output from step S101. and will Converted to AI computing power levels of each processing area The initial allocation results of the AI ​​computing power. Here... Represents a single processing region The AI ​​computing power level is allocated based on the overall result of processing areas within the same video frame. Together they constitute. The system has already saved the region location identifier and region quantization description according to video frames and processing areas. Regional motion vector stability and regional residual description Therefore, this step directly processes the area. The system determines the basic computing power allocation unit. Next, the arrangement of processing regions within the current frame is restored based on the region location identifiers, and then each processing region is read. corresponding , and .in, The sorting results of the block-level quantization intensity in the processing area from step S101 are used to represent the state of compression detail loss in that area; The motion vector stability obtained from step S101 based on the block-level motion vector consistency is used to represent the degree of motion consistency in the region. The results of sorting out the block-level residual intensity and residual concentration location from step S101 are used to indicate the insufficient prediction or local complex detail state of the region. Since these three types of information correspond to different risk sources in video super-resolution reconstruction, this step adopts a hierarchical trigger-based judgment, so that compression damage, motion instability and residual concentration can enter the AI ​​computing power level judgment process respectively.

[0069] The system is reading Then, first and Transformed into a discrete state. For The system classifies video sequences or processing batches into three categories—light compression, medium compression, and heavy compression—based on the quantization intensity distribution. For example, in a low-bitrate video, if the quantization intensity of most coded blocks within a processing region is in the high-bit range of the current frame or segment, that region is considered heavily compressed; if it is near the main distribution, it is considered medium compression; and if it is in the low-bit range, it is considered light compression. The system classifies residuals into three categories—low residuals, dispersed residuals, and concentrated residuals—based on the distribution pattern of significantly coded residual blocks within a region. If the residuals are weak and do not form a continuous structure, they are classified as low residuals; if the residuals are distributed across multiple discontinuous locations, they are classified as dispersed residuals; and if the residuals are concentrated near edges, text strokes, thin lines, or motion contours, they are classified as concentrated residuals. The boundaries for light compression, medium compression, heavy compression, and the classification of low, dispersed, and concentrated residuals are determined by the same video sequence, the same processing batch, or the system configuration table, and are adjusted accordingly. The generation process records the corresponding classification results, enabling subsequent computing power level determination to directly read the discrete state. Continue to use it as a continuous stability result, and compare it with the preset motion stability judgment threshold. Compare. The system pre-configures the parameters based on video frame rate, encoding configuration, and AI super-resolution reconstruction path processing capabilities; its values ​​are related to... For stability ratios between zero and one, such as sports, live broadcasts, or surveillance videos that are more sensitive to motion boundaries, higher stability requirements can be set to allow areas of inconsistent motion to enter high computing power levels earlier; for videos with slower image changes, relatively lenient judgment conditions can be set.

[0070] The system first based on , and Generate processing area Basic computing power level The initial source of this calculation formula is piecewise functions in mathematics and threshold segmentation judgment rules in engineering control. Its original idea is to divide the input state into several condition intervals and map different condition intervals to a defined control level. This application extends this segmentation rule to the video super-resolution AI computing power allocation scenario, transforming the condition intervals into three types of encoding degradation triggering conditions: by... Indicates compressive damage, by Indicates unstable motion, caused by This indicates insufficient prediction or localized complex details. After this modification, the basic computing power level... Determine as follows:

[0071] ;

[0072] in, Indicates the processing area Basic computing power level, numerical value , , These correspond to high computing power level, regular computing power level, and low computing power level, respectively. This refers to the processing area obtained in step S101 according to the preset area size, whose position is determined by... The regional location identifiers are determined; Indicates the processing area The region quantization description is obtained by sorting the block-level quantization intensity in step S101, and is converted into a light compression, medium compression or heavy compression state in this step. Indicates the processing area The regional residual description is obtained by organizing the block-level residual intensity and residual concentration location in step S101, and is converted into low residual, dispersed residual or concentrated residual state in this step; Indicates the processing area The regional motion vector stability is calculated by step S101; This represents the preset threshold for determining motion stability. In this formula, and These are similar stability values, and their magnitudes can be directly compared. and Participating in conditional matching with discrete states; The output is a discrete level. Both sides of the formula represent the level determination result. The condition term is only used to select the level branch, and the calculation relationship is consistent with the segmentation determination rule.

[0073] The logical order of the above segmentation rules is as follows: first, determine whether there are regions requiring priority processing with high computing power; then, determine whether they require processing with regular computing power; finally, classify the remaining regions into the low computing power level. The first type of condition in the high computing power level is... For heavy compression and To concentrate residuals, this condition corresponds to the scenario in video super-resolution where "compression impairment and difficulty in local structural recovery coexist," such as subtitle strokes, screen text, grid lines, fine line structures, or complex texture edges; the second type of condition in high-computing-power levels is... This condition corresponds to regions with low motion vector stability, such as the outlines of moving objects, occlusion boundaries, or locations with mixed local motion. Standard computing power levels are used to handle regions with single heavy compression, medium compression, or dispersed residuals, ensuring stable initial AI super-resolution processing for regions with general compression damage and general texture restoration. Low computing power levels are used to process lightly compressed, motion-stable regions with weak residuals, allowing subsequent AI computing power to be concentrated on locations with a greater impact on super-resolution reconstruction quality.

[0074] After obtaining the basic computing power level Afterwards, the system continued to utilize The region location identifier in the process generates the processing area. The final AI computing power level This processing method originates from the neighborhood expansion concept in mathematical morphology and the hierarchical continuity constraint on discrete meshes: in a discrete region mesh, if low-level regions surrounding high-level regions directly maintain their low level, subsequent AI super-resolution paths are prone to abrupt changes in texture sharpness, brightness transitions, or detail density at the region boundaries. This application modifies this neighborhood expansion concept into a region adjacency protection approach for partitioned AI super-resolution, setting a minimum normal computing power level for low-level regions sharing edges with high-computing-power regions, thereby forming a transitional processing zone around the high-computing-power regions. Final AI computing power level Determine as follows:

[0075] ;in, Indicates the processing area The final AI computing power level is also the basis for selecting the AI ​​super-resolution reconstruction path in the subsequent step S103. This indicates the processing area obtained from the previous formula. Basic computing power level; Indicates the current processing area The positional relationship between adjacent processing regions located in the same frame and sharing an edge is determined by... The regional location identifiers are determined; Indicates adjacent processing areas The basic computing power level. Edge-sharing relationships are determined by the intra-frame coordinates of the processing regions. For example, two processing regions that are vertically or horizontally adjacent within the same frame and share a region boundary are considered to share an edge. The input and output of this formula are both discrete levels. A condition term is used to determine whether to upgrade a lower computing power level to a regular computing power level; when the condition is met, the output level is determined. When the conditions are not met, the basic computing power level will be used. Therefore, the calculation results remain within the range of high, normal, and low AI computing power levels.

[0076] The two formulas above have a clear sequential relationship: the first formula is based on... In , and Obtain the base computing power level for each processing region. The second form is... Using regional location identifiers as input, the final AI computing power level is further determined. The first equation addresses the mapping problem from degradation risk to computing power level, while the second equation addresses the regional level transition problem before partitioned AI super-resolution execution. Thus, the regional location identifier, regional quantization description, regional motion vector stability, and regional residual description obtained in step S101 are all practically used: the regional location identifier is used to establish shared edge relationships and bind the output level. Used for compression damage triggering. Used for motion instability triggering Used for residual settling and triggering of complex details.

[0077] During the processing of an online course video, a certain processing area near the subtitles at the bottom of the screen... Depend on The following description of regional degradation is given: For heavy compression, To concentrate residuals, If the system is preset Therefore, although the motion stability of this region is relatively high, due to... For heavy compression and To centralize the residuals, we substitute them into the first equation and obtain... This indicates that the edge area of ​​the subtitle has entered a high computing power level. The adjacent background area above the subtitle area... Depend on Given: For light compression, For low residuals, Substituting into the first equation, we get ;because and Shared side, and The basic computing power level is After substituting into the second equation The final AI computing power level becomes Therefore, in step S103, when processing the high-computing-power area of ​​the subtitle, the adjacent background area will use a conventional AI super-resolution path, which can form a transition processing range near the edge of the subtitle, reducing the sudden change in detail density caused by direct splicing of high and low computing-power paths.

[0078] In the processing of another sports video, the processing area near the outline of the moving object... Depend on Given: For medium compression, To disperse residuals, .like ,because Substituting into the first equation, we get This indicates that the region has entered a high computing power level due to its low motion vector stability. Grassland areas with shared borders It may have light compression, low residuals, and high motion stability. Substituting this into the first equation, we get... ;because High computing power region Sharing a common side, substituting into the second equation, we get... In this way, the area near the motion contour not only receives high-performance AI super-resolution processing, but also forms a transition zone with conventional computing power, which is beneficial for the subsequent step S103 to maintain the texture continuity near the motion boundary.

[0079] After the system completes the above judgment, it will process each area. The final AI computing power level The initial allocation of AI computing power is organized by video frame and region location. Each record includes a region location identifier and the corresponding region. ,in This is used to instruct step S103 to call the high-precision AI super-resolution reconstruction path. This is used to instruct step S103 to call the regular AI super-resolution reconstruction path. This is used to instruct step S103 to call the low-computational-cost AI super-resolution reconstruction path. Because each All are related to a processing area in step S101 One-to-one correspondence; step S103 allows direct reading of each region. The partitioned AI super-resolution reconstruction is performed according to the AI ​​computing power level of different processing regions within the same frame. Through the above processing, this step transforms the region degradation description from the video encoding process in step S101 into an executable initial allocation result of AI computing power, so that compression impairment, motion instability, local residual concentration, and region stitching transition are all clearly arranged before the initial AI super-resolution reconstruction.

[0080] like Figure 4 As shown, in step S103, the outer boundary width of the image block cropping is determined according to the AI ​​computing power level, and the temporal image block of the processing area is cropped according to the outer boundary width; the temporal image block is reconstructed by calling the corresponding preset AI super-resolution reconstruction path according to the AI ​​computing power level to obtain the regional super-resolution result; the initial AI reconstructed video data is synthesized according to the regional super-resolution result.

[0081] Specifically, this step receives the initial AI computing power allocation result output from step S102, and then processes it according to the results of each processing region. Corresponding AI computing power level The processing regions in the low-resolution video are fed into a preset AI super-resolution reconstruction path of corresponding intensity to generate initial AI reconstructed video data. The initial AI computing power allocation results are arranged by video frame and processing area. Each record contains the area location identifier and the area's information. The grade value is used to distinguish between three levels of AI super-resolution execution paths: low computational load, standard, and high precision. The system first reads the region location identifier from each record to locate the processing region in the current frame. The spatial range is determined, and the corresponding temporal image content is extracted from the low-resolution frame buffer retained in the preceding decoding process to form a set of low-resolution temporal image blocks. . The cropping location is determined by the area location identifier in the record, and the cropping range is determined by this step based on the area. Calculations show that in real-time video scenarios, It can be composed of the current frame and the previously decoded frame. In offline enhancement scenarios, It can be composed of the current frame, the previous frame, and the next frame. Therefore, each output in step S102... It also determines the location of the corresponding processing area, the path strength for entering AI super-resolution, and the scope of the clipping context.

[0082] The system is cutting At that time, a boundary expansion method jointly associated with the AI ​​computing power level and the adjacent region level is adopted. The classic boundary expansion idea in image block processing is to add overlapping regions outside the processing block so that the intra-block computation can utilize the surrounding context and reduce block boundary discontinuities. This application extends this idea to the partitioned AI video super-resolution scenario, transforming the fixed boundary expansion into an expansion driven by the AI ​​computing power level, and adding a protection item for adjacent high-level regions: when the processing region When the AI ​​computing power level is high, the system extracts a larger surrounding context for it; when the processing area... When sharing an edge with a processing region of higher AI computing power, the system also increases its outward expansion range, allowing for overlapping transition space between the regional results of different AI paths. The adjacency protection in step S102 is used to determine the AI ​​computing power level of the processing region. The adjacent protection marker in this step Used to determine cutting The context of the time frame, and the two respectively apply to path selection and image patch cropping. Processing area The width of the outer boundary is determined by the following formula:

[0083] ;

[0084] in, Indicates the processing area Cut The outward expansion of the boundary width is determined by this step based on the region. Calculated based on the levels of adjacent regions; This indicates the basic outer width, which is configured by the system based on the convolutional receptive range and region stitching requirements of the preset AI super-resolution reconstruction path; This indicates the increased outer width between adjacent AI computing power levels, which is configured by the system based on the processing area size and boundary transition requirements. This indicates the processing area output in step S102. AI computing power level; This indicates a neighbor-level protection marker, derived from the region location identifier and adjacent region level in the initial AI computing power allocation result. The system traverses and processes the region. Adjacent processing regions sharing a common edge (top, bottom, left, right) If there are adjacent processing areas satisfy ,but In other cases This formula originates from arithmetic progressions in mathematics, with the basic form being "initial value plus level increment". This application adds to this basis... As a neighbor-level protection term, it provides additional context to processing regions closer to higher computing power regions. In the formula... , and Both represent the boundary width. and The width is a non-negative pixel. and These are numerical coefficients corresponding to the level or adjacency state. Therefore, the sum of the items on the right still yields the boundary width, which can be directly used for image patch cropping. When the outward expansion range exceeds the current frame boundary, the system uses boundary copying, mirror filling, or truncation cropping to form valid input, avoiding the edge area from being unable to enter the corresponding AI super-resolution path due to exceeding the screen.

[0085] In specific configurations, if , A static background area And there are no higher-level shared border regions nearby, then After substituting, we get The system extracts the basic context for this region; if a regular texture region And it shares an edge with the high computing power region, then After substituting, we get The system extracts a larger context for it to connect with the reconstruction results of adjacent high-computation-power regions; if a certain subtitle stroke, motion contour, or thin line structure is located in an area... ,even though This also results in a larger expansion range, allowing the AI ​​super-resolution path to simultaneously obtain target structure and surrounding background information when reconstructing the central region. The above calculations enable... The level information is transformed into the actual scope of selection, so as to avoid the AI ​​computing power level remaining only at the scheduling and marking level.

[0086] The system obtains Afterwards, according to Invoke the corresponding preset AI super-resolution reconstruction path. Low-computation AI super-resolution reconstruction paths are used for... The processing area, this path will The current frame image block is stitched together with the adjacent frame image blocks according to the channel direction, first through one A two-dimensional convolutional layer extracts shallow features, with an output channel count of 32; this is followed by four residual blocks, each containing two... A convolutional layer, a non-linear activation layer, and a skip connection are used to recover basic edges and low-frequency textures; then, resolution upsampling is achieved through a subpixel re-sorting upsampling layer, followed by a... The reconstructed convolutional layer outputs a high-resolution result for the corresponding processed region. Conventional AI super-resolution reconstruction paths are used for... The processing region's input organization is consistent with the low-computational-cost path, with 48 shallow feature channels and 8 residual blocks configured, and a [missing information - likely a specific feature block or parameter] is set before upsampling. Channel compression convolutional layers are used to fuse features from adjacent frames and control computational scale, suitable for handling moderate compression, general textures, and scattered residual regions. High-precision AI super-resolution reconstruction paths are used for... In the processing area, the number of shallow feature channels is configured to be 64, the number of residual blocks is configured to be 12, and a temporal feature fusion layer is set after the residual blocks; this temporal feature fusion layer concatenates the current frame features, previous frame features, and subsequent frame features according to the channel direction, through... Convolution compresses back to the number of trunk channels, then passes through two... Convolutional layers generate fused features, which are then passed to subpixel rearrangement upsampling layers and reconstruction convolutional layers. The three paths undergo training, model conversion, or parameter fixing before system deployment, and the input scale, magnification, number of channels, number of residual blocks, and available computing power are registered in the model library; at runtime, the system only needs to adjust the model based on these parameters. Call the corresponding model file or inference subgraph; do not retrain the model in this step. The input for all three paths is... The output scale is consistent, with differences in channel size, number of residual blocks, temporal fusion strength, and extended context range, making the output generated in step S102 consistent. It can clearly correspond to the differences in AI computational load.

[0087] The system traverses frame by frame. The region record. For any processing region The system first determines the location based on the area identifier and Cut from low-resolution frame buffer And then according to Inference is performed using either a low-computational-load, conventional, or high-precision AI super-resolution reconstruction path to obtain high-resolution region results for that area. Because... The system includes an extended range, and the regional super-resolution results also include a central effective region and overlapping boundary regions. When the system synthesizes the same frame, the central effective region prioritizes using the super-resolution results of the corresponding processing region itself. The overlapping boundary positions are smoothly transitioned according to their proximity to the central effective region of each region. Positions closer to the center of a region use more of that region's output, and positions closer to the boundary between two regions use the outputs of adjacent regions more evenly. This fusion rule originates from the weighted average transition concept in image stitching. When applied to the scenario of this application, it is used to connect the output results of different AI computing power paths, enabling a continuous transition between high-computing-power regions, normal-computing-power regions, and low-computing-power regions at spatial boundaries.

[0088] For example, in an online course video, step S102 might determine the processing area where the subtitle strokes are located. Define the background area adjacent to the top of the subtitle as This step is based on To crop a larger portion of the subtitle area This allows the high-precision AI super-resolution reconstruction path to simultaneously see the subtitle strokes, stroke edges, and surrounding background; for adjacent background areas, it is based on... The system invokes a conventional AI super-resolution reconstruction path. When two regions overlap near the subtitle edges, the system synthesizes the region results according to the aforementioned transition rules, using a higher-precision path for areas closer to the subtitle stroke center and a conventional path for areas closer to the background center, thus creating a natural transition between improved subtitle edge clarity and preserved background texture. In sports videos, if the moving object outline region is determined by step S102 as... This step will have a larger impact. Clipping includes both the inner and outer sides of the outline. High-precision AI super-resolution reconstruction path is used to restore motion boundary details; if the adjacent grassland or background area is The texture continuity around the contour is maintained through regular paths and boundary transitions.

[0089] After the system completes AI super-resolution inference and region synthesis for all processed regions of the current frame, it obtains the initial AI reconstruction result for that frame; after processing the complete video frame by frame, it generates the initial AI reconstructed video data. . By each processing area according to The image is synthesized after calling the corresponding AI super-resolution reconstruction path, preserving the video frame order and the content of the initial reconstructed image. This will serve as direct input for the next step, used to determine the local reprocessing area and perform subsequent compensation processing. Through the above execution process, the initial AI computing power allocation result output in step S102 is transformed into the cropping range, AI path strength, and region fusion strategy. The resource allocation driven by the encoded information forms the actual video super-resolution reconstruction result in this step.

[0090] like Figure 5 As shown, in step S104, based on the initial AI reconstructed video data, the internal structural continuity of the processing area and the structural stability of adjacent frames are calculated; based on the internal structural continuity of the area and the structural stability of adjacent frames, a local reprocessing area is determined; local high-precision AI compensation processing is performed on the local reprocessing area to obtain a local compensation result; the local compensation result is combined with the initial AI reconstructed video data to obtain the target super-resolution video data.

[0091] Specifically, this step receives the initial AI reconstruction video data output from step S103. and with Frame sequences and processing areas For each target area, the region requiring local reprocessing is identified, and then pre-defined local high-precision AI compensation processing is performed on that region to generate target super-resolution video data. . It has already been done in step S103 according to The initial reconstruction results, including those from different AI computing power levels and their output paths, are obtained after completing the partitioned AI super-resolution reconstruction and synthesis. Therefore, the judgment object in this step comes directly from the already generated super-resolution results, rather than re-parse the bitstream or redistribute the initial computing power. The system follows the processing area division method in step S103. Locating each processing region in each frame It reads the high-resolution image content of the current frame's processing area and the corresponding areas of adjacent frames. Here... Consistent with the meaning of the processing area in the aforementioned steps, it is determined by the location of the area within the video frame; It is the initial AI-reconstructed video data output in step S103; This refers to the target super-resolution video data output after local reprocessing and synthesis in this step. Through this connection, the initial AI reconstruction result obtained in step S103 serves as both the verification object and the image source for compensation processing in this step.

[0092] The system processes each region When performing local structure verification, the focus is on extracting the structural continuity within a region and the structural stability between corresponding regions in adjacent frames. The structural continuity within a region is achieved through fixed edge operators and connectivity checks. Specifically, the Sobel or Scharr operator can be used to calculate the horizontal and vertical edge responses of the brightness image within the processed region, respectively. The absolute values ​​of the edge responses in the two directions are added to form a region edge response map. Then, the edge responses within the processed region are averaged and processed to obtain a region gradient description that can be used in subsequent calculations. The same fixed edge operator and the same processing method are used to obtain the region gradient description for corresponding regions in adjacent frames, making the structural responses of the current frame and adjacent frames comparable. Subsequently, the system checks for phenomena such as short-distance line breaks, local adhesion of text strokes, broken thin lines, parallel trailing on the outer side of motion boundaries, and abrupt changes in sharpness at the boundaries of adjacent regions. These checks can be determined by the length of edge connectivity segments, the interval between adjacent edge segments, the number of parallel edge responses, and the changes in edge responses at the boundaries of regions. The relevant judgment boundaries are configured by the system before deployment based on the verification results of subtitles, thin line structures, motion contours, and ordinary background samples, and written into the local structure verification parameter table. This review process is suitable for reviewing the results of the initial AI reconstruction in video super-resolution scenarios, because local failures are usually manifested as a sudden oversharpening of a frame, a sudden weakening of edges, flickering of textures between adjacent frames, or unstable ghosting near motion contours.

[0093] Processing area The local temporal structure jump variables are obtained by the following formula:

[0094] ;

[0095] The initial source of this formula is the relative difference measure in mathematics, whose basic form is to divide the absolute difference of two similar non-negative quantities by their total amount, which is used to express the relative degree of change between them; in this application, the two similar non-negative quantities are specifically set as the initial AI reconstruction video data. The gradient descriptions of the current frame's processing region and the corresponding regions in adjacent frames are used to obtain information for judging local structural transitions. .in, Indicates the processing area The local temporal structure jump variables are determined by this step based on The structural changes in corresponding regions of adjacent frames are calculated. express Processing area in the current frame The regional gradient description is obtained by calculating the horizontal and vertical edge responses using fixed edge operators and then organizing the edge responses into the average response within the region. express Adjacent frames and processing area The regional gradient description at the corresponding location is obtained by the same fixed edge operator and the same averaging method; This represents a pre-configured, minimal, stable term greater than zero, used to ensure that the denominator can still be calculated stably in low-texture regions. and For describing the responses of similar structures, the numerator represents the difference between the two, and the denominator represents the overall response intensity of the two with a stability term added. The calculation result is a proportional jump variable, which can be compared with a preset structural jump judgment threshold. Compare; and Both are proportional judgment values, which are pre-configured by the system based on video type, frame rate, and requirements for timing stability.

[0096] In the comparison of adjacent frames, if the previous frame is used for real-time processing, then The gradient is derived from the previous frame. If offline enhancement can obtain both previous and next frames, the system can compare the regional gradient descriptions of the current frame with the previous frame and the current frame with the next frame in the same way, and use the one with the more significant change as the gradient description for that region. For example, in an online course video, after the initial AI reconstruction in step S103, the edges of the characters in a certain subtitle area are significantly sharpened in the current frame, while the edges at the corresponding positions in the previous frame are weaker. After refining with a fixed edge operator, the result is... , ,like Substituting into the formula, we get In another sports video, a certain motion outline region shows obvious ghosting in the current frame, while the corresponding boundary in the previous frame is clearer. After processing, the image is... , ,but The latter case The significantly higher value indicates that the region exhibits a stronger temporal structural shift after the initial AI reconstruction, making it more suitable for local reprocessing.

[0097] The system will With preset threshold The comparisons were made, and the local reprocessing markers were determined based on the results of the internal structure review of the region. .like Reaching or exceeding This indicates a significant change in structural response between corresponding regions in adjacent frames. If the internal structure of a region reveals short-distance line breaks, merging of text strokes, broken thin lines, motion boundary blurring, or abrupt sharpness changes at region boundaries, it also indicates a local structural anomaly in that region. Local reprocessing markers. Determine as follows:

[0098] ;

[0099] in, Indicates the processing area The local reprocessing markers are determined by this step based on The partial verification results were obtained; Indicates the processing area Entering local high-precision AI compensation processing. Indicates the processing area Retain the initial AI reconstruction results generated in step S103; Indicates the aforementioned local temporal structure jump variable; This represents the preset structural transition judgment threshold. This formula originates from the threshold triggering and state triggering combination rules in engineering control, that is, when the continuous detection quantity exceeds the preset threshold, or the discrete structure state enters an abnormal condition, subsequent execution actions are triggered; in this application, the continuous detection quantity is... Discrete structural states are the processing regions Whether there are local structural anomalies, the triggered execution action is local high-precision AI compensation processing.

[0100] for Processing area The system from The region and its surrounding context are extracted as input to a local high-precision AI compensation path. This compensation path performs detailed corrections and temporal stabilization processing based on the initial AI reconstruction results. Its output scale corresponds to the input region, facilitating backfilling. The same spatial location. The local high-precision AI compensation path includes a structural feature extraction part, an edge preservation part, and a temporal consistency part. The structural feature extraction part receives the local region of the current frame and the corresponding region of the adjacent frame, concatenates the two according to the channel direction, and inputs them into a single... A two-dimensional convolutional layer with 64 output channels is followed by 6 residual blocks, each containing two channels. A convolutional layer, a non-linear activation layer, and a skip connection are used to extract local texture, edge, and detail features. The edge-preserving part uses a fixed edge operator to extract edge responses from the input local region, and then passes them through two... Convolutional layers map edge responses to auxiliary edge features, enhancing the continuity of character strokes, thin line structures, and motion boundaries. The temporal consistency part integrates features from the current frame with those from adjacent frames. After convolutional compression, the mixture is fused, and then processed by two... The convolutional layer outputs temporal correction features to reduce local flicker and motion boundary drift between adjacent frames. Finally, the system concatenates the structural features, edge-aided features, and temporal correction features along the channel direction, and then... Convolutional layers are fused, and then... The output convolutional layer generates local compensation results.

[0101] Before performing local compensation, the system will consider adjacent components that all satisfy... Multiple processing regions are merged into a continuous reprocessing block, which is then fed into a local high-precision AI compensation path. This processing can adapt to continuous anomalies in video super-resolution, such as subtitle strokes often spanning multiple adjacent regions, and motion contour trails often being continuously distributed along object boundaries; after merging these regions, the compensation path can be corrected within a more complete local structural range. The system reserves the processing area. The corresponding initial AI reconstruction result. For the region obtained through local compensation, the system will put the compensation result back into the original region. The corresponding positions are determined, and a transition zone is set at the boundary of the compensation area; within the transition zone, the position closer to the center of the compensation area uses more local compensation results, while the position closer to the uncompensated area uses more local compensation results. The initial AI reconstruction results in the model allow for a smooth transition between the localized correction area and the surrounding area.

[0102] In online course videos, if a subtitle's strokes are continuously distributed across multiple adjacent processing areas, some areas may experience issues due to... The area is relatively high, and some areas are identified as having overlapping strokes or broken lines. The system will merge these adjacent areas into a continuous reprocessing block and input a local high-precision AI compensation path to ensure consistent detail correction across the entire subtitle edge. In sports videos, if multiple areas near the outline of a moving object exhibit motion blur and edge flickering, and these are identified as... The system also merges these regions and performs local compensation to ensure the continuity of the contour boundary correction along the motion direction. After completing local reprocessing and region synthesis for each frame, the system outputs the target super-resolution video data in the original video frame order. . Depend on middle The region and the area after local high-precision AI compensation processing The regions together form a whole, maintaining target resolution and frame sequence continuity.

[0103] Figure 6 A structural block diagram of an AI computing power adaptive allocation system for video super-resolution reconstruction according to some embodiments of this disclosure is shown. Figure 6 As shown, the system includes: a region degradation description generation module 601, a computing power level determination module 602, a partitioned AI super-resolution reconstruction module 603, and a local compensation module 604.

[0104] The region degradation description generation module 601 is used to obtain the block-level quantization intensity, block-level motion vector and block-level residual intensity of the video stream to be processed during the decoding process, divide the video frames into processing regions according to the preset region size, and organize the block-level quantization intensity, the block-level motion vector and the block-level residual intensity into region degradation description data.

[0105] The computing power level determination module 602 is used to generate the AI ​​computing power level of the processing area based on the area degradation description data.

[0106] The partitioned AI super-resolution reconstruction module 603 is used to determine the outer boundary width of the image block cropping according to the AI ​​computing power level, crop the temporal image block of the processing area according to the outer boundary width, call the corresponding preset AI super-resolution reconstruction path according to the AI ​​computing power level to reconstruct the temporal image block, and obtain the regional super-resolution result; and synthesize the initial AI reconstructed video data according to the regional super-resolution result.

[0107] The local compensation module 604 is used to calculate the internal structural continuity of the processing area and the structural stability of adjacent frames based on the initial AI reconstructed video data; determine the local reprocessing area based on the internal structural continuity of the area and the structural stability of adjacent frames; perform local high-precision AI compensation processing on the local reprocessing area to obtain the local compensation result; and synthesize the local compensation result with the initial AI reconstructed video data to obtain the target super-resolution video data.

[0108] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. An AI computing power adaptive allocation method for video super-resolution reconstruction, characterized in that, include: Obtain the block-level quantization intensity, block-level motion vector, and block-level residual intensity of the video stream to be processed during the decoding process. Divide the video frames into processing regions according to the preset region size. Organize the block-level quantization intensity, the block-level motion vector, and the block-level residual intensity into region degradation description data. The AI ​​computing power level of the processing area is generated based on the regional degradation description data. The outer boundary width of the image patch is determined according to the AI ​​computing power level, and the temporal image patch of the processing area is cropped according to the outer boundary width; the temporal image patch is reconstructed by calling the corresponding preset AI super-resolution reconstruction path according to the AI ​​computing power level to obtain the region super-resolution result; the initial AI reconstructed video data is synthesized according to the region super-resolution result. Based on the initial AI-reconstructed video data, the internal structural continuity of the processing area and the structural stability of adjacent frames are calculated; based on the internal structural continuity and the structural stability of adjacent frames, a local reprocessing area is determined; local high-precision AI compensation processing is performed on the local reprocessing area to obtain a local compensation result; the local compensation result is combined with the initial AI-reconstructed video data to obtain the target super-resolution video data.

2. The AI ​​computing power adaptive allocation method for video super-resolution reconstruction according to claim 1, characterized in that, The step of organizing the block-level quantization intensity, the block-level motion vector, and the block-level residual intensity into region degradation description data includes: Read the quantization parameters of the coded blocks within the processing area to obtain the region quantization description; Read the motion vectors of the coded blocks within the processing area, determine the main motion vector, calculate the relative deviation of the motion vectors of the coded blocks relative to the main motion vector, and obtain the regional motion vector stability based on the relative deviation. The distribution of residual coding blocks within the processing area is statistically analyzed to obtain a description of the regional residuals; The region quantization description, the region motion vector stability, and the region residual description are combined into the region degradation description data.

3. The AI ​​computing power adaptive allocation method for video super-resolution reconstruction according to claim 1, characterized in that, The step of generating the AI ​​computing power level of the processing region based on the region degradation description data includes: The region quantization description in the region degradation description data is converted into a compressed state, and the region residual description in the region degradation description data is converted into a residual state; The basic computing power level of the processing region is determined based on the compression state, the residual state, and the regional motion vector stability in the regional degradation description data. The AI ​​computing power level of the processing area is determined based on the base computing power level of the processing area and the base computing power level of adjacent processing areas.

4. The AI ​​computing power adaptive allocation method for video super-resolution reconstruction according to claim 3, characterized in that, The step of determining the AI ​​computing power level of the processing region based on the base computing power level of the processing region and the base computing power levels of adjacent processing regions includes: Determine whether the basic computing power level of the processing area is a low computing power level; If the base computing power level of the processing region is the low computing power level, and there is an adjacent processing region sharing an edge with the processing region, and the base computing power level of the adjacent processing region is the high computing power level, then the AI ​​computing power level of the processing region is set to the normal computing power level. If the basic computing power level of the processing region is not the low computing power level, or there is no adjacent processing region sharing an edge with the processing region and having a high computing power level, then the basic computing power level of the processing region shall be used as the AI ​​computing power level of the processing region.

5. The AI ​​computing power adaptive allocation method for video super-resolution reconstruction according to claim 1, characterized in that, Determining the outer boundary width of image patch cropping based on the AI ​​computing power level includes: Obtain the base expansion width and the expansion width of adjacent levels; Obtain a neighbor protection flag, which is used to indicate whether there is an adjacent processing region that shares an edge with the processing region and has an AI computing power level higher than that of the processing region; The outer expansion boundary width is calculated based on the basic outer expansion width, the adjacent level outer expansion width, the AI ​​computing power level, and the adjacent level protection flag.

6. The AI ​​computing power adaptive allocation method for video super-resolution reconstruction according to claim 1, characterized in that, The step of reconstructing the temporal image block by calling the corresponding preset AI super-resolution reconstruction path according to the AI ​​computing power level includes: If the AI ​​computing power level is low, the low-computation AI super-resolution reconstruction path is called to reconstruct the time-series image block; If the AI ​​computing power level is a conventional computing power level, the conventional AI super-resolution reconstruction path is called to reconstruct the temporal image block; If the AI ​​computing power level is high, the high-precision AI super-resolution reconstruction path is invoked to reconstruct the time-series image block; The number of residual blocks in the low-computation AI super-resolution reconstruction path, the conventional AI super-resolution reconstruction path, and the high-precision AI super-resolution reconstruction path increases sequentially.

7. The AI ​​computing power adaptive allocation method for video super-resolution reconstruction according to claim 1, characterized in that, The process involves calculating the internal structural continuity of the processing area and the structural stability of adjacent frames based on the initial AI-reconstructed video data. Based on the structural continuity within the region and the structural stability of adjacent frames, a local reprocessing region is determined, including: The region gradient description of the processing area is calculated using a fixed edge operator, and the region gradient description of the corresponding region in adjacent frames is also calculated. Calculate the local temporal structure jump variable of the processing region based on the regional gradient description of the processing region and the regional gradient description of the corresponding region of the adjacent frame; The internal structure of the processed area is reviewed to obtain the results of local structural anomaly detection. If the local temporal structure jump variable reaches the preset structure jump judgment threshold, or if the local structure anomaly check result indicates the existence of a local structure anomaly, then the processing area is determined as the local reprocessing area.

8. The AI ​​computing power adaptive allocation method for video super-resolution reconstruction according to claim 7, characterized in that, The step of performing local high-precision AI compensation processing on the local reprocessed region to obtain the local compensation result includes: The adjacent processing regions that are both identified as the local reprocessing regions are merged to obtain a continuous reprocessing block; The continuous reprocessing block is input into the local high-precision AI compensation path to obtain the local compensation result; The local high-precision AI compensation path includes a structural feature extraction part, an edge preservation part, and a temporal consistency part.

9. The AI ​​computing power adaptive allocation method for video super-resolution reconstruction according to claim 1, characterized in that, The initial AI-reconstructed video data synthesized based on the region super-resolution results includes: For the processing region within the same video frame, the central effective region in the region super-resolution result is taken as the final reconstruction result of the processing region; For the overlapping regions of the processing regions adjacent to the processing region, the super-resolution results of the adjacent processing regions are weighted and transitioned according to the proximity of the pixels in the overlapping region to the central effective region, so as to obtain the initial AI reconstructed video data.

10. An AI computing power adaptive allocation system for video super-resolution reconstruction, characterized in that, include: The region degradation description generation module is used to obtain the block-level quantization intensity, block-level motion vector and block-level residual intensity of the video bitstream to be processed during the decoding process, divide the video frames into processing regions according to the preset region size, and organize the block-level quantization intensity, the block-level motion vector and the block-level residual intensity into region degradation description data. The computing power level determination module is used to generate the AI ​​computing power level of the processing area based on the area degradation description data. The partitioned AI super-resolution reconstruction module is used to determine the outer boundary width of the image block cropping according to the AI ​​computing power level, crop the temporal image block of the processing area according to the outer boundary width, call the corresponding preset AI super-resolution reconstruction path according to the AI ​​computing power level to reconstruct the temporal image block, and obtain the regional super-resolution result; and synthesize the initial AI reconstructed video data according to the regional super-resolution result. The local compensation module is used to calculate the internal structural continuity of the processing area and the structural stability of adjacent frames based on the initial AI reconstructed video data; determine the local reprocessing area based on the internal structural continuity of the area and the structural stability of adjacent frames; perform local high-precision AI compensation processing on the local reprocessing area to obtain the local compensation result; and synthesize the local compensation result with the initial AI reconstructed video data to obtain the target super-resolution video data.