Resolution video transcoding and super-resolution fusion processing method for ai large model

Abstract

The present application relates to the technical field of video processing, and especially relates to a resolution video transcoding and super-resolution fusion processing method for AI large models, which comprises the following steps: identifying scene switching nodes to segment independent scenes, obtaining subject and background division information and target distribution information; selecting initial transcoding parameters that meet a recovery rate threshold and have minimum bandwidth according to a recoverability curve to perform compression transcoding; extracting and analyzing video frames in each scene; performing recognizable degree evaluation on the transcoded and super-resolution reconstructed video to obtain average recognizable degree, and triggering parameter correction when the average recognizable degree does not meet the qualified condition; super-resolution reconstructing the low-code-rate video stream after correction, and inputting the low-code-rate video stream into a downstream large model to perform video understanding. The present application realizes hierarchical adaptive correction based on recognizable degree feedback, thereby ensuring the detection accuracy of the large model and reducing the transmission bandwidth and storage cost to the maximum extent.

Description

Technical Field

[0001] This invention relates to the field of video processing technology, and in particular to a resolution video transcoding and super-resolution fusion processing method for large AI models. Background Technology

[0002] With the rapid development of artificial intelligence (AI) technology, large AI models have been widely applied in computer vision tasks such as video understanding, object detection, and behavior recognition. Typically, large AI models require high-resolution input videos; high-resolution videos provide richer detail information, resulting in higher detection accuracy and recognition precision. However, in scenarios where edge devices (such as cameras and mobile terminals) upload video data to the cloud, the transmission of high-resolution video consumes significant network bandwidth, and cloud storage costs also increase significantly. To reduce bandwidth and storage costs, existing technologies typically employ video transcoding and compression to compress the original high-resolution video into a low-bitrate video before uploading. However, traditional transcoding and compression introduce varying degrees of compression distortion while reducing the bitrate, leading to a decrease in video quality and consequently a significant degradation in the detection accuracy and recognition performance of downstream large AI models. On the other hand, while cloud-based super-resolution reconstruction technology can restore the resolution and visual quality of compressed videos to some extent, existing super-resolution models are mostly trained based on ideal degradation assumptions, and their ability to recover complex distortions such as blockiness and blur introduced by real transcoding and compression is limited, making it difficult to guarantee effective restoration of the large model's detection accuracy after reconstruction.

[0003] Chinese Patent Application No. CN202511868758.1 discloses a post-processing method and system for improving video smoothness and clarity. The method first responds to the generation of upstream video by transcoding the upstream video to generate corresponding video frames, a PTS frame code table, and audio. Then, the video frames are input into a preset scene detection and video scene segmentation model to generate a target scene frame series for auxiliary frame interpolation. Next, the target scene frame series is input into a frame interpolation model based on a diffusion Transformer architecture for frame interpolation to obtain an updated frame series. Simultaneously, based on the updated frame series, the PTS frame code table is updated synchronously. The updated frame series is then sent to a real-time streaming video super-resolution model for super-resolution updates to obtain a super-resolution continuous frame series. Finally, the super-resolution continuous frame series, audio, and the synchronously updated PTS frame code table are aligned, validated, and fused to output an improved video. This invention solves the problem that existing video optimization techniques struggle to simultaneously achieve both smoothness and clarity.

[0004] However, existing technologies still have the following problems: Scene segmentation only serves frame interpolation and super-resolution processes to improve visual quality, without addressing the perception and adaptive correction of transcoding parameters on the detection accuracy of large models. It cannot solve the balance between bandwidth consumption and the performance of downstream tasks of large models. Summary of the Invention

[0005] To address this, the present invention provides a resolution video transcoding and super-resolution fusion processing method for large AI models, which overcomes the limitations of existing technologies where scene segmentation only serves frame interpolation and super-resolution processes to improve visual quality, without addressing the perception and adaptive correction of transcoding parameters on the detection accuracy of large models, and thus cannot solve the problem of balancing bandwidth consumption and the performance of downstream tasks of large models.

[0006] To achieve the above objectives, this invention provides a method for high-resolution video transcoding and super-resolution fusion processing for large AI models. It includes: Step S1: Identify scene switching nodes in the input video to segment it into several independent scenes, and obtain the division information of the main body region and background region and the target distribution information in the current scene video frame; Step S2: Based on the pre-established recoverability curve, combined with the partitioning information and target distribution information, select the initial transcoding parameters that meet the preset recovery rate threshold and have the minimum bandwidth usage to perform compression transcoding. Step S3: In each of the independent scenes, at least one video frame for analysis is extracted based on the distribution statistical characteristics of the main body region; Step S4: Evaluate the recognizability of the analyzed video frame after transcoding and super-resolution reconstruction, obtain the average recognizability of the current scene, and trigger parameter correction when the average recognizability does not meet the preset qualification conditions. Step S5: The low bitrate video stream, after parameter correction and transcoding parameter processing, is super-reconstructed and input into the downstream AI big model to perform video understanding tasks.

[0007] Furthermore, the division between the main body region and the background region is achieved through region analysis based on boundary density, including: Divide the current video frame into several regions to be analyzed; Based on the grayscale edge detection algorithm, the cumulative edge length of each region to be analyzed is calculated, and the ratio of the cumulative edge length to the area of ​​the region to be analyzed is used as the boundary density of the region. The regions to be analyzed with a boundary density greater than a preset density threshold are identified as the main regions, and the regions to be analyzed with a boundary density less than or equal to the preset density threshold are identified as the background regions.

[0008] Furthermore, the scene switching node is identified through the regional switching change rate, including: For two adjacent frames, the first number of times the main area changes to the background area in the region to be analyzed, and the second number of times the background area changes to the main area, are counted. The sum of the first number and the second number is taken as the region switching change rate. When the area switching change rate is greater than the preset switching rate threshold, it is determined that a scene switch has occurred between the two adjacent frames, and the first frame after the switch is taken as the starting frame of the new scene.

[0009] Further, in step S3, at least one video frame for analysis is extracted based on the distribution statistical characteristics of the main region, including: Calculate the proportion of the main body region in each video frame in the current scene, and select the video frame with the largest proportion of the main body region as the first candidate frame; The cumulative frequency of each region to be analyzed being identified as the main region in the scene is counted, and video frames covering high-frequency main regions are selected as second candidate frames based on the cumulative frequency of occurrence. By combining the first candidate frame and the second candidate frame, at least one video frame for analysis is determined.

[0010] Further, in step S4, the identifiability assessment includes: Based on the detection confidence or classification accuracy output by the downstream AI large model, the identifiable quantization value of each analyzed video frame is calculated, and the average of the quantization values ​​of each analyzed video frame is taken as the average identifiability of the current scene.

[0011] Furthermore, in step S4, the correction strategy after triggering parameter correction includes: Determine the anomaly type based on the trend of identifiability changes in the current scene over time: If the trend of change shows abrupt fluctuations, it is determined that the scene segmentation is abnormal, and the first correction strategy is executed. The first correction strategy includes raising the switching rate threshold in the judgment criteria of the scene switching node. If the changing trend shows a gradual deterioration, it is determined that the compression capability is insufficient, and a second correction strategy is executed. The second correction strategy includes lowering the bitrate or resolution lower limit in the transcoding parameters.

[0012] Furthermore, the trend of change is quantified by the magnitude of the variation: Based on multiple periodically determined average recognizability values, a time-average recognizability curve for the scene is plotted, and the slope at the current time node is calculated as the alienation magnitude. When the degree of alienation is greater than the preset alienation threshold, it is determined that the change trend exhibits abrupt fluctuations. When the degree of alienation is less than or equal to the preset alienation threshold, it is determined that the change trend exhibits a gradual deterioration.

[0013] Furthermore, in the first correction strategy, after increasing the switching rate threshold in the determination criteria of the scene switching node, it also includes: Statistical analysis of the actual scene switching frequency within a preset time period after the current scene switch; If the actual scene switching frequency is greater than the preset switching frequency threshold, then the boundary density threshold for dividing the main area and the background area will be further increased.

[0014] Furthermore, if the average recognizability still does not meet the preset qualification condition after the first correction strategy is executed, the second correction strategy is switched to lower the bitrate or resolution lower limit in the transcoding parameters.

[0015] Furthermore, in the second correction strategy, lowering the lower limit of the bitrate or resolution in the transcoding parameters includes: Obtain the recoverability curve corresponding to the current scene, and identify the cliff contribution node in the recoverability curve where the detection accuracy recovery rate drops sharply with the decrease of the bit rate; The bitrate value corresponding to the cliff contributing node is used as the critical bitrate, and the target bitrate in the compression standard is adjusted to be no lower than the critical bitrate.

[0016] Compared with existing technologies, the beneficial effects of this invention are as follows: This embodiment uses a boundary density-based region analysis method to determine texture-dense regions with boundary density higher than a threshold as the main body region and flat regions with boundary density lower than a threshold as the background region. This achieves rapid and accurate segmentation of the main body and background, with low computational overhead and good real-time performance, providing a precise spatial region foundation for subsequent scene switching recognition, target distribution statistics, and transcoding parameter decisions. At the same time, the preset density threshold is determined by either pre-calibration or adaptive calculation, which is applicable to conventional scenes with calibration samples and can also autonomously adapt the threshold based on the texture features of the first frame in cold start scenarios. Furthermore, the threshold can be periodically updated as the scene changes, ensuring the consistency and robustness of region segmentation under different scenes, and providing a reliable spatial region segmentation guarantee for subsequent differentiated compression strategies.

[0017] Furthermore, this embodiment uses the dual-criteria extraction method based on the proportion of the main body region and the cumulative frequency of occurrence. The selected video frames for analysis not only include the most prominent key moments of the main body in the scene, but also take into account the overall spatial coverage of the main body's activities in the scene. This ensures that the subsequent recognition evaluation can effectively reflect the true recognition performance level of the scene after transcoding compression and super-resolution reconstruction.

[0018] Furthermore, this embodiment, through anomaly type discrimination and hierarchical correction strategy based on changing trend patterns, can accurately locate the root cause of the decline in the recognition performance of large models, avoid taking ineffective correction measures due to misjudgment, and improve the efficiency and accuracy of parameter correction. At the same time, the binary discrimination method of abrupt fluctuations and gradual degradation is simple to implement, has low computational overhead, and is suitable for real-time execution in the cloud.

[0019] Furthermore, this embodiment transforms the temporal trend of identifiability into an objective numerical comparison by using a quantitative judgment method based on the magnitude of anomalies. This enables the automatic and accurate identification of two types of anomalies: abrupt fluctuations and gradual degradation, providing a quantitative decision-making basis for subsequent implementation of targeted hierarchical correction strategies. Through the two-level progressive adjustment of the switching rate threshold and boundary density threshold in the first correction strategy, a hierarchical optimization link is formed from the switching judgment standard to the region division basis, which can accurately solve the scene segmentation anomaly problem. By verifying whether the identifiability meets the standard after the first correction strategy is executed, and then switching to the second correction strategy if it does not, a layer-by-layer investigation and correction from scene segmentation anomalies to insufficient compression capability is achieved, ensuring that the effect of each correction measure is traceable and verifiable. By using a bitrate reduction method based on the constraint of the recovery curve interruption cliff contribution node, the target bitrate in the compression standard is constrained above the critical bitrate that the super-resolution model can effectively recover, avoiding the risk of over-compression caused by blind parameter tuning, and maintaining a low transmission bandwidth to the maximum extent while ensuring the detection accuracy of the large model. Attached Figure Description

[0020] Figure 1 This is a flowchart of the resolution video transcoding and super-resolution fusion processing method for large AI models according to the present invention. Figure 2 This is a flowchart illustrating the division of the main body region and background region in the resolution video transcoding and super-resolution fusion processing method for large AI models according to the present invention. Detailed Implementation

[0021] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0022] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0023] Furthermore, it should be noted that, in the description of this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0024] Please see Figures 1-2 As shown, Figure 1 This is a flowchart of the resolution video transcoding and super-resolution fusion processing method for large AI models according to the present invention. Figure 2 This is a flowchart illustrating the division of the main body region and background region in the resolution video transcoding and super-resolution fusion processing method for large AI models according to the present invention.

[0025] This invention provides a resolution video transcoding and super-resolution fusion processing method for large-scale AI models, comprising: Step S1: Identify scene switching nodes in the input video to segment it into several independent scenes, and obtain the division information of the main body region and background region and the target distribution information in the current scene video frame; Step S2: Based on the pre-established recoverability curve, combined with the partitioning information and target distribution information, select the initial transcoding parameters that meet the preset recovery rate threshold and have the minimum bandwidth usage to perform compression transcoding. Step S3: In each of the independent scenes, at least one video frame for analysis is extracted based on the distribution statistical characteristics of the main body region; Step S4: Evaluate the recognizability of the analyzed video frame after transcoding and super-resolution reconstruction, obtain the average recognizability of the current scene, and trigger parameter correction when the average recognizability does not meet the preset qualification conditions. Step S5: The low bitrate video stream, after parameter correction and transcoding parameter processing, is super-reconstructed and input into the downstream AI big model to perform video understanding tasks.

[0026] Specifically, the division between the main body region and the background region is achieved through region analysis based on boundary density, including: Divide the current video frame into several regions to be analyzed; Based on the grayscale edge detection algorithm, the cumulative edge length of each region to be analyzed is calculated, and the ratio of the cumulative edge length to the area of ​​the region to be analyzed is used as the boundary density of the region. The regions to be analyzed with a boundary density greater than a preset density threshold are identified as the main regions, and the regions to be analyzed with a boundary density less than or equal to the preset density threshold are identified as the background regions.

[0027] In this embodiment, the edge detection module processes consecutive video frames. First, each frame undergoes grayscale preprocessing, converting the RGB color image to grayscale to reduce the computational complexity of subsequent edge detection. Then, the edge detection module divides each frame into several equally sized rectangular regions for analysis using a fixed spatial grid. For example, with a full HD input video resolution, each frame can be divided into tens by tens of regions, each region being tens by tens of pixels. For other resolutions, the grid parameters can be adapted according to a proportional scaling principle, but it should be ensured that each region contains sufficient pixel information to support effective edge statistics.

[0028] After segmentation, the edge detection module independently performs edge detection operations on each region to be analyzed. This embodiment uses the Sobel operator as the grayscale edge detection algorithm, calculating the gradient magnitude of each pixel in both the horizontal and vertical directions. When the gradient magnitude exceeds a preset edge determination threshold, the pixel is marked as an edge pixel. After completing pixel-by-pixel edge determination within the region, the edge detection module counts the total number of edge pixels within the region to be analyzed, using this number as the cumulative edge length. Since the cumulative edge length is positively correlated with the region area, to eliminate the influence of region area differences on the determination results, the cumulative edge length is divided by the total number of pixels in the region to be analyzed (i.e., the region area) to obtain a normalized boundary density value.

[0029] After obtaining the boundary density of each region to be analyzed, the edge end compares it with a preset density threshold. The preset density threshold is determined in advance by collecting several frames of video images of the same type as the current application scenario as calibration samples, calculating the boundary density values ​​of the manually labeled main body region and background region in each frame, and statistically analyzing the minimum value of the boundary density of the main body region and the maximum value of the boundary density of the background region. The preset density threshold is then set to a value between the two. For monitoring scenarios, the minimum boundary density of the main body region is approximately slightly more than one-tenth, and the maximum boundary density of the background region is approximately slightly less than one-fifteenth. In this embodiment, the preset density threshold is set to one-tenth. When the boundary density of a region to be analyzed is greater than this threshold, it indicates that the image texture in this region is rich and the edges are dense, usually corresponding to the target object or foreground subject in the video image. Therefore, this region is identified as the main body region. Conversely, when the boundary density is less than or equal to this threshold, it indicates that the image in this region is relatively flat and the edges are sparse, usually corresponding to the background region such as the sky, walls, and ground in the image. Therefore, this region is identified as the background region.

[0030] For cold-start scenarios lacking calibration samples, an adaptive determination method can be adopted: calculate the average and dispersion of the boundary density of all regions to be analyzed in the first frame of the current scene, and set a preset density threshold as the result of proportionally superimposing the average and dispersion, where the contribution ratio of dispersion is approximately between three-fifths and four-fifths. In this embodiment, four-fifths is used, making the threshold adaptive to the overall texture richness of the current image. This adaptive threshold is periodically updated as the scene content dynamically changes, ensuring that the segmentation accuracy remains stable during long-term operation.

[0031] This embodiment employs a boundary density-based region analysis method to identify texture-dense regions with boundary density exceeding a threshold as subject regions and flat regions with boundary density below a threshold as background regions. This achieves rapid and accurate subject-background segmentation with low computational overhead and good real-time performance, providing a precise spatial region foundation for subsequent scene switching recognition, target distribution statistics, and transcoding parameter decisions. Furthermore, by determining the preset density threshold through either pre-calibration or adaptive calculation, it is applicable to both conventional scenes with calibrated samples and cold-start scenarios where the threshold can be autonomously adapted based on the texture features of the first frame. The threshold can also be periodically updated according to scene changes, ensuring consistency and robustness of region segmentation across different scenarios and providing a reliable spatial region segmentation guarantee for subsequent differentiated compression strategies.

[0032] Specifically, the scene switching node is identified through the area switching change rate, including: For two adjacent frames, the first number of times the main area changes to the background area in the region to be analyzed, and the second number of times the background area changes to the main area, are counted. The sum of the first number and the second number is taken as the region switching change rate. When the area switching change rate is greater than the preset switching rate threshold, it is determined that a scene switch has occurred between the two adjacent frames, and the first frame after the switch is taken as the starting frame of the new scene.

[0033] In this embodiment, the edge processing unit takes frame sequences as the processing object and reads two adjacent frames sequentially to determine scene switching. Before determining the switching, the edge processing unit has completed the subject / background marking of each region to be analyzed in each frame according to the aforementioned boundary density-based region analysis method. That is, each region to be analyzed is marked as a subject region or a background region in the current frame.

[0034] After reading the region labeling results of the current frame and the previous frame, the edge processor iterates through all regions to be analyzed, comparing the labeling status changes between the two frames for each region. For each region to be analyzed, if it was labeled as a main region in the previous frame but as a background region in the current frame, it is counted in the first count; if it was labeled as a background region in the previous frame but as a main region in the current frame, it is counted in the second count. After the traversal is complete, the edge processor adds the first and second counts to obtain the region switching change rate.

[0035] The physical meaning of the region switching change rate is that when a large number of areas to be analyzed in a video frame simultaneously switch between the subject and the background, it indicates a drastic change in the spatial layout of the image. This change usually corresponds to events such as shot switching, scene transitions, or significant shifts in perspective. Therefore, the edge processing unit compares the region switching change rate with a preset switching rate threshold. When the region switching change rate exceeds this threshold, it is determined that a scene switch has occurred between two adjacent frames, and the current frame after the switch is used as the starting frame of the new scene. This divides the input video into several independent scenes with coherent content.

[0036] The preset switching rate threshold is determined in advance as follows: Test videos containing multiple known scene switching moments are collected; the region switching change rate of adjacent frames before and after each scene switching moment is calculated; the minimum value among the region switching change rates corresponding to all scene switching moments is taken as the lower reference bound; simultaneously, the region switching change rate of adjacent frames within the same scene is calculated; the maximum value among the corresponding region switching change rates of all non-switching frames is taken as the upper reference bound; the preset switching rate threshold is set to a value between this lower and upper reference bound. In this embodiment, for typical surveillance video and short video scenarios, the preset switching rate threshold is approximately one-quarter of the total number of regions to be analyzed. That is, when more than one-quarter of the total regions to be analyzed undergo subject-background switching, a scene switching determination is triggered.

[0037] For cold start scenarios lacking calibration videos, an adaptive determination method can be adopted: calculate the average and fluctuation range of the region switching change rate of each adjacent frame within the initial segment of the current video, and set the preset switching rate threshold as the result of proportionally adding the average and fluctuation ranges, so that the threshold adapts to the overall content change intensity of the current video. This adaptive threshold is periodically updated as the video content progresses, ensuring that the scene segmentation accuracy remains stable during long-term operation.

[0038] Specifically, in step S3, at least one video frame for analysis is extracted based on the distribution statistical characteristics of the main region, including: Calculate the proportion of the main body region in each video frame in the current scene, and select the video frame with the largest proportion of the main body region as the first candidate frame; The cumulative frequency of each region to be analyzed being identified as the main region in the scene is counted, and video frames covering high-frequency main regions are selected as second candidate frames based on the cumulative frequency of occurrence. By combining the first candidate frame and the second candidate frame, at least one video frame for analysis is determined.

[0039] In this embodiment, after completing scene segmentation, the edge processing unit performs video frame extraction for each independent scene. The purpose of extracting video frames is to select the most representative frames from the numerous video frames contained in the scene for subsequent recognition evaluation, thereby minimizing computational overhead while ensuring the effectiveness of the evaluation. The edge processing unit first calculates the proportion of the main body region in each video frame of the current scene. Specifically, for each frame in the scene, the edge processing unit counts the number of all regions marked as main bodies to be analyzed in that frame and divides it by the total number of regions to be analyzed in that frame to obtain the proportion of the main body region in that frame. Subsequently, the edge processing unit traverses all video frames in the scene and selects the video frame with the largest proportion of the main body region as the first candidate frame. This frame usually corresponds to the moment when the target object occupies the largest area in the scene and contains the richest subject information.

[0040] Meanwhile, the edge processing unit calculates the cumulative frequency of occurrence of each region to be analyzed as a main subject region within the scene. Specifically, for each region to be analyzed within the scene, the edge processing unit traverses all video frames of the scene, counts the number of frames marked as a main subject region at that location, and then divides this number by the total number of frames in the scene to obtain the cumulative frequency of occurrence at that location. This frequency reflects the density of main subject information carried by each spatial location within the scene in the temporal dimension. The edge processing unit sorts all regions to be analyzed from high to low according to their cumulative frequency of occurrence, selects several regions with the highest cumulative frequency of occurrence as high-frequency main subject regions, and then selects a video frame from the scene that can simultaneously cover the vast majority of these high-frequency main subject regions as the second candidate frame. In most cases, this frame is usually the one with the most dispersed distribution of main targets in the scene and the widest coverage, reflecting the overall spatial distribution characteristics of main subject activities within the scene.

[0041] After obtaining the first and second candidate frames, the edge processing unit combines both to determine the set of video frames for analysis of the current scene. Specifically, if the first and second candidate frames are the same frame, only that frame is selected as the analysis video frame; if they are different frames, both are used as analysis video frames. For scenes with long durations or significant changes in the distribution of subjects within the scene, the number of extracted frames can be increased proportionally to the total number of frames in the scene. However, typically, the number of analysis video frames extracted for each scene is between one and three to balance evaluation accuracy and computational efficiency.

[0042] This embodiment uses the dual-criteria extraction method based on the proportion of the main body region and the cumulative frequency of occurrence. The selected video frames for analysis not only include the most prominent key moments of the main body in the scene, but also take into account the overall spatial coverage of the main body's activities in the scene. This ensures that the subsequent recognition evaluation can effectively reflect the true recognition performance level of the scene after transcoding compression and super-resolution reconstruction.

[0043] Specifically, in step S4, the identifiability assessment includes: Based on the detection confidence or classification accuracy output by the downstream AI large model, the identifiable quantization value of each analyzed video frame is calculated, and the average of the quantization values ​​of each analyzed video frame is taken as the average identifiability of the current scene.

[0044] In this embodiment, after receiving the low-bitrate compressed video stream uploaded from the edge, the cloud first extracts the compressed frame data corresponding to each analysis video frame determined in step S3. Then, the cloud inputs these compressed frames into a preset super-resolution model for super-resolution reconstruction to obtain reconstructed high-resolution analysis frames. After reconstruction, the cloud inputs the reconstructed analysis frames into the target downstream AI large model. Taking a target detection large model as an example, the large model outputs several detection results for each frame. Each detection result includes the target category label, the coordinates of the detection bounding box, and the corresponding detection confidence score. For classification task large models, the large model outputs the classification probability distribution of the frame across each candidate category. The cloud extracts confidence information from the results output by the large model: for detection tasks, the average confidence score of the highest-confidence detection boxes among all detection results in the frame is taken as the identifiable quantification value of the frame; for classification tasks, the maximum classification probability is taken as the identifiable quantification value of the frame. For large models that simultaneously output detection boxes and classification results, the average detection confidence score can be used as the quantification basis.

[0045] After calculating the identifiable quantization value of each analyzed video frame in the current scene, the cloud adds up the quantization values ​​of all analyzed frames and divides them by the total number of analyzed frames to obtain the average identifiability of the scene. This average identifiability serves as a comprehensive evaluation metric to measure the degree to which the compressed video in the scene, after super-resolution reconstruction, retains the recognition performance of the large model under the current transcoding parameters. The cloud compares the calculated average identifiability with preset qualification conditions. The preset qualification conditions are determined in advance based on the actual application requirements of the downstream large model. For example, they can be set as an average detection confidence score not lower than a certain threshold, or the percentage of the identifiable quantization value relative to the baseline quantization value of the original uncompressed video on the large model not lower than a certain percentage. If the average identifiability meets the qualification conditions, the current transcoding parameters are deemed valid, and no correction is triggered; if the average identifiability does not meet the qualification conditions, the subsequent parameter correction process is triggered.

[0046] Specifically, in step S4, the correction strategy after triggering parameter correction includes: Determine the anomaly type based on the trend of identifiability changes in the current scene over time: If the trend of change shows abrupt fluctuations, it is determined that the scene segmentation is abnormal, and the first correction strategy is executed. The first correction strategy includes raising the switching rate threshold in the judgment criteria of the scene switching node. If the changing trend shows a gradual deterioration, it is determined that the compression capability is insufficient, and a second correction strategy is executed. The second correction strategy includes lowering the bitrate or resolution lower limit in the transcoding parameters.

[0047] In this embodiment, the cloud triggers a parameter correction process after determining that the average recognizability of the current scene does not meet the preset qualification conditions. The core of the parameter correction process is to first identify the type of anomaly causing the recognizability to fail, and then execute a targeted correction strategy based on the anomaly type, rather than blindly adjusting the transcoding parameters. The cloud continuously calculates and records the average recognizability of each independent scene at fixed time intervals. For the scene currently being processed, the cloud has accumulated average recognizability data for multiple time nodes from the start frame to the current moment. The cloud plots this sequence data as a time-recognition change curve for the scene and analyzes the trend of recognizability changes over time based on this curve.

[0048] The cloud-based system categorizes anomalies into two types based on their trends: abrupt fluctuations and gradual degradation. Abrupt fluctuations refer to a significant, sudden drop in identifiability between two consecutive assessments, with the drop exceeding the normal inter-frame fluctuation range. This drop may then partially recover or remain persistently low, exhibiting a volatile overall pattern. The physical root cause of this fluctuation is usually inaccurate identification of scene transition nodes. When the scene transition threshold is set too leniently, the edge processing unit may classify adjacent frames with significantly different content into the same scene. This results in extracted analysis video frames containing content from different scenes, failing to represent the true distribution of the main elements in the current scene, leading to abrupt changes in the assessment results. Based on this, the cloud-based system determines that the root cause of abrupt fluctuations is scene segmentation anomalies and executes the first correction strategy.

[0049] Gradual degradation refers to a monotonically decreasing trend in recognizability over multiple consecutive time periods. The decline is gradual rather than drastic, resembling a slow landslide. The physical root cause of this degradation is usually insufficient compression capability. When the bitrate or resolution lower limit in the initial transcoding parameters is set too low, the distortion introduced by compression gradually worsens over time. The super-resolution model cannot effectively recover the detailed information required for large-scale model detection, leading to a continuous and slow decline in recognizability. Based on this, the cloud determines that the root cause of gradual degradation is insufficient compression capability and implements a second correction strategy.

[0050] The first correction strategy is implemented by increasing the switching rate threshold in the scene switching node's judgment criteria. In step S1, the edge device uses a switching recognition method based on the regional switching change rate. When the regional switching change rate is greater than the preset switching rate threshold, scene switching is triggered. The first correction strategy increases this threshold by a certain amount from its current value, making subsequent scene switching judgments more stringent. Switching is only triggered when more areas to be analyzed simultaneously undergo subject-background transitions, thus avoiding misjudging adjacent frames with different content as the same scene due to an excessively low threshold. The magnitude of the threshold increase can be determined proportionally based on the current degree of distortion; the greater the distortion, the greater the increase. The second correction strategy is implemented by lowering the bitrate or resolution lower limit in the transcoding parameters. After the cloud sends the correction command to the edge device, the edge device appropriately increases the compressed bitrate level or raises the resolution lower limit during subsequent transcoding processes to retain more image detail information for the super-resolution model to recover. If the current transcoding parameters are already at the bitrate or resolution lower limit, they are further increased based on that lower limit. The downward adjustment can also be determined proportionally based on the slope of the decrease in recognizability.

[0051] This embodiment, through an anomaly type discrimination and hierarchical correction strategy based on changing trend patterns, can accurately pinpoint the root cause of the performance degradation of large models, avoiding ineffective correction measures due to misjudgment, and improving the efficiency and accuracy of parameter correction. Meanwhile, the binary discrimination method between abrupt fluctuations and gradual degradation is simple to implement, has low computational overhead, and is suitable for real-time execution in the cloud.

[0052] Specifically, the trend of change is quantified by the magnitude of the aberration: Based on multiple periodically determined average recognizability values, a time-average recognizability curve for the scene is plotted, and the slope at the current time node is calculated as the alienation magnitude. When the degree of alienation is greater than the preset alienation threshold, it is determined that the change trend exhibits abrupt fluctuations. When the degree of alienation is less than or equal to the preset alienation threshold, it is determined that the change trend exhibits a gradual deterioration.

[0053] In this embodiment, the cloud uses the anomaly magnitude as a quantitative indicator to numerically determine the temporal trend of identifiability observed in step S4, thereby achieving objective and accurate differentiation of anomaly types. In step S4, the cloud continuously calculates and records the average identifiability at each time point in the current scene according to a fixed time period. Based on this data, the cloud constructs a two-dimensional data sequence of time and identifiability for the scene, with time as the horizontal axis and average identifiability as the vertical axis. When step S4 determines that the current average identifiability does not meet the preset qualification conditions and triggers the correction process, the cloud immediately extracts data points from the most recent consecutive time points from the data sequence, performs linear fitting on these data points, and calculates the slope of the fitted line at the current time point. This slope value is used as the anomaly magnitude. The sign and magnitude of the slope reflect the direction and rate of change of identifiability in the recent time period; a negative slope indicates a decreasing trend in identifiability, and a larger absolute value of the slope indicates a faster rate of change.

[0054] After obtaining the magnitude of the distortion, the cloud compares the absolute value of the distortion magnitude with a preset distortion threshold and determines the type of change trend based on the comparison result. The specific determination logic is as follows: When the absolute value of the distortion magnitude is greater than the preset distortion threshold, it indicates that the identifiability has changed drastically in a short period of time, and the cloud determines that the change trend shows abrupt fluctuations. In this case, the identifiability drops sharply between adjacent evaluation periods, and the rate of change far exceeds the normal fluctuation range. The root cause is usually a misjudgment at the scene switching node, that is, the edge end mistakenly classifies adjacent frames with large content differences into the same scene, resulting in heterogeneous frames in the extracted analysis video frames that do not match the main distribution of the scene, thus causing a drastic jump in identifiability. When the absolute value of the distortion magnitude is less than or equal to the preset distortion threshold, it indicates that the change in identifiability is relatively gradual, and the cloud determines that the change trend shows a gradual deterioration. In this case, the recognizability decreases slightly over multiple consecutive periods, with a steady rather than drastic rate of change. The root cause is usually insufficient compression capability, i.e., the bitrate or resolution lower limit in the initial transcoding parameters is set too low, which causes compression distortion to gradually increase over time. The super-resolution model cannot effectively recover, causing the recognizability to continue to decline in a slow and gradual manner.

[0055] The preset anomaly threshold is determined in advance by collecting test video data containing known anomaly types, calculating the anomaly amplitude values ​​for abrupt fluctuations and gradual degradation in each video, statistically analyzing the range of anomaly amplitude values ​​corresponding to each type of anomaly, and setting the preset anomaly threshold to a value that can effectively distinguish between the two types of anomalies. In this embodiment, the preset anomaly threshold is set such that when the anomaly amplitude exceeds the threshold, it is determined to be an abrupt fluctuation, and when the anomaly amplitude is within the threshold range, it is determined to be a gradual degradation.

[0056] Specifically, the first correction strategy, after increasing the switching rate threshold in the determination criteria for the scene switching node, further includes: Statistical analysis of the actual scene switching frequency within a preset time period after the current scene switch; If the actual scene switching frequency is greater than the preset switching frequency threshold, then the boundary density threshold for dividing the main area and the background area will be further increased.

[0057] In this embodiment, after the cloud executes the first correction strategy to increase the switching rate threshold of the scene switching node, it is necessary to further verify whether the correction measure is sufficient to solve the scene segmentation anomaly problem. To this end, the cloud introduces scene switching frequency as an auxiliary judgment indicator. Specifically, after increasing the switching rate threshold, the cloud sends the updated switching rate threshold to the edge device. The edge device uses this updated threshold to perform scene switching judgment during subsequent video processing. Starting from the current scene switching moment, the cloud counts the actual number of scene switching events within a preset time period, and then divides this count by the length of the time period to obtain the actual scene switching frequency.

[0058] The cloud platform compares the actual scene switching frequency with a preset switching frequency threshold. The preset switching frequency threshold is determined in advance based on the characteristics of the video content and the application scenario. For surveillance videos, scene switching is usually infrequent, so the threshold is set to a lower value; for short videos or advertising videos, scene switching is more frequent, so the threshold is set to a higher value. In this embodiment, for a typical surveillance scenario, the preset switching frequency threshold is set to no more than a few switches per minute.

[0059] If the actual scene switching frequency exceeds the preset switching frequency threshold, it indicates that although the switching rate threshold has been increased, scene switching is still too frequent. The root cause of this situation is usually not the switching threshold itself, but insufficient precision in dividing the main body area and the background area. When the boundary density threshold is set too low, a large number of sparsely textured background areas are misjudged as main body areas, causing a large number of spurious changes to be mixed into the calculation of the subsequent area switching change rate. This makes it easy for the change rate based on the incorrect area marking to exceed the threshold and trigger frequent switching, even if the switching rate threshold has been increased.

[0060] In response to this situation, the cloud performs further correction measures: increasing the boundary density threshold when dividing the subject and background regions. The cloud sends this correction instruction to the edge processing unit, which uses the increased boundary density threshold to determine the subject and background in subsequent video frames. Increasing the boundary density threshold ensures that only regions with denser textures and more prominent edge features are identified as subject regions, thus excluding misjudged background regions from the subject statistics. This makes the region switching rate more accurately reflect the true changes in the image content, thereby making scene switching determination more reliable. If the actual scene switching frequency is less than or equal to the preset switching frequency threshold, it indicates that the scene segmentation anomaly has been effectively mitigated after increasing the switching rate threshold. No further adjustment of the boundary density threshold is needed, and the cloud ends the correction process, continuing to process subsequent videos with the current parameters.

[0061] Specifically, after executing the first correction strategy, if the average recognizability still does not meet the preset qualification condition, then switch to the second correction strategy to lower the bitrate or resolution lower limit in the transcoding parameters.

[0062] In this embodiment, after executing the first correction strategy, the cloud does not immediately determine the end of the correction process, but instead verifies the correction effect. Specifically, after the first correction strategy is executed and takes effect, the cloud re-extracts and analyzes the subsequent video frames of the current scene according to step S4, evaluates their average recognizability, and compares the average recognizability with the preset qualification conditions. If the re-evaluated average recognizability meets the preset qualification conditions, it indicates that the first correction strategy has effectively solved the scene segmentation anomaly problem, and the recognizability has been restored to the standard level. The cloud then ends the current parameter correction process and continues to process subsequent videos with the current parameters. If the re-evaluated average recognizability still does not meet the preset qualification conditions, it indicates that the scene segmentation anomaly is not the only reason for the substandard recognizability, or that the first correction strategy has failed to completely solve the problem. In this case, the cloud switches to the second correction strategy, performing the operation of lowering the bitrate or resolution lower limit in the transcoding parameters. The cloud sends the corresponding correction instructions to the edge device. During the subsequent transcoding process, the edge device appropriately increases the compressed bitrate or raises the lower limit of resolution to retain more image detail information for the super-resolution model to recover, thereby improving the recognizability of the large model.

[0063] Specifically, in the second correction strategy, lowering the lower limit of the bitrate or resolution in the transcoding parameters includes: Obtain the recoverability curve corresponding to the current scene, and identify the cliff contribution node in the recoverability curve where the detection accuracy recovery rate drops sharply with the decrease of the bit rate; The bitrate value corresponding to the cliff contributing node is used as the critical bitrate, and the target bitrate in the compression standard is adjusted to be no lower than the critical bitrate.

[0064] In this embodiment, when the cloud determines that a second correction strategy needs to be executed, the cloud first obtains the recoverability curve corresponding to the current scene from the local storage at the edge or the cloud cache. As mentioned earlier, this curve is pre-established in the offline stage using a standard test video set, recording the recovery rate of the detection accuracy of the downstream AI large model after super-resolution reconstruction of compressed videos at different bitrate levels. After obtaining the recoverability curve, the cloud performs a traversal analysis of the curve to identify the cliff contribution nodes. The so-called cliff contribution node refers to the critical point on the curve where the detection accuracy recovery rate drops sharply as the bitrate decreases. On the recoverability curve, there is usually a relatively flat area where moderately reducing the bitrate will not cause a significant decrease in the recovery rate; however, when the bitrate is reduced to below a certain critical value, the recovery rate will drop sharply as the bitrate decreases further. The curve point corresponding to this critical value is the cliff contribution node, and its physical meaning is that below this critical bitrate, the compression distortion has exceeded the upper limit of the effective recovery capability of the super-resolution model, and even after super-resolution reconstruction, the large model cannot extract enough semantic information to maintain the detection accuracy.

[0065] After identifying the contributing node to the precipitate, the cloud extracts the corresponding bitrate value as the critical bitrate. Subsequently, the cloud adjusts the target bitrate in the compression standard to a range not lower than this critical bitrate. Specifically, if the currently used target bitrate is already lower than the critical bitrate, it is increased to the critical bitrate or a value slightly higher than it. If the currently used target bitrate is not lower than the critical bitrate, but the average identifiability still does not meet the acceptable criteria, it can be slightly increased further above the critical bitrate until the identifiability meets the standard. The cloud sends the adjusted target bitrate value to the edge as a correction command. Upon receiving this command, the edge constrains the bitrate parameter to near the updated target bitrate during subsequent transcoding, ensuring that the compression distortion remains within the range that the super-resolution model can effectively recover, avoiding irreversible performance loss due to over-compression.

[0066] This embodiment transforms the temporal trend of identifiability into an objective numerical comparison through a quantitative judgment method based on the magnitude of anomalies. This enables the automatic and accurate identification of two types of anomalies: abrupt fluctuations and gradual degradation, providing a quantitative decision-making basis for subsequent implementation of targeted hierarchical correction strategies. The two-level progressive adjustment of the switching rate threshold and boundary density threshold in the first correction strategy forms a hierarchical optimization link from the switching judgment standard to the region division basis, accurately resolving scene segmentation anomalies. The progressive execution logic of verifying whether identifiability meets the standard after the first correction strategy is implemented, and then switching to the second correction strategy if it does not, achieves layer-by-layer investigation and correction from scene segmentation anomalies to insufficient compression capability, ensuring that the effect of each correction measure is traceable and verifiable. The bitrate reduction method based on the constraint of the recoverability curve interruption cliff contribution node constrains the target bitrate in the compression standard above the critical bitrate that the super-resolution model can effectively recover, avoiding the risk of over-compression caused by blind parameter tuning and maintaining a low transmission bandwidth to the maximum extent while ensuring the detection accuracy of the large model.

[0067] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

[0068] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention can have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for high-resolution video transcoding and super-resolution fusion processing for large AI models, characterized in that, include: Step S1: Identify scene switching nodes in the input video to segment it into several independent scenes, and obtain the division information of the main body region and background region and the target distribution information in the current scene video frame; Step S2: Based on the pre-established recoverability curve, combined with the partitioning information and target distribution information, select the initial transcoding parameters that meet the preset recovery rate threshold and have the minimum bandwidth usage to perform compression transcoding. Step S3: In each of the independent scenes, at least one video frame for analysis is extracted based on the distribution statistical characteristics of the main body region; Step S4: Evaluate the recognizability of the analyzed video frame after transcoding and super-resolution reconstruction, obtain the average recognizability of the current scene, and trigger parameter correction when the average recognizability does not meet the preset qualification conditions. Step S5 involves super-resolution reconstruction of the low-bitrate video stream after parameter correction and transcoding parameter processing, and inputting it into the downstream AI large model to perform video understanding tasks.

2. The method according to claim 1, characterized in that, The division between the main body region and the background region is achieved through region analysis based on boundary density, including: Divide the current video frame into several regions to be analyzed; Based on the grayscale edge detection algorithm, the cumulative edge length of each region to be analyzed is calculated, and the ratio of the cumulative edge length to the area of ​​the region to be analyzed is used as the boundary density of the region. The regions to be analyzed with a boundary density greater than a preset density threshold are identified as the main regions, and the regions to be analyzed with a boundary density less than or equal to the preset density threshold are identified as the background regions.

3. The method according to claim 1, characterized in that, The scene switching node is identified based on the area switching change rate, including: For two adjacent frames, the first number of times the main area changes to the background area in the region to be analyzed, and the second number of times the background area changes to the main area, are counted. The sum of the first number and the second number is taken as the region switching change rate. When the area switching change rate is greater than the preset switching rate threshold, it is determined that a scene switch has occurred between the two adjacent frames, and the first frame after the switch is taken as the starting frame of the new scene.

4. The method according to claim 1, characterized in that, In step S3, at least one video frame for analysis is extracted based on the distribution statistical characteristics of the main region, including: Calculate the proportion of the main body region in each video frame in the current scene, and select the video frame with the largest proportion of the main body region as the first candidate frame; The cumulative frequency of each region to be analyzed being identified as the main region in the scene is counted, and video frames covering high-frequency main regions are selected as second candidate frames based on the cumulative frequency of occurrence. By combining the first candidate frame and the second candidate frame, at least one video frame for analysis is determined.

5. The method according to claim 1, characterized in that, In step S4, the identifiability assessment includes: Based on the detection confidence or classification accuracy output by the downstream AI large model, the identifiable quantization value of each analyzed video frame is calculated, and the average of the quantization values ​​of each analyzed video frame is taken as the average identifiability of the current scene.

6. The method according to claim 5, characterized in that, In step S4, the correction strategy after triggering parameter correction includes: Determine the anomaly type based on the trend of identifiability changes in the current scene over time: If the trend of change shows abrupt fluctuations, it is determined that the scene segmentation is abnormal, and the first correction strategy is executed. The first correction strategy includes raising the switching rate threshold in the judgment criteria of the scene switching node. If the changing trend shows a gradual deterioration, it is determined that the compression capability is insufficient, and a second correction strategy is executed. The second correction strategy includes lowering the bitrate or resolution lower limit in the transcoding parameters.

7. The method according to claim 6, characterized in that, The trend of change is quantified by the magnitude of the aberration: Based on multiple periodically determined average recognizability values, a time-average recognizability curve for the scene is plotted, and the slope at the current time node is calculated as the alienation magnitude. When the degree of alienation is greater than the preset alienation threshold, it is determined that the change trend exhibits abrupt fluctuations. When the degree of alienation is less than or equal to the preset alienation threshold, it is determined that the change trend exhibits a gradual deterioration.

8. The method according to claim 6, characterized in that, The first correction strategy, after increasing the switching rate threshold in the determination criteria for the scene switching node, further includes: Statistical analysis of the actual scene switching frequency within a preset time period after the current scene switch; If the actual scene switching frequency is greater than the preset switching frequency threshold, then the boundary density threshold for dividing the main area and the background area will be further increased.

9. The method according to claim 8, characterized in that, If the average recognizability still does not meet the preset qualification condition after the first correction strategy is executed, the second correction strategy is switched to lower the bitrate or resolution lower limit in the transcoding parameters.

10. The method according to claim 9, characterized in that, In the second correction strategy, lowering the lower limit of the bitrate or resolution in the transcoding parameters includes: Obtain the recoverability curve corresponding to the current scene, and identify the cliff contribution node in the recoverability curve where the detection accuracy recovery rate drops sharply with the decrease of the bit rate; The bitrate value corresponding to the cliff contributing node is used as the critical bitrate, and the target bitrate in the compression standard is adjusted to be no lower than the critical bitrate.

Images