A method and system for audio and video signal transmission with independent and controllable architecture and hardware / software collaboration

Abstract

This application relates to the field of audio and video encoding / decoding and processing technology, providing a method and system for audio and video signal transmission with an independently controllable architecture and hardware / software collaboration, solving the problem of low bandwidth utilization. The method includes: synchronously collecting bandwidth fluctuations, packet loss rate, and transmission delay parameters of the mobile terminal link, generating a network status level through correlation analysis; performing resolution scaling on the luminance component of the video frame and dynamic downsampling on the chrominance component to generate a down-resolution frame adapted to the network status; combining the packet loss rate to adjust the quantization step size, frame group length, and inter-frame prediction mode to generate a fault-tolerant encoded stream; aligning the luminance and chrominance components through entropy decoding and motion compensation to form a decoded and reconstructed frame with parallel multi-scale feature extraction and detail compensation, generating an enhanced luminance map; independently reconstructing the chrominance decoding component; and outputting the original resolution video frame through color reconstruction and fusion. This application achieves low-latency transmission of high-definition video under network fluctuations, balancing bandwidth adaptability and image quality restoration accuracy.

Description

Technical Field

[0001] This application relates to the field of audio and video encoding and decoding technology, and in particular to an audio and video signal transmission method and system with an independently controllable architecture that integrates hardware and software. Background Technology

[0002] In mobile audio and video communication scenarios, such as remote medical consultations, emergency command systems, and mobile vehicle monitoring, network environments often face problems such as drastic bandwidth fluctuations, high packet loss rates, and significant transmission latency jitter. These scenarios require transmission systems to possess adaptive capabilities, high fault tolerance, and low processing latency to meet interactive needs.

[0003] Currently, the mainstream solution adopts the SFU architecture transmission technology with end-to-end state awareness. This solution collects network status through server-side collaboration with multiple clients. The sending end periodically reports the uplink network status, the receiving end reports the downlink status, and the server integrates the data and filters out the worst downlink status to feed back to the sending end.

[0004] However, while this solution can improve the adaptability of weak networks, it still has some core shortcomings. First, the granularity of state awareness is insufficient, relying only on the macro-level link state and failing to distinguish the spatiotemporal differences in audio and video data; second, the encoding strategy is disconnected from the content, using a uniform redundancy mechanism to process all video frames without considering the complexity of the frame content, resulting in low bandwidth utilization. Summary of the Invention

[0005] This application provides a method and system for audio and video signal transmission with an independent and controllable architecture and hardware-software collaboration, in order to solve the problem of low bandwidth utilization in the prior art.

[0006] Firstly, this application provides a method for transmitting audio and video signals using a self-controllable architecture with coordinated hardware and software, including:

[0007] During audio and video transmission, bandwidth fluctuation parameters, packet loss rate parameters, and transmission delay parameters of the mobile terminal link are collected synchronously. Based on the correlation analysis of the bandwidth fluctuation parameters and transmission delay parameters, a network status level is generated.

[0008] Based on the network status level, the original luminance component of the video frame is scaled up and the original chrominance component is selectively sampled using a frame adjustment device to generate a down-resolution frame that matches the current network status level.

[0009] Based on the total number of pixels and spatial complexity parameters of the down-resolution frame, and combined with the packet loss rate parameter, the quantization step size range, frame group length and inter-frame prediction mode are simultaneously adjusted to generate a fault-tolerant coding stream with a fused anti-packet loss structure.

[0010] At the receiving end, an entropy decoding operation is performed on the fault-tolerant coded stream, and motion compensation alignment is performed on the decoded luminance component and decoded chrominance component of the decoded and recovered down-resolution frame to obtain the decoded reconstructed frame.

[0011] Based on the neural network reconstruction algorithm, multi-scale feature extraction is performed on the luminance component of the decoded and reconstructed frame, and local detail features are superimposed for detail compensation to generate an enhanced luminance feature map. Chromaticity resolution reconstruction is performed on the decoded chrominance component to generate a chrominance reconstruction component. The enhanced luminance feature map and the chrominance reconstruction component are then synchronously reconstructed in color space to output a video frame restored to the original resolution.

[0012] Optionally, based on a neural network reconstruction algorithm, multi-scale feature extraction is performed on the luminance component of the decoded and reconstructed frame, and local detail features are superimposed for detail compensation to generate an enhanced luminance feature map. Chromaticity resolution reconstruction is then performed on the decoded chroma component to generate a chroma reconstruction component. Finally, the enhanced luminance feature map and the chroma reconstruction component are simultaneously reconstructed in color space to output a video frame restored to its original resolution.

[0013] The luminance components of the decoded and reconstructed frame are subjected to multi-level convolution operations based on a neural network reconstruction algorithm to extract luminance features at different scales.

[0014] From the brightness features at different scales, the finest brightness features are selected and superimposed and fused with local detail features to perform detail compensation on the brightness components of the decoded and reconstructed frame, generating an enhanced brightness feature map.

[0015] Bilinear interpolation reconstruction is performed on the chroma components of the decoded and reconstructed frame to generate chroma reconstructed components that match the original resolution;

[0016] The enhanced brightness feature map and the chromaticity reconstruction component are synchronously superimposed according to a preset color space conversion rule to restore the original resolution while ensuring color consistency, and output the restored video frame.

[0017] Optionally, from the brightness features at different scales, the finest-scale brightness features are selected and superimposed and fused with local detail features to perform detail compensation on the brightness components of the decoded and reconstructed frame, generating an enhanced brightness feature map, including:

[0018] From the brightness features of different scales, the finest brightness feature is selected, and the spatial coordinates corresponding to the finest brightness feature are obtained. A pixel neighborhood of a fixed size is delineated with each of the spatial coordinates as the center point.

[0019] The feature values ​​of all locations within the pixel neighborhood are weighted and calculated to generate local detail features containing all locations, wherein the weight coefficient of the spatial location coordinates is higher than that of the edge locations;

[0020] The brightness features at the finest scale and the local detail features are added point by point according to the same spatial coordinates to obtain the superimposed and fused result after detail compensation. A nonlinear activation operation is performed on the superimposed and fused result to generate an enhanced brightness feature map.

[0021] Optionally, based on the total number of pixels and spatial complexity parameters of the down-resolution frame, and in conjunction with the packet loss rate parameters, the quantization step size range, frame group length, and inter-frame prediction mode are simultaneously adjusted to generate a fault-tolerant coded stream with a fused packet loss-resistant structure, including:

[0022] The data volume per unit frame is calculated based on the total number of pixels in the down-resolution frame, and the data volume per unit frame is combined with the space complexity parameter to generate a frame complexity weight.

[0023] The upper limit of the quantization step size range is increased by multiplying the frame complexity weight and the packet loss rate parameter. The maximum value of the frame group length is mapped inversely according to the packet loss rate parameter, and the maximum value is used as the upper limit threshold of the reference frame number for limiting the inter-frame prediction mode.

[0024] During the encoding process, the adjusted quantization step size range, frame group length, and inter-frame prediction mode parameters are injected into the encoder to generate a fault-tolerant coding stream.

[0025] Optionally, based on the network state level, resolution scaling is performed on the original luminance component of the video frame using a frame adjustment device, and selective sampling is performed on the original chrominance component to generate a down-resolution frame matching the current network state level, including:

[0026] According to the network status level, a preset resolution scaling ratio table is queried to determine the target scaling ratio of the luminance component. The original luminance component is uniformly sampled according to the target scaling ratio to generate the scaled luminance component.

[0027] Non-uniform region division is performed on the original chromaticity components. Based on the spatial gradient distribution of the original chromaticity components, sparse sampling is performed on regions with spatial gradients below a threshold to obtain sampled chromaticity components.

[0028] The scaled luminance component is combined with the sampled chrominance component to generate a down-resolution frame that matches the current network state level.

[0029] Optionally, at the receiving end, entropy decoding is performed on the fault-tolerant coded stream, and motion compensation alignment is performed on the decoded luminance and chrominance components of the decoded and recovered down-resolution frame to obtain a decoded reconstructed frame, including:

[0030] The fault-tolerant encoded stream is subjected to syntax element type determination to determine the category of the element to be decoded and its corresponding context model index. Based on the current state value of the context model index, the corresponding probability interval is calculated by the arithmetic decoding engine.

[0031] The value at the current parsing position in the fault-tolerant coding stream is compared with the boundary value of the probability interval to parse out the binary character sequence. The binary character sequence is then debinarized to restore the decoded luminance component and decoded chrominance component of the down-resolution frame.

[0032] Extract the motion vector of the decoded luminance component, calculate the spatial coordinate offset of the decoded chrominance component based on the motion vector, adjust the spatial position of the decoded chrominance component using the spatial coordinate offset, and obtain the offset-compensated decoded chrominance component.

[0033] The time-domain motion trajectory matching is performed on the offset-compensated decoded chroma component and the decoded luminance component. The matched decoded luminance component and decoded chroma component are then merged to obtain the reconstructed decoded frame.

[0034] Optionally, during audio and video transmission, bandwidth fluctuation parameters, packet loss rate parameters, and transmission delay parameters of the mobile terminal link are collected synchronously. Based on the correlation analysis of the bandwidth fluctuation parameters and transmission delay parameters, a network status level is generated, including:

[0035] Within a fixed time window, the bandwidth fluctuation parameters, packet loss rate parameters, and transmission delay parameters of the mobile terminal link are periodically acquired, and the change in the bandwidth fluctuation parameters within a continuous time window is calculated.

[0036] The transmission delay parameter is divided into multiple level intervals, and each level interval is associated with the amount of change of the bandwidth fluctuation parameter. Based on the distribution of the amount of change in the corresponding level interval, the bandwidth stability score corresponding to each level interval is determined.

[0037] Based on a preset scoring threshold mapping table, the bandwidth stability score corresponding to each level range is converted into a discrete network status level.

[0038] Secondly, this application provides an audio and video signal transmission system with an independently controllable architecture and hardware-software collaboration, comprising:

[0039] The analysis module is used to simultaneously collect bandwidth fluctuation parameters, packet loss rate parameters, and transmission delay parameters of the mobile terminal link during audio and video transmission. Based on the correlation analysis of the bandwidth fluctuation parameters and transmission delay parameters, a network status level is generated.

[0040] The sampling module is used to perform resolution scaling on the original luminance component of the video frame and selective sampling on the original chrominance component according to the network status level, through the frame adjustment device, to generate a down-resolution frame that matches the current network status level.

[0041] The generation module is used to generate a fault-tolerant coded stream with a fused anti-packet-loss structure by simultaneously adjusting the quantization step size range, frame group length, and inter-frame prediction mode based on the total number of pixels and spatial complexity parameters of the down-resolution frame and the packet loss rate parameters.

[0042] The reconstruction module is used to perform entropy decoding on the fault-tolerant coded stream at the receiving end, and to perform motion compensation alignment on the decoded luminance component and decoded chrominance component of the decoded and recovered down-resolution frame to obtain the decoded reconstructed frame.

[0043] The restoration module is used to perform multi-scale feature extraction on the luminance component of the decoded and reconstructed frame based on a neural network reconstruction algorithm, and to perform detail compensation by superimposing local detail features to generate an enhanced luminance feature map. It then performs chroma resolution reconstruction on the decoded chroma component to generate a chroma reconstruction component, and performs color space synchronous reconstruction on the enhanced luminance feature map and the chroma reconstruction component to output a video frame restored to the original resolution.

[0044] Thirdly, this application provides a computing device, including a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are to be invoked and executed by the processing component to implement an audio and video signal transmission method with autonomous and controllable architecture and hardware-software collaboration as described in the first aspect above.

[0045] Fourthly, this application provides a computer storage medium storing a computer program, which, when executed by a computer, implements an audio and video signal transmission method with an autonomous and controllable architecture and hardware-software collaboration as described in the first aspect.

[0046] The beneficial effects of this application are:

[0047] This application collects bandwidth fluctuation, packet loss rate, and transmission delay parameters and generates network status levels through correlation analysis, achieving dynamic perception and quantitative classification of network status to provide a decision-making basis for adaptive transmission. Based on the network status level, it performs resolution scaling on the luminance component of video frames and selective sampling on the chrominance component to generate down-resolution frames, differentially reducing data load, maintaining the sharpness of the main subject outline under weak network conditions, and improving bandwidth utilization. Based on the total number of pixels, spatial complexity, and packet loss rate of the down-resolution frames, it synchronously adjusts the quantization step size, frame group length, and inter-frame prediction mode, dynamically injecting anti-packet loss structures to improve... The receiver performs entropy decoding on the fault-tolerant encoded stream and aligns the luminance / chrominance components with motion compensation to eliminate displacement distortion during transmission and ensure audio-visual synchronization. A neural network is used to extract multi-scale features from the luminance component of the decoded and reconstructed frame, and local detail compensation is superimposed to generate an enhanced luminance feature map, restoring high-frequency edges and texture details and resolving blurring caused by resolution reduction. Resolution reconstruction is performed on the chrominance component, and color space is synchronously reconstructed with the luminance feature map to achieve high-fidelity restoration of the chrominance component, outputting video frames with the same resolution as the original.

[0048] Furthermore, multi-level convolutional operations are performed on the luminance component of the decoded and reconstructed frame to extract multi-scale features, capturing luminance information from global structure to local micro-texture, providing a multi-granular feature foundation for detail compensation; the finest-scale luminance features are fused with local detail features for superposition compensation, enhancing high-frequency details such as edge sharpness and suppressing compression artifacts, thereby improving subjective image quality clarity; bilinear interpolation is performed on the decoded chrominance component to reconstruct high-resolution chrominance data, restoring chrominance resolution with low computational complexity and avoiding end-side latency caused by complex models; the enhanced luminance feature map and chrominance reconstruction component are synchronously superimposed according to preset color space conversion rules to ensure color consistency and alignment accuracy of the luminance and chrominance channels and eliminate color distortion.

[0049] These or other aspects of this application will become more apparent in the following description of the embodiments. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 A flowchart of an audio and video signal transmission method with autonomous and controllable architecture and hardware-software collaboration provided in this application is shown;

[0052] Figure 2The illustration shows a scenario diagram of an audio and video signal transmission method with autonomous and controllable architecture and hardware-software collaboration provided in this application.

[0053] Figure 3 This paper presents a schematic diagram of the structure of an audio and video signal transmission system with an autonomous and controllable architecture that integrates hardware and software, as provided in this application.

[0054] Figure 4 A schematic diagram of the structure of a computing device provided in this application is shown. Detailed Implementation

[0055] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0056] In some of the processes described in the specification, claims, and accompanying drawings of this application, multiple operations appearing in a specific order are included. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or may be executed in parallel. The operation numbers, such as 101, 102, etc., are merely used to distinguish different operations and do not themselves represent any execution order. Furthermore, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first," "second," etc., in this document are used to distinguish different messages, devices, modules, etc., and do not represent a chronological order, nor do they limit "first" and "second" to different types.

[0057] Research has revealed significant shortcomings in existing audio and video transmission technologies in mobile network environments. Traditional methods typically employ fixed encoding rules and uniform bandwidth allocation, making it difficult to address dynamic changes in network conditions, such as bandwidth fluctuations, data packet loss, and transmission latency. This rigid approach leads to three core flaws: First, resource allocation lacks flexibility, forcing the transmission of high-resolution images when network conditions deteriorate, resulting in stuttering or wasted bandwidth, while failing to improve image quality when the network is good. Second, it fails to differentiate the characteristics of different elements in the video, applying the same compression strategy to critical details and color information, leading to distortion in dynamic images. Third, fault tolerance mechanisms are inefficient, relying on static redundant data rather than dynamic adjustment strategies, increasing unnecessary data volume and failing to protect the integrity of critical images. These problems are particularly pronounced in weak network scenarios such as mobile live streaming and video conferencing, urgently requiring a new transmission method that can intelligently sense network conditions and dynamically adapt to content characteristics.

[0058] To address the aforementioned issues, this invention proposes a content-aware driven dynamic collaborative optimization method. Its core lies in the coordinated regulation of three factors: network state, content complexity, and fault-tolerant coding. Specifically, the process first generates multi-level network state labels to achieve refined perception by synchronously monitoring the correlation between bandwidth fluctuations, packet loss rate, and transmission latency. Then, video frames are adaptively processed according to the network state level, dynamically scaling the resolution of luminance data while implementing a key region priority sampling strategy for color data, preserving core color information while reducing data volume. Further, based on the processed image spatial complexity and real-time packet loss rate, three coding parameters are dynamically bound: adjusting compression fineness to protect details, shortening keyframe intervals to reduce error propagation, and selecting a prediction mode based on image motion characteristics, thereby generating a coded stream that balances low redundancy and high fault tolerance. Finally, at the receiving end, a neural network performs multi-layer feature extraction and detail compensation for luminance and reconstructs chrominance resolution, ultimately fusing luminance and color information to output a complete image. This method forms a complete closed loop of perception, decision-making, and reconstruction, significantly improving transmission efficiency in weak networks, resolving the contradiction between resource idleness and image distortion, and is plug-and-play on general-purpose hardware platforms without relying on dedicated codecs. Its innovative breakthrough lies in upgrading the traditional link adaptation mechanism to a content adaptation mechanism, which completely solves the key problems of crude state awareness and the disconnect between coding strategy and content.

[0059] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0060] Figure 1 This application provides a flowchart of an audio / video signal transmission method based on a self-controllable architecture with hardware and software collaboration, as shown in the following embodiments. Figure 1 As shown, the method includes:

[0061] 101. During audio and video transmission, the bandwidth fluctuation parameters, packet loss rate parameters, and transmission delay parameters of the mobile terminal link are collected synchronously. Based on the correlation analysis of the bandwidth fluctuation parameters and transmission delay parameters, a network status level is generated.

[0062] Optionally, step 101 may specifically include the following steps:

[0063] 1011. Within a fixed time window, periodically acquire the bandwidth fluctuation parameters, packet loss rate parameters, and transmission delay parameters of the mobile terminal link, and calculate the change in the bandwidth fluctuation parameters within a continuous time window;

[0064] 1012. Divide the transmission delay parameter into multiple level intervals, and associate each level interval with the change in the bandwidth fluctuation parameter. Based on the distribution of the change in the corresponding level interval, determine the bandwidth stability score corresponding to each level interval.

[0065] 1013. Based on a preset scoring threshold mapping table, the bandwidth stability score corresponding to each level interval is converted into a discrete network status level.

[0066] In the above steps, the bandwidth fluctuation parameter refers to the degree of bandwidth variation of the mobile link during audio and video transmission, measured in bits per second; the packet loss rate parameter represents the proportion of data packets lost during data transmission, given as a percentage; the transmission delay parameter describes the time delay from the sender to the receiver, measured in milliseconds; the network status level represents network quality as discrete levels; the change amount is calculated by continuously comparing the values ​​of the bandwidth fluctuation parameter over different time periods; the level interval divides the transmission delay parameter into multiple ranges to evaluate different levels of network performance; the distribution state is used to observe the dispersion of the change amount within each level interval; the bandwidth stability score is a numerical score calculated based on the distribution state to represent network stability; and the preset score threshold mapping table is a predefined table used to map the score to specific discrete network status levels.

[0067] In this embodiment, firstly, network parameter acquisition and change calculation are performed through step 1011. This allows the system to periodically acquire bandwidth fluctuation parameters, packet loss rate parameters, and transmission delay parameters of the mobile terminal link within a fixed time window. The bandwidth fluctuation parameter is calculated using the difference between adjacent time windows: First, the bandwidth value at time T is recorded. After a fixed time interval Record the new bandwidth value later. Then, the bandwidth change formula is used for calculation. The bandwidth fluctuation change is obtained, where This represents the change in bandwidth, measured in megabits per second (Mbps). Simultaneously, the system records the transmission delay and packet loss rate parameters at that moment, forming a dataset containing three data points: bandwidth change, transmission delay, and packet loss rate. For example, in a flight simulator intercom system monitoring scenario, the system collects the audio transmission link bandwidth every second, recording the initial data... =500Mbps, recorded after 1 second =520Mbps, calculate bandwidth fluctuation change. =1520−500=20Mbps. At this time, the synchronous recording transmission delay parameter is 150 milliseconds and the packet loss rate parameter is 0.3%.

[0068] Secondly, delay grading and stability scoring are calculated in step 1012. The transmission delay parameter is first divided into four grade ranges: 0 to 100 milliseconds is excellent, 101 to 200 milliseconds is good, 201 to 300 milliseconds is average, and over 300 milliseconds is poor. This grading standard is based on the flight simulation aviation communication standard. Then, the current transmission delay parameter value is categorized into the corresponding range; for example, a 150-millisecond delay is categorized into the good range. Next, the most recent N bandwidth fluctuation change samples are obtained within the corresponding delay range, and the arithmetic mean of these samples is calculated. ,in Here are the sample values, and N is the sample size; calculate the standard deviation. ,in, This indicates the degree of data dispersion. Finally, a stability scoring formula is used. Calculate the bandwidth stability score.

[0069] For example, when monitoring the flight intercom system, a latency of 150 milliseconds falls into the "good" range, and 10 bandwidth variation samples are taken. Calculate the mean Standard deviation Substituting into the scoring formula, we get (Keep two decimal places).

[0070] Finally, network status level mapping is implemented through step 1013. A predefined scoring threshold mapping table is used, specifying that 9.0 to 10.0 points correspond to excellent network status, 7.0 to 8.9 points to good, 5.0 to 6.9 points to medium, and below 5.0 points to poor. After obtaining the bandwidth stability score calculated in step 1012, the system queries the corresponding discrete network status level in the mapping table based on the numerical range of the score value. In particular, when monitoring critical flight alarm systems, if the score falls into an abnormal range, a log recording mechanism is triggered. For example, in the engine alarm sound transmission scenario, if the input bandwidth stability score is 9.74 points, the mapping table determines that it is in the 9.0 to 10.0 range, and the output network status level is excellent; when a score of 8.5 points is calculated, the mapping table determines it to be good, and the system automatically generates an alarm log to record the network status degradation event.

[0071] In practical applications, during the network performance monitoring of a flight simulator's audio system, the system periodically collects the latency parameters of the audio transmission channel, the time from audio signal generation to playback, bandwidth fluctuation parameters, and the amount of data transmitted per second every 2 seconds. For example, in the first round of collection, the latency value is 25 milliseconds, and in the second round, the latency value becomes 40 milliseconds. Therefore, the bandwidth fluctuation change is calculated to be 40-25=15 milliseconds. Next, the system divides the transmission latency parameters into three levels: 0 to 20 milliseconds is the excellent level (Level 1), 21 to 50 milliseconds is the good level (Level 2), and above 51 milliseconds is the level requiring optimization (Level 3). The current latency value of 40 milliseconds falls within the good level range. The system analyzes the distribution of historical bandwidth fluctuation changes within this level. If historical data shows that most changes are concentrated in the 5 to 15 millisecond range (e.g., 7 out of 10 similar data sets fall within this range), the bandwidth stability score is determined to be 7 points, with a maximum score of 10 indicating higher stability. Conversely, if the changes frequently exceed 20 milliseconds, the score is 5 points. Finally, based on a preset scoring threshold mapping table, for example, a score of 8 to 10 corresponds to a stable network status level, 6 to 7 corresponds to slight fluctuations, and below 5 corresponds to the need for optimization, the current instance's score of 7 is converted to a slight fluctuation level, thereby triggering the system to automatically increase the allocation of audio buffer resources to ensure the real-time synchronization effect of the cockpit ambient sound.

[0072] In the overall scheme of step 101 above, bandwidth fluctuation parameters, packet loss rate parameters, and transmission delay parameters of the mobile terminal link are collected synchronously. Based on the correlation analysis between bandwidth fluctuation parameters and transmission delay parameters, discrete network status levels are automatically generated. The technical implementation includes calculating the change in bandwidth fluctuation parameters within a periodic time window, dividing the transmission delay parameter level intervals and associating them with the bandwidth change distribution to evaluate bandwidth stability scores. Then, the scores are converted into network status levels according to a preset mapping table, thereby achieving efficient identification of network health status and dynamically optimizing transmission strategies to improve audio and video transmission quality and stability.

[0073] 102. Based on the network status level, the original luminance component of the video frame is scaled up and the original chrominance component is selectively sampled using a frame adjustment device to generate a down-resolution frame that matches the current network status level.

[0074] Optionally, step 102 may specifically include the following steps:

[0075] 1021. According to the network status level, query the preset resolution scaling ratio table, determine the target scaling ratio of the luminance component, perform uniform sampling on the original luminance component according to the target scaling ratio, and generate the scaled luminance component.

[0076] 1022. Perform non-uniform region division on the original chromaticity components. Based on the spatial gradient distribution of the original chromaticity components, select regions with spatial gradients below the threshold for sparse sampling to obtain sampled chromaticity components.

[0077] 1023. Combine the scaled luminance component with the sampled chrominance component to generate a down-resolution frame that matches the current network state level.

[0078] In the above steps, the frame adjustment device is a hardware or software component used to dynamically adjust the size of video frames. A video frame refers to the complete representation of a single image in a video sequence. The original luminance component is the original data part of the video frame describing pixel luminance information, and the original chrominance component is the original data part of the video frame describing pixel color information. The resolution scaling table is a preset mapping table used to convert network status levels into specific scaling ratio values. The target scaling ratio is the specific scaling ratio value of the luminance component obtained from the resolution scaling ratio table. Uniform sampling processes the luminance component by extracting pixel values ​​at fixed intervals. The scaled luminance component is the luminance data result after uniform sampling scaling. Non-uniform region division... This method divides an image into blocks of varying sizes to identify features in different regions. Spatial gradient distribution refers to the pattern of change in the rate of change of pixel values ​​in the image in the spatial direction. Spatial gradient is a quantity that calculates the intensity of change of pixels in the horizontal or vertical direction. Threshold is a preset numerical limit used to determine whether to perform a sampling operation. Sparse sampling is a method of selecting pixels with a lower density to process the chroma component. The sampled chroma component is the chroma data result obtained through sparse sampling. Combination is the process of merging the scaled luminance component and the sampled chroma component into a complete frame. Down-resolution frame is the final output video frame with reduced resolution. The current network status level is a discrete evaluation level of network quality obtained from the previous steps, such as excellent or good.

[0079] In this embodiment, firstly, adaptive scaling of the luminance component is achieved through step 1021. A preset resolution scaling table is consulted based on the network status level to determine the target scaling ratio for the luminance component. The scaling ratio table is set according to the following principles: excellent network status scaling ratio is 100% (maintaining the original resolution), good level scaling is 80%, medium level scaling is 60%, and poor level scaling is 40%. After obtaining the target ratio, a uniform sampling operation is performed on the original luminance component, that is, pixel values ​​are extracted at fixed intervals to generate the scaled luminance component. For example, in the video stream processing of a flight simulator instrument panel, when the network status level is good, the scaling ratio of 80% is determined by consulting the table. The original 1080P luminance component (1920×1080 pixels) is uniformly sampled by 4 pixels every 5 pixels in the row and column directions to generate a scaled luminance component of 864×576 pixels.

[0080] Secondly, intelligent sampling of the chroma components is achieved through step 1022. First, the original chroma components are divided into non-uniform regions, dividing the image into 16×16 pixel blocks; then, the spatial gradient value of each block is calculated. :

[0081] ;

[0082] Wherein, the spatial gradient value G represents the mean gradient value used to measure the intensity of color change within an image block; N represents the total number of pixels within the block; It is the chromaticity component The gradient in the horizontal direction reflects the amount of change in color level; It is the chromaticity component The gradient in the vertical direction reflects the amount of vertical change in color; It is the chromaticity component Gradient in the horizontal direction; It is the chromaticity component The gradient in the vertical direction. and For chromaticity components, and This represents the horizontal and vertical gradients; finally, smooth regions with gradient values ​​below a threshold T (e.g., T=15) are selected for sparse sampling, with the sampling density increased by 10% for every 5 decrease in gradient value, generating sampled chromaticity components. For example, when processing flight terrain display videos, the gradient value of the sky region... G =8 is below the threshold of 15, sampled at 30% density; gradient value of the cockpit instrument area. G =35 is higher than the threshold, so 100% sampling is maintained, and the final amount of chroma data is reduced.

[0083] Finally, step 1023 completes the down-resolution frame synthesis. The scaled luminance component generated in step 1021 and the sampled chrominance component generated in step 1022 are pixel-aligned and recombined. The luminance component retains complete spatial information, while the chrominance component uses bilinear interpolation based on the sampling position index to fill unsampled areas. Ultimately, a down-resolution frame matching the network state level is generated. For example, in a good network state, the 864×576 pixel luminance component and the sparsely sampled chrominance component are combined using the interpolation formula:

[0084] ;

[0085] in, This represents the chromaticity value reconstructed from the target interpolation point, i.e., the color data of the unsampled location that needs to be calculated in the final calculation. The chromaticity values ​​of neighboring sampling points. Distance weights This represents the reciprocal of the Euclidean distance, used to calculate the weights. The closer the distance, the larger the reciprocal, and the higher the weight. Complete chroma information is reconstructed, and a down-resolution frame of 960×540 is output, reducing the data volume to 45% of the original video, meeting aviation QTG standard requirements.

[0086] In practical applications, in a video transmission optimization scenario for an aviation training simulator, when the network status level is assessed as "moderate fluctuation," for example, level 2 as output in step 1013, the system initiates image scaling processing: First, based on the current level, it queries the preset resolution scaling table to determine that the luminance component needs to be reduced to 1 / 3 of its original size. If the original resolution is 1920×1080, the target is 640×360. The system performs uniform sampling on the original luminance component—for example, taking one sample every three pixels horizontally and one sample every three pixels vertically to generate the scaled luminance component. Next, it processes the chrominance component: the image is divided into 5×5 non-uniform regions, and the spatial gradient of the color change intensity within each region is calculated. If the change value in a certain region is detected to be lower than a set threshold, such as a gradient value ≤ 8 in the sky background region, then only one sparse sample is taken for every four pixels in that region; while in the rapidly changing dashboard region, the gradient value > 8, the original sampling density is maintained. Finally, the obtained sampled chrominance component retains the color details of key areas. Finally, the 640×360 luminance component is combined with the selectively sampled chrominance component to generate a down-resolution frame. This frame maintains the natural gradation of the sky area while ensuring clear edges on the dashboard text, reducing the overall data volume, and adapting to the transmission needs under current network fluctuation conditions.

[0087] In the overall scheme of step 102 above, based on the dynamically evaluated network status level, the resolution of the video content is adaptively adjusted using a frame adjustment device. Specifically, uniform resolution scaling based on a preset ratio table is performed on the original luminance component of the video frame, while spatial gradient-driven intelligent sampling is performed on the original chrominance component. By identifying low-gradient texture regions and performing selective sparse sampling, the scaled luminance component and the optimized sampled chrominance component are finally recombined to generate a down-resolution frame that accurately matches the real-time network status. This scheme significantly reduces the amount of data transmitted in high-latency or high-packet-loss network environments while ensuring that core visual information is not lost, achieving a dynamic balance between transmission quality and bandwidth usage.

[0088] 103. Based on the total number of pixels and spatial complexity parameters of the down-resolution frame, and combined with the packet loss rate parameters, the quantization step size range, frame group length and inter-frame prediction mode are simultaneously adjusted to generate a fault-tolerant coding stream with a fused anti-packet loss structure.

[0089] Optionally, step 103 may specifically include the following steps:

[0090] 1031. Calculate the data volume per unit frame based on the total number of pixels in the down-resolution frame, and combine the data volume per unit frame with the space complexity parameter to generate a frame complexity weight;

[0091] 1032. Increase the upper limit of the quantization step size range based on the product of the frame complexity weight and the packet loss rate parameter, reverse map the maximum value of the frame group length according to the packet loss rate parameter, and use the maximum value as the upper limit threshold of the reference frame number for limiting the inter-frame prediction mode.

[0092] 1033. During the encoding process, the adjusted quantization step size range, frame group length, and inter-frame prediction mode parameters are injected into the encoder to generate a fault-tolerant coding stream.

[0093] In the above steps, a down-resolution frame refers to a video frame whose resolution is reduced after processing in step 102; the total number of pixels represents the total number of pixels in the frame; the spatial complexity parameter is a numerical indicator describing the complexity of image texture details, calculated through edge density or gradient changes; the packet loss rate parameter refers to the proportion of data packets lost during data transmission; the quantization step size range refers to the allowable range of quantization precision changes during encoding and compression, with a larger quantization step size resulting in a higher compression ratio; the frame group length is the number of consecutive frames grouped together for encoding; the inter-frame prediction mode refers to an encoding method that utilizes the similarity between consecutive frames for compression; and the frame complexity weight is the compression difficulty calculated using the total number of pixels and the spatial complexity parameter. Degree coefficient; Unit frame data size refers to the original byte size of a single frame when uncompressed; Upper limit value indicates the maximum allowed numerical boundary of the parameter; Reference frame number upper limit threshold limits the maximum number of historical frames that can be referenced during inter-frame prediction; Fault-tolerant coded stream is a compressed video data stream with the ability to resist data loss; Fusion anti-packet loss structure refers to the anti-packet loss mechanism formed by synchronously adjusting the coding parameters; Synchronous adjustment means simultaneously adjusting three parameters: quantization step size range, frame group length, and inter-frame prediction mode; Product increase refers to the operation of expanding the parameter range through multiplication; Inverse mapping refers to the method of deriving parameter values ​​inversely proportional to the packet loss rate; Injecting into the encoder refers to inputting the adjusted parameters into the video compression system.

[0094] In this embodiment of the application, firstly, the total number of pixels of the down-resolution frame is obtained through step 1031. This value is calculated by multiplying the resolution width by the height. For example, in a flight simulator terrain display video, a down-resolution frame has a resolution of 960 pixels wide by 540 pixels high, so the total number of pixels P = 960 × 540 = 518,400 pixels. Then, assuming a pixel bit depth of 8 bits, the value is calculated using the formula... Calculate the data size per unit frame, where Indicates the size of the raw data in bytes for a single frame. The pixel bit depth is in bits; substituting the parameters yields... =8518,400 × 8 = 518,400 bytes. Next, extract the space complexity parameter. S This parameter is calculated using image processing algorithms such as the Sobel edge detector to determine the image edge density. For example, if the measured edge ratio of a terrain image is 35%, then... S =0.35. Finally, the frame complexity weight coefficient is generated. Substituting into W =518400 × 0.35 = 181440, this weight is used to quantify the compression difficulty coefficient of video frames. The entire process, from pixel statistics to weight output, is sequentially connected, providing basic data for subsequent parameter adjustment.

[0095] Secondly, step 1032 first uses frame complexity weights. W and packet loss rate parameters L Calculate the upper limit of the quantization step size range using the formula. ,in, This indicates that the default value for the basic quantization step size is 30. k The adjustment coefficient is fixed at 0.02 (used to standardize the frame complexity weight coefficient). W It is the frame complexity weight. L This is a parameter for packet loss rate; for example, in the video processing of a flight simulator dashboard, the input... W =181440 and L =0.005 (i.e., a packet loss rate of 0.5%), substituting into the formula, we get... =30+0.02×181440×0.005=48, which means that the upper limit of the quantization step size is extended to 48 to improve compression efficiency.

[0096] Next, based on the packet loss rate parameter L The maximum value of the reverse-mapped frame group length is obtained using the formula. The symbol ⌊⋅⌋ represents the floor function. L This represents the packet loss rate. For example, when the same packet loss rate is 0.005, the calculated result is... =⌊10×(1−0.005)⌋=⌊9.95⌋=9, this value indicates that the maximum allowed frame group length is 9 frames to reduce the amount of data per packet. Then, this maximum value is directly used as the upper limit threshold for the number of reference frames in the inter-frame prediction mode, that is... To reduce the risk of error propagation, predictions are limited to referencing only the most recent 9 frames. The entire process sequentially performs quantization step size adjustment, frame group length mapping, and reference frame threshold setting, with output parameters such as a maximum quantization step size of 48, a maximum frame group length of 9 frames, and a maximum reference frame of 9 frames, forming a coordinated control mechanism.

[0097] Finally, a fault-tolerant coded stream is generated through step 1033. Specifically, the parameters adjusted in step 1032, including the quantization step size range (e.g., minimum 4 to maximum 48), the frame group length limit of 9 frames, and the inter-frame prediction reference frame limit of 9 frames, are input into a standard video encoder such as H.265. When the encoder performs compression, the quantization step size limit is increased to 48 to improve compression efficiency, the frame group length is limited to 9 frames to reduce the size of a single packet, and inter-frame prediction only references the most recent 9 frames to reduce the risk of error propagation, thus incorporating a packet loss-resistant structure. The final output is a fault-tolerant coded stream. For example, in a flight terrain video coding scenario, the original video fails to decode at a packet loss rate of 0.8%, but after this parameter adjustment, the compression rate increases by 35%, and the system can still maintain smooth decoding even with a packet loss rate of 1.2%. The entire process, from parameter injection to stream output, is seamless, ensuring the authenticity of aviation training in harsh network environments.

[0098] In a practical application, in a video transmission optimization scenario of a flight simulator, after the system receives a down-resolution frame (e.g., 640 pixels wide x 360 pixels high = 230,400 pixels), it immediately initiates a fault-tolerant processing procedure: First, it calculates the data volume of a single frame. Each pixel contains three color values ​​(red, green, and blue) and occupies 3 bytes of storage space. Therefore, the total data volume is 230,400 pixels x 3 bytes = 691,200 bytes. Next, it analyzes the image features and detects that 60% of the area, such as the sky background, has a smooth color change, while the instrument panel area has a significant color change. The percentage of smooth areas (0.6) is multiplied by the total data volume, i.e., 691,200 multiplied by 0.6, resulting in 414,720, which is used as the image complexity index. Subsequently, based on the current network packet loss situation (e.g., 10 packets are lost for every 100 packets transmitted), dynamic parameter adjustments are made: the upper limit of the basic quantization parameter is set to 20 points. The image complexity of 414,720 is multiplied by the 10% packet loss rate, resulting in 41,472. This is then multiplied by a fine-tuning coefficient of 0.00005 to calculate an adjustment value of approximately 2 points, raising the new upper limit to 22 points. Simultaneously, the original maximum frame group length of 60 frames is divided by 1.1, which corresponds to the adjustment coefficient for 100% + 10% packet loss. This results in approximately 54.5, which is rounded down to 54 frames, serving as the maximum frame group length. This 54 frames are then set as the upper limit for the number of reference frames. Ultimately, the system configures these three quantization parameters (range 5-22 points), frame group length (no more than 54 frames), and reference frame count (limited to 54 frames) to the encoder, generating a video stream with packet loss resilience. The entire process ensures the clarity of critical images such as dashboards while enabling the video stream to adapt to unstable network environments.

[0099] In the overall scheme of step 103 above, the total number of pixels and spatial complexity parameters of the down-resolution frames are coordinated with the packet loss rate parameter to dynamically adjust key encoding parameters to generate a fault-tolerant video stream with high packet loss resistance. Specifically, a frame complexity weight is generated based on the frame data volume and spatial complexity. The product of this weight and the packet loss rate is used to expand the upper limit of the quantization step size range. At the same time, the maximum threshold of the frame group length is inversely mapped according to the packet loss rate, and this is used to constrain the upper limit of the number of reference frames in the inter-frame prediction mode. The adjusted quantization step size range, frame group length, and number of reference frames are injected into the encoding process to form a triple composite protection mechanism, thereby significantly enhancing the packet loss resistance and decoding stability of the video stream in harsh network environments while maintaining basic image quality.

[0100] 104. At the receiving end, perform entropy decoding on the fault-tolerant coded stream, and perform motion compensation alignment on the decoded luminance component and decoded chrominance component of the decoded and recovered down-resolution frame to obtain the decoded reconstructed frame.

[0101] Optionally, step 104 may specifically include the following steps:

[0102] 1041. Perform syntax element type determination on the fault-tolerant encoded stream to determine the category of the element to be decoded and the corresponding context model index. Based on the current state value of the context model index, calculate the corresponding probability interval through the arithmetic decoding engine.

[0103] 1042. Compare the value of the current parsing position in the fault-tolerant coding stream with the boundary value of the probability interval to parse out the binary character sequence, perform debinarization on the binary character sequence, and restore it to the decoded luminance component and decoded chrominance component of the down-resolution frame;

[0104] 1043. Extract the motion vector of the decoded luminance component, calculate the spatial coordinate offset of the decoded chrominance component based on the motion vector, adjust the spatial position of the decoded chrominance component using the spatial coordinate offset, and obtain the offset-compensated decoded chrominance component.

[0105] 1044. Perform time-domain motion trajectory matching on the offset-compensated decoded chroma component and the decoded luminance component, and merge the matched decoded luminance component and decoded chroma component to obtain the reconstructed decoded frame.

[0106] In the above steps, the fault-tolerant coded stream refers to the generated compressed video data resistant to packet loss; entropy decoding is the inverse process of encoding, used to recover the original information from the compressed data; down-resolution frames refer to frames with reduced resolution; decoded luma and chroma components are the luma and color data recovered after entropy decoding, respectively; motion compensation alignment refers to adjusting the chroma space position according to the motion data to match the luma; decoded reconstructed frames are the finally restored video frames; syntax element type judgment identifies the categories of different encoded elements in the data stream; context model index identifies the probability model to which the current decoded element belongs; current state value is the dynamic probability state of the model; arithmetic decoding engine is the algorithm unit that uses probability intervals to recover data; probability interval represents the numerical range of the probability of a symbol appearing; boundary value comparison refers to the operation of comparing data values ​​with probability intervals; binary character sequences are intermediate data composed of 0s and 1s; debinarization refers to restoring the binary sequence to its original value; motion vectors are vector data describing inter-frame motion offsets; spatial coordinate offset is the position correction value of chroma relative to luma; offset compensation is the process of adjusting the chroma position; temporal motion trajectory matching refers to the process of tracking motion trajectories across frames.

[0107] In this embodiment, after receiving the fault-tolerant encoded stream in step 1041, the header data is parsed to identify the syntax element type and determine the category of the current element to be decoded, such as a motion vector element. Based on this category, the predefined context model index number is queried to point to model M1, and the current state value of the model is obtained. (like =0.6), the probability interval is calculated using the arithmetic decoding engine, where the interval for the symbol "0" is calculated using the following formula: Parameter description: This indicates that the base value for the cumulative probability is 0 by default. Assuming the state range value is 100, the calculated interval is [0, 60) (0.6 × 100 = 60). For example, when decoding flight navigation video, the motion vector element type is detected, the index model M1 state value is 0.6, and the symbol "0" interval [0, 60) is generated, providing a basis for the next decoding step.

[0108] Secondly, the video component data is recovered through step 1042, specifically by reading the current values ​​of the fault-tolerant encoded stream. V like V =45, compare it with the probability interval boundary of step 1041: if V Belongs to [0, If the input value is within the range [0, 60), the output is "0"; otherwise, the output is "1". This process is repeated to generate a binary sequence such as "0010". Then, the original value is restored using a debinarization mapping table, which converts the binary value "0010" to the value 12 according to predefined rules. The data stream is separated to obtain the decoded luminance component of the down-resolution frame, such as a 512×288 matrix, and the decoded chrominance component, such as a 256×144 matrix. For example, when processing flight dashboard video, if the input value is 45, the output is "0" within the range [0, 60). The accumulated sequence "0010" is mapped to the value 12. Finally, the luminance and chrominance component matrices are restored.

[0109] Next, the chroma components are aligned in step 1043, specifically by extracting motion vectors from the decoded luminance components. , such as horizontal displacement For pixels, based on a chroma sampling format such as 4:2:0, the formula for calculating spatial coordinate offset is: ,in, The subsampling ratio is set to 2, and the offset is calculated. Pixel; bilinear interpolation compensation is performed on the decoded chroma components, using the following interpolation formula: ;

[0110] in, This represents the new chromaticity value after offset compensation. The chromaticity values ​​of neighboring points. For positional weights, the actual operation is to use chroma pixel coordinates from Adjusted to For example, the chromaticity pixels of an aircraft altitude indicator. Offset to Take the nearest point , , , Interpolation is used to calculate the new chromaticity value.

[0111] Finally, a reconstructed frame is generated in step 1044. Specifically, this involves performing temporal motion trajectory matching on the offset-compensated chroma and luminance components, i.e., comparing the motion vectors of consecutive frames to track the object's trajectory. For example, if the pointer position changes, the trajectory deviation... When using pixels (threshold), fine-tune the chroma position based on the luminance component; after matching, merge the components: the luminance matrix is ​​directly used as the Y channel, and the interpolated chroma matrix is ​​used as... The channels are reconstructed into YUV format decoded and reconstructed frames. For example, in the flight speedometer animation, the pointer brightness position (100, 200) in frame t is detected. If the chromaticity trajectory deviation ΔD = 0.5 pixels does not exceed the threshold, the 960×540 pixel reconstructed frame is directly merged and output.

[0112] In practical applications, in the optimization scenario of the video receiver of an aviation simulator, the system immediately starts the decoding and reconstruction process after receiving the video stream that is resistant to network packet loss: First, the data stream is identified by type. For example, based on the data characteristics, it is determined that the current segment belongs to the brightness information of the screen, and the third model in the pre-stored decoding rule library is called. The probability range is calculated by the arithmetic decoding engine. Assuming that the model indicates that the probability interval of the current symbol is 0.78 to 0.92, the value 0.85 in the data stream is read. Since this value falls between 0.78 and 0.92, the system parses out the binary sequence "1011", and then converts it into a specific digital matrix, that is, restores the luminance component matrix with a resolution of 640×360 and the chrominance component matrix after sparse sampling. Motion compensation is then performed: motion displacement information is extracted from the luminance component. For example, if a dashboard area is detected to have shifted laterally by 32 pixels between two adjacent frames, since the chroma resolution is half that of the luminance, the system automatically calculates that the chroma component needs to be compensated by 32 multiplied by 0.5, which equals 16 pixels. Based on this offset, the chroma component coordinates are adjusted, moving the chroma data at the original coordinates (x, y) to the new position (x+16, y). Finally, timeline matching is performed, precisely combining the adjusted chroma components, such as the data from the red indicator lights on the cockpit panel, with the corresponding luminance components to generate a completely reconstructed image frame. The entire process ensures that the dynamic image of the dashboard is completely restored at the receiving end, eliminating ghosting during transmission.

[0113] In the overall scheme of step 104 above, the probability interval boundary is resolved by the arithmetic decoding engine based on the state value of the context model index. The luminance and chrominance components of the down-resolution frame are then restored through numerical comparison and debinarization. Subsequently, the motion vector of the luminance component is extracted, the spatial coordinate offset of the chrominance component is calculated for component alignment compensation, and spatiotemporal synchronization calibration of the chrominance and luminance components is completed using temporal motion trajectory matching. Finally, a highly complete decoded and reconstructed frame is generated. This process significantly enhances the decoder's fault tolerance and recovery capability in data loss scenarios, ensuring stable and smooth visual content output even in environments with network fluctuations.

[0114] 105. Based on the neural network reconstruction algorithm, perform multi-scale feature extraction on the luminance component of the decoded and reconstructed frame and superimpose local detail features for detail compensation to generate an enhanced luminance feature map. Perform chroma resolution reconstruction on the decoded chroma component to generate a chroma reconstruction component. Perform color space synchronous reconstruction on the enhanced luminance feature map and the chroma reconstruction component to output a video frame restored to the original resolution.

[0115] Optionally, step 105 may specifically include the following steps:

[0116] 1051. Based on the neural network reconstruction algorithm, perform multi-level convolution operations on the luminance components of the decoded and reconstructed frame to extract luminance features at different scales;

[0117] 1052. From the brightness features of different scales, select the finest brightness feature and the local detail feature, and superimpose and fuse them to perform detail compensation on the brightness component of the decoded and reconstructed frame to generate an enhanced brightness feature map.

[0118] Step 1052 may specifically include the following process: selecting the finest brightness feature from the brightness features of different scales, and obtaining the spatial coordinates corresponding to the finest brightness feature; defining a fixed-size pixel neighborhood with each spatial coordinate as the center point; performing weighted calculation on the feature values ​​of all positions within the pixel neighborhood to generate local detail features containing all positions, wherein the weight coefficient of the spatial coordinates is higher than that of the edge positions; adding the finest brightness feature and the local detail feature point by point according to the same spatial coordinates to obtain the superimposed fusion result after detail compensation; performing a nonlinear activation operation on the superimposed fusion result to generate an enhanced brightness feature map.

[0119] 1053. Perform bilinear interpolation reconstruction on the chroma components of the decoded and reconstructed frame to generate chroma reconstruction components that match the original resolution;

[0120] 1054. The enhanced brightness feature map and the chromaticity reconstruction component are synchronously superimposed according to the preset color space conversion rules to restore the original resolution while ensuring color consistency, and output the restored video frame.

[0121] In the above steps, the neural network reconstruction algorithm refers to a deep learning-based image enhancement method; the decoded reconstruction frame refers to the restored low-resolution video frame; the luminance component and chrominance component are the separate representations of the luminance and color data of the reconstructed frame; multi-scale feature extraction captures macroscopic to microscopic features of the image through convolutional operations at different levels; local detail features describe the enhanced information of image texture details; detail compensation is the operation of adding details to the original luminance to improve sharpness; the enhanced luminance feature map is the high-definition luminance data after compensation; chrominance resolution reconstruction is the process of restoring low-resolution chrominance data to the original resolution; the chrominance reconstruction component is the restored complete chrominance data; the color space is the same. Step reconstruction is the process of merging luminance and chrominance in a unified color space; the original resolution is the video size before downscaling; the restored video frame is the final high-quality full-resolution frame output; multi-level convolutional operations are steps in a neural network to perform multiple feature extractions on the input data; the finest-scale luminance feature refers to the feature map with the highest resolution; spatial coordinates indicate the coordinates of feature points in the image; pixel neighborhood refers to a square area centered on the coordinates; weighted calculation is the summation of feature values ​​within the neighborhood according to weights; bilinear interpolation reconstruction is a method of amplifying the chrominance component by linear interpolation of neighboring pixels; the preset color space conversion rule is the color model standard for merging luminance and chrominance.

[0122] In this embodiment, firstly, multi-scale feature extraction is performed on the luminance component of the decoded and reconstructed frame through step 1051. The specific process is as follows: the luminance component is input into a neural network for three-level convolution processing: the first layer uses a 5×5 large convolution kernel to extract macroscopic features and generate a feature map F1 with its size halved; the second layer uses a 3×3 medium convolution kernel to process F1 and generate a feature map F2 with its size halved again; the third layer uses a 1×1 small convolution kernel to process F2 and generate a feature map F3 that retains details. For example, the luminance component of a flight instrument panel (512×288 pixels) is processed in three levels: F1, with a size of 256×144, contains the overall outline of the instrument panel; F2, with a size of 128×72, contains the pointer outline; and F3, with a size of 128×72, contains the scale texture, thus fully realizing multi-scale feature extraction.

[0123] Secondly, in step 1052, the finest feature map F3 is selected, and the spatial coordinates (x, y) of its feature points are obtained; a 5×5 pixel neighborhood is defined centered on these coordinates, and local detail features are calculated using a weighted formula:

[0124] ;

[0125] Among them, spatial coordinates Indicates the position of the currently processed feature point in the image; offset It is relative to the center coordinates The horizontal / vertical offset value, ranging from -2 to 2, is used to traverse 25 positions within a 5×5 neighborhood; Indicates relative position within the neighborhood. The finest-scale eigenvalue at the specified location; weighting coefficients This refers to the feature weights assigned to each offset position. The weight coefficient is 0.4 at the center point w(0,0), and decreases linearly with increasing distance until it reaches 0.01 at the edge w(2,2). The F3 value is added point-by-point to the local feature map along the same coordinates, resulting in enhanced = F3 + local feature. Finally, the ReLU activation function f(x) = max(0,x) is used to generate an enhanced brightness feature map. For example, at the pointer tip coordinates (120,85): the F3 value is 0.7, the neighborhood weighted local feature is 0.25, and the sum of the values ​​is 0.95. After ReLU, the output is 0.95, significantly improving sharpness.

[0126] Next, chroma resolution reconstruction is performed in step 1053. First, the chroma components of the decoded and reconstructed frame are obtained. The resolution of these components is lower than the original video size; for example, if the original resolution is 512×288, the chroma component is only 256×144. The system uses bilinear interpolation for reconstruction: for each pixel to be reconstructed in the chroma image, the positions of its four nearest neighbor pixels are found, and the horizontal and vertical distance ratios between this point and its four neighbors are calculated respectively. Then, based on these distance ratios, a weighted average of the chroma values ​​of the four neighboring points is calculated, assigning higher weights to closer neighboring points (e.g., the closest point has a weight of 40%, the second closest 30%, the farther point 20%, and the farthest point 10%). Finally, the reconstructed chroma value of this point is calculated using the weighted average. This operation is repeated until all target pixels are reconstructed, restoring the chroma components to their original resolution. For example, in the chromaticity reconstruction of a flight map, the target point is located at coordinates (200.5, 150.5), and the coordinates of the four neighboring points are (200, 150), (201, 150), (200, 151), and (201, 151). The new chromaticity value is calculated based on the position ratio weighting to eliminate the color mosaic effect and generate a full-size chromaticity reconstruction component.

[0127] Finally, in step 1054, the restored video frame is output by first aligning the enhanced luminance feature map generated in step 1052 with the chromaticity reconstruction components generated in step 1053. A standard YUV to RGB color space conversion method is used: for each pixel location, the Y value of the luminance feature map, and the Cb and Cr values ​​of the chromaticity reconstruction components are taken; then, the RGB primary color values ​​are calculated according to aviation display standards: the red R value is calculated by converting Y and Cr values, the green G value is calculated by converting Y, Cb, and Cr, and the blue B value is calculated by converting Y and Cb values. Specific coefficients follow international standards to ensure color accuracy. During the compositing process, a one-to-one correspondence between luminance and chromaticity space positions is strictly maintained, ultimately outputting an RGB format video frame at the original resolution. For example, in processing the red area of ​​an aircraft altimeter, the luminance Y value is 180, the chromaticity Cr value is 120, and the red R value is calculated to be 170 using international standard coefficients, preserving a vibrant red and accurately restoring the color system of the aviation instrument panel.

[0128] In practical applications, during the high-definition video reconstruction of an aviation simulator, the system performs enhancement processing on the 640×360 pixel image recovered from the receiving end. First, for the luminance component, feature maps of different scales are extracted using a three-layer neural network, with the smallest scale feature map maintaining its original resolution. The system defines a 3×3 pixel region as the analysis range, centered on each pixel location (e.g., points at coordinates 100 and 200). All pixels within this region are weighted: the center point is assigned a higher weight (e.g., 70%), and the remaining 30% weight is evenly distributed among the eight surrounding points. The weighted calculation result is superimposed on the smallest scale feature map and then subjected to non-linear amplification to generate a luminance image with enhanced details. Simultaneously, the chrominance component is processed. The original sparsely sampled chrominance data is only 180×120 pixels. A bilinear interpolation method is used for reconstruction: when it is necessary to fill in the color value at coordinates 150 and 300, the four nearest known sampling points are searched, and a weighted average is calculated based on the distance ratio between each point and the target location, directly yielding a chrominance value of 129 for that point. Finally, a color space synchronous reconstruction is performed, merging the enhanced luminance map and the reconstructed chrominance map according to standard conversion rules. For example, a luminance value of 210 at a certain location is combined with chrominance values ​​of 129 and 125, and the red, green, and blue values ​​are calculated using a general conversion formula, ultimately generating a restored image with the original resolution of 1920×1080. The entire process improves the sharpness of the dashboard digital edges, makes the sky background color transition natural and smooth, and completely restores the visual quality before transmission.

[0129] In the overall scheme of step 105 above, a neural network reconstruction algorithm is used to perform fine-grained image quality restoration on the decoded and reconstructed frame. Multi-level convolution is performed on the luminance component to extract multi-scale features, and the finest scale luminance features are superimposed and compensated with local detail features generated based on spatial position coordinates. An enhanced luminance feature map is generated through nonlinear activation. At the same time, bilinear interpolation is used to perform resolution reconstruction on the chrominance component. Finally, the enhanced luminance feature map and the chrominance reconstruction component are reconstructed synchronously through a preset color space conversion rule. Under the premise of strictly maintaining color consistency, the original resolution is accurately restored, and a restored video frame with rich visual details and accurate colors is output.

[0130] The following is a complete embodiment for steps 101 to 105:

[0131] like Figure 2As shown, during the audio and video transmission process of an aviation simulator training system, the system first monitors the network status in real time: transmission latency and bandwidth data are collected every 2 seconds, and the latency is divided into three levels: 0-20 milliseconds (excellent), 21-50 milliseconds (good), and above 51 milliseconds (needs optimization). The current latency of 40 milliseconds is considered good. Combined with the historical data showing bandwidth variations concentrated between 5-15 milliseconds, the system automatically scores it at 7 points and converts it to a "moderate fluctuation" network status level. Based on this level, the system initiates image adjustment: the luminance component of the original 1920×1080 image is sampled from every 3 pixels, reducing it to a 640×360 resolution; simultaneously, the image is divided into 5×5 blocks, sampling every 4 pixels in areas with smooth color (such as the sky background), while maintaining the original sampling in areas with dense detail (such as the instrument panel), generating a down-resolution frame. At this point, the total number of pixels is calculated to be 640 x 360 = 230,400 pixels. Combining this with the image characteristics of the 60% smooth area, a complexity value of 414,720 is obtained.

[0132] Based on a 10% packet loss rate, the system dynamically adjusts the encoding parameters: the quantization step size upper limit of 20 is adjusted from approximately 20 (414,720 x 10% x 0.00005) to 22; the frame group length is changed from 60 frames to approximately 54 frames (60 divided by 1.1); and the prediction reference frame is limited to 54 frames. After adjustment, the encoder generates a packet loss-resistant data stream. Upon receiving the data, the receiver first identifies the data type and uses a probabilistic model to reconstruct the luminance matrix from the binary sequence "1011". Based on the 32-pixel displacement of the dashboard in the luminance matrix, the required luminance compensation of 16 pixels is calculated by halving the chrominance resolution, and the chrominance data coordinates are moved to achieve alignment.

[0133] Finally, a three-layer neural network is used for processing: multi-scale features of the luminance component are extracted, and in the smallest scale layer, each pixel is used as the center, such as coordinates [100, 200], and the pixels in the 3×3 area are weighted (70% in the center and 30% in the periphery). After superposition, a luminance map with sharpened edge of the dashboard is generated; bilinear interpolation is performed on the 180×120 chroma sampling points to reconstruct to 640×360; the luminance value 210 and the chroma value 129 / 125 are converted according to the YUV-RGB conversion rules to output red 206, green 212, and blue 212, reconstructing the original resolution image and completely restoring the details and natural colors of the cockpit instrument panel.

[0134] Figure 3 This application provides a schematic diagram of the structure of an audio and video signal transmission system with an independently controllable architecture and hardware-software collaboration, as shown in the embodiments. Figure 3 As shown, the system includes:

[0135] Analysis module 31 is used to synchronously collect bandwidth fluctuation parameters, packet loss rate parameters and transmission delay parameters of the mobile terminal link during audio and video transmission, and generate network status level based on the correlation analysis of the bandwidth fluctuation parameters and transmission delay parameters.

[0136] The sampling module 32 is used to perform resolution scaling on the original luminance component of the video frame and selective sampling on the original chrominance component according to the network status level, and generate a down-resolution frame that matches the current network status level.

[0137] The generation module 33 is used to generate a fault-tolerant coding stream with a fused anti-packet-loss structure by simultaneously adjusting the quantization step size range, frame group length, and inter-frame prediction mode based on the total number of pixels and spatial complexity parameters of the down-resolution frame and the packet loss rate parameters.

[0138] The reconstruction module 34 is used to perform entropy decoding operation on the fault-tolerant coded stream at the receiving end, and to perform motion compensation alignment on the decoded luminance component and decoded chrominance component of the decoded and recovered down-resolution frame to obtain the decoded reconstructed frame.

[0139] The restoration module 35 is used to perform multi-scale feature extraction on the luminance component of the decoded and reconstructed frame based on a neural network reconstruction algorithm and superimpose local detail features for detail compensation to generate an enhanced luminance feature map. It also performs chroma resolution reconstruction on the decoded chroma component to generate a chroma reconstruction component, and performs color space synchronous reconstruction on the enhanced luminance feature map and the chroma reconstruction component to output a video frame restored to the original resolution.

[0140] Figure 3 The aforementioned autonomous and controllable architecture hardware and software collaborative audio and video signal transmission system can execute... Figure 1 The implementation principle and technical effects of the audio and video signal transmission method with autonomous and controllable architecture and hardware-software collaboration described in the illustrated embodiment will not be repeated here. The specific methods by which each module and unit performs operations in the audio and video signal transmission system with autonomous and controllable architecture and hardware-software collaboration in the above embodiments have been described in detail in the embodiments related to this method, and will not be elaborated upon here.

[0141] In one possible design, Figure 3 The illustrated embodiment of an autonomous and controllable architecture hardware and software collaborative audio and video signal transmission system can be implemented as a computing device, such as... Figure 4 As shown, the computing device may include a storage component 41 and a processing component 42;

[0142] The storage component 41 stores one or more computer instructions, wherein the one or more computer instructions are invoked and executed by the processing component 42.

[0143] The processing component 42 is used for the above Figure 1 The embodiment describes an audio and video signal transmission method based on an autonomous and controllable architecture with hardware and software collaboration.

[0144] The processing component 42 may include one or more processors to execute computer instructions to complete all or part of the steps in the above-described method. Alternatively, the processing component may be implemented as one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-described method.

[0145] Storage component 41 is configured to store various types of data to support operations at the terminal. The storage component can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0146] Of course, computing devices may also include other components, such as input / output interfaces, display components, communication components, etc.

[0147] Input / output interfaces provide interfaces between processing components and peripheral interface modules, which can be output devices, input devices, etc.

[0148] The communication components are configured to facilitate wired or wireless communication between computing devices and other devices.

[0149] The computing device can be a physical device or an elastic computing host provided by a cloud computing platform. In this case, the computing device can refer to a cloud server, and the aforementioned processing components, storage components, etc., can be basic server resources rented or purchased from the cloud computing platform.

[0150] This application also provides a computer storage medium storing a computer program, which, when executed by a computer, can perform the above-described functions. Figure 1 The illustrated embodiment presents an audio and video signal transmission method based on a self-controllable architecture with hardware and software collaboration.

[0151] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0152] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0153] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0154] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for transmitting audio and video signals using a self-controllable architecture with hardware and software collaboration, characterized in that, include: During audio and video transmission, bandwidth fluctuation parameters, packet loss rate parameters, and transmission delay parameters of the mobile terminal link are collected synchronously. Based on the correlation analysis of the bandwidth fluctuation parameters and transmission delay parameters, a network status level is generated. Based on the network status level, the original luminance component of the video frame is scaled up and the original chrominance component is selectively sampled using a frame adjustment device to generate a down-resolution frame that matches the current network status level. Based on the total number of pixels and spatial complexity parameters of the down-resolution frame, and combined with the packet loss rate parameter, the quantization step size range, frame group length and inter-frame prediction mode are simultaneously adjusted to generate a fault-tolerant coding stream with a fused anti-packet loss structure. At the receiving end, an entropy decoding operation is performed on the fault-tolerant coded stream, and motion compensation alignment is performed on the decoded luminance component and decoded chrominance component of the decoded and recovered down-resolution frame to obtain the decoded reconstructed frame. Based on the neural network reconstruction algorithm, multi-scale feature extraction is performed on the luminance component of the decoded and reconstructed frame, and local detail features are superimposed for detail compensation to generate an enhanced luminance feature map. Chromaticity resolution reconstruction is performed on the decoded chrominance component to generate a chrominance reconstruction component. The enhanced luminance feature map and the chrominance reconstruction component are then synchronously reconstructed in color space to output a video frame restored to the original resolution.

2. The audio and video signal transmission method with autonomous and controllable architecture and hardware-software collaboration according to claim 1, characterized in that, Based on a neural network reconstruction algorithm, multi-scale feature extraction is performed on the luminance component of the decoded and reconstructed frame, and local detail features are superimposed for detail compensation to generate an enhanced luminance feature map. Chromaticity resolution reconstruction is then performed on the decoded chroma component to generate a reconstructed chroma component. Finally, the enhanced luminance feature map and the reconstructed chroma component are synchronously reconstructed in color space to output a video frame restored to its original resolution, including: The luminance components of the decoded and reconstructed frame are subjected to multi-level convolution operations based on a neural network reconstruction algorithm to extract luminance features at different scales. From the brightness features at different scales, the finest brightness features are selected and superimposed and fused with local detail features to perform detail compensation on the brightness components of the decoded and reconstructed frame, generating an enhanced brightness feature map. Bilinear interpolation reconstruction is performed on the chroma components of the decoded and reconstructed frame to generate chroma reconstructed components that match the original resolution; The enhanced brightness feature map and the chromaticity reconstruction component are synchronously superimposed according to a preset color space conversion rule to restore the original resolution while ensuring color consistency, and output the restored video frame.

3. The audio and video signal transmission method with autonomous and controllable architecture and hardware-software collaboration according to claim 2, characterized in that, From the brightness features at different scales, the finest brightness features are selected and superimposed and fused with local detail features to perform detail compensation on the brightness components of the decoded and reconstructed frame, generating an enhanced brightness feature map, including: From the brightness features of different scales, the finest brightness feature is selected, and the spatial coordinates corresponding to the finest brightness feature are obtained. A pixel neighborhood of a fixed size is delineated with each of the spatial coordinates as the center point. The feature values ​​of all locations within the pixel neighborhood are weighted and calculated to generate local detail features containing all locations, wherein the weight coefficient of the spatial location coordinates is higher than that of the edge locations; The brightness features at the finest scale and the local detail features are added point by point according to the same spatial coordinates to obtain the superimposed and fused result after detail compensation. A nonlinear activation operation is performed on the superimposed and fused result to generate an enhanced brightness feature map.

4. The audio and video signal transmission method with autonomous and controllable architecture and hardware-software collaboration according to claim 1, characterized in that, Based on the total number of pixels and spatial complexity parameters of the down-resolution frame, and combined with the packet loss rate parameters, the quantization step size range, frame group length, and inter-frame prediction mode are simultaneously adjusted to generate a fault-tolerant coded stream with a fused packet loss-resistant structure, including: The data volume per unit frame is calculated based on the total number of pixels in the down-resolution frame, and the data volume per unit frame is combined with the space complexity parameter to generate a frame complexity weight. The upper limit of the quantization step size range is increased by multiplying the frame complexity weight and the packet loss rate parameter. The maximum value of the frame group length is reverse-mapped according to the packet loss rate parameter, and the maximum value is used as the upper limit threshold of the reference frame number for limiting the inter-frame prediction mode. During the encoding process, the adjusted quantization step size range, frame group length, and inter-frame prediction mode parameters are injected into the encoder to generate a fault-tolerant coding stream.

5. The audio and video signal transmission method with autonomous and controllable architecture and hardware-software collaboration according to claim 1, characterized in that, Based on the network state level, resolution scaling is performed on the original luminance component of the video frame using a frame adjustment device, and selective sampling is performed on the original chrominance component to generate a down-resolution frame that matches the current network state level, including: According to the network status level, a preset resolution scaling ratio table is queried to determine the target scaling ratio of the luminance component. The original luminance component is uniformly sampled according to the target scaling ratio to generate the scaled luminance component. Non-uniform region division is performed on the original chromaticity components. Based on the spatial gradient distribution of the original chromaticity components, sparse sampling is performed on regions with spatial gradients below a threshold to obtain sampled chromaticity components. The scaled luminance component is combined with the sampled chrominance component to generate a down-resolution frame that matches the current network state level.

6. The audio and video signal transmission method with autonomous and controllable architecture and hardware-software collaboration according to claim 1, characterized in that, At the receiving end, entropy decoding is performed on the fault-tolerant coded stream, and motion compensation alignment is performed on the decoded luminance and chrominance components of the decoded and recovered down-resolution frame to obtain a decoded reconstructed frame, including: The fault-tolerant encoded stream is subjected to syntax element type determination to determine the category of the element to be decoded and its corresponding context model index. Based on the current state value of the context model index, the corresponding probability interval is calculated by the arithmetic decoding engine. The value at the current parsing position in the fault-tolerant coding stream is compared with the boundary value of the probability interval to parse out the binary character sequence. The binary character sequence is then debinarized to restore the decoded luminance component and decoded chrominance component of the down-resolution frame. Extract the motion vector of the decoded luminance component, calculate the spatial coordinate offset of the decoded chrominance component based on the motion vector, adjust the spatial position of the decoded chrominance component using the spatial coordinate offset, and obtain the offset-compensated decoded chrominance component. The time-domain motion trajectory matching is performed on the offset-compensated decoded chroma component and the decoded luminance component. The matched decoded luminance component and decoded chroma component are then merged to obtain the reconstructed decoded frame.

7. The audio and video signal transmission method with autonomous and controllable architecture and hardware-software collaboration according to claim 1, characterized in that, During audio and video transmission, bandwidth fluctuation parameters, packet loss rate parameters, and transmission delay parameters of the mobile terminal link are collected synchronously. Based on the correlation analysis of the bandwidth fluctuation parameters and transmission delay parameters, a network status level is generated, including: Within a fixed time window, the bandwidth fluctuation parameters, packet loss rate parameters, and transmission delay parameters of the mobile terminal link are periodically acquired, and the change in the bandwidth fluctuation parameters within a continuous time window is calculated. The transmission delay parameter is divided into multiple level intervals, and each level interval is associated with the change in the bandwidth fluctuation parameter. Based on the distribution of the change in the corresponding level interval, the bandwidth stability score corresponding to each level interval is determined. Based on a preset scoring threshold mapping table, the bandwidth stability score corresponding to each level range is converted into a discrete network status level.

8. A self-controllable architecture hardware and software collaborative audio and video signal transmission system, characterized in that, include: The analysis module is used to simultaneously collect bandwidth fluctuation parameters, packet loss rate parameters, and transmission delay parameters of the mobile terminal link during audio and video transmission. Based on the correlation analysis of the bandwidth fluctuation parameters and transmission delay parameters, a network status level is generated. The sampling module is used to perform resolution scaling on the original luminance component of the video frame and selective sampling on the original chrominance component according to the network status level, thereby generating a down-resolution frame that matches the current network status level. The generation module is used to generate a fault-tolerant coded stream with a fused anti-packet-loss structure by simultaneously adjusting the quantization step size range, frame group length, and inter-frame prediction mode based on the total number of pixels and spatial complexity parameters of the down-resolution frame and the packet loss rate parameters. The reconstruction module is used to perform entropy decoding on the fault-tolerant coded stream at the receiving end, and to perform motion compensation alignment on the decoded luminance component and decoded chrominance component of the decoded and recovered down-resolution frame to obtain the decoded reconstructed frame. The restoration module is used to perform multi-scale feature extraction on the luminance component of the decoded and reconstructed frame based on a neural network reconstruction algorithm, and to perform detail compensation by superimposing local detail features to generate an enhanced luminance feature map. It then performs chroma resolution reconstruction on the decoded chroma component to generate a chroma reconstruction component, and performs color space synchronous reconstruction on the enhanced luminance feature map and the chroma reconstruction component to output a video frame restored to the original resolution.

Images