A satellite communication-based image enhancement method, system, device and medium

Abstract

The application provides a satellite communication-based image enhancement method, system, device and medium, wherein the method comprises: in-bucket sorting of time bucket grouping data to obtain sorted time bucket data; based on a trigger processing water line and a forced processing water line of a dynamic buffer, performing handover condition detection on the sorted time bucket data, and when the handover condition is met, handing over the sorted time bucket data to a preset image processing unit to obtain reorganized image data; performing content-aware error concealment on a missing area in the reorganized image data to obtain a concealed and repaired image; and performing lightweight deep learning model optimization on the concealed and repaired image to obtain an enhanced final image, so as to achieve the technical effects of improving key frame recovery success rate, optimizing bandwidth utilization, reducing end-to-end processing delay, reducing memory occupation, and improving image visual quality.

Description

Technical Field

[0001] This invention relates to the field of satellite communication technology, and in particular to an image enhancement method, system, device and medium based on satellite communication. Background Technology

[0002] With the widespread application of satellite communication technology in remote sensing monitoring, deep space exploration, and emergency communications, the real-time and reliable transmission of high-resolution image data has become a critical requirement. However, the satellite communication environment is characterized by long distances, high levels of interference, long latency, and drastic fluctuations in packet loss rates, posing severe challenges to the complete reception and quality assurance of image data.

[0003] Traditional technologies use a fixed-ratio forward error correction mechanism for redundant error correction. However, this mechanism is prone to wasting bandwidth resources when network conditions fluctuate, and it cannot effectively recover key frame data packets to ensure normal video stream decoding. When facing scenarios with dynamically changing satellite channels, traditional technologies use reordering buffers that rely on fixed timeout waiting mechanisms for data reassembly. However, this is difficult to adapt to channel characteristics and can also cause problems such as excessive end-to-end latency and a surge in memory consumption. In addition, traditional technologies often ignore the differences in the importance of data content and visually significant features, apply the same level of protection to different types of frames, and fail to specifically optimize error masking and quality enhancement strategies. Summary of the Invention

[0004] Therefore, it is necessary to provide an image enhancement method, system, device, and medium based on satellite communication to address the above-mentioned technical problems, so as to improve the success rate of key frame recovery, optimize bandwidth utilization, reduce end-to-end processing latency, reduce memory usage, and improve the visual quality of images.

[0005] In a first aspect, this application provides an image enhancement method based on satellite communication, the method comprising:

[0006] A circular buffer is established, and the water level of the circular buffer is dynamically configured based on the real-time network packet loss rate to obtain a dynamic buffer with a trigger processing water level and a forced processing water level.

[0007] Receive image data packets that have been marked with content importance classification and the generated forward error correction code; perform packet loss recovery based on the image data packets marked with content importance classification and the forward error correction code to obtain the recovered image data packets;

[0008] Based on the coarse-grained timestamp and fine-grained sequence number carried by the restored image data packets, the restored image data packets are distributed into time buckets to obtain time bucket grouped data; the time bucket grouped data is sorted within each bucket to obtain sorted time bucket data; based on the trigger processing water level and the forced processing water level of the dynamic buffer, the sorted time bucket data is checked for handover conditions, and when the handover conditions are met, the sorted time bucket data is handed over to the preset image processing unit to obtain reconstructed image data;

[0009] Content-aware error masking is performed on the missing regions in the reconstructed image data to obtain a masked and repaired image; a lightweight deep learning model is then used to optimize the masked and repaired image to obtain the enhanced final image.

[0010] In one embodiment, the masked restoration image is optimized using a lightweight deep learning model to obtain the enhanced final image, including:

[0011] Visual saliency analysis was performed on the masked restoration image to generate a visual attention heatmap;

[0012] Based on visual attention heatmap, multi-scale feature extraction is performed on the defect region in the masking repair image to generate a multi-scale defect feature map containing local texture features and global structural features.

[0013] Adversarial feature reconstruction is performed on multi-scale defect feature maps using a pre-defined generative adversarial network model to generate perceptually optimized image regions.

[0014] The image region optimized by perception is fused with the non-defect region in the masked repair image in the gradient domain to obtain the enhanced final image.

[0015] In one embodiment, gradient domain fusion is performed between the perceptually optimized image region and the non-defective region in the masking repair image to obtain the enhanced final image, including:

[0016] Boundary gradient fields are constructed for the image regions of the perception optimization and the non-defect regions in the masking restoration image to generate the region boundary gradient fields.

[0017] Based on the Poisson equation, the gradient field of the region boundary is optimized across regions to generate a seamlessly transitioned fused gradient field.

[0018] Gradient field reconstruction is performed on the seamlessly transitioned fused gradient field to generate the enhanced final image. The expression for the enhanced final image is:

[0019]

[0020] in, This represents the final enhanced image. This indicates the gradient field reconstruction operation. Indicates the integration of regional spatial domains. Indicates the fused gradient field. Represents the gradient field at the region boundary. Represents the gradient operator, This represents minimizing the solution of a variational problem. This represents a two-dimensional spatial integral.

[0021] In one embodiment, adversarial feature reconstruction of multi-scale defect feature maps is performed using a preset generative adversarial network model to generate perceptually optimized image regions, including:

[0022] The generator of the generative adversarial network model performs multi-level residual reconstruction on the multi-scale defect feature map to generate residual reconstructed feature maps.

[0023] The residual reconstruction feature map and the multi-scale defect feature map are aggregated across scales to generate an aggregated feature map.

[0024] Spatial-aware reconstruction of the aggregated feature map is performed through the generator's output layer to generate perceptually optimized image regions.

[0025] In one embodiment, content-aware error masking is performed on missing regions in the reconstructed image data to obtain a masked and repaired image, including:

[0026] Spatial interpolation is performed to fill in the missing static image regions in the reconstructed image data to obtain a masked and repaired image;

[0027] Temporal motion compensation is performed on the lost regions of the video sequence in the reconstructed image data to obtain the masked and restored image.

[0028] In one embodiment, spatial interpolation is performed to fill in the static image loss region in the reconstructed image data to obtain a masked restoration image, including:

[0029] Edge-aware analysis is performed on the missing regions of static images to generate an edge direction distribution map describing the texture direction;

[0030] Based on the edge orientation distribution map, a set of intact pixel blocks that meet the preset texture continuity threshold are selected in the area surrounding the lost region;

[0031] Multi-directional dynamic weighted interpolation is performed on the set of intact pixel blocks that meet the preset texture continuity threshold to generate repaired pixel regions. The expression for the repaired pixel regions is:

[0032]

[0033] in, Indicates repairing pixel areas. This represents the number of intact pixel blocks that meet the threshold. Indicates the first Dynamic weights of individual pixel blocks Indicates the first A complete pixel block, This represents the gradient of the edge direction distribution map. Represents pixel blocks gradient, This represents the function for calculating directional similarity.

[0034] The repaired pixel region is integrated with the lost region of the static image through topological structure matching to obtain the masked repair image.

[0035] In one embodiment, based on the coarse-grained timestamp and fine-grained sequence number carried by the recovered image data packets, time-bucket distribution is performed on the recovered image data packets to obtain time-bucket group data, including:

[0036] Parse the coarse-grained timestamps carried in the recovered image data packets to generate time bucket identifiers;

[0037] Based on the time bucket identifier, the recovered image data packets are dynamically allocated to the corresponding time bucket storage units;

[0038] For data packets within the same time bucket storage unit, perform sequence continuity verification based on the fine-grained sequence number carried by the data packet, and remove data packets with abnormal sequence number jumps;

[0039] By integrating the valid data packets from all time-bucket storage units, the time-bucket grouped data is obtained.

[0040] Secondly, this application also provides an image enhancement system based on satellite communication, the system comprising:

[0041] The dynamic water level configuration module is used to establish a circular buffer. Based on the real-time network packet loss rate, the water level of the circular buffer is dynamically configured to obtain a dynamic buffer with a trigger processing water level and a forced processing water level.

[0042] The hierarchical error correction and recovery module is used to receive image data packets that have been marked with content importance hierarchy and the generated forward error correction code; based on the image data packets marked with content importance hierarchy and the forward error correction code, it performs packet loss recovery to obtain the recovered image data packets;

[0043] The hybrid reassembly decision module is used to distribute the recovered image data packets into time buckets based on the coarse-grained timestamps and fine-grained sequence numbers carried by the recovered image data packets, thereby obtaining time bucket grouped data; sorting the time bucket grouped data within each bucket to obtain sorted time bucket data; and performing handover condition detection on the sorted time bucket data based on the trigger processing water level and the forced processing water level of the dynamic buffer. When the handover condition is met, the sorted time bucket data is handed over to the preset image processing unit to obtain reconstructed image data.

[0044] The perceptual enhancement processing module is used to perform content-aware error masking on the missing regions in the reconstructed image data to obtain a masked and repaired image; the masked and repaired image is then optimized using a lightweight deep learning model to obtain the enhanced final image.

[0045] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the methods in the first aspect of this application.

[0046] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any of the methods in the first aspect of this application.

[0047] Compared with existing technologies, this application provides an image enhancement method, system, device, and medium based on satellite communication, wherein the method and its effects include:

[0048] 1. Based on the real-time network packet loss rate, the water level of the ring buffer is dynamically configured. The water level for trigger processing and forced processing is set to form a dynamic buffer. This method can not only adapt to the fluctuation characteristics of the satellite channel and avoid excessive space occupation of the buffer, thus reducing memory occupation, but also promote data processing in a timely manner through the handover condition detection corresponding to the water level, thus helping to reduce end-to-end processing latency.

[0049] 2. After receiving image data packets with content importance classification tags and forward error correction codes, the two are combined for packet loss recovery. This can strengthen the recovery guarantee for important content, help improve the success rate of key frame recovery, and avoid resource redundancy caused by fixed-ratio error correction, which is conducive to optimizing bandwidth utilization.

[0050] 3. Based on the coarse-grained timestamp and fine-grained sequence number of the recovered data packet, time buckets are distributed and sorted within the buckets. Then, the water level of the dynamic buffer is used to detect the handover conditions and hand over the data to the image processing unit to obtain reconstructed image data. This process can make data reconstruction more efficient and further reduce the end-to-end processing time.

[0051] 4. By performing content-aware error masking on the lost areas of the reconstructed image data, and then optimizing the masked and repaired image using a lightweight deep learning model, the visual quality of the image can be improved while controlling resource consumption. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a flowchart of an image enhancement method based on satellite communication in one embodiment of the present invention;

[0054] Figure 2 This is a flowchart illustrating how a pre-defined generative adversarial network model is used to perform adversarial feature reconstruction on a multi-scale defect feature map to generate a perceptually optimized image region in one embodiment of the present invention.

[0055] Figure 3 This is a structural diagram of an image enhancement system based on satellite communication, according to one embodiment of the present invention. Detailed Implementation

[0056] To make the above-mentioned objects, features, and advantages of this application more apparent and understandable, the specific embodiments of this application will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the application. Therefore, this application is not limited to the specific embodiments disclosed below.

[0057] First, the application scenarios of the embodiments of this application will be described. In the embodiments of this application, a satellite communication-based image enhancement method, system, device, and medium are provided, applicable to, but not limited to, remote sensing monitoring image transmission scenarios.

[0058] Example 1:

[0059] In illustrative purposes, the image enhancement method, system, device and medium based on satellite communication provided in this application embodiment can also be applied to other image-related application scenarios under satellite communication, such as deep space exploration image data backhaul scenarios and image information transmission scenarios in emergency communication scenarios. This is only an example and does not limit the specific application scenarios.

[0060] like Figure 1As shown, this application provides an image enhancement method based on satellite communication, the method comprising:

[0061] S101: Establish a circular buffer and dynamically configure the water level of the circular buffer based on the real-time network packet loss rate to obtain a dynamic buffer configured with trigger processing water level and forced processing water level.

[0062] For example, the storage structure and data flow rules of the circular buffer are determined based on the transmission characteristics of the satellite communication link to ensure that the circular buffer can achieve ordered data access and cyclic reuse. The network packet loss rate parameters of the satellite communication link are collected in real time, and dynamic adjustment rules for the water level are formulated based on the changing characteristics of these parameters. The configuration conditions for the trigger processing water level and the forced processing water level are clearly defined. The trigger processing water level is used to identify the start threshold of the data processing flow, and the forced processing water level is used to define the execution boundary of the data processing operation. The water level parameters of the circular buffer are dynamically configured according to the established rules. After completing all configuration steps, a dynamic buffer with the trigger processing water level and the forced processing water level is obtained.

[0063] S102: Receive the image data packets that have been marked with content importance classification and the generated forward error correction code; perform packet loss recovery based on the image data packets that have been marked with content importance classification and the forward error correction code to obtain the recovered image data packets.

[0064] For example, the received image data packets marked with content importance classification and the generated forward error correction code are subjected to integrity verification. After confirming that both types of data are complete and error-free, the packet loss recovery process begins. The loss status of the image data packets marked with content importance classification is identified, and the recovery priority of different levels of data packets is determined according to the content importance classification results, prioritizing the recovery needs of high-importance data packets.

[0065] The generated forward error correction code is invoked to match the lost image data packets of the corresponding level. Redundancy information from the forward error correction code is used to complete and reconstruct the lost image data packets. After reconstruction, the obtained data packets are validated to confirm their content integrity and sequence coherence, resulting in the recovered image data packets.

[0066] S103: Based on the coarse-grained timestamp and fine-grained sequence number carried by the recovered image data packet, the recovered image data packet is distributed into time buckets to obtain time bucket group data; the time bucket group data is sorted within the bucket to obtain sorted time bucket data; based on the trigger processing water level and the forced processing water level of the dynamic buffer, the sorted time bucket data is checked for handover conditions; when the handover conditions are met, the sorted time bucket data is handed over to the preset image processing unit to obtain reconstructed image data.

[0067] For example, the coarse-grained timestamp and fine-grained sequence number carried in the recovered image data packets are extracted, and the two types of identification information are parsed and verified to ensure that the identification information is complete and valid. Based on the time attribute of the coarse-grained timestamp, corresponding time bucket storage intervals are divided, and the recovered image data packets are distributed to the corresponding time bucket storage intervals according to the matching time interval rules. After all recovered image data packets have been distributed, the valid data packets in each time bucket storage interval are aggregated to obtain the time bucket grouped data.

[0068] For each time bucket storage interval contained in the time bucket group data, the sequence continuity is verified based on the fine-grained sequence number. Data packets with normal sequence are filtered out and data packets with abnormal sequence are removed. Then, the data packets with normal sequence in each time bucket storage interval are sorted to obtain the sorted time bucket data.

[0069] The system retrieves the preset trigger and forced processing thresholds from the dynamic buffer and performs dual checks on the accumulated data volume and storage duration of the sorted time bucket data to determine if it meets the handover requirements. When the sorted time bucket data reaches the thresholds corresponding to either the trigger or forced processing thresholds, it is transferred to the preset image processing unit according to the preset data transmission specifications. Upon receiving the sorted time bucket data, the preset image processing unit performs data integration and structural reorganization operations to obtain reconstructed image data.

[0070] S104: Perform content-aware error masking on the missing regions in the reconstructed image data to obtain a masked and repaired image; optimize the masked and repaired image using a lightweight deep learning model to obtain the enhanced final image.

[0071] For example, the content type of the reconstructed image data is analyzed to distinguish between static images and video sequences. For missing regions in the reconstructed image data of static images, features of surrounding intact pixel blocks are extracted and interpolated based on spatial correlation to fill in the missing content. For missing regions in the reconstructed image data of video sequences, motion-compensated copying is performed using temporal correlation features between consecutive frames to fill in the missing content, resulting in a masked and repaired image.

[0072] The defective regions in the masked and repaired image are identified, their local texture features and global structural features are extracted and input into a preset lightweight deep learning model. After model feature reconstruction and optimization, an optimized region that meets the requirements of visual perception is generated. The optimized region is then naturally merged with the non-defective regions in the masked and repaired image to obtain the enhanced final image.

[0073] Example 2

[0074] This application also provides an image enhancement method based on satellite communication, including: dynamically configuring the water level of a circular buffer based on the real-time network packet loss rate, setting water level lines for triggered and forced processing to form a dynamic buffer. This method can adapt to the fluctuating characteristics of satellite channels, avoid excessive buffer space occupation and reduce memory usage, and also promptly advance data processing through handover condition detection corresponding to the water level lines, helping to reduce end-to-end processing latency. After receiving image data packets with content importance classification tags and forward error correction codes, combining the two for packet loss recovery can strengthen recovery protection for important content, helping to improve the success rate of key frame recovery, and also avoid resource redundancy caused by fixed-ratio error correction, which is beneficial to optimizing bandwidth utilization.

[0075] By leveraging the coarse-grained timestamps and fine-grained sequence numbers of the recovered data packets for time-bucket distribution and sorting within each bucket, and then combining this with dynamic buffer waterline detection to determine handover conditions and transfer the data to the image processing unit to obtain reconstructed image data, this process makes data reconstruction more efficient and further reduces end-to-end processing time. Furthermore, by performing content-aware error masking on the lost areas of the reconstructed image data, and then optimizing the masked and repaired image using a lightweight deep learning model, the visual quality of the image can be improved while controlling resource consumption.

[0076] In one embodiment, the masked restoration image is optimized using a lightweight deep learning model to obtain the enhanced final image, including:

[0077] (1) Visual saliency analysis was performed on the masked restoration image to generate a visual attention heatmap.

[0078] For example, the masked restoration image is preprocessed to remove redundant interference information and retain the core features and effective content of the image. Then, the pixel grayscale distribution features, texture direction features and spatial structure association features of the masked restoration image are extracted. Based on the correlation of various features and the visual perception priority, the visual salience weight of each region is calculated. The visual importance of different regions is presented intuitively according to the weight distribution, and a visual attention heatmap is generated.

[0079] Among them, the visual attention heatmap is an image representation that intuitively reflects the visual importance level of each region in the masked restoration image, highlighting key areas that are easily perceived by the eyes.

[0080] (2) Based on the visual attention heatmap, multi-scale feature extraction is performed on the defect area in the masking repair image to generate a multi-scale defect feature map containing local texture features and global structural features.

[0081] For example, using a visual attention heatmap as the core guide, the defective regions with high visual weight and quality defects in the masked restoration image are delineated. Corresponding feature extraction windows and extraction parameters are set for different feature scales. At a fine scale, local texture features such as edge details and texture particles of the defective region are captured. At a coarse scale, global structural features such as the spatial position relationship and structural contour association between the defective region and the overall masked restoration image are mined. The local texture features and global structural features extracted at different scales are integrated and normalized to generate a multi-scale defect feature map containing local texture features and global structural features.

[0082] Among them, the multi-scale defect feature map is a collection representation that integrates the relevant features of the defect region at different feature scales, and covers feature information at both the local texture level and the global structure level.

[0083] (3) Adversarial feature reconstruction of multi-scale defect feature maps is performed using a pre-defined generative adversarial network model to generate perceptually optimized image regions.

[0084] For example, multi-scale defect feature maps are input into a pre-defined generative adversarial network (GAN) model. After receiving the multi-scale defect feature maps, the generator performs preliminary reconstruction of the defect features through multi-level feature transformation, residual connections, and cross-scale feature fusion operations, generating candidate reconstruction regions. The discriminator compares the candidate reconstruction regions output by the generator with standard defect-free image regions and outputs a realism evaluation result. Based on the discriminator's evaluation result, the generator adjusts its internal parameters, iteratively optimizes, corrects feature deviations, and supplements reasonable feature information until a perceptually optimized image region conforming to the laws of visual perception is generated.

[0085] The core components of the generative adversarial network model include a generator and a discriminator, which achieve high-quality feature reconstruction through a mechanism of mutual adversarial and mutual optimization.

[0086] (4) The image region of the perception optimization is fused with the non-defect region in the masking repair image in the gradient domain to obtain the enhanced final image.

[0087] For example, gradient information such as gradient direction and gradient magnitude are calculated for the perceptually optimized image region to construct its gradient field. Similarly, gradient information for the non-defective regions in the masked restoration image is calculated to construct their gradient fields. By comparing and analyzing the boundary gradient differences and internal gradient distribution patterns of the two gradient fields, fusion strategies such as gradient smoothing adjustment and boundary gradient transition are employed to coordinate and adapt the two gradient fields, eliminating abrupt gradient changes between regions. Based on the adapted fused gradient field, image pixel values ​​are reconstructed, achieving a visually seamless connection between the perceptually optimized image region and the non-defective regions in the masked restoration image, resulting in the enhanced final image.

[0088] Among them, gradient domain fusion is a technique for fusion of regions based on image gradient features. Its core is to achieve visual coherence of the fused region by coordinating the gradient distribution of different regions.

[0089] In one embodiment, gradient domain fusion is performed between the perceptually optimized image region and the non-defective region in the masking repair image to obtain the enhanced final image, including:

[0090] (1) Construct boundary gradient fields for the image regions of the perception optimization and the non-defect regions in the masking repair image to generate the region boundary gradient fields.

[0091] For example, the physical boundary contour between the image region of the perception optimization and the non-defect region in the masking restoration image is located. For each spatial location point on the boundary contour, the gradient direction and gradient magnitude of the image region of the perception optimization at that location point are collected, and the gradient direction and gradient magnitude of the non-defect region in the masking restoration image at that location point are collected. The two types of gradient information corresponding to each location point are spatially correlated. Then, according to the spatial arrangement order of the boundary contour, the correlated gradient information of all location points is orderly integrated to form a gradient feature set covering the entire boundary range, and the region boundary gradient field is generated.

[0092] Among them, the region boundary gradient field is a feature set that integrates two types of gradient information at each location point, which is the boundary between the image region of the coverage-sensory optimization and the non-defect region in the masking repair image. It is used to reflect the gradient distribution state at the boundary of the two regions.

[0093] (2) Based on the Poisson equation, the gradient field of the region boundary is optimized across the region to generate a seamless transition fused gradient field.

[0094] For example, the gradient constraint boundary conditions of the Poisson equation are clearly defined, and the gradient information of each location point in the gradient field of the region boundary is used as the boundary input condition of the Poisson equation. The solution objective of the Poisson equation is set as coordinating the gradient distribution inside the image region of the perception optimization and the non-defect region in the mask repair image, so that the gradient inside the region and the boundary gradient remain continuous and consistent. The Poisson equation is solved to obtain the gradient distribution result across the region. The gradient smoothness of the result is checked, the locally existing gradient abrupt parts are corrected, and a seamless transition fused gradient field is generated.

[0095] Among them, the seamless transition fusion gradient field is a set of gradient features that, after solving the Poisson equation and smoothing verification, can achieve natural connection between the gradients of the image region in the perception optimization and the non-defect region in the masked restoration image.

[0096] (3) Reconstruct the gradient field of the seamlessly transitioned fused gradient field to generate the enhanced final image. The expression of the enhanced final image is:

[0097]

[0098] in, This represents the final enhanced image. This indicates the gradient field reconstruction operation. Indicates the integration of regional spatial domains. Indicates the fused gradient field. Represents the gradient field at the region boundary. Represents the gradient operator, This represents minimizing the solution of a variational problem. This represents a two-dimensional spatial integral.

[0099] For example, a minimization variational problem is constructed with a seamless transition fused gradient field as a constraint. The core objective of this problem is to find the spatial distribution of image pixel values ​​that satisfies the gradient characteristics of the seamless transition fused gradient field. By solving the minimization variational problem, the spatial distribution of pixel values ​​that meets the gradient constraints is obtained. Then, the gradient field reconstruction operation is performed, and the spatial distribution of pixel values ​​is mapped and rendered pixel by pixel according to the spatial dimension of the image. The rendered image is then visually verified to ensure that there are no missing pixels or gradient anomalies. Finally, the enhanced final image is generated.

[0100] Among them, the gradient field reconstruction operation is the process of converting the spatial distribution of pixel values ​​corresponding to the gradient feature set into a complete visual image; the fusion region spatial domain is the spatial range for gradient fusion between the perceptually optimized image region and the non-defective region in the masked repair image.

[0101] like Figure 2 As shown, a pre-defined generative adversarial network model is used to perform adversarial feature reconstruction on multi-scale defect feature maps, generating perceptually optimized image regions, including:

[0102] S201: Multi-level residual reconstruction is performed on the multi-scale defect feature map by the generator of the generative adversarial network model to generate residual reconstruction feature map.

[0103] For example, the generator of the generative adversarial network model is invoked, and the hierarchical division of the multi-level residual units built into the generator is determined, with each level corresponding to a specific feature scale. The multi-scale defect feature map is decomposed into sub-feature maps of different scales, and the residual units of the corresponding level are matched according to the scale of the sub-feature maps. After receiving the sub-feature map of the corresponding scale, each residual unit extracts the defect feature region in the sub-feature map, and then preserves the basic structural information of the sub-feature map through residual connections. At the same time, the missing details of the defect region are filled in by the convolution operation within the unit. After processing, each residual unit outputs a repair sub-feature map of the corresponding scale. The repair sub-feature maps output by each level of residual units are sorted according to the original scale hierarchy, and the repair sub-features of different scales are fused through feature concatenation operations to generate a residual reconstruction feature map.

[0104] Among them, the multi-level residual unit is a number of processing units divided according to the feature scale within the generator, and the residual reconstruction feature map is a feature set that integrates the repair sub-features of each scale with the basic structural information.

[0105] S202: Perform cross-scale feature aggregation on the residual reconstruction feature map and the multi-scale defect feature map to generate an aggregated feature map.

[0106] For example, the scale levels of the residual reconstruction feature map and the multi-scale defect feature map are clearly defined, and the scale correspondence between the two is established. For each corresponding scale level, the sub-feature map of the residual reconstruction feature map at that scale and the sub-feature map of the multi-scale defect feature map at that scale are aligned in channel dimension to ensure that the number of feature channels of the two types of sub-feature maps is consistent.

[0107] Through element-level weighted fusion, weights are assigned to the two types of sub-features based on their effectiveness. The repair information of the residual reconstruction feature map sub-features and the original structural information of the multi-scale defect feature map sub-features are preserved, resulting in a fused sub-feature map for that scale. All fused sub-feature maps at all scales are then processed to achieve uniform resolution in order of scale from fine to coarse. Finally, a feature stacking operation is used to integrate the fused sub-features from all scales, generating an aggregated feature map.

[0108] Among them, element-level weighted fusion is an operation method that allocates weights according to the effectiveness of features and fuses two types of sub-feature information, while aggregated feature map is a comprehensive feature set that integrates sub-features fused at various scales.

[0109] S203: Spatial-aware reconstruction of the aggregated feature map is performed through the output layer of the generator to generate a perceptually optimized image region.

[0110] For example, the generator's output layer is invoked, activating the spatial awareness unit built into the output layer. This unit first performs spatial dimension parsing on the aggregated feature map, extracting the spatial coordinate information of the aggregated feature map and the feature intensity at the corresponding position. Based on visual perception prior rules, the feature intensity at different spatial positions is adjusted to make the feature details of high visual weight regions richer and the features of low visual weight regions smoother.

[0111] The adjusted features are mapped to the pixel space, and the abstract features are transformed into a specific pixel value distribution through deconvolution operations within the unit. Simultaneously, the range of pixel values ​​is calibrated to the effective display range of the image. Local consistency checks are performed on the mapped pixel regions to correct abrupt differences between adjacent pixels, ensuring texture continuity within the region and generating a perceptually optimized image region.

[0112] Among them, the spatial perception unit is a processing unit in the output layer that adjusts features according to visual rules and realizes feature-to-pixel mapping. The image region of perception optimization is a repaired image region with continuous texture and conforming to visual rules.

[0113] In one embodiment, content-aware error masking is performed on missing regions in the reconstructed image data to obtain a masked and repaired image, including:

[0114] (1) Spatial interpolation is performed to fill the missing static image regions in the reconstructed image data to obtain the masked repair image.

[0115] For example, content type identification is performed on the reconstructed image data to filter out static image data. Through pixel distribution consistency verification and structural feature analysis, the spatial boundaries, size, and shape features of the lost static image region in the reconstructed image data are located. Intact pixel blocks surrounding the lost static image region are extracted, and their texture direction, grayscale gradient distribution, and spatial adjacency are analyzed. Based on these features, suitable pixel blocks with texture continuity and grayscale coordination with the lost static image region are selected. A spatial interpolation strategy matching texture features is used to gradually fill the lost static image region with the feature information of the suitable pixel blocks. During the filling process, pixel grayscale values ​​and texture details are calibrated in real time to ensure a natural visual transition and smooth structural connection between the lost static image region and the surrounding intact pixel regions, resulting in a masked and repaired image.

[0116] Spatial interpolation filling is a processing method that supplements the pixel information of the lost area by using the visual features of intact pixel blocks around the lost area of ​​a static image through a smooth transition. The core is to maintain the consistency of texture and grayscale between the filled area and the surrounding area.

[0117] (2) Perform temporal motion compensation processing on the lost areas of the video sequence in the reconstructed image data to obtain the masked and repaired image.

[0118] For example, content type recognition is performed on the reconstructed image data to distinguish video sequence data. Through intra-frame pixel integrity detection and inter-frame correlation analysis, the frame location, spatial coverage, and region contour of the lost video sequence in the reconstructed image data are located. The preceding and following valid frames containing the lost video sequence region are extracted. Motion vector estimation is performed on the preceding and following valid frames to analyze the motion trajectory and positional offset information of the target region within the frame. Reference pixel regions in the preceding and following valid frames that have similar features to the lost video sequence region are matched.

[0119] Compensation coefficients are calculated based on motion vectors and position offset information. Pixel data from the reference pixel area is then mapped to the lost area of ​​the video sequence according to the compensation coefficients. Simultaneously, the brightness and contrast parameters of the mapped pixels are adjusted to adapt to the visual environment of the current frame, ensuring that the pixels in the lost area of ​​the video sequence are connected and continuous with those in other areas within the frame, thus obtaining a masked and repaired image.

[0120] Among them, temporal motion compensation processing utilizes the temporal correlation between video sequence frames to fill the lost area through motion estimation and pixel mapping. The core is to achieve cross-frame pixel compensation based on the motion information of consecutive frames.

[0121] In one embodiment, spatial interpolation is performed to fill in the static image loss region in the reconstructed image data to obtain a masked restoration image, including:

[0122] (1) Perform edge perception analysis on the lost area of ​​the static image to generate an edge direction distribution map describing the texture direction.

[0123] For example, the spatial boundary of the lost area in a static image is located, the pixel gradient information of the intact pixel area around the lost area is extracted, the texture extension direction of the surrounding area is analyzed based on the gradient information, and the turning and continuous features of the texture are identified. The texture direction features are marked according to their spatial positions to form a visual feature map that can intuitively reflect the texture distribution pattern around the lost area of ​​the static image, and generate an edge direction distribution map describing the texture direction.

[0124] Among them, the edge direction distribution map is a visual feature map that marks the extension direction, turning point and continuity features of the texture around the lost area of ​​the static image, which is used to guide the selection of subsequent intact pixel blocks.

[0125] (2) Based on the edge direction distribution map, select a set of intact pixel blocks that meet the preset texture continuity threshold in the area surrounding the lost area.

[0126] For example, using the edge direction distribution map as a reference, the texture direction benchmark of the area surrounding the lost area of ​​the static image is determined. The intact pixel blocks in the area surrounding the lost area of ​​the static image are extracted one by one. The texture features of each intact pixel block are analyzed. The texture features are compared with the texture direction in the edge direction distribution map to verify the texture continuity of each intact pixel block with the surrounding area. The intact pixel blocks whose texture continuity meets the preset texture continuity threshold are selected. The intact pixel blocks that meet the above conditions are summarized to form a set of intact pixel blocks that meet the preset texture continuity threshold.

[0127] Among them, the set of intact pixel blocks that meet the preset texture continuity threshold is a set of intact pixel blocks in the area surrounding the lost area of ​​the static image that meet the threshold requirement for the continuity with the surrounding texture.

[0128] (3) Perform multi-directional dynamic weighted interpolation on the set of intact pixel blocks that meet the preset texture continuity threshold to generate repaired pixel regions. The expression for the repaired pixel regions is:

[0129]

[0130] in, Indicates repairing pixel areas. This represents the number of intact pixel blocks that meet the threshold. Indicates the first Dynamic weights of individual pixel blocks Indicates the first A complete pixel block, This represents the gradient of the edge direction distribution map. Represents pixel blocks gradient, This represents the function for calculating directional similarity.

[0131] For example, the feature weights of each intact pixel block in the set of intact pixel blocks that meet the preset texture continuity threshold are parsed. The gradient features of the edge direction distribution map and the gradient features of each intact pixel block are combined to calculate the directional similarity between the two. The dynamic weights of each intact pixel block are assigned according to the directional similarity results. At the same time, the pixel information of the intact pixel blocks is interpolated from multiple texture extension directions. The pixel information obtained by interpolation in different directions is integrated according to the dynamic weights to form a set of pixel information covering the lost area of ​​the static image, thereby generating the repaired pixel area.

[0132] Among them, multi-directional dynamic weighted interpolation processing is a pixel filling method that combines directional similarity to assign weights, interpolates from multiple texture directions, and integrates the results. The repaired pixel region is a set of pixel information that covers the lost area of ​​the static image after this processing.

[0133] (4) The repaired pixel area is integrated with the lost area of ​​the static image by topological structure matching to obtain the masked repair image.

[0134] For example, the spatial topology of the lost region in the static image is analyzed, including the shape, boundary contour and spatial dimension features of the region. At the same time, the topological features of the repair pixel region are analyzed. The topology of the repair pixel region is aligned and matched with the topology of the lost region in the static image. The pixel position of the repair pixel region is adjusted to fit the spatial contour of the lost region in the static image. The texture connection details between the repair pixel region and the surrounding area of ​​the lost region in the static image are calibrated. The adjusted repair pixel region is filled into the lost region in the static image to obtain the masked repair image.

[0135] Among them, topology matching integration is a processing method that aligns, adjusts and fills the topology of the repaired pixel region with the topology of the lost region in the static image. The masked repair image is the image obtained after this integration process, which completes the filling of the lost region.

[0136] In one embodiment, based on the coarse-grained timestamp and fine-grained sequence number carried by the recovered image data packets, time-bucket distribution is performed on the recovered image data packets to obtain time-bucket group data, including:

[0137] (1) Parse the coarse-grained timestamp carried by the recovered image data packet and generate a time bucket identifier.

[0138] For example, the core identification information of the coarse-grained timestamp carried by the recovered image data packet is extracted, the core identification information is subjected to integrity verification and format standardization processing, redundant interference information is removed, key time features are retained, and then the standardized coarse-grained timestamp is converted into an identification code with a unique correspondence according to the preset time division rules to generate a time bucket identifier.

[0139] The time bucket identifier is a unique identifier code that corresponds to the coarse-grained timestamp of the recovered image data packet, used to achieve accurate matching between the data packet and the time bucket storage unit.

[0140] (2) Based on the time bucket identifier, the recovered image data packets are dynamically allocated to the corresponding time bucket storage units.

[0141] For example, a mapping relationship is established between time bucket identifiers and time bucket storage units. Based on this mapping relationship, the target time bucket storage unit corresponding to the recovered image data packet is queried. The storage status and available space of the target time bucket storage unit are verified. After confirming that the status is normal and the space is sufficient, the recovered image data packet is transmitted to the target time bucket storage unit. At the same time, the storage ownership information of the data packet is recorded to complete the dynamic allocation operation.

[0142] The time bucket storage unit is an independent storage medium used to store the recovered image data packets under the same time characteristics, and each unit corresponds to a unique time bucket identifier.

[0143] (3) For data packets in the same time bucket storage unit, perform sequence continuity verification based on the fine-grained sequence number carried by the data packet, and remove data packets with abnormal sequence number jumps.

[0144] For example, all recovered image data packets within the same time bucket storage unit are organized, the fine-grained sequence number carried by each data packet is extracted, the data packets are initially sorted according to the natural sorting rules of the sequence number, and then the continuity of the fine-grained sequence number of the sorted data packets is checked one by one. The sequence number difference of adjacent data packets is compared, and data packets with abnormal sequence number jumps that exceed the normal sequence interval are marked. The marked data packets with abnormal sequence number jumps are removed from the time bucket storage unit, and valid data packets with continuous sequences are retained.

[0145] Among them, sequence continuity verification is a verification method that judges whether the data packet sequence is complete by comparing the interval and sorting relationship of fine-grained sequence numbers. Its core is to filter out valid data packets with normal sequences.

[0146] (4) Integrate the valid data packets of all time bucket storage units to obtain time bucket group data.

[0147] For example, all time bucket storage units are traversed, and the valid data packets retained in each time bucket storage unit after sequence continuity verification are collected. The valid data packets are classified and grouped according to the correspondence of time bucket identifiers. The number and storage information of valid data packets in each category are recorded. All the classified and grouped valid data packet sets are integrated as a whole to form a structured data packet set, thus obtaining the time bucket grouped data.

[0148] Among them, the time-bucket grouped data is a structured data set formed by classifying, collecting and integrating all valid data packets in the time-bucket storage units according to the time-bucket identifier.

[0149] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0150] Example 3:

[0151] In this embodiment, as Figure 3 As shown, this application also provides an image enhancement system 300 based on satellite communication, the system 300 comprising:

[0152] The dynamic water level configuration module 301 is used to establish a ring buffer and dynamically configure the water level of the ring buffer based on the real-time network packet loss rate to obtain a dynamic buffer with a triggered processing water level and a forced processing water level.

[0153] The hierarchical error correction and recovery module 302 is used to receive image data packets that have been marked with content importance hierarchy and the generated forward error correction code; and to perform packet loss recovery based on the image data packets marked with content importance hierarchy and the forward error correction code to obtain the recovered image data packets;

[0154] The hybrid reconstruction decision module 303 is used to distribute the recovered image data packets into time buckets based on the coarse-grained timestamp and fine-grained sequence number carried by the recovered image data packets, to obtain time bucket grouped data; sort the time bucket grouped data within each bucket to obtain sorted time bucket data; and perform handover condition detection on the sorted time bucket data based on the trigger processing water level and the forced processing water level of the dynamic buffer. When the handover condition is met, the sorted time bucket data is handed over to the preset image processing unit to obtain reconstructed image data.

[0155] The perceptual enhancement processing module 304 is used to perform content-aware error masking on the missing regions in the reconstructed image data to obtain a masked and repaired image; and to optimize the masked and repaired image using a lightweight deep learning model to obtain the enhanced final image.

[0156] Specifically, the dynamic water level configuration module 301 establishes a circular buffer, determines its storage structure and data flow rules, and ensures that data can be accessed and reused in an orderly manner. It collects real-time network packet loss rate variation characteristics and, based on these characteristics, formulates configuration rules for trigger processing water level lines and forced processing water level lines, clarifying the adjustment conditions and corresponding threshold ranges for both types of water level lines. The water level parameters of the circular buffer are dynamically adjusted according to the configuration rules. After completing all adjustment steps, a dynamic buffer with trigger processing water level lines and forced processing water level lines is obtained.

[0157] The dynamic buffer is a circular buffer after the water level parameters are configured. It contains two types of threshold indicators used to determine the timing of data processing: the trigger processing water level and the forced processing water level.

[0158] The hierarchical error correction and recovery module 302 receives image data packets that have been categorized by content importance and the generated forward error correction code. It performs integrity checks on both the categorized image data packets and the generated forward error correction code. After confirming that neither type of data is missing information, it identifies the loss status of the categorized image data packets and assigns recovery priorities to data packets of different levels based on the content importance categorization results. It then calls the generated forward error correction code, matches it with the lost data packets of the corresponding priority, and uses the redundant information of the generated forward error correction code to perform data completion and structural reconstruction on the lost data packets. After the reconstruction operation is completed, it verifies the content validity of the data packets to obtain the recovered image data packets.

[0159] Among them, the image data packets that have been marked with content importance are image data carriers divided according to the visual or information priority of the content, and the generated forward error correction code is a redundant information encoding used to complete the missing data.

[0160] The hybrid recombination decision module 303 extracts the coarse-grained timestamp carried by the recovered image data packets, parses the temporal features of the coarse-grained timestamp and generates a corresponding time bucket identifier, and allocates the recovered image data packets to the matching time bucket storage units according to the time bucket identifier. For the recovered image data packets in the same time bucket storage unit, the fine-grained sequence number carried by each data packet is extracted and arranged in an ordered manner within the bucket to obtain the sorted time bucket data.

[0161] The system retrieves the criteria for triggering and forcing the processing thresholds of the dynamic buffer, and performs dual checks on the data accumulation and storage duration of the sorted time bucket data. When the sorted time bucket data meets the criteria, it transfers the sorted time bucket data to the preset image processing unit according to the preset specifications. The preset image processing unit then integrates the received sorted time bucket data and restructures it to obtain reconstructed image data.

[0162] The time bucket storage unit is an independent storage medium used to collect image data packets recovered under the same time characteristics, and the preset image processing unit is a processing medium responsible for integrating and reorganizing the received data.

[0163] The perceptual enhancement processing module 304 performs content type recognition on the reconstructed image data, distinguishing between static image loss areas and video sequence loss areas. For static image loss areas, it extracts surrounding intact pixel blocks and performs interpolation filling based on spatial correlation. For video sequence loss areas, it performs motion-compensated copying using the temporal correlation features of consecutive valid frames. After completing the filling operation for all loss areas, it obtains the masked restoration image. The masked restoration image then performs defect region identification, extracts feature information of the defective regions, and inputs it into a lightweight deep learning model for feature reconstruction and optimization. The optimized regions are then naturally fused with the non-defective regions in the masked restoration image to obtain the enhanced final image.

[0164] The lightweight deep learning model is a computational model used to optimize the features of the masked image. The enhanced final image is the result of filling in the missing regions and optimizing the features.

[0165] The perception enhancement processing module 304 is also used for:

[0166] Visual saliency analysis was performed on the masked restoration image to generate a visual attention heatmap;

[0167] Based on visual attention heatmap, multi-scale feature extraction is performed on the defect region in the masking repair image to generate a multi-scale defect feature map containing local texture features and global structural features.

[0168] Adversarial feature reconstruction is performed on multi-scale defect feature maps using a pre-defined generative adversarial network model to generate perceptually optimized image regions.

[0169] The image region optimized by perception is fused with the non-defect region in the masked repair image in the gradient domain to obtain the enhanced final image.

[0170] The perception enhancement processing module 304 is also used for:

[0171] Boundary gradient fields are constructed for the image regions of the perception optimization and the non-defect regions in the masking restoration image to generate the region boundary gradient fields.

[0172] Based on the Poisson equation, the gradient field of the region boundary is optimized across regions to generate a seamlessly transitioned fused gradient field.

[0173] Gradient field reconstruction is performed on the seamlessly transitioned fused gradient field to generate the enhanced final image. The expression for the enhanced final image is:

[0174]

[0175] in, This represents the final enhanced image. This indicates the gradient field reconstruction operation. Indicates the integration of regional spatial domains. Indicates the fused gradient field. Represents the gradient field at the region boundary. Represents the gradient operator, This represents minimizing the solution of a variational problem. This represents a two-dimensional spatial integral.

[0176] The perception enhancement processing module 304 is also used for:

[0177] The generator of the generative adversarial network model performs multi-level residual reconstruction on the multi-scale defect feature map to generate residual reconstructed feature maps.

[0178] The residual reconstruction feature map and the multi-scale defect feature map are aggregated across scales to generate an aggregated feature map.

[0179] Spatial-aware reconstruction of the aggregated feature map is performed through the generator's output layer to generate perceptually optimized image regions.

[0180] The perception enhancement processing module 304 is also used for:

[0181] Spatial interpolation is performed to fill in the missing static image regions in the reconstructed image data to obtain a masked and repaired image;

[0182] Temporal motion compensation is performed on the lost regions of the video sequence in the reconstructed image data to obtain the masked and restored image.

[0183] The perception enhancement processing module 304 is also used for:

[0184] Edge-aware analysis is performed on the missing regions of static images to generate an edge direction distribution map describing the texture direction;

[0185] Based on the edge orientation distribution map, a set of intact pixel blocks that meet the preset texture continuity threshold are selected in the area surrounding the lost region;

[0186] Multi-directional dynamic weighted interpolation is performed on the set of intact pixel blocks that meet the preset texture continuity threshold to generate repaired pixel regions. The expression for the repaired pixel regions is:

[0187]

[0188] in, Indicates repairing pixel areas. This represents the number of intact pixel blocks that meet the threshold. Indicates the first Dynamic weights of individual pixel blocks Indicates the first A complete pixel block, This represents the gradient of the edge direction distribution map. Represents pixel blocks gradient, This represents the function for calculating directional similarity.

[0189] The repaired pixel region is integrated with the lost region of the static image through topological structure matching to obtain the masked repair image.

[0190] Hybrid recombination decision module 303 is also used for:

[0191] Parse the coarse-grained timestamps carried in the recovered image data packets to generate time bucket identifiers;

[0192] Based on the time bucket identifier, the recovered image data packets are dynamically allocated to the corresponding time bucket storage units;

[0193] For data packets within the same time bucket storage unit, perform sequence continuity verification based on the fine-grained sequence number carried by the data packet, and remove data packets with abnormal sequence number jumps;

[0194] By integrating the valid data packets from all time-bucket storage units, the time-bucket grouped data is obtained.

[0195] In one embodiment, the receiving end establishes a circular buffer with dynamic watermark management; a "trigger processing watermark" and a "forced processing watermark" are set. The system monitors the network packet loss rate in real time: when the instantaneous packet loss rate is lower than a first threshold, the "trigger processing watermark" is set to 50% of the total buffer capacity. When the instantaneous packet loss rate is higher than the first threshold, the "trigger processing watermark" is dynamically lowered to 30% to reduce processing latency, although some recovery rate may be sacrificed due to insufficient waiting time. When the amount of data in the buffer reaches the "forced processing watermark" (e.g., 80%), the subsequent processing flow is immediately triggered to prevent buffer overflow.

[0196] Before sending image data, the sending end marks the data packets according to their importance. For example, I-frame data packets in the H.264 bitstream are marked as "high priority", P-frames as "medium priority", and B-frames as "low priority".

[0197] The sending end employs asymmetric FEC coding: For "high priority" data blocks, high-redundancy FEC coding is used (e.g., generating 3 redundant packets for every 4 data packets). For "medium priority" data blocks, medium-redundancy FEC coding is used (e.g., generating 2 redundant packets for every 6 data packets). For "low priority" data blocks, low-redundancy or no FEC coding is used. When the receiving end detects data packet loss, it first attempts to recover the data using the received FEC redundant packets, with the recovery priority being: high > medium > low.

[0198] Each data packet carries both a coarse-grained timestamp (e.g., UTC seconds) and a fine-grained sequence number (incrementing within the current second). The reassembly mechanism is as follows: Data packets are first distributed to different "time buckets" based on their coarse-grained timestamps. Within each "time bucket," data packets are precisely ordered based on their fine-grained sequence numbers. A "time bucket" is considered complete and handed over to the image processing module when it meets one of the following conditions:

[0199] a. Receive all data packets with consecutive sequence numbers within the time bucket.

[0200] b. The "maximum waiting time" of the time bucket is reached (this time is dynamically calculated based on the typical jitter range of satellite communication).

[0201] c. Subsequent time buckets of this time bucket have started receiving data, and the data volume of this time bucket has reached the "trigger processing water level".

[0202] For lost data packets that cannot be recovered even after FEC, a content-based error masking technique is employed: for static images, spatial interpolation is used to fill in the missing macroblocks using neighboring intact pixel blocks. For video sequences, motion-compensated copying is performed using the temporal correlation between consecutive frames to fill in the missing macroblocks. After image decoding, for mosaic or blurred areas caused by data loss, a lightweight deep learning model (such as edge enhancement using a Generative Adversarial Network (GAN)) is used for local image enhancement to subjectively improve image quality.

[0203] Through adaptive FEC and dynamic water level management, the method can flexibly cope with drastic fluctuations in satellite channel quality, minimizing bandwidth waste and processing latency while ensuring recovery success rate.

[0204] The hybrid reassembly mechanism avoids fixed timeouts and, combined with the concept of a time bucket, can quickly determine data integrity, significantly reducing end-to-end processing latency and meeting near real-time processing requirements. Through asymmetric FEC based on data importance and subsequent content-aware error masking and quality enhancement, the method prioritizes the integrity of critical data and maximizes visual quality restoration when data is indeed lost, improving user experience. Dynamic buffer management and differentiated FEC strategies enable the system to operate efficiently even in environments with limited memory and bandwidth resources (such as satellite ground stations).

[0205] Example 4:

[0206] In this embodiment, the present application also provides a computer device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0207] Example 5:

[0208] In this embodiment, the present application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps in the above-described method embodiments.

[0209] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0210] The above-described embodiments are merely illustrative of several implementation methods of the embodiments of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the embodiments of this application, and these modifications and improvements all fall within the protection scope of the embodiments of this application.

Claims

1. An image enhancement method based on satellite communication, characterized in that, The method includes: A circular buffer is established, and the water level of the circular buffer is dynamically configured based on the real-time network packet loss rate to obtain a dynamic buffer configured with trigger processing water level and forced processing water level. Receive image data packets that have been marked with content importance classification and the generated forward error correction code; perform packet loss recovery based on the image data packets marked with content importance classification and the forward error correction code to obtain the recovered image data packets; Based on the coarse-grained timestamp and fine-grained sequence number carried by the recovered image data packet, the recovered image data packet is distributed into time buckets to obtain time bucket group data; the time bucket group data is sorted within each bucket to obtain sorted time bucket data; based on the trigger processing water level and the forced processing water level of the dynamic buffer, the sorted time bucket data is subjected to handover condition detection; when the handover condition is met, the sorted time bucket data is handed over to the preset image processing unit to obtain reconstructed image data; The missing regions in the reconstructed image data are subjected to content-aware error masking to obtain a masked and repaired image; the masked and repaired image is then optimized using a lightweight deep learning model to obtain the enhanced final image.

2. The image enhancement method based on satellite communication according to claim 1, characterized in that, The step of optimizing the masked and repaired image using a lightweight deep learning model to obtain the enhanced final image includes: Visual saliency analysis is performed on the masked and repaired image to generate a visual attention heatmap; Based on the visual attention heatmap, multi-scale feature extraction is performed on the defect region in the masking repair image to generate a multi-scale defect feature map containing local texture features and global structural features. Adversarial feature reconstruction is performed on the multi-scale defect feature map using a pre-defined generative adversarial network model to generate a perceptually optimized image region. The image region optimized by perception is fused with the non-defect region in the masked repair image in the gradient domain to obtain the enhanced final image.

3. The image enhancement method based on satellite communication according to claim 2, characterized in that, The step of fusing the perceptually optimized image region with the non-defect region in the masked repair image in the gradient domain to obtain the enhanced final image includes: A boundary gradient field is constructed between the image region optimized by perception and the non-defect region in the masking repair image to generate a region boundary gradient field. Based on the Poisson equation, the gradient field of the boundary of the region is optimized across regions to generate a seamlessly transitioned fused gradient field. Gradient field reconstruction is performed on the seamlessly transitioned fused gradient field to generate the enhanced final image, the expression of which is: in, This represents the final enhanced image. This indicates the gradient field reconstruction operation. Indicates the integration of regional spatial domains. Indicates the fused gradient field. Represents the gradient field at the region boundary. Represents the gradient operator. This represents minimizing the solution of a variational problem. This represents a two-dimensional spatial integral.

4. The image enhancement method based on satellite communication according to claim 2, characterized in that, The step of performing adversarial feature reconstruction on the multi-scale defect feature map using a preset generative adversarial network model to generate a perceptually optimized image region includes: The generator of the generative adversarial network model performs multi-level residual reconstruction on the multi-scale defect feature map to generate a residual reconstruction feature map. The residual reconstruction feature map and the multi-scale defect feature map are subjected to cross-scale feature aggregation to generate an aggregated feature map. The generator's output layer performs spatially perceptual reconstruction on the aggregated feature map to generate the perceptually optimized image region.

5. The image enhancement method based on satellite communication according to claim 1, characterized in that, The step of performing content-aware error masking on the missing regions in the reconstructed image data to obtain a masked and repaired image includes: Spatial interpolation is performed to fill in the lost static image regions in the reconstructed image data to obtain the masked repair image; Temporal motion compensation processing is performed on the lost video sequence regions in the reconstructed image data to obtain the masked restoration image.

6. The image enhancement method based on satellite communication according to claim 5, characterized in that, The step of spatially interpolating and filling the lost static image region in the reconstructed image data to obtain the masked and repaired image includes: Edge-aware analysis is performed on the lost areas of the static image to generate an edge direction distribution map describing the texture direction; Based on the edge direction distribution map, a set of intact pixel blocks that meet the preset texture continuity threshold is selected in the area surrounding the lost region; The set of intact pixel blocks that meet the preset texture continuity threshold is subjected to multi-directional dynamic weighted interpolation to generate repaired pixel regions. The expression for the repaired pixel regions is: in, Indicates the repair of pixel areas. This represents the number of intact pixel blocks that meet the threshold. Indicates the first Dynamic weights of individual pixel blocks Indicates the first A complete pixel block, This represents the gradient of the edge direction distribution map. Represents pixel blocks gradient, This represents the function for calculating directional similarity. The repaired pixel region is integrated with the lost region of the static image through topological structure matching to obtain the masked repair image.

7. The image enhancement method based on satellite communication according to claim 1, characterized in that, The recovered image data packets are distributed into time buckets based on the coarse-grained timestamps and fine-grained sequence numbers carried in the recovered image data packets, resulting in time bucket grouped data, including: Parse the coarse-grained timestamp carried in the recovered image data packet to generate a time bucket identifier; Based on the time bucket identifier, the recovered image data packet is dynamically allocated to the corresponding time bucket storage unit; For data packets within the same time bucket storage unit, perform sequence continuity verification based on the fine-grained sequence number carried by the data packet, and remove data packets with abnormal sequence number jumps; The time-bucket group data is obtained by integrating the valid data packets from all time-bucket storage units.

8. An image enhancement system based on satellite communication, characterized in that, The system includes: The dynamic water level configuration module is used to establish a ring buffer and dynamically configure the water level of the ring buffer based on the real-time network packet loss rate, so as to obtain a dynamic buffer configured with a trigger processing water level and a forced processing water level. The hierarchical error correction and recovery module is used to receive image data packets that have been marked with content importance hierarchy and the generated forward error correction code; and to perform packet loss recovery based on the image data packets that have been marked with content importance hierarchy and the forward error correction code to obtain the recovered image data packets; The hybrid reconstruction decision module is used to distribute the recovered image data packets into time buckets based on the coarse-grained timestamps and fine-grained sequence numbers carried by the recovered image data packets, thereby obtaining time bucket grouped data; sort the time bucket grouped data within each bucket to obtain sorted time bucket data; and perform handover condition detection on the sorted time bucket data based on the trigger processing water level and the forced processing water level of the dynamic buffer. When the handover condition is met, the sorted time bucket data is handed over to a preset image processing unit to obtain reconstructed image data. The perceptual enhancement processing module is used to perform content-aware error masking on the missing regions in the reconstructed image data to obtain a masked and repaired image; and to optimize the masked and repaired image using a lightweight deep learning model to obtain the enhanced final image.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the image enhancement method based on satellite communication according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the image enhancement method based on satellite communication according to any one of claims 1 to 7.

Images