A distributed remote sensing video coding method based on space-time semantic features and diffusion model

Abstract

This invention discloses a distributed remote sensing video encoding and decoding method based on spatiotemporal semantic features and a diffusion model, belonging to the field of remote sensing video compression and distributed encoding and decoding. The encoding end uses a simplified approach to obtain the potential representation of key information in the source-view video frames; the decoding end extracts spatially shared feature representations from the side-information view video frames; a temporal feature extraction module obtains temporal features from the previous two reconstructed frames, which are then processed by a diffusion model to generate semantically richer temporal feature information; a multi-scale spatiotemporal feature mining module fully integrates spatially shared features and temporal features to obtain spatiotemporally semantically enhanced multi-scale feature representations. These feature representations further guide the diffusion model to generate robust reconstructed features and serve as semantic prior conditions to promote the inverse feature transformation module to obtain high-fidelity reconstructed frames, achieving low-bitrate, high-fidelity remote sensing video compression, suitable for distributed remote sensing video compression in satellite constellation scenarios.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing video compression and distributed encoding and decoding technology, and more specifically, relates to a distributed remote sensing video encoding and decoding method based on spatiotemporal semantic features and a diffusion model. Background Technology

[0002] Remote sensing satellite imaging has wide applications in military reconnaissance, battlefield situational awareness, and natural disaster monitoring. With the rapid development of spaceborne imaging technology, the spatiotemporal and spectral resolution of remote sensing imaging has increased dramatically. Spaceborne platforms with limited computing and storage resources cannot efficiently process massive amounts of imaging data. Simultaneously, limited satellite-to-ground bandwidth severely hinders the transmission efficiency of remote sensing image data, restricting the application of remote sensing images. Furthermore, remote sensing satellite imaging technology has evolved from the traditional single-satellite Earth observation mode to a constellation mode with multiple satellites conducting collaborative detection. The amount of satellite imaging data has increased explosively, posing an even greater challenge to the transmission efficiency of limited satellite-to-ground bandwidth. Image compression technology can effectively remove spatiotemporal redundancy in remote sensing imaging and alleviate the bandwidth pressure on satellite-to-ground transmission links, making it a key technology for achieving efficient applications of massive amounts of remote sensing imaging. In the satellite constellation mode, there are overlapping imaging perspectives between each satellite. Therefore, the compression paradigm of a single satellite cannot effectively utilize the perspectives of other satellites to remove spatiotemporal redundancy information, hindering further improvements in compression efficiency.

[0003] Chinese patent applications 201510325575.5 and 201910376206.7 combine traditional compressed sensing technology with distributed theory to design distributed image and video compressed sensing reconstruction methods; however, traditional compressed sensing technology cannot effectively extract key image information, and it is difficult to jointly optimize between modules. Chinese patent applications 202011318867.3 and 202210593207.9 utilize distributed coding theory to propose distributed video coding methods based on adaptive interval overlap factors and robust adaptive DAC codes, respectively; however, traditional methods cannot perform end-to-end joint optimization, resulting in poor quality of reconstructed video frames. With the rapid development of deep learning technology, deep neural networks have achieved significant compression results in the field of image compression due to their powerful representation capabilities. Chinese patent application 202410705931.5 proposes a cross-channel distributed coding based on multi-channel attention, but this method has a complex encoding end, making it unsuitable for resource-constrained satellite terminals, and the decoding end does not effectively utilize inter-frame information, resulting in poor quality of reconstructed video frames. In recent years, generative models have made significant progress in the field of image reconstruction. Chinese patent application 201910413811.7 uses a GAN network at the decoding end to further improve the quality of reconstructed images. However, GAN networks are unstable and difficult to converge during training. In addition, the above-mentioned processing objects are natural images, and there are still great challenges when processing large-format remote sensing images with rich texture information.

[0004] Therefore, the field of remote sensing imaging urgently needs a distributed video compression method that features an extremely simple encoding end and joint decoding using a generative model, suitable for satellite constellations. Summary of the Invention

[0005] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides a distributed remote sensing video encoding and decoding method based on spatiotemporal semantic features and a diffusion model. This solves the technical challenge that existing distributed remote sensing video frame compression methods in satellite constellation scenarios cannot effectively utilize inter-frame and side information as conditional priors to enhance high-fidelity visual perception of reconstructed video frames.

[0006] To achieve the above objectives, according to a first aspect of the present invention, a distributed remote sensing video encoding and decoding method and system based on spatiotemporal semantic features and a diffusion model is provided, including a feature transformation module, a quantization module and an arithmetic encoder deployed at the satellite end, and an arithmetic decoder, a shared feature extraction module, a private feature transformation module, a private feature inverse transformation module, a temporal feature extraction module, a first diffusion model, a multi-scale spatiotemporal feature mining module, a second diffusion model and a feature inverse transformation module deployed at the ground end; The feature transformation module is used to convert video frames from the source viewpoint of the satellite into a compact, continuous latent representation from the pixel domain. The quantization module is used to quantize the compact continuous latent representation to obtain an encodeable discretized latent representation; The arithmetic encoder is used to perform arithmetic encoding on the discretized latent representation to obtain the compressed bitstream of the source view video frame; The arithmetic decoder is used to re-decode the compressed bitstream of the source view video frame into a discretized latent representation; The shared feature extraction module is used to extract common feature information between the source view video frame and the side information view video frame based on the side information view video frame with the same shooting angle as the source view video frame from the ground. The private feature transformation module is used to extract the private feature information of the side information view video frame; The private feature inverse transformation module is used to generate a side information perspective reconstructed video frame and the multi-scale side information features of the side information perspective video frame based on the complete feature representation obtained by the channel splicing operation of the common feature information and the private feature information. The temporal feature extraction module is used to extract the temporal features of the previous two reconstructed video frames as the temporal features of the current video frame; The first diffusion model is used to perform semantic enhancement processing on the temporal features of the current video frame to obtain enhanced temporal features; The multi-scale spatiotemporal feature mining module is used to obtain multi-scale spatiotemporal semantic features based on the temporal features, the common feature information, and the enhanced temporal features. The second diffusion model is used to generate a reconstructed feature representation based on the discretized latent representation, the shared feature information, and the multi-scale spatiotemporal semantic features; The inverse feature transformation module is used to reconstruct the source view reconstructed video frame based on the reconstructed feature representation and the multi-scale spatiotemporal semantic features.

[0007] According to a second aspect of the present invention, an electronic device is provided, comprising: a computer-readable storage medium and a processor; The computer-readable storage medium is used to store executable instructions; The processor is configured to read executable instructions stored in the computer-readable storage medium and execute the method as described in the first aspect.

[0008] According to a third aspect of the invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to perform the method as described in the first aspect.

[0009] According to a fourth aspect of the invention, a computer program product is provided, comprising a computer program or instructions that, when executed by a processor, implement the method described in the first aspect.

[0010] Compared with the prior art, the above-described technical solutions conceived in this invention can achieve the following beneficial effects: (1) The distributed remote sensing video encoding and decoding method proposed in this invention, based on spatiotemporal semantic features and diffusion models, adopts a simplified pixel shuffling downsampling as a nonlinear transformation operation at the encoding end. This transforms the source video frame into a compact latent representation, achieving decoupling from the high-dimensional pixel domain to the low-dimensional latent feature domain. At the same time, this operation effectively removes spatial redundancy while retaining key data in the source video frame. In addition, this operation can achieve parameterless and non-multiplicative accumulation lightweight downsampling, completing the dimensionality transformation only through tensor rearrangement, resulting in extremely low computational and memory overhead, making it very suitable for spaceborne platforms with limited computing power and memory. Furthermore, the motion estimation and motion compensation modules are removed at the encoding end, greatly reducing the computational and memory resources required for the encoded segment.

[0011] (2) The distributed remote sensing video encoding and decoding method based on spatiotemporal semantic features and diffusion model proposed in this invention makes full use of multi-view side information and inter-frame temporal information at the decoding end to generate richer multi-scale spatiotemporal feature priors, which is beneficial for the decoding end to obtain high-quality reconstructed frames. At the same time, the introduction of another view side information brings richer spatial feature information representation, thereby further reducing the information redundancy of the source view video frame at the encoding end and improving the compression efficiency. In addition, the multi-scale spatiotemporal features are used as prior information input into the diffusion model and inverse transform module at the decoding end, so that the diffusion model can generate richer reconstructed feature representations. Meanwhile, the inverse transform module that introduces multi-scale spatiotemporal features also ensures the reconstruction of high-fidelity source view video frames.

[0012] (3) The present invention proposes a distributed remote sensing video encoding and decoding method based on spatiotemporal semantic features and diffusion model. By introducing multi-view side information and generating diffusion model at the decoding end, higher quality reconstructed frames can be obtained compared with the existing methods that only use inter-frame temporal information. Attached Figure Description

[0013] Figure 1 This is a structural diagram of a distributed remote sensing video encoding and decoding system based on spatiotemporal semantic features and a diffusion model, according to an embodiment of the present invention.

[0014] Figure 2 This is a network structure diagram of the multi-scale spatiotemporal feature mining module in an embodiment of the present invention. Detailed Implementation

[0015] To make the design scheme and technical features of the present invention clearer, the technical solutions and implementation methods of the embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be noted that the described embodiments are merely some embodiments of the present invention and do not limit the implementation methods of the present invention. All other embodiments proposed by those skilled in the art based on the embodiments of the present invention with adaptive modifications and without creative effort are within the scope of protection of the present invention. It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0016] Distributed compression utilizes distributed source coding theory to achieve a compression process where multiple nodes independently encode at the encoding end and jointly decode at the decoding end. During the encoding process, side-view information is used to efficiently remove spatiotemporal redundancy, improving compression efficiency. Therefore, distributed compression technology can effectively utilize the perspective information between multiple satellites in a satellite constellation to achieve efficient compressed transmission. Simultaneously, the ground end uses multi-view information for joint decoding to achieve high-fidelity, high-quality video frame reconstruction, significantly reducing on-board resource consumption, improving the transmission efficiency of satellite-to-ground links under limited bandwidth, and promoting the efficient utilization of remote sensing imaging data.

[0017] Based on this, the present invention provides a distributed remote sensing video encoding and decoding system based on spatiotemporal semantic features and a diffusion model, such as... Figure 1 As shown, it includes a feature transformation module, a quantization module, and an arithmetic encoder deployed on the satellite, and an arithmetic decoder, a shared feature extraction module, a private feature transformation module, a private feature inverse transformation module, a temporal feature extraction module, a first diffusion model, a multi-scale spatiotemporal feature mining module, a second diffusion model, and a feature inverse transformation module deployed on the ground. The feature transformation module is used to transform video frames from the satellite source perspective. Transformation from pixel domain to continuous latent representation In some embodiments, to reduce the computational load and data transmission volume at the satellite end, the feature transformation module can perform multi-layer pixel shuffling downsampling on the source view video frame to obtain a continuous latent representation at a unique scale. It should be noted that the source view video frame can be a single-frame remote sensing image acquired independently by the satellite, or a video frame from a video stream acquired by the satellite. Therefore, this scheme can be used for image compression and decompression in independent image encoding and decoding scenarios, as well as for intra-frame encoding and decoding in video encoding and decoding scenarios.

[0018] The quantization module is used to convert the continuous latent representation Quantization is performed to obtain an encodeable discretized latent representation. In some embodiments, during model training, the quantization module can be replaced by adding uniform noise to the continuous latent representation to ensure the backpropagation of gradient information of the network model; during inference, the quantization module can perform quantization operations by rounding.

[0019] An arithmetic encoder is used for the discretized latent representation Arithmetic encoding is performed to obtain the compressed bitstream of the source view video frame. Subsequently, the satellite will transmit the compressed bitstream of the source view video frame to the ground.

[0020] After receiving the compressed bitstream of the source view video frame transmitted from the satellite, the ground terminal will re-decode the compressed bitstream of the source view video frame into a discretized latent representation.

[0021] The shared feature extraction module is used to extract side information from video frames with the same shooting angle as the source viewpoint video frames, based on the ground. Extract the common feature information between the source view video frame and the side information view video frame. In some embodiments, the shared feature extraction module employs a four-layer pixel shuffling downsampling operation to analyze edge information from the perspective of video frames. Perform nonlinear transformation operations to extract video frames from the source's perspective. With edge information perspective video frames Shared feature information .

[0022] The private feature transformation module is used to extract private feature information from the side information viewpoint video frames. In some embodiments, the private feature transformation module employs a four-layer pixel shuffling downsampling operation to process edge information viewpoint video frames. Perform nonlinear transformation operations to extract the private feature information representation of the side information viewpoint video frames. .

[0023] The private feature inverse transformation module is used to transform the shared feature information. With the private feature information Complete feature representation obtained through channel splicing operation Generate side information viewpoint to reconstruct video frames And the multi-scale side information features of the side information viewpoint video frames (e.g.) However, the number of scales is not specifically limited in the embodiments of the present invention. It should be noted that the shared feature extraction module and the private feature transformation module can be constructed based on any existing image feature extraction network (e.g., multi-layer pixel shuffling downsampling network, encoder network, etc.), as long as it has image feature extraction capability. The embodiments of the present invention do not specifically limit this. According to the selected private feature transformation module, a private feature inverse transformation module (e.g., multi-layer pixel shuffling upsampling network, decoder network, etc.) corresponding to its structure can be selected. In some embodiments, in order to simplify the network structure, a multi-layer pixel shuffling downsampling network can be used to construct the shared feature extraction module and the private feature transformation module, so that the shared feature extraction module and the private feature transformation module perform multi-layer pixel shuffling downsampling operations on the side information perspective video frames respectively to obtain the common feature information and private feature information of a unique scale. A multi-layer pixel shuffling upsampling network is used to construct the private feature inverse transformation module, so that the private feature inverse transformation module performs multi-layer pixel shuffling upsampling operations on the complete feature representation of the side information perspective video frames, wherein the first few layers of pixel shuffling upsampling operations can obtain the above-mentioned multi-scale side information features, and the last layer of pixel shuffling upsampling operations can obtain the side information perspective reconstructed video frames.

[0024] The temporal feature extraction module is used to extract the video frames from the current source viewpoint in the buffer. The previous two reconstructed video frames and The temporal characteristics of the current video frame Temporal characteristics .

[0025] The first diffusion model is used to analyze the temporal features of the current video frame. Semantic enhancement is performed to obtain enhanced temporal features. ; The multi-scale spatiotemporal feature mining module is used to mine features based on the temporal features. Common feature information With enhanced temporal features To obtain multi-scale spatiotemporal semantic features To fully utilize spatiotemporal feature information, the shared feature information of the space... With the aforementioned time-series feature information and semantically enhanced temporal feature information By using a multi-scale spatiotemporal feature mining module, multi-scale feature information representations with spatiotemporal feature semantic enhancement are obtained. In some embodiments, such as Figure 2 As shown, the multi-scale spatiotemporal feature mining module includes multiple upsampling modules, downsampling modules, and convolution modules corresponding to different scales; Among them, shared features The output of each upsampling module and the temporal semantic feature representation and The concatenated features obtained by concatenating features with the same scale from the upsampling module are input into the next upsampling module. The concatenated features are then fed into the corresponding scale convolutional module after being added element-wise by the output of the downsampling module corresponding to the corresponding scale through a 1×1 convolutional block, thus obtaining the spatiotemporal semantic features of the corresponding scale. The concatenated features obtained by concatenating the output of the last upsampling module with the temporal feature representation of the same scale are then processed by a 1×1 convolutional block and fed into the corresponding scale downsampling module and convolutional module respectively, thus obtaining the spatiotemporal semantic features of the corresponding scale output by the corresponding convolutional module.

[0026] The second diffusion model is used based on the discretized latent representation. The shared feature information and the multi-scale spatiotemporal semantic features , generate the reconstructed feature representation F.

[0027] The feature inverse transformation module is used to transform the reconstructed feature representation F and the multi-scale spatiotemporal semantic features. The video frames reconstructed from the source perspective are obtained. In some embodiments, the inverse feature transformation module employs a four-layer pixel shuffling upsampling method to perform a nonlinear inverse transformation operation on the reconstructed feature representation F; simultaneously, multi-scale spatiotemporal features are introduced. Prior to this, high-fidelity source perspective video frames are reconstructed. .

[0028] To ensure that the aforementioned distributed remote sensing video encoding and decoding system can simultaneously meet the requirements of minimal encoding and high reconstruction quality at the decoding end, the feature transformation module, shared feature extraction module, private feature transformation module, private feature inverse transformation module, multi-scale spatial feature mining module, and feature inverse transformation module can be trained based on the following loss function:

[0029] in, They represent the discretized latent representations respectively. Private characteristic information Common feature information Entropy estimation, Specifically, it could be the transmission bit rate from the satellite to the ground. , α and β are the reconstruction distortion terms for the source view video frame and the side information view video frame, respectively, used to represent the differences between the source view reconstructed video frame and the source view video frame, as well as the differences between the side information view reconstructed video frame and the side information view video frame; α and β are hyperparameters, and η represents the weight of the reconstruction distortion term for the side information view video frame.

[0030] Since the core objective of this system is the video frame from the source's perspective. x High-quality reconstruction, therefore and The term can serve as a regularization constraint for optimizing the transmission bit rate and distortion at the satellite end.

[0031] Combination Figure 1 This system leverages the synergistic effect of the hard constraints of the network structure and the soft constraints of the aforementioned loss function: structurally, it shares feature information. As the sole shared intermediate feature, it must simultaneously support video frames from the source's perspective. With edge information perspective video frames In dual-path reconstruction, to simultaneously meet the restoration requirements of two tasks, the feature transformation module is forced to utilize shared feature information. Medium-coded source view video frames With edge information perspective video frames Shared structural and semantic information; in the loss function, Item through punishment To reduce coding complexity, the forced feature transformation module removes unique information useful only for a single task, retaining only common features that are crucial for both tasks and can be efficiently compressed. This, combined with the multi-scale feature mining module's multi-scale feature interaction, further enhances the shared feature representation, ultimately enabling effective learning of common features. Additionally, side information perspective video frames... The reconstruction task depends on shared feature information. With private feature information To meet the restoration requirements of the side-information perspective video frame reconstruction task, the private feature transformation module is forced to extract unique information of the side-information perspective video frames that is useful for the task. This allows the subsequent private feature inverse transformation module and multi-scale spatial feature mining module to combine the aforementioned shared and unique information to extract the complete information of the side-information perspective video frames. This helps the diffusion model and feature inverse transformation module achieve high-fidelity source perspective video frame reconstruction based on the discretized latent representation decoded from the satellite end and the complete information of the side-information perspective video frames available from the ground end. Simultaneously, the loss function is set with... This method can also minimize the number of bits transmitted from the satellite to the ground, thereby improving the compression rate of video frames from the source's perspective.

[0032] In summary, the system provided by this invention performs simple feature transformation and arithmetic coding at the encoding end, and extracts common features between the side information perspective and the source perspective, as well as private features of the side information perspective, using the side information perspective video frames at the decoding end. Then, a multi-scale spatial feature mining module effectively extracts rich spatial multi-scale semantic feature representations of the side information perspective video frames based on these common and private features. These multi-scale semantic feature representations contain most of the semantic information of the source perspective video frames, thus effectively reducing the number of bits transmitted at the encoding end of the source perspective video frames and improving the compression transmission efficiency under severely limited satellite-to-ground bandwidth. In addition, these multi-scale semantic feature representations can serve as semantic priors, helping the diffusion model generate more robust reconstruction feature representations and ensuring higher fidelity reconstructed video frames at the inverse transformation module. Thus, high-fidelity source perspective reconstructed video frames can be obtained while reducing the number of bits transmitted from the satellite to the ground. Furthermore, with the powerful reconstruction capabilities of the decoding end, the feature transformation module of the encoding end can use a simplified pixel shuffling downsampling as a non-linear transformation operation to transform the original source view video frame into a compact latent representation, thereby achieving decoupling from the high-dimensional pixel domain to the low-dimensional latent feature domain. At the same time, this operation effectively removes spatial redundancy while retaining key data in the source view video frame. Moreover, this operation can achieve parameterless and non-multiplicative accumulation lightweight downsampling, completing the dimensionality transformation only through tensor rearrangement, reducing computational and memory overhead, and making it suitable for spaceborne platforms with limited computing power and memory.

[0033] The distributed remote sensing video encoding and decoding method provided by the present invention is described below. The distributed remote sensing video encoding and decoding method described below can be referred to in correspondence with the distributed remote sensing video encoding and decoding system described above.

[0034] This invention provides a distributed remote sensing video encoding / decoding method based on any of the above-mentioned distributed remote sensing video encoding / decoding systems based on spatiotemporal semantic features and diffusion models, comprising: Based on the feature transformation module, video frames from the source viewpoint of the satellite are converted from the pixel domain into a continuous latent representation; The continuous latent representation is quantized based on the quantization module to obtain an encodeable discretized latent representation; The discretized latent representation is arithmetically encoded based on the arithmetic encoder to obtain the compressed bitstream of the source view video frame; The compressed bitstream of the source view video frame is re-decoded into a discretized latent representation based on the arithmetic decoder. Based on the shared feature extraction module, the common feature information between the source view video frame and the side information view video frame is extracted according to the side information view video frame with the same shooting angle as the source view video frame from the ground. The private feature transformation module extracts the private feature information of the side information viewpoint video frame; Based on the private feature inverse transformation module, a side information perspective reconstructed video frame and multi-scale side information features of the side information perspective video frame are generated according to the complete feature representation obtained by channel splicing operation of the common feature information and the private feature information. The temporal feature extraction module extracts the temporal features of the previous two reconstructed video frames as the temporal features of the current video frame. Based on the first diffusion model, semantic enhancement processing is performed on the temporal features of the current video frame to obtain enhanced temporal features; Based on the multi-scale spatial feature mining module, multi-scale spatiotemporal semantic features are obtained according to the multi-scale side information features of the video frame from the side information perspective, the common feature information, and the enhanced temporal features. Based on the second diffusion model, a reconstructed feature representation is generated according to the discretized latent representation, the shared feature information, and the multi-scale spatiotemporal semantic features; Based on the feature inverse transformation module, and according to the reconstructed feature representation and the multi-scale spatiotemporal semantic features, the source view reconstructed video frame is obtained.

[0035] This invention provides an electronic device, including: a computer-readable storage medium and a processor; The computer-readable storage medium is used to store executable instructions; The processor is configured to read executable instructions stored in the computer-readable storage medium and execute the method as described in any of the above embodiments.

[0036] This invention provides a computer-readable storage medium storing computer instructions that cause a processor to perform the method described in any of the above embodiments.

[0037] This invention provides a computer program product, including a computer program or instructions, which, when executed by a processor, implement the method described in any of the above embodiments.

[0038] Those skilled in the art will readily understand that the above examples are merely one embodiment of the present invention and are not intended to limit the invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention. Furthermore, the technical features involved in the above embodiments of the present invention can be combined with each other as long as they do not conflict with each other, and used in other embodiments.

Claims

1. A distributed remote sensing video encoding and decoding system based on spatiotemporal semantic features and a diffusion model, characterized in that, It includes a feature transformation module, a quantization module, and an arithmetic encoder deployed on the satellite, and an arithmetic decoder, a shared feature extraction module, a private feature transformation module, a private feature inverse transformation module, a temporal feature extraction module, a first diffusion model, a multi-scale spatiotemporal feature mining module, a second diffusion model, and a feature inverse transformation module deployed on the ground. The feature transformation module is used to convert video frames from the source viewpoint of the satellite into a compact, continuous latent representation from the pixel domain. The quantization module is used to quantize the compact continuous latent representation to obtain an encodeable discretized latent representation; The arithmetic encoder is used to perform arithmetic encoding on the discretized latent representation to obtain the compressed bitstream of the source view video frame; The arithmetic decoder is used to re-decode the compressed bitstream of the source view video frame into a discretized latent representation; The shared feature extraction module is used to extract common feature information between the source view video frame and the side information view video frame based on the side information view video frame with the same shooting angle as the source view video frame from the ground. The private feature transformation module is used to extract the private feature information of the side information view video frame; The private feature inverse transformation module is used to generate a side information perspective reconstructed video frame and the multi-scale side information features of the side information perspective video frame based on the complete feature representation obtained by the channel splicing operation of the common feature information and the private feature information. The temporal feature extraction module is used to extract the temporal features of the previous two reconstructed video frames as the temporal features of the current video frame; The first diffusion model is used to perform semantic enhancement processing on the temporal features of the current video frame to obtain enhanced temporal features; The multi-scale spatiotemporal feature mining module is used to obtain multi-scale spatiotemporal semantic features based on the temporal features, the common feature information, and the enhanced temporal features. The second diffusion model is used to generate a reconstructed feature representation based on the discretized latent representation, the shared feature information, and the multi-scale spatiotemporal semantic features; The inverse feature transformation module is used to reconstruct the source view reconstructed video frame based on the reconstructed feature representation and the multi-scale spatiotemporal semantic features.

2. The distributed remote sensing video encoding and decoding system based on spatiotemporal semantic features and a diffusion model according to claim 1, characterized in that, The feature transformation module, shared feature extraction module, private feature transformation module, private feature inverse transformation module, multi-scale spatiotemporal feature mining module, and feature inverse transformation module in the distributed remote sensing video encoding and decoding system are trained based on the following loss function: in, These represent the entropy estimates of the discretized latent representation, the private feature information, and the shared feature information, respectively. , α and β are the reconstruction distortion terms for the source view video frame and the side information view video frame, respectively, used to represent the differences between the source view reconstructed video frame and the source view video frame, as well as the differences between the side information view reconstructed video frame and the side information view video frame; α and β are hyperparameters, and η represents the weight of the reconstruction distortion term for the side information view video frame.

3. A distributed remote sensing video encoding and decoding system based on spatiotemporal semantic features and a diffusion model according to claim 2, characterized in that, The multi-scale spatiotemporal feature mining module includes multiple upsampling modules, downsampling modules, and convolution modules corresponding to different scales; Among them, shared features The output of each upsampling module and the temporal semantic feature representation and The concatenated features obtained by concatenating features with the same scale from the upsampling module are input into the next upsampling module. The concatenated features are then fed into the corresponding scale convolutional module after being added element-wise by the output of the downsampling module corresponding to the corresponding scale through a 1×1 convolutional block, thus obtaining the spatiotemporal semantic features of the corresponding scale. The concatenated features obtained by concatenating the output of the last upsampling module with the temporal feature representation of the same scale are then processed by a 1×1 convolutional block and fed into the corresponding scale downsampling module and convolutional module respectively, thus obtaining the spatiotemporal semantic features of the corresponding scale output by the corresponding convolutional module.

4. A distributed remote sensing video encoding and decoding system based on spatiotemporal semantic features and a diffusion model according to any one of claims 1 to 3, characterized in that, The feature transformation module includes sequentially connected... A multi-layer shuffling downsampling layer is used to perform multi-layer pixel shuffling downsampling operations on video frames from the source's perspective to obtain a continuous latent representation at a unique scale.

5. A distributed remote sensing video encoding and decoding system based on spatiotemporal semantic features and a diffusion model according to any one of claims 1 to 3, characterized in that, The feature inverse transformation module includes sequentially connected... A multi-layer pixel shuffling upsampling is used to perform multi-layer pixel shuffling upsampling operations on the reconstructed features and the multi-scale spatiotemporal semantic feature representations. In each pixel shuffling upsampling operation, the feature information obtained from the previous pixel shuffling upsampling operation is fused with the spatiotemporal semantic features of the same scale before the pixel shuffling upsampling operation is performed.

6. A distributed remote sensing video encoding and decoding system based on spatiotemporal semantic features and a diffusion model according to any one of claims 1 to 3, characterized in that, The shared feature extraction module and the private feature transformation module are used to perform multi-layer pixel shuffling downsampling operations on the side information viewpoint video frame to obtain the shared feature information and the private feature information at a unique scale; the private feature inverse transformation module is used to perform multi-layer pixel shuffling upsampling operations on the complete feature representation of the side information viewpoint video frame to obtain the multi-scale side information features obtained by the previous several layers of pixel shuffling upsampling operations, and the side information viewpoint reconstructed video frame obtained by the last layer of pixel shuffling upsampling operations.

7. A distributed remote sensing video encoding and decoding method based on the distributed remote sensing video encoding and decoding system based on spatiotemporal semantic features and diffusion models as described in any one of claims 1 to 6, characterized in that, include: Based on the feature transformation module, video frames from the source viewpoint of the satellite are converted from the pixel domain into a continuous latent representation; The continuous latent representation is quantized based on the quantization module to obtain an encodeable discretized latent representation; The discretized latent representation is arithmetically encoded based on the arithmetic encoder to obtain the compressed bitstream of the source view video frame; The compressed bitstream of the source view video frame is re-decoded into a discretized latent representation based on the arithmetic decoder. Based on the shared feature extraction module, the common feature information between the source view video frame and the side information view video frame is extracted according to the side information view video frame with the same shooting angle as the source view video frame from the ground. The private feature transformation module extracts the private feature information of the side information viewpoint video frame; Based on the private feature inverse transformation module, a side information perspective reconstructed video frame and multi-scale side information features of the side information perspective video frame are generated according to the complete feature representation obtained by channel splicing operation of the common feature information and the private feature information. The temporal feature extraction module extracts the temporal features of the previous two reconstructed video frames as the temporal features of the current video frame. Based on the first diffusion model, semantic enhancement processing is performed on the temporal features of the current video frame to obtain enhanced temporal features; Based on the multi-scale spatial feature mining module, multi-scale spatiotemporal semantic features are obtained according to the multi-scale side information features of the video frame from the side information perspective, the common feature information, and the enhanced temporal features. Based on the second diffusion model, a reconstructed feature representation is generated according to the discretized latent representation, the shared feature information, and the multi-scale spatiotemporal semantic features; Based on the feature inverse transformation module, and according to the reconstructed feature representation and the multi-scale spatiotemporal semantic features, the source view reconstructed video frame is obtained.

8. An electronic device, characterized in that, include: Computer-readable storage media and processors; The computer-readable storage medium is used to store executable instructions; The processor is configured to read executable instructions stored in the computer-readable storage medium and execute the method as described in claim 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a processor to perform the method as described in claim 7.

10. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by the processor, they implement the method as described in claim 7.

Images