Video transmission semantic communication system
By introducing a video semantic encoding and decoding model with a neural network architecture into the wireless video transmission system, the problem of reconstructing video information under adverse channel conditions in traditional solutions has been solved, achieving efficient and noise-resistant video recovery.
Patent Information
- Application Number
- CN202311312264.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-09
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2043-10-09
AI Technical Summary
Traditional wireless video transmission cannot effectively reconstruct the original information when channel conditions are below a certain threshold, and existing deep learning improvement schemes fail to balance semantic information and channel bit rate, resulting in insufficient attention to important semantic information.
Design a video transmission semantic communication system, which adopts a video semantic encoding and decoding model based on a neural network architecture. It extracts inter-frame information through a bidirectional optical flow module and combines feature learning, selection, fusion and prediction modules to optimize the channel encoding and decoding process, reduce redundancy and improve noise resistance.
It improves the communication efficiency and noise resistance of video transmission, enhances the robustness of the system, and ensures effective recovery of video images in harsh channel environments.
Smart Images

Figure CN117579840B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of semantic communication, and more specifically to a video transmission semantic communication system. Background Technology
[0002] Traditional wireless video transmission is based on separate digital communication, consisting of a source codec (AVC / H.264, HEVC / H.265) and a channel codec (LDPC, Turbo). However, traditional communication schemes suffer from the "cliff effect" when transmitting data through physical channels. The receiver cannot reconstruct the original information from the received signal and instead generates data randomly when channel conditions fall below a certain threshold. Furthermore, some work has used deep learning to improve traditional video transmission schemes, increasing transmission efficiency. However, these schemes fail to consider the balance between semantic information and channel bit rate, resulting in insufficient attention to important semantic information.
[0003] Semantic communication can process data within the semantic domain, extracting its meaning, filtering out useless, irrelevant, and unimportant information, and further compressing the data while preserving its semantic meaning. Furthermore, semantic communication is expected to be robust to harsh channel environments (i.e., low signal-to-noise ratio regions), making it more suitable for applications requiring high reliability. These factors have prompted the development of semantic communication systems that consider the semantics behind digital bits, thereby improving the accuracy and efficiency of communication.
[0004] Therefore, how to complete the data reconstruction at the receiving end in a noisy channel with limited communication volume is a task that urgently needs to be studied. Summary of the Invention
[0005] To overcome the shortcomings of existing technologies, this invention provides a video transmission semantic communication system.
[0006] The objective of this invention is achieved through the following technical solution: a video transmission semantic communication system, which is deployed at the transmitting end and receiving end of a wireless video device. The transmitting end includes a video acquisition module, a central processing module 1, and a wireless communication module 1, and the receiving end includes a central processing module 2 and a wireless communication module 2.
[0007] The video acquisition module of the transmitting end acquires video images through an optical lens and performs digital processing; the central computing module 1 deploys a video semantic coding model based on a neural network architecture, performs semantic coding on the acquired video images, and sends the encoded information to the receiving end after channel coding through the channel encoder in the wireless communication module 1.
[0008] The receiving end performs channel decoding on the received signal through the channel decoder in the wireless communication module 2, and sends the decoding information to the central processing module 2. The central processing module 2 deploys a video semantic decoding model based on a neural network architecture to perform semantic decoding on the received signal, and finally completes the restoration of the video image.
[0009] The video semantic coding model includes a bidirectional optical flow module and a semantic encoder. The semantic encoder includes a feature learning module 1, a feature selection module, and a feature fusion module. The video semantic decoding model includes a feature prediction module, a feature learning module 2, and an interpolation module.
[0010] The digitization process of the video acquisition module includes the normalization operation of video frames, and the digitized video frame sequence S is input into the bidirectional optical flow module and the semantic encoder respectively.
[0011] The bidirectional optical flow module extracts inter-frame information of video images by estimating residual information through bidirectional optical flow and inputs the bidirectional optical flow V into the semantic encoder.
[0012] The feature learning module 1 and feature learning module 2 extract semantic information through a convolutional neural network, use the PReLU activation function to perform non-linear mapping on the output of all convolutional layers, and embed a noise attention module after each PReLU activation function to redistribute the weights of the learned potential semantic information; the bidirectional optical flow feature F and video frame feature S′ are obtained through feature learning module 1 and input into the feature selection module.
[0013] In the feature selection module, the current frame is defined as the key frame, and the frames before and after the current frame are defined as prediction frames. Bidirectional optical flow features are selected through a forward selection mechanism. The selected bidirectional optical flow features and the key frame / prediction frame are fused using a dual feature module to obtain the fused feature F′.
[0014] The feature fusion module reweights the fused feature F′ output by the feature selection module and the video frame feature S′ output by the feature learning module 1, performs feature learning on the input features using multiple receptive fields, gives higher weight to important features, and outputs semantic feature F″.
[0015] The channel encoder and channel decoder consist of convolutional layers and shaping layers; the cascaded convolutional layers of the channel encoder map video frame features S′ and semantic features F″ to channel features Y, respectively, and the shaping layer obtains the channel features Y′ to be transmitted; the receiving end receives the channel features... The channel features are shaped by a shaping layer. Input concatenated convolutional layers to recover video frame features and semantic features
[0016] The feature prediction module is used to map latent semantic features to recover the original video frame sequence; for the received video frame features and semantic features To perform prediction and reconstruction, the feature information is learned by Res-UNet with embedded noise attention module to obtain the predicted semantic features L;
[0017] The feature learning module 2 combines semantic features L and video frame features. They are respectively mapped and reconstructed into video frame sequence P and video frame sequence K;
[0018] The interpolation module restores the reconstructed video frame sequence P and K to the original frame order, thus completing the video image restoration.
[0019] Furthermore, the implementation of the bidirectional optical flow module includes:
[0020] (1) Use a weight-shared convolutional network to extract the fully connected features F1 and F2 of two consecutive video frames;
[0021] (2) By calculating the correlation between the fully connected features F1 and F2 Obtain the inter-frame correspondence. This is a normalization factor to limit the results of matrix dot product;
[0022] (3) Obtain the inter-frame correlation by using the weighted average value G of the inter-frame correspondence and pixel grid.
[0023] (4) Obtain inter-frame bidirectional optical flow based on the differences between corresponding pixels. The residual information of consecutive frames is estimated by bidirectional optical flow, and the bidirectional optical flow V is input into the semantic encoder.
[0024] Furthermore, feature learning module 1 and feature learning module 2 extract semantic information through an m×m×o / s convolutional neural network, where m is the kernel size, o is the number of convolutional channels, and s is the stride. A generalized normalization transformation layer is applied to each convolutional layer for normalization. In feature learning module 1, the generalized normalization transformation layer uses GDN, and in feature learning module 2, the generalized normalization transformation layer uses IGDN. Feature learning module 2 performs feature upsampling through pixel shuffle, and its last layer is a sigmoid activation function.
[0025] Furthermore, the implementation of the noise attention module includes: expanding the SNR value to the same dimension as the latent semantic feature U; processing the semantic feature U with an average pooling function and concatenating it with the expanded SNR value; inputting the concatenated result into the squeezing and activation module to obtain output weights; processing the output weights with a sigmoid activation function to obtain a scaling factor; multiplying the input semantic feature U with the scaling factor to obtain the scaled feature U′; and finally using a residual network to learn the impact of noise on the system.
[0026] Furthermore, the dual-feature module is used to fuse video frames and optical flow features. Its input is bidirectional optical flow features and keyframe features, or bidirectional optical flow features and predicted frame features. Feature information is extracted through a convolutional neural network, and the extracted features are cross-fused. Finally, the fused features are output through a CA attention mechanism.
[0027] Furthermore, the implementation of the feature fusion module includes: inputting the fused feature F′ and video frame feature S′ into the feature learning module and the noise attention module for processing, respectively, and using different convolution kernel sizes to achieve different receptive fields to perform feature fusion in a soft attention manner. The fused feature information is then weighted and redistributed with the input fused feature F′ and video frame feature S′, and the redistribution result is input into the CNN network to finally output the semantic feature F″.
[0028] Furthermore, the implementation of the feature prediction module includes: firstly, processing the received semantic features... and video frame features Joint operations are performed to obtain joint features, which are then input into the Res-UNet network. Feature mapping is performed through two downsampling and two upsampling operations to generate the predicted semantic features L.
[0029] Furthermore, the implementation of the interpolation module includes: first, taking a frame from the video frame sequence P as the starting frame, then taking a frame from the video frame sequence K as the subsequent frame, and so on alternately, to finally complete the restoration of the video image.
[0030] The beneficial effects of this invention are as follows: Based on the structure of keyframes and prediction frames, this invention designs a continuous inter-frame residual information estimation method based on bidirectional optical flow, reducing information redundancy between consecutive frames; it designs a feature learning module to perform feature learning and size transformation on semantic information, improving feature extraction capability; it designs a feature selection module to reduce semantic redundancy and further improve the compression rate of data features; it designs a feature fusion module, which improves the system's tolerance to noise while focusing on important semantic features by embedding a noise attention module; and it designs a feature prediction module to perform data mapping on received semantic features, improving the expressive power of data features. The system of this invention has the advantages of high communication efficiency, strong noise resistance, and high system robustness. Attached Figure Description
[0031] Figure 1 This is a structural diagram of a video transmission semantic communication system provided in an embodiment of the present invention;
[0032] Figure 2 This is a structural diagram of the feature learning module provided in an embodiment of the present invention;
[0033] Figure 3 This is a structural diagram of the noise attention module provided in an embodiment of the present invention;
[0034] Figure 4(a) is a structural diagram of the feature selection module provided in an embodiment of the present invention;
[0035] Figure 4(b) is a structural diagram of the dual-feature module provided in an embodiment of the present invention;
[0036] Figure 5 This is a structural diagram of the feature fusion module provided in an embodiment of the present invention;
[0037] Figure 6 This is a structural diagram of the feature prediction module provided in an embodiment of the present invention;
[0038] Figure 7 This is a schematic diagram of the interpolation module provided in an embodiment of the present invention. Detailed Implementation
[0039] To better understand the technical solution of this application, the embodiments of this application will be described in detail below with reference to the accompanying drawings.
[0040] It should be understood that the described embodiments are merely some, not all, of the embodiments in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.
[0041] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. The singular forms “a,” “the,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.
[0042] like Figure 1 As shown, this application embodiment provides a video transmission semantic communication system. The system is deployed at the transmitting end and receiving end of a wireless video device. The transmitting end includes a video acquisition module, a central processing module 1, and a wireless communication module 1. The receiving end includes a central processing module 2 and a wireless communication module 2. The wireless video device can be an intelligent wireless terminal with computing capabilities, including wireless monitoring equipment, wireless cameras, etc.
[0043] The video acquisition module at the transmitting end acquires video images through an optical lens and performs digital processing; the central computing module 1 deploys a video semantic coding model based on a neural network architecture to perform semantic coding on the acquired video images, and then sends the coded information to the receiving end after channel coding through the channel encoder in the wireless communication module 1.
[0044] The receiving end performs channel decoding on the received signal through the channel decoder in the wireless communication module 2, and sends the decoding information to the central processing module 2. The central processing module 2 deploys a video semantic decoding model based on a neural network architecture to perform semantic decoding on the received signal, and finally completes the restoration of the video image.
[0045] Specifically, the video semantic coding model includes a bidirectional optical flow module and a semantic encoder. The semantic encoder includes a feature learning module, a feature selection module, and a feature fusion module.
[0046] Specifically, the video semantic decoding model includes a feature prediction module, a feature learning module 2, and an interpolation module.
[0047] In the process of video image feature extraction in the central computing module, this invention divides continuous video image frames into key frames and prediction frames, and reduces semantic redundancy and improves communication efficiency by using inter-frame correlation features.
[0048] The following description further provides some embodiments of the implementation of the various modules of the video transmission semantic communication system that meet the requirements of this application.
[0049] I. Video Acquisition Module
[0050] The digitization process of the video acquisition module includes video frame normalization, and the digitized video frame sequence is denoted as... Let S represent the i-th video frame in the n-th video group, where H and W represent the height and width of the video frame, respectively, C represents the number of channels in the video frame, and N represents the number of video groups. The video frame sequence S is input into the bidirectional optical flow module and the semantic encoder, respectively.
[0051] II. Bidirectional Optical Flow Module
[0052] The bidirectional optical flow module is denoted as GMFlow. GMFlow uses bidirectional optical flow to estimate residual information and extract inter-frame information from video images. Based on this information, it extracts correlation features between consecutive frames and reduces frame compression information through these correlation features. The specific implementation is as follows:
[0053] (1) Use a weight-shared convolutional network to extract the fully connected features of two consecutive video frames. D represents the number of channels in the fully connected feature;
[0054] (2) By calculating the correlation between the fully connected features F1 and F2 Obtain the inter-frame correspondence. This is a normalization factor to limit the results of matrix dot product;
[0055] (3) Through the inter-frame correspondence and the weighted average of the pixel grid Obtain inter-frame correlation
[0056] (4) Obtain inter-frame bidirectional optical flow based on the differences between corresponding pixels. The residual information of consecutive frames is estimated by bidirectional optical flow, and the bidirectional optical flow V is input into the semantic encoder.
[0057] III. Feature Learning Module
[0058] The structures of feature learning module 1 and feature learning module 2 are as follows: Figure 2 As shown, the feature learning module extracts semantic information through an m×m×o / s convolutional neural network, where m is the kernel size, o is the number of convolutional channels, and s is the stride. Furthermore, a generalized normalization (GDN) transformation layer is applied to each convolutional layer for normalization; in feature learning module 1, the GDN is used, while in feature learning module 2, the GDN is used. The PReLU activation function is used to perform a non-linear mapping on the outputs of all convolutional layers. Feature learning module 2 performs feature upsampling through pixel shuffle, and the last layer uses a sigmoid activation function.
[0059] A noisy attention module is embedded after each PReLU activation function to reallocate the weights of the learned latent semantic information, thereby mitigating the impact of channel noise on the system. The specific structure is as follows: Figure 3As shown, the SNR value is first expanded to the same dimension as the latent semantic feature U. The semantic feature U is then processed by an average pooling function and concatenated with the expanded SNR value. The concatenated result is then input into the squeezing and activation module (SE Layer) to obtain the output weights. The output weights are processed by a sigmoid activation function to obtain a scaling factor. The scaled feature U′ is obtained by multiplying the input semantic feature U by the scaling factor. Finally, a residual network is used to learn the impact of noise on the system. By embedding the noise attention module into different modules, the system's robustness to channel noise is improved.
[0060] Bidirectional optical flow features are obtained through feature learning module 1. and video frame features The input is then fed into the feature selection module, where t is the downsampling factor, and c1 and c2 are the number of channels for optical flow features and video frame features, respectively. In this embodiment, c1 = 512 and c2 = 128. The large amount of input semantic information poses a significant challenge to the transmitter, which needs to identify the important semantic information. To address this issue, a feature selection module is proposed to refine the semantic information while reducing complexity.
[0061] IV. Feature Selection Module
[0062] In the feature selection module, the current frame is defined as the keyframe, and the frames before and after the current frame are defined as prediction frames. A forward selection mechanism is used to select bidirectional optical flow features. The dual-feature module (DFF) then fuses the selected bidirectional optical flow features with the keyframe / prediction frames to obtain the fused features. Where t is the downsampling factor and c3 is the number of channels of the fused feature. In this embodiment, c3 = 256. The structure of the feature selection module is shown in Figure 4(a). In this module, the selected optical flow features and the corresponding video frame information are marked with solid lines, and the discarded semantic information is marked with dashed lines.
[0063] Among them, the dual-feature module DFF is used to fuse video frames and optical flow features. Its structure is shown in Figure 4(b). The input of DFF is bidirectional optical flow features and key frame features, or bidirectional optical flow features and predicted frame features. Feature information is extracted through convolutional neural network, and the extracted features are cross-fused. Finally, the fused features are output through CA attention mechanism.
[0064] V. Feature Fusion Module
[0065] For the complex inter-frame information in the spatiotemporal domain of video transmission on physical channels, a feature fusion module is proposed to focus on important semantic features, thereby achieving better reconstruction performance.
[0066] The feature fusion module reweights the fused feature F′ output by the feature selection module and the video frame feature S′ output by the feature learning module 1. It then uses multiple receptive fields to learn the input features, assigning higher weights to important features, and finally outputs semantic features. Where t is the downsampling factor, and c4 is the number of channels for the semantic features; in this embodiment, c4 = 256. The structure of the feature fusion module is as follows: Figure 5 As shown, the specific implementation is as follows:
[0067] The fused feature F′ and video frame feature S′ are input into the feature learning module and the noise attention module, respectively, for processing. The output features are fused using different receptive fields through different convolutional kernel sizes in a soft attention manner. The fused feature information is then weighted and redistributed with the input fused feature F′ and video frame feature S′. The weighted result is input into the CNN network, and the final output is the semantic feature F″. This module uses a more flexible method to update the weights of the semantic information, thereby improving the efficiency and effectiveness of the latent semantic representation through soft attention.
[0068] VI. Channel Encoder and Channel Decoder
[0069] The channel encoder and channel decoder consist of convolutional layers and shaping layers.
[0070] The cascaded convolutional layers of the channel encoder map video frame features S′ and semantic features F″ to channel features, respectively. Then, the channel characteristics to be transmitted are obtained through the shaping layer. Where y1 and y2 are the number of output channels for the keyframe and prediction frame, respectively, and c′ is... At the receiving end, the received channel characteristics Through plastic surgery to It has the same dimensionality as Y. The shaped channel features Input to cascaded convolutional layers to recover video frame features and semantic features
[0071] VII. Feature Prediction Module
[0072] The feature prediction module is used to map latent semantic features to recover the original video frame sequence; for the received video frame features and semantic features Predictive reconstruction is performed by learning feature information through a simplified Res-UNet with an embedded noisy attention module; the structure of the feature prediction module is as follows: Figure 6 As shown, the specific implementation is as follows:
[0073] First, the received semantic features and video frame features Joint operations are performed to obtain joint features, which are then input into a simplified Res-UNet network. Feature mapping is performed through two downsampling and two upsampling operations to generate predicted semantic features. Where t is the downsampling factor and c5 is the number of channels for the predicted semantic features. In this embodiment, c5 = 128.
[0074] Feature learning module 2 combines semantic features L and video frame features The video frames are reconstructed into video frame sequences P and K, respectively, and the resulting video frames have the same dimensions as the original video frames.
[0075] VIII. Interpolation Module
[0076] The interpolation module restores the reconstructed video frame sequence to the original frame order. The structure of the interpolation module is as follows: Figure 7 As shown, first, a frame is taken from the video frame sequence P as the starting frame, and then a frame is taken from the video frame sequence K as the subsequent frame. This process is repeated alternately until the video image is finally restored.
[0077] The video transmission semantic communication system proposed in this invention has the advantages of high communication efficiency, strong anti-noise capability and high system robustness.
[0078] The above embodiments are typical implementations of the present invention, but the implementation of the present invention is not limited to the embodiments described above. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.
Claims
1. A video transmission semantics communication system, characterized by, The system is deployed at the transmitting and receiving ends of a wireless video device. The transmitting end includes a video acquisition module, a central processing module 1, and a wireless communication module 1. The receiving end includes a central processing module 2 and a wireless communication module 2. The video acquisition module of the transmitting end acquires video images and performs digital processing; the central computing module 1 deploys a video semantic coding model based on a neural network architecture to perform semantic coding on the acquired video images, and sends the encoded information to the receiving end after channel coding through the channel encoder in the wireless communication module 1. The receiving end performs channel decoding on the received signal through the channel decoder in the wireless communication module 2, and sends the decoding information to the central processing module 2. The central processing module 2 deploys a video semantic decoding model based on a neural network architecture to perform semantic decoding on the received signal, and finally completes the restoration of the video image. The video semantic coding model includes a bidirectional optical flow module and a semantic encoder. The semantic encoder includes a feature learning module 1, a feature selection module, and a feature fusion module. The video semantic decoding model includes a feature prediction module, a feature learning module 2, and an interpolation module. The digitization process of the video acquisition module includes the normalization operation of video frames, and the digitized video frame sequence S is input into the bidirectional optical flow module and the semantic encoder respectively. The bidirectional optical flow module extracts inter-frame information of video images by estimating residual information through bidirectional optical flow and inputs the bidirectional optical flow V into the semantic encoder. The feature learning module 1 and feature learning module 2 extract semantic information through a convolutional neural network, use the PReLU activation function to perform non-linear mapping on the output of all convolutional layers, and embed a noise attention module after each PReLU activation function to redistribute the weights of the learned potential semantic information. Bidirectional optical flow features are obtained through feature learning module 1. and video frame features And input the feature selection module; In the feature selection module, the current frame is defined as the prediction frame, and the frames before and after the current frame are defined as keyframes. A forward selection mechanism selects bidirectional optical flow features, and a dual-feature module fuses the selected bidirectional optical flow features with the keyframes / prediction frames to obtain the fused features. ; The feature fusion module reweights the input features using multiple receptive fields and gives higher weights to important features to output semantic features and the video frame features output by the feature learning module 1 The feature fusion module reweights the input features using multiple receptive fields and gives higher weights to important features to output semantic features ; The channel encoder and channel decoder consist of convolutional layers and shaping layers; the cascaded convolutional layers of the channel encoder process video frame features. and semantic features Mapped to channel features respectively The channel characteristics to be transmitted are obtained through the shaping layer. The receiving end will receive the channel characteristics. The channel features are shaped by a shaping layer. Input concatenated convolutional layers to recover video frame features and semantic features ; The feature prediction module is used to map latent semantic features to recover the original video frame sequence; for the received video frame features and semantic features Predictive reconstruction is performed by learning the feature information through a Res-UNet with an embedded noisy attention module to obtain the predicted semantic features. ; The feature learning module 2 will learn semantic features and video frame features Reconstructed into video frame sequences by mapping and video frame sequence ; The interpolation module reconstructs the video frame sequence. and The original frame order is restored, completing the video image recovery.
2. The system according to claim 1, characterized in that, The implementation of the bidirectional optical flow module includes: (1) Use a weight-shared convolutional network to extract the fully connected features of two consecutive video frames. ; (2) By calculating the fully connected features and Correlation between The inter-frame correspondence is obtained, where D represents the number of channels in the fully connected feature. This is a normalization factor to limit the results of matrix dot product; (3) Through the inter-frame correspondence and the weighted average of the pixel grid Obtain inter-frame correlation ; (4) Obtain inter-frame bidirectional optical flow based on the differences between corresponding pixels. The residual information of consecutive frames is estimated by bidirectional optical flow, and the bidirectional optical flow V is input into the semantic encoder.
3. The system according to claim 1, characterized in that, The feature learning module 1 and feature learning module 2 are connected through Convolutional neural networks extract semantic information. The kernel size is the convolution kernel size. The number of convolution channels. The step size is used; a generalized normalization transformation layer is applied to each convolutional layer for normalization. In feature learning module 1, the generalized normalization transformation layer adopts GDN, and in feature learning module 2, the generalized normalization transformation layer adopts IGDN. The feature learning module 2 performs feature upsampling operation through pixel shuffle, and its last layer is a sigmoid activation function.
4. The system according to claim 1, characterized in that, The implementation of the noise attention module includes: expanding the SNR value to the same dimension as the latent semantic feature U; processing the semantic feature U with an average pooling function and concatenating it with the expanded SNR value; inputting the concatenated result into the squeezing and activation module to obtain output weights; processing the output weights with a sigmoid activation function to obtain a scaling factor; and obtaining the scaled feature by multiplying the input semantic feature U with the scaling factor. Finally, a residual network is used to learn the effect of noise on the system.
5. The system according to claim 1, characterized in that, The dual-feature module is used to fuse video frames and optical flow features. Its input is bidirectional optical flow features and keyframe features, or bidirectional optical flow features and predicted frame features. Feature information is extracted through a convolutional neural network, and the extracted features are cross-fused. Finally, the fused features are output through a CA attention mechanism.
6. The system according to claim 1, characterized in that, The implementation of the feature fusion module includes: fusing features and video frame features The input features are processed separately into the feature learning module and the noise attention module. The output features are then processed using different convolutional kernel sizes to achieve different receptive fields. Feature fusion is performed using soft attention, and the fused feature information is then combined with the input fused features. and video frame features The weights are redistributed, the result is fed into the CNN network, and the final output is semantic features. .
7. The system according to claim 1, characterized in that, The implementation of the feature prediction module includes: first, processing the received semantic features... and video frame features Joint operations are performed to obtain joint features, which are then input into the Res-UNet network. Feature mapping is performed through two downsampling and two upsampling operations to generate predicted semantic features. .
8. The system according to claim 1, characterized in that, The implementation of the interpolation module includes: first, from the video frame sequence Take one frame from the video frame sequence as the starting frame, and then extract the next frame from the video frame sequence. One frame is extracted from the video and used as the subsequent frame. This process is repeated alternately until the video image is finally restored.
9. The system according to claim 1, characterized in that, The wireless video device uses an intelligent wireless terminal with computing capabilities, including wireless monitoring equipment and wireless cameras.
