A light field image transmission method based on multi-dimensional feature fusion of light field images

Abstract

This invention discloses a method for transmitting light field images based on multi-dimensional feature fusion. The method includes: the sender converting the light field image into sub-aperture images, performing sparse sampling to obtain a set of sparsely sampled sub-aperture images. These sub-aperture image sets are then converted into pseudo-video sequences and VVC encoded using a VTM encoder to generate a pseudo-video sequence bitstream, which is then sent to the receiver. Upon receiving the bitstream, the receiver decodes it, reconstructs the pseudo-video sequence, and extracts the set of sparsely sampled sub-aperture images using a video extraction module. This set is then converted into a first macro-pixel image and input into a multi-dimensional feature fusion network for further processing. This network fuses multi-dimensional features through multiple feature extraction modules to reconstruct the light field image. This method improves the transmission efficiency and quality of light field images by introducing multi-dimensional feature fusion and multi-scale attention mechanisms, making it particularly suitable for bandwidth-constrained environments.

Description

Technical Field

[0001] This invention belongs to the field of light field image processing technology, specifically relating to a light field image transmission method based on the fusion of multi-dimensional features of light field images. Background Technology

[0002] Light field technology is an advanced imaging technique capable of capturing spatial and angular information of a scene, widely used in virtual reality, augmented reality, computational photography, and 3D modeling. However, due to the large number of sub-aperture images contained in light field data, its data volume is extremely large, placing high demands on bandwidth and storage resources for transmission and storage. Therefore, how to efficiently encode light field data to reduce the number of transmitted views and improve transmission efficiency has become a key research issue. To address these issues, it is necessary to develop an efficient light field coding framework that fully utilizes the spatial and angular characteristics of light field images to improve coding performance and reduce transmission bandwidth requirements. Simultaneously, it should enhance the processing capability for viewpoint correlation at the decoding end to improve the visual quality of the decoded image, thereby providing a more efficient solution for the practical application of light field technology. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a light field image transmission method based on the fusion of multi-dimensional features of light field images.

[0004] The objective of this invention can be achieved through the following technical solutions:

[0005] This invention provides a method for transmitting light field images based on multi-dimensional feature fusion of light field images, comprising the following steps:

[0006] Step S1: The sender converts the light field image into a sub-aperture image, performs sparse sampling on the sub-aperture image, and obtains a set of sparsely sampled sub-aperture images;

[0007] Step S2: Combine the sparsely sampled sub-aperture images into a pseudo-video sequence;

[0008] Step S3: Input the pseudo-video sequence into the VTM encoder for VVC encoding to obtain the pseudo-video sequence bitstream, and send the pseudo-video sequence bitstream to the receiver;

[0009] Step S4: The receiver receives the pseudo video sequence bitstream and restores the pseudo video sequence bitstream to obtain the pseudo video sequence.

[0010] Step S5: Input the pseudo-video sequence into the video extraction module to extract sub-aperture images and output a sparsely sampled sub-aperture image set;

[0011] Step S6: Convert the sparsely sampled sub-aperture image set into a first macro-pixel image;

[0012] Step S7: Input the first macro-pixel image into the multi-dimensional feature fusion network, output the second macro-pixel image, convert the output second macro-pixel image into a sub-aperture image, and obtain the light field image through the sub-aperture image.

[0013] Furthermore, step S1 includes the following steps:

[0014] Multiple sub-aperture images are extracted from the light field image. Each sub-aperture image represents a two-dimensional image at a specific viewpoint in the light field image. The extracted sub-aperture images are an n×n sub-aperture image array.

[0015] The n×n sub-aperture image array is sparsely sampled according to a preset sparse sampling rule to obtain a sparsely sampled sub-aperture image set.

[0016] Furthermore, the step of forming a pseudo-video sequence from the sparsely sampled sub-aperture images includes: arranging the sparsely sampled sub-aperture images in a preset order to generate a pseudo-video sequence, wherein each sub-aperture image corresponds to a frame in the video, simulating the order of the timeline.

[0017] Furthermore, step S5 includes the following steps:

[0018] The pseudo-video sequence is input into the video extraction module. Each frame Fi of the pseudo-video sequence corresponds to a sub-aperture image. The video extraction module analyzes the input pseudo-video sequence and extracts the viewpoint image in each frame.

[0019] The video extraction module will perform position analysis on each frame of the image according to the preset sparse sampling rules, and extract the sub-aperture image at a specific position from each frame of the pseudo video sequence.

[0020] The extracted sub-aperture image set is output as a sparsely sampled sub-aperture image set.

[0021] Furthermore, the multidimensional feature fusion network includes a convolutional layer, multiple identical multidimensional feature fusion modules, and an upsampling module connected in sequence.

[0022] Furthermore, the multiple identical multidimensional feature fusion modules are connected using a residual connection method.

[0023] Furthermore, the multidimensional feature fusion module includes a spatial feature extractor, a dense spatial feature extractor, an angular feature extractor, a horizontal polar plane feature extractor, a vertical polar plane feature extractor, a dense spatial fusion module, an angular domain feature fusion module, and a fusion convolution.

[0024] Furthermore, the spatial feature extractor includes two identical 3×3 convolutional layers with a dilation factor of 3, a padding size consistent with the dilation factor, a stride of 1, and a LeakyReLU activation function following each convolutional layer.

[0025] The dense spatial feature extractor includes two identical 5×5 convolutional layers with padding of 2 and stride of 1. Each convolutional layer is followed by a LeakyReLU activation function.

[0026] The angle feature extractor includes two identical 2D convolution and PixelShuffle modules, with a 2D convolution kernel size and stride of 3;

[0027] The horizontal polar plane feature extractor includes an asymmetric convolution kernel of size [1,9], a channel splicing module, and a 1×1 convolution;

[0028] The vertical polar plane feature extractor includes an asymmetric convolution kernel of size [9,1], a channel splicing module, and a 1×1 convolution.

[0029] Furthermore, the dense spatial fusion module includes a multi-scale attention mechanism module and a spatial fusion convolution. The spatial fusion convolution has 2 input channels and 1 output channel, and uses LeakyReLU (0.1) for non-linear activation.

[0030] The corner feature fusion module includes a multi-scale attention mechanism module and a corner fusion convolution. The corner fusion convolution has 3 input channels and 1 output channel, and uses LeakyReLU (0.1) for non-linear activation.

[0031] Furthermore, the step of inputting the first macro-pixel image into the multi-dimensional feature fusion network and outputting the second macro-pixel image includes the following steps:

[0032] The first macro-pixel image is input into a convolutional layer of a multi-dimensional feature fusion network, and the first feature map is output.

[0033] The first feature map is input into the spatial feature extractor, dense spatial feature extractor, angular feature extractor, horizontal polar plane feature extractor, and vertical polar plane feature extractor of the multi-dimensional feature fusion module for feature extraction, and the spatial feature map, local spatial feature map, angular feature map, horizontal feature map, and vertical feature map are output respectively.

[0034] The spatial feature map and the local spatial feature map are input into the dense spatial fusion module for feature fusion, and the second feature map is output.

[0035] The angular feature map, horizontal feature map, and vertical feature map are input into the angular domain feature fusion module for feature fusion, and the third feature map is output.

[0036] The second and third feature maps are input into the fusion convolution of the multi-dimensional feature fusion module, and the fourth feature map is output.

[0037] The fourth feature map is input into the subsequent multi-dimensional feature fusion module, the last multi-dimensional feature fusion module outputs the multi-dimensional feature fusion feature map, the multi-dimensional feature fusion feature map is input into the upsampling module, and the second macro-pixel image is output.

[0038] Compared with the prior art, the present invention has the following advantages:

[0039] (1) This invention reduces the amount of data during the transmission of light field images by converting them into sub-aperture images and performing sparse sampling to generate pseudo-video sequences. This method effectively reduces data redundancy and lowers the transmission bandwidth requirements.

[0040] (2) This invention performs multi-dimensional feature fusion on the recovered image through a multi-dimensional feature fusion network. By combining the fusion of multiple features such as space, dense space, angle, and polar plane, it significantly improves the accuracy and detail of light field image recovery and can better reconstruct various viewpoints of the light field image.

[0041] (3) The present invention applies a multi-scale attention mechanism in the dense space fusion module and the corner domain feature fusion module, which can weight important features according to the specific content of the image, thereby improving the image quality and visual effect and avoiding interference from irrelevant information.

[0042] (4) By employing convolution and attention mechanisms at different scales, this invention can effectively process light field images of varying complexity, demonstrating good adaptability and processing performance in various light field image transmission tasks. Whether the light field image is low-resolution or high-resolution, this invention maintains good image restoration results.

[0043] (5) Due to the use of sparse sampling and pseudo-video sequence, the present invention can effectively reduce the computational burden and reduce transmission delay, which is suitable for the needs of real-time light field image transmission. Attached Figure Description

[0044] Figure 1 This is a flowchart of the present invention;

[0045] Figure 2 This is a flowchart illustrating the overall process framework of the present invention.

[0046] Figure 3 This is a block diagram of the multi-scale attention mechanism of the present invention. Detailed Implementation

[0047] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0048] Example 1:

[0049] This embodiment provides a light field image transmission method based on multi-dimensional feature fusion of light field images, such as... Figure 1 As shown, it includes the following steps:

[0050] Step S1: The sender converts the light field image into a sub-aperture image, performs sparse sampling on the sub-aperture image, and obtains a set of sparsely sampled sub-aperture images;

[0051] Step S2: Combine the sparsely sampled sub-aperture images into a pseudo-video sequence;

[0052] Step S3: Input the pseudo-video sequence into the VTM encoder for VVC encoding to obtain the pseudo-video sequence bitstream, and send the pseudo-video sequence bitstream to the receiver;

[0053] Step S4: The receiver receives the pseudo video sequence bitstream and restores the pseudo video sequence bitstream to obtain the pseudo video sequence.

[0054] Step S5: Input the pseudo-video sequence into the video extraction module to extract sub-aperture images and output a sparsely sampled sub-aperture image set;

[0055] Step S6: Convert the sparsely sampled sub-aperture image set into a first macro-pixel image;

[0056] Step S7: Input the first macro-pixel image into the multi-dimensional feature fusion network, output the second macro-pixel image, convert the output second macro-pixel image into a sub-aperture image, and obtain the light field image through the sub-aperture image.

[0057] Step S1 includes the following steps:

[0058] Multiple sub-aperture images are extracted from the light field image. Each sub-aperture image represents a two-dimensional image at a specific viewpoint in the light field image. The extracted sub-aperture images are an n×n sub-aperture image array.

[0059] The n×n sub-aperture image array is sparsely sampled according to a preset sparse sampling rule to obtain a sparsely sampled sub-aperture image set.

[0060] The process of creating a pseudo-video sequence from a set of sparsely sampled sub-aperture images includes: arranging the set of sparsely sampled sub-aperture images in a preset order to generate a pseudo-video sequence, wherein each sub-aperture image corresponds to a frame in the video, simulating the order of the timeline.

[0061] Step S5 includes the following steps:

[0062] The pseudo-video sequence is input into the video extraction module. Each frame Fi of the pseudo-video sequence corresponds to a sub-aperture image. The frequency extraction module analyzes the input pseudo-video sequence and extracts the viewpoint image in each frame.

[0063] The video extraction module will perform position analysis on each frame of the image according to the preset sparse sampling rules, and extract the sub-aperture image at a specific position from each frame of the pseudo video sequence.

[0064] The extracted sub-aperture image set is output as a sparsely sampled sub-aperture image set.

[0065] The multidimensional feature fusion network consists of a convolutional layer, multiple identical multidimensional feature fusion modules, and an upsampling module connected in sequence.

[0066] In this process, multiple identical multidimensional feature fusion modules are connected using residual connections.

[0067] The multidimensional feature fusion module includes a spatial feature extractor, a dense spatial feature extractor, an angular feature extractor, a horizontal polar plane feature extractor, a vertical polar plane feature extractor, a dense spatial fusion module, an angular domain feature fusion module, and a fusion convolution.

[0068] The spatial feature extractor consists of two identical 3×3 convolutional layers with a dilation factor of 3, padding size matching the dilation factor, and a stride of 1. Each convolutional layer is followed by a LeakyReLU activation function.

[0069] The dense spatial feature extractor consists of two identical 5×5 convolutional layers with padding of 2 and stride of 1, followed by a LeakyReLU activation function after each convolutional layer;

[0070] The angle feature extractor consists of two identical 2D convolutions with a PixelShuffle module, the 2D convolution kernel size and stride being 3;

[0071] The horizontal polar plane feature extractor includes an asymmetric convolution kernel of size [1,9], a channel splicing module, and a 1×1 convolution;

[0072] The vertical polar plane feature extractor includes an asymmetric convolution kernel of size [9,1], a channel splicing module, and a 1×1 convolution.

[0073] The dense spatial fusion module includes a multi-scale attention mechanism module and a spatial fusion convolution. The spatial fusion convolution has 2 input channels and 1 output channel, and uses LeakyReLU (0.1) for non-linear activation.

[0074] The corner feature fusion module includes a multi-scale attention mechanism module and a corner fusion convolution. The corner fusion convolution has 3 input channels and 1 output channel, and uses LeakyReLU (0.1) for non-linear activation.

[0075] The process of inputting the first macro-pixel image into a multi-dimensional feature fusion network and outputting the second macro-pixel image includes the following steps:

[0076] The first macro-pixel image is input into a convolutional layer of a multi-dimensional feature fusion network, and the first feature map is output.

[0077] The first feature map is input into the spatial feature extractor, dense spatial feature extractor, angular feature extractor, horizontal polar plane feature extractor, and vertical polar plane feature extractor of the multi-dimensional feature fusion module for feature extraction, and the spatial feature map, local spatial feature map, angular feature map, horizontal feature map, and vertical feature map are output respectively.

[0078] The spatial feature map and the local spatial feature map are input into the dense spatial fusion module for feature fusion, and the second feature map is output.

[0079] The angular feature map, horizontal feature map, and vertical feature map are input into the angular domain feature fusion module for feature fusion, and the third feature map is output.

[0080] The second and third feature maps are input into the fusion convolution of the multi-dimensional feature fusion module, and the fourth feature map is output.

[0081] The fourth feature map is input into the subsequent multi-dimensional feature fusion module, the last multi-dimensional feature fusion module outputs the multi-dimensional feature fusion feature map, the multi-dimensional feature fusion feature map is input into the upsampling module, and the second macro-pixel image is output.

[0082] Example 2:

[0083] The parts not mentioned in this embodiment are the same as in Embodiment 1.

[0084] This invention proposes a light field image transmission method based on multi-dimensional feature fusion of light field images. This method aims to achieve efficient reconstruction of the light field image by converting the light field image into a sparsely sampled sub-aperture image set, compressing and transmitting it using video coding technology, and combining it with a multi-dimensional feature fusion network. The specific implementation process is as follows:

[0085] First, the sender extracts sub-aperture images from the light field image. The light field image is essentially a two-dimensional image array captured from different viewpoints, typically containing multiple sub-aperture images, each representing a two-dimensional image of the light field image from a specific viewpoint. For ease of transmission and processing, a 7×7 sub-aperture image array is extracted from the light field image, with each sub-aperture image representing a different viewpoint. From these sub-aperture images, a subset is selected according to a preset sparse sampling rule. Typically, this is done according to certain rules (e.g., selecting sub-aperture images every few rows or columns) to extract a sparse sub-aperture image array from the 7×7 array, such as a 3×3 or 5×5 array. This results in a sparsely sampled set of sub-aperture images.

[0086] Next, these sparsely sampled sub-aperture images are arranged in a predetermined order to form a pseudo-video sequence. Each frame of this pseudo-video sequence corresponds to a sub-aperture image. By simulating a timeline, the sparse spatial information is transformed into time-series data, forming the pseudo-video format. This step can be achieved using software such as FFmpeg. By setting a fixed frame rate of 30 frames per second, the image sequence is synthesized into a video format, and the output is in the standard YUV420p format.

[0087] The sender then feeds the pseudo-video sequence into the VTM encoder for VVC encoding. The VTM encoder (Video Transmission Module Encoder) is based on the VVC (Versatile Video Coding) video coding standard. Through VVC encoding, we can achieve high-compression bitstream output and effectively compress the video sequence, significantly reducing the amount of data during transmission. The encoded bitstream contains information from the pseudo-video sequence and is sent to the receiver via a network or other transmission medium.

[0088] After receiving the bitstream of the pseudo-video sequence, the receiver first performs VVC decoding to recover the pseudo-video sequence. During this process, the decoder re-decodes the compressed bitstream into a video sequence, recovering the original video frames. Next, the receiver inputs the recovered pseudo-video sequence into the video extraction module for further processing. The video extraction module extracts specific sub-aperture images from each frame of the video. Since each frame of the pseudo-video sequence is actually a sub-aperture image, the extraction module extracts the image from each decoded frame, ultimately restoring it into a sparsely sampled set of sub-aperture images.

[0089] Building upon this, the receiver further processes these sparsely sampled sub-aperture image sets, converting them into a first macro-pixel image. In this invention, a macro-pixel image refers to an image with a large sampling area, after appropriate spatial transformation and mapping. The first macro-pixel image undergoes further processing by a multi-dimensional feature fusion network. The purpose of the multi-dimensional feature fusion network is to extract and fuse features from multiple dimensions of the input macro-pixel image, thereby recovering a more complete and clearer light field image.

[0090] like Figure 2 As shown, the Multidimensional Feature Fusion Module (MDFFM) comprises multiple modules, specifically including a Spatial Feature Extractor (SFE), a Dense Spatial Feature Extractor (DSSFE), an Angle Feature Extractor (AFE), a Horizontal Polar Plane Feature Extractor (EPI-S), a Vertical Polar Plane Feature Extractor (EPI-T), a Dense Spatial Fusion Module (FuseSDC), an Angle Domain Feature Fusion Module (FuseAEC), and a Fusion Convolution (Fuse). Each module extracts image features at different levels and in different directions, and achieves feature extraction and enhancement through a series of convolution operations and activation functions.

[0091] The spatial feature extraction module uses two 3×3 convolutional layers to extract basic spatial features of the image. The dilation coefficient of each convolutional layer is 3, the padding size is the same as the dilation coefficient, and the stride is 1. Each convolutional layer is followed by a LeakyReLU activation function. The dense spatial feature extraction module uses two 5×5 convolutional layers with padding of 2 and a stride of 1. Each convolutional layer is also followed by a LeakyReLU activation function. Through the combination of these two modules, the network can extract detailed spatial features from macro-pixel images.

[0092] The angle feature extraction module extracts the angle features of the image through two layers of two-dimensional convolution with a kernel size and stride of 3. The output features are rearranged into spatial dimensions through a PixelShuffle operation. The epipolar plane feature extraction module extracts EPI (Epipolar Plane Image) features in the horizontal and vertical directions using asymmetric convolution kernels ([1,9]) respectively, and fuses the features through channel concatenation and 1×1 convolution, finally outputting the joint angle and EPI features.

[0093] The extracted spatial, angular, and polar plane features will be fused in the dense spatial fusion module and the angular domain feature fusion module. The dense spatial fusion module uses a multi-scale attention mechanism to process spatial and dense spatial features and perform feature fusion, outputting a fused spatial feature map. The angular domain feature fusion module, on the other hand, uses a multi-scale attention mechanism to fuse angular features, horizontal polar plane features, and vertical polar plane features, outputting a fused angular domain feature map. Finally, these fused features will be further processed by the multi-dimensional feature fusion module to output the final multi-dimensional feature fusion map.

[0094] like Figure 3 As shown, the multi-scale attention module first weights the input feature map using a channel attention mechanism. The channel attention part obtains global information through global average pooling and generates channel attention weights through a series of convolutional operations. Next, a spatial attention mechanism processes the channel-weighted feature map spatially, obtaining spatial feature maps through average pooling and max pooling operations, and generating spatial attention weights through convolutional operations. The combination of these two attention modules enables the model to adaptively adjust the degree of attention given to different features based on information at different scales, thereby improving the quality of the reconstructed image.

[0095] The multidimensional feature fusion map is then fed into the upsampling module for resolution enhancement. The upsampling module maps the feature map to an angular resolution-related space through a convolutional layer, then expands the channel dimensions through a 1×1 convolution, and then rearranges the channel features to the spatial dimensions through a PixelShuffle operation. Finally, dilated convolution completes feature aggregation and resolution enhancement to obtain the restored second macropixel image.

[0096] Finally, the second macropixel image is inversely transformed to recover the sub-aperture image, thereby ultimately reconstructing the light field image. Through these steps, the receiver can not only efficiently transmit the light field image, but also effectively recover a high-quality light field image through a feature fusion network.

[0097] In summary, the transmission method based on multi-dimensional feature fusion of light field images proposed in this invention, through techniques such as sparse sampling, video coding, and multi-dimensional feature fusion, can ensure effective image quality recovery while maintaining the transmission efficiency of light field images, and has broad application prospects.

[0098] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0099] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for transmitting light field images based on multi-dimensional feature fusion of light field images, characterized in that, Includes the following steps: Step S1: The sender converts the light field image into a sub-aperture image, performs sparse sampling on the sub-aperture image, and obtains a set of sparsely sampled sub-aperture images; Step S2: Combine the sparsely sampled sub-aperture images into a pseudo-video sequence; Step S3: Input the pseudo-video sequence into the VTM encoder for VVC encoding to obtain the pseudo-video sequence bitstream, and send the pseudo-video sequence bitstream to the receiver; Step S4: The receiver receives the pseudo video sequence bitstream and restores the pseudo video sequence bitstream to obtain the pseudo video sequence. Step S5: Input the pseudo-video sequence into the video extraction module to extract sub-aperture images and output a sparsely sampled sub-aperture image set; Step S6: Convert the sparsely sampled sub-aperture image set into a first macro-pixel image; Step S7: Input the first macro-pixel image into the multi-dimensional feature fusion network, output the second macro-pixel image, convert the output second macro-pixel image into a sub-aperture image, and obtain the light field image through the sub-aperture image; The process of inputting the first macro-pixel image into a multi-dimensional feature fusion network and outputting the second macro-pixel image includes the following steps: The first macro-pixel image is input into a convolutional layer of a multi-dimensional feature fusion network, and the first feature map is output. The first feature map is input into the spatial feature extractor, dense spatial feature extractor, angular feature extractor, horizontal polar plane feature extractor, and vertical polar plane feature extractor of the multi-dimensional feature fusion module for feature extraction, and the spatial feature map, local spatial feature map, angular feature map, horizontal feature map, and vertical feature map are output respectively. The spatial feature map and the local spatial feature map are input into the dense spatial fusion module for feature fusion, and the second feature map is output. The angular feature map, horizontal feature map, and vertical feature map are input into the angular domain feature fusion module for feature fusion, and the third feature map is output. The second and third feature maps are input into the fusion convolution of the multi-dimensional feature fusion module, and the fourth feature map is output. The fourth feature map is input into the subsequent multi-dimensional feature fusion module, the last multi-dimensional feature fusion module outputs the multi-dimensional feature fusion feature map, the multi-dimensional feature fusion feature map is input into the upsampling module, and the second macro-pixel image is output.

2. The light field image transmission method based on multi-dimensional feature fusion of light field images according to claim 1, characterized in that, Step S1 includes the following steps: Multiple sub-aperture images are extracted from the light field image. Each sub-aperture image represents a two-dimensional image from a specific viewpoint within the light field image. The extracted sub-aperture images are: Sub-aperture image array; right The sub-aperture image array is sparsely sampled according to a preset sparse sampling rule to obtain a sparsely sampled sub-aperture image set.

3. The light field image transmission method based on multi-dimensional feature fusion of light field images according to claim 1, characterized in that, The step of creating a pseudo-video sequence from a set of sparsely sampled sub-aperture images includes: arranging the set of sparsely sampled sub-aperture images in a preset order to generate a pseudo-video sequence, wherein each sub-aperture image corresponds to a frame in the video, simulating the order of the timeline.

4. The light field image transmission method based on multi-dimensional feature fusion of light field images according to claim 1, characterized in that, Step S5 includes the following steps: The pseudo-video sequence is input into the video extraction module. Each frame Fi of the pseudo-video sequence corresponds to a sub-aperture image. The video extraction module analyzes the input pseudo-video sequence and extracts the viewpoint image in each frame. The video extraction module will perform position analysis on each frame of the image according to the preset sparse sampling rules, and extract the sub-aperture image at a specific position from each frame of the pseudo video sequence. The extracted sub-aperture image set is output as a sparsely sampled sub-aperture image set.

5. The light field image transmission method based on multi-dimensional feature fusion of light field images according to claim 1, characterized in that, The multidimensional feature fusion network includes a convolutional layer, multiple identical multidimensional feature fusion modules, and an upsampling module connected in sequence.

6. The light field image transmission method based on multi-dimensional feature fusion of light field images according to claim 5, characterized in that, The multiple identical multidimensional feature fusion modules are connected using a residual connection method.

7. The light field image transmission method based on multi-dimensional feature fusion of light field images according to claim 5, characterized in that, The multidimensional feature fusion module includes a spatial feature extractor, a dense spatial feature extractor, an angular feature extractor, a horizontal polar plane feature extractor, a vertical polar plane feature extractor, a dense spatial fusion module, an angular domain feature fusion module, and a fusion convolution.

8. The light field image transmission method based on multi-dimensional feature fusion of light field images according to claim 7, characterized in that, The spatial feature extractor includes two identical... Convolutional layers have an inflation factor of 3, padding size that matches the inflation factor, and a stride of 1. Each convolutional layer is followed by a LeakyReLU activation function. The dense spatial feature extractor includes two identical 5×5 convolutional layers with padding of 2 and stride of 1. Each convolutional layer is followed by a LeakyReLU activation function. The angle feature extractor includes two identical 2D convolution and PixelShuffle modules, with a 2D convolution kernel size and stride of 3; The horizontal polar plane feature extractor includes an asymmetric convolution kernel of size [1, 9], a channel splicing module, and a 1×1 convolution; The vertical polar plane feature extractor includes an asymmetric convolution kernel of size [9, 1], a channel splicing module, and a 1×1 convolution.

9. A light field image transmission method based on multi-dimensional feature fusion of light field images according to claim 7, characterized in that, The dense spatial fusion module includes a multi-scale attention mechanism module and a spatial fusion convolution. The spatial fusion convolution has 2 input channels and 1 output channel, and uses LeakyReLU (0.1) for non-linear activation. The corner feature fusion module includes a multi-scale attention mechanism module and a corner fusion convolution. The corner fusion convolution has 3 input channels and 1 output channel, and uses LeakyReLU (0.1) for non-linear activation.

Images