Light field image transmission method based on light field image multi-dimensional feature fusion

Abstract

The invention discloses a light field image transmission method based on light field image multi-dimensional feature fusion. The method comprises the following steps: a sender converts a light field image into a sub-aperture image and carries out sparse sampling to obtain a sparse sampling sub-aperture image set; and the sub-aperture image sets are converted into a pseudo video sequence, VVC coding is carried out through a VTM encoder, and a pseudo video sequence bit stream is generated and sent to a receiver. And after receiving the bit stream, a receiver decodes the bit stream, restores the pseudo video sequence, and extracts a sparse sampling sub-aperture image set through a video extraction module. And converting the set into a first macro-pixel image, and inputting the first macro-pixel image into a multi-dimensional feature fusion network for further processing. The network fuses multi-dimensional features through a plurality of feature extraction modules, and reconstructs a light field image through the image. According to the method, by introducing multi-dimensional feature fusion and a multi-scale attention mechanism, the transmission efficiency and quality of the light field image are improved, and the method is particularly suitable for an environment with limited bandwidth.

Description

Technical Field

[0001] The present invention belongs to the technical field of light field image processing, and in particular relates to a light field image transmission method based on multi-dimensional feature fusion of light field images. Background Art

[0002] Light field technology is an advanced imaging technology that can capture the spatial and angular information of a scene. It is widely used in fields such as virtual reality, augmented reality, computational photography, and 3D modeling. However, since light field data contains a large number of sub-aperture images, the amount of data is extremely large, and transmission and storage place high demands on bandwidth and storage resources. Therefore, how to efficiently encode light field data to reduce the number of transmitted views and improve transmission efficiency has become a key issue in current research. In response to the above problems, it is necessary to develop an efficient light field coding framework that fully utilizes the spatial and angular characteristics of light field images to improve coding performance and reduce transmission bandwidth requirements. At the same time, it enhances the processing capability of viewpoint correlation at the decoding end and improves the visual quality of the decoded image, thereby providing a more efficient solution for the practical application of light field technology. Summary of the invention

[0003] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a light field image transmission method based on multi-dimensional feature fusion of light field images.

[0004] The purpose of the present invention can be achieved by the following technical solutions:

[0005] The present invention provides a light field image transmission method based on multi-dimensional feature fusion of light field images, comprising the following steps:

[0006] Step S1: The sender converts the light field image into a sub-aperture image, performs sparse sampling on the sub-aperture image, and obtains a set of sparsely sampled sub-aperture images;

[0007] Step S2: assembling the sparsely sampled sub-aperture images into a pseudo video sequence;

[0008] Step S3: input the pseudo video sequence into the VTM encoder for VVC encoding to obtain a pseudo video sequence bit stream, and send the pseudo video sequence bit stream to the receiver;

[0009] Step S4: The receiving party receives the pseudo video sequence bit stream, and restores the pseudo video sequence bit stream to obtain a pseudo video sequence;

[0010] Step S5: input the pseudo video sequence into the video extraction module to extract sub-aperture images, and output a sparsely sampled sub-aperture image set;

[0011] Step S6: converting the sparsely sampled sub-aperture image set into a first macro-pixel image;

[0012] Step S7: input the first macro-pixel image into a multi-dimensional feature fusion network, output a second macro-pixel image, convert the output second macro-pixel image into a sub-aperture image, and obtain a light field image through the sub-aperture image.

[0013] Furthermore, the step S1 includes the following steps:

[0014] Extracting multiple sub-aperture images from the light field image, each sub-aperture image represents a two-dimensional image at a specific viewing angle in the light field image, and the extracted sub-aperture images are an n×n sub-aperture image array;

[0015] The n×n sub-aperture image array is sparsely sampled according to a preset sparse sampling rule to obtain a sparsely sampled sub-aperture image set.

[0016] Furthermore, the method of forming a pseudo video sequence from a sparsely sampled sub-aperture image set includes: arranging the sparsely sampled sub-aperture image set in a preset order to generate a pseudo video sequence, wherein each sub-aperture image corresponds to a frame in the video, simulating the order of the timeline.

[0017] Furthermore, step S5 includes the following steps:

[0018] Input the pseudo video sequence into the video extraction module, wherein each frame Fi of the pseudo video sequence corresponds to a sub-aperture image, and the video extraction module analyzes the input pseudo video sequence and extracts the view image in each frame;

[0019] The video extraction module will perform position analysis on each frame of the image according to the preset sparse sampling rules, and extract the sub-aperture image at a specific position from each frame of the pseudo video sequence;

[0020] The extracted sub-aperture image set is output as a sparsely sampled sub-aperture image set.

[0021] Furthermore, the multi-dimensional feature fusion network includes a convolutional layer, a plurality of identical multi-dimensional feature fusion modules and an upsampling module connected in sequence.

[0022] Furthermore, the multiple identical multi-dimensional feature fusion modules are connected in a residual connection manner.

[0023] Furthermore, the multidimensional feature fusion module includes a spatial feature extractor, a dense spatial feature extractor, an angle feature extractor, a horizontal polar plane feature extractor, a vertical polar plane feature extractor, a dense spatial fusion module, an angular domain feature fusion module, and a fused convolution.

[0024] Furthermore, the spatial feature extractor includes two identical 3×3 convolutional layers, the expansion coefficient of the convolutional layer is 3, the padding size is consistent with the expansion coefficient, the stride is 1, and each convolutional layer is followed by a LeakyReLU activation function;

[0025] The dense spatial feature extractor includes two identical 5×5 convolutional layers with padding of 2 and stride of 1, and each convolutional layer is followed by a LeakyReLU activation function;

[0026] The angle feature extractor includes two identical two-dimensional convolution and PixelShuffle modules, and the two-dimensional convolution kernel size and stride are 3;

[0027] The horizontal polar plane feature extractor includes an asymmetric convolution kernel of size [1,9], a channel splicing module and a 1×1 convolution;

[0028] The vertical polar plane feature extractor includes an asymmetric convolution kernel of size [9, 1], a channel splicing module and a 1×1 convolution.

[0029] Furthermore, the dense spatial fusion module includes a multi-scale attention mechanism module and a spatial fusion convolution. The spatial fusion convolution has 2 channels as input channels and 1 channel as output channels, and uses LeakyReLU (0.1) for nonlinear activation.

[0030] The angular domain feature fusion module includes a multi-scale attention mechanism module and an angular domain fusion convolution. The angular domain fusion convolution has 3 channels as input channels and 1 channel as output channels, and uses LeakyReLU (0.1) for nonlinear activation.

[0031] Furthermore, the step of inputting the first macro-pixel image into a multi-dimensional feature fusion network and outputting a second macro-pixel image comprises the following steps:

[0032] Inputting the first macro-pixel image into a convolutional layer of a multi-dimensional feature fusion network, and outputting a first feature map;

[0033] The first feature map is respectively input into the spatial feature extractor, the dense spatial feature extractor, the angle feature extractor, the horizontal polar plane feature extractor, and the vertical polar plane feature extractor of the multidimensional feature fusion module for feature extraction, and the spatial feature map, the local spatial feature map, the angle feature map, the horizontal feature map, and the vertical feature map are respectively output;

[0034] Input the spatial feature map and the local spatial feature map into the dense spatial fusion module for feature fusion, and output a second feature map;

[0035] The angle feature map, the horizontal feature map, and the vertical feature map are input into the angle domain feature fusion module for feature fusion, and a third feature map is output;

[0036] Input the second feature map and the third feature map into the fused convolution of the multi-dimensional feature fusion module, and output a fourth feature map;

[0037] The fourth feature map is input into the subsequently connected multidimensional feature fusion module, the last multidimensional feature fusion module outputs the multidimensional feature fusion feature map, the multidimensional feature fusion feature map is input into the upsampling module, and the second macro-pixel image is output.

[0038] Compared with the prior art, the present invention has the following advantages:

[0039] (1) The present invention reduces the amount of data in the light field image transmission process by converting the light field image into a sub-aperture image and performing sparse sampling to generate a pseudo video sequence. This method effectively reduces data redundancy and reduces transmission bandwidth requirements.

[0040] (2) The present invention performs multi-dimensional feature fusion on the restored image through a multi-dimensional feature fusion network, and combines the fusion of multiple features such as space, dense space, angle, and polar plane, which significantly improves the accuracy and detail expression of light field image restoration, and can better reconstruct various viewing angles of the light field image.

[0041] (3) The present invention applies a multi-scale attention mechanism in the dense space fusion module and the angular domain feature fusion module, which can weight important features according to the specific content of the image, thereby better improving the quality and visual effect of the image and avoiding the interference of irrelevant information.

[0042] (4) The present invention can effectively process light field images of different complexity by using convolution and attention mechanisms of different scales, and can show good adaptability and processing effects in various light field image transmission tasks. Whether it is a low-resolution or high-resolution light field image, the present invention can maintain a good image restoration effect.

[0043] (5) Due to the use of sparse sampling and pseudo video sequence, the present invention can effectively reduce the computational burden and reduce transmission delay, and is suitable for the needs of real-time light field image transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is a flow chart of the present invention;

[0045] Figure 2 It is the overall process framework diagram of the present invention;

[0046] Figure 3 This is a module diagram of the multi-scale attention mechanism of the present invention. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0048] Embodiment 1:

[0049] This embodiment provides a light field image transmission method based on multi-dimensional feature fusion of light field images. Figure 1 As shown, the following steps are included:

[0050] Step S1: The sender converts the light field image into a sub-aperture image, performs sparse sampling on the sub-aperture image, and obtains a set of sparsely sampled sub-aperture images;

[0051] Step S2: assembling the sparsely sampled sub-aperture images into a pseudo video sequence;

[0052] Step S3: input the pseudo video sequence into the VTM encoder for VVC encoding to obtain a pseudo video sequence bit stream, and send the pseudo video sequence bit stream to the receiver;

[0053] Step S4: The receiving party receives the pseudo video sequence bit stream, and restores the pseudo video sequence bit stream to obtain a pseudo video sequence;

[0054] Step S5: input the pseudo video sequence into the video extraction module to extract sub-aperture images, and output a sparsely sampled sub-aperture image set;

[0055] Step S6: converting the sparsely sampled sub-aperture image set into a first macro-pixel image;

[0056] Step S7: input the first macro-pixel image into a multi-dimensional feature fusion network, output a second macro-pixel image, convert the output second macro-pixel image into a sub-aperture image, and obtain a light field image through the sub-aperture image.

[0057] Wherein, step S1 comprises the following steps:

[0058] Extracting multiple sub-aperture images from the light field image, each sub-aperture image represents a two-dimensional image at a specific viewing angle in the light field image, and the extracted sub-aperture images are an n×n sub-aperture image array;

[0059] The n×n sub-aperture image array is sparsely sampled according to a preset sparse sampling rule to obtain a sparsely sampled sub-aperture image set.

[0060] The sparsely sampled sub-aperture image set is formed into a pseudo video sequence, including: arranging the sparsely sampled sub-aperture image set in a preset order to generate a pseudo video sequence, wherein each sub-aperture image corresponds to a frame in the video, simulating the order of the time axis.

[0061] Wherein, step S5 comprises the following steps:

[0062] The pseudo video sequence is input into the video extraction module. Each frame Fi of the pseudo video sequence corresponds to a sub-aperture image. The frequency extraction module analyzes the input pseudo video sequence and extracts the view image in each frame.

[0063] The video extraction module will perform position analysis on each frame of the image according to the preset sparse sampling rules, and extract the sub-aperture image at a specific position from each frame of the pseudo video sequence;

[0064] The extracted sub-aperture image set is output as a sparsely sampled sub-aperture image set.

[0065] The multi-dimensional feature fusion network includes a convolutional layer, a plurality of identical multi-dimensional feature fusion modules and an upsampling module connected in sequence.

[0066] Among them, multiple identical multi-dimensional feature fusion modules are connected using residual connections.

[0067] Among them, the multi-dimensional feature fusion module includes spatial feature extractor, dense spatial feature extractor, angle feature extractor, horizontal polar plane feature extractor, vertical polar plane feature extractor, dense spatial fusion module, angular domain feature fusion module, and fused convolution.

[0068] The spatial feature extractor consists of two identical 3×3 convolutional layers, with a convolutional layer expansion factor of 3, a padding size that is consistent with the expansion factor, a stride of 1, and each convolutional layer is followed by a LeakyReLU activation function.

[0069] The dense spatial feature extractor consists of two identical 5×5 convolutional layers with padding of 2 and stride of 1. Each convolutional layer is followed by a LeakyReLU activation function.

[0070] The angle feature extractor consists of two identical 2D convolution and PixelShuffle modules with a 2D convolution kernel size and stride of 3;

[0071] The horizontal polar plane feature extractor includes an asymmetric convolution kernel of size [1,9], a channel splicing module and a 1×1 convolution;

[0072] The vertical polar plane feature extractor consists of an asymmetric convolution kernel of size [9, 1], a channel splicing module and a 1×1 convolution.

[0073] Among them, the dense spatial fusion module includes a multi-scale attention mechanism module and a spatial fusion convolution. The number of input channels of the spatial fusion convolution is 2 channels, the number of output channels is channel, and LeakyReLU (0.1) is used for nonlinear activation;

[0074] The corner domain feature fusion module includes a multi-scale attention mechanism module and corner domain fusion convolution. The corner domain fusion convolution has 3 input channels and 1 output channel, and uses LeakyReLU (0.1) for nonlinear activation.

[0075] The first macro-pixel image is input into a multi-dimensional feature fusion network, and a second macro-pixel image is output, comprising the following steps:

[0076] Inputting the first macro-pixel image into a convolutional layer of a multi-dimensional feature fusion network, and outputting a first feature map;

[0077] The first feature map is respectively input into the spatial feature extractor, the dense spatial feature extractor, the angle feature extractor, the horizontal polar plane feature extractor, and the vertical polar plane feature extractor of the multidimensional feature fusion module for feature extraction, and the spatial feature map, the local spatial feature map, the angle feature map, the horizontal feature map, and the vertical feature map are respectively output;

[0078] Input the spatial feature map and the local spatial feature map into the dense spatial fusion module for feature fusion, and output a second feature map;

[0079] The angle feature map, the horizontal feature map, and the vertical feature map are input into the angle domain feature fusion module for feature fusion, and a third feature map is output;

[0080] Input the second feature map and the third feature map into the fused convolution of the multi-dimensional feature fusion module, and output a fourth feature map;

[0081] The fourth feature map is input into the subsequently connected multidimensional feature fusion module, the last multidimensional feature fusion module outputs the multidimensional feature fusion feature map, the multidimensional feature fusion feature map is input into the upsampling module, and the second macro-pixel image is output.

[0082] Embodiment 2:

[0083] The parts not mentioned in this embodiment are the same as those in Embodiment 1.

[0084] The present invention proposes a light field image transmission method based on multi-dimensional feature fusion of light field images. The method aims to realize efficient restoration of light field images by converting light field images into a set of sparsely sampled sub-aperture images, compressing and transmitting them through video coding technology, and combining a multi-dimensional feature fusion network. The specific implementation process is as follows:

[0085] First, the sender extracts sub-aperture images from the light field image. A light field image is essentially a two-dimensional image array captured from different perspectives, and usually contains multiple sub-aperture images, each of which represents a two-dimensional image of the light field image at a specific perspective. In order to facilitate transmission and processing, a 7×7 sub-aperture image array is extracted from the light field image, and each sub-aperture image represents a different perspective. Among these sub-aperture images, we select a part of the sub-aperture images according to the preset sparse sampling rules, usually according to certain rules (for example, select sub-aperture images every few rows or columns) to extract a sparse sub-aperture image array, such as a 3×3 or 5×5 array, from the 7×7 array. In this way, we obtain a set of sparsely sampled sub-aperture images.

[0086] Next, these sparsely sampled sub-aperture images are arranged in a predetermined order to form a pseudo video sequence. Each frame of the pseudo video sequence corresponds to a sub-aperture image. By simulating the time axis, the sparse spatial information is converted into time series data to form the style of the pseudo video. This step can be achieved through software such as the FFmpeg tool. By setting a fixed frame rate of 30 frames per second, the image sequence is synthesized into a video format and output in a standard YUV420p format.

[0087] The sender then feeds the pseudo video sequence into the VTM encoder for VVC encoding. The VTM encoder (Video Transport Module Encoder) is an encoder based on the VVC (Versatile Video Coding) video coding standard. Through VVC encoding, we can achieve high compression rate bitstream output and effectively compress the video sequence, which significantly reduces the amount of data during transmission. The encoded bitstream contains the information of the pseudo video sequence and is sent to the receiver through the network or other transmission media.

[0088] After the receiver receives the bit stream of the pseudo video sequence, it first performs VVC decoding to restore the pseudo video sequence. In this process, the decoder decodes the compressed bit stream back into a video sequence and restores the original video frame. Then, the receiver inputs the restored pseudo video sequence into the video extraction module for further processing. The function of the video extraction module is to extract a specific sub-aperture image from each frame of video. Since each frame of the pseudo video sequence is actually a sub-aperture image, the extraction module extracts the decoded image of each frame of video and finally restores it to a set of sparsely sampled sub-aperture images.

[0089] On this basis, the receiver continues to process these sparsely sampled sub-aperture image sets and converts them into a first macro-pixel image. In the present invention, a macro-pixel image refers to an image with a larger area sampling, which has undergone appropriate spatial transformation and mapping. The first macro-pixel image is further processed by a multi-dimensional feature fusion network. The purpose of the multi-dimensional feature fusion network is to extract and fuse features of multiple dimensions from the input macro-pixel image, thereby restoring a more complete and clear light field image.

[0090] like Figure 2 As shown in the figure, the multi-dimensional feature fusion module MDFFM includes multiple modules, including spatial feature extractor SFE, dense spatial feature extractor DSSFE, angle feature extractor AFE, horizontal polar plane feature extractor EPI-S, vertical polar plane feature extractor EPI-T, dense spatial fusion module FuseSDC, angular domain feature fusion module FuseAEC, and fused convolution Fuse. The function of each module is to extract image features at different levels and directions, and realize feature extraction and enhancement through a series of convolution operations and activation functions.

[0091] The spatial feature extraction module uses two 3×3 convolutional layers to extract the basic spatial features of the image. The expansion coefficient of the convolutional layer is 3, the padding size is the same as the expansion coefficient, the stride is 1, and each convolution layer is followed by a LeakyReLU activation function. The dense spatial feature extraction module uses two 5×5 convolutional layers, with a padding of 2 and a stride of 1. The LeakyReLU activation function is also used after each convolution layer. Through the combination of these two modules, the network can extract detailed spatial features from macro-pixel images.

[0092] The angle feature extraction module extracts the angle features of the image through two layers of two-dimensional convolution, with a convolution kernel size and stride of 3. The output features are rearranged to the spatial dimension through the PixelShuffle operation. The polar plane feature extraction module uses asymmetric convolution kernels ([1,9]) in the horizontal and vertical directions to extract EPI (Epipolar Plane Image) features, and performs feature fusion through channel concatenation and 1×1 convolution, and finally outputs the joint angle and EPI features.

[0093] The multi-dimensional features extracted above, such as space, angle, polar plane, etc., will be fused in the dense space fusion module and the angular domain feature fusion module. The dense space fusion module uses the multi-scale attention mechanism module to process spatial features and dense space features, and performs feature fusion, and outputs the fused spatial feature map. The angular domain feature fusion module fuses angle features, horizontal polar plane features, and vertical polar plane features through a multi-scale attention mechanism, and outputs the fused angular domain feature map. Finally, these fused features will be further processed by the multi-dimensional feature fusion module, and the final multi-dimensional feature fusion map will be output.

[0094] like Figure 3 As shown in the figure, the multi-scale attention module first performs weighted processing on the input feature map through the channel attention mechanism. The channel attention part obtains global information through global average pooling, and generates channel attention weights through a series of convolution operations. Then, the spatial direction of the channel-weighted feature map is processed through the spatial attention mechanism, and the spatial feature map is obtained through average pooling and maximum pooling operations, and the spatial attention weight is generated through convolution operations. The combination of these two attention modules enables the model to adaptively adjust the degree of attention to different features according to information of different scales, thereby improving the quality of the restored image.

[0095] The multi-dimensional feature fusion map is then input into the upsampling module for resolution improvement. The upsampling module maps the feature map to a space related to the angular resolution through a convolution layer, then expands the channel dimension through a 1×1 convolution, and then rearranges the channel features to the spatial dimension through the PixelShuffle operation, and finally completes the feature aggregation and resolution improvement through the expansion convolution to obtain the restored second macro pixel image.

[0096] Finally, the second macro-pixel image is restored to a sub-aperture image through inverse transformation, thereby finally restoring the light field image. Through these steps, the receiver can not only efficiently transmit the light field image, but also effectively restore the high-quality light field image through the feature fusion network.

[0097] In summary, the transmission method based on multi-dimensional feature fusion of light field images proposed in the present invention can ensure the transmission efficiency of light field images while ensuring the effective restoration of image quality through technologies such as sparse sampling, video coding and multi-dimensional feature fusion, and has broad application prospects.

[0098] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program code.

[0099] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A light field image transmission method based on multi-dimensional feature fusion of light field images, characterized in that: The following steps are involved: Step S1: The sender converts the light field image into a sub-aperture image, performs sparse sampling on the sub-aperture image, and obtains a set of sparsely sampled sub-aperture images; Step S2: assembling the sparsely sampled sub-aperture images into a pseudo video sequence; Step S3: input the pseudo video sequence into the VTM encoder for VVC encoding to obtain a pseudo video sequence bit stream, and send the pseudo video sequence bit stream to the receiver; Step S4: The receiving party receives the pseudo video sequence bit stream, and restores the pseudo video sequence bit stream to obtain a pseudo video sequence; Step S5: input the pseudo video sequence into the video extraction module to extract sub-aperture images, and output a sparsely sampled sub-aperture image set; Step S6: converting the sparsely sampled sub-aperture image set into a first macro-pixel image; Step S7: input the first macro-pixel image into a multi-dimensional feature fusion network, output a second macro-pixel image, convert the output second macro-pixel image into a sub-aperture image, and obtain a light field image through the sub-aperture image.

2. The light field image transmission method based on multi-dimensional feature fusion of light field images according to claim 1, characterized in that: The step S1 comprises the following steps: Extracting multiple sub-aperture images from the light field image, each sub-aperture image represents a two-dimensional image at a specific viewing angle in the light field image, and the extracted sub-aperture images are an n×n sub-aperture image array; The n×n sub-aperture image array is sparsely sampled according to a preset sparse sampling rule to obtain a sparsely sampled sub-aperture image set.

3. The light field image transmission method based on multi-dimensional feature fusion of light field images according to claim 1, characterized in that: The method of forming a pseudo video sequence from a sparsely sampled sub-aperture image set includes: arranging the sparsely sampled sub-aperture image set in a preset order to generate a pseudo video sequence, wherein each sub-aperture image corresponds to a frame in the video, simulating the order of the time axis.

4. The light field image transmission method based on multi-dimensional feature fusion of light field images according to claim 1, characterized in that: The step S5 comprises the following steps: Input the pseudo video sequence into the video extraction module, wherein each frame Fi of the pseudo video sequence corresponds to a sub-aperture image, and the video extraction module analyzes the input pseudo video sequence and extracts the view image in each frame; The video extraction module will perform position analysis on each frame of the image according to the preset sparse sampling rules, and extract the sub-aperture image at a specific position from each frame of the pseudo video sequence; The extracted sub-aperture image set is output as a sparsely sampled sub-aperture image set.

5. The light field image transmission method based on multi-dimensional feature fusion of light field images according to claim 1, characterized in that: The multi-dimensional feature fusion network includes a convolution layer, a plurality of identical multi-dimensional feature fusion modules and an upsampling module which are connected in sequence.

6. The light field image transmission method based on multi-dimensional feature fusion of light field images according to claim 5, characterized in that: The multiple identical multi-dimensional feature fusion modules are connected in a residual connection manner.

7. The light field image transmission method based on multi-dimensional feature fusion of light field images according to claim 5, characterized in that: The multi-dimensional feature fusion module includes a spatial feature extractor, a dense spatial feature extractor, an angle feature extractor, a horizontal polar plane feature extractor, a vertical polar plane feature extractor, a dense spatial fusion module, an angular domain feature fusion module, and a fused convolution.

8. The light field image transmission method based on multi-dimensional feature fusion of light field images according to claim 7, characterized in that: The spatial feature extractor includes two identical 3×3 convolutional layers, the expansion coefficient of the convolutional layer is 3, the padding size is consistent with the expansion coefficient, the stride is 1, and each convolutional layer is followed by a LeakyReLU activation function; The dense spatial feature extractor includes two identical 5×5 convolutional layers with padding of 2 and stride of 1, and each convolutional layer is followed by a LeakyReLU activation function; The angle feature extractor includes two identical two-dimensional convolution and PixelShuffle modules, and the two-dimensional convolution kernel size and stride are 3; The horizontal polar plane feature extractor includes an asymmetric convolution kernel of size [1,9], a channel splicing module and a 1×1 convolution; The vertical polar plane feature extractor includes an asymmetric convolution kernel of size [9, 1], a channel splicing module and a 1×1 convolution.

9. The light field image transmission method based on multi-dimensional feature fusion of light field images according to claim 7, characterized in that: The dense spatial fusion module includes a multi-scale attention mechanism module and a spatial fusion convolution. The spatial fusion convolution has 2 channels as input channels and 1 channel as output channels, and uses LeakyReLU (0.1) for nonlinear activation. The angular domain feature fusion module includes a multi-scale attention mechanism module and an angular domain fusion convolution. The angular domain fusion convolution has 3 channels as input channels and 1 channel as output channels, and uses LeakyReLU (0.1) for nonlinear activation.

10. The light field image transmission method based on multi-dimensional feature fusion of light field images according to claim 1, characterized in that: The step of inputting the first macro-pixel image into a multi-dimensional feature fusion network and outputting a second macro-pixel image comprises the following steps: Inputting the first macro-pixel image into a convolutional layer of a multi-dimensional feature fusion network, and outputting a first feature map; The first feature map is respectively input into the spatial feature extractor, the dense spatial feature extractor, the angle feature extractor, the horizontal polar plane feature extractor, and the vertical polar plane feature extractor of the multidimensional feature fusion module for feature extraction, and the spatial feature map, the local spatial feature map, the angle feature map, the horizontal feature map, and the vertical feature map are respectively output; Input the spatial feature map and the local spatial feature map into the dense spatial fusion module for feature fusion, and output a second feature map; The angle feature map, the horizontal feature map, and the vertical feature map are input into the angle domain feature fusion module for feature fusion, and a third feature map is output; Input the second feature map and the third feature map into the fused convolution of the multi-dimensional feature fusion module, and output a fourth feature map; The fourth feature map is input into the subsequently connected multidimensional feature fusion module, the last multidimensional feature fusion module outputs the multidimensional feature fusion feature map, the multidimensional feature fusion feature map is input into the upsampling module, and the second macro-pixel image is output.

Images