A light field angle super-resolution reconstruction method, device, medium and product

Abstract

The application discloses a light field angle super-resolution reconstruction method and device, medium and product, and relates to the field of light field imaging. The method comprises the following steps: obtaining a high-angle resolution light field image by using a light field image reconstruction model according to a low-angle resolution light field image; and the process of generating the high-angle resolution light field image by the light field image reconstruction model comprises the following steps: performing initial convolution processing on the light field image; extracting multi-stage fused angle residual features of the initial convolution features by using a plurality of cascaded multi-feature fusion extraction groups; performing non-integer multiple up-sampling on the angle dimension of the angle residual features by using an up-sampling module; adding the up-sampled features and an initial light field image obtained by bicubic interpolation up-sampling to generate the high-angle resolution light field image. The application can improve the image detail recovery quality while ensuring the geometric consistency between angles.

Description

Technical Field

[0001] This application relates to the field of light field imaging, and in particular to a method, device, medium, and product for super-resolution reconstruction of light field angles. Background Technology

[0002] With the rapid development of light field imaging technology, it has shown broad application prospects in fields such as virtual reality (VR), augmented reality (AR), 3D reconstruction, and free-viewpoint video. Light field cameras, by introducing a microlens array between the main lens and the image sensor, can simultaneously record the spatial position and propagation direction of light, generating light field data with a four-dimensional structure. However, limited by the physical structure of the microlens array, there is an unavoidable trade-off between spatial and angular resolution in light field imaging; that is, it is difficult to guarantee high angular resolution while obtaining high spatial resolution, and vice versa. Light Field Super-Resolution (LFSR) technology aims to reconstruct high-quality spatial or angular information from low-resolution light fields using algorithms such as deep learning. Among many research tasks, Angular Super-Resolution (ASR) is one of the important directions in light field research because it directly affects free-viewpoint rendering and multi-view consistency.

[0003] Angular Super-Resolution (ASR) of light field images aims to generate dense viewpoint images from sparse sub-aperture images to improve the angular sampling density of the light field, supporting applications such as free viewpoint rendering, dynamic focusing, and 3D reconstruction. To address the issues of geometric consistency and detail recovery in the viewpoint synthesis process, researchers both domestically and internationally have proposed various methods, which can be mainly divided into three categories: traditional geometric methods, deep learning methods, and implicit neural representation methods.

[0004] Early methods relied on the geometry of the light field and parallax information. Existing techniques propose variational optimization methods based on epipolar images, utilizing the linear structure properties in EPIs to generate intermediate viewpoints through frequency domain interpolation. While this method maintains geometric coherence between viewpoints, it suffers from high computational complexity and poor adaptability to noise and complex textures. Existing techniques also employ Sheared EPI representations, enhancing the correlation between viewpoints through frequency domain filtering, and then generating dense viewpoints through inverse transformation. This approach performs well in regular scenes but has limited effectiveness when dealing with complex geometry or dynamic objects.

[0005] In recent years, data-driven deep learning methods have become mainstream. Existing techniques propose a deep learning-based viewpoint synthesis framework, synthesizing new viewpoints through depth estimation and geometric transformations, but they are sensitive to occlusion regions and depth errors. Existing techniques propose high-order residual networks, combining local and global residual learning to extract spatial-angular features; GANet combines disparity estimation with viewpoint synthesis and introduces consistency loss, showing good performance in occluded regions. The EPI-Volume network designed in existing techniques utilizes 3D convolution to capture sub-pixel motion between viewpoints, but its generalization ability to real data is limited. Existing techniques use adversarial learning to improve synthesis quality, but the training process is unstable; further, existing techniques propose viewpoint consistency GANs, effectively reducing artifacts and improving visual quality.

[0006] In recent years, implicit neural representation methods, such as the new paradigm of angle synthesis, NeRF, have emerged. NeRF models the light field as a continuous radiation field and implicitly encodes the geometric and appearance information of the scene using a multilayer perceptron (MLP), enabling the generation of arbitrary new viewpoints from sparse viewpoints. While it performs excellently on synthetic data, the original NeRF suffers from low computational efficiency and poor adaptability to real light fields. LF-NeRF enhances the geometric modeling capability between viewpoints by introducing angular coordinate parameterization and explicitly modeling the correlation between viewpoints, but it still requires significant computational resources and training data, and its robustness needs improvement.

[0007] Light field images typically employ a two-plane model. ,in Indicates angular resolution. This refers to spatial resolution. To better analyze parallax in a light field, pixels from the same row / column of all images at a single horizontal / vertical viewpoint are often stacked into a 2D image, known as an Epipolar Plane Image (EPI). EPI projects four-dimensional data onto a two-dimensional plane by fixing one spatial dimension and one angular dimension of the light field, forming an image with a linear structure. For example, it can fix... and To obtain the horizontal EPI-HH, i.e. Similarly, the vertical EPI-VV can be obtained, i.e. This representation method has become a core tool for geometric analysis in light field processing.

[0008] Traditional angular super-resolution methods primarily rely on interpolation algorithms (such as bilinear interpolation) or disparity-guided viewpoint synthesis techniques. However, these methods make strong assumptions about scene geometry and struggle to handle complex dynamic changes in real-world scenes. In recent years, deep learning-based methods have significantly improved the performance of angular super-resolution through end-to-end feature learning. For example, convolutional neural network (CNN)-based methods (such as LF-ATO and LFSSR) generate intermediate views through multi-view feature fusion; Transformer-based methods (such as LFT and EPIT) utilize self-attention mechanisms to model long-range dependencies between views. However, existing methods still have limitations. For instance, most methods fuse views by stacking images or simply stitching features together, failing to fully exploit the potential correlations within the 4D structure of the light field, leading to information redundancy and high computational complexity. In occluded or high disparity regions, existing methods struggle to maintain disparity consistency between views, resulting in structural misalignment or artifacts in the generated images.

[0009] To address the aforementioned issues, there is an urgent need for a novel light field angle super-resolution reconstruction method that can improve the quality of image detail restoration while ensuring geometric consistency between angles. Summary of the Invention

[0010] The purpose of this application is to provide a method, device, medium, and product for super-resolution reconstruction of light field angles, which can improve the quality of image detail recovery while ensuring geometric consistency between angles.

[0011] To achieve the above objectives, this application provides the following solution: In a first aspect, this application provides a method for super-resolution reconstruction of light field angles, the method comprising: Construct a light field image reconstruction model; Acquire a low-angle resolution light field image; the low-angle resolution light field image is a sub-aperture image with a sparse viewpoint; Based on the low-angle resolution light field image, a light field image reconstruction model is used to obtain a high-angle resolution light field image; The process by which the light field image reconstruction model generates a high-angular-resolution light field image is as follows: The light field image is subjected to initial convolution processing to obtain initial convolution features; Multiple cascaded multi-feature fusion extraction groups are used to extract angular residual features after multi-stage fusion of initial convolutional features. Each multi-feature fusion extraction group includes multiple multi-feature fusion extraction modules and an angle residual group. The multi-feature fusion extraction module is used to extract multi-dimensional features, including spatial features, angular features, and EPI features, from the initial convolutional features and fuse the multi-dimensional features. The angle residual group is used to fuse angular features through multiple cascaded angle residual blocks and learn angular features by introducing a channel attention mechanism. An upsampling module is used to perform non-integer multiple upsampling of the angle residual features in the angle dimension; the upsampling module is used to reconstruct the relationship between the channel dimension and the pixel arrangement, and to perform non-integer multiple upsampling of the angle dimension. The upsampled features are added to the initial light field image obtained by bicubic interpolation upsampling to generate a high-angular-resolution light field image.

[0012] Optionally, the multi-feature fusion extraction module includes: a parallel spatial feature extraction branch, an angular feature extraction branch, and a multi-feature EPI fusion branch, as well as a fusion module; The spatial feature extraction branch is used to extract spatial dimension features; the angle feature extraction branch is used to extract angle dimension features; the multi-feature EPI fusion branch is used to extract and fuse multiple EPI features. The fusion module is used to integrate the features output by the parallel spatial feature extraction branch, the angle feature extraction branch, and the multi-feature EPI fusion branch.

[0013] Optionally, the multi-feature EPI fusion branch includes: a multi-scale separable EPI extraction module; The multi-scale separable EPI extraction module is used for multi-branch parallel processing to extract the angle and EPI features of different EPI image features.

[0014] Optionally, the angle residual group includes multiple cascaded angle residual blocks; each angle residual block learns angle features through convolution and channel attention mechanisms.

[0015] Optionally, the processing procedure of the upsampling module is as follows: The angular residual features are transformed and reshaped; Based on the reshaped features, channel upsampling is performed using 1×1 convolution; Based on the upsampled features, the features are rearranged using a pixel rearrangement method.

[0016] Optionally, the light field image reconstruction model is implemented based on the PyTorch framework.

[0017] Optionally, the light field image reconstruction model is trained and tested on an NVIDIA 3090Ti GPU.

[0018] Secondly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the light field angle super-resolution reconstruction method described above.

[0019] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned light field angle super-resolution reconstruction method.

[0020] Fourthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the aforementioned light field angle super-resolution reconstruction method.

[0021] According to the specific embodiments provided in this application, this application has the following technical effects: This application provides a method, device, medium, and product for optical field angle super-resolution reconstruction. It generates high-angle-resolution optical field images from low-angle-resolution images using an optical field image reconstruction model. Specifically, it achieves efficient modeling and feature fusion for geometric consistency between viewpoints by utilizing the generalized EPI representation and multi-scale separable convolution mechanism within multiple cascaded multi-feature fusion extraction groups (IMEGs) in the optical field image reconstruction model. Through an upsampling module and a multi-feature fusion extraction structure, more flexible and efficient angle view reconstruction is achieved. Furthermore, the network's ability to model key viewpoint features is enhanced through Angular Residual Groups (ARGs) and a channel attention mechanism, improving image detail recovery quality while maintaining geometric consistency. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a schematic diagram of a light field angle super-resolution reconstruction method in one embodiment of this application; Figure 2 A schematic diagram of the light field image reconstruction model structure; Figure 3 This is a schematic diagram of the multi-feature fusion extraction module structure; Figure 4 This is a schematic diagram of the structure of a multi-scale separable EPI extraction module; Figure 5 This is a schematic diagram of the angular residual group structure. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0026] In one exemplary embodiment, such as Figure 1 As shown, a method for super-resolution reconstruction of light field angles is provided, which includes the following steps S1011 to S103. Wherein: S101, Construct a light field image reconstruction model, such as Figure 2 As shown; S102, acquire a low-angle resolution light field image; the low-angle resolution light field image is a sparse-view sub-aperture image (SAI). S103, Based on the low-angle resolution light field image, a light field image reconstruction model is used to obtain a high-angle resolution light field image; The process by which the light field image reconstruction model generates a high-angular-resolution light field image is as follows: S1, Perform initial convolution processing on the light field image to obtain initial convolution features; Specifically, the light field image undergoes a 3×3 convolution to obtain initial convolutional features. Where H and W represent the height and width of each SAI, U=V=A represent the angular resolution, and C is the number of channels; S2, multiple cascaded multi-feature fusion extraction groups are used to extract the angular residual features after multi-stage fusion of the initial convolutional features; each multi-feature fusion extraction group includes multiple multi-feature fusion extraction blocks (IMEBs) and angular residual groups (ARGs); the multi-feature fusion extraction blocks are used to extract multi-dimensional features including spatial features, angular features and EPI features from the initial convolutional features, and fuse the multi-dimensional features; the angular residual groups are used to fuse angular features through multiple cascaded angular residual blocks, and angular features are learned by introducing a channel attention mechanism. As a specific implementation, four IMEGs are cascaded to gradually decouple different view features from a small number of light field image views to obtain more angular information. The main goal is to effectively model the redundancy and correlation of light field images in the angular dimension. Each IMEG contains multiple sub-modules for extracting angular features (multi-feature fusion extraction modules), capturing local and global structural information in the sparse view domain in a progressive manner.

[0027] Specifically, each IMEG contains four multi-feature fusion extraction modules, a 3×3 convolution, and an angle residual group, which progressively integrate feature information from different viewpoints through a hierarchical cascade approach. Simultaneously, the feature transfer structure between IMEGs helps maintain consistency and coherence of features across viewpoints. Specifically, the output features of the i-th IMEG... It can be represented as: ; in, This represents the output feature of the i-th IMEG. The number of IMEG values. This indicates the internal feature processing procedure. This represents the high-dimensional features of the light field image after 3×3 convolution. This layer-by-layer propagation and fusion structure enables IMEG to effectively encode disparity relationships and nonlocal correlations in the angular domain, laying a solid foundation for subsequent high-quality angular image restoration.

[0028] The multi-feature fusion extraction module includes: a parallel spatial feature extraction branch (Spa Conv), an angular feature extraction branch (Ang Conv), and a multi-feature EPI fusion branch (MEFM), as well as a fusion module (Fuse Module). The spatial feature extraction branch is used to extract spatial dimension features; the angle feature extraction branch is used to extract angle dimension features; the multi-feature EPI fusion branch is used to extract and fuse multiple EPI features. The fusion module is used to integrate the features output by the parallel spatial feature extraction branch, the angle feature extraction branch, and the multi-feature EPI fusion branch.

[0029] IMEB is responsible for extracting and fusing spatial, angular, and EPI multidimensional features from light field images. Through multi-branch parallel processing and residual learning mechanisms, IMEB can fully exploit the geometric correlations and complementary information between light field viewpoints. The main components of IMEB are as follows: Figure 3 As shown in part (a), it consists of three main branches: spatial feature extraction branch, angular feature extraction branch, and multi-feature EPI fusion branch. The output features of the three branches are dynamically integrated through the fusion module to ultimately form a unified feature representation.

[0030] Figure 3 The spatial feature extraction branch in part (b) consists of two 3×3 convolutional layers with LeakyReLU; Figure 3 The angle feature extraction branch in part (c) consists of two convolutional layers, A×A and 1×1, with LeakyReLU, and recovers the high-frequency information of the high-angle resolution light field image through pixel shuffle. In the multi-feature EPI fusion module, four different EPI image features (EPI-VV, EPI-HH, EPI-HV, EPI-VH) are input into a multi-scale separable EPI extraction module (MSEB) to extract angle and EPI features. EPI-HV and EPI-VH are extensions of traditional EPI; specifically, by fixing the angle dimension... and spatial dimensions The light field data is stacked along the horizontal angle and horizontal space to form a new two-dimensional slice. And denoted as EPI-VH; through a fixed angular dimension and spatial dimensions The light field data is stacked along the vertical angle and vertical space to form another two-dimensional slice. This is denoted as EPI-HV. The high-dimensional features obtained after MSEB processing... , , and The dimensions of the aforementioned high-dimensional features are all The above high-dimensional features are concatenated to obtain the features after MEFM processing. .

[0031] Features extracted from these three branches , , The input features are connected and spatial information is merged through a fusion module. The fusion module consists of a 1×1 convolution with LeakyReLU and a 3×3 convolutional layer. The integrated high-dimensional features can be represented as... Specifically, after passing through each IMEB module, the high-dimensional features are further learned, that is: ; in, This represents the output feature of the image after passing through the j-th IMEB in the i-th IMEG. This indicates the processing procedure within the IMEB module. This represents the number of IMEBs.

[0032] like Figure 4As shown, the multi-scale separable EPI extraction module is used for multi-branch parallel processing to extract angle and EPI features of different EPI images. In each MSEB, multi-scale separable EPI convolution is used for multi-branch parallel processing, which improves model performance without excessively increasing the number of model parameters, and uses dilated convolution kernels with different dilation rates. Each branch has the same convolution kernel size A, but different dilation rates (1,1), (1,2), (1,4), which means that the receptive field in the horizontal direction will be approximately 1-3 times larger than A. This allows the model to better learn sub-pixel information within different disparity ranges and further learn the spatial angle correlation on the EPI image.

[0033] like Figure 5 As shown, the ARG is composed of multiple Angular Residual Blocks (ARBs) connected in series. Each ARB performs feature extraction and enhancement within a local range. The entire ARG aggregates multi-level features through residual connections to achieve deeper angular feature learning.

[0034] Specifically, each ARB works together with convolution and channel attention mechanisms to fully extract discriminative features between viewpoints and adaptively adjust feature responses using global context information to improve the quality of information reconstruction in the angular direction.

[0035] Let the output of the last IMEB module in the i-th IMEG be represented as Then the features after stacking and fusing the residuals of K ARBs can be expressed as: ; in, This represents the processing operation of the k-th ARB. This represents the convolution operation. Let represent a convolutional layer with a kernel size of 3×3 and a stride of 1. The final output features of the ARG are then dimensionally integrated through a 3×3 convolution operation and form a residual connection with the initial input. The output features of the i-th IMEG are further represented as: ; The ARG structure in this application helps maintain stable feature transfer while enhancing the model's ability to model detailed information in the angular direction. Through the cascading of multiple ARBs, the ARG can deeply explore the correlation between different viewpoints in the angular dimension, effectively improving the angular representation capability of the light field image, thereby enhancing the final angular super-resolution reconstruction effect. The introduction of the ARG enables the network to not only fully integrate angular features but also to use attention mechanisms to filter redundant information, achieving more efficient and accurate angular dimension modeling.

[0036] S3, using the upsampling module to perform non-integer multiple upsampling of the angle residual features in the angle dimension; the upsampling module is used to reconstruct the relationship between the channel dimension and the pixel arrangement, and to perform non-integer multiple upsampling of the angle dimension. After performing depth extraction and processing on the light field angular features, it is necessary to upsample the light field features to the angular resolution of the target. Due to the upsampling factor... It may be a non-integer value (e.g., in angular super-resolution ranging from 2×2 to 7×7). Therefore, directly applying the pixel shuffle method for upsampling at non-integer multiples is not feasible. To address this, a downsampling-upsampling method for angle upsampling is employed; specifically, the high-dimensional features generated after the last IMEG are first... Transform and reshape into Then, 1×1 convolution is used to perform channel upsampling to obtain... Finally, pixel rearrangement is used to rearrange the features. .

[0037] S4. Add the upsampled features to the initial light field image obtained by bicubic interpolation upsampling to generate a high-angular-resolution light field image. ,in, This indicates the magnification factor.

[0038] The light field image reconstruction model (GMS-Net-ASR) provided in this application is implemented based on the PyTorch framework and trained and tested on an NVIDIA 3090Ti GPU. The structural parameters of the light field image reconstruction model are set as follows: I=4 (number of IMEG groups), J=4 (number of modules in each group), and C=64 (number of channels).

[0039] The L1 loss function is used during training, and the Adam optimizer is selected with the following parameters: , The initial learning rate is set to The batch size is reduced to half every 15 epochs, for a total of 70 epochs, with a batch size of 4.

[0040] During the testing phase, images were converted to the YCbCr color space, and PSNR and SSIM were calculated on the Y channel as evaluation metrics. Specifically, the evaluation method involved first calculating the average score for the 45 newly generated views in the LFASR2×2→7×7 task, then averaging the scores for each scene to obtain the overall performance for each dataset. Additionally, FLOPs were calculated to measure the model's computational efficiency.

[0041] This application has the following effects: (1) Multi-view modeling of generalized EPI: Horizontal-vertical (EPI-HV) and vertical-horizontal (EPI-VH) EPI subspaces are proposed. Subpixel displacement information between viewpoints is captured through multi-directional slicing strategy to enhance the adaptability to irregular parallax.

[0042] (2) Multi-magnification angle-spatial collaborative upsampling: To address the issues of viewpoint alignment and continuity in optical field angle super-resolution, a separable upsampling module based on the angle dimension was designed. By reconstructing the channel dimension and pixel arrangement relationship, the limitation that pixel shuffle cannot directly support non-integer magnification was overcome. This effectively improved the adaptability and flexibility of angle upsampling, significantly enhanced the support for arbitrary magnification angle interpolation tasks while maintaining the network's lightweight nature, avoided feature misalignment and viewpoint discontinuity, and fully ensured the maintenance of geometric consistency between viewpoints.

[0043] (3) Multi-feature fusion architecture and angle residual modeling mechanism: A structural module IMEG is proposed that integrates multi-feature fusion extraction block IMEB and angle residual group ARG. ARG introduces a channel attention mechanism while fusing angle features by cascading multiple angle residual blocks ARB, which adaptively enhances key perspective information and effectively improves the consistency and expressive power of perspective features.

[0044] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal. The computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is connected to the system bus via the I / O interfaces. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The I / O interfaces of the computer device are used for exchanging information between the processor and external devices. The communication interface of the computer device is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a light field angle super-resolution reconstruction method.

[0045] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0046] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0047] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0048] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0049] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0050] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchain. The processors involved in the embodiments provided in this application may be, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc.

[0051] In this application, all actions to acquire signals, information, or data are carried out in compliance with the relevant data protection laws and policies of the country where the location is situated, and with the authorization granted by the owner of the relevant device.

[0052] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0053] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for super-resolution reconstruction of light field angles, characterized in that, The light field angle super-resolution reconstruction method includes: Construct a light field image reconstruction model; Acquire a low-angle resolution light field image; the low-angle resolution light field image is a sub-aperture image with a sparse viewpoint; Based on the low-angle resolution light field image, a light field image reconstruction model is used to obtain a high-angle resolution light field image; The process by which the light field image reconstruction model generates a high-angular-resolution light field image is as follows: The light field image is subjected to initial convolution processing to obtain initial convolution features; Multiple cascaded multi-feature fusion extraction groups are used to extract angular residual features after multi-stage fusion of initial convolutional features. Each multi-feature fusion extraction group includes multiple multi-feature fusion extraction modules and an angle residual group. The multi-feature fusion extraction module is used to extract multi-dimensional features, including spatial features, angular features, and EPI features, from the initial convolutional features and fuse the multi-dimensional features. The angle residual group is used to fuse angular features through multiple cascaded angle residual blocks and learn angular features by introducing a channel attention mechanism. An upsampling module is used to upsample the angle residual features at non-integer multiples in the angle dimension. This upsampling module is used to reconstruct the relationship between channel dimensions and pixel arrangement, performing non-integer multiple upsampling in the angle dimension. Specifically, it upsamples the high-dimensional features generated after the last multi-feature fusion extraction group. Transform and reshape into features Then, 1×1 convolution is used to perform channel upsampling to obtain features. Finally, pixel rearrangement is used to rearrange the features into a new feature set. ;in, For the set of real numbers, and As an angle dimension, and Let A be the spatial dimension, and let A be the branches with the same convolution kernel size. C is the upsampling factor, and C is the number of channels; The upsampled features are added to the initial light field image obtained by bicubic interpolation upsampling to generate a high angular resolution light field image; The multi-feature fusion extraction module includes: a parallel spatial feature extraction branch, an angular feature extraction branch, and a multi-feature EPI fusion branch, as well as a fusion module; The spatial feature extraction branch is used to extract spatial dimension features; the angle feature extraction branch is used to extract angle dimension features; the multi-feature EPI fusion branch is used to extract and fuse multiple EPI features. The fusion module is used to integrate the features output by the parallel spatial feature extraction branch, the angle feature extraction branch, and the multi-feature EPI fusion branch; The multi-feature EPI fusion branch includes: a multi-scale separable EPI extraction module; The multi-scale separable EPI extraction module is used for multi-branch parallel processing to extract the angle and EPI features of different EPI image features.

2. The light field angle super-resolution reconstruction method according to claim 1, characterized in that, The angle residual group comprises multiple cascaded angle residual blocks; each angle residual block learns angle features through convolution and channel attention mechanisms.

3. The light field angle super-resolution reconstruction method according to claim 1, characterized in that, The light field image reconstruction model is implemented based on the PyTorch framework.

4. The light field angle super-resolution reconstruction method according to claim 1, characterized in that, The light field image reconstruction model was trained and tested on an NVIDIA 3090Ti GPU.

5. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the light field angle super-resolution reconstruction method according to any one of claims 1-4.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the light field angle super-resolution reconstruction method as described in any one of claims 1-4.

7. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the light field angle super-resolution reconstruction method as described in any one of claims 1-4.

Images