Light field batch representation method and system based on implicit neural network

Abstract

The present application relates to the technical field of computer vision, and provide a kind of light field batch characterization method and system based on implicit neural network, method includes initialization scene prompt set and view angle prompt set;Through the weight group of learnable, scene prompt set and view angle prompt set are weighted summation, obtain scene weighted prompt and view angle weighted prompt;Merging scene weighted prompt and view angle weighted prompt obtains joint prompt;Joint prompt is input into implicit neural network and obtains reconstructed light field image;The mean square error loss between reconstructed light field image and original light field image is calculated, and implicit neural network weight, scene prompt set and view angle prompt set and the weight group of learnable are updated by back propagation;Repeat the above steps, complete the batch characterization of light field.The present application solves the batch characterization problem of light field through single implicit neural network, and realizes the light field compression reconstruction task of high efficiency and high quality.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method and system for batch representation of light fields based on implicit neural networks. Background Technology

[0002] Light fields record the intensity and direction of light rays in three-dimensional space. Compared to ordinary planar images, light field imaging systems capture three-dimensional spatial information through angular dimensions, providing users with a better immersive experience and improving the performance of many downstream computer vision tasks. However, on the other hand, light fields need to record different light ray directions, which inevitably leads to a significant degree of redundancy and high data storage requirements, thus limiting the efficiency of light field transmission and the universality of hardware support. Therefore, developing high-performance compression methods is a crucial factor in promoting the application of light field technology.

[0003] Early research focused on directly compressing microlens images captured by light field cameras using the High Efficiency Video Coding (HEVC-Intra) framework, but this coding-based solution had limited performance. Subsequent research used sorted light field images as pseudo-video sequences, then applied video compression standards or video compression learning networks for compression. Compression methods based on view synthesis separate compression and reconstruction into encoding and decoding ends. The latter reconstructs the complete light field from the received key aperture images using view rendering methods, achieving the goal of light field compression. Furthermore, depth-based light field coding and decoding systems and multi-dimensional light field compression methods utilizing 4D redundancy of the light field have emerged.

[0004] The emergence of neural radiation fields has provided a new approach to light field compression, and research has begun to focus on using the weights in implicit neural networks to represent light field information. Methods based on implicit neural representations establish a connection between light field compression and neural network compression, allowing for further compression using methods such as pruning, quantization, and tensor rank optimization. For example, by introducing a dual-core dynamic encoding mechanism that divides neural network weights into recording scene description information and information from different viewpoints, the current best compression representation method for single light fields, SAIs (Single-Aperture Images), has been obtained. Based on the implicit neural radiation representation of single light fields, a joint representation method for light fields of thousands of orders of magnitude, SVD-Siren (a multi-field sinusoidal representation method based on singular value decomposition), has been implemented.

[0005] Despite significant progress in various studies, existing light field compression methods still suffer from the following limitations: pseudo-video sequence-based methods exhibit large performance differences between keyframes and non-keyframes; view synthesis-based methods show significant performance degradation in complex environments and under strong noise interference; transform-based methods are mostly applicable only to narrow baseline light fields; implicit neural network-based methods are only applicable to single light fields, and the few existing multi-light field compression methods perform poorly under low bandwidth. Currently, a method that maintains high representation efficiency while representing light fields in batches is still lacking. Summary of the Invention

[0006] This invention aims to at least solve one of the technical problems existing in related technologies. To this end, this invention provides a method and system for batch representation of light fields based on implicit neural networks. By using a single implicit neural network, the problem of batch representation of light fields is solved, achieving high efficiency and high quality in completing the task of light field compression and reconstruction.

[0007] This invention provides a method for batch characterization of light fields based on implicit neural networks, comprising: S1: Initialize the scene cue set and view cue set based on the original light field image; S2: Obtain scene-weighted cues by weighting the set of scene cues using learnable weighted reassemblies, and obtain viewpoint-weighted cues by weighting the set of viewpoint cues using learnable weighted reassemblies. S3: Combine scene-weighted cues and viewpoint-weighted cues to obtain a joint cue; S4: Input the joint cue into the implicit neural network, and obtain the reconstructed light field image through multiple iterations of convolution and upsampling, and then through the frame convolutional layer and frame activation layer. S5: Calculate the mean squared error loss between the reconstructed light field image and the original light field image, and update the implicit neural network weights, scene cue set, view cue set, and learnable weight reassembly through backpropagation; S6: Repeat steps S2 to S5 to complete the batch characterization of the light field.

[0008] Furthermore, the shape of the scene cue set is ,in, For the number of scenes, Number of scene channels Provides the height of the scene cue matrix. The width of the scene matrix; The shape of the set is indicated by the viewpoint. ,in, The number of vertical viewing angles. The number of horizontal viewpoints, Number of view channels To indicate the height of the matrix for the viewpoint, Provides a viewpoint indication of the matrix width; The scene cue set and the view cue set have the same height and the same width.

[0009] Furthermore, in step S2, obtaining scene-weighted hints includes: S211: Extracting scenarios from learnable weighted reconfigurations The corresponding weighted reorganization has the following shape: , S212: Scene The corresponding weighting and shape are The scene hint set is dimensionally compressed in terms of the number of scenes, retaining the channel, height, and width dimensions, resulting in the scene... Scene-weighted prompts, shape is .

[0010] Furthermore, in step S2, the view-weighted hints include row hints and column hints, and obtaining the view-weighted hints includes: S221: Extract the hints from all rows below the column, resulting in a shape of... Slices; S222: Extracting perspectives from learnable weight reconfigurations Corresponding equity restructuring; S223: Perspective The corresponding weighting and shape are The slices in Dimensional compression is performed, preserving the channel, height, and width dimensions, to obtain the shape as follows: perspective Line position hints; S224: Extract the hints for all columns below the row, resulting in a shape of... Slices; S225: Perspective The corresponding weighting and shape are The slices in Dimensional compression is performed, preserving the channel, height, and width dimensions, to obtain the shape as follows: perspective The following is a prompt from the team.

[0011] Furthermore, dimensional compression is achieved through Einstein's summation convention.

[0012] Furthermore, the implicit neural network includes five decoder single-layer structures, each including a decoder two-dimensional convolutional layer, an upsampling layer, a batch normalization layer, and a decoder activation layer.

[0013] Furthermore, the calculation expression for the single-layer structure of the decoder is as follows: in, This is the output of the decoder's single-layer structure. For each layer's input, the shape is... , For the number of channels, For height, For width, This is a two-dimensional convolutional layer operation. For upsampling layer operations, For batch normalization layer operations, This is the operation for activating the decoder layer.

[0014] Furthermore, in step S4, the frame convolutional layer is a two-dimensional convolutional layer, initialized using Kaiming, and the frame activation layer uses... Activation function.

[0015] Furthermore, in step S6, multiple images are trained and their weights are updated simultaneously using multidimensional tensors from the PyTorch library.

[0016] The present invention also provides a light field batch characterization system based on an implicit neural network for performing any of the above-described light field batch characterization methods based on an implicit neural network, comprising: An initialization module is used to initialize a scene cue set and a view cue set based on the original light field image; The weighted prompt module is used to perform weighted summation on the set of scene prompts through learnable weighted recombinations to obtain scene-weighted prompts, and to perform weighted summation on the set of view prompts through learnable weighted recombinations to obtain view-weighted prompts. The merging module is used to merge scene-weighted cues and viewpoint-weighted cues to obtain a joint cues; The reconstruction module is used to input the joint cue into the implicit neural network, and obtain the reconstructed light field image through multiple iterations of convolution and upsampling, and through frame convolutional layers and frame activation layers. The update module is used to calculate the mean square error loss between the reconstructed light field image and the original light field image, and to update the implicit neural network weights, scene cue set, view cue set, and learnable weight reassembly through backpropagation. A batch characterization module is used to perform batch characterization of the light field.

[0017] The above-described one or more technical solutions in the embodiments of the present invention have at least one of the following technical effects: By displaying and distinguishing light field information and viewpoint information as input, the light field compression and reconstruction task can be completed with high efficiency and high quality.

[0018] This method uses learnable cues as network input to participate in the training of implicit neural networks. By setting different sets of cues, it can simultaneously focus on both scene and perspective information, and learn light field information through cues and weights.

[0019] By introducing learnable weight reassemblies, similar information in both the scene and perspective components is identified, thereby improving parameter utilization. The parameter ratios of the cue set and network weights are balanced to achieve a more reasonable independent-shared ratio.

[0020] The design scheme is based on an image-by-image approach and incorporates batch training to make the most of the computing device's performance and perceive more information at the same time. Random sampling is used to collect data from the dataset to accelerate the training and convergence efficiency of the network under all lighting conditions.

[0021] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0023] Figure 1 This is a flowchart illustrating a method for batch characterization of light fields based on implicit neural networks provided by the present invention.

[0024] Figure 2 This is a schematic diagram of the model framework for a batch representation method of light field based on implicit neural networks provided by the present invention.

[0025] Figure 3 This is a comparison diagram of the center image error of the light field image reconstructed according to an embodiment of the present invention.

[0026] Figure 4 This is a comparison chart of the peak signal-to-noise ratio of the reconstructed light field image from different viewpoints according to an embodiment of the present invention.

[0027] Figure 5 This is a schematic diagram of the structure of a batch light field characterization system based on an implicit neural network provided by the present invention.

[0028] Figure label: 101. Initialization module; 102. Weighted suggestion module; 103. Merging module; 104. Reconstruction module; 105. Update module; 106. Batch representation module. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention. The following embodiments are used to illustrate this invention but cannot be used to limit the scope of this invention.

[0030] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0031] The following is combined with Figures 1 to 5 This invention describes a method and system for batch characterization of light fields based on implicit neural networks.

[0032] This invention addresses the batch representation of light fields using a single implicit neural network, focusing on the relationships and balances between different angles: how to guide the network to accurately distinguish different viewpoints within the same scale of light fields; how to enable the network to clearly differentiate between different light field scenes; and how to allow a single network to effectively learn the unique features of multiple light fields. To achieve these goals, this invention uses explicit viewpoints and scene cues as inputs, enabling a shared-weight neural network to represent light fields in batches.

[0033] like Figure 1 As shown, a method for batch characterization of light fields based on implicit neural networks includes: S1: Initialize the scene cue set and view cue set based on the original light field image; For all viewpoint ranges and all light field scenes that need to be represented, the cue sets for viewpoint and scene are organized separately using random initialization; the scene cue set and the angle cue set are the original input sets. The shape of the scene cue set is ,in, For the number of scenes, Number of scene channels Provides the height of the scene cue matrix. The width of the scene matrix; The shape of the set is indicated by the viewpoint. ,in, The number of vertical viewing angles. The number of horizontal viewpoints, Number of view channels To indicate the height of the matrix for the viewpoint, Provides a viewpoint indication of the matrix width.

[0034] The scene cue set and the view cue set have the same height and the same width.

[0035] The set of scene cues and the set of viewpoint cues are essentially multiple overlapping sets. A two-dimensional matrix, where, The height of the two-dimensional matrix. The width of the two-dimensional matrix.

[0036] In some specific embodiments of the present invention , .

[0037] The dimensions gradually increase in both height and width until they match the original image. The remaining dimensions are sliced ​​or compressed to unify them into the channel dimension, and the number of channels continues to decrease until the RGB three-color channels are obtained.

[0038] Both the scene hint set and the view hint set use tensor data types to facilitate various operations and match the batch function designed in this invention.

[0039] S2: Obtain scene-weighted cues by weighting the set of scene cues using learnable weighted reassemblies, and obtain viewpoint-weighted cues by weighting the set of viewpoint cues using learnable weighted reassemblies. Learnable weight reorganization includes scene weight reorganization and perspective weight reorganization. Scene weight reorganization is used to perform weighted summation on the set of scene cues to obtain scene-weighted cues, and perspective weight reorganization is used to perform weighted summation on the set of perspective cues to obtain perspective-weighted cues.

[0040] Obtaining scene-weighted hints involves the following steps: S211: Extracting Scenes from Scene Weight Restructuring The corresponding weighted reorganization has the following shape: , S212: Scene The corresponding weighting and shape are The scene hint set is dimensionally compressed in terms of the number of scenes, retaining the channel, height, and width dimensions, resulting in the scene... Scene-weighted prompts, shape is .

[0041] The logic of multiplication and summation to compress dimensions is described using Einstein's summation convention, and the calculation expression is: in, For the scene Context-weighted hints, with dimensions as follows: , Let Einstein's summation convention function be used. For the scene The weight matrix has dimensions of , This is a set of scene prompts.

[0042] View-weighted hints include row hints and column hints.

[0043] The view-weighted hints include: S221: Extract the hints from all rows below the column, resulting in a shape of... Slices; S222: Extracting perspective from perspective weight reorganization Corresponding equity restructuring; S223: Perspective The corresponding weighting and shape are The slices in Dimensional compression is performed, preserving the channel, height, and width dimensions, to obtain the shape as follows: perspective Line position hints; The method for obtaining column hints corresponds to that for row hints. S224: Extract the hints for all columns below the row, resulting in a shape of... Slices; S225: Perspective The corresponding weighting and shape are The slices in Dimensional compression is performed, preserving the channel, height, and width dimensions, to obtain the shape as follows: perspective The following is a prompt from the team.

[0044] Scene weight reassembly and viewpoint weight reassembly are randomly initialized before training and optimized as updatable parameters during backpropagation gradient descent to better characterize the similarity and difference between different cues.

[0045] Scene weight reorganization and viewpoint weight reorganization are also included in the total number of parameters, but due to their small size, they are ignored in the content related to balance tips and weight parameter proportions.

[0046] S3: Combine scene-weighted cues and viewpoint-weighted cues to obtain a joint cue; because , Therefore, merging and splicing are performed directly along the channel dimension to obtain the shape as follows. The joint reminder.

[0047] S4: Input the joint cue into the implicit neural network, and obtain the reconstructed light field image through multiple iterations of convolution and upsampling, and then through the frame convolutional layer and frame activation layer. An implicit neural network is a multi-layer neural network with a decoder architecture, consisting of five decoder single-layer structures. Each decoder single-layer structure includes a decoder 2D convolutional layer, an upsampling layer, a batch normalization layer, and a decoder activation layer. The decoder's two-dimensional convolutional layer is the part where parameters are mainly piled up. It is responsible for sensing and extracting features in the image, while also reducing the number of channels. The upsampling layer doubles the width and height of the tensor output by the two-dimensional convolutional layer. It uses a double cubic interpolation algorithm to consider the influence of the gray values ​​of neighboring points and the gray change rate to obtain a higher quality interpolated image. Batch normalization is used to keep the input distribution stable and avoid oscillations during gradient propagation.

[0048] During forward propagation, the input sequentially passes through the decoder's 2D convolutional layer, upsampling layer, batch normalization layer, and decoder activation layer. The tensor output from the decoder activation layer is either fed back into the next decoder single-layer structure or passed to the frame convolutional layer. The model frame is as follows: Figure 2 As shown.

[0049] The calculation expression for a single-layer structure of the decoder is: in, This is the output of the decoder's single-layer structure. For each layer's input, the shape is... , For the number of channels, For height, For width, This is a two-dimensional convolutional layer operation. For upsampling layer operations, For batch normalization layer operations, This is the operation for activating the decoder layer.

[0050] If using the batch function, add the batch dimension before the first dimension, and the shape will change. , For batch number, The frame convolutional layer is a two-dimensional convolutional layer, using Kaiming initialization, and the frame activation layer uses... Activation function.

[0051] S5: Calculate the mean squared error loss between the reconstructed light field image and the original light field image, and update the implicit neural network weights, scene cue set, view cue set, and learnable weight reassembly through backpropagation; The tensor processed by the 5-layer decoder single-layer structure, and then passed through the frame convolutional layer, has its shape changed. Select As an activation function, the tensor, after activation, becomes the number in this iteration. Scene OK For the reconstructed sub-aperture images, the reference objects for error calculation during backpropagation are the source image and the reconstructed sub-aperture image. The mean squared error loss (MSE) is chosen as the reference for the loss function, and the peak signal-to-noise ratio (PSNR), which is logarithmically related to the MSE loss, is used as the evaluation metric. The expression for the loss function is as follows: in, To rebuild the losses, Let the mean squared error loss function be . For the corresponding light field In the sub-aperture image matrix located in row and column A single image at a location, To rebuild the network, The number of rows or columns.

[0052] S6: Repeat steps S2 to S5 to complete the batch characterization of the light field.

[0053] Simultaneous training and weight updates of multiple images can be performed using multidimensional tensors from the PyTorch library.

[0054] Representing light fields using neural networks essentially involves training a network that is sufficiently fitted, or even overfitted, to the corresponding light field. The ability to quickly achieve the desired training effect is another crucial factor in evaluating technical performance, besides compression quality. This invention leverages the multidimensional tensor settings in the PyTorch library (an open-source deep learning framework for machine learning and deep learning) to introduce batch training capabilities. This allows for simultaneous training and weight updates of multiple images on a computing device, significantly accelerating the training speed.

[0055] When organizing training samples, the scenes and locations are selected completely randomly. During training, the network fully experiences changes in the scene and location to avoid repeated training on a specific or nearby sample, which could lead to unnecessary oscillations when encountering other samples. Compared to methods that train replacement weights on only one image at a time and methods that train weights pixel-by-pixel, the method designed in this invention offers a significant advantage in training speed.

[0056] This invention designs a multi-light field batch representation framework (SAPR) based on implicit neural networks, which uses light field information and viewpoint information as input to complete the light field compression and reconstruction task with high efficiency and high quality.

[0057] In terms of information learning, a method using learnable cues as network input to participate in the training of implicit neural networks is proposed. By setting different sets of cues, both scene and perspective information can be focused on simultaneously, and light field information can be learned through cues and weights.

[0058] In terms of parameter utilization, a learnable weight reassembly is introduced to indicate similar information in both the scene and perspective parts, thereby improving parameter utilization. The parameter ratio of the cue set and network weights is balanced to achieve a more reasonable independent-shared ratio.

[0059] In terms of training efficiency, the scheme is designed according to the image-by-image approach and batch training design is added to make the most of the computing device performance and perceive more information in the same time. Random sampling is used to collect data from the dataset to accelerate the training convergence efficiency of the network under all light fields.

[0060] To evaluate the effectiveness of this experimental design, the test data consisted of four light fields from the classic EPFL light field dataset: Bikes (bicycle scene), Danger (dangerous scene), Fountain (fountain scene), and Stone (rock / static scene). This light field dataset has unique and representative characteristics. Due to the vignetting effect, only 9×9 sub-aperture images at the center of each light field were used to compose the dataset. The microlens matrix weakens the light intensity reaching the sensor, resulting in relatively noisy captured light field images containing some noise, which allows for additional measurement and comparison of the network's noise robustness performance. The four light fields have low content similarity to verify the performance of a wider range of multi-light field compression.

[0061] This experiment compares the performance of current state-of-the-art methods in the field of optical field compression with those tested on the given data. SAIs-LFC (Structure Adaptive Iterative Filtering) and DDLF (Dynamic Deep Learning Filtering) are single-field compression schemes, while SVD-Siren (Singular Value Decomposition-based Feature Enhancement) is a multi-field compression scheme. The comparative experiment uses the average total number of parameters in a single optical field as a reference to evaluate the peak signal-to-noise ratio (PSNR) of different methods in the four optical fields with the same number of parameters. The reconstruction results are shown in Table 1. A higher PSNR value indicates a smaller difference between the comparison image and the source image.

[0062] Table 1. Reconstruction effects of existing solutions and the method of this invention in different light fields. Table 1 shows the reconstruction performance of the present invention and existing schemes in the above four scenarios when near convergence. The present invention has 827k parameters. Although SVD-Siren uses Float32 weight data type, the number of parameters in SVD-Siren is kept close to that of the present invention at 842k. SAIs-LFC and DDLF train a network for each optical field separately. Each network in SAIs-LFC has 206k parameters, and each network in DDLF has 208k parameters. The results in Table 1 show that the present invention significantly outperforms the SVD-Siren method and the single-optical-field method DDLF under the same parameter settings, while maintaining similar performance to the SAIs-LFC method.

[0063] Figure 3 This section presents an error map showing the center viewpoint of each light field in the output of each method after training, compared to the original image. Blue to red indicates that the error between that pixel and the original image gradually increases. Figure 3 As can be clearly seen, the reddish portion of the error map for SVD-Siren is much larger than that for SAIs-LFC and this invention, while the performance of SAIs-LFC and this invention is basically similar. This indicates that the performance of this invention is close to that of SAIs-LFC, the current best single-field performance, and significantly better than that of SVD-Siren, which corresponds to the results of the comparison using the PSNR index.

[0064] also, Figure 4 The average PSNR performance of different schemes in four light fields is shown. The horizontal axis represents the row-first viewpoint split, arranging the 9×9 two-dimensional viewpoints in one dimension. It can be observed that the closer the reconstructed light field image is to the center viewpoint, the higher its peak signal-to-noise ratio, which is consistent with the representation characteristics of implicit neural networks trained image by image. Compared with other schemes in the horizontal direction, the line graph shows a significant advantage, indicating the effectiveness of the present invention.

[0065] like Figure 5 As shown, a batch light field characterization system based on an implicit neural network is used to execute a batch light field characterization method based on an implicit neural network, including: Initialization module 101 is used to initialize the scene cue set and the view cue set based on the original light field image; The weighted prompt module 102 is used to perform weighted summation on the set of scene prompts through learnable weighted reassemblies to obtain scene weighted prompts, and to perform weighted summation on the set of view prompts through learnable weighted reassemblies to obtain view weighted prompts. The merging module 103 is used to merge scene-weighted cues and viewpoint-weighted cues to obtain a joint cue; The reconstruction module 104 is used to input the joint cue into the implicit neural network, and obtain the reconstructed light field image through multiple iterations of convolution and upsampling, and through frame convolutional layers and frame activation layers. The update module 105 is used to calculate the mean square error loss between the reconstructed light field image and the original light field image, and to update the implicit neural network weights, scene cue set, view cue set, and learnable weight reassembly through backpropagation; The batch characterization module 105 is used to complete the batch characterization of the light field.

[0066] Through the collaborative work of the above modules, the problem of batch representation of light fields is solved by a single implicit neural network, thereby achieving high efficiency and high quality in completing the light field compression and reconstruction task.

[0067] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for batch characterization of light fields based on implicit neural networks, characterized in that, include: S1: Initialize the scene cue set and view cue set based on the original light field image; The shape of the scene cue set is ,in, For the number of scenes, Number of scene channels Provides the height of the scene cue matrix. The width of the scene matrix; The shape of the set is indicated by the viewpoint. ,in, The number of vertical viewing angles. The number of horizontal viewpoints, Number of view channels To indicate the height of the matrix for the viewpoint, Provides a viewpoint indication of the matrix width; The scene cue set and the view cue set have the same height and the same width; S2: Obtain scene-weighted cues by weighting the set of scene cues using learnable weighted reassemblies, and obtain viewpoint-weighted cues by weighting the set of viewpoint cues using learnable weighted reassemblies. S3: Combine scene-weighted cues and viewpoint-weighted cues to obtain a joint cue; S4: Input the joint cue into the implicit neural network, and obtain the reconstructed light field image through multiple iterations of convolution and upsampling, and then through the frame convolutional layer and frame activation layer. S5: Calculate the mean squared error loss between the reconstructed light field image and the original light field image, and update the implicit neural network weights, scene cue set, view cue set, and learnable weight reassembly through backpropagation; S6: Repeat steps S2 to S5 to complete the batch characterization of the light field.

2. The method for batch characterization of light fields based on implicit neural networks according to claim 1, characterized in that, In step S2, the scene-weighted hints obtained include: S211: Extracting scenarios from learnable weighted reconfigurations The corresponding weighted reorganization has the following shape: , S212: Scene The corresponding weighting and shape are The scene hint set is dimensionally compressed in terms of the number of scenes, retaining the channel, height, and width dimensions, resulting in the scene... Scene-weighted prompts, shape is .

3. The method for batch characterization of light fields based on implicit neural networks according to claim 1, characterized in that, In step S2, view-weighted hints include row hints and column hints. The view-weighted hints are obtained by: S221: Extract column The hints from all the lines below give the shape as Slices; S222: Extracting perspectives from learnable weight reconfigurations Corresponding equity restructuring; S223: Perspective The corresponding weighting and shape are The slices in Dimensional compression is performed, preserving the channel, height, and width dimensions, to obtain the shape as follows: perspective Line position hints; S224: Truncate line The hints in all columns below give the shape as Slices; S225: Perspective The corresponding weighting and shape are The slices in Dimensional compression is performed, preserving the channel, height, and width dimensions, to obtain the shape as follows: perspective The following is a prompt from the team.

4. The method for batch characterization of light fields based on implicit neural networks according to claim 1, characterized in that, Dimensional compression is achieved through Einstein's summation convention.

5. The method for batch characterization of light fields based on implicit neural networks according to claim 1, characterized in that, The implicit neural network includes five decoder single-layer structures, each consisting of a decoder two-dimensional convolutional layer, an upsampling layer, a batch normalization layer, and a decoder activation layer.

6. The method for batch characterization of light fields based on implicit neural networks according to claim 5, characterized in that, The calculation expression for a single-layer structure of the decoder is: in, This is the output of the decoder's single-layer structure. For each layer's input, the shape is... , For the number of channels, For height, For width, This is a two-dimensional convolutional layer operation. For upsampling layer operations, For batch normalization layer operations, This is the operation for activating the decoder layer.

7. The method for batch characterization of light fields based on implicit neural networks according to claim 1, characterized in that, In step S4, the frame convolutional layer is a two-dimensional convolutional layer, initialized using Kaiming, and the frame activation layer uses... Activation function.

8. The method for batch characterization of light fields based on implicit neural networks according to claim 1, characterized in that, In step S6, multiple images are trained and their weights are updated simultaneously using multidimensional tensors from the PyTorch library.

9. A batch representation system for light fields based on implicit neural networks, characterized in that, To perform a batch representation method of light field based on an implicit neural network as described in any one of claims 1 to 8, comprising: An initialization module is used to initialize a scene cue set and a view cue set based on the original light field image; The weighted prompt module is used to perform weighted summation on the set of scene prompts through learnable weighted recombinations to obtain scene-weighted prompts, and to perform weighted summation on the set of view prompts through learnable weighted recombinations to obtain view-weighted prompts. The merging module is used to merge scene-weighted cues and viewpoint-weighted cues to obtain a joint cues; The reconstruction module is used to input the joint cue into the implicit neural network, and obtain the reconstructed light field image through multiple iterations of convolution and upsampling, and through frame convolutional layers and frame activation layers. The update module is used to calculate the mean square error loss between the reconstructed light field image and the original light field image, and to update the implicit neural network weights, scene cue set, view cue set, and learnable weight reassembly through backpropagation. A batch characterization module is used to perform batch characterization of the light field.

Images