A Dynamic Flock Observation Method and System Based on a Neural Hybrid Camera

Abstract

The present invention discloses a dynamic bird flock observation method and system based on a neural hybrid camera, including: Step 1, obtaining a central view and a surrounding view of the bird flock; Step 2, first training the neural central view field CI of #imgabs2# with the central view according to #imgabs0##imgabs1#, and then training the disparity field #imgabs4# of #imgabs3# with the surrounding view. Step 3, according to the bi-plane light field coordinates under a new perspective, using the neural light field network model #imgabs5# to output a panoramic view of the dynamic bird flock under the new perspective. The present invention can render images of any light field perspective only through the image data collected by the hybrid camera system, and fully exploit the light field color and angle information of the hybrid camera data. θ , and then training the disparity field #imgabs4# of #imgabs3# with the surrounding view. Step 3, according to the bi-plane light field coordinates under a new perspective, using the neural light field network model #imgabs5# to output a panoramic view of the dynamic bird flock under the new perspective. The present invention can render images of any light field perspective only through the image data collected by the hybrid camera system, and fully exploit the light field color and angle information of the hybrid camera data.

Description

Technical Field

[0001] The present invention relates to the technical fields of computer vision and computer graphics, and particularly to a method and system for observing dynamic bird flocks based on a neural hybrid camera. Background Art

[0002] In the observation of dynamic bird flocks, there are many challenges in achieving a large field of view, high resolution, and three-dimensional imaging. First of all, bird flocks usually carry out complex flight activities in a vast space, requiring the observation system to have a wide field of view to capture the dynamic changes of the entire group. Traditional imaging devices are difficult to balance a large field of view and high resolution. A too small field of view may not be able to fully cover the activities of the bird flock, while an enlarged field of view usually leads to a decrease in resolution. In addition, the flight trajectories of bird flocks are complex, often accompanied by high deformations and rapid movements, which makes it more difficult for the observation system to perform three-dimensional imaging. High-quality three-dimensional imaging requires both high spatial and temporal resolutions to ensure capturing the fine motion trajectories of the bird flock and its distribution in three-dimensional space, but this requirement often brings a large amount of data processing and storage pressure.

[0003] Neural hybrid light field cameras have significant advantages in the observation of dynamic bird flocks. First of all, it can greatly increase the imaging field of view, enabling the precise capture of bird flock activities in a larger range, while maintaining a high-resolution light field output to ensure that the observation system can obtain detailed information about the bird flock. In addition, neural hybrid light field cameras can store light field data in a neural network, significantly reducing the amount of data required for traditional light field storage. This storage method not only effectively alleviates the data processing and storage bottlenecks generated in high-resolution and large-field-of-view observations, but also improves the efficiency of data transmission and processing, enabling the observation of dynamic bird flocks to be efficiently realized under large-scale and high-precision requirements. Summary of the Invention

[0004] The purpose of the present invention is to provide a method and system for observing dynamic bird flocks based on a neural hybrid camera to meet the requirements of achieving a large field of view, high resolution, and three-dimensional imaging in the observation of dynamic bird flocks.

[0005] To achieve the above purpose, the present invention provides a method for observing dynamic bird flocks based on a neural hybrid camera, which includes:

[0006] Step 1, collecting bird flock image data with different resolutions through a hybrid camera system, where the hybrid camera system includes a camera arranged at the central viewpoint (u 0 , v 0 ) position for obtaining a central view The hybrid camera system further includes cameras arranged around the central viewpoint (u 0 , v 0 ) position for obtaining a peripheral view

[0007] Step 2: Use the central view obtained in Step 1 to train the neural light field network model for the neural central view field CI θ (x, y), where θ represents the set of network parameters of the neural central view field CI θ (x, y), and use the surrounding views obtained in Step 1 to train the neural disparity field of the neural light field network model , where φ represents the set of network parameters of the neural disparity field ; among them, the acquisition method of the neural light field network model includes:

[0008] Take the position coordinates (u, v, x, y) of the bi - plane light field as the input of the disparity propagation equation L(u, v, x, y) for anti - occlusion and anti - Lambertian radiation described in Equation (2). Through the neural central view field CI θ (x, y) and the neural disparity field represent the central view function and the disparity function d u,v (x, y) respectively, as shown in Equation (3) and Equation (4), to obtain the neural light field network model described in Equation (1)

[0009] In the formula, represents the physical process of obtaining the surrounding view of the light field view point (u, v) from the central view and the disparity map relative to the central view point ;

[0010] Step 3: According to the bi - plane light field coordinates (u, v, x, y) under the new view, use the neural light field network model obtained in Step 2 to map the bi - plane light field coordinates (u, v, x, y) to the corresponding color values, so as to obtain the image information of the dynamic bird flock at this position, and then output the panoramic view of the dynamic bird flock under the new view.

[0011] Furthermore, the training method of the neural central view field CI θ (x, y) in Step 2 specifically includes: Obtain the neural central view field CI θ (x, y) using the loss function described in Equation (5):

[0012]

[0013] In the formula, is for CI θ (x, y) and The reconstruction loss, represents the central view with a resolution of αH×αW in the biplane light field of π xy The set of coordinates of the plane, || represents taking the absolute value.

[0014] Furthermore, the neural disparity field in step 2 The training method specifically includes: using the loss function described in Equation (9) to obtain the neural disparity field

[0015]

[0016] where λ 1 , λ 2 , λ 3 are hyperparameters, is the structure-aware smoothness loss described in Equation (6), is the structural similarity index loss described in Equation (7), is the predicted value of the surrounding view described in Equation (8) and the reconstruction loss of the surrounding view and the of the surrounding view;

[0017]

[0018] In the formula, x S respectively represent the surrounding views with a resolution of H×W respectively in the π uv plane and π xy plane coordinate sets in the biplane light field, represents the gradient along the horizontal and vertical directions, ⊙ represents the Hadamard product, and η represents the adjustment parameter that controls the strength of edge sensitivity, represents the disparity map of the light field viewpoint (u, v) relative to the central viewpoint (u 0 , v 0 ), || represents taking the absolute value, and SSIM is an index used to measure the similarity of two images.

[0019] Furthermore, λ 1 = 1, λ 2 = 0.1, λ 3 = 0.01.

[0020] The present invention also provides a dynamic bird flock observation system based on a neural hybrid camera, which includes:

[0021] Bird flock image acquisition unit, which is used to acquire bird flock image data with different resolutions through a hybrid camera system. The hybrid camera system includes a camera arranged at the central viewpoint (u 0 , v 0 ) position for obtaining the central view The hybrid camera system further includes cameras arranged at positions around the central viewpoint (u 0 , v 0 ) for obtaining the surrounding views

[0022] Neural light field network model creation unit, which is used to utilize the central view of the bird flock image acquisition unit to train the neural central view field CI (x, y) of the neural light field network model, where θ represents the set of network parameters of the neural central view field CI θ (x, y), and utilize the surrounding views of the bird flock image acquisition unit θ to train the neural disparity field of the neural light field network model , where φ represents the set of network parameters of the neural disparity field ; among them, the acquisition method of the neural light field network model includes: Taking the position coordinates (u, v, x, y) of the biplane light field as the input of the disparity propagation equation L(u, v, x, y) for anti-occlusion and anti-Langmuir radiation described in Equation (2), and passing through the neural central view field CI

[0023] (x, y) and the neural disparity field θ respectively represent the central view function and the disparity function d (x, y), as shown in Equation (3) and Equation (4), to obtain the neural light field network model described in Equation (1) u,v In the formula,

[0024] wherein, represents the physical process of obtaining the surrounding view of the light field viewpoint (u, v) from the central view and the disparity map relative to the central viewpoint ;

[0025] Scene reconstruction unit, which is used to map the biplane light field coordinates (u, v, x, y) at a new perspective to the corresponding color values by using the neural light field network model in step 2 so as to obtain the image information of the dynamic bird flock at this position, and further output the panoramic view of the dynamic bird flock at the new perspective.

[0026] Furthermore, the training method of the neural central view field CI θ (x, y) specifically includes: obtaining the neural central view field CI using the loss function described in Equation (5) θ (x, y):

[0027]

[0028] In the formula,[[]] is the reconstruction loss between CI θ (x, y) and . represents the central view with a resolution of αH × αW in the π xy plane coordinate set of the two-plane light field, and || represents taking the absolute value.[[]]

[0029] Furthermore, the training method of the neural disparity field specifically includes: using the loss function described in Equation (9)[[]] to obtain the neural disparity field

[0030]

[0031] where λ 1 , λ 2 , λ 3 are hyperparameters,[[]] is the structure-aware smoothness loss described in Equation (6),[[]] is the structural similarity index loss described in Equation (7),[[]] is the predicted value of the surrounding view described in Equation (8)[[]] and the reconstruction loss between the surrounding view ;[[]]

[0032]

[0033] In the formula,[[]] x S respectively represent the coordinate sets of the surrounding views with a resolution of H × W in the π uv plane and π xy plane of the two-plane light field,[[]] represents the gradient along the horizontal and vertical directions, ⊙ represents the Hadamard product, η represents the adjustment parameter that controls the strength of edge sensitivity,[[]] represents the disparity map of the light field view point (u, v) relative to the central view point (u 0 , v 0 ), || represents taking the absolute value, and SSIM is an index used to measure the similarity between two images.[[]]

[0034] Further, λ 1 = 1, λ 2 = 0.1, λ 3 = 0.01.

[0035] The present invention can achieve the synchronization of capturing dynamic data of birds without using a large light field camera array. By specifically designing the arrangement and quantity of cameras in the hybrid camera system, it can capture the dynamic data of birds. Only through the image data collected by the hybrid camera system, it can render images from any light field perspective, and may be able to fully exploit the light field color and angle information of the hybrid camera data, which helps to reconstruct the flight trajectories and postures of bird flocks under different perspectives and lighting conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is a schematic flowchart of a method for observing dynamic bird flocks based on a neural hybrid camera provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0037] In the drawings, the same or similar reference numerals are used to denote the same or similar elements or elements having the same or similar functions. The embodiments of the present invention will be described in detail below with reference to the drawings.

[0038] In the description of the present invention, the orientation or positional relationship indicated by the terms "center", "longitudinal", "transverse", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as limiting the protection scope of the present invention.

[0039] Term Explanation: Neural Fields is a technology that implicitly represents continuous fields through neural networks, and is usually used to represent physical quantities or attributes in space. It takes continuous spatial information (such as coordinates or time) as the input of the neural network and outputs some attributes in this space, such as color, density, lighting, geometric shape, etc. Neural Fields can be widely applied in fields such as 3D reconstruction, scene representation, and physical simulation.

[0040] As Figure 1 shown, the method for observing dynamic bird flocks based on a neural hybrid camera provided by an embodiment of the present invention includes:

[0041] Step 1, collecting bird flock image data with different resolutions through a hybrid camera system, and preprocessing the bird flock image data to obtain hybrid light field data.

[0042] Among them, the hybrid camera system includes a camera arranged at the central viewpoint (u 0 , v 0 ) position, which is used to collect high-resolution images of the bird flock, hereinafter referred to as the central view The hybrid camera system also includes cameras arranged around the central viewpoint (u 0 , v 0 ) position, which is used to collect low-resolution images of the bird flock, hereinafter referred to as the surrounding view The relative position relationship between the camera at the central viewpoint position and the cameras around the central viewpoint can be preset to obtain the position information of the bird flock relative to the camera, and then the position information of the bird flock can be obtained. Therefore, the bird flock image contains both the position information of the bird flock and the attitude information of the bird flock.

[0043] The preprocessing method can be but not limited to using the imresize method of matlab for image scaling processing, etc.

[0044] Step 2, since the central view collected by the hybrid camera is high-resolution and sufficient color information of the scene is collected, in this embodiment, an implicit neural representation is used to fit the central high-resolution view to estimate the scene color, fully considering the role of the high-resolution camera in the hybrid imaging system in Step 1. Using the central view obtained in Step 1 to train the neural central view field CI of the neural light field network model θ (x, y). The input of the neural central view field CI θ (x, y) is a two-dimensional vector, and the two-dimensional position coordinates x = (x, y) in the π xy plane of the light field double plane are sent into the MLP for training, as outputting the color c of the sampling point in the four-dimensional space.

[0045] θ represents the set of network parameters of the neural central view field CI θ (x, y). θ can be set, for example, but not limited to: the neural central view field CI θ is a 6-layer MLP (the full English name is Multilayer Perceptron, and the full Chinese name is the fully connected network layer), using the sine activation function, and the neuron size of the middle layer is 256; the position encodings of the image space coordinates are set to 8 respectively, the Adam optimizer is used in the training process, the initial learning rate is 1×10 -2 , and the final learning rate is 1×10 -5 , and it drops by 0.1 times every 4000 rounds.

[0046] Using the surrounding view obtained in Step 1 to the neural light field network model Neural disparity field for training. The neural disparity field takes a four-dimensional vector as input, which is the four-dimensional position coordinates r=(u, v, x, y) of the light field's double plane. The four-dimensional vector is fed into the MLP for training, as to output the disparity d(u, v, x, y) of the sampling points in the four-dimensional space (u, v, x, y).

[0047] φ represents the set of network parameters of the neural disparity field and can be set, for example but not limited to: the neural disparity field is a 5-layer MLP, using the Relu activation function, with the neuron size in the middle layer being 256; the position encodings of the light field angle and space are set to 6 and 8 respectively, and the Adam optimizer is used in the training process with a learning rate of 1×10 -2 .

[0048] The implicit neural representation learns a continuous function through a neural network to represent the characteristics of an object or certain information in a spatial scene, such as color, density, depth, etc. In this embodiment, there are two types of implicit neural representations. The neural central view field CI θ (x, y) is used to learn the central view function of the light field, and the neural disparity field is used to learn the disparity function of the light field. The dynamic adjustment of the required number of parameters and spectral characteristics on different scenes means that the resolutions and spectral characteristics of the light field data collected in different scenes are different. The neural central view field CI θ (x, y) and the neural disparity field need to adjust network parameters such as the number of layers N of the MLP, the number of neurons W, and the choice of activation function, as well as the dimension L of the position encoding. When performing position encoding on the light field double plane coordinates, the hyperparameters are adjusted according to the frequency characteristics of the light field space and angle. The activation function of the neural disparity field selects the ReLU activation function, and the neural central view field selects the sine activation function.

[0049] Among them, the acquisition method of the neural light field network model includes:[[]]

[0050] Taking the position coordinates (u, v, x, y) of the double-plane light field as the input of the disparity propagation equation L(u, v, x, y) for anti-occlusion and anti-Lambertian radiation described in Equation (2), and through the neural central view field CI θ (x, y) and the neural disparity field representing the central view function and the disparity function d u,v (x, y) respectively, as shown in Equation (3) and Equation (4), to obtain the neural light field network model described in Equation (1)

[0051] In the formula, represents the physical process of obtaining the surrounding view of the light field view point (u, v) from the central view and the disparity map relative to the central view point, and θ represents the neural central view field CI (x, y), and φ represents the network parameter set of the neural disparity field θ (x, y), and φ represents the network parameter set of the neural disparity field .

[0052] In one embodiment, the training method of the neural central view field CI θ (x, y) specifically includes:

[0053] Using a loss function for measuring the gap between the model prediction value and the true value, performing gradient backpropagation to obtain the neural central view field CI θ (x, y). For example: using the loss function described in Equation (5) to obtain the neural central view field CI θ (x, y):

[0054]

[0055] In the formula, is the reconstruction loss of CI θ (x, y) and , represents the central view with a resolution of αH×αW in the π xy plane coordinate set of the two-plane light field, and || represents the absolute value.

[0056] In one embodiment, the training method of the neural disparity field specifically includes: using the loss function described in Equation (9) to obtain the neural disparity field

[0057]

[0058] where λ 1 , λ 2 , λ 3 are hyperparameters, and the specific values are adjusted according to the specific task. For example, λ 1 = 1, λ 2 = 0.1, λ 3 = 0.01, is the structure-aware smoothness loss described in Equation (6), is the structural similarity index SSIM (fully English name: Structural Similarity Index) loss described in Equation (7), for the neural disparity field in the surrounding views and the neural central view field CI θ perform region matching between the continuous data of (x, y), is the predicted value of the surrounding view described by Equation (8) and the reconstruction loss of the surrounding view of ;

[0059]

[0060] wherein x S respectively represent the surrounding views with a resolution of H×W respectively in the π uv plane and the π xy plane coordinate sets in the two-plane light field, represents the gradients along the horizontal and vertical directions, ⊙ represents the Hadamard product, and η represents the adjustment parameter that controls the strength of edge sensitivity, represents the disparity map of the light field viewpoint (u, v) relative to the central viewpoint (u 0 , v 0 ), || represents taking the absolute value, is obtained from step 2 SSIM is an index used to measure the similarity between two images, especially suitable for tasks related to image quality assessment and visual perception. SSIM takes into account the sensitivity of the human eye to image structure, brightness, and contrast, so it is more in line with human visual characteristics in image quality evaluation.

[0061] Step 3, according to the two-plane light field coordinates (u, v, x, y) in the new view, these coordinates respectively represent the sampling positions of the virtual camera in the π uv plane and the π xy plane of the light field. Use the neural light field network model in step 2 to map the two-plane light field coordinates (u, v, x, y) to the corresponding color values, so as to obtain the image information of the dynamic bird flock at this position. For the color value of each sampling point combine the color data at each position into a complete image. In this way, the final output is a panoramic view of the dynamic bird flock in the new view, which can accurately restore the flight postures and spatial relationships of the bird flock.

[0062] An embodiment of the present invention also provides a dynamic bird flock observation system based on a neural hybrid camera, which includes a bird flock image acquisition unit, a neural light field network model creation unit, and a scene reconstruction unit:

[0063] The bird flock image acquisition unit is used to acquire bird flock image data with different resolutions through a hybrid camera system. The hybrid camera system includes a camera arranged at the central viewpoint (u 0 , v 0 ) position for obtaining the central view The hybrid camera system further includes cameras arranged at positions around the central viewpoint (u 0 , v 0 ) for obtaining the surrounding view

[0064] The neural light field network model creation unit is used to utilize the central view of the bird flock image acquisition unit to train the neural central view field CI (x, y) of the neural light field network model, where θ represents the set of network parameters of the neural central view field CI θ (x, y), and utilize the surrounding view of the bird flock image acquisition unit θ to train the neural disparity field of the neural light field network model , where φ represents the set of network parameters of the neural disparity field ; among them, the acquisition method of the neural light field network model includes: Taking the position coordinates (u, v, x, y) of the bi - plane light field as the input of the disparity propagation equation L(u, v, x, y) for anti - occlusion and anti - Lambertian radiation described in Equation (2), and passing through the neural central view field CI

[0065] θ (x, y) and the neural disparity field respectively represent the central view function and the disparity function d u,v (x, y), as shown in Equation (3) and Equation (4), to obtain the neural light field network model described in Equation (1)

[0066] The scene reconstruction unit is used to utilize the neural light field network model to obtain the color value of any coordinate of the bi - plane light field, and combine this color value with the corresponding angle information included in the neural light field network model to generate a new perspective and flight attitude map of the dynamic bird flock

[0067] Finally, it should be pointed out that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than limiting it. Those of ordinary skill in the art should understand that: the technical solutions recorded in the foregoing embodiments can be modified, or some of the technical features can be equivalently replaced; these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention

Claims

1. A method for dynamic bird flock observation based on a neural hybrid camera, characterized in that: include: Step 1: Collect bird flock image data of different resolutions through a hybrid camera system. The hybrid camera system includes a camera arranged at the central viewpoint (u0, v0) to obtain the central view. The hybrid camera system also includes cameras arranged around the central viewpoint (u0, v0) for acquiring surrounding views. Step 2, using the center view from step 1 Neural Light Field Network Model The Neurocentric View Field CI θ (x, y) for training, θ represents the neural center view field CI θ The network parameter set of (x, y) uses the surrounding view obtained in step 1 Neural Light Field Network Model Neural disparity field For training, φ represents the neural disparity field A set of network parameters; among them, the neural light field network model Ways to obtain include: The position coordinates (u, v, x, y) of the biplane light field are used as the input of the parallax propagation equation L(u, v, x, y) for anti-occlusion and anti-radiation described by equation (2), and the neural center view field CI is used to calculate the θ (x, y) and the neural disparity field Represent the center view function and the disparity function d u，v (x, y), as shown in equations (3) and (4), we get the neural light field network model described by equation (1) In the formula, Represents the surrounding views of the light field viewpoint (u, v) obtained by the central view and the disparity map relative to the central viewpoint physical process; Step 3: Based on the position coordinates (u, v, x, y) of the dual-plane light field under the new perspective, use the neural light field network model of step 2 The position coordinates (u, v, x, y) of the dual-plane light field are mapped to corresponding color values ​​to obtain the image information of the dynamic bird flock at the position coordinates, and then output a panoramic view of the dynamic bird flock under a new perspective.

2. The method for dynamic bird flock observation based on neural hybrid camera according to claim 1, characterized in that: Step 2 of the Neurocentric View Field CI θ The training method of (x, y) specifically includes: using the loss function described by formula (5) to obtain the neural center view field CI θ (x, y): In the formula, l h CI θ (x, y) and The reconstruction loss, Represents the central view with a resolution of αH×aW π in a biplane light field xy The coordinate set of the plane, || means taking the absolute value.

3. The method for dynamic bird flock observation based on a neural hybrid camera according to claim 1 or 2, characterized in that: Neural disparity field at step 2 The training method specifically includes: using the loss function l described by formula (9) l Obtaining Neural Disparity Fields l l =λ1l1+λ2l SSIM +λ3l smooth (9) Among them, λ1, λ2, λ3 are hyperparameters, l smooth is the structure-aware smoothness loss described by equation (6), l SSIM is the structural similarity index loss described by formula (7), l1 is the surrounding view described by formula (8) The predicted value of and surrounding views reconstruction losses; In the formula, x S Represented as the surrounding view with resolution H×W π in the two-plane light field uv Plane, π xy The coordinate set of the plane, represents the gradient along the horizontal and vertical directions, ⊙ represents the Hadamard product, and η represents the adjustment parameter for controlling the strength of edge sensitivity. It represents the disparity map of the light field viewpoint (u, v) relative to the central viewpoint (u0, v0), || represents the absolute value, and SSIM is an indicator used to measure the similarity of two images.

4. The method for dynamic bird flock observation based on neural hybrid camera according to claim 3, characterized in that: λ1=1, λ2=0.1, λ3=0.

01.

5. A dynamic bird flock observation system based on neural hybrid camera, characterized in that: include: A flock image acquisition unit is used to acquire flock image data of different resolutions through a hybrid camera system. The hybrid camera system includes a camera arranged at a central viewpoint (u0, v0) to obtain a central view. The hybrid camera system also includes cameras arranged around the central viewpoint (u0, v0) for acquiring surrounding views. Neural Light Field Network model creation unit for utilizing the center view of the bird flock image acquisition unit Neural Light Field Network Model The Neurocentric View Field CI θ (x, y) for training, θ represents the neural center view field CI θ (x, y) network parameter set, using the surrounding view of the bird flock image acquisition unit Neural Light Field Network Model Neural disparity field For training, φ represents the neural disparity field A set of network parameters; among them, the neural light field network model Ways to obtain include: The position coordinates (u, v, x, y) of the biplane light field are used as the input of the parallax propagation equation L(u, v, x, y) for anti-occlusion and anti-radiation described by equation (2), and the neural center view field CI is used to calculate the θ (x, y) and the neural disparity field Represent the center view function and the disparity function d u，v (x, y), as shown in equations (3) and (4), we get the neural light field network model described by equation (1) In the formula, Represents the surrounding views of the light field viewpoint (u, v) obtained by the central view and the disparity map relative to the central viewpoint physical process; A scene reconstruction unit is used to use the neural light field network model in step 2 according to the position coordinates (u, v, x, y) of the biplane light field under the new perspective. The position coordinates (u, v, x, y) of the dual-plane light field are mapped to corresponding color values ​​to obtain the image information of the dynamic bird flock at the position coordinates, and then output a panoramic view of the dynamic bird flock under a new perspective.

6. The dynamic bird flock observation system based on neural hybrid camera according to claim 5, characterized in that: NeuroCenter View Field CI θ The training method of (x, y) specifically includes: using the loss function described by formula (5) to obtain the neural center view field CI θ (x, y): In the formula, l h CI θ (x, y) and The reconstruction loss, Represents the central view with a resolution of αH×αW π in a biplane light field xy The coordinate set of the plane, || means taking the absolute value.

7. The dynamic bird flock observation system based on neural hybrid camera according to claim 5 or 6, characterized in that: Neural disparity field The training method specifically includes: using the loss function l described by formula (9) l Obtaining Neural Disparity Fields l l =λ1l1+λ2l SSIM +λ3l smooth (9) Among them, λ1, λ2, λ3 are hyperparameters, l smooth is the structure-aware smoothness loss described by equation (6), l SSIM is the structural similarity index loss described by formula (7), l1 is the surrounding view described by formula (8) The predicted value of and surrounding views reconstruction losses; In the formula, x S Represented as the surrounding view with resolution H×W π in the two-plane light field uv Plane, π xy The coordinate set of the plane, represents the gradient along the horizontal and vertical directions, ⊙ represents the Hadamard product, and η represents the adjustment parameter for controlling the strength of edge sensitivity. It represents the disparity map of the light field viewpoint (u, v) relative to the central viewpoint (u0, v0), || represents the absolute value, and SSIM is an indicator used to measure the similarity of two images.

8. The dynamic bird flock observation system based on neural hybrid camera according to claim 7, characterized in that: λ1=1, λ2=0.1, λ3=0.01.

Images