Rare bird virtual digitalization method and system based on implicit neural representation

Abstract

The invention discloses a rare bird virtual digitalization method and system based on implicit neural representation. The method comprises the following steps: step 1, creating input data; step 2, mapping the spatial position coordinate X of the reconstruction area of the rare birds into a high-dimensional Fourier feature vector gamma (X) through Fourier features, setting an adaptive learnable filter HB (alpha (X)) at the X position, the HB (alpha (X)) including B for controlling the dimension of the HB (alpha (X)) and alpha (X) for controlling the filtering initial position of the HB (alpha (X)); 3, combining the HB (alpha (X)) with the gamma (X) to obtain a hidden unit z (1) of the first layer of the MLPs network F theta so as to obtain the color and the volume density at the X position; and step 4, generating a new visual angle and a new posture graph of the static background of the rare birds through a body rendering technology. According to the method, the high-quality picture of the rare birds at any visual angle can be quickly rendered only through the data collected by the monocular camera, and the method is excellent in performance when complex forms and details of the rare birds are processed.

Description

Technical Field

[0001] The invention relates to the technical field of computer vision and computer graphics, and in particular to a virtual digitization method for rare birds based on implicit neural representation. Background Art

[0002] Virtual digitization of rare birds faces many difficulties, mainly due to the need for high precision and authenticity. First, rare birds usually have complex feather structures and unique dynamic behaviors, and the meticulous capture and accurate reproduction of these features are an indispensable part of the digitization process. However, traditional 3D modeling methods can hardly meet the high-precision requirements for appearance, details, and motion at the same time, especially in simulating the texture of feathers, light and shadow changes, and subtle dynamic performance. In addition, the number of rare birds is limited, and it is difficult to obtain high-quality shooting and scanning data, especially in the wild. This data scarcity further exacerbates the complexity of digital modeling. In order to achieve highly realistic performance of virtual models, a lot of manual adjustments and data processing must be done, which is time-consuming and labor-intensive, and it is difficult to ensure the adaptability and authenticity of the model in different scenarios.

[0003] Learnable implicit neural representations offer significant advantages for the virtual digitization of rare birds. First, this method can provide high-quality, three-dimensional, highly realistic representations of rare birds, not only accurately capturing the appearance characteristics of the birds, but also dynamically presenting their complex feather structures and natural movement behaviors. Unlike traditional explicit modeling techniques, implicit neural representations do not rely on specific geometric forms such as meshes or point clouds, but instead generate high-resolution three-dimensional models through continuous functions of neural network learning objects. This enables it to perform well in dealing with the complex forms and details of rare birds, and can more naturally simulate the interaction between light and feathers, thereby improving the overall visual effect. In addition, this representation method can effectively deal with the problem of data scarcity, and can generate high-quality models through a small amount of data training, and has good generalization ability, which is suitable for realistic reproduction in different environments and scenarios. Summary of the invention

[0004] The purpose of the present invention is to provide a rare bird virtual digitization method based on implicit neural representation to meet the demand for high-quality reconstruction of rare bird virtual digitization models in current practical applications.

[0005] To achieve the above object, the present invention provides a method for virtual digitization of rare birds based on implicit neural representation, which comprises:

[0006] Step 1, collect images of rare birds from multiple angles through a camera, record the camera parameters of each image, align the images according to the camera parameters, and use the color value of each pixel in the aligned image and the ray direction of each pixel relative to the camera as input data;

[0007] Step 2: Determine the reconstruction area of ​​rare birds based on the input data of step 1, and transform the spatial position coordinate X of the reconstruction area of ​​rare birds into a high-dimensional Fourier feature vector γ(X) shown in formula (1) through Fourier feature mapping:

[0008] γ(X)=(sin(2 0 πX), cos(2 0 πX),…,sin(2 L-1 πX), cos(2 L-1 πX)) (1)

[0009] Where L is the hyperparameter in the Fourier feature network;

[0010] Assign a three-dimensional trainable grid of a set resolution to the rare bird reconstruction area, and store a one-dimensional learnable variable α(X) at the preset position of the three-dimensional trainable grid as a task parameter, and set an adaptive learnable filter H at X. B (α(X)), filter H B (α(X)) is used to filter some redundant information in γ(X). The filter H B (α(X)) contains the hyperparameter B, which is used to control the filter H B The dimension of (α(X)) does not exceed the dimension of γ(X), thus controlling the filter H B (α(X)) corresponds to the filter H in γ(X) B The redundant information of the dimension (α(X)) is filtered, and the filter H B (α(X)) also contains the task parameter α(X), which is used to control the filter H B (α(X)) Initial position of the filter;

[0011] Step 3: Replace the filter H in step 2 B (α(X)) is combined with the high-dimensional Fourier feature vector γ(X) through the Hadamard product ⊙, as shown in formula (2), and the MLPs (in Chinese: used to learn the mapping from encoding features to signal values) network F is obtained. θ The first layer of hidden units z (1) , z (1) It is then input into the MLPs network F θ In the output, the color C and volume density σ at X are output;

[0012]

[0013] Where σ is the MLPs network F θ The activation function, and Represents the MLPs network F θ The weights and biases of the i-th layer, i = 1, ..., k-1, Represents the MLPs network F θ The hidden unit of the i-th layer, f represents the MLPs network F θ ;

[0014] Step 4, using volume rendering technology, the color C and volume density σ at X obtained in step 3 are integrated to generate a new perspective and new posture map of the static background of the rare bird.

[0015] Furthermore, the filter H B (α(X)) is composed of its components Composition, each Calculated by formula (3):

[0016]

[0017] Among them, ∈ is a filter performance adjustment parameter whose value can be preset, the value of j is twice the dimension of X, and the filter H B The hyperparameter B of (α(X)) is less than j, thus controlling the filter H B The dimension of (α(X)) does not exceed the dimension of γ(X).

[0018] Further, ∈=-1000.

[0019] Furthermore, a method for obtaining the task parameter α(X) includes: the task parameter α(X) is learned and obtained through a trainable grid that is coordinated with the neural network parameters during training, and the learning method includes: first finding the grid where X is located in the rare bird reconstruction area, and obtaining the value of the task parameter α(X) at X through an interpolation method.

[0020] Furthermore, when α(X) = -1, the filter H B (α(X)) is converted into a low-pass filter; when α(X) = 11, the filter H B (α(X)) is converted into a bandpass filter; when α(X) = 21, the filter H B (α(X)) is transformed into a high-pass filter, and the hyperparameter B is set to 15.

[0021] The present invention also provides a rare bird virtual digitization system based on implicit neural representation, which comprises:

[0022] An input data creation unit, which is used to collect images of rare birds from multiple angles according to a camera, record camera parameters of each image, align the images according to the camera parameters, and use the color value of each pixel in the aligned image and the ray direction of each pixel relative to the camera as input data;

[0023] The Fourier feature mapping unit is used to determine the rare bird reconstruction area on the input data of the input data creation unit, and transform the spatial position coordinate X of the rare bird reconstruction area into a high-dimensional Fourier feature vector γ(X) shown in formula (1) through Fourier feature mapping:

[0024] Y(X)=(sin(2 0 πX), cos(2 0 πX),…,sin(2 L-1 πX), cos(2 L-1 πX)) (1)

[0025] Where L is the hyperparameter in the Fourier feature network;

[0026] The adaptive learnable filter setting unit is used to allocate a three-dimensional trainable grid with a set resolution to the rare bird reconstruction area, and store a one-dimensional learnable variable α(X) at a preset position of the three-dimensional trainable grid as a task parameter, and set the adaptive learnable filter H at X. B (α(X)), filter H B (α(X)) is used to filter some redundant information in γ(X). The filter H B (α(X)) contains the hyperparameter B, which is used to control the filter H B The dimension of (α(X)) does not exceed the dimension of γ(X), thus controlling the filter H B (α(X)) corresponds to the filter H in γ(X) B The redundant information of the dimension (α(X)) is filtered, and the filter H B (α(X)) also contains the task parameter α(X), which is used to control the filter H B (α(X)) Initial position of the filter;

[0027] The reconstruction parameter acquisition unit is used to transform the filter H B (α(X)) is combined with the high-dimensional Fourier eigenvector γ(X) through the Hadamard product ⊙, as shown in formula (2), to obtain the MLPs network F θ The first layer of hidden units z (1) , z (1) It is then input into the MLPs network F θ In the output, the color C and volume density σ at X are output;

[0028]

[0029] Where σ is the MLPs network F θ The activation function, and Represents the MLPs network F θ The weights and biases of the i-th layer, i = 1, ..., k-1, Represents the MLPs network F θ The hidden unit of the i-th layer, f represents the MLPs network F θ ;

[0030] The scene reconstruction unit is used to integrate the color C and the volume density σ at X through volume rendering technology to generate a new perspective and a new posture map of the static background of the rare bird.

[0031] Furthermore, the filter H B (α(X)) is composed of its components Composition, each Calculated by formula (3):

[0032]

[0033] Among them, ∈ is a filter performance adjustment parameter whose value can be preset, the value of j is twice the dimension of X, and the filter H B The hyperparameter B of (α(X)) is less than j, thus controlling the filter H B The dimension of (α(X)) does not exceed the dimension of γ(X).

[0034] Further, ∈=-1000.

[0035] Furthermore, a method for obtaining the task parameter α(X) includes: the task parameter α(X) is learned and obtained through a trainable grid that is coordinated with the neural network parameters during training, and the learning method includes: first finding the grid where X is located in the rare bird reconstruction area, and obtaining the value of the task parameter α(X) at X through an interpolation method.

[0036] Furthermore, when α(X) = -1, the filter H B (α(X)) is converted into a low-pass filter; when α(X) = 11, the filter H B (α(X)) is converted into a bandpass filter; when α(X) = 21, the filter H B (α(X)) is transformed into a high-pass filter, and the hyperparameter B is set to 15.

[0037] The present invention can quickly render high-quality pictures of rare birds from any perspective using only data collected by a monocular camera, and performs well in processing the complex forms and details of rare birds. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a flowchart of a method for virtual digitization of rare birds based on implicit neural representation provided by an embodiment of the present invention.

[0039] Figure 2 , Figure 3 and Figure 4 They are respectively visualization schematic diagrams of converting a learnable filter into a low-pass filter when α(X) takes different values ​​according to embodiments of the present invention. DETAILED DESCRIPTION

[0040] In the drawings, the same or similar reference numerals are used to represent the same or similar elements or elements with the same or similar functions. The embodiments of the present invention are described in detail below in conjunction with the drawings.

[0041] In the description of the present invention, the terms "center", "longitudinal", "lateral", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate directions or positional relationships based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore should not be understood as limiting the scope of protection of the present invention.

[0042] like Figure 1 As shown, the rare bird virtual digitization method based on implicit neural representation provided by the embodiment of the present invention includes:

[0043] Step 1: Collect images of rare birds from multiple angles through a camera, record the camera parameters of each image, align the images according to the camera parameters, and use the color value of each pixel in the aligned image and the ray direction of each pixel relative to the camera as input data.

[0044] The sampling angle is required to be as sparse as possible and to collect a large range. The camera parameters include camera intrinsics and extrinsics. The camera intrinsics include focal length, and the camera extrinsics include rotation matrix and translation vector. The ray direction defines the direction of each pixel in the image from the camera's viewpoint in 3D space. Each ray is calculated through the camera intrinsics and extrinsics, pointing to the pixel on the image plane, and then extending to a specific position in the 3D scene.

[0045] Step 2, determining the rare bird reconstruction area based on the input data of step 1, and the rare bird reconstruction area is often determined by continuously adjusting parameters.

[0046] The spatial coordinates X of the reconstruction area of ​​rare birds are mapped to the high-dimensional Fourier eigenvector γ(X) shown in formula (1) through Fourier eigenmapping. Figure 1 , X represents the three-dimensional coordinate vector at position x in the world coordinate system xyz0, corresponding to Figure 1 InputPosition in:

[0047] γ(X)=(sin(2 0 πX), cos(2 0 πX),…,sin(2 L-1 πX), cos(2 L-1 πX)) (1)

[0048] Where L is a hyperparameter in the Fourier feature network, and its value can be pre-set for different tasks to avoid the problem of slowing down the running speed of the MLPs network Fθ due to excessively large value.

[0049] A three-dimensional trainable grid of a set resolution is allocated to the rare bird reconstruction area, and a one-dimensional learnable variable α(X) is stored at a preset position of the three-dimensional trainable grid, such as the spatial position of the corner point of the three-dimensional trainable grid, as a task parameter.

[0050] Set the adaptive learnable filter H at X B (α(X)), filter H B (α(X)) is used to filter part of the redundant information in γ(X) to reduce the training time of the MLPs network Fθ.

[0051] In one embodiment, the filter H B (α(X)) contains the hyperparameter B, which is used to control the filter H B The dimension of (α(X)) does not exceed the dimension of γ(X), thus controlling the filter H B (α(X)) corresponds to the filter H in γ(X) B The redundant information of the dimension (α(X)) is filtered out.

[0052] In another embodiment, the filter H B (α(X)) also contains the task parameter α(X), which is used to control the filter H BThe initial position of the (α(X)) filter is determined to meet the needs of different tasks. The method for obtaining the task parameter α(X) includes: the task parameter α(X) is learned and obtained through a trainable grid that is optimized in coordination with the neural network parameters during the training process. The learning method includes: first finding the grid where X is located in the rare bird reconstruction area, and obtaining the value of the task parameter α(X) at X through an interpolation method. Therefore, in this embodiment, the value of the task parameter α(X) can be adjusted by adjusting the resolution of the grid where X is located, thereby controlling the initial position of the filter and meeting the needs of different tasks. For example: by reducing the task parameter α(X), a filter H is assigned to an area in the rare bird reconstruction area. B (α(X)), which can remove noise and outlier points in the region. Generally, in 2D image fitting tasks, the resolution of the trainable grid of α(X) is set to the image resolution, which significantly improves the ability to accurately express natural scenes in the image.

[0053] Step 3: Replace the filter H in step 2 B (α(X)) is combined with the high-dimensional Fourier eigenvector γ(X) through the Hadamard product ⊙, as shown in formula (2), to obtain the MLPs network F θ The first layer of hidden units z (1) , z (1) It is then input into the MLPs network F θ In the example above, the color C and volume density σ at X are output.

[0054]

[0055] Where σ is the MLPs network F θ The activation function, and Represents the MLPs network F θ The weights and biases of the i-th layer, i = 1, ..., k-1, Represents the MLPs network F θ The hidden unit of the i-th layer, f represents the MLPs network F θ .

[0056] In other embodiments, other network structures other than the MLPs network can also be used with the filter H B (α(X)) integration can replace the ReLU activation function in the network with a sine activation function and achieve better performance than the original structure.

[0057] In the fields of computer vision and computer graphics, the radiation field represents the radiation emitted by each point in space. Through the neural network of the radiation field and with the help of deep learning technology, the color information and volume density information of each point are learned, and then the scene is reconstructed through volume rendering.

[0058] Step 4: By using volume rendering technology, the color C and volume density σ at X obtained in step 3 are integrated, as shown in formula (4), to generate a new perspective and new posture map of the static background of the rare bird:

[0059]

[0060] Where C(r) is the color information of the pixel in the new view under the camera light r(t) of any viewing angle, t n and t f are the near and far boundaries of the object passed by r(t), respectively, and T(t) is the camera ray r(t) from t n to f The cumulative transmittance is described as equation (5), σ(r(t)) is the volume density of the three-dimensional point in the camera space under r(t), c(r(t)) is the color of the three-dimensional point in the camera space under r(t), r(t) = o + td, d represents the direction, o represents the optical center, and t represents the distance:

[0061]

[0062] In one embodiment, step 2 filter H B (α(X)) is composed of its components Composition, each Calculated by formula (3):

[0063]

[0064] Among them, for the filter H B The dimension of (α(X)) is the same as that of formula γ(X). When ∈ takes a larger value, the filter performance deviates from the ideal state; the smaller the value of ∈, the closer the filter is to the ideal performance. Experimental verification shows that ∈=-1000 can meet the design requirements. Therefore, ∈=-1000 is selected. Formula (3) designs an adaptive differentiable filter, which can achieve multiple filtering functions such as low-pass, band-pass and high-pass by adjusting a single parameter. Of course, according to actual filtering requirements, technicians in this field can also adjust form.

[0065] Figure 2 , Figure 3 and Figure 4 They are respectively visualization schematic diagrams of converting a learnable filter into a low-pass filter when α(X) takes different values ​​according to embodiments of the present invention. Figure 2 It shows the visualization of the transformation of the learnable filter into a low-pass filter when α(X)=-1. Figure 3 It shows the visualization of the transformation of the learnable filter into a bandpass filter when α(X)=11. Figure 4 This is a visualization of the transformation of the learnable filter into a high-pass filter when α(X) = 21. The horizontal axis j in the figure represents the index value of the learnable filter component, and the vertical axis H represents Response value. The parameter B controls the width of the filter. Figure 1 , 2, and 3 all have values ​​of 15.

[0066] The parameters of the filter provided in the above embodiment can be automatically adjusted through the learning process of the neural network. This method allows the network to dynamically adjust the response of the filter to the frequency components according to different task requirements.

[0067] In natural scenes containing rare birds, there are obvious differences in the frequency spectrum of local areas. First, rare birds usually have complex feather structures and unique dynamic behaviors. The meticulous capture and accurate reproduction of these features are an indispensable part of the digitization process. Usually the main part presents low frequency and the edge part presents high frequency. Fixed coding frequency Fourier features will lead to redundant input information and reduce the running speed of the model. Sine coding is the main factor in obtaining high-quality signal reconstruction, aiming to embed multiple orthogonal Fourier bases into subsequent networks. Although Fourier coding has been successful, there are still some unresolved issues. The selection of the correct frequency scale often requires manual adjustment and usually involves a complex parameter adjustment process. This method selects the frequency for the entire input in a global and spatially invariant manner, thus missing the opportunity to better adapt to local high-frequency features, which may eventually lead to the model being insufficient when processing areas with rich details, manifested as undesirable representation effects such as missing details and blurred edges. The learnable filter network proposed in this embodiment performs low-pass, band-pass and high-pass operations on the Fourier feature vector by a learnable filter controlled by a learnable parameter, and the filtered Fourier feature can better represent local high-frequency details. The convergence speed of MLPs is significantly accelerated by adaptively filtering Fourier features of different frequencies through differentiable filters with learnable parameters.

[0068] The present invention also provides a rare bird virtual digitization system based on implicit neural representation, which includes an input data creation unit, a Fourier feature mapping unit, an adaptive learnable filter setting unit, a reconstruction parameter acquisition unit and a scene reconstruction unit, wherein:

[0069] The input data creation unit is used to collect images of rare birds from multiple angles according to the camera, record the camera parameters of each image, align the images according to the camera parameters, and use the color value of each pixel in the aligned image and the ray direction of each pixel relative to the camera as input data.

[0070] The Fourier feature mapping unit is used to determine the rare bird reconstruction area on the input data of the input data creation unit, and transform the spatial position coordinate X of the rare bird reconstruction area into a high-dimensional Fourier feature vector γ(X) shown in formula (1) through Fourier feature mapping.

[0071] The adaptive learnable filter setting unit is used to allocate a three-dimensional trainable grid with a set resolution to the rare bird reconstruction area, and store a one-dimensional learnable variable α(X) at a preset position of the three-dimensional trainable grid as a task parameter, and set the adaptive learnable filter H at X. B (α(X)), filter H B (α(X)) is used to filter some redundant information in γ(X). The filter H B (α(X)) contains the hyperparameter B, which is used to control the filter H B The dimension of (α(X)) does not exceed the dimension of γ(X), thus controlling the filter H B (α(X)) corresponds to the filter H in γ(X) B The redundant information of the dimension (α(X)) is filtered, and the filter H B (α(X)) also contains the task parameter α(X), which is used to control the filter H B (α(X)) Initial position of the filter.

[0072] The reconstruction parameter acquisition unit is used to transform the filter H B (α(X)) is combined with the high-dimensional Fourier eigenvector γ(X) through the Hadamard product ⊙, as shown in formula (2), and the hidden unit z( 1) , z(1) is then input into the MLPs network Fθ, which outputs the color C and volume density σ at X.

[0073] The scene reconstruction unit is used to integrate the color C and volume density σ at X through volume rendering technology to generate a new perspective and new posture map of the static background of the rare bird.

[0074] The present invention provides a framework for learnable implicit neural representation, aiming to solve the limitations of traditional implicit neural representation methods (INR) in expressing high-frequency information and convergence speed. The core idea is to introduce an adaptive learnable filter, which controls the filter through a learnable parameter to achieve low-pass, band-pass and high-pass functions, thereby realizing spatial adaptive filtering of Fourier features. Spatial adaptive filtering of Fourier features can enhance the network's ability to express natural scenes and significantly improve the convergence speed.

[0075] The framework for learning implicit neural representation provided by the invention can also be applied to other tasks. This framework can not only efficiently capture the complex structure in the data, but also has extremely high flexibility and is suitable for various types of data processing tasks. By training the network parameters, the network can implicitly represent the characteristics of the input data, thereby achieving efficient learning and expression of the data.

[0076] Taking rare birds as an example, the application prospects of this framework are particularly broad. First, in the field of one-dimensional audio signal processing, this framework can be used to fit the calls of rare birds. By learning a large number of bird call samples, the neural network can capture the unique characteristics of the sounds of different types of birds, and then achieve high-precision recognition and classification of specific bird calls. Secondly, in the field of two-dimensional image signal processing, it can be used to represent high-definition pictures of rare birds. By learning visual features such as texture and color in the image, the neural network can reconstruct high-quality bird images. Finally, it can also play a role in text information processing. By implicitly storing the text information of the recorded birds in the parameters and weights of the network, it is possible to quickly retrieve and understand bird-related literature. It is worth mentioning that all of the above functions can be achieved through a lightweight neural network architecture. This means that while ensuring performance, this method also has low computational costs and storage requirements, and is very suitable for deployment in resource-limited environments, such as field survey sites or mobile devices.

[0077] The present invention explores the frequency domain characteristics of the local area (usually the main part presents low frequency and the edge part presents high frequency), and filters the Fourier feature coding of the local area through a learnable filter, which effectively avoids the information redundancy caused by the fixed coding frequency and improves the operating efficiency of the model.

[0078] Finally, it should be pointed out that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. Those skilled in the art should understand that the technical solutions described in the above embodiments may be modified, or some of the technical features thereof may be replaced by equivalents; these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A virtual digitization method for rare birds based on implicit neural representation, characterized in that: include: Step 1, collect images of rare birds from multiple angles through a camera, record the camera parameters of each image, align the images according to the camera parameters, and use the color value of each pixel in the aligned image and the ray direction of each pixel relative to the camera as input data; Step 2: Determine the reconstruction area of ​​rare birds based on the input data of step 1, and transform the spatial position coordinate X of the reconstruction area of ​​rare birds into a high-dimensional Fourier feature vector γ(X) shown in formula (1) through Fourier feature mapping: γ(X)=(sin(2 0 πX),cos(2 0 πX),…,sin(2 L-1 πX),cos(2 L-1 πX)) (1) Where L is the hyperparameter in the Fourier feature network; Assign a three-dimensional trainable grid of set resolution to the reconstruction area of ​​rare birds, and store a one-dimensional learnable variable α(X) at the preset position of the three-dimensional trainable grid as a task parameter, and set an adaptive learnable filter H at X. B (α(X)), filter H B (α(X)) is used to filter some redundant information in γ(X). The filter H B (α(X)) contains the hyperparameter B, which is used to control the filter H B The dimension of (α(X)) does not exceed the dimension of γ(X), thus controlling the filter H B (α(X)) corresponds to the filter H in γ(X) B The redundant information of the dimension (α(X)) is filtered, and the filter H B (α(X)) also contains the task parameter α(X), which is used to control the filter H B (α(X)) Initial position of the filter; Step 3: Replace the filter H in step 2 B (α(X)) is combined with the high-dimensional Fourier eigenvector γ(X) through the Hadamard product ⊙, as shown in formula (2), to obtain the MLPs network F θ The first layer of hidden units z (1) , z (1) It is then input into the MLPs network F θ In the output, the color C and volume density σ at X are output; Where σ is the MLPs network F θ The activation function, and Represents the MLPs network F θ The weights and biases of the i-th layer, i = 1, ..., k-1, Represents the MLPs network F θ The hidden unit of the i-th layer, f represents the MLPs network F θ ; Step 4, using volume rendering technology, the color C and volume density σ at X obtained in step 3 are integrated to generate a new perspective and new posture map of the static background of the rare bird.

2. The method for virtual digitization of rare birds based on implicit neural representation according to claim 1, characterized in that: Filter H B (α(X)) is composed of its components Composition, each Calculated by formula (3): Among them, ∈ is a filter performance adjustment parameter whose value can be preset, the value of j is twice the dimension of X, and the filter H B The hyperparameter B of (α(X)) is less than j, thus controlling the filter H B The dimension of (α(X)) does not exceed the dimension of γ(X).

3. The method for virtual digitization of rare birds based on implicit neural representation according to claim 2, characterized in that: ∈＝-1000。 4. The method for virtual digitization of rare birds based on implicit neural representation according to any one of claims 1 to 3, characterized in that: The method for obtaining the task parameter α(X) includes: the task parameter α(X) is learned and obtained through a trainable grid that is coordinated with the neural network parameters during the training process. The learning method includes: first finding the grid where X is located in the rare bird reconstruction area, and obtaining the value of the task parameter α(X) at X through an interpolation method.

5. The method for virtual digitization of rare birds based on implicit neural representation according to claim 4, characterized in that: When α(X) = -1, the filter H B (α(X)) is converted into a low-pass filter; when α(X) = 11, the filter H B (α(X)) is converted into a bandpass filter; when α(X) = 21, the filter H B (α(X)) is transformed into a high-pass filter, and the hyperparameter B is set to 15.

6. A rare bird virtual digitization system based on implicit neural representation, characterized in that: include: An input data creation unit, which is used to collect images of rare birds from multiple angles according to a camera, record camera parameters of each image, align the images according to the camera parameters, and use the color value of each pixel in the aligned image and the ray direction of each pixel relative to the camera as input data; The Fourier feature mapping unit is used to determine the rare bird reconstruction area on the input data of the input data creation unit, and transform the spatial position coordinate X of the rare bird reconstruction area into a high-dimensional Fourier feature vector γ(X) shown in formula (1) through Fourier feature mapping: γ(X)=(sin(2 0 πX),cos(2 0 πX),…,sin(2 L-1 πX),cos(2 L-1 πX)) (1) Where L is the hyperparameter in the Fourier feature network; The adaptive learnable filter setting unit is used to allocate a three-dimensional trainable grid with a set resolution to the rare bird reconstruction area, and store a one-dimensional learnable variable α(X) at a preset position of the three-dimensional trainable grid as a task parameter, and set the adaptive learnable filter H at X. B (α(X)), filter H B (α(X)) is used to filter some redundant information in γ(X). The filter H B (α(X)) contains the hyperparameter B, which is used to control the filter H B The dimension of (α(X)) does not exceed the dimension of γ(X), thus controlling the filter H B (α(X)) corresponds to the filter H in γ(X) B The redundant information of the dimension (α(X)) is filtered, and the filter H B (α(X)) also contains the task parameter α(X), which is used to control the filter H B (α(X)) Initial position of the filter; The reconstruction parameter acquisition unit is used to transform the filter H B (α(X)) is combined with the high-dimensional Fourier eigenvector γ(X) through the Hadamard product ⊙, as shown in formula (2), to obtain the MLPs network F θ The first layer of hidden units z( 1 0,z( 1 ) is then input into the MLPs network F θ In the output, the color C and volume density σ at X are output; Where σ is the MLPs network F θ The activation function, and Represents the MLPs network F θ The weights and biases of the i-th layer, i = 1, ..., k-1, Represents the MLPs network F θ The hidden unit of the i-th layer, f represents the MLPs network F θ ; The scene reconstruction unit is used to integrate the color C and the volume density σ at X through volume rendering technology to generate a new perspective and a new posture map of the static background of the rare bird.

7. The rare bird virtual digitization system based on implicit neural representation according to claim 6, characterized in that: Filter H B (α(X)) is composed of its components Composition, each Calculated by formula (3): Among them, ∈ is a filter performance adjustment parameter whose value can be preset, the value of j is twice the dimension of X, and the filter H B The hyperparameter B of (α(X)) is less than j, thus controlling the filter H B The dimension of (α(X)) does not exceed the dimension of γ(X).

8. The rare bird virtual digitization system based on implicit neural representation according to claim 7, characterized in that: ∈＝-1000。 9. The rare bird virtual digitization system based on implicit neural representation according to any one of claims 6 to 8, characterized in that: The method for obtaining the task parameter α(X) includes: the task parameter α(X) is learned and obtained through a trainable grid that is coordinated with the neural network parameters during the training process. The learning method includes: first finding the grid where X is located in the rare bird reconstruction area, and obtaining the value of the task parameter α(X) at X through an interpolation method.

10. The rare bird virtual digitization system based on implicit neural representation according to claim 9, characterized in that: When α(X) = -1, the filter H B (α(X)) is converted into a low-pass filter; when α(X) = 11, the filter H B (α(X)) is converted into a bandpass filter; when α(X) = 21, the filter H B (α*X)) is transformed into a high-pass filter, and the hyperparameter B is set to 15.

Images