Signal reconstruction method and system based on implicit neural representation

Abstract

The invention provides a signal reconstruction method and system based on implicit neural representation, and relates to the technical field of signal reconstruction, and the method comprises the steps: building a series of basic measurement grids with different distance metrics for to-be-processed multimedia data, building different feature spaces through a plurality of distance metrics, building feature grids in the spaces, and carrying out the reconstruction of the to-be-processed multimedia data through the feature grids; storing related characteristics of different order derivatives; the related characteristic grids are indexed through the coordinates of the to-be-reconstructed signal, so that the characteristics of the signal at different orders are obtained; a high-order term extrapolation decoder is used, features of different orders are fused in each layer, a reconstruction result is refined step by step, and the reconstructed features are converted into target signal values through an output layer; according to the implicit neural representation method, any nonlinearity of a target signal in a feature space is fitted by using a plurality of basic measurement grids, different measurement grids are respectively stored by using Hash mapping to improve the compactness of the model, and the accuracy, adaptability and stability of the implicit neural representation method are improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of signal reconstruction, and in particular to a signal reconstruction method and system based on implicit neural representation. Background Art

[0002] Driven by the digital age, technologies such as augmented reality (AR), virtual reality (VR) and 3D reconstruction play an increasingly important role in scientific research and development and actual production life. These technologies generate a large amount of multimodal data every day, including images, videos and 3D models. However, most of the current signal characterization methods focus on a specific form of data, which also leads to the need to adopt different technical solutions for downstream tasks such as transmission and compression of these signals.

[0003] With the rapid development of deep learning technology, many researchers have applied implicit neural representation technology (INRs) to the fields of computer vision and graphics. As a unified signal representation method, implicit neural representation trains a dedicated neural network for each target signal. By utilizing the flexibility of neural network input and output, it can adapt to different types of signals by simply modifying the supervision information in training. In particular, it can effectively process three-dimensional data and contribute to the efficient storage and processing of different modal signals. The expressive power of implicit neural representation mainly depends on three aspects: first, the ability to fit complex signals, second, the compactness of the neural network, and third, the time required to fit the target signal. This method has a simple structure and a small number of parameters, but it is difficult to fit complex or large-scale signals, and the training time is also long, which is subject to many limitations in practical applications.

[0004] In recent years, implicit neural representation methods (also known as hybrid representations) based on explicit structures such as voxel grids have been widely studied. These methods use a set of feature vectors to represent the local area of ​​the signal, and use interpolation as an index rule to calculate the features of the query point based on the local feature weights, and convert the features into target signal values ​​through a lightweight decoder. Such methods greatly speed up the fitting speed and accuracy by storing the local features of the signal, and can adjust the size of the feature grid according to the complexity of the signal, thereby achieving efficient reconstruction of large-scale signals. However, explicit storage of features reduces the compactness of neural representation, and the use of simple linear interpolation index rules will also limit model performance. When the signal is more complex, a denser local feature representation is required, further exacerbating parameter inefficiency; since the complexity of the signal varies in different regions, using the same number of features at all locations to fit the local signal will result in greater redundancy. Existing methods use heuristic strategies such as pruning and merging to construct data structures for specific signals during training, or introduce sparsity constraints by simplifying the network structure to reduce the number of stored features, but they cannot improve the fitting ability of complex signals.

[0005] The shortcomings of the above implicit neural representation methods in terms of accuracy, adaptability and stability can be summarized as follows:

[0006] 1) The implicit neural representation based on the feature grid uses simple line interpolation, which results in poor reconstruction effect;

[0007] 2) The feature grid storage is redundant and the parameters are inefficient;

[0008] 3) The decoder has a simple structure and weak nonlinear fitting capability. Summary of the invention

[0009] In order to solve the above problems, the present disclosure proposes a signal reconstruction method and system based on implicit neural representation, which uses multiple basic metric grids to fit arbitrary nonlinearity of the target signal in the feature space, utilizes hash mapping to store different metric grids respectively to improve the compactness of the model, and uses a high-order term extrapolation decoder to gradually refine the nonlinear features and convert them into target signals, thereby improving the accuracy, adaptability and stability of the implicit neural representation method.

[0010] According to some embodiments, the present disclosure adopts the following technical solutions:

[0011] A signal reconstruction method based on implicit neural representation, comprising:

[0012] Based on Taylor expansion, a series of basic metric grids with different distance metrics are established for the multimedia data to be processed. Different feature spaces are constructed through multiple distance metrics, and feature grids are established in these spaces to store features related to different order derivatives.

[0013] The coordinates of the signal to be reconstructed are used to index the relevant feature grid, thereby obtaining the features of the signal at different orders;

[0014] Using the high-order term extrapolation decoder, features of different orders are fused in each layer, the reconstruction results are gradually refined, and the reconstructed features are converted into target signal values ​​through the output layer.

[0015] According to some embodiments, the present disclosure adopts the following technical solutions:

[0016] A signal reconstruction system based on implicit neural representation, comprising:

[0017] The grid construction module is configured to: refer to Taylor expansion to establish a series of basic metric grids with different distance metrics for the multimedia data to be processed, construct different feature spaces through multiple distance metrics, and establish feature grids in these spaces to store features related to different order derivatives;

[0018] The feature indexing module is configured to: index the relevant feature grids through the coordinates of the signal to be reconstructed, thereby obtaining the features of the signal at different orders;

[0019] The signal decoding module is configured to: utilize a high-order term extrapolation decoder to fuse features of different orders in each layer, gradually refine the reconstruction results, and convert the reconstructed features into target signal values ​​through the output layer.

[0020] According to some embodiments, the present disclosure adopts the following technical solutions:

[0021] A computer program product comprises a computer program, wherein when the computer program is executed by a processor, the signal reconstruction method based on implicit neural representation is implemented.

[0022] According to some embodiments, the present disclosure adopts the following technical solutions:

[0023] A non-transitory computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by a processor, a signal reconstruction method based on implicit neural representation is implemented.

[0024] According to some embodiments, the present disclosure adopts the following technical solutions:

[0025] An electronic device comprises: a processor, a memory and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory so that the electronic device implements the signal reconstruction method based on implicit neural representation.

[0026] Compared with the prior art, the present invention has the following beneficial effects:

[0027] The present invention uses multiple basic metric grids to fit arbitrary nonlinearity of the target signal in the feature space, utilizes hash mapping to store different metric grids respectively to improve the compactness of the model, uses the features indexed in the basic metric grids as the input of the high-order term extrapolation decoder, multiplies the features of different metric grids with the features of the decoder through the Hadamard product at each layer of the decoder to predict the high-order terms, gradually refines the nonlinear features and converts them into the target signal, improves the reconstruction quality, and has more obvious advantages on signals with larger scale and more complex details, significantly reduces erroneous holes and spikes, and maintains the compactness of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings constituting a part of the present disclosure are used to provide a further understanding of the present disclosure. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation on the present disclosure.

[0029] Figure 1 This is the overall framework diagram of Example 1.

[0030] Figure 2 This is a structural diagram of the high-order term extrapolation decoder of Example 1.

[0031] Figure 3 This is a comparison chart of the 2D image fitting results of Example 1.

[0032] Figure 4 This is a comparison chart of the 2D image fitting results errors of Example 1.

[0033] Figure 5 This is a comparison chart of the fitting results of the three-dimensional model of Example 1.

[0034] Figure 6 This is a comparison chart of the 5D neural radiation field reconstruction results of Example 1. DETAILED DESCRIPTION

[0035] The present disclosure is further described below in conjunction with the accompanying drawings and embodiments.

[0036] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present disclosure belongs.

[0037] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present disclosure. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0038] Terminology explanation:

[0039]

[0040]

[0041] Example 1

[0042] In one embodiment of the present disclosure, a signal reconstruction method based on implicit neural representation is provided, comprising:

[0043] Step S1: Using Taylor expansion, a series of basic metric grids with different distance metrics are established for the multimedia data to be processed, different feature spaces are constructed through multiple distance metrics, and feature grids are established in these spaces to store features related to different order derivatives;

[0044] Step S2: indexing the relevant feature grid through the coordinates of the signal to be reconstructed, thereby obtaining the features of the signal at different orders;

[0045] Step S3: Using the high-order term extrapolation decoder, the features of different orders are fused in each layer, the reconstruction results are gradually refined, and the reconstructed features are converted into target signal values ​​through the output layer.

[0046] As an embodiment, the present disclosure provides a signal reconstruction method based on implicit neural representation, such as Figure 1 As shown in the figure, firstly, the basic metric grid is established by drawing on the idea of ​​Taylor expansion, and different nonlinear spaces are constructed through a variety of nonlinear distance metrics to approximate different high-order terms in Taylor expansion, and feature grids are generated in these spaces. The coordinate x of the input signal to be reconstructed is used to index these feature grids, so as to obtain the nonlinear approximation of the signal in the latent space; secondly, according to the sparsity characteristics of different high-order terms, hash maps of different lengths are designed for each feature grid to achieve efficient storage and maintain the compactness of the model; finally, in the decoding stage, the features indexed in each metric grid are summed as the initial input, and the features of different orders are fused in each layer, the reconstruction results are gradually refined, and the reconstructed features are converted into target signal values ​​through the output layer. The specific implementation process is described in detail below.

[0047] First, let’s briefly introduce the theoretical knowledge used:

[0048] Given a set of observations of the target signal with T samples The implicit neural representation is defined as a modeling signal coordinate x to the corresponding signal value The mapping neural network:

[0049]

[0050] The original implicit neural representation method directly fits the signal through a fully connected neural network, takes the coordinates of the signal as input, and uses the signal value for end-to-end supervised training, which can be expressed as:

[0051]

[0052] in, represents the output feature of the lth layer, W l and b l is the learnable parameter of this layer, ψ(·) is the activation function, and L is the total number of network layers.

[0053] The grid-based implicit neural representation uses a set of feature vectors to represent the local area of ​​the signal, and uses interpolation as the index rule to calculate the feature of the query point based on the local feature weights, and converts the feature into the target signal value through a light decoder, which is expressed as:

[0054]

[0055] Among them, the feature grid is represented as Define the discrete coordinates x in the input domain i In the figure, g(·) is an index function related to distance. The distance metric generally used is linear distance | i |, decoder f d Typically a simple fully connected network.

[0056] Taylor expansion is a formula that uses the information of a function at a point to describe its nearby values. The original intention was to use a polynomial to approximate the situation of a function around a point. Let n be a positive integer. If a function f defined on an interval containing a is differentiable n+1 times at point a, then for any x on this interval, we have:

[0057]

[0058] The polynomial is called the Taylor expansion of the function at a, and the remaining R n (x) is the remainder of Taylor's formula, which is (xa) n Higher-order infinitesimals.

[0059] Based on the above theoretical knowledge, the following Figure 1 The three stages are described in:

[0060] 1. Build a basic measurement grid:

[0061] In the grid-based implicit neural representation, the grid Z can be regarded as the discrete feature representation of the continuous signal in the feature space, and a continuous linear approximation is obtained through the index function. The original method uses linear interpolation as the index rule, and the target signal often exhibits complex nonlinear characteristics, corresponding to a complex nonlinear feature space. Therefore, this embodiment is based on the fact that any point of the continuous function can be approximated by a polynomial containing terms of different orders at its adjacent points through the Taylor expansion, and a series of basic metric grids are used to fit the complex nonlinear representation of the signal in the feature space.

[0062] First, according to Taylor expansion, the nonlinear high-order terms have the following form:

[0063]

[0064] Among them, x and x i They represent the coordinates of the current query point (i.e., the coordinates of the signal to be reconstructed) and the vertex coordinates of the stored features in the grid, respectively. (n) (x i ) indicates that the feature is at vertex x iThe nth-order derivative at (xx i ) n represents a nonlinear distance metric based on polynomials as basis functions.

[0065] Then, a series of basic metric grids are established according to the above nonlinear high-order term formulas. Different orders of polynomials are approximately represented by specifying different nonlinear distance metrics, and feature grids are established in these metric spaces to store features related to different orders of derivatives.

[0066] Therefore, a series of basic metric grids with different nonlinear metrics are constructed as:

[0067]

[0068] in, It means that the definition is based on the metric d p The characteristic grid under, in order to be consistent with Taylor expansion, the exponential form of the p-norm can be used as a measure of different orders.

[0069] For a query point x, this embodiment indexes all basic metric grids according to the defined different metrics:

[0070]

[0071] In the optimization process of implicit neural representation, through the indexing process designed in this embodiment, different order derivatives of the ideal characteristics of the signal can be learned through the basic metric grid, and high-order terms in the Taylor expansion can be generated through the metric-based indexing.

[0072] As a special case, the first basic metric grid index characteristics It provides a linear approximation of the latent space features, which is consistent with existing grid-based neural representation methods; while other basic metrics are grid-indexed features. Represents features induced by higher-order derivatives to provide nonlinear terms.

[0073] By establishing a series of basic metric grids, it is possible to fit any nonlinearity exhibited by complex signals in the feature space, thereby achieving accurate fitting of complex signals; in addition to using the p-norm as a nonlinear metric in accordance with the Taylor expansion, different basic metric grids can also be defined, because these metrics can be further expressed using Taylor expansion, thereby achieving more complex nonlinear fitting.

[0074] 2. Compact representation based on hash mapping:

[0075] Compared with the general method, for each vertex coordinate x in the mesh i, it is necessary to store multiple features from different basic metric grids, which represent the linear approximation and high-order terms of the signal features respectively. Using a hash table to store feature grids is a commonly used compact representation. Its essence is to use a hash table that stores features much less than the number of features contained in the feature grid itself by storing the sparsity of the features themselves; however, simply using the number of basic metric grids as an additional dimension to construct a hash map will not only increase the probability of hash collisions in space, but also cause hash collisions in different basic metric grids; since different basic metric grids contain different-order features, this hash collision will have a greater impact on the fitting accuracy; therefore, this embodiment prevents the occurrence of such collisions by hash encoding the basic metric grids separately.

[0076] Since different basic metric grids store information of different-order derivatives, the sparsity of the grids is different, and the sparsity increases with the increase of the derivative order. In the smooth region of the signal, the high-order derivatives are all 0, making The sparsity of the feature is better than the linear approximation of is much larger; therefore, this embodiment further encourages feature fusion in high-order terms by setting a shorter hash table length for the high-order metric grid. The designed hash map can be expressed as:

[0077]

[0078] in, Represents the bitwise XOR operation, π j is the specified large prime number, and t is the set hash table length.

[0079] The model compactness is improved through the above-mentioned basic metric grid compression representation based on hash coding, so the number of grid parameters is similar to the general method, but due to the use of nonlinear metrics to index features from different grids, this example obtains a more accurate nonlinear representation of the target signal in the latent space.

[0080] 3. Design of high-order extrapolation decoder:

[0081] The use of hash mapping effectively reduces the redundancy in the basic metric grid, but in order to maintain the compactness and fitting speed of the method, three basic metric grids (including linear metrics) are used to maintain a balance between representation accuracy and efficiency. In order to further reduce the approximation error caused by the limited number of grids, this embodiment further develops a high-order term extrapolation decoder, which gradually predicts high-order terms based on the linear approximation and low-order terms stored in the grid, such as Figure 2 As shown:

[0082] First, the different basic metric features indexed by the coordinate x of the reconstructed signal are treated Perform the sum operation as the decoder input:

[0083]

[0084] h0 can be regarded as an initial approximation of the nonlinear feature, obtained from the basic metric grid.

[0085] In each layer of the decoder, the features first pass through the linear layer and the activation function, and then the linear approximation features and relatively low-order terms stored in different metric grids are input into the modulation layer respectively. The output and the linear layer output pass through the activation function respectively and are aggregated using the Hadamard product to generate high-order terms. Taking the lth layer of the decoder as an example, the process is expressed as follows:

[0086]

[0087]

[0088] in, represents the approximate value of the gradual refinement of nonlinear features during decoding, ω l (·) and γ l (·) are the decoder backbone layer and the modulation layer with different order features, σ ω and σ γ are the activation functions of the backbone layer and the modulation layer. In this embodiment, sin and sin are used respectively. 2 .

[0089] Under the premise of simplifying the role of the activation function, since each layer of the decoder multiplies the current nonlinear feature with the features of different orders through the Hadamard product, the highest order of the predicted nonlinear feature is actually continuously improved. After M layers of decoders, the theoretically achievable highest order of nonlinear approximate features is Finally, the refined nonlinear feature h M Transformed to the signal domain through a linear layer:

[0090]

[0091] Among them, W o and b o represents the learnable parameters of the linear layer, Represents the predicted signal value.

[0092] The effect of the solution of this embodiment is verified by experiments. In the experimental setting, the algorithm model of this embodiment is deployed on the PyTorch platform, and the model training and testing are completed on the NVIDIA GeForce RTX 3090 GPU, so that the network can be trained end-to-end. The number of basic metric grids is set to 3, using [x 2 ,x 3], [sin(·), arcsin(·)], [sin(·), cos(·)] are used as nonlinear measures for testing. The high-order term extrapolation decoder consists of five layers, with a hidden layer width of 64 and modulation layer inputs of The initial learning rate is 1e-3, and the cosine annealing learning rate decay strategy is used. Different loss functions are used for different types of signals, including mean-square error (MSE) and mean absolute percentage error (MAPE).

[0093] This embodiment is tested on a variety of implicit neural representation tasks, including 2D image fitting, 3D signed distance function fitting, and 5D neural radiation field reconstruction (3D scene reconstruction). The specific implementation of different tasks will be described below:

[0094] For 2D image signal fitting, the Kodak dataset is first used for testing. The parameter amount of the model is controlled to be about 207k by adjusting the hash table length T and the resolution of the basic metric grid, and the training is 20,000 steps. PSNR and SSIM are used to evaluate the quality of the reconstructed image. For 3D signed distance function signals, this embodiment uses the Stanford 3D scan repository and a complex "engine" three-dimensional model for testing. The parameter amount is set to 905K, and the training is 20,000 steps. Intersection over Union IoU, Normal Angular Error NAE, and Chamfer Distance CD are used as evaluation indicators for comprehensive comparison. For 5D neural radiation field reconstruction (three-dimensional scene signal), this embodiment uses the Blender dataset for testing. First, the model of this method is set based on the parameters of NerfAcc to maintain comparability, and training is performed for 30,000 steps. PSNR, SSIM, and LPIPS (VGG) are used to evaluate the quality of new perspective images of three-dimensional scenes.

[0095] Table 1 Comparison of results with existing methods on 2D image signal fitting tasks

[0096]

[0097]

[0098] The following is a comparative analysis of the experimental results. The 2D image signal fitting test results are shown in Table 1. "↑" and "↓" respectively indicate that the lower and higher the index, the better the effect. It can be seen that compared with the original method, this embodiment improves the quality of the reconstructed image while maintaining the compactness of the model. In addition, in the tests with different parameter values, the method proposed in this embodiment always achieves the highest reconstruction quality. Comparison of 2D image fitting results Figure 3 As shown in FIG. 1 , it can be seen that this embodiment achieves more accurate reconstruction in the image detail area. Second, this embodiment uses a larger scale image to test the scalability of the method. The error diagram of this embodiment and the original method is shown in FIG. Figure 4 As shown, it can be seen that it has more obvious advantages on signals with larger scale and more complex details.

[0099] Table 2 Comparison of results with existing methods on the 3D signed distance function fitting task

[0100]

[0101] The test results of the 3D signed distance function signal are shown in Table 2. From the results, the normal vector angle error of this embodiment on the Stanford 3D scan repository is reduced by an average of 9.6% compared with the original method, and is reduced by 3.4% compared with the current best method. Other indicators also maintain the best performance; the fitting results of the "engine" three-dimensional model are compared. Figure 5 As shown, in complex areas, the present embodiment has a more complete reconstruction result, and erroneous holes and spurs are significantly reduced.

[0102] Table 3 compares the results with existing methods on the 5D neural radiation field reconstruction task.

[0103]

[0104] The test results of 5D neural radiation field reconstruction (3D scene signal) are shown in Table 3. The reconstruction results of this embodiment on 8 scenes of the Blender dataset are improved compared with the original method in all evaluation indicators, while the model parameters remain compact; the comparison of 5D neural radiation field reconstruction results is as follows: Figure 6 As shown in Figure 3, the advantages of this method are more obvious in the scene detail area.

[0105] In summary, this embodiment is aimed at a hybrid representation method based on feature grids in implicit neural representation. Multiple basic metric grids are used to fit any nonlinearity of the target signal in the feature space, and different metric grids are stored separately using hash mapping to improve the compactness of the model. The features indexed in the basic metric grids are used as the input of the high-order term extrapolation decoder. At each layer of the decoder, the features of different metric grids are multiplied by the features of the decoder through the Hadamard product to predict the high-order terms, and the nonlinear features are gradually refined and converted into target signals. Experiments on various signal fitting tasks such as 2D images, 3D signed distance functions, and three-dimensional scenes (5D neural radiation fields) have proved that this embodiment has a certain improvement in fitting accuracy and model compactness compared to the original method.

[0106] Example 2

[0107] In one embodiment of the present disclosure, a signal reconstruction system based on implicit neural representation is provided, comprising:

[0108] The grid construction module is configured to: refer to Taylor expansion to establish a series of basic metric grids with different distance metrics for the multimedia data to be processed, construct different feature spaces through multiple distance metrics, and establish feature grids in these spaces to store features related to different order derivatives;

[0109] The feature indexing module is configured to: index the relevant feature grids through the coordinates of the signal to be reconstructed, thereby obtaining the features of the signal at different orders;

[0110] The signal decoding module is configured to: utilize a high-order term extrapolation decoder to fuse features of different orders in each layer, gradually refine the reconstruction results, and convert the reconstructed features into target signal values ​​through the output layer.

[0111] Example 3

[0112] In one embodiment of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the signal reconstruction method based on implicit neural representation is implemented.

[0113] Example 4

[0114] In one embodiment of the present disclosure, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by a processor, the signal reconstruction method based on implicit neural representation is implemented.

[0115] Example 5

[0116] In one embodiment of the present disclosure, an electronic device is provided, including: a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory, so that the electronic device executes the signal reconstruction method based on implicit neural representation.

[0117] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0118] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0119] Although the above describes the specific implementation methods of the present disclosure in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present disclosure. Technical personnel in the relevant field should understand that on the basis of the technical solution of the present disclosure, various modifications or variations that can be made by those skilled in the art without creative work are still within the scope of protection of the present disclosure.

Claims

1. A signal reconstruction method based on implicit neural representation, characterized in that: include: Based on Taylor expansion, a series of basic metric grids with different distance metrics are established for the multimedia data to be processed. Different feature spaces are constructed through multiple distance metrics, and feature grids are established in these spaces to store features related to different order derivatives. The coordinates of the signal to be reconstructed are used to index the relevant feature grid, thereby obtaining the features of the signal at different orders; Using the high-order term extrapolation decoder, features of different orders are fused in each layer, the reconstruction results are gradually refined, and the reconstructed features are converted into target signal values ​​through the output layer.

2. A signal reconstruction method based on implicit neural representation as claimed in claim 1, characterized in that: The series of basic metric grids with different distance metrics are expressed as follows: in, It means that the definition is based on the metric d p The basic metric grid, x and x i They represent the coordinates of point x in the multimedia data and the vertex coordinates of the features stored in the grid, respectively.

3. A signal reconstruction method based on implicit neural representation as claimed in claim 2, characterized in that: The series of basic metric grids with different distance metrics are specifically: First Basic Metrics Grid Provides a linear approximation of the feature space, while other basic metric grids It is a nonlinear approximation constructed based on Taylor expansion.

4. A signal reconstruction method based on implicit neural representation as claimed in claim 3, characterized in that: The nonlinear approximation constructed according to the Taylor expansion is expressed by nonlinear high-order terms: Among them, x and x i Represents the coordinates of the point x in the multimedia data and the vertex coordinates of the stored features in the grid, z (n) (x i ) indicates that the feature is at vertex x i The nth-order derivative at (xx i ) n represents a nonlinear distance metric based on polynomials as basis functions.

5. The signal reconstruction method based on implicit neural representation according to claim 1, characterized in that: Also includes: According to the sparsity characteristics of different high-order terms, hash maps of different lengths are designed for each feature grid.

6. The signal reconstruction method based on implicit neural representation according to claim 1, characterized in that: In each layer of the high-order term extrapolation decoder, the features first pass through the linear layer and the activation function, and then are input into the linear modulation layer with the linear approximation and the relatively low-order terms stored in the basic metric grid, respectively, and the output is aggregated with the decoder output through the Hadamard product to generate the high-order terms.

7. A signal reconstruction system based on implicit neural representation, characterized in that: include: The grid construction module is configured to: refer to Taylor expansion to establish a series of basic metric grids with different distance metrics for the multimedia data to be processed, construct different feature spaces through multiple distance metrics, and establish feature grids in these spaces to store features related to different order derivatives; The feature indexing module is configured to: index the relevant feature grids through the coordinates of the signal to be reconstructed, thereby obtaining the features of the signal at different orders; The signal decoding module is configured to: utilize a high-order term extrapolation decoder to fuse features of different orders in each layer, gradually refine the reconstruction results, and convert the reconstructed features into target signal values ​​through the output layer.

8. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the signal reconstruction method based on implicit neural representation described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by the processor, a signal reconstruction method based on implicit neural representation as described in any one of claims 1-6 is implemented.

10. An electronic device, characterized in that: include: A processor, a memory and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory so that the electronic device executes a signal reconstruction method based on implicit neural representation as described in any one of claims 1-6.

Images