Multivariate task prediction neural network and method based on meta-principle physical architecture

Abstract

The application relates to a multivariate task prediction neural network and method based on a meta-principle physical architecture, and relates to the technical field of artificial intelligence deep learning.The method comprises the following steps: inputting an input sequence into an input embedding module to perform vector conversion, outputting an initial feature vector sequence to stacked meta-principle encoding layers to perform iterative feature extraction, outputting final slice fusion features, conservation constraint features and periodic law features to corresponding independent gate fusion modules for gate weight fusion after being processed by the meta-principle encoding layers, and outputting a fusion feature sequence; and inputting the fusion feature sequence into a feedforward output module to output a probability distribution prediction result of a multivariate task.The method can embed three general meta-principles of connection association, conservation constraint and periodic oscillation into a network forward reasoning architecture, and is suitable for neural network prediction of complex unknown multivariate tasks.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence and deep learning technology, and in particular to a multivariate task prediction neural network and method based on a meta-principle physical architecture. Background Technology

[0002] Multivariate task prediction is widely used in various practical engineering scenarios such as industrial, energy, and environmental simulation. Its core task is to use historical multi-dimensional observation data of the system to infer the future state change trend of the system.

[0003] Existing multivariate task prediction neural networks are primarily based on a phenomenological fitting paradigm. Large-scale autoregressive neural networks, exemplified by the Transformer architecture, learn the statistical correlation between inputs and outputs using massive amounts of parameters and training data, achieving significant success in tasks such as language understanding and computer vision. However, these models are essentially phenomenological fitting systems, lacking an intrinsic understanding of the fundamental laws of the physical world and unable to intrinsically model the causal mechanisms and physical principles behind phenomena. In the area of ​​Physics-Informed Neural Networks (PINNs), existing methods embed known physical governing equations as soft constraints into the loss function, enabling the neural network to satisfy physical laws while fitting observed data. However, PINN-like methods have limitations: they require prior knowledge of the specific form of the physical equations; they struggle to handle complex multi-physics coupling problems; their generalization ability is limited by the distribution of training data; and physical knowledge only applies to the loss function level, rather than being embedded in the network architecture itself. Regarding efficient Transformer variants, models such as Longformer, BigBird, Reformer, and Performer reduce the computational complexity of attention and optimize long sequence processing capabilities through methods like local window attention, stochastic attention, locality-sensitive hashing, and orthogonal random features. However, the core optimization objectives of these models focus on computational efficiency and long sequence modeling capabilities, without addressing explicit embedding or intrinsic modeling of physical principles. In terms of state-space models, Mamba et al. achieved linear-time sequence modeling through selective state spaces. However, the core focus of these models is also on computational efficiency, without exploring explicit embedding of physical principles. Therefore, there is an urgent need for a neural network prediction architecture that embeds three general principles—connectivity, conservation constraints, and periodic oscillations—into the network's forward inference architecture, adapting to complex, unknown, and multivariate tasks. Summary of the Invention

[0004] Therefore, it is necessary to address the technical problem that the existing technologies have difficulty embedding the three general meta-principles of connection association, conservation constraints, and periodic oscillations into the network forward inference architecture, and are unable to adapt to neural network prediction of complex unknown multivariate tasks. To provide a multivariate task prediction neural network and method based on a meta-principles physical architecture, which can embed the three general meta-principles of connection association, conservation constraints, and periodic oscillations into the network forward inference architecture and adapt to neural network prediction of complex unknown multivariate tasks.

[0005] To achieve the above-mentioned technical effects, the technical solution of the present invention is as follows: A multivariate task prediction neural network based on a meta-principle physical architecture includes an input embedding module, several stacked meta-principle encoding modules, several independent gated fusion modules, and a feedforward output module connected in sequence. Each layer of the meta-principle coding module includes a parallel slice-layered attention mechanism submodule, an energy encoder, and a periodic encoder; The output of the input embedding module is connected to the original input of the slice hierarchical attention mechanism submodule, the energy encoder, and the periodic encoder, respectively. The final output of the slice hierarchical attention mechanism submodule, the energy encoder, and the periodic encoder is connected to the input of an independent gated fusion module, and the output of each independent gated fusion module is connected to the feedforward output module.

[0006] This invention also proposes a multivariate task prediction method based on a meta-principle physical architecture multivariate task prediction neural network as described above, comprising the following steps: S1. Obtain the input sequence, input the input sequence into the input embedding module for vector transformation, and output the initial feature vector sequence; S2. The initial feature vector sequence is input into each stacked meta-principle coding layer for iterative feature extraction. The slice fusion feature is output through the slice hierarchical attention mechanism submodule in each meta-principle coding layer, the conservation constraint feature is output through the energy encoder in each meta-principle coding layer, and the periodic regularity feature is output through the periodic encoder in each meta-principle coding layer. After processing by each meta-principle coding layer, the final slice fusion feature, conservation constraint feature and periodic regularity feature are output. S3. Input the final slice fusion features, conservation constraint features and periodic regularity features into the corresponding connected independent gated fusion modules for gated weight fusion, and output the fusion feature sequence; S4. Input the fused feature sequence into the feedforward output module to output the probability distribution prediction result of the multivariate task.

[0007] Preferably, the input embedding module includes a word embedding layer and a position embedding layer connected in sequence; the step of inputting the input sequence into the input embedding module for vector transformation includes: S111. Input the input sequence into the word embedding layer, extract the high-dimensional semantic feature vector of the input sequence from the word embedding layer, and output the word vector to the position embedding layer; S112. The position embedding layer adds temporal position encoding to the word vector, outputs a position embedding vector, and normalizes the position embedding vector through the layer to output the initial feature vector sequence after layer normalization.

[0008] Preferably, the step of outputting slice fusion features through the slice hierarchical attention mechanism submodule in each meta-principle coding layer includes: S211. Divide the input initial feature vector sequence into N consecutive slices along the time dimension; S212. Within each slice, calculate the attention score using the query matrix Q and the key matrix K. The attention score S is numerically stabilized using a soft-clipping function, and the stabilized attention score is output. The stabilized attention score is then normalized and weighted and summed with the value matrix V to obtain the local interaction features. ; S213. Local interaction features The statistical values ​​are input into a preset weighted aggregation function to calculate the representative value for each slice. The expression is as follows:

[0009] in, As the first initialization parameter, For the second initialization parameter, For the third initialization parameter, Represents the vector of maximum values ​​in a slice. Represents the vector of minimum values ​​in a slice. Represents the slice mean vector; S214. Utilize preset learnable attention weights The representative value of each slice is merged into the current token to obtain the expression for the slice fusion feature as follows:

[0010] Where H represents the original feature matrix of the current token, Indicates slice fusion features, It represents the Hadamardi (or Hadama) stack.

[0011] Preferably, the step of outputting conservation constraint features through the energy encoder in each meta-principle coding layer includes: S221. Calculate the average energy for each slice. ; S222. Based on the average energy Logarithmic cumulative energy and ; S223. Based on the energy and Global geometric mean energy estimation is performed to obtain the global geometric mean energy. ; S224. Calculate the current slice energy relative to the global geometric mean energy. deviation ; S225. Based on the aforementioned deviation The conservation constraint characteristics are calculated using a delayed two-step compensation mechanism. .

[0012] Preferably, the conservation constraint feature The calculation expression is as follows:

[0013] in, This represents the original input feature matrix of the i-th slice. Let represent the Sigmoid activation function, and I represent the learnable compensation strength parameter.

[0014] Preferably, the step of outputting periodic regularity features through the periodic encoder in each meta-principle coding layer includes: S231. Apply a Fourier transform to each slice along the sequence dimension and output the Fourier transform result. ; S232. Based on the Fourier transform result Calculate the amplitude spectrum ; S233. Based on the amplitude spectrum Historical spectrum information is accumulated using the exponential moving average mechanism. ; S234. Based on the historical spectrum information The periodic characteristics are calculated using a one-step delay compensation mechanism. .

[0015] Preferably, the periodic regularity feature The calculation expression is as follows:

[0016] in, This indicates a learnable multilayer perceptron. Indicates the first Historical spectral information of the slice.

[0017] Preferably, the step of inputting the final slice fusion features, conservation constraint features, and periodicity features into corresponding connected independent gated fusion modules for gated weight fusion includes: S311. Input the final slice fusion features, conservation constraint features and periodicity features into the corresponding connected independent gated fusion modules to calculate the gate weights, and obtain the gate weights of the slice fusion features, conservation constraint features and periodicity features respectively. S312. The final slice fusion features, conservation constraint features, and periodic regularity features are fused with their corresponding gating weights and added to the residual connection to obtain the fusion feature sequence.

[0018] Preferably, inputting the fused feature sequence into the feedforward output module includes: S411. Input the fused feature sequence into a feedforward network for nonlinear transformation and output the nonlinear transformation features; S412. Perform layer normalization on the nonlinear transformation features and output the layer normalized features; S413. Input the layer normalized features into the linear projection layer, and have the linear projection layer output a fraction matrix; S414. Input the score matrix into the Softmax layer and output the probability distribution prediction result of the multivariate task.

[0019] Compared with the prior art, the beneficial effects of the technical solution of the present invention are: This invention constructs a multivariate task prediction neural network. Through parallel slice-layered attention mechanism submodules, energy encoders, and periodic encoders within the network, it extracts intrinsic features of multivariate sequences from three dimensions: variable association and temporal dependence, information flow conservation constraints, and sequence periodicity. It deeply embeds three general physical principles—connectivity, conservation constraints, and periodic oscillations—into the network's forward inference process. This allows for the realization of intrinsic constraints of physical laws without prior knowledge of the system's precise control equations, overcoming the application limitations of traditional physical information neural networks that rely on known explicit control equations. Simultaneously, the final outputs of the slice-layered attention mechanism submodule, energy encoder, and periodic encoder are connected to independent gating fusion modules, performing dynamic weight adaptation on the three types of meta-principle features. This automatically adjusts the contribution ratio of each principle branch according to the inherent characteristics of different multivariate prediction scenarios, significantly enhancing the intrinsic physical rationality and out-of-distribution generalization ability of the prediction results while improving the accuracy of multivariate sequence prediction. This adapts to neural network prediction of complex and unknown multivariate tasks, effectively compensating for the core deficiency of purely data-driven phenomenological fitting models that lack an intrinsic understanding of the fundamental laws of the physical world. Attached Figure Description

[0020] Figure 1 This is a structural diagram of a multivariate task prediction neural network based on a meta-principle physical architecture proposed in one embodiment; Figure 2 This is a flowchart of a multivariate task prediction method based on a meta-principle physical architecture multivariate task prediction neural network proposed in one embodiment. Figure 3 This is an architecture diagram of a multivariate task prediction neural network based on a meta-principle physical architecture proposed in one embodiment; 1. Input embedding module; 2. Meta-principle encoding module; 3. Independent gating fusion module; 4. Feedforward output module. Detailed Implementation

[0021] To make the purpose, technical solution, and advantages of this application clearer, the prior art involved in this application is described below: Existing neural network architectures rely solely on data-driven statistical learning, lacking an intrinsic understanding of the fundamental meta-principles of the physical world (connectivity, conservation, and periodicity), resulting in weak performance in tasks involving physical reasoning, mathematical proof, and counterfactual inference. Existing physical information neural networks only impose physical constraints at the loss function level, failing to embed physical principles into the neural network's underlying architecture and thus lacking true intrinsic deductive capabilities. Existing efficient sequence modeling methods, while pursuing computational efficiency, neglect the embedding of physical principles, failing to achieve global physical principle embedding while maintaining causal constraints. Existing methods struggle to simultaneously satisfy the following three requirements: global physical principle embedding, strict causal constraints on autoregressive models, and parallel training efficiency. To address these issues, this invention proposes a Multivariate Task Prediction Neural Network (MPPA) based on a meta-principles physical architecture. By explicitly mapping the three core meta-principles of the physical world—connectivity, conservation, and periodicity—to differentiable modular components within the neural network, it achieves a paradigm shift from phenomenological fitting to intrinsic deduction.

[0022] The present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the scope of the present application.

[0023] Example 1 See Figure 1 This embodiment proposes a multivariate task prediction neural network based on a meta-principle physical architecture, including an input embedding module 1, an N-layer stacked meta-principle encoding module 2, three independent gated fusion modules 3, and a feedforward output module 4 connected in sequence; N is a positive integer; Each layer of the meta-principle coding module includes a parallel slice-layered attention mechanism submodule, an energy encoder, and a periodic encoder; The output of the input embedding module is connected to the original input of the slice hierarchical attention mechanism submodule, the energy encoder, and the periodic encoder, respectively. The final output of the slice hierarchical attention mechanism submodule, the energy encoder, and the periodic encoder is connected to the input of an independent gated fusion module, and the output of each independent gated fusion module is connected to the feedforward output module.

[0024] See Figure 2 and Figure 3 This embodiment also proposes a multivariate task prediction method based on the meta-principle physical architecture of the above-mentioned multivariate task prediction neural network, including the following steps: S1. Obtain the input sequence, input the input sequence into the input embedding module for vector transformation, and output an initial feature vector sequence; the input embedding module includes a word embedding layer and a position embedding layer connected in sequence; the step of inputting the input sequence into the input embedding module for vector transformation includes: S111. Input the input sequence into the word embedding layer, extract the high-dimensional semantic feature vector of the input sequence from the word embedding layer, and output the word vector to the position embedding layer; S112. The position embedding layer adds temporal position encoding to the word vector, outputs a position embedding vector, and normalizes the position embedding vector through the layer to output the initial feature vector sequence after layer normalization.

[0025] S2. The initial feature vector sequence is input into each stacked meta-principle coding layer for iterative feature extraction. The slice fusion feature is output through the slice hierarchical attention mechanism submodule in each meta-principle coding layer, the conservation constraint feature is output through the energy encoder in each meta-principle coding layer, and the periodic regularity feature is output through the periodic encoder in each meta-principle coding layer. After processing by each meta-principle coding layer, the final slice fusion feature, conservation constraint feature and periodic regularity feature are output. S3. Input the final slice fusion features, conservation constraint features and periodic regularity features into the corresponding connected independent gated fusion modules for gated weight fusion, and output the fusion feature sequence; S4. Input the fused feature sequence into the feedforward output module to output the probability distribution prediction result of the multivariate task.

[0026] In this embodiment, a multivariate task prediction neural network is first constructed. Through parallel slice-layered attention mechanism submodules, energy encoders, and periodic encoders within the multivariate task prediction neural network, intrinsic features of the multivariate sequence are extracted from three dimensions: variable association and temporal dependence, information flow conservation constraints, and sequence periodicity. The three general physical principles of connectivity association, conservation constraints, and periodic oscillations are deeply embedded into the network's forward inference process. This allows for the realization of intrinsic constraints of physical laws without prior knowledge of the system's precise control equations, overcoming the application limitations of traditional physical information neural networks that rely on known explicit control equations. Simultaneously, the final outputs of the slice-layered attention mechanism submodule, energy encoder, and periodic encoder are connected to independent gating fusion modules, performing dynamic weight adaptation on the three types of meta-principle features. This automatically adjusts the contribution ratio of each principle branch according to the inherent characteristics of different multivariate prediction scenarios, significantly enhancing the intrinsic physical rationality and out-of-distribution generalization ability of the prediction results while improving the accuracy of multivariate sequence prediction. This adapts to neural network prediction of complex and unknown multivariate tasks, effectively compensating for the core deficiency of purely data-driven phenomenological fitting models that lack an intrinsic understanding of the fundamental laws of the physical world.

[0027] Example 2 This embodiment further illustrates the multivariate task prediction method based on the meta-principle physical architecture of the multivariate task prediction neural network proposed in the above embodiment.

[0028] The slice-based hierarchical attention mechanism submodule described in S2 is used to implement the connectivity principle. It divides the input sequence into fixed-size slices, calculates local associations between tokens within each slice using a self-attention mechanism, and models cross-slice long-distance dependencies using a hierarchical attention aggregation mechanism. Specifically, S2 describes the output of slice fusion features through the slice hierarchical attention mechanism submodule in each meta-principle coding layer, including: S211. Input the initial feature vector sequence Divide the slice into N consecutive slices along the time dimension; where n represents the total length of the input sequence, and C represents the size of each slice; S212. Within each slice, calculate the attention score using the query matrix Q and the key matrix K. The attention score S is numerically stabilized using a soft-clipping function, and the stabilized attention score is output. The stabilized attention score is then normalized and weighted and summed with the value matrix V to obtain the local interaction features. ; Among them, the Soft-Clipping function f ( x The calculation expression is as follows:

[0029] c represents the threshold, with a value of 10; S213. Local interaction features The statistical values ​​are input into a preset weighted aggregation function to calculate the representative value for each slice. The expression is as follows:

[0030] in, As the first initialization parameter, For the second initialization parameter, For the third initialization parameter, Represents the vector of maximum values ​​in a slice. Represents the vector of minimum values ​​in a slice. Represents the slice mean vector; S214. Utilize preset learnable attention weights The representative value of each slice is merged into the current token to obtain the expression for the slice fusion feature as follows:

[0031] Where H represents the original feature matrix of the current token, Indicates slice fusion features, It represents the Hadamardi (or Hadama) stack.

[0032] Learnable attention weights The expression is as follows:

[0033] This represents the query vector for the current slice. For the pre-slice j The key vector; The energy encoder described in S2 is used to implement the conservation principle, maintaining conservation constraints during information propagation based on logarithmic domain energy tracking and delay compensation mechanisms. The energy encoder in each meta-principle coding layer of S2 outputs conservation constraint features, including: S221. Calculate the average energy for each slice. as follows:

[0034] in, This represents the feature vector corresponding to the t-th time step; S222. Based on the average energy Logarithmic cumulative energy and as follows:

[0035] S223. Based on the energy and Global geometric mean energy estimation is performed to obtain the global geometric mean energy. as follows:

[0036] S224. Calculate the current slice energy relative to the global geometric mean energy. deviation as follows:

[0037] S225. Based on the aforementioned deviation The conservation constraint characteristics are calculated using a delayed two-step compensation mechanism. .

[0038] The conservation constraint features The calculation expression is as follows:

[0039] in, This represents the original input feature matrix of the i-th slice. Let represent the Sigmoid activation function, and I represent the learnable compensation strength parameter.

[0040] The periodic encoder described in S2 is used to implement the periodicity principle, modeling the periodicity and oscillatory behavior of the system through spectral analysis and delay modulation mechanisms based on Fast Fourier Transform (FFT). The periodic encoder in each meta-principle coding layer of S2 outputs periodic characteristics, including: S231. Apply a Fourier transform to each slice along the sequence dimension and output the Fourier transform result. as follows:

[0041] S232. Based on the Fourier transform result Calculate the amplitude spectrum as follows: ; S233. Based on the amplitude spectrum Historical spectrum information is accumulated using the exponential moving average mechanism. as follows:

[0042] S234. Based on the historical spectrum information The periodic characteristics are calculated using a one-step delay compensation mechanism. ; The periodic characteristics The calculation expression is as follows:

[0043] in, This indicates a learnable multilayer perceptron. Indicates the first Historical spectral information of the slice.

[0044] S3 describes inputting the final slice fusion features, conservation constraint features, and periodic regularity features into corresponding connected independent gated fusion modules for gated weight fusion. The independent gated fusion module dynamically controls the contribution weights of the three meta-principle components, employing an independent gated fusion mechanism, including: S311. Input the final slice fusion features, conservation constraint features, and periodicity features into the corresponding connected independent gated fusion modules to calculate the gate weights, thereby obtaining the gate weights of the slice fusion features. Gating weights with conservation constraints Gating weights with periodic characteristics The specific calculations are as follows:

[0045] S312. The final slice fusion features, conservation constraint features, and periodicity features are fused with their corresponding gate weights and then added together with the residual connections to obtain the fused feature sequence. as follows:

[0046] T represents the periodicity characteristic of the output of the periodic encoder, and Z, E, and T are the outputs of the slice-layer attention mechanism submodule, the energy encoder, and the periodic encoder, respectively. x For residual connections; The architecture of the meta-principle encoding module strictly follows autoregressive causal constraints, and all components operate under causal mask protection to ensure that the current position depends only on historical information.

[0047] The step of inputting the fused feature sequence into the feedforward output module includes: S411. Input the fused feature sequence into a feedforward network for nonlinear transformation and output the nonlinear transformation features; S412. Perform layer normalization on the nonlinear transformation features and output the layer normalized features; S413. Input the layer normalized features into the linear projection layer, and have the linear projection layer output a fraction matrix; S414. Input the score matrix into the Softmax layer and output the probability distribution prediction result of the multivariate task.

[0048] In this embodiment, the input sequence is first divided into slices of fixed size in the slice hierarchical attention mechanism submodule. Local associations are calculated within each slice using self-attention, and hierarchical aggregation is achieved between slices using learnable attention weights. The fusion formula is as follows: This mechanism replaces traditional global attention, significantly reducing computational complexity while maintaining global dependency modeling capabilities. Secondly, the conservation constraint implementation method based on number domain energy tracking and delay compensation achieves cross-slice global conservation constraints under strict causal constraints through number domain energy accumulation, geometric mean estimation, debt queue storage, and delay feedback mechanisms. Furthermore, the periodic modeling method based on FFT spectrum analysis and EMA delay modulation achieves endogenous modeling of system periodicity through slice-level FFT transform, amplitude spectrum calculation, exponential moving average accumulation, and delay modulation. Finally, the independent gating fusion module dynamically adjusts the contribution weights of the three meta-principle components through learnable independent gating parameters, achieving selective activation at the physical principle level. This realizes the overall architectural design idea of ​​explicitly mapping the three physical meta-principles—connectivity, conservation, and periodicity—to differentiable modular components in a neural network.

[0049] The multivariate task prediction method based on a meta-principle physical architecture and a multivariate task prediction neural network proposed in this embodiment also has the following advantages: (1) Breakthrough improvement in physical reasoning ability: Experiments show that the architecture of this invention has achieved a breakthrough improvement from zero to 0.436 in physical reasoning tasks (the baseline model score is 0.000), and has achieved significant improvements in system identification, parameter prediction, trajectory quality, energy conservation and other dimensions.

[0050] (2) Significant improvement in mathematical task performance: In mathematical tasks, the present invention achieves a 2.18-fold improvement in overall performance (score increased from 0.151 to 0.330), especially in symbol generation accuracy (token matching rate increased by 13 times) and derivation structure integrity.

[0051] (3) Enhanced logical reasoning ability: A 52% improvement in overall performance on logical tasks (score increased from 0.300 to 0.456), of which clause structure generation ability improved by 758%.

[0052] (4) Complete retention of general language modeling capability: In language tasks, the performance of this invention is on par with the baseline model (both scores are 0.889), which verifies that the meta-principle embedding has no negative impact on general capability and solves the industry pain point of existing physical information models that "professional capability is improved but general capability is reduced".

[0053] (5) Reduction of verification perplexity: The verification set perplexity decreased from 269.40 to 259.45, a relative decrease of 3.69%, indicating that the architecture of the present invention has stronger sequence modeling ability and generalization performance.

[0054] (6) Advantages of parameter efficiency: Only 11.8% of the parameters (242.40M vs 216.91M) are needed to significantly improve the above-mentioned capabilities, which has good parameter efficiency.

[0055] (7) Enhanced generalization ability outside the distribution: Since the physical principle has universality and causal explanatory power, the architecture of this invention exhibits strong generalization ability in physical scenarios outside the training distribution.

[0056] (8) Improved interpretability: The three meta-principle components have clear physical meanings—the slice layered attention mechanism reflects the intensity of causal influence between entities, the energy encoder reflects the conservation measure of information flow, and the periodic encoder reflects the extraction of periodic patterns in system evolution—making the model decision-making process more transparent and interpretable.

[0057] Example 3 See Figure 3 This embodiment proposes an architecture for a multivariate task prediction neural network based on a meta-principle physical architecture. The input embedding module adopts a 12-layer stacked meta-principle encoding module, three independent gating fusion modules, and a feedforward output module. Each layer contains three parallel slice hierarchical attention mechanism sub-modules, an energy encoder, and a periodic encoder.

[0058] The architecture of the neural network for multivariate task prediction is configured as follows: 12 layers, 1024 hidden dimensions, 16 attention heads, 16 slices, and a total of 242.40M parameters. The baseline comparison model is the GPT architecture with the same configuration (12 layers, 1024 hidden dimensions, 16 attention heads, and 216.91M parameters).

[0059] The input text sequence is first converted into a vector representation through the word embedding layer and position embedding layer in the input embedding module, and then input to the first layer meta-principle encoding module.

[0060] In the slice-layer attention mechanism module, the input initial feature vector sequence (The sequence, with a length of 256 and a dimension of 1024, is divided into N=16 slices, each slice being C=16 in size. Within each slice, the sequence is projected through three linear projection matrices Q, K, and ... (Generate query, key, and value matrix, and calculate attention score) Apply the Soft-Clipping function Numerical stabilization is performed, and then a weighted aggregation function is used. Calculate the representative value for each slice. In modeling relationships between slices, the attention weights are calculated between the query vector of the current slice and the key vector of the preceding slice. This merges the representative value of the previous slice into the current token.

[0061] In the energy encoder, the average energy of each slice is first calculated. Maintaining the accumulated energy in the logarithmic field and Estimate the global geometric mean energy Calculate the energy deviation of the current slice. It is then stored in the debt queue and finally compensated through a two-step delayed mechanism. Perform energy compensation on the current slice.

[0062] In a periodic encoder, an FFT transform is applied along the sequence dimension to each slice. Calculate the amplitude spectrum Historical spectrum information is accumulated through the EMA mechanism. , The learnable parameters are constrained to between 0.1 and 0.9 by a sigmoid constraint and modulated by a one-step delay. Modulate the current slice.

[0063] The independent gated fusion module calculates the gate weights of the slice fusion features. Gating weights with conservation constraints Gating weights with periodic characteristics as follows:

[0064] The final fusion output is a fusion feature sequence. as follows:

[0065] Where T represents the periodicity of the output of the periodic encoder, and Z, E, and T are the outputs of the slice-layered attention mechanism submodule, the energy encoder, and the periodic encoder, respectively. x For residual connections; After 12 layers of meta-principle encoding, the final representation is the probability distribution prediction result of the multivariate task generated through layer normalization and linear projection layers in the feedforward output module.

[0066] The experimental results of the multivariate task prediction neural network based on the meta-principle physical architecture proposed in this embodiment are as follows: In the physics task, the overall score of this embodiment is 0.436, while the baseline GPT is 0.000, achieving a breakthrough improvement of 436 times; in the mathematical task, the overall score improves from 0.151 to 0.330, an improvement of 2.18 times; in the logic task, the overall score improves from 0.300 to 0.456, an improvement of 52%; in the language task, both scores are 0.889, verifying the lossless preservation of general capabilities. The validation set perplexity decreased from 269.40 to 259.45, a relative reduction of 3.69%.

[0067] This embodiment also included an energy encoder ablation experiment. To verify the independent contribution of the energy encoder, the energy encoder component was disabled in this embodiment, retaining only the slice hierarchical attention mechanism module and the periodic encoder. Experimental results showed that disabling the energy encoder reduced the overall score of the physics task from 0.436 to 0.423, a relative decrease of 3.03%. The decrease was most significant in the system identification and parameter prediction dimensions, verifying the core role of conservation constraint mechanisms in physics reasoning.

[0068] This embodiment also conducted an ablation experiment on the slice-layered attention mechanism. To verify the independent contribution of the slice-layered attention mechanism module, this module was disabled in this embodiment, and only the energy encoder and the periodic encoder were retained. The experimental results show that after disabling the module, the overall score of the physics task decreased from 0.436 to 0.430, a relative decrease of 1.47%, mainly affecting the system's recognition dimension, thus verifying the key role of the connectivity principle in modeling the interaction of physical entities.

[0069] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A multivariate task prediction neural network based on a meta-principle physical architecture, characterized in that, It includes an input embedding module connected in sequence, several layers of stacked meta-principle encoding modules, several independent gated fusion modules, and a feedforward output module; Each layer of the meta-principle coding module includes a parallel slice-layered attention mechanism submodule, an energy encoder, and a periodic encoder; The output of the input embedding module is connected to the original input of the slice hierarchical attention mechanism submodule, the energy encoder, and the periodic encoder, respectively. The final output of the slice hierarchical attention mechanism submodule, the energy encoder, and the periodic encoder is connected to the input of an independent gated fusion module, and the output of each independent gated fusion module is connected to the feedforward output module.

2. A multivariate task prediction method employing a multivariate task prediction neural network based on a meta-principle physical architecture as described in claim 1, characterized in that, Includes the following steps: S1. Obtain the input sequence, input the input sequence into the input embedding module for vector transformation, and output the initial feature vector sequence; S2. The initial feature vector sequence is input into each stacked meta-principle coding layer for iterative feature extraction. The slice fusion feature is output through the slice hierarchical attention mechanism submodule in each meta-principle coding layer, the conservation constraint feature is output through the energy encoder in each meta-principle coding layer, and the periodic regularity feature is output through the periodic encoder in each meta-principle coding layer. After processing by each meta-principle coding layer, the final slice fusion feature, conservation constraint feature and periodic regularity feature are output. S3. Input the final slice fusion features, conservation constraint features and periodic regularity features into the corresponding connected independent gated fusion modules for gated weight fusion, and output the fusion feature sequence; S4. Input the fused feature sequence into the feedforward output module to output the probability distribution prediction result of the multivariate task.

3. The multivariate task prediction method based on a meta-principle physical architecture multivariate task prediction neural network according to claim 2, characterized in that, The input embedding module includes a word embedding layer and a position embedding layer connected in sequence; the step of inputting the input sequence into the input embedding module for vector transformation includes: S111. Input the input sequence into the word embedding layer, extract the high-dimensional semantic feature vector of the input sequence from the word embedding layer, and output the word vector to the position embedding layer; S112. The position embedding layer adds temporal position encoding to the word vector, outputs a position embedding vector, and normalizes the position embedding vector through the layer to output the initial feature vector sequence after layer normalization.

4. The multivariate task prediction method based on a meta-principle physical architecture multivariate task prediction neural network according to claim 2, characterized in that, The slice fusion feature output through the slice hierarchical attention mechanism submodule in each meta-principle coding layer includes: S211. Divide the input initial feature vector sequence into N consecutive slices along the time dimension; S212. Within each slice, calculate the attention score using the query matrix Q and the key matrix K. The attention score S is numerically stabilized using a soft-clipping function, and the stabilized attention score is output. The stabilized attention score is then normalized and weighted and summed with the value matrix V to obtain the local interaction features. ; S213. Local interaction features The statistical values ​​are input into a preset weighted aggregation function to calculate the representative value for each slice. The expression is as follows: in, As the first initialization parameter, For the second initialization parameter, For the third initialization parameter, Represents the vector of maximum values ​​in a slice. Represents the vector of minimum values ​​in a slice. Represents the slice mean vector; S214. Utilize preset learnable attention weights The representative value of each slice is merged into the current token to obtain the expression for the slice fusion feature as follows: Where H represents the original feature matrix of the current token, Indicates slice fusion features, It represents the Hadamardi (or Hadama) stack.

5. The multivariate task prediction method based on a meta-principle physical architecture multivariate task prediction neural network according to claim 4, characterized in that, The energy encoder outputting conservation constraint features through each meta-principle coding layer includes: S221. Calculate the average energy for each slice. ; S222. Based on the average energy Logarithmic cumulative energy and ; S223. Based on the energy and Global geometric mean energy estimation is performed to obtain the global geometric mean energy. ; S224. Calculate the current slice energy relative to the global geometric mean energy. deviation ; S225. Based on the aforementioned deviation The conservation constraint characteristics are calculated using a delayed two-step compensation mechanism. .

6. The multivariate task prediction method based on a meta-principle physical architecture multivariate task prediction neural network according to claim 5, characterized in that, The conservation constraint features The calculation expression is as follows: in, This represents the original input feature matrix of the i-th slice. Let represent the Sigmoid activation function, and I represent the learnable compensation strength parameter.

7. The multivariate task prediction method based on a meta-principle physical architecture multivariate task prediction neural network according to claim 4, characterized in that, The periodic regularity features output by the periodic encoder in each meta-principle coding layer include: S231. Apply a Fourier transform to each slice along the sequence dimension and output the Fourier transform result. ; S232. Based on the Fourier transform result Calculate the amplitude spectrum ; S233. Based on the amplitude spectrum Historical spectrum information is accumulated using the exponential moving average mechanism. ; S234. Based on the historical spectrum information The periodic characteristics are calculated using a one-step delay compensation mechanism. .

8. The multivariate task prediction method based on a meta-principle physical architecture multivariate task prediction neural network according to claim 7, characterized in that, The periodic characteristics The calculation expression is as follows: in, This indicates a learnable multilayer perceptron. Indicates the first Historical spectral information of the slice.

9. The multivariate task prediction method based on a meta-principle physical architecture multivariate task prediction neural network according to claim 2, characterized in that, The step of inputting the final slice fusion features, conservation constraint features, and periodic regularity features into the corresponding connected independent gated fusion modules for gated weight fusion includes: S311. Input the final slice fusion features, conservation constraint features and periodicity features into the corresponding connected independent gated fusion modules to calculate the gate weights, and obtain the gate weights of the slice fusion features, conservation constraint features and periodicity features respectively. S312. The final slice fusion features, conservation constraint features, and periodic regularity features are fused with their corresponding gating weights and added to the residual connection to obtain the fusion feature sequence.

10. The multivariate task prediction method based on a meta-principle physical architecture multivariate task prediction neural network according to claim 2, characterized in that, The step of inputting the fused feature sequence into the feedforward output module includes: S411. Input the fused feature sequence into the feedforward network for nonlinear transformation and output the nonlinear transformation features; S412. Perform layer normalization on the nonlinear transformation features and output the layer normalized features; S413. Input the layer normalized features into the linear projection layer, and have the linear projection layer output a fraction matrix; S414. Input the score matrix into the Softmax layer and output the probability distribution prediction result of the multivariate task.

Images