Cross-layer path scheduling mechanism based on BioCircuit Transformer prototype feedback

Abstract

The present application relates to the technical field of artificial intelligence and neural network structure optimization, and particularly relates to a cross-layer path scheduling mechanism based on BioCircuit Transformer prototype feedback, which is applied to input sequences of the L+1 layer of a neural network model and comprises the following steps: S1, cross-layer feedback construction; S2, gate determination; S3, three-path execution and write-back; and S4, fusion and docking; through the novel combination of "cross-layer feedback gate + dynamic path switch", the present application first realizes three-state path control (activation / inhibition / cutting) in the interlayer propagation process of token, effectively reduces redundant propagation, and enhances the adaptability of deep modeling; the cross-layer feedback constraint significantly alleviates the problems of over-regulation and under-regulation, and maintains the balance of gradient and energy distribution; at the same time, the dynamic path switch reduces the calculation overhead, so that the model shows higher efficiency and more stable inference performance in high-redundancy tasks such as long text and code.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence and neural network structure optimization technology, and in particular to a cross-layer path scheduling mechanism based on BioCircuit Transformer prototype feedback. Background Technology

[0002] As a core foundational model in current artificial intelligence tasks such as natural language processing and computer vision, the Transformer architecture achieves parallel processing of sequential data through a self-attention mechanism, significantly improving the model's ability to capture long-distance dependencies. However, with the increase in the number of model layers and the complexity of processing tasks, the traditional Transformer has gradually exposed a series of key technical problems in the process of cross-layer information propagation, which seriously restricts the stability of deep semantic representation and inference efficiency. In traditional Transformers, the cross-layer propagation path of tokens is typically static, relying mainly on residual connections or fixed skip connections. This approach has the following drawbacks: 1. Unstable information propagation: As the number of model layers increases, the token activation state is easily amplified or weakened due to the cumulative effect during the layer-by-layer propagation process, resulting in semantic drift between the deep semantic representation and the initial input semantics. This problem is more prominent when processing long text sequences. 2. Severe redundancy in computation: All tokens must be forcibly passed layer by layer and participate in the complete computation process of each layer. There is a lack of dynamic pruning mechanism for redundant tokens. A large amount of computing power is consumed in the processing of tokens that have very low contribution to the task, resulting in low model inference efficiency. 3. Lack of semantic adaptability: Fixed jump connections adopt an indiscriminate treatment strategy for key tokens and redundant tokens, and cannot dynamically adjust the propagation path according to the semantic importance of tokens. This makes it difficult for deep models to maintain robustness to core semantic information, and performance degrades significantly in noisy data or highly redundant tasks. To address the aforementioned issues of traditional Transformers, the BioCircuitTransformer (BCT) architecture has been proposed in related technical fields. The BCT architecture introduces prototype feedback and energy regulation mechanisms. By constructing prototype aggregation and energy constraints among tokens within the same layer or locally upstream and downstream, it optimizes the local semantic representation of tokens to a certain extent. Specifically, BCT maps local tokens to the prototype space through Prototype Aggregation Units (PAUs / MSPAs), generates local prototype responses, and calculates prototype energy scales, thereby achieving semantic calibration and energy balance for local tokens. However, the prototype feedback mechanism of BCT still has significant limitations: its feedback scope is limited to local tokens within the same layer or a few adjacent layers, and it has not yet formed a systematic regulation path for cross-layer information propagation. This local feedback design leads to two key problems in the BCT architecture for deep models: first, feedback regulation imbalance, where some layers suffer from insufficient feedback signals (underregulation), failing to effectively calibrate semantic drift, while others experience excessively strong feedback signals (overregulation), causing token activation state oscillations; second, gradient distribution imbalance, where the incoherence of inter-layer feedback makes gradients prone to vanishing or exploding during backpropagation, further exacerbating the training instability of deep models. Therefore, we propose a cross-layer path scheduling mechanism based on the BioCircuit Transformer prototype feedback to address these issues. Summary of the Invention

[0003] This invention addresses the shortcomings of feedback regulation imbalance and gradient distribution imbalance in existing technologies by providing a cross-layer path scheduling mechanism based on BioCircuit Transformer prototype feedback.

[0004] This invention is achieved through the following technical solution: A cross-layer path scheduling mechanism based on BioCircuit Transformer prototype feedback is applied to the input sequence of the (L+1)th layer of a neural network model. This includes the following steps: S1, Cross-layer Feedback Construction: Based on the prototype set of the Lth layer. and prototype energy Calculate the cross-layer prototype response weights for each position i. ; S2. Gating Decision: First calculate the cross-layer feedback characteristics at position i in the (L+1)th layer. Then, the input representation of the i-th token in the (L+1)-th layer... With corresponding cross-layer feedback features splicing, and calculating the gating coefficient through gating mapping. And set a high threshold and low threshold ,and > According to the gating coefficient The three-state set of tokens at level L+1: activation set Suppression set ; S3, Three-Path Execution and Write-back: Including the activation set during main path execution. The token in the process performs QKV attention computation to obtain the main path update result. ; Suppression set during secondary path execution The tokens in the middle utilize the corresponding cross-layer feedback features. Feedback adjustment yields secondary path update results. ; During the execution of the pruning path, the pruning set is... The tokens in the data are processed using either equal-length pruning or physical pruning to obtain the pruning path results. ; S4. Fusion and Integration: Based on the mask information of the three-state set, the main path results are... Secondary path results Pruning path results Bitwise fusion yields the output of layer L+1. This output serves as the input to the next layer of the neural network model. .

[0005] In a preferred embodiment of the present invention, the cross-layer prototype response weights in S1 The calculation formula is ,in The temperature / sharpening coefficient is greater than 0. β is a set of prototype weights for the current token. It combines directional consistency (cos) and prototype credibility strength (log), quantitatively mapping the pattern consensus of the upper layer to "which prototypes the token should refer to and what proportion of each should be included".

[0006] In a preferred embodiment of the present invention, the prototype set of the Lth layer The basic form is This can be obtained through the following steps: Prototype spatial projection: The input representation of the i-th token in the L-th layer is as follows Through two layers of feedforward network Input representation for each token Projecting is performed to obtain the prototype space projection vector. ; Neighborhood similarity and response weight calculation: For each position i, calculate its similarity with other positions j in the neighborhood. Where d is the hidden dimension. As the scaling factor, select the top-k neighborhood N of position i. k (i) Normalize the similarity within the neighborhood to obtain the response weight. ; Local prototype response calculation: For each position i, calculate the local prototype response based on the neighborhood response weights and the prototype space projection vector. ; Layer Prototype Aggregation: Set the number of prototypes m in each layer, and use non-negative normalized weights. ∈[0,1] for all local prototype responses Aggregation is performed to obtain the prototype set of the Lth layer. .

[0007] In a preferred embodiment of the present invention, the prototype energy For each prototype at level L... Its energy scale is defined in its basic form as follows: , It is a numerical stability constant greater than 0, with a value range of 10. -8 ~10 -6 The L2 norm of the corresponding vector measures the vector strength at that location, and ZƐ is the normalization factor for the overall layer response strength. For the u-th prototype, is the energy scale, which is the proportionality of the response intensity borne by the prototype relative to the total layer intensity.

[0008] In a preferred embodiment of the present invention, the cross-layer feedback feature at each position i in the (L+1)th layer The calculation formula is . Cross-layer feedback features are the synthesis suggestions for the token from the previous layer. They are combinations of the prototype set (β is non-negative and sums to 1), thus located within the semantic bag of the prototypes, expressing which stable patterns the token should gravitate towards. Subsequent gating uses both f and e as criteria to determine the propagation strength of the token.

[0009] In a preferred embodiment of the present invention, the gating coefficient The calculation process is to first calculate After mapping through the Sigmoid function, .

[0010] In a preferred embodiment of the present invention, the high threshold and low threshold There are two setting methods: fixed threshold and adaptive threshold. Setting a fixed threshold directly... and Specific values, for example ( , = (0.30, 0.70); Adaptive threshold is set to... and , where quantile(·,q) is the empirical quantile; , It is at the quantile level, and , , ∈(0,1).

[0011] A system based on BioCircuit Transformer prototype feedback cross-layer path scheduling mechanism includes an input and embedding module, a prototype processing module, an inter-layer feedback gating module, a dynamic path scheduling module, and a layer output fusion module. The input and embedding module is used to segment the discrete token sequence, generate token index and position index, and perform a word embedding lookup function. and position embedding matrix The initial embedding representation of the token is calculated. ,in ; Let i be the index of the i-th discrete token; For location index; This is a word embedding lookup table function, which corresponds to a trainable matrix. ∈R v×d v is the vocabulary size, d is the embedding dimension / hidden dimension, and R is the real number field; ∈R p×d Given an existing trainable embedding matrix, P is the maximum sequence length covered by the positional embeddings; The prototype processing module is used to process the input sequence of the Lth layer of the neural network model. Processing is performed to generate the prototype set of the Lth layer. and prototype energy scale The prototype processing module includes a prototype space projection unit, a neighborhood similarity calculation unit, a local prototype response unit, a layer prototype aggregation unit, and a prototype energy calculation unit. The inter-layer feedback gating module is used to determine the Lth layer prototype set output by the prototype processing module. and prototype energy scale For the input sequence of the (L+1)th layer Processing is performed to generate gating coefficients. Furthermore, the interlayer feedback gating module includes a cross-layer prototype response weight calculation unit, a cross-layer feedback feature calculation unit, and a gating coefficient mapping unit; The dynamic path scheduling module is used to determine the gating coefficient output by the inter-layer feedback gating module. Dynamic path scheduling is performed on the tokens of the L+1 layer, and the dynamic path scheduling module includes a three-state set partitioning unit, a main path execution unit, a secondary path execution unit, and a pruned path execution unit. The layer output fusion module is used to fuse the main path result, secondary path result, and pruning path result output by the dynamic path scheduling module bit by bit according to the mask information of the three-state set, and output the final output of the (L+1)th layer. The output is then used as the input to the next layer of the neural network model. .

[0012] In a preferred embodiment of the present invention, the prototype spatial projection unit in the prototype processing module employs a two-layer feedforward network. Input representation for each token Perform projection and output the prototype space projection vector. ; The neighborhood similarity calculation unit calculates the similarity between each position i and its neighboring position j. It also normalizes the similarity within the neighborhood and outputs the response weights. ; Local prototype response units are based on neighborhood response weights. and prototype space projection vector Calculate and output the local prototype response. ; Layer prototype aggregation units are obtained through non-negative normalized weights. Response to all local prototypes Perform aggregation and output the prototype set of the Lth layer. ; The prototype energy calculation unit is based on the L2 norm and non-negative normalized weights of the local prototype response. Calculate and output the prototype energy scale of the Lth layer. .

[0013] In a preferred embodiment of the present invention, the cross-layer prototype response weight calculation unit in the inter-layer feedback gating module calculates the weight based on the cosine similarity and the logarithm of the prototype energy scale. Normalize, calculate and output cross-layer prototype response weights ; The cross-layer feedback feature calculation unit calculates the cross-layer prototype response weights. and the Lth level prototype set Calculate and output cross-layer feedback features ; The gating coefficient mapping unit concatenates the input representation of the L+1 layer token with the cross-layer feedback features, and then calculates and outputs the gating coefficients through linear transformation and Sigmoid mapping. .

[0014] The beneficial effects of this invention are: This invention combines "cross-layer feedback gating" with "dynamic path switching." The cross-layer feedback gating module acts as the signal generation end, generating precise cross-layer semantic feedback signals for each token based on the upper-layer global prototype set and energy scale through cosine similarity calculation, softmax normalization, and sigmoid mapping. Simultaneously, the dynamic path switch acts as the decision execution end, using a gating coefficient g as the basis and employing a dual-threshold approach. The decision mechanism transforms feedback signals into three path instructions: activation, inhibition, and pruning. It also incorporates these signals into the closed-loop design of decision execution, upgrading cross-layer feedback from local semantic calibration to global path scheduling. This breaks through the architectural limitations of separating feedback and path in existing technologies, effectively reducing the ineffective propagation of redundant tokens and lowering unnecessary computational overhead. At the same time, it allows deep models to dynamically adjust their processing strategies based on the semantic value of tokens, significantly improving their adaptive modeling capabilities for complex tasks. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the first half of the cross-layer path scheduling mechanism based on BioCircuit Transformer prototype feedback of the present invention. Figure 2 This is a schematic diagram of the latter part of the cross-layer path scheduling mechanism based on BioCircuit Transformer prototype feedback of the present invention. Detailed Implementation

[0016] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby providing a clearer and more explicit definition of the scope of protection of the present invention.

[0017] like Figure 1-2 The illustrated cross-layer path scheduling mechanism based on BioCircuit Transformer prototype feedback is applied to the input sequence of the (L+1)th layer of a neural network model. This includes the following steps: S1, Cross-layer Feedback Construction: Based on the prototype set of the Lth layer. and prototype energy Calculate the cross-layer prototype response weights for each position i. ; The prototype set of the Lth layer The basic form is This can be obtained through the following steps: Prototype spatial projection: The input representation of the i-th token in the L-th layer is as follows Through two layers of feedforward network Input representation for each token Projecting is performed to obtain the prototype space projection vector. ; Neighborhood similarity and response weight calculation: For each position i, calculate its similarity with other positions j in the neighborhood. Where d is the hidden dimension, As the scaling factor, select the top-k neighborhood N of position i. k (i) Normalize the similarity within the neighborhood to obtain the response weight. ; Local prototype response calculation: For each position i, calculate the local prototype response based on the neighborhood response weights and the prototype space projection vector. ; Layer Prototype Aggregation: Set the number of prototypes m in each layer, and use non-negative normalized weights. ∈[0,1] for all local prototype responses Aggregation is performed to obtain the prototype set of the Lth layer. ; Prototype energy For each prototype at level L... Its energy scale is defined in its basic form as follows: , It is a numerical stability constant greater than 0, with a value range of 10. -8 ~10 -6 The L2 norm of the corresponding vector measures the vector strength at that location. ZƐ is the normalization factor for the overall layer response strength. For the energy scale of the u-th prototype, it is the proportionality of the response intensity carried by the prototype relative to the total layer strength; Cross-layer prototype response weights The calculation formula is ,in The temperature / sharpening coefficient is greater than 0. β is a set of prototype weights for the current token. It combines directional consistency (cos) and prototype credibility strength (log), quantitatively mapping the pattern consensus of the upper layer to "which prototypes should the token refer to and what proportion of each should be included". S2. Gating Decision: First calculate the cross-layer feedback characteristics at position i in the (L+1)th layer. Cross-layer feedback characteristics The calculation formula is ; Cross-layer feedback features are the synthesis suggestions for the token from the previous layer. They are combinations of the prototype set (β is non-negative and sums to 1), thus located within the semantic package of the prototypes, expressing which stable patterns the token should gravitate towards. Subsequent gating uses both f and e as criteria to determine the propagation strength of the token. Represent the input of the i-th token in the (L+1)-th layer. With corresponding cross-layer feedback features splicing, and calculating the gating coefficient through gating mapping. Gating coefficient The calculation process is to first calculate After mapping through the Sigmoid function, ; Set a high threshold and low threshold ,and > According to the gating coefficient The three-state set of tokens at level L+1: activation set Suppression set ; High threshold and low threshold There are two setting methods: fixed threshold and adaptive threshold. Setting a fixed threshold directly... and Specific values, for example ( , = (0.30, 0.70); Adaptive threshold is set to... and Where quantile(·,q) is the empirical quantile. , It is at the quantile level, and , , ∈(0,1); S3, Three-Path Execution and Write-back: Including the activation set during main path execution. The token in the process performs QKV attention computation to obtain the main path update result. ; The specific process involves extracting subsequences. `gather` is a standard tensor indexing operation. By performing QKV projections respectively, we get Q=E A W Q K=E A W K V=E A WV W Q W K W V， W O Calculate the scaled dot product similarity matrix to obtain the trainable projection matrix. ,in Given dimensions Q and K, normalize S to obtain the attention weights A = softmax. row (S), calculate the attention aggregation value U=AV, and obtain the output mapping back to the channel dimension Y=UWo through the output mapping matrix Wo. After calculating the residual with Y, the corresponding position in the original sequence is updated through a write-back operation to obtain... ; Suppression set during secondary path execution The tokens in the middle utilize the corresponding cross-layer feedback features. Feedback adjustment yields secondary path update results. ; The specific process involves extracting the input subsequence corresponding to the suppression set. Cross-layer feedback feature subsequence ,in ,Will As the update value of the suppression set token, it is updated to the corresponding position in the original sequence through a write-back operation, resulting in... ,in ; During the execution of the pruning path, the pruning set is... The tokens in the data are processed using either equal-length pruning or physical pruning to obtain the pruning path results. ; If equal-length pruning is used, the token representation at the corresponding position in the pruning set is replaced with a placeholder vector r. pad ,Right now ,in This means putting the same r pad Write to all Prune lines. If r pad =0 (constant), Prune rows do not propagate gradients back to the input. If r pad Set as a trainable empty embedding, its gradient only updates the embedding and does not backpropagate to the input branch of the pruned token, which is stable and has low overhead; If physical pruning is used, the token row corresponding to the pruning set is removed in subsequent calculations, and only the tokens corresponding to the active set and the suppressed set are retained for calculation; S4. Fusion and Integration: Based on the mask information of the three-state set, the main path results are... Secondary path results Pruning path results Bitwise fusion yields the output of layer L+1. This output serves as the input to the next layer of the neural network model. .

[0018] A system based on BioCircuit Transformer prototype feedback cross-layer path scheduling mechanism includes an input and embedding module, a prototype processing module, an inter-layer feedback gating module, a dynamic path scheduling module, and a layer output fusion module. The input and embedding module is used to segment discrete token sequences, generate token indices and position indices, and perform table lookup using word embedding functions. and position embedding matrix The initial embedding representation of the token is calculated. ,in ; Let i be the index of the i-th discrete token; For location index; This is a word embedding lookup table function, which corresponds to a trainable matrix. ∈R v×d v is the vocabulary size, d is the embedding dimension / hidden dimension, and R is the real number field; ∈R p×d Given an existing trainable embedding matrix, P is the maximum sequence length covered by the positional embeddings; The prototype processing module is used to process the input sequence of the Lth layer of the neural network model. Processing is performed to generate the prototype set of the Lth layer. and prototype energy scale The prototype processing module includes a prototype space projection unit, a neighborhood similarity calculation unit, a local prototype response unit, a layer prototype aggregation unit, and a prototype energy calculation unit. The prototype space projection unit uses a two-layer feedforward network. Input representation for each token Perform projection and output the prototype space projection vector. ; The neighborhood similarity calculation unit calculates the similarity between each position i and its neighboring position j. It also normalizes the similarity within the neighborhood and outputs the response weights. ; Local prototype response units are based on neighborhood response weights. and prototype space projection vector Calculate and output the local prototype response. ; Layer prototype aggregation units are obtained through non-negative normalized weights. Response to all local prototypes Perform aggregation and output the prototype set of the Lth layer. ; The prototype energy calculation unit is based on the L2 norm and non-negative normalized weights of the local prototype response. Calculate and output the prototype energy scale of the Lth layer. .

[0019] The inter-layer feedback gating module is used to process the Lth layer prototype set output by the prototype processing module. and prototype energy scale For the input sequence of the (L+1)th layer Processing is performed to generate gating coefficients. Furthermore, the interlayer feedback gating module includes a cross-layer prototype response weight calculation unit, a cross-layer feedback feature calculation unit, and a gating coefficient mapping unit; The cross-layer prototype response weight calculation unit calculates the weights based on cosine similarity and the logarithm of the prototype energy scale, through... Normalize, calculate and output cross-layer prototype response weights ; The cross-layer feedback feature calculation unit calculates the cross-layer prototype response weights. and the Lth level prototype set Calculate and output cross-layer feedback features ; The gating coefficient mapping unit concatenates the input representation of the L+1 layer token with the cross-layer feedback features, and then calculates and outputs the gating coefficients through linear transformation and Sigmoid mapping. .

[0020] The dynamic path scheduling module is used to determine the gating coefficients output by the inter-layer feedback gating module. Dynamic path scheduling is performed on the tokens of the L+1 layer, and the dynamic path scheduling module includes a three-state set partitioning unit, a main path execution unit, a secondary path execution unit, and a pruned path execution unit. The three-state set partitioning unit is based on a set high threshold. Determine the gating coefficient, divide and output the activation set. Suppression set ; The main path execution unit performs QKV attention calculation and residual update on the tokens in the activation set, and outputs the main path update result. The main path execution unit supports two computation modes: single-head attention and multi-head attention. When using multi-head attention, Q, K, and V are split by head, the attention results of each head are calculated separately, and then concatenated at the end. Finally, the output mapping matrix WO is used to map back to the hidden dimension d. The main path execution unit only applies to the Active subset. Perform standard self-attention to avoid redundant QKV computations on Inhibit / Prune, thus concentrating computational power on high-confidence tokens and the input sequence of this layer. Active index set , mask Main path update output , where only i∈ The position is updated, while other positions remain placeholders in this section, and the subsequence is extracted by the Active index. `gather` is a standard tensor indexing operation, followed by QKV projection: Q=E A W Q K=E A W K V=E A W V Q, K, V projection and output mapping matrix, W Q W K W V W O These are trainable parameters; Then, apply row-normalized softmax weights A = softmax row (S), scaled dot product similarity matrix The aggregated value after attention is U=AV, and the output is mapped back to the channel dimension Y=UWo. The intermediate result calculated by attention is projected back to the standard dimension d of the model using a linear transformation, so that it can be added to the input residual and fed into subsequent layers.

[0021] The secondary path execution unit adjusts and updates the tokens in the suppression set using cross-layer feedback features, and outputs the secondary path update result. ; Cross-layer feedback guidance is used to correct tokens flagged as Inhibits, while preserving the original representation. At the same time, utilize the comprehensive feedback from the previous prototype. Directional corrections are made to mitigate over-modulation / under-modulation and semantic drift in deep propagation. Output Only when i∈I (L+1) The position is updated, while the remaining positions remain placeholders. Then, it is merged bit by bit with the main path to obtain the full layer output.

[0022] enter: The input sequence for this layer.

[0023] ⊆{1,...,n}, the set of Inhibit indices.

[0024] Cross-layer feedback characteristics.

[0025] Output: Only when i∈ The position is updated, while the remaining positions remain unchanged; then it is merged bit by bit with the main path to obtain the full layer output.

[0026] Extract the corresponding row from matrix X according to the index set S: ; Using E as the base, write X back to rows in the index set S, while keeping the original values ​​of E for the other rows: The pruning path execution unit processes the tokens in the pruning set using either equal-length pruning or physical pruning methods, and outputs the pruning path results. ; Input sequence for this layer Based on the three-state set, tensor operators. For The token within the layer skips subsequent operators, blocking unstable / redundant paths at minimal cost; while ensuring consistency with the main / secondary paths and subsequent fusion interfaces. Two implementation modes: (A) Equal-length pruning (logical mask, recommended default): Keeps the sequence length n unchanged, only replaces the representation of the Prune position with a placeholder vector: in, Indicates putting the same Write to all Prune lines. If =0 (constant), Prune rows do not propagate gradients back to the input. If Assuming a trainable empty embedding, its gradient only updates this embedding and is not backpropagated to the input branch of the pruned token, making it stable and low-cost.

[0027] (B) Physical pruning (index compression, further reducing computation): Remove Prune rows in subsequent operators at this level, only for the retained set. It preserves the output concatenation of the set after traversing the main / secondary paths. This truly reduces the memory and computational load of this layer. Only the K index and write-back mapping need to be maintained.

[0028] Align with primary and secondary paths: The three paths are mutually exclusive and cover the entire set {1,...,n}. This is a placeholder for an empty set.

[0029] The layer output fusion module is used to fuse the main path result, secondary path result, and pruning path result output by the dynamic path scheduling module bit by bit according to the mask information of the three-state set, and output the final output of the (L+1)th layer. The output is then used as the input to the next layer of the neural network model. .

[0030] The embodiments described above are merely examples of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A cross-layer path scheduling mechanism based on BioCircuit Transformer prototype feedback, applied to the input sequence of the (L+1)th layer of a neural network model. Its characteristics are, Includes the following steps: S1, Cross-layer Feedback Construction: Based on the prototype set of the Lth layer. and prototype energy Calculate the cross-layer prototype response weights for each position i. ; S2. Gating Decision: First calculate the cross-layer feedback characteristics at position i in the (L+1)th layer. Then, the input representation of the i-th token in the (L+1)-th layer... With corresponding cross-layer feedback features splicing, and calculating the gating coefficient through gating mapping. And set a high threshold and low threshold And according to the gating coefficient The three-state set of tokens at level L+1: activation set Suppression set ; S3, Three-Path Execution and Write-back: Including the activation set during main path execution. The token in the process performs QKV attention computation to obtain the main path update result. ; Suppression set during secondary path execution The tokens in the middle utilize the corresponding cross-layer feedback features. Feedback adjustment yields secondary path update results. ; During the execution of the pruning path, the pruning set is... The tokens in the data are processed using either equal-length pruning or physical pruning to obtain the pruning path results. ; S4. Fusion and Integration: Based on the mask information of the three-state set, the main path results are... Secondary path results Pruning path results Bitwise fusion yields the output of layer L+1. This output serves as the input to the next layer of the neural network model. .

2. The cross-layer path scheduling mechanism based on BioCircuit Transformer prototype feedback as described in claim 1, characterized in that: The cross-layer prototype response weights in S1 The calculation formula is: .

3. The cross-layer path scheduling mechanism based on BioCircuit Transformer prototype feedback as described in claim 2, characterized in that: The prototype set of the Lth layer The basic form is This can be obtained through the following steps: Prototype spatial projection: The input representation of the i-th token in the L-th layer is as follows Through two layers of feedforward network Input representation for each token Projecting is performed to obtain the prototype space projection vector. ; Neighborhood similarity and response weight calculation: For each position i, calculate its similarity with other positions j in the neighborhood. Select the top-k neighborhood N of position i k (i) Normalize the similarity within the neighborhood to obtain the response weight. ; Local prototype response calculation: For each location i, calculate the local prototype response based on the neighborhood response weights and the prototype space projection vector. ; Layer prototype aggregation: Set the number of prototypes m in each layer, and use non-negative normalized weights. ∈[0,1] for all local prototype responses Aggregation is performed to obtain the prototype set of the Lth layer. .

4. The cross-layer path scheduling mechanism based on BioCircuit Transformer prototype feedback as described in claim 2, characterized in that: The prototype energy For each prototype at level L... Its energy scale is defined in its basic form as follows: 。 5. The cross-layer path scheduling mechanism based on BioCircuit Transformer prototype feedback as described in claim 1, characterized in that: The cross-layer feedback feature at each position i in the (L+1)th layer The calculation formula is: .

6. The cross-layer path scheduling mechanism based on BioCircuit Transformer prototype feedback as described in claim 5, characterized in that: The gating coefficient The calculation process is to first calculate After mapping through the Sigmoid function, .

7. The cross-layer path scheduling mechanism based on BioCircuit Transformer prototype feedback as described in claim 1, characterized in that: The high threshold and low threshold There are two setting methods: fixed threshold and adaptive threshold. The fixed threshold method allows for direct setting. and The specific value is set using an adaptive threshold. and .

8. A system for cross-layer path scheduling based on BioCircuit Transformer prototype feedback, characterized in that: It includes an input and embedding module, a prototype processing module, an inter-layer feedback gating module, a dynamic path scheduling module, and a layer output fusion module; The input and embedding module is used to segment the discrete token sequence, generate token index and position index, and perform a word embedding lookup function. and position embedding matrix The initial embedding representation of the token is calculated. ; The prototype processing module is used to process the input sequence of the Lth layer of the neural network model. Processing is performed to generate the prototype set of the Lth layer. and prototype energy scale ; The inter-layer feedback gating module is used to determine the Lth layer prototype set output by the prototype processing module. and prototype energy scale For the input sequence of the (L+1)th layer Processing is performed to generate gating coefficients. ; The dynamic path scheduling module is used to determine the gating coefficient output by the inter-layer feedback gating module. Dynamic path scheduling is performed on the tokens at level L+1. The layer output fusion module is used to fuse the main path result, secondary path result, and pruning path result output by the dynamic path scheduling module bit by bit according to the mask information of the three-state set, and output the final output of the (L+1)th layer. The output is then used as the input to the next layer of the neural network model. .

9. The system based on the cross-layer path scheduling mechanism using BioCircuit Transformer prototype feedback as described in claim 8, characterized in that: The prototype processing module includes a prototype space projection unit, a neighborhood similarity calculation unit, a local prototype response unit, a layer prototype aggregation unit, and a prototype energy calculation unit.

10. The system based on the cross-layer path scheduling mechanism using BioCircuit Transformer prototype feedback as described in claim 8, characterized in that: The interlayer feedback gating module includes a cross-layer prototype response weight calculation unit, a cross-layer feedback feature calculation unit, and a gating coefficient mapping unit.

Images