A bio-inspired text sequence processing method

Abstract

This invention discloses a bio-inspired text sequence processing method, belonging to the field of text sequence processing, and applied to sequence retrieval and sequence recovery tasks. This method mimics the micropillar structure of the human cerebral cortex, encapsulating a large number of parallel Spiking neurons within each micropillar structure; it incorporates synaptic delay and Theta oscillation mechanisms to ensure periodic learning and prediction of text sequences; it designs a sparse temporal group coding scheme to transform input text characters into sparse distributed representations; and it proposes Spiking-based unsupervised learning rules to realize the storage and association process of text sequences. By periodically storing and associating the distributed representations of input sequence characters, this method can complete sequence retrieval tasks from partial context and sequence recovery tasks from damaged information, providing a new approach to the construction of artificial association systems and expanding the application scope of Spiking-based neuromorphic chips.

Description

Technical Field

[0001] This invention relates to the field of text sequence processing, specifically a bio-inspired text sequence processing method. Background Technology

[0002] The associative system in the human brain possesses powerful information storage and inference capabilities, making it an indispensable part of human cognition. With the rapid development of deep learning technology and inspired by the human brain's associative system, various associative memory models have been proposed. Utilizing the powerful sequence learning capabilities of deep neural networks, associative memory models can learn a wealth of contextual features from training data, enabling tasks such as sequence memory and temporal data processing. However, traditional associative models, when applied to text sequences, only borrow from the brain's hierarchical information processing structure at the brain region level, neglecting the neuronal structure and biological optimization methods within the biological nervous system. Although these models can accomplish certain complex sequence processing tasks, they struggle to achieve human-like associative memory functions, cannot better understand and process incomplete contextual information, and cannot extract accurate associations from it, thus failing to solve complex practical problems more efficiently. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of existing associative memory models, which only borrow from the structure of the brain for hierarchical information processing at the brain region scale, ignore the neuronal structure and biological optimization methods in the biological nervous system, cannot better understand and process incomplete contextual information, and cannot solve complex practical problems more efficiently. This invention provides a bio-inspired text sequence processing method.

[0004] The objective of this invention is mainly achieved through the following technical solutions:

[0005] A bio-inspired text sequence processing method includes the following steps:

[0006] S1: Represent each input character as a sparse distributed neural micropillar, and further convert the first input character into the pulse firing time of the Spiking neurons in the receiving neural micropillar;

[0007] S2: Calculate the activation state of the Spiking neuron in the first character receiving neural micropillar;

[0008] S3: Find the Spiking neuron that enters the prediction state for the next input character;

[0009] S4: Calculate the activation state of the Spiking neuron in the next input character receiving neural micropillar and define a unique winning neuron for that character receiving neural micropillar;

[0010] S5: Update the distal synaptic connections of the winning neuron;

[0011] S6: Repeat steps S3 - S5 until the activation state is defined for each spiking neuron contained in each neural microcolumn, and select a unique winning neuron;

[0012] S7: Sequence retrieval: Based on the sparse distributed neural microcolumns of characters, use the neural microcolumn where the neuron predicting the state is located to infer the next possible character;

[0013] S8: Sequence recovery: Use two types of information, the receiving neural microcolumn and the predicting neural microcolumn, to perform anomaly detection and correction on the damaged sequence.

[0014] Furthermore, step S1 is implemented using a sparse temporal group coding scheme, and this sparse temporal group coding scheme includes the following steps:

[0015] S11: Encode each input character through the BERT model to convert it into a feature vector;

[0016] S12: Reduce the dimension of the feature vector;

[0017] S13: Represent the low - dimensional feature vector as sparse distributed neural microcolumns, and further convert the first input character into the spike - firing time of the spiking neurons in the corresponding neural microcolumn through the Gaussian phase encoding method.

[0018] Furthermore, step S12 is implemented using a similarity transformation method, and this similarity transformation method includes the following steps:

[0019] S121: For a feature vector with dimension N Perform matrix multiplication with 3H random matrices with dimension C×N to obtain a feature vector

[0020]

[0021] where W hi represents the random matrix;

[0022] S122: For the feature vector Obtain a vector through scaled dot - product

[0023]

[0024] where · represents the inner product of vectors, and C is a constant;

[0025] S123: Concatenate the vectors to form a vector Then, through matrix multiplication with the random matrix W′ and normalization, obtain a low - dimensional feature vector with dimension M, M < N

[0026]

[0027] Wherein, the random matrix W gi Both W and W′ are sampled from a standard normal distribution.

[0028] Furthermore, step S13 is implemented using a sparse time group coding method, which includes the following steps:

[0029] S131: Low-dimensional feature vectors for character encoding using L neural micropillars: targeting the feature value range [I min ,I max The receptive domain of the neural micropillar l is:

[0030]

[0031] In the formula, l≥1, I min Let I be the smallest eigenvalue that can be encoded by L neural micropillars. max The maximum feature value that can be encoded by L neural micropillars is given. Each feature value of the vector falls into a neural micropillar, providing feedforward input to the neural micropillars. That is, a character can be encoded into a maximum of M neural micropillars.

[0032] S132: For the first character of the input, the feature value falling into the neural micropillar l is further converted into the pulse firing time t of the Spiking neuron in the neural micropillar using a Gaussian function:

[0033]

[0034] In the formula, x m Represents the eigenvalue, t max This represents the maximum encoding time window. b and c represent the mean and variance of the Gaussian function corresponding to the neural micropillar l, respectively. c is a constant value, set to 0.1, and b is set to:

[0035]

[0036] S133: Phase encoding is used to distinguish feature values ​​falling into the same neural micropillar. The encoding process is described using the cosine function.

[0037] f(t) = A cos(ωt + φ) j )

[0038] In the formula, f(t) represents the cosine function corresponding to the j-th dimension of the eigenvector, A represents the amplitude, ω represents the phase velocity, and φ represents the phase velocity. j The phase of the j-th feature dimension of the eigenvector is defined as:

[0039] φ j =φ0+(j-1)Δφ

[0040] In the formula, φ0 represents the initial phase, and Δφ represents the constant phase difference between adjacent feature dimensions.

[0041] Furthermore, the method for defining the activation state of the Spiking neuron includes the following steps:

[0042] S21: For the first character input, all the Spiking neurons in its neural micropillars will enter an activated state;

[0043] S22: For other input characters, calculate the membrane voltage of all Spiking neurons in their neural micropillars:

[0044]

[0045] In the formula, Γ represents the membrane voltage of the j-th Spiking neuron in the neural micropillar l at time t. j I represents the distal dendrite set of the j-th Spiking neuron in neural micropillar l. l (t) represents the feedforward input current, and W represents I. l The weights of (t) are set to 1, V θ (t) represents theta oscillation, V s (t) represents the membrane voltage of the distal dendrite s of the Spiking neuron after receiving distal synaptic input:

[0046]

[0047] In the formula, w is t represents the synaptic weight between presynaptic neuron i and synaptic dendrite s. i t represents the pulse firing time of presynaptic neuron i. d This represents synaptic delay, where k represents the kernel function, expressed by factors V0 and constants τ and τ'. s Together they control the postsynaptic membrane voltage:

[0048]

[0049] In the formula, u = t - (t i +t d () represents the current time t and the delayed presynaptic pulse firing time t. i +t d The difference, V θ (t) can be described using the sine function:

[0050] Vθ (t)=A θ sin(ω θ t+φ θ )

[0051] In the formula, A θ ω represents the amplitude below a threshold. θ φ represents the oscillation phase velocity. θ This represents the oscillation phase shift; each input character is learned and predicted within a single theta-cycle time T; current I l (t) can be described using a piecewise function:

[0052]

[0053] In the formula, I ext Set as a constant, and n is set as an integer greater than 1; when the membrane voltage Exceeding the ignition threshold V th The j-th Spiking neuron in the neural micropillar l will release a pulse, entering an activated state, with the ignition threshold V. th Set to V th =A θ +WI ext ;

[0054] The method for defining the predicted state of the Spiking neuron includes the following steps:

[0055] S31: Calculate the membrane voltage of the distal dendrite s of the Spiking neuron after receiving distal synaptic input:

[0056]

[0057] S32: Determine V s (t) Whether the dendritic activation threshold V is exceeded within the first quarter of the cycle. ath ;

[0058] S33: Determine whether the number of presynaptic neurons activated by the distal dendrite s in the previous cycle exceeds N. a ;

[0059] S34: Distal dendrites s that simultaneously satisfy S32 and S33 are defined as active dendrites, and Spiking neurons with at least one active dendrite are defined as predictive neurons.

[0060] The method for defining the winning neuron includes the following steps:

[0061] S41: Determine whether there is a predictive neuron in the neural micropillar. If it exists, define the predictive neuron as the winning neuron. If it does not exist, proceed to step S42.

[0062] S42: Determine whether there are matching dendrites in the Spiking neurons of the neural micropillars. If there are, define the Spiking neurons with matching dendrites as winning neurons. If not, go to step S43.

[0063] S43: Define the Spiking neuron with the fewest dendritic segments as the winning neuron and initialize N. s Each synapse forms a synaptic connection with the activated neuron of the previous cycle. If the number of synaptic connections on its distal dendrite exceeds the maximum number of synapses limit Synapse_max, the neuron will create a new distal dendrite and then continue to form synaptic connections with the activated neuron of the previous cycle. The initial synaptic weight is set to 0.21.

[0064] In the above steps, synaptic delay and Theta oscillation mechanisms were incorporated when calculating the membrane voltage of the Spiking neuron to ensure periodic learning and prediction of the text sequence.

[0065] Furthermore, the method for defining the matching dendrites is as follows:

[0066] Calculate the membrane voltage of the distal dendrite s of the Spiking neuron after receiving distal synaptic input:

[0067]

[0068] If V s (t) exceeds the dendrite matching threshold V within the first quarter of the cycle. pth Then, the distal dendrite s is defined as the matching dendrite.

[0069] Furthermore, step S5 is implemented using the STDP algorithm, which includes the following steps:

[0070] S51: Calculate the pulse firing time t of neuron j to which the distal postsynaptic dendrite s belongs. j And the delayed spiking time t of the presynaptic neuron i i +t d The difference u:

[0071] u = t j -(t i +t d )

[0072] S52: Calculate the synaptic weight change Δw between neuron j, to which the postsynaptic distal dendrite s belongs, and neuron i, to which the presynaptic neuron i belongs. is :

[0073]

[0074] In the formula, A ± and τ± All are constants greater than 0.

[0075] Furthermore, the anomaly detection and correction process in step S8 includes the following steps:

[0076] S81: For sequences In the formula, n is the length of the sequence, using M1(Y) i ) represents the character Y i The set of receiving neural micropillars, M2(Y) i ) represents the character Y i-1 The set of neural micropillars where the predicted neurons are located;

[0077] S82: Calculate character Y i abnormal score S i :

[0078]

[0079] In the formula, |·| represents the length of the set, and the abnormal score ranges from [0,1]. If the current input character is abnormal, the abnormal score will be close to 1; otherwise, the abnormal score will be close to 0.

[0080] S83: Set an exception threshold to determine if a character is abnormal. If the exception score S i If the value is greater than the abnormal threshold, input the character Y. i If this is considered abnormal, proceed to step S84;

[0081] S84: Based on the predicted neural micropillar set M2(Y) i Infer the correct character and replace the abnormal character Y. i .

[0082] In summary, the present invention has the following advantages compared with the prior art:

[0083] 1. This invention incorporates synaptic delay and Theta oscillation mechanisms to ensure periodic learning and prediction of text sequences.

[0084] 2. This invention designs a sparse temporal group coding scheme that can represent characters as sparse distributed neural micropillars, which not only makes the sequence learning process more robust and flexible, but also improves the performance of text sequence processing while stabilizing training.

[0085] 3. This invention proposes an unsupervised learning rule based on Spiking to regulate synaptic connections in the sequence learning process, effectively realizing the storage and association process of text sequences. Attached Figure Description

[0086] The accompanying drawings, which are included to provide a further understanding of embodiments of the invention and form part of this application, do not constitute a limitation thereof. In the drawings:

[0087] Figure 1 This is a schematic diagram illustrating the steps of an embodiment of the present invention;

[0088] Figure 2 This is a schematic diagram of the similarity conversion method according to an embodiment of the present invention;

[0089] Figure 3 This is a schematic diagram of the sparse temporal group coding method according to an embodiment of the present invention. Detailed Implementation

[0090] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of this invention are only for explaining this invention and are not intended to limit this invention.

[0091] Example:

[0092] like Figure 1 As shown, a bio-inspired text sequence processing method includes the following steps: S1: Represent each input character as a sparse distributed neural micropillar, and further convert the first input character into the pulse firing time of the Spiking neurons in the receiving neural micropillar; S2: Calculate the activation state of the Spiking neurons in the receiving neural micropillar of the first character; S3: Find the Spiking neurons that enter the prediction state for the next input character; S4: Calculate the activation state of the Spiking neurons in the receiving neural micropillar of the next input character, and define a unique winning neuron for the receiving neural micropillar of that character; S5: Update the distal synaptic connections of the winning neuron; S6: Repeat steps S3 to S5 until each neural micropillar defines the activation state of the Spiking neurons it contains, and selects a unique winning neuron; S7: Sequence retrieval: Based on the sparse distributed neural micropillar of the character, use the neural micropillar where the neuron in the prediction state is located to infer the next possible character; S8: Sequence recovery: Use both receiving neural micropillar and prediction neural micropillar information to detect and correct anomalies in the damaged sequence.

[0093] In one possible implementation, step S1 is implemented using a time group coding scheme, which includes the following steps:

[0094] S11: Encode each input character into a feature vector using the BERT model;

[0095] S12: Reduce the dimensionality of the feature vector;

[0096] S13: Represent the low-dimensional feature vector as sparse distributed neural microcolumns, and further convert the first input character into the spike firing time of the Spiking neurons in the corresponding neural microcolumns through the Gaussian phase encoding method.

[0097] Specifically, the feature vector encoded by the BERT model can well integrate context information, which is more conducive to further optimizing the Spiking neural network to complete more complex sequence processing tasks.

[0098] In a possible implementation, as Figure 2 shown, step S12 is implemented by the similarity transformation method, and this similarity transformation method includes the following steps:

[0099] S121: For the feature vector with dimension N Perform matrix multiplication with 3H random matrices with dimension C×N to obtain the feature vector

[0100]

[0101] where, W gi represents the random matrix;

[0102] S122: For the feature vector Use the scaling dot product to obtain the vector

[0103]

[0104] where, · represents the inner product of vectors, C is a constant, and the purpose of setting the constant C is to avoid the inner product of vectors being too large and thus changing the numerical distribution of the vector;

[0105] S123: Concatenate the vectors into the vector Then, through matrix multiplication with the random matrix W′ and normalization processing, obtain a low-dimensional feature vector with dimension M, M < N

[0106] <00003​​​​​​​​​In one possible implementation, step S13 is implemented using a sparse time group coding method, which includes the following steps:

[0110] S131: Encoding low-dimensional feature vectors using L neural micropillars: targeting the feature value range [I min ,I max The receptive domain of the neural micropillar l is:

[0111]

[0112] In the formula, l≥1, I min Let I be the smallest eigenvalue that can be encoded by L neural micropillars. max The maximum feature value that can be encoded by L neural micropillars is given. Each feature value of the vector falls into a neural micropillar, providing feedforward input to the neural micropillars. That is, a character can be encoded into a maximum of M neural micropillars.

[0113] S132: For the first character of the input, the feature value falling into the neural micropillar l is further converted into the pulse firing time t of the Spiking neuron in the neural micropillar using a Gaussian function:

[0114]

[0115] In the formula, x m Represents the eigenvalue, t max This represents the maximum encoding time window. b and c represent the mean and variance of the Gaussian function corresponding to the neural micropillar l, respectively. c is a constant value, set to 0.1, and b is set to:

[0116]

[0117] S133: Phase encoding is used to distinguish feature values ​​falling into the same neural micropillar. The encoding process is described using the cosine function.

[0118] f(t) = A cos(ωt + φ) j )

[0119] In the formula, f(t) represents the cosine function corresponding to the j-th dimension of the eigenvector, A represents the amplitude, ω represents the phase velocity, and φ represents the phase velocity. j The phase of the j-th feature dimension of the eigenvector is defined as:

[0120] φ j =φ0+(j-1)Δφ

[0121] In the formula, φ0 represents the initial phase, and Δφ represents the constant phase difference between adjacent feature dimensions.

[0122] Specifically, the process of the sparse temporal group coding method is as follows: Figure 3 As shown, Gaussian encoding is used to... Figure 3 Low-dimensional feature vectors in (a) Transform into Figure 3 In (b), the pulse firing time t = (t1, t2, t3, t4, t5) is... Figure 3 In (b), the pulse firing times t = (t1, t2, t3, t4, t5) are assigned different phase information according to the feature dimensions. After alignment and compression operations, each pulse time t = (t1, t2, t3, t4, t5) is forced to be generated at the nearest peak of the corresponding cosine function. After phase encoding is completed, the encoded pulse firing times t = (t1, t2, t3, t4, t5) are assigned to the corresponding receiving neural micropillars, such as... Figure 3 As shown in (c).

[0123] Furthermore, the method for defining the activation state of the Spiking neuron includes the following steps:

[0124] S21: For the first character input, all the Spiking neurons in its neural micropillars will enter an activated state;

[0125] S22: For other input characters, calculate the membrane voltage of all Spiking neurons in their neural micropillars:

[0126]

[0127] In the formula, Γ represents the membrane voltage of the j-th Spiking neuron in the neural micropillar l at time t. j I represents the distal dendrite set of the j-th Spiking neuron in neural micropillar l. l (t) represents the feedforward input current, and W represents I. l The weights of (t) are set to 1, V θ (t) represents theta oscillation, V s (t) represents the membrane voltage of the distal dendrite s of the Spiking neuron after receiving distal synaptic input:

[0128]

[0129] In the formula, w is t represents the synaptic weight between presynaptic neuron i and synaptic dendrite s. i t represents the pulse firing time of presynaptic neuron i. d This represents synaptic delay, where k represents the kernel function, expressed by factors V0 and constants τ and τ'. s Together they control the postsynaptic membrane voltage:

[0130]

[0131] In the formula, u = t - (t i +t d () represents the current time t and the delayed presynaptic pulse firing time t. i +t d The difference, V θ (t) can be described using the sine function:

[0132] V θ (t)=A θ sin(ω θ t+φ θ )

[0133] In the formula, A θ ω represents the amplitude below a threshold. θ φ represents the oscillation phase velocity. θ This represents the oscillation phase shift; each input character is learned and predicted within a single theta-cycle time T; current I l (t) can be described using a piecewise function:

[0134]

[0135] In the formula, I ext Set as a constant, and n is set as an integer greater than 1; when the membrane voltage Exceeding the ignition threshold V th The j-th Spiking neuron of the neural micropillar l releases a pulse and enters an activated state; the ignition threshold V th Set to V th =A θ +WI ext ;

[0136] The method for defining the predicted state of the Spiking neuron includes the following steps:

[0137] S31: Calculate the membrane voltage of the distal dendrite s of the Spiking neuron after receiving transverse synaptic input:

[0138]

[0139] S32: Determine V s (t) Whether the dendritic activation threshold V is exceeded within the first quarter of the cycle. ath ;

[0140] S33: Determine whether the number of presynaptic neurons activated by the distal dendrite s in the previous cycle exceeds N. a ;

[0141] S34: Distal dendrites s that simultaneously satisfy S32 and S33 are defined as active dendrites, and Spiking neurons with at least one active dendrite are defined as predictive neurons.

[0142] The method for defining the winning neuron includes the following steps:

[0143] S41: Determine whether there is a predictive neuron in the neural micropillar. If it exists, define the predictive neuron as the winning neuron. If it does not exist, proceed to step S42.

[0144] S42: Determine whether there are matching dendrites in the Spiking neurons of the neural micropillars. If there are, define the Spiking neurons with matching dendrites as winning neurons. If not, go to step S43.

[0145] S43: Define the Spiking neuron with the fewest dendritic segments as the winning neuron and initialize N. s Each synapse forms a synaptic connection with the activated neuron of the previous cycle. If the number of synaptic connections on its distal dendrite exceeds the maximum number of synapses limit Synapse_max, the neuron will create a new distal dendrite and then continue to form synaptic connections with the activated neuron of the previous cycle. The initial synaptic weight is set to 0.21.

[0146] When calculating the membrane voltage of Spiking neurons, synaptic delay and Theta oscillation mechanisms are incorporated to ensure periodic learning and prediction of text sequences.

[0147] Furthermore, the method for defining the matching dendrites is as follows:

[0148] Calculate the membrane voltage of the distal dendrite s of the Spiking neuron after receiving transverse synaptic input:

[0149]

[0150] If V s (t) exceeds the dendrite matching threshold V within the first quarter of the cycle. pth Then, the distal dendrite s is defined as the matching dendrite.

[0151] In one possible implementation, step S5 is implemented using the STDP algorithm, which includes the following steps:

[0152] S51: Calculate the pulse firing time t of neuron j to which the distal postsynaptic dendrite s belongs. j And the delayed spiking time t of the presynaptic neuron i i +t d The difference u:

[0153] u = t j -(t i +t d )

[0154] S52: Calculate the synaptic weight change Δw between neuron j, to which the postsynaptic distal dendrite s belongs, and neuron i, to which the presynaptic neuron i belongs. is :

[0155]

[0156] In the formula, A ± and τ ± All are constants greater than 0.

[0157] Furthermore, the anomaly detection and correction process in step S8 includes the following steps:

[0158] S81: For sequences In the formula, n is the length of the sequence, using M1(Y) i ) represents the character Y i The set of receiving neural micropillars, M2(Y) i ) represents the character Y i-1 The set of neural micropillars where the predicted neurons are located;

[0159] S82: Calculate character Y i abnormal score S i :

[0160]

[0161] In the formula, |·| represents the length of the set. The abnormality score ranges from [0,1]. If the current input character is abnormal, the abnormality score will be close to 1; otherwise, the abnormality score will be close to 0.

[0162] S83: Set an exception threshold to determine if a character is abnormal. If the exception score S i If the value is greater than the abnormal threshold, input the character Y. i If this is considered abnormal, proceed to step S84;

[0163] S84: Based on the predicted neural micropillar set M2(Y) i Infer the correct character and replace the abnormal character Y. i .

[0164] Based on the aforementioned bio-inspired text sequence processing method, an unsupervised learning rule based on Spiking neurons is proposed, comprising the following steps:

[0165] Step 1: Determine if a predictive neuron exists within the neural micropillar. If it exists, define the predictive neuron as the winning neuron; otherwise, proceed to Step 2.

[0166] Step 2: Determine whether there are matching dendrites in the Spiking neurons within the neural micropillars. If they do, define the Spiking neurons with matching dendrites as winning neurons; otherwise, proceed to Step 3.

[0167] Step 3: Define the Spiking neuron with the fewest dendritic segments as the winning neuron, create a new dendrite for this neuron, and then initialize N. s Each synapse forms a synaptic connection with the activated neuron of the previous cycle;

[0168] Step 4: Update the distal synaptic connection weights of the winning neuron using the STDP algorithm.

[0169] The following is a more specific embodiment for illustration.

[0170] A bio-inspired text sequence processing method includes the following steps:

[0171] S1: Represent each input character as a sparse distributed neural micropillar, and further convert the first input character into the pulse firing time of the Spiking neurons in the receiving neural micropillar;

[0172] In one possible implementation, step S1 includes the following steps:

[0173] S11: First, each input character is encoded into a 128-dimensional feature vector using the Tiny-BERT model;

[0174] S12: Use the similarity conversion method to reduce the dimension of the feature vector to 10 dimensions;

[0175] S13: The setup method contains 512 neural micropillars, each containing 32 Spiking neurons. The low-dimensional feature vector is represented as at least 10 neural micropillars using a sparse temporal group coding method. The first input character is further converted into the pulse firing time t of the corresponding Spiking neuron in the neural micropillar using Gaussian phase coding.

[0176] Specifically, such as Figure 2 As shown, the similarity conversion method in step S12 includes the following steps:

[0177] S121: For feature vectors with dimension 128 The eigenvector is obtained by performing matrix multiplication with 12 random matrices of dimension 16×128.

[0178] S122: For feature vectors Vectors are obtained by scaling dot product.

[0179] S123: Transfer vector Concatenate into a vector After matrix multiplication and normalization with a 64×10 random matrix, a low-dimensional eigenvector with a dimension of 10 is obtained.

[0180] Specifically, the sparse temporal group coding method in step S13 includes the following steps:

[0181] S131: Encoding low-dimensional feature vectors using 512 neural micropillars The eigenvalue range is [0,1], and the receptive domain of the neural micropillar l is: One character can be encoded into a maximum of 10 neural micropillars;

[0182] S132: For the first character of the input, the feature value falling into the neural micropillar l is further converted into the pulse firing time of the Spiking neuron in the neural micropillar using a Gaussian function;

[0183] S133: Phase encoding is used to distinguish feature values ​​falling into the same neural micropillar, described by the cosine function.

[0184] Thus, we transformed the 128-dimensional feature vector into a 10-dimensional low-dimensional feature vector, and then encoded the low-dimensional feature vector into at least 10 neural micropillars using a sparse temporal group coding scheme. Furthermore, we represented the first character of the input as the pulse firing time of the Spiking neuron in the neural micropillar.

[0185] S2: Calculate the activation state of the Spiking neuron in the first character receiving neural micropillar;

[0186] S3: Find the Spiking neuron that enters the prediction state for the next input character;

[0187] S4: Calculate the activation state of the Spiking neuron in the next input character receiving neural micropillar and define a unique winning neuron for that character receiving neural micropillar;

[0188] S5: Update the distal synaptic connections of the winning neuron using the STDP algorithm;

[0189] S6: Repeat steps S3 to S5 until each neural micropillar defines the activation state for the Spiking neurons it contains and selects the only winning neuron.

[0190] Furthermore, the method for defining the activation state of the Spiking neuron includes the following steps:

[0191] S21: For the first character input, all Spiking neurons in its neural micropillars will enter an activated state;

[0192] S22: For other input characters, calculate the membrane voltage of the j-th Spiking neuron in the neural micropillar l at time t. When the membrane voltage Exceeding the ignition threshold V th =2, the j-th Spiking neuron of the neural micropillar l will release a pulse and then enter the activated state. Each Spiking neuron releases at most one pulse within a single cycle time of 10 seconds.

[0193] Furthermore, the Spiking neuron prediction state definition method includes the following steps:

[0194] S31: Calculate the membrane voltage V after the distal dendrite s of the Spiking neuron receives distal synaptic input. s (t);

[0195] S32: Determine V s (t) Whether the dendritic activation threshold 1 is exceeded within the first quarter of the cycle;

[0196] S33: Determine whether the number of presynaptic neurons activated in the distal dendrite s during the previous cycle exceeds 13;

[0197] S34: Distal dendrites *s* that simultaneously satisfy steps S32 and S33 are defined as activated dendrites. Spiking neurons with at least one activated dendrite are defined as predictive neurons.

[0198] Furthermore, the method for defining the winning neuron includes the following steps:

[0199] S41: Determine whether a predictor neuron exists in the neural micropillar. If it exists, define the predictor neuron as the winning neuron; otherwise, proceed to step S42.

[0200] S42: Determine whether there is a matching dendrite in the Spiking neuron within the receiving neural micropillar. If there is, define the Spiking neuron with the matching dendrite as the winning neuron; otherwise, proceed to step S43.

[0201] S43: Define the Spiking neuron with the fewest dendritic segments as the winning neuron and initialize 32 synapses to form synaptic connections with the activated neuron of the previous cycle. If the synaptic connections on its distal dendrites exceed the maximum number of synapses of 128, the neuron will create a new distal dendrite and then continue to form synaptic connections with the activated neuron of the previous cycle. The initial synaptic weight is set to 0.21.

[0202] Specifically, the matching dendrite definition method is as follows: V s (t) Distal dendrites s that exceed the dendrite matching threshold of 0.5 within the first quarter of the cycle are defined as matched dendrites.

[0203] At this point, each neural micropillar has defined the activation state of the neurons it contains and selected a unique winning neuron.

[0204] S7: Sequence retrieval: Based on the sparsely distributed neural micropillars of characters, the next possible character is inferred by using the neural micropillar where the neuron in the predicted state is located;

[0205] S8: Sequence Recovery: Utilizes both received neural micropillars and predicted neural micropillars to detect and correct anomalies in damaged sequences.

[0206] Furthermore, step S8 includes the following steps:

[0207] The anomaly score S is calculated based on both received neural micropillars and predicted neural micropillars. i If the abnormal score S i If the value is greater than 0.5, the character is abnormal and the damaged sequence needs to be restored to its original pattern by predicting neural micropillars.

[0208] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A biologically inspired text sequence processing method, characterized in that, Includes the following steps: S1: Represent each input character as a sparse distributed neural micropillar, and further convert the first input character into the pulse firing time of the Spiking neurons in the receiving neural micropillar; S2: Calculate the activation state of the Spiking neuron in the first character receiving neural micropillar; S3: Find the Spiking neuron that enters the prediction state for the next input character; S4: Calculate the activation state of the Spiking neuron in the next input character receiving neural micropillar and define a unique winning neuron for that character receiving neural micropillar; S5: Update the distal synaptic connections of the winning neuron; S6: Repeat steps S3 to S5 until each neural micropillar defines the activation state for the Spiking neurons it contains and selects the only winning neuron. S7: Sequence retrieval: Based on the sparsely distributed neural micropillars of characters, the next possible character is inferred by using the neural micropillar where the neuron in the predicted state is located; S8: Sequence recovery: using both received neural micropillars and predicted neural micropillars to detect and correct anomalies in damaged sequences; in, The method for defining the predicted state of the Spiking neuron includes the following steps: S31: Calculate the membrane voltage of the distal dendrite s of the Spiking neuron after receiving distal synaptic input: S32: Determine V s (t) Whether the dendritic activation threshold V is exceeded in the first quarter of the cycle. ath ; S33: Determine whether the number of presynaptic neurons activated by the distal dendrite s in the previous cycle exceeds N. a ; S34: Distal dendrites s that simultaneously satisfy S32 and S33 are defined as active dendrites, and Spiking neurons with at least one active dendrite are defined as predictive neurons. The method for defining the winning neuron includes the following steps: S41: Determine whether there is a predictive neuron in the neural micropillar. If it exists, define the predictive neuron as the winning neuron. If it does not exist, proceed to step S42. S42: Determine whether there are matching dendrites in the Spiking neurons of the neural micropillars. If there are, define the Spiking neurons with matching dendrites as winning neurons. If not, go to step S43. S43: Define the Spiking neuron with the fewest dendritic segments as the winning neuron and initialize N. s Each synapse forms a synaptic connection with the activated neuron of the previous cycle. If the number of synaptic connections on its distal dendrite exceeds the maximum number of synapses limit Synapse_max, the neuron will create a new distal dendrite and then continue to form synaptic connections with the activated neuron of the previous cycle. The initial synaptic weight is set to 0.

21. The method for defining the matching dendrites is as follows: Calculate the membrane voltage of the distal dendrite s of the Spiking neuron after receiving distal synaptic input: If V s (t) exceeds the dendrite matching threshold V within the first quarter of the cycle. pth Then, the distal dendrite s is defined as the matching dendrite.

2. The bio-inspired text sequence processing method according to claim 1, characterized in that, Step S1 is implemented using a sparse time group coding scheme, which includes the following steps: S11: Encode each input character into a feature vector using the BERT model; S12: Reduce the dimensionality of the feature vector; S13: Represent the low-dimensional feature vector as sparse distributed neural micropillars, and further convert the first character of the input into the pulse firing time of the Spiking neuron in the corresponding neural micropillar through Gaussian phase encoding.

3. The bio-inspired text sequence processing method according to claim 2, characterized in that, Step S12 is implemented using a similarity conversion method, which includes the following steps: S121: For feature vectors of dimension N Perform matrix multiplication with 3H random matrices of dimension C×N to obtain the eigenvectors. In the formula, V hi Represents a random matrix; S122: For feature vectors Vectors are obtained by scaling dot product. In the formula, · represents the dot product of vectors, and C is a constant; S123: Concatenate the vector to form a vector Then, through matrix multiplication with the random matrix W ′ and normalization processing, obtain a low-dimensional feature vector with dimension M, where M < N Wherein, the random matrix W hi Both W and W′ are sampled from a standard normal distribution.

4. The bio-inspired text sequence processing method according to claim 2, characterized in that, Step S13 is implemented using a sparse time group coding method, which includes the following steps: S131: Low-dimensional feature vectors for character encoding using L neural micropillars: targeting the feature value range [I min ,I max The receptive domain of the neural micropillar l is: In the formula, l≥1, I min Let I be the smallest eigenvalue that can be encoded by L neural micropillars. max The maximum feature value that can be encoded by L neural micropillars is given. Each feature value of the vector falls into a neural micropillar, providing feedforward input to the neural micropillars. That is, a character can be encoded into a maximum of M neural micropillars. S132: For the first character of the input, the feature value falling into the neural micropillar l is further converted into the pulse firing time t of the Spiking neuron in the neural micropillar using a Gaussian function: In the formula, x m t represents the eigenvalue. max This represents the maximum encoding time window. b and c represent the mean and variance of the Gaussian function corresponding to the neural micropillar l, respectively. c is a constant value, set to 0.1, and b is set to: S133: Phase encoding is used to distinguish feature values ​​falling into the same neural micropillar. The encoding process is described using the cosine function. f(t)=Acos(ωt+φ j ) In the formula, f(t) represents the cosine function corresponding to the j-th dimension of the eigenvector, A represents the amplitude, ω represents the phase velocity, and φ represents the phase velocity. j The phase of the j-th feature dimension of the eigenvector is defined as: f j =φ0+(j-1)Δφ In the formula, φ0 represents the initial phase, and Δφ represents the constant phase difference between adjacent feature dimensions.

5. The bio-inspired text sequence processing method according to claim 1, characterized in that, The method for defining the activation state of Spiking neurons includes the following steps: S21: For the first character input, all the Spiking neurons in its neural micropillars will enter an activated state; S22: For other input characters, calculate the membrane voltage of all Spiking neurons in their neural micropillars: In the formula, Γ represents the membrane voltage of the j-th Spiking neuron in the neural micropillar l at time t. j I represents the distal dendrite set of the j-th Spiking neuron in neural micropillar l. l (t) represents the feedforward input current, and W represents I. l The weights of (t) are set to 1, V θ (t) represents theta oscillation, V s (t) represents the membrane voltage of the distal dendrite s of the Spiking neuron after receiving distal synaptic input: In the formula, w is t represents the synaptic weight between presynaptic neuron i and synaptic dendrite s. i t represents the pulse firing time of presynaptic neuron i. d This represents synaptic delay, where k represents the kernel function, expressed by factors V0 and constants τ and τ'. s Together they control the postsynaptic membrane voltage: In the formula, u = t - (t i +t d () represents the current time t and the delayed presynaptic pulse firing time t. i +t d The difference, V θ (t) can be described using the sine function: V θ (t)=A θ sin(ω θ t+φ θ ) In the formula, A θ ω represents the amplitude below a threshold. θ φ represents the oscillation phase velocity. θ This represents the oscillation phase shift; each input character is learned and predicted within a single theta-cycle time T; current I l (t) can be described using a piecewise function: In the formula, I ext Set as a constant, and n is set as an integer greater than 1; when the membrane voltage Exceeding the ignition threshold V th The j-th Spiking neuron in the neural micropillar l will release a pulse and enter an activated state; the ignition threshold V th Set to V th =A θ +WI ext .

6. The bio-inspired text sequence processing method according to claim 1, characterized in that, Step S5 is implemented using the STDP algorithm, which includes the following steps: S51: Calculate the pulse firing time t of neuron j to which the distal postsynaptic dendrite s belongs. j And the delayed spiking time t of the presynaptic neuron i i +t d The difference u: u=t j -(t i +t d ) S52: Calculate the synaptic weight change Δw between neuron j, to which the postsynaptic distal dendrite s belongs, and neuron i, to which the presynaptic neuron i belongs. is : In the formula, A ± and τ ± All are constants greater than 0.

7. The bio-inspired text sequence processing method according to claim 1, characterized in that, The anomaly detection and correction process in step S8 includes the following steps: S81: For sequences In the formula, n is the length of the sequence, using M1(Y) i ) represents the character Y i The set of receiving neural micropillars, M2(Y) i ) represents the character Y i-1 The set of neural micropillars where the predicted neurons are located; S82: Calculate character Y i abnormal score S i : In the formula, |·| represents the length of the set; the abnormal score ranges from [0,1]. If the current input character is abnormal, the abnormal score will be close to 1, and vice versa. S83: Set an exception threshold to determine if a character is abnormal. If the exception score S i If the value is greater than the abnormal threshold, input the character Y. i If this is considered abnormal, proceed to step S84; S84: Based on the predicted neural micropillar set M2(Y) i Infer the correct character and replace the abnormal character Y. i .

Images