A working memory reorganization system and method based on brain-like cognitive computing

Abstract

The application discloses a working memory reorganization system and method based on brain-like cognitive computing, relates to the technical field of working memory reorganization, and comprises the following steps: analyzing the synaptic connection degree between different neurons in a pulse signal sequence; analyzing the similarity between the retrieved multi-modal data and the candidate historical memories in the working memory library of the platform; performing synaptic connection smooth cutting on the neurons to be cut off, generating a clustering cluster, analyzing the synaptic connection increase condition between different neurons in the clustering cluster, and updating and reorganizing the global connection weight matrix in the working memory library; acquiring current retrieval multi-modal data at the current moment, performing brain-like retrieval in the platform, obtaining retrieval memory data, and performing sending, which fundamentally coordinates the conflict between plasticity and stability, overcomes the catastrophic forgetting problem in continuous learning, and realizes efficient dynamic reorganization and accurate retrieval of working memory.

Description

Technical Field

[0001] This invention relates to the field of working memory reorganization technology, specifically a working memory reorganization system and method based on brain-like cognitive computing. Background Technology

[0002] In the fields of cognitive science and brain-inspired computing, working memory is widely regarded as the core cognitive function of the biological brain to achieve temporary information storage and online processing. Its interaction with long-term memory forms the basis of advanced human cognitive activities. In recent years, working memory reconstruction methods based on brain-inspired cognitive computing have gradually become a research hotspot in the field of artificial intelligence. Compared with traditional fixed memory storage methods, this method is more flexible and adaptive, and can realize real-time maintenance and reconstruction of working memory during information processing.

[0003] Working memory reorganization systems based on neuromorphic cognitive computing need to effectively retain previously consolidated memory content while continuously receiving new information and forming new neural clusters to support subsequent reasoning and decision-making. However, when new information is input and triggers working memory reorganization, the synaptic connection patterns of the original neural clusters are partially reconstructed. That is, highly relevant connections are enhanced, while low-relevance connections are weakened or severed. This leads to a problem: during this dynamic pruning and reorganization process at the local connection level, interference may continue to accumulate over time. That is, after multiple reorganization iterations, the connection weights corresponding to old memories that have not been activated recently or have low relevance to new inputs will be gradually weakened or even completely eliminated. This results in the old knowledge representation being covered or destroyed by new knowledge, leading to catastrophic forgetting in the process of neuromorphic cognitive computing. Summary of the Invention

[0004] The purpose of this invention is to provide a working memory reorganization system and method based on brain-like cognitive computing to solve the problems raised in the prior art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a working memory reorganization method based on brain-like cognitive computing, the method comprising: Step S1: Obtain the multimodal data from the platform, construct a pulse signal sequence, analyze the degree of synaptic connection between different neurons in the pulse signal sequence, and obtain the retrieval pulse data; Step S2: Analyze the similarity between the retrieved multimodal data and the candidate historical memories in the platform's working memory bank to obtain a set of historically relevant records; Step S3: determining the historical memory to be cut off in the working memory bank, performing smooth cutting of synaptic connections on neurons of the historical memory to be cut off, generating clustering clusters, analyzing the increase status of synaptic connections between different neurons in the clustering clusters, updating and reorganizing the global connection weight matrix in the working memory bank, and obtaining an updated global connection matrix; Step S4: acquiring current retrieval multimodal data at the current moment, performing brain-like retrieval in the platform to obtain retrieved memory data, and sending the retrieved memory data.

[0006] Further, step S3 comprises: acquiring a historical correlation record set of retrieved multimodal data in a current cycle, and writing pulse signal sequences of the retrieved modal data into the working memory bank in the platform; calculating a composite retrieval score R of the c-th historical memory in the working memory bank according to the historical correlation record set c , calculating an average value μ and a standard deviation σ of composite retrieval scores of each historical memory in the working memory bank, and calculating a memory saliency value G of the c-th historical memory in the working memory bank c , setting a composite retrieval score threshold R´ and a memory saliency threshold G´, when R c <R´ or G c <G´, marking the c-th historical memory as the historical memory to be cut off; otherwise, not processing the c-th historical memory; acquiring a synaptic connection matrix W of the historical memory to be cut off from the working memory bank, performing smooth cutting of neural connections on the synaptic connection matrix W, and obtaining an attenuated synaptic connection matrix; acquiring the d-th historical memory that is not the historical memory to be cut off in the current cycle, acquiring a plurality of related historical memories of the d-th historical memory and collecting them to obtain a similar historical memory set, clustering the similar historical memory set by using a clustering algorithm to obtain clustering clusters in the similar historical memory set, and constructing an increased synaptic connection matrix of the clustering clusters; acquiring a global connection weight matrix W´ in the working memory bank, replacing the synaptic connection matrix of the historical memory to be cut off in the global connection weight matrix W´ of the current cycle by using the attenuated synaptic connection matrix of the historical memory to be cut off, and superposing the increased synaptic connection matrix of the clustering clusters into the global connection weight matrix W´ to obtain an updated global connection matrix W in the current cycle △ ; in the above steps, active forgetting is performed on historical memories with low retrieval calls and low relevance in the working memory bank, and fusion reorganization is performed on historical memories with high retrieval calls and high relevance to form clustering clusters, which breaks the serial processing mode of learning first then forgetting or protection first then learning in the prior art, and fundamentally eliminates the sequential decision dilemma of when to forget and when to learn.

[0007] Furthermore, step S2 includes: The pulse signal sequence of the multimodal data in the spiking neuron model is obtained from the retrieved pulse data, and the working memory bank is obtained from the platform to obtain the candidate historical memory of the pulse signal sequence in the current cycle. Calculate the exponentially decaying kernel between the candidate historical memory and the retrieved multimodal data, obtain each neuron of the pulse signal sequence, calculate the temporal similarity between the historical pulse signal sequence of the candidate historical memory and the pulse signal sequence of the retrieved multimodal data at each neuron dimension, and calculate the original similarity score Q between the historical pulse signal sequence of the candidate historical memory and the pulse signal sequence of the retrieved multimodal data. sum ; Calculate the self-similarity score Q´ of the historical pulse signal sequence of candidate historical memories, and calculate the self-similarity score Q' of the pulse signal sequence for retrieving multimodal data. △ Calculate the feature similarity score P between the historical pulse signal sequence of candidate historical memory and the pulse signal sequence of retrieved multimodal data; Calculate the feature similarity score between the historical pulse signal sequence of each candidate historical memory and the pulse signal sequence of the retrieved multimodal data. Set a feature similarity threshold P'. When the feature similarity score between the historical pulse signal sequence of a candidate historical memory and the pulse signal sequence of the retrieved multimodal data is greater than the feature similarity threshold P', then the candidate historical memory is recorded as the relevant historical memory for the retrieved multimodal data. Retrieve all relevant historical memories of the retrieved multimodal data, increment the retrieval count of each relevant historical memory by 1, update the retrieval time of each relevant historical memory to the time point when the retrieved multimodal data was entered into the platform in the current period, and aggregate the retrieval count and retrieval time of each relevant historical memory of the retrieved multimodal data to obtain a set of historical relevant records of the retrieved multimodal data.

[0008] Furthermore, step S1 includes: The system retrieves the multimodal data to be processed within the current period from the platform, preprocesses the multimodal data, extracts and fuses features from the multimodal data, and obtains the multimodal fusion feature vector of the multimodal data. Set the encoding time window T=[t start ,t end Given a time step t', each component in the multimodal fusion feature vector is mapped to a neuron. The input current of each neuron in the predefined spiking neuron model is calculated, where t' is the input current of the multimodal fusion feature vector. start and t end These are the encoding start time and encoding end time, respectively; The encoding time window T is divided according to the time step t´ to obtain each time step within the encoding time window T. The membrane potential of each neuron at each time step within the encoding time window T is calculated iteratively. Set the membrane potential threshold V th Set a pulse signal variable S. When the membrane potential of a neuron at a certain time step is less than the membrane potential threshold V... th If the pulse signal variable of a neuron at a certain time step is set to 1, then the pulse signal variable of a neuron at a certain time step is set to 0. The pulse signal variables of each neuron at each time step are obtained and aggregated to obtain the pulse signal sequence of the retrieved multimodal data in the spiking neuron model. Calculate the synaptic connection weights between one neuron and another neuron at a certain time step, and construct the synaptic connection matrix of the pulse signal sequence for retrieving multimodal data; The retrieved multimodal data is obtained by combining the pulse signal sequences and synaptic connection matrices in the spiking neuron model.

[0009] Furthermore, step S4 includes: Obtain the pulse signal sequence of the currently retrieved multimodal data at the current time within the current period, and update the global connectivity matrix W within the current period. △ ; Calculate the feature similarity score between the historical working memory in the working memory bank and the pulse signal sequence of the currently retrieved multimodal data, obtain a relevant historical memory of the currently retrieved multimodal data, and update the global connectivity matrix W from the current period. △ The sum of synaptic connection weights H between neurons of a specific historical memory is obtained. sum ; Set the connection accumulation threshold H', when H' sum When >H´, a certain related historical memory is recorded as the target retrieval historical memory for the current multimodal data retrieval; otherwise, no related historical memory is processed. The system retrieves and aggregates the historical retrieval data of each target in the working memory of the current multimodal data retrieval, obtaining the retrieval memory data of the current retrieval modality, and then sends the retrieval memory data to the retrieval personnel of the current multimodal data retrieval through the platform.

[0010] To better implement the above method, a working memory reorganization system is also proposed, which includes a synaptic connection analysis module, a similarity analysis module, a reorganization update module, and a memory retrieval module. The synaptic connection analysis module is used to acquire multimodal data retrieved from the platform, construct pulse signal sequences, analyze the degree of synaptic connection between different neurons in the pulse signal sequences, and obtain retrieved pulse data. The similarity analysis module is used to analyze the similarity between the retrieved multimodal data and the candidate historical memories in the platform's working memory bank to obtain a set of historically relevant records; The reorganization and update module is used to update and reorganize the global connection weight matrix in the working memory to obtain the updated global connection matrix. The memory retrieval module is used to perform brain-like retrieval on the current multimodal data in the platform to obtain retrieval memory data, and then send the retrieval memory data through the platform.

[0011] Furthermore, the synaptic connection analysis module includes a signal sequence construction unit and a synaptic connection analysis unit; The signal sequence construction unit is used to acquire and retrieve multimodal data from the platform and construct the pulse signal sequence of the retrieved multimodal data; The synaptic connection analysis unit is used to analyze the degree of synaptic connection between different neurons in a pulse signal sequence to obtain retrieved pulse data.

[0012] Furthermore, the similarity analysis module includes candidate historical memory determination units and similarity analysis units; The candidate historical memory determination unit is used to obtain the pulse signal sequence of the retrieval multimodal data in the spiking neuron model from the retrieval pulse data, and determine the candidate historical memory of the pulse signal sequence from the working memory bank; The similarity analysis unit is used to analyze the similarity between the retrieved multimodal data and the candidate historical memories in the working memory bank of the platform, determine the relevant historical memories of the retrieved multimodal data, and obtain the historical relevant record set of the retrieved multimodal data.

[0013] Furthermore, the recombination and update module includes a memory clustering unit and a recombination and update unit; The memory clustering unit is used to identify the historical memories to be cut in the working memory bank, smoothly sever the synaptic connections of the neurons of the historical memories to be cut, and generate clusters. The reorganization and update unit is used to analyze the increase in synaptic connections between different neurons in the cluster, and to update and reorganize the global connection weight matrix in the working memory to obtain the updated global connection matrix.

[0014] Furthermore, the memory retrieval module includes a memory retrieval unit; The memory retrieval unit is used to acquire the current retrieval multimodal data at the current moment, and combine it with the updated global connection matrix to perform brain-like retrieval on the platform to obtain retrieval memory data. The retrieval memory data is then sent to the retrieval personnel of the current retrieval multimodal data through the platform.

[0015] Compared with existing technologies, the beneficial effects of this invention are as follows: converting the input multimodal data into a pulse signal sequence eliminates the barrier of heterogeneity between different simulated data. Furthermore, by analyzing the temporal similarity between the retrieved multimodal data and historical memory in the current cycle, the number of retrieval calls and retrieval time of historical memory are updated, further activating neurons in historical memory. Moreover, by analyzing the retrieval call situation of historical memory, neurons detecting retrieved historical memory are strengthened in the global connection weight matrix, while the connections of neurons in historical memory with fewer retrieval calls are weakened. This allows historical memory in the working memory bank to be truly updated and reorganized. Through the system's inherent connection optimization and multi-pathway retrieval synergy, the conflict between plasticity and stability is fundamentally coordinated, overcoming the catastrophic forgetting problem in continuous learning, and achieving efficient dynamic reorganization and accurate retrieval of working memory. Attached Figure Description

[0016] Figure 1 This is a flowchart of a working memory reorganization method based on brain-like cognitive computing according to the present invention; Figure 2 This is a flowchart of a working memory reorganization system according to the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Example: Figures 1-2 As shown, the present invention provides a technical solution, a working memory reorganization method based on brain-like cognitive computing, the method comprising: Step S1: Obtain the multimodal data from the platform, construct a pulse signal sequence, analyze the degree of synaptic connection between different neurons in the pulse signal sequence, and obtain the retrieval pulse data; Step S1 includes: The system retrieves the multimodal data to be processed within the current period from the platform, preprocesses the multimodal data, extracts and fuses features from the multimodal data, and obtains the multimodal fusion feature vector of the multimodal data. For example, retrieving various multimodal data in multimodal data includes any one or more combinations of data such as image data, text data, voice data, and sensor time-series data; For example, preprocessing of retrieved multimodal data includes the following specific preprocessing steps: The image data preprocessing process in the retrieved multimodal data is as follows: the image data is normalized in size and pixel value, and the image is augmented; The preprocessing of text data in the retrieved multimodal data is as follows: the text data is segmented into words, stop words are removed, and digital encoding is performed. The preprocessing process for speech data in the retrieved multimodal data is as follows: pre-emphasis is applied to the input speech signal, followed by framing, windowing, and short-time Fourier transform processing to extract the corresponding acoustic features; The preprocessing process for sensor time-series data in the retrieved multimodal data is as follows: filtering and denoising are performed on the time-series data acquired by the sensors; For example, feature extraction and feature fusion are performed on retrieved multimodal data to obtain a multimodal fused feature vector of the retrieved multimodal data. The specific process is as follows: A pre-trained ResNet model is used as an image feature extractor, taking pre-processed image data as input to extract feature vectors from the image data; Using a pre-trained BERT model as a text feature extractor, the pre-processed text data is taken as input to extract the feature vector of the text data; The pre-trained wav2vec2.0 model is used as the speech feature extractor. The pre-processed speech data is taken as input and the feature vector of the speech data is extracted. The pre-trained InceptionTime model is used as the sensor feature extractor. The pre-processed sensor time-series data is taken as input to extract the feature vector of the sensor time-series data. The dimensions of each multimodal data point in the retrieved multimodal data are unified, and the data are aggregated in a preset order to obtain the multimodal fusion feature vector of the retrieved multimodal data. Set the encoding time window T=[t start ,t end Given a time step t', each component in the multimodal fusion feature vector is mapped to a neuron. The input current of each neuron in the predefined spiking neuron model is calculated, where t' is the input current of the multimodal fusion feature vector. start and t end These are the encoding start time and encoding end time, respectively; The encoding time window T is divided according to the time step t´ to obtain each time step within the encoding time window T. The membrane potential of each neuron at each time step within the encoding time window T is calculated iteratively. Set the membrane potential threshold V thSet a pulse signal variable S. When the membrane potential of a neuron at a certain time step is less than the membrane potential threshold V... th If the pulse signal variable of a neuron at a certain time step is set to 1, then the pulse signal variable of a neuron at a certain time step is set to 0. The pulse signal variables of each neuron at each time step are obtained and aggregated to obtain the pulse signal sequence of the retrieved multimodal data in the spiking neuron model. For example, the time step t' is 1ms in a specific embodiment; For example, each component in the multimodal fusion feature vector is mapped to a neuron. Specifically, each component in the multimodal fusion feature vector is obtained, and each component is used as a neuron in a preset spiking neuron model as the multimodal fusion feature vector. Among them, the components in the multimodal fusion feature vector are: the latent semantic feature dimension in the multimodal fusion feature vector, which in a specific embodiment is represented as the joint features of dynamic texture and rapid tone, etc. For example, calculating the input current I of the a-th neuron in a predefined spiking neuron model using a multimodal fusion feature vector. a (t), the specific process is as follows: Calculate the input current I of the a-th neuron in the predefined spiking neuron model using the multimodal fusion feature vector. a (t)=g·F a ·1 [ t start, t end] (t), where g represents the preset current gain coefficient, F a Represents the a-th component in the modality fusion feature vector, 1 [ t start, t end] (t) represents the input current I. a (t) is valid within the encoding time window T, and the input current I... a (t) is 0 if the time t is outside the encoding time window T; For example, the encoding time window T is divided according to the time step t´ to obtain each time step within the encoding time window T. The distance between each time step within the encoding time window T is the time step t´. For example, the preset spiking neuron model is a LIF neuron model in a specific embodiment; For example, the membrane potential of each neuron at each time step within the encoding time window T is calculated iteratively. The specific calculation process is as follows: The a-th component in the multimodal fusion feature vector is mapped to the input current I of the neuron. a(t), calculate the membrane potential V of the a-th neuron at the next time step t´ at time t. a (t+t´): , Among them, V rest R is the preset resting point; R is the preset membrane resistance; V a (t) represents the membrane potential of the a-th neuron at time t; τ is a preset membrane time constant; In a specific embodiment, V rest The value is -80mV, the membrane resistance R is 1MΩ, and the membrane time constant τ is 10ms; For example, when the pulse signal variable S takes a value of 1, it means that the neuron has fired a pulse signal within the time step; when the pulse signal variable S takes a value of 0, it means that the neuron has not fired a pulse signal within the time step. Calculate the synaptic connection weights between one neuron and another neuron at a certain time step, and construct the synaptic connection matrix of the pulse signal sequence for retrieving multimodal data; For example, to calculate the synaptic connection weight w = η × S × S between one neuron and another neuron at a certain time step. △ Where η is the preset initial learning rate, and S △ This represents the pulse signal variable of another neuron at a certain time step. For example, in a specific embodiment, the initial learning rate η is 0.01; For example, the synaptic connection matrix is ​​constructed as follows: The synaptic connection weights between neurons in the spiking neuron model at each time step of the retrieved multimodal data are obtained and aggregated according to the time order of the time steps and the order of the neurons to obtain the synaptic connection matrix of the spiking signal sequence of the retrieved multimodal data. The retrieved multimodal data is obtained by combining the pulse signal sequences and synaptic connection matrices in the spiking neuron model.

[0019] Step S2: Analyze the similarity between the retrieved multimodal data and the candidate historical memories in the platform's working memory bank to obtain a set of historically relevant records; Step S2 includes: The pulse signal sequence of the multimodal data in the spiking neuron model is obtained from the retrieved pulse data, and the working memory bank is obtained from the platform to obtain the candidate historical memory of the pulse signal sequence in the current cycle. For example, the working memory bank contains all the historical memories that have already been stored in the platform; For example, the specific process for obtaining candidate historical memories of pulse signal sequences within the current period is as follows: Retrieve the time steps and total number of neurons in the historical pulse signal sequence of historical memory within the historical encoding time window from the working memory bank; Obtain the time step and total number of neurons in the encoding time window of the pulse signal sequence in the spiking neuron model for the retrieved multimodal data. When the time step and total number of neurons in the historical pulse signal sequence of a certain historical memory in the working memory bank are the same as the time step and total number of neurons in the pulse signal sequence in the current period, then the certain historical memory is recorded as a candidate historical memory of the pulse signal sequence in the current period. Calculate the exponentially decaying kernel between the candidate historical memory and the retrieved multimodal data, obtain each neuron of the pulse signal sequence, calculate the temporal similarity between the historical pulse signal sequence of the candidate historical memory and the pulse signal sequence of the retrieved multimodal data at each neuron dimension, and calculate the original similarity score Q between the historical pulse signal sequence of the candidate historical memory and the pulse signal sequence of the retrieved multimodal data. sum ; For example, the exponential decay kernel between candidate historical memories and retrieved multimodal data is calculated as follows: Obtain the time difference Δt between the historical pulse signal sequence of the candidate historical memory and the pulse signal sequence of the retrieved multimodal data, where the pulse signal of the same neuron fires at the same time step. Calculate the exponentially decaying kernel K(Δt) = exp(-|Δt| / t) between the historical pulse signal sequence of the candidate historical memory and the pulse signal sequence of the retrieved multimodal data, where the pulse signal of the same neuron fires at the same time step. ker ), where t ker This is a preset time constant used to control the effective range of timing dependencies; in a specific embodiment, it is set to 10ms. For example, calculating the temporal similarity Q between the historical pulse signal sequence of candidate historical memories and the pulse signal sequence of retrieved multimodal data at the b-th neuron dimension. b The specific calculation process is as follows: The pulse signal variables of the b-th neuron at each time step are obtained from the historical pulse signal sequence of the candidate historical memory, and are aggregated in chronological order to obtain a one-dimensional pulse time sequence of the b-th neuron obtained from the historical pulse signal sequence of the candidate historical memory. Obtain a one-dimensional pulse time series of the pulse signal sequence for retrieving multimodal data, and calculate the temporal similarity Q between the historical pulse signal sequence of the candidate historical memory and the pulse signal sequence for retrieving multimodal data at the b-th neuron dimension. b : , Where M is the total number of time steps in the one-dimensional pulse time sequence of the pulse signal sequence used to retrieve multimodal data; S b (β) represents the pulse signal variable of the b-th neuron at the β-th time step in a one-dimensional pulse time sequence of the pulse signal sequence; S´ b (α) represents the pulse signal variable of the b-th neuron in the one-dimensional pulse time series of the historical pulse signal sequence at the α-th time step; K(β-α) represents the exponentially decaying kernel between the b-th neuron in the one-dimensional pulse time series of the pulse signal sequence and the b-th neuron in the one-dimensional pulse time series of the historical pulse signal sequence at the α-th time step and the β-th time step. For example, the original similarity score Q between the historical pulse signal sequence of candidate historical memories and the pulse signal sequence for retrieving multimodal data. sum The specific calculation process is as follows: The sum of temporal similarities between the historical pulse signal sequences of candidate historical memories and the pulse signal sequences of retrieved multimodal data at each neuron dimension is calculated to obtain the original similarity score between the historical pulse signal sequences of candidate historical memories and the pulse signal sequences of retrieved multimodal data. Calculate the self-similarity score Q´ of the historical pulse signal sequence of candidate historical memories, and calculate the self-similarity score Q' of the pulse signal sequence for retrieving multimodal data. △ Calculate the feature similarity score P between the historical pulse signal sequence of candidate historical memory and the pulse signal sequence of retrieved multimodal data; For example, the self-similarity score of the historical pulse signal sequence of candidate historical memory is calculated as follows: Calculate the sum of temporal similarities between the historical pulse signal sequences of candidate historical memories at each neuron dimension, and record it as the self-similarity score of the historical pulse signal sequence of candidate historical memories; For example, the self-similarity score of a pulse signal sequence for retrieving multimodal data is calculated as follows: The sum of temporal similarities between the pulse signal sequence for retrieving multimodal data at each neuron dimension and the pulse signal sequence for retrieving multimodal data is recorded as the self-similarity score of the pulse signal sequence for retrieving multimodal data. For example, the feature similarity score P between the historical pulse signal sequence of candidate historical memory and the pulse signal sequence of retrieved multimodal data is specifically calculated using the formula: P = Q sum / (Q´×Q △ ) 1 / 2 ; Calculating a feature similarity score between a historical pulse signal sequence of each candidate historical memory and a pulse signal sequence of retrieved multimodal data, setting a feature similarity threshold P´, when the feature similarity score between the historical pulse signal sequence of a candidate historical memory and the pulse signal sequence of the retrieved multimodal data is greater than the feature similarity threshold P´, marking the candidate historical memory as a relevant historical memory of the retrieved multimodal data; Acquiring each relevant historical memory of the retrieved multimodal data, increasing the number of retrieval calls of each relevant historical memory by 1, updating the retrieval time of each relevant historical memory to the time point when the retrieved multimodal data of the current cycle is input into the platform, and aggregating the number of retrieval calls and retrieval time of each relevant historical memory of the retrieved multimodal data to obtain a historical correlation record set of the retrieved multimodal data.

[0020] Step S3: determining a to-be-cut historical memory in a working memory library, smoothly performing synaptic disconnection on neurons of the to-be-cut historical memory, generating a clustering cluster, analyzing an increase condition of synaptic connections between different neurons in the clustering cluster, updating and reorganizing a global connection weight matrix in the working memory library, to obtain an updated global connection matrix; wherein step S3 comprises: Acquiring a historical correlation record set of the retrieved multimodal data in the current cycle, and writing the pulse signal sequence of the retrieved modal data into the working memory library in the platform; for example, writing the pulse signal sequence of the retrieved modal data into the working memory library in the platform, the specific process is: writing the pulse signal sequence of the retrieved modal data as a piece of memory written by the platform in the current cycle into the working memory library in the platform, acquiring the input time point of the pulse signal sequence of the retrieved modal data as the retrieval time of the pulse signal sequence, and initializing the number of retrieval calls of the pulse signal sequence of the retrieved modal data to 0; According to the historical correlation record set, calculating a composite retrieval score R of the c-th historical memory in the working memory library c , calculating the mean value μ and standard deviation σ of composite retrieval scores of each historical memory in the working memory library, and calculating a memory significance value G of the c-th historical memory in the working memory library c , setting a composite retrieval score threshold R´ and a memory significance threshold G´, when R c < R´ or G c < G´, marking the c-th historical memory as a to-be-cut historical memory, otherwise, not processing the c-th historical memory; for example, for the composite retrieval score R of the c-th historical memory in the working memory library, the specific calculation process is: When the c-th historical memory is a candidate c-th historical memory among several retrieval multimodal data in the current period, the average value of the feature similarity scores between the historical pulse signal sequence of the c-th historical memory and the pulse signal sequences of several retrieval multimodal data is obtained from the historical related record set and recorded as the retrieval similarity score of the c-th historical memory in the current period. The retrieval call count of the c-th historical memory is obtained from the platform. The retrieval similarity score and retrieval call count of the c-th historical memory within the current period are then normalized to their maximum and minimum values ​​to obtain the normalized retrieval call count N of the c-th historical memory. c Similarity score E for retrieval c ; Calculate the composite retrieval score R of the c-th historical memory in the working memory bank. c =λ N ×N c +λ E ×E c , λ N λ represents the preset search similarity weight. E λ represents the preset retrieval call weight. N +λ E =1, λ N and λ E All are greater than 0; Obtain the synaptic connection matrix W of the historical memory to be severed from the working memory bank, and perform smooth severing of neural connections on the synaptic connection matrix W to obtain the decayed synaptic connection matrix; For example, performing smooth transection of neural connections on the synaptic connection matrix W yields a decayed synaptic connection matrix. The specific process is as follows: The synaptic connection weight of the element in the x-th row and y-th column of the synaptic connection matrix W in the current period is multiplied by a preset severing attenuation coefficient λ to obtain the synaptic connection weight of the element in the x-th row and y-th column after smooth severing of neural connections, where λ>0. The synaptic connection weights of each row and column of the elements in the smoothed transection of neural connections are obtained and aggregated to obtain the decayed synaptic connection matrix. Get the d-th historical memory that is not to be severed in the current period, get several related historical memories of the d-th historical memory and aggregate them to obtain a set of similar historical memories, use a clustering algorithm to cluster the set of similar historical memories to obtain clusters in the set of similar historical memories, and construct the synaptic connection matrix of the clusters. For example, the clustering algorithm is a density-based clustering algorithm, specifically the DBSCAN clustering algorithm; For example, clustering algorithms are used to cluster similar historical memory sets to obtain clusters within the similar historical memory sets. The specific process is as follows: Obtain the temporal similarity Q between pairs of neurons from several related historical memories in a similar historical memory set. △ And calculate the similarity distance L=1-Q between each pair of neurons. △ Using similarity distance as a distance metric, neurons in several related historical memories are automatically grouped to obtain clusters of similar historical memories. For example, the process of constructing the synaptic connection matrix for clustering is as follows: Multiply the preset intensity coefficient by the pulse signal variable between each pair of neurons in the cluster, and then multiply by the preset initial learning rate η to obtain the synaptic connection weights between each pair of neurons in the cluster. Obtain the synaptic connection weights between each pair of neurons in the cluster and aggregate them in a preset order to obtain the cluster's increased synaptic connection matrix. Obtain the global connectivity weight matrix W' from the working memory. Use the decaying synaptic connection matrix of the historical memory to be severed to replace the synaptic connection matrix of the historical memory to be severed in the global connectivity weight matrix W' for the current period. Then, add the increasing synaptic connection matrix of the cluster to the global connectivity weight matrix W' to obtain the updated global connectivity matrix W' for the current period. △ ; For example, the global connection weight matrix W' is as follows: The matrix is ​​constructed by aggregating the synaptic connection weights between all neurons within the working memory bank.

[0021] Step S4: Obtain the current multimodal data of the current retrieval, perform brain-like retrieval on the platform, obtain the retrieval memory data, and send it; Step S4 includes: Obtain the pulse signal sequence of the currently retrieved multimodal data at the current time within the current period, and update the global connectivity matrix W within the current period. △ ; Calculate the feature similarity score between the historical working memory in the working memory bank and the pulse signal sequence of the currently retrieved multimodal data, obtain a relevant historical memory of the currently retrieved multimodal data, and update the global connectivity matrix W from the current period. △ The sum of synaptic connection weights H between neurons of a specific historical memory is obtained. sum ; Set the connection accumulation threshold H', when H' sum When >H´, a certain related historical memory is recorded as the target retrieval historical memory for the current multimodal data retrieval; otherwise, no related historical memory is processed. The system retrieves and aggregates the historical retrieval data of each target in the working memory of the current multimodal data retrieval, obtaining the retrieval memory data of the current retrieval modality, and then sends the retrieval memory data to the retrieval personnel of the current multimodal data retrieval through the platform.

[0022] To better implement the above method, a working memory reorganization system is also proposed, which includes a synaptic connection analysis module, a similarity analysis module, a reorganization update module, and a memory retrieval module. The synaptic connection analysis module is used to acquire multimodal data retrieved from the platform, construct pulse signal sequences, analyze the degree of synaptic connection between different neurons in the pulse signal sequences, and obtain retrieved pulse data. The similarity analysis module is used to analyze the similarity between the retrieved multimodal data and the candidate historical memories in the platform's working memory bank to obtain a set of historically relevant records; The reorganization and update module is used to update and reorganize the global connection weight matrix in the working memory to obtain the updated global connection matrix. The memory retrieval module is used to perform brain-like retrieval on the current multimodal data in the platform to obtain retrieval memory data, and then send the retrieval memory data through the platform.

[0023] The synaptic connection analysis module includes a signal sequence construction unit and a synaptic connection analysis unit. The signal sequence construction unit is used to acquire and retrieve multimodal data from the platform and construct the pulse signal sequence of the retrieved multimodal data; The synaptic connection analysis unit is used to analyze the degree of synaptic connection between different neurons in a pulse signal sequence to obtain retrieved pulse data.

[0024] The similarity analysis module includes a candidate historical memory determination unit and a similarity analysis unit. The candidate historical memory determination unit is used to obtain the pulse signal sequence of the retrieval multimodal data in the spiking neuron model from the retrieval pulse data, and determine the candidate historical memory of the pulse signal sequence from the working memory bank; The similarity analysis unit is used to analyze the similarity between the retrieved multimodal data and the candidate historical memories in the working memory bank of the platform, determine the relevant historical memories of the retrieved multimodal data, and obtain the historical relevant record set of the retrieved multimodal data.

[0025] The recombination and update module includes a memory clustering unit and a recombination and update unit. The memory clustering unit is used to identify the historical memories to be cut in the working memory bank, smoothly sever the synaptic connections of the neurons of the historical memories to be cut, and generate clusters. The reorganization and update unit is used to analyze the increase in synaptic connections between different neurons in the cluster, and to update and reorganize the global connection weight matrix in the working memory to obtain the updated global connection matrix.

[0026] The memory retrieval module includes a memory retrieval unit; The memory retrieval unit is used to acquire the current retrieval multimodal data at the current moment, and combine it with the updated global connection matrix to perform brain-like retrieval on the platform to obtain retrieval memory data. The retrieval memory data is then sent to the retrieval personnel of the current retrieval multimodal data through the platform.

[0027] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A working memory reorganization method based on brain-like cognitive computing, characterized in that, The method includes: Step S1: Obtain the multimodal data from the platform, construct a pulse signal sequence, analyze the degree of synaptic connection between different neurons in the pulse signal sequence, and obtain the retrieval pulse data; Step S2: Analyze the similarity between the retrieved multimodal data and the candidate historical memories in the platform's working memory bank to obtain a set of historically relevant records; Step S3: Identify the historical memories to be severed in the working memory bank, smoothly sever the synaptic connections of the neurons of the historical memories to be severed, generate clusters, analyze the increase in synaptic connections between different neurons in the clusters, update and reorganize the global connection weight matrix in the working memory bank to obtain the updated global connection matrix. Step S4: Obtain the current multimodal data of the current retrieval, perform brain-like retrieval on the platform, obtain the retrieval memory data, and send it; Step S4 includes: Obtain the pulse signal sequence of the currently retrieved multimodal data at the current time within the current period, and update the global connectivity matrix W within the current period. △ ; Calculate the feature similarity score between the historical working memory in the working memory bank and the pulse signal sequence of the currently retrieved multimodal data, obtain a relevant historical memory of the currently retrieved multimodal data, and update the global connectivity matrix W from the current period. △ The sum of synaptic connection weights H between neurons of a specific historical memory is obtained. sum ; Set the connection accumulation threshold H', when H' sum When >H´, a certain related historical memory is recorded as the target retrieval historical memory for the current retrieval of multimodal data; otherwise, no processing is performed on a certain related historical memory. The system retrieves and aggregates the historical retrieval data of each target in the working memory of the current multimodal data retrieval, obtaining the retrieval memory data of the current retrieval modality, and then sends the retrieval memory data to the retrieval personnel of the current multimodal data retrieval through the platform.

2. The working memory reorganization method based on brain-like cognitive computing according to claim 1, characterized in that, Step S3 includes: Obtain the historical relevant record set of the retrieved multimodal data within the current period, and write the pulse signal sequence of the retrieved modal data into the working memory of the platform; According to the historical related record set, calculate the composite retrieval score R of the c-th historical memory in the working memory library c calculate the mean value μ and standard deviation σ of the composite retrieval scores of each historical memory in the working memory library, and calculate the memory salience value G of the c-th historical memory in the working memory library c set the composite retrieval score threshold R´ and the memory salience threshold G´, when R c < R´ or G c < G´, the c-th historical memory is recorded as a historical memory to be cut off; otherwise, no processing is performed on the c-th historical memory; Obtain the synaptic connection matrix W of the historical memory to be severed from the working memory bank, and perform smooth severing of neural connections on the synaptic connection matrix W to obtain the decayed synaptic connection matrix; Get the d-th historical memory that is not to be severed in the current period, get several related historical memories of the d-th historical memory and aggregate them to obtain a set of similar historical memories, use a clustering algorithm to cluster the set of similar historical memories to obtain clusters in the set of similar historical memories, and construct the synaptic connection matrix of the clusters. Obtain the global connectivity weight matrix W' from the working memory. Use the decaying synaptic connection matrix of the historical memory to be severed to replace the synaptic connection matrix of the historical memory to be severed in the global connectivity weight matrix W' for the current period. Then, add the increasing synaptic connection matrix of the cluster to the global connectivity weight matrix W' to obtain the updated global connectivity matrix W' for the current period. △ .

3. The working memory reorganization method based on brain-like cognitive computing according to claim 1, characterized in that, Step S2 includes: The pulse signal sequence of the multimodal data in the spiking neuron model is obtained from the retrieved pulse data, and the working memory bank is obtained from the platform to obtain the candidate historical memory of the pulse signal sequence in the current cycle. Calculate the exponentially decaying kernel between the candidate historical memory and the retrieved multimodal data, obtain each neuron of the pulse signal sequence, calculate the temporal similarity between the historical pulse signal sequence of the candidate historical memory and the pulse signal sequence of the retrieved multimodal data at each neuron dimension, and calculate the original similarity score Q between the historical pulse signal sequence of the candidate historical memory and the pulse signal sequence of the retrieved multimodal data. sum ; Calculate the self-similarity score Q´ of the historical pulse signal sequence of candidate historical memories, and calculate the self-similarity score Q' of the pulse signal sequence for retrieving multimodal data. △ Calculate the feature similarity score P between the historical pulse signal sequence of candidate historical memory and the pulse signal sequence of retrieved multimodal data; Calculate the feature similarity score between the historical pulse signal sequence of each candidate historical memory and the pulse signal sequence of the retrieved multimodal data. Set a feature similarity threshold P'. When the feature similarity score between the historical pulse signal sequence of a candidate historical memory and the pulse signal sequence of the retrieved multimodal data is greater than the feature similarity threshold P', then the candidate historical memory is recorded as the relevant historical memory for the retrieved multimodal data. Retrieve all relevant historical memories of the retrieved multimodal data, increment the retrieval count of each relevant historical memory by 1, update the retrieval time of each relevant historical memory to the time point when the retrieved multimodal data was entered into the platform in the current period, and aggregate the retrieval count and retrieval time of each relevant historical memory of the retrieved multimodal data to obtain a set of historical relevant records of the retrieved multimodal data.

4. The working memory reorganization method based on brain-like cognitive computing according to claim 1, characterized in that, Step S1 includes: The system retrieves the multimodal retrieval data to be processed within the current period from the platform, preprocesses the multimodal retrieval data, extracts and fuses features from the multimodal retrieval data, and obtains the multimodal fusion feature vector of the multimodal retrieval data. Set the encoding time window T=[t start ,t end Given a time step t', each component in the multimodal fusion feature vector is mapped to a neuron. The input current of each neuron in the predefined spiking neuron model is calculated, where t' is the input current of the multimodal fusion feature vector. start and t end These are the encoding start time and encoding end time, respectively; The encoding time window T is divided according to the time step t´ to obtain each time step within the encoding time window T. The membrane potential of each neuron at each time step within the encoding time window T is calculated iteratively. Set the membrane potential threshold V th Set a pulse signal variable S. When the membrane potential of a neuron at a certain time step is less than the membrane potential threshold V... th If the pulse signal variable of a neuron at a certain time step is set to 1, then the pulse signal variable of a neuron at a certain time step is set to 0. The pulse signal variables of each neuron at each time step are obtained and aggregated to obtain the pulse signal sequence of the retrieved multimodal data in the spiking neuron model. Calculate the synaptic connection weights between one neuron and another neuron at a certain time step, and construct the synaptic connection matrix of the pulse signal sequence for retrieving multimodal data; The retrieved multimodal data is obtained by combining the pulse signal sequences and synaptic connection matrices in the spiking neuron model.

5. A working memory reorganization system for executing a working memory reorganization method based on brain-like cognitive computing as described in any one of claims 1-4, characterized in that, The system includes a synaptic connection analysis module, a similarity analysis module, a recombination and update module, and a memory retrieval module; The synaptic connection analysis module is used to acquire multimodal data retrieved from the platform, construct pulse signal sequences, analyze the degree of synaptic connection between different neurons in the pulse signal sequences, and obtain retrieved pulse data. The similarity analysis module is used to analyze the similarity between the retrieved multimodal data and the candidate historical memories in the working memory bank of the platform to obtain a set of historically relevant records; The reorganization and update module is used to update and reorganize the global connection weight matrix in the working memory to obtain an updated global connection matrix. The memory retrieval module is used to perform brain-like retrieval on the current multimodal data in the platform to obtain retrieval memory data, and then send the retrieval memory data through the platform.

6. A working memory reorganization system according to claim 5, characterized in that, The synaptic connection analysis module includes a signal sequence construction unit and a synaptic connection analysis unit; The signal sequence construction unit is used to acquire and retrieve multimodal data from the platform, and construct the pulse signal sequence of the retrieved multimodal data; The synaptic connection analysis unit is used to analyze the degree of synaptic connection between different neurons in the pulse signal sequence to obtain the retrieved pulse data.

7. A working memory reorganization system according to claim 5, characterized in that, The similarity analysis module includes a candidate historical memory determination unit and a similarity analysis unit; The candidate historical memory determination unit is used to obtain the pulse signal sequence of the retrieval multimodal data in the spiking neuron model from the retrieved pulse data, and determine the candidate historical memory of the pulse signal sequence from the working memory bank; The similarity analysis unit is used to analyze the similarity between the retrieved multimodal data and the candidate historical memories in the working memory bank of the platform, determine the relevant historical memories of the retrieved multimodal data, and obtain the historical relevant record set of the retrieved multimodal data.

8. A working memory reorganization system according to claim 5, characterized in that, The recombination and update module includes a memory clustering unit and a recombination and update unit; The memory clustering unit is used to identify the historical memories to be severed in the working memory bank, smoothly sever the synaptic connections of the neurons of the historical memories to be severed, and generate clusters. The reorganization and update unit is used to analyze the increase in synaptic connections between different neurons in the cluster, update and reorganize the global connection weight matrix in the working memory to obtain an updated global connection matrix.

9. A working memory reorganization system according to claim 5, characterized in that, The memory retrieval module includes a memory retrieval unit; The memory retrieval unit is used to acquire the current retrieval multimodal data at the current moment, and in combination with updating the global connection matrix, perform brain-like retrieval on the platform to obtain retrieval memory data, and send the retrieval memory data to the retrieval personnel of the current retrieval multimodal data through the platform.

Images