Dynamic neuron based open environment pattern recognition method

Abstract

The application discloses an open environment pattern recognition method based on dynamic neurons, and steps include: initializing and configuring a structural development type neural network, setting the attribute of information saturation, and constructing a required structural development type neural network model; acquiring an image or text data stream to be recognized and inputting into the structural development type neural network model for learning and classification, and outputting a classification result, wherein when the structural development type neural network is in structural development and parameter adjustment, the information saturation of a winning neuron is used to control the neuron parameter and the splitting time, and the information saturation of the winning neuron is continuously adjusted, and when the information saturation of the neuron reaches a preset saturation threshold, a new neuron is split out. The application can improve the plasticity and stability of the competitive neural network, and is suitable for accurately and reliably learning and classifying non-steady-state data streams.

Description

Technical Field

[0001] This invention relates to the field of pattern recognition technology in open environments, and more particularly to a pattern recognition method for open environments based on dynamic neurons. Background Technology

[0002] Artificial intelligence (AI) operating in the real world is often exposed to non-stationary information flows in open environments, requiring the ability to continuously learn—that is, to integrate new knowledge while retaining previously learned experience. This is generally considered one of the essential attributes of future general artificial intelligence. However, current mainstream advanced deep learning models typically rely on a complete, one-time-use training set. When learning new knowledge, the new information often overwrites previously learned knowledge, leading to catastrophic forgetting and making continuous learning in non-stationary environments difficult. Therefore, the problem of continuous learning has regained significant attention in recent years. However, most current continuous learning methods still focus on supervised learning, while open environments often lack prior information and external supervision. Large amounts of data often lack category labels, with each data point assigned a category label. Consequently, current continuous learning methods often contradict the characteristics of open environments.

[0003] Unsupervised continuous learning aims to learn from non-stationary, unlabeled data streams, where the distribution of data or the number of categories changes over time. However, neural network models often face a stability-plasticity dilemma in unsupervised continuous learning, meaning that when a learning network attempts to adapt to a constantly changing environment in real time, it is difficult to balance adaptability to rapidly changing environments with maintaining the stability of previously learned knowledge.

[0004] Most current structural developmental neural networks (SGRNs) are designed based on classic competitive neural networks such as self-organizing maps (SOMs) and neural gas networks (NGs). A representative SGRN is the Growing Neural Gas (GNG), which improves network dynamics compared to NGs but still lacks the ability to track rapidly changing, non-stationary data. Several SGRN-based methods are currently applied in various continuous learning scenarios. Examples include the knowledge consolidation (KC) algorithm for incremental learning of old and new class features in GWR video action recognition; combining pre-trained CNNs with SOINNs for continuous target recognition; and the general associative memory (GAM) system based on SOINN. These extended algorithms introduce the incremental learning properties of SGRNs into classification algorithms, making classifiers suitable for continuous learning scenarios.

[0005] The aforementioned structural developmental neural networks effectively improve the plasticity and stability of neural networks by using a mechanism of adaptive development of network topology. However, when performing pattern recognition on image data streams in open environments, the following problems still exist in achieving unsupervised continuous learning and simulating the human learning process:

[0006] 1. Open environments place higher demands on the real-time performance of neural network models. Since there is no distinction between training and testing phases, a seamless transition between the learning and usage phases of the network model is required. Traditional methods of adding and deleting neurons based on connection age thresholds and pre-set periodic parameters can lead to fluctuations in the number of neurons during network development and instability in classification results, which is detrimental to the model's real-time performance and consequently affects the accuracy and real-time performance of the final pattern recognition.

[0007] 2. Learning in an open environment requires neural networks to be able to recognize things more and more precisely, just like the human learning process. That is, as the input information increases, the learning results should be subdivided. In the current technology, structural developmental neural networks cannot subdivide the learning results as the input information increases, that is, they cannot make full use of continuous data streams to achieve increasingly refined pattern recognition. Summary of the Invention

[0008] The technical problem to be solved by this invention is: In view of the technical problems existing in the prior art, this invention provides an open environment pattern recognition method based on dynamic neurons that is simple to implement, has high recognition accuracy and real-time performance, and can gradually refine the recognition as the input information increases. It can combine a highly plastic and stable competitive neural network to achieve efficient and accurate learning and classification of non-steady-state data streams.

[0009] To solve the above-mentioned technical problems, the technical solution proposed by this invention is as follows:

[0010] An open environment pattern recognition method based on dynamic neurons, comprising the following steps:

[0011] The Structural Developmental Neural Network with Information Saturation (SDNNIS) is initialized and configured, and the information saturation attribute is set for its neurons to construct the required Structural Developmental Neural Network model.

[0012] The image or text data stream to be identified is input into the structural developmental neural network model for learning and classification, and the classification result is output. During structural development and parameter adjustment, the structural developmental neural network adjusts the neuron parameters and splitting timing according to the information saturation of the winning neuron, and continuously adjusts the information saturation of the competing winning neuron. The information saturation is calculated based on the winning frequency of the neuron and the information carrying capacity. When the information saturation of the neuron reaches a preset saturation threshold, a new neuron splits out.

[0013] Furthermore, the structural developmental neural network is an SDNNIS comprising an input layer and a competition layer. The input layer is used to transmit sample features. After receiving the sample features from the input layer, the competition layer performs neuron coverage detection, meaning only neurons covering the input sample respond. During the competitive learning phase, responding neurons compete with each other, activating the neuron with the largest response value, thus obtaining the winning neuron. The information saturation of the winning neuron is updated. If the information saturation is not yet saturated, the center weight vector of the corresponding neuron is adjusted, and the index of the winning neuron is output. If the information saturation is saturated, the corresponding neuron splits to generate a new neuron, and the index of the new neuron is output. Based on the network output, the number of cluster layers corresponding to the neuron, and the clustering layer parameters, a label is assigned to the current input sample, completing the clustering process.

[0014] Furthermore, during the neuron coverage detection, if the input signal x i With neuron n j The distance d(x) between the center vectors i ,μ j )≤r j Then determine the input signal x i There is a neuron n j The probability of representation, neuron n j A response occurs; if the input data stream does not fall within the coverage area of ​​any neuron, that is, the current output layer does not have a neuron that can represent the input, a new neuron is generated directly with the current input signal as the center vector.

[0015] Furthermore, when initializing the newly generated neuron, the new neuron n... j Coverage parameter σ j and coverage radius r j Specifically set as follows:

[0016] σ j =d(μ j ,μ p ) / 3

[0017] r j=3σ j

[0018] Where, μ j For neuron n j The center vector, μ p For neuron n j The center vector of the parent element, d() is the distance function.

[0019] Furthermore, a Gaussian function is used as the response function of neurons in the structural developmental neural network, that is, if a new input signal x is added... i It landed on neuron n j Within the coverage area, then neuron n j For input signal x i The response is:

[0020]

[0021] Where, μ j The mean of a Gaussian function and the number of neurons n j The center vector, σ j For neuron n j The coverage parameter is d, and d() is the distance function for calculating the input signal and the center vector of the neuron.

[0022] Furthermore, during the competitive learning phase, the neuron with the largest local response wins, and the index k of the winning neuron is obtained according to the following formula:

[0023]

[0024] Among them, IdR i The coverage area encompasses the input signal x i The set of subscripts of neurons.

[0025] Furthermore, in the aforementioned structural developmental neural network, the root node of each tree is the root neuron, which is the first layer of hierarchical clustering. The child neurons of the root neuron are the second layer, and so on. Each mature neuron whose layer number is less than the clustering layer number CL represents a class. The class label is the neuron subscript, and the center vector is the prototype of the class. The class labels of immature neurons and neurons whose layer number is greater than CL directly inherit the parent label.

[0026] Furthermore, the winning neuron n k Regarding the input signal x i The change in information saturation Δs k,i Based on input sample x i With the winning neuron n k The similarity is calculated as follows:

[0027] △sk,i (o k,i )=a(1-o k,i )

[0028]

[0029] Where a is a constraint coefficient and takes the value 1-R, and R is the information saturation reset threshold.

[0030] Furthermore, in the aforementioned structural developmental neural network model, if the information saturation of a neuron has not reached saturation, the index k of the winning neuron is output, and the center weight vector of the winning neuron is updated so that it moves toward the input data. Specifically, the center weight vector is updated using the following formula:

[0031]

[0032] Where, μ k ′ is the updated center weight vector, μ k Let Δs be the center weight vector before the update. k,i For the winning neuron n k Regarding the input signal x i The change in information saturation, s k ′ represents the updated information saturation.

[0033] Furthermore, the structural developmental neural network model also includes a mother element, the competitive layer develops from the mother element, and the center weight vector of the mother element is configured as the origin of the sample space, the coverage parameter is positive infinity, and the information saturation is configured to be in a saturated state.

[0034] Compared with the prior art, the advantages of the present invention are as follows:

[0035] 1. This invention adaptively adjusts the network topology based on competitive learning and neuron splitting mechanisms to adapt to unsupervised continuous learning of non-stationary data streams. At the same time, the splitting mechanism based on information saturation enables the structural developmental neural network to split into new neurons when needed. It does not rely on periodic parameters that require certain prior knowledge to add or prune neurons, which can improve the plasticity, stability and real-time performance of the model. The model can also subdivide the categories during continuous learning, so that the learning results can be continuously subdivided as the input information increases. Thus, combined with this structural developmental neural network, it can achieve efficient, accurate and progressively subdivided learning and classification of non-stationary data streams.

[0036] 2. This invention further improves the efficiency and real-time performance of pattern recognition by setting the coverage domain characteristics of neurons in the structural developmental neural network and setting a coverage domain matching stage before the network enters competitive learning. This avoids global computation of the network and improves the parallelism and operating efficiency of the network. Furthermore, by adopting a method in which only local neurons participate in the computation, the consumption of computing power can be further reduced significantly.

[0037] 3. The present invention further sets up a mother cell in the structural developmental neural network, which unifies the network's discovery of new samples and the subdivision of categories into the neuron splitting mechanism. This makes the network model more concise and avoids global network operations, further improving the network's sensitivity to novel features and the network's parallelism, thereby further improving the efficiency and real-time performance of pattern recognition. Attached Figure Description

[0038] Figure 1 This is a schematic diagram illustrating the implementation process of the open environment pattern recognition method based on dynamic neurons in this embodiment.

[0039] Figure 2 This is a schematic diagram of the structural principle of SDNNIS used in this embodiment.

[0040] Figure 3 This is a schematic diagram of the similarity curve and the corresponding information saturation change curve obtained in this embodiment.

[0041] Figure 4 This is a schematic diagram of the neuron category labeling in the output layer of this embodiment.

[0042] Figure 5 This is a schematic diagram showing the changes in the number of neurons and the number of clusters identified during the network's development in specific application scenarios under instance incremental and class incremental learning scenarios.

[0043] Figure 6 This is a schematic diagram of the category segmentation results of SDNNIS in a specific application embodiment.

[0044] Figure 7 This is an example diagram illustrating how the number of clustering layers determines the degree of subdivision in a specific application embodiment.

[0045] Figure 8 The information saturation reset threshold affects the number of neurons generated by SDNNIS in instance-incremental and class-incremental learning scenarios (corresponding to...). Figure 8 (a) and clustering effect (corresponding) Figure 8 Schematic diagram of the impact of (b)).

[0046] Figure 9 This is a schematic diagram of clustering results on an artificial dataset in a specific application embodiment.

[0047] Figure 10 This is a schematic diagram of the coil dataset used in a specific application embodiment.

[0048] Figure 11 This is a schematic diagram of a clustering result of SDNNIS on the coil dataset in a specific application embodiment. Detailed Implementation

[0049] The present invention will be further described below with reference to the accompanying drawings and specific preferred embodiments, but this does not limit the scope of protection of the present invention.

[0050] like Figure 1 As shown, the steps of the open environment pattern recognition method based on dynamic neurons in this embodiment include:

[0051] S01. Initialize and configure the structural developmental neural network and set the information saturation attribute for its neurons to construct the required structural developmental neural network model;

[0052] S02. The image or text data stream to be identified is input into the structural developmental neural network model for learning and classification, and the classification result is output. During structural development and parameter adjustment, the structural developmental neural network adjusts the neuron parameters and splitting timing according to the information saturation of the winning neuron, and continuously adjusts the information saturation of the competing winning neuron. The information saturation is calculated based on the winning frequency of the neuron and the information carrying capacity. When the information saturation of the neuron reaches the preset saturation threshold, a new neuron splits out.

[0053] This embodiment employs a structural developmental neural network (SDNNIS) with information saturation. This network, based on competitive learning and neuron splitting mechanisms, can adaptively adjust its topology to adapt to unsupervised continuous learning of non-stationary data streams. Simultaneously, the information saturation-based splitting mechanism allows the SDNIS to split into new neurons when needed, without relying on periodic parameters that require prior knowledge for neuron addition or pruning. This improves the model's stability and real-time performance. Based on the parent-child relationships generated by neuron splitting, the model can also simulate long and short-term memory in human learning processes, further subdividing categories during continuous learning. This makes the network's learning mechanism closer to human continuous learning methods, thereby improving not only the network's plasticity and stability but also enabling continuous subdivision of learning outcomes as input information increases.

[0054] Structural developmental neural networks are a type of growing neural network based on competitive learning. They typically use competitive learning and certain rules for adding and removing neurons to adaptively adjust the weights and number of neurons, enabling the network topology to autonomously develop and approximate the distribution of input data without affecting previous learning outcomes. This alleviates the stability-plasticity dilemma of competitive learning and allows for unsupervised continuous learning of data streams that change over time.

[0055] Due to the irreversibility of time and the instability of the world, humans have evolved the ability to continuously learn in changing environments, learning new knowledge more effectively based on old knowledge, and not forgetting previous knowledge while learning new knowledge. The theory of complementary learning systems (CLS) defines the complementary roles of the hippocampus and neocortex in learning and memory, indicating the existence of specialized mechanisms in the human cognitive system for protecting and consolidating knowledge. The hippocampus is responsible for short-term adaptation, allowing for rapid learning of new information, which is then transferred and integrated into the neocortex for long-term storage. In the hippocampus region of the mammalian brain, new functional neurons are constantly generated by neuronal stem cells and embedded in existing neural networks. When people repeatedly encounter the same type of objects, the associated memory neurons are continuously activated, and the memory of that type of object is continuously strengthened. Furthermore, in the process of human cognition, as the amount of information received increases, the understanding of things becomes more precise. When the number of samples corresponding to a learned category becomes rich enough, this category will naturally be subdivided. For example, from a macro perspective, domestic cats and tigers have many similar characteristics and both belong to the cat family, so they can be classified as one category. As more and more people become familiar with tigers, this animal can be further subdivided into subspecies such as Siberian tiger, Siberian tiger, and South China tiger.

[0056] Inspired by the aforementioned biological neural mechanisms and human cognitive processes, this embodiment, based on SDNNIS, adds an information saturation attribute to neurons in the network to reflect the amount of information carried by the neuron. This information saturation level determines the developmental stage of a single neuron and whether it splits. Specifically, the information saturation of the winning neuron increases with the amount of information brought in by the input data, the neuron gradually matures, the adjustment range becomes smaller, and it tends to stabilize. This process simulates the long short-term memory mechanism of the human brain, improving the stability of the neural network model. When the information saturation reaches the splitting threshold, it indicates that the information carried by the neuron is rich enough, and the neuron splits to produce new neurons, simulating the subdivision process in human cognition and increasing the model's plasticity.

[0057] SDNNIS is a growing competitive neural network consisting of an input layer and an output layer. The input layer receives the feature vector of the target object, while the output layer develops from scratch. Each neuron possesses three internal attributes: a center vector, a coverage parameter, and information saturation. Neurons use a Gaussian function as their response function, with the mean and standard deviation of the Gaussian function representing the neuron's center vector and coverage parameter, respectively, and these values ​​change as the neuron is adjusted. SDNNIS learns and classifies the input data stream using an incremental competitive learning approach. When an input signal is passed to the input layer, neurons in the output layer only respond to input signals falling within their coverage area. The responding neurons compete to produce a winning neuron based on the output values ​​of their respective response functions, and the network output indicates the category of the winning neuron. If the input signal does not fall within the coverage area of ​​any neuron, a new neuron is directly generated, achieving anomaly detection or acquiring new knowledge.

[0058] In this embodiment, each neuron in the structural developmental neural network represents a class. During competitive learning, the neuron most "similar" to the input sample is activated, becoming the winning neuron. The input sample is then classified into the class represented by the winning neuron. The information saturation of the neuron gradually increases with the number of activations. The higher the information saturation, the smaller the adjustment amplitude of the center weight vector. When the information saturation reaches a saturation threshold, the neuron splits to generate new neurons, representing further subdivision of the class or a new class. After classifying the signal, SDNNIS enters the neuron attribute and lateral connection update phase. The attributes and lateral connections of the winning neuron are adjusted according to its developmental stage and the input signal. The developmental stage of the neuron is determined by the information saturation; neurons with higher information saturation are more mature, have smaller update amplitudes, and tend to be stable, effectively simulating the long short-term memory mechanism in human learning. When the information saturation of a neuron reaches the saturation threshold, it means that the neuron carries a sufficiently rich amount of information, or that the sample features appear frequently within the neuron's coverage area, causing the neuron to split into new neurons. These new neurons tend to further subdivide the original class, effectively simulating the increasingly refined process of human cognition as knowledge increases.

[0059] like Figure 2As shown, the number of neurons in the input layer is equal to the dimension of the input space sample features. The function of these neurons is to transmit sample features. After receiving the sample features from the input layer, the competition layer first performs coverage detection of the neurons, meaning only neurons covering the local area of ​​the input sample respond. Then, it enters the competitive learning phase, where responding neurons compete with each other. The neuron with the largest response value is activated, and the winning neuron gets the opportunity to update its information saturation. If the information saturation is not reached, the neurons adjust their center weight vectors, and the output is the index of the winning neuron; if saturation is reached, the neurons split to generate new neurons, and the output is the index of the new neurons. Finally, the network output, the number of cluster layers corresponding to the neurons, and the user-defined clustering layer parameters are combined to assign a label to the current input sample, completing the clustering.

[0060] For ease of description, this embodiment defines the following parameters: input signal x i =(x i1 ,x i2 ,...,x id ) represents the i-th signal input to the network in the sample space χ, possessing d-dimensional features. The output layer consists of a set of neurons. Describe the process, where i represents the number of signals the model has processed, and n represents the number of signals the model has processed. j Let k represent the j-th neuron. i This represents the number of neurons generated after the algorithm processes the i-th piece of information. Each neuron is a triple n. j =(μ j ,σ j ,s j ,p j ), where μ j σ represents the center vector. j Indicates the coverage parameter, s j p represents information saturation. j Represents the parent index. The neuron's coverage area D. j ={x|d(x,μ j )≤r j}, where d() is the distance function. The splitting mechanism of neurons leads to parent-child relationships between neurons, thus forming a hierarchical tree structure T. SDNNIS clusters neurons by classifying them according to T and the user-defined number of clustering layers CL, n j The category label is denoted as l k .

[0061] To avoid global computations in the network and improve its parallelism and efficiency, this embodiment sets coverage domain characteristics for neurons and a coverage domain matching stage before the network enters competitive learning. When a new sample feature is input into the input layer, neurons in the competitive layer perform coverage domain matching; only neurons whose coverage domains contain the input will respond and participate in the competition. Specifically, when the input sample x... i ∈D j ={x|d(x,μ j )≤r j When, that is, when the input signal x i With neuron n j The distance d(x) between the center vectors i ,μ j )≤r j At that time, it was believed that x i There is n j The probability of representation, neuron n j A response occurs. If the input signal does not fall within the coverage area of ​​any neuron, meaning the current output layer has no neuron capable of representing the input, a new neuron is directly generated using the input signal as the center vector. This embodiment significantly reduces computational power consumption by involving only local neurons in the computation.

[0062] In this embodiment, based specifically on the 3-sigma rule of Gaussian distribution, when initializing newly generated neurons, the new neuron n... j Coverage parameter σ j Set to σ j =d(μ j ,μ p ) / 3, where μ p For neuron n j The center vector of the parent neuron, d() is the distance function. Let neuron n be... j The coverage radius r j for:

[0063] r j =3σ j (1)

[0064] Where, σ j For n j Coverage parameters.

[0065] Because the von Neumann architecture has the characteristic of sequential execution, when simulating the model on a computer, a global comparison is required in the coverage matching stage, which cannot reflect the local computation and parallelism of the network. However, by hardwareizing the model, each matching can be calculated independently, thus achieving parallelism.

[0066] By matching, the index set IdR of the local neurons that responded can be obtained. i In this embodiment, SDNNIS is specifically configured to use a Gaussian function as the response function of the output layer neurons; that is, if a new input signal x is added... i It landed on neuron n j Within the coverage area, then neuron n j For x i The response is:

[0067]

[0068] The mean of the Gaussian function is the center vector μ of the neuron. j The standard deviation is the coverage parameter σ of the neuron. j d() is a function that calculates the distance between the input signal and the neuron's center vector. Since the input signal and the neuron's center vector are of the same dimension, Euclidean distance is specifically used in this embodiment.

[0069] From the above equation, we can see that the center and standard deviation of the Gaussian response function are the center weight vector and coverage parameters of the neuron, respectively.

[0070] The neuron with the largest local response ultimately wins; that is, the neuron with the largest response value among those that respond is activated. The index k of the winning neuron is obtained, expressed by the formula:

[0071]

[0072] Among them, IdR i The coverage area encompasses the input x i The set of indices of neurons. Once the winning neuron is activated, it immediately enters the learning phase, either by adjusting the center weight vector or splitting to generate a new neuron. Whether to adjust or split depends on whether the information saturation reaches a saturation threshold (value of 1) after the update.

[0073] In this embodiment, the winning neuron n k Regarding the input sample x i The change in information saturation is denoted as Δs. k,i Its size depends on the input sample x i With the winning neuron n k Similarity o k,j The similarity is calculated. Specifically, it can be calculated using the Gaussian radial basis function, i.e.:

[0074]

[0075] The formula for calculating the change in information saturation is as follows:

[0076] △s k,i (ok,i )=a(1-o k,i (5)

[0077] Where 'a' is the constraint coefficient, which takes the value 1-R, and R is the information saturation reset threshold, i.e., the information saturation drop-off value after neuron splitting. In a specific application embodiment, the similarity curve and the corresponding information saturation change curve obtained according to the above method are shown below. Figure 3 As shown. Information saturation s k The update formula is s k ′=s k +△s k,i , where s k ' represents the updated information saturation, s k This represents the information saturation level before the update.

[0078] In this embodiment, if the information saturation of a neuron has not reached saturation, the network output is the index k of the winning neuron. The winning neuron will update its center weight vector, moving it towards the input sample. The specific formula for updating the center weight vector in this embodiment is:

[0079]

[0080] Where, μ k ′ is the updated center weight vector, μ k Let Δs be the center weight vector before the update. k,i For the winning neuron n k Regarding the input signal x i The change in information saturation.

[0081] Furthermore, considering that in open environments, intelligent systems often encounter samples that have not been learned from before, meaning that input samples may not fall within the coverage of any neuron in the current neural network, this embodiment further sets up a parent unit (similar to the existence of stem cells in a biological organism) to ensure the network's sensitivity to novel features and its parallelism. The competitive layer develops starting from the parent unit, with the central weight vector of the parent unit being the origin of the sample space and the coverage parameter being positive infinity, to ensure that it can respond to input samples in the entire sample space. The information saturation is 1, indicating a saturated state, so that when only the parent unit responds (novel features appear), it immediately splits to generate new neurons to represent new features. In this embodiment, the neuron set is initialized as N0 = {n0}, where n0 is the parent unit. The quadruple form of the parent unit can be specifically represented as n0 = (0, ∞, 1, -1), which remains unchanged throughout the network development process. This embodiment unifies the network's discovery of new samples and the subdivision of categories into the neuron splitting mechanism by setting the mother unit. This makes the network model more concise, avoids global computation of the network, and further improves the network's sensitivity to novel features and the network's parallelism.

[0082] In a specific application embodiment, if the information saturation reaches saturation, it means that the neuron carries a sufficiently rich amount of information, and the neuron will split to generate a new neuron. The network output is the index of the new neuron. The new neuron uses the input sample that triggered the split of the winning neuron as its center weight vector, and its coverage parameter is one-third of the distance between the neuron and the parent's center weight vector, that is, the parent's center weight vector is located at the 3σ boundary of the new neuron. The information saturation of the new neuron is initialized to the supremum 'a' of the information saturation change, the parent is labeled as the index of the winning neuron, and the initialization formula for the new neuron is:

[0083] n Ni-1 ={x i ,d(x i ,μ k ) / 3,a,k} (7)

[0084] This embodiment quantifies the winning frequency and information carrying capacity of neurons by using information saturation. Based on information saturation, it regulates the parameter adjustment and splitting of neurons. By simulating the human brain's memory mechanism and cognitive process in SDNNIS, it can effectively improve the plasticity and stability of the network model, thereby effectively improving the accuracy and reliability of pattern recognition.

[0085] SDNNIS's clustering method combines prototype-based and hierarchical clustering. The center vector of each neuron can serve as a prototype in the cluster, and the number of prototypes does not need to be pre-specified but is dynamically adjusted based on the input signal. The parent-child relationships formed by neuron splitting establish a tree-like hierarchical structure, with child neurons further subdividing the classes represented by their parent neurons. Users can set the number of clustering layers according to the required clustering accuracy.

[0086] In this embodiment of the structural developmental neural network, the root node of each tree is the root neuron, which is the first layer of hierarchical clustering. The child neurons of the root neuron are the second layer, and so on. Each mature neuron whose layer number is less than the clustering layer number CL represents a class. The class label is the neuron subscript, and the center vector is the prototype of the class. The class labels of immature neurons and neurons whose layer number is greater than CL directly inherit the parent label.

[0087] Specifically, such as Figure 4 As shown, the parent node is the root node of the clustering tree, which is at level 0. The child neurons of the parent node constitute the second level of the clustering tree, and so on, up to n. j The floor number is denoted as lr j Assuming the user-set clustering layer parameter is 2, neurons in layers less than 2 each form a cluster center with their own category label, such as their own index. Neurons in layers greater than 2 directly inherit their parent's label; therefore, neurons in layers greater than the clustering layer parameter under the same parent have the same category label and are clustered into the same class. Finally, the input sample is labeled with the same label as its corresponding neuron, completing the incremental clustering of the samples.

[0088] To verify the effectiveness of this invention, the developmental process and subdivision capability of SDNNIS were tested in a specific application embodiment. The effectiveness of the structural developmental competition learning rule introducing information saturation was also verified, as well as the impact of hyperparameters on SDNNIS structural development and performance. Specifically, this embodiment verifies the incremental clustering effect of SDNNIS on an artificially constructed image dataset and compares it with typical traditional structural developmental neural networks GNG, GWR, and SOINN. Furthermore, the effectiveness of SDNNIS in continuous unsupervised learning is further verified on a real-world dataset. The artificially constructed image dataset consists of images following a two-dimensional normal distribution with different distribution characteristics (as shown in Table 1). The real-world dataset comes from the coil-100 dataset, which is a dataset composed of images of different objects at different angles during a 360-degree rotation. The learning scenarios are divided into instance-incremental and class-incremental types based on the sample input method. In the instance-incremental learning scenario, the order in which samples from different classes appear is random. In the class-incremental learning scenario, samples are input in class order, i.e., all samples of one class are input before the next class is input. By setting up the above datasets and learning scenarios, the effectiveness of the algorithm can be comprehensively verified under different feature datasets and different continuous learning scenarios.

[0089] Table 1: Artificial Datasets

[0090]

[0091] This embodiment first tests the changes in the number of neurons and the number of identified clusters under two continuous learning scenarios: instance increment and class increment, on the DS1 dataset. The initial SDNNIS network is configured (including configuring the coverage area, competitive learning mechanism, etc.), and the information saturation attribute is set for the neurons in the network to construct the required SDNNIS model. Image data from the DS1 dataset is input into the configured SDNNIS model for learning and classification. During structural development and parameter adjustment, SDNNIS adjusts the neuron parameters and splitting timing based on the information saturation of the winning neuron, continuously adjusting the information saturation of the competing winning neurons. The information saturation is calculated based on the neuron's winning frequency and information capacity. When the information saturation of a neuron reaches a preset saturation threshold, a new neuron splits out, and the SDNNIS model outputs the classification result, as described above. The changes in the number of neurons and the number of identified clusters during the network development process under instance increment and class increment learning scenarios are as follows: Figure 5As shown in (a) and (b), the number of neurons in structural developmental neural networks changes with the increase in the number of input samples. In scenarios with increasing instance increments, the number of neurons in SDNNIS (Structured Neural Networks) grows rapidly, enabling it to quickly identify all current categories and further subdivide categories in subsequent input samples. However, a drawback is that the continuous splitting of neurons makes it difficult to effectively control the network size. The number of neurons in GWR (Growth WR) stabilizes after reaching a certain stage of development, while the number of neurons in SOINN fluctuates significantly, and the number of neurons in GNG (Growth GNG) increases periodically. The number of neurons and the connections between neurons affect the clustering results, such as... Figure 5 As shown in (c) and (d), in the case of instance increments, the SDNNIS of this invention can quickly identify and maintain stability of the categories present in the environment, while other networks suffer from large fluctuations in the number of categories identified during development and the inability to quickly and effectively distinguish categories. In the scenario of class increments, when new categories appear, the SDNNIS of this invention can quickly identify the new categories and maintain stability. GNG cannot identify new categories in real time, and GWR and SOINN suffer from large fluctuations in the number of categories identified. In summary, the neuron splitting mechanism based on information saturation of this invention can effectively improve the plasticity of the model and promptly discover new categories in the environment. At the same time, since information saturation and the parent-child relationship between neurons maintain the network's learning results of old knowledge, it can also improve the stability of the model.

[0092] This embodiment further verifies the segmentation capability of the SDNNIS invention on the DS4 dataset. This test dataset consists of 5 clusters following a normal distribution, with overlap between clusters. Overlapping clusters can be considered as forming a larger cluster; therefore, the dataset can be viewed as consisting of two large clusters, each composed of 2 and 3 sub-clusters, respectively. The learning method is specifically instance-incremental. In the initial learning stage, when the number of input samples is small, the number of generated neurons is small, and the classification is coarse. The SDNNIS invention is able to identify the major categories present in the environment. For example... Figure 6 As shown in (a) and (b), when the number of samples is only 10, the SDNNIS of this invention considers it to be one class. When the number of learned samples increases to 30, it can identify two major classes. As the number of learned samples increases, when the information saturation of neurons in the SDNNIS of this invention reaches a saturation threshold, new neurons will split off, leading to a further subdivision of the original classes. The newly generated neurons can represent more refined classes, simulating the human cognitive process. Figure 6 As shown in (c), (d), and (e), with the increase in the number of training samples, the SDNNIS of this invention is gradually able to identify subclasses within two major categories. It is noteworthy that, because the clustering layer parameter CL is set to 3, even as neurons continue to grow during subsequent sample additions, the number of categories remains constant, as shown in... Figure 6 As shown in (f).

[0093] The choice of class subdivision level depends on the clustering layer parameter CL set by the user. Figure 7 The corresponding clustering results are 1, 2, and 3 for CL values, with the test dataset being DS4. When CL=1, the clustering result is 1 class; when CL=2, the clustering result is 3 classes (including the parent class); and when CL=3, the clustering result is further subdivided into 5 classes. It can be intuitively seen that using larger clustering layer parameters yields more refined clustering results. The specific clustering layer parameters can be set according to actual needs. Furthermore, from... Figure 7 As can be seen from this, the present invention SDNNIS can well reflect the categories contained in the learning samples at different levels of subdivision, which is consistent with the intuitive perception of the human brain.

[0094] The present invention, SDNNIS, has only one hyperparameter: the information saturation reset threshold R. Compared to previously proposed structural developmental neural networks (GNG, GWR, and SOINN have 6, 5, and 10 hyperparameters respectively), SDNNIS significantly reduces the number of hyperparameters, decreasing reliance on prior information and trial-and-error processes, thus effectively improving the algorithm's adaptability in open environments. This embodiment tested the number of neurons generated by SDNNIS and the DBI value of the clustering results on DS1 with different values ​​of R, which can reflect the impact of R on SDNNIS to a certain extent. According to the meaning of the information saturation reset threshold, the information saturation of neurons should be in a relatively stable state when it is at the reset threshold. Therefore, this embodiment tested the impact of R∈[0.50,0.95] (interval of 0.02) on the number of SDNNIS neurons and the DBI value of the clustering results under two learning scenarios: instance increment and class increment. Since SDNNIS is sensitive to the order of sample input, each group was tested 30 times with different sample input orders, and the final result of each group was the average of the 30 test results.

[0095] like Figure 8 As shown, where Figure 8 Figure (a) reflects the effect of the reset threshold on the number of neurons generated by the network. In both different learning scenarios, the number of neurons decreases as the reset threshold increases, reflecting that the constraint effect of the reset threshold on the growth of information saturation is dominant. The larger R is, the slower the growth of information saturation and the more difficult it is for neurons to split. Figure 8Figure (b) reflects the impact of the reset threshold on the clustering effect. With incremental instance counts, the DBI decreases as the reset threshold increases. However, with incremental class counts, the DBI is relatively low when R ∈ [0.65, 0.80], resulting in better clustering. Therefore, a larger reset threshold can improve the stability of neurons after splitting and reduce the number of neurons, thus improving network efficiency. However, an excessively large reset threshold is detrimental to network plasticity, thus affecting the clustering effect. Considering network efficiency, plasticity-stability, and clustering effect, this embodiment sets the hyperparameter R to 0.8, which maintains a good DBI level while reducing the number of neurons. It should be noted that the hyperparameters are inherent to SDNNIS and do not need to be set differently for different datasets and tasks.

[0096] This embodiment further validates the clustering performance of SDNNIS on artificial datasets DS1, DS2, and DS3. DS1 contains 5 clusters, each with the same number of samples and variance, and the sample input method is class increment. DS2 contains 4 clusters, each with the same number of samples but different variances (0.01, 0.02, 0.03, 0.05), and the sample input method is instance increment. DS3 contains 4 clusters, each with the same variance but different number of samples (25, 50, 100, 200), and the sample input method is instance increment. The clustering results on different datasets are as follows: Figure 9 As shown, the clustering layer parameter CL=2, and the comparison of clustering results is shown in Table 2.

[0097] Table 2: Comparison of clustering performance of structural developmental neural networks on datasets with different distribution characteristics and in different incremental learning scenarios.

[0098]

[0099] This embodiment further tests the sustained unsupervised learning performance of the SDNNIS invention on real-world samples using real-world images. The real-world images are from the coil-100 dataset, which contains color images of 100 objects with various complex geometric and reflective properties. These objects are placed on an electric turntable against a black background, which rotates 360 degrees to obtain images from different angles and poses. This embodiment selects five simple object types (such as...) Figure 10 As shown), each object contains 10 images from different angles. Experimental results show that the SDNNIS method of this invention achieves 100% accuracy in clustering images of 5 physical objects from different angles in scenarios with both instance increment and class increment. Figure 11As shown. Unlike mainstream supervised learning-based image classification methods that require a large number of samples to train the network to obtain a good classifier, SDNNIS, as described in this invention, can quickly identify new categories and effectively classify samples from a small number of samples appearing in the environment, even when sufficient training samples are unavailable and there is no prior knowledge about the task. The above clustering results on real-world images further demonstrate that SDNNIS can adapt to clustering tasks in different continuous learning scenarios.

[0100] This invention simulates the long and short term memory of the human brain by using neuronal information saturation and neuronal maturation stages based on information saturation, making the network model both plastic and stable. At the same time, based on the parent-child relationship generated by the splitting of neurons, the network can subdivide the original categories as the number of learning samples increases, simulating the gradually refined cognitive process of human cognition of things, thereby achieving pattern recognition and classification with plasticity, stability and gradual subdivision in an open environment.

[0101] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the invention. Therefore, any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention should fall within the protection scope of the present invention.

Claims

1. A method for pattern recognition in open environments based on dynamic neurons, characterized in that the steps include... include: The structural developmental neural network is initialized and configured, and the information saturation attribute of its neurons is set to construct the required structural developmental neural network model. The image or text data stream to be identified is input into the structural developmental neural network model for learning and classification, and the classification result is output. During structural development and parameter adjustment, the structural developmental neural network adjusts the neuron parameters and splitting timing according to the information saturation of the winning neuron, and continuously adjusts the information saturation of the competing winning neuron. The information saturation is calculated based on the winning frequency of the neuron and the information carrying capacity. When the information saturation of the neuron reaches a preset saturation threshold, a new neuron splits out. The structural developmental neural network includes an input layer and a competition layer. The input layer transmits sample features. After receiving the sample features from the input layer, the competition layer performs neuron coverage detection, meaning only neurons covering the input sample respond. During the competitive learning phase, responding neurons compete with each other, activating the neuron with the largest response value, thus obtaining the winning neuron. The information saturation of the winning neuron is updated. If the information saturation is not yet saturated, the center vector of the corresponding neuron is adjusted, and the index of the winning neuron is output. If the information saturation is saturated, the corresponding neuron splits to generate a new neuron, and the index of the new neuron is output. Based on the network output, the number of cluster layers the corresponding neuron belongs to, and the clustering layer parameters, the current input sample is labeled, completing the clustering process.

2. The open environment pattern recognition method based on dynamic neurons according to claim 1, characterized in that, When performing neuron coverage detection, if the input signal With neurons center vector Distance between Then determine the input signal There are neurons The possibility of representation, neurons A response occurs; if the input data stream does not fall within the coverage area of ​​any neuron, that is, the current output layer does not have a neuron that can represent the input, a new neuron is generated directly with the current input signal as the center vector.

3. The open environment pattern recognition method based on dynamic neurons according to claim 2, characterized in that, When initializing the configuration of newly generated neurons, the new neurons Coverage parameters and coverage radius Specifically set as follows: in, For new neurons The center vector, For new neurons The center vector of the parent element, This is the distance function.

4. The open environment pattern recognition method based on dynamic neurons according to claim 1, characterized in that, A Gaussian function is used as the response function of neurons in a structural developmental neural network, i.e., if a new input signal is added... It landed on the neuron Within the coverage area, the neuron For input signal The response is: in, The mean of a Gaussian function and is a neuron The center vector, For neurons Coverage parameters, This is a distance function used to calculate the distance between the input signal and the center vector of the neuron.

5. The open environment pattern recognition method based on dynamic neurons according to claim 4, characterized in that, During the competitive learning phase, the neuron with the largest local response wins, and the index of the winning neuron is obtained according to the following formula. : in, The coverage area encompasses the input signal. The set of subscripts of neurons.

6. The open environment pattern recognition method based on dynamic neurons according to any one of claims 1 to 5, characterized in that, In the aforementioned structural developmental neural network, the root node of each tree is the root neuron, which is the first layer of hierarchical clustering. The child neurons of the root neuron are in the second layer, and so on; the layer number is less than the clustering layer number. Each mature neuron represents a class, labeled with the neuron's subscript, and the center vector serves as the class prototype. Immature neurons and those in layers greater than [a certain number] are also represented. The class label of a neuron is directly inherited from its parent label.

7. The open environment pattern recognition method based on dynamic neurons according to any one of claims 1 to 5, characterized in that, Winning neurons Regarding input signals Change in information saturation Based on the input sample With winning neurons The similarity is calculated as follows: in, The constraint coefficient has a value of , Reset the information saturation threshold.

8. The open environment pattern recognition method based on dynamic neurons according to any one of claims 1 to 5, characterized in that, In the aforementioned structural developmental neural network model, if the information saturation of a neuron has not reached saturation, the index of the winning neuron is output. k And update the center vector of the winning neuron so that it moves toward the input data, the center vector being updated specifically by the following formula: in, For the updated center vector, The center vector before the update. For the winning neuron Regarding input signals The change in information saturation This represents the updated information saturation.

9. The open environment pattern recognition method based on dynamic neurons according to any one of claims 1 to 5, characterized in that, The structural developmental neural network model also includes a mother element, and the competitive layer develops from the mother element. The center vector of the mother element is configured as the origin of the sample space, the coverage parameter is positive infinity, and the information saturation is configured to make it saturated.

Images