Optimization method of structural development neural network for open environment pattern recognition

Abstract

The application discloses a structural development type neural network optimization method for open environment pattern recognition, and steps include: obtaining to-be-recognized data flow input into the structural development type neural network for learning and classification; when the network is in structural development and parameter adjustment, the information saturation of the winning neuron is used to control the neuron parameter, development stage and splitting time, and the information saturation of the winning neuron is continuously adjusted; when the information saturation of the neuron reaches a reset threshold, the neuron matures; when a preset saturation threshold is reached, a new neuron is split out; in the structural development process, an inhibition mechanism of neuron splitting is used; after the information saturation of the neuron reaches saturation, if neuron generation is unsuccessful, the neuron that inhibits neuron generation is used as the winning neuron. The application has the advantages of simple implementation method, good network plasticity and stability, inhibition of network size, and high network development stability.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of pattern recognition in open environment, and particularly relates to a structure development type neural network optimization method for pattern recognition in open environment. BACKGROUND

[0002] Artificial intelligence often runs in the real world and is exposed to non-steady-state information flow in an open environment, and needs to have the ability of continuous learning, that is, integrating new knowledge while retaining previously learned experience, which is usually considered as one of the necessary attributes of future general artificial intelligence. However, the current mainstream advanced deep learning model usually relies on a complete training set available at one time, and when learning new knowledge, the new information will cover the previously learned knowledge, often leading to catastrophic forgetting, and it is difficult to continuously learn in a non-steady-state environment. Therefore, in recent years, the problem of continuous learning has attracted a lot of attention again, but the current continuous learning methods are still mostly concentrated on supervised learning. However, the open environment often lacks prior information and external supervision information, and a large amount of data usually has no class label, so each data is assigned a class label. Therefore, the current continuous learning method is usually contrary to the characteristics of the open environment.

[0003] Unsupervised continuous learning aims to learn from non-steady-state unlabeled data flow, in which the distribution of data or the number of classes changes over time. However, neural network models often face the stability-plasticity dilemma in unsupervised continuous learning, that is, when a learning network tries to adapt to the changing environment in real time, it is difficult to balance the plasticity of adapting to the rapidly changing environment and the stability of retaining previously learned knowledge.

[0004] Most of the current structure development type neural networks are designed based on the classic competitive neural network self-organizing map (SOM) and neural gas (NG). Representative structure development type neural networks such as growing neural gas (GNG) improve the network dynamics based on NG, but still do not have the ability to track rapidly changing non-steady-state data. There are currently a variety of methods based on structure development type networks in different continuous learning scenarios, for example, the knowledge consolidation (KC) algorithm based on GWR video action recognition class incremental learning separates the new and old class feature manifold; the pre-trained CNN is combined with SOINN and applied to the continuous target recognition scenario; the general associative memory (GAM) based on SOINN. The above expansion algorithms introduce the incremental learning property of the structure development type neural network into the classification algorithm, making the classifier applicable to the continuous learning scenario.

[0005] The structural development type neural network effectively improves the plasticity and stability of the neural network by using the mechanism of adaptive development of network topology structure, but when applied to pattern recognition in an open environment such as image data stream, the following problems may exist:

[0006] 1. The open environment puts higher requirements on the real-time performance of the neural network model, and since the training and testing stages are not distinguished, the learning stage and the use stage of the network model need to be seamlessly connected. The traditional method of adding and deleting neurons based on the connection age threshold and the pre-set period parameter may cause the number of neurons to fluctuate during the development of the network and the classification results to be unstable, which is not conducive to the real-time performance of the model.

[0007] 2. In the learning in the open environment, the neural network needs to be able to recognize things more and more precisely, that is, as the input information increases, the learning results are subdivided, and the existing structural development type neural network cannot achieve the subdivision of learning results as the input information increases.

[0008] 3. The traditional structural development type neural network only focuses on the incremental learning problem of the network, and does not consider the network size control problem. In the network development process, if the neurons grow unlimitedly, it will lead to the problem of being unable to effectively control the network size, and when the same type of sample appears frequently, the corresponding neurons are also prone to uncontrolled division, so the development stability of the traditional structural development type neural network using the traditional development method is not high. SUMMARY

[0009] The technical problem to be solved by the present application is to provide a structural development type neural network optimization method for open environment pattern recognition, which has the advantages of simple implementation, good plasticity and stability, and can inhibit the network size, so that the network development stability is high.

[0010] To solve the above technical problems, the technical solution provided by the present application is:

[0011] A structural development type neural network optimization method for open environment pattern recognition, comprising the following steps:

[0012] Acquiring the input of the image or text data stream to be recognized into the structural development type neural network for learning and classification;

[0013] The structural development type neural network adjusts the information saturation degree of the winning neuron during the competition, and splits a new neuron when the information saturation degree of the winning neuron reaches a preset saturation threshold.

[0014] During the structural development of the structural development type neural network, an inhibition mechanism for neuron splitting is set, after the information saturation degree of the neuron reaches saturation, it is determined whether to generate a new neuron, if the neuron generation is unsuccessful, the target neuron which inhibits the generation of the new neuron is taken as the winning neuron, and the attributes of the target neuron are adjusted.

[0015] Further, during the development process of the structural development type neural network, the attributes and lateral connections of the winning neuron are adjusted according to the development stage and the input signal, wherein the development stage of the neuron is determined according to the information saturation degree, and the development stage of the neuron is divided into an immature stage, a mature stage and a parent stage according to the information saturation degree.

[0016] Further, when the neuron is just generated, the initial saturation degree is set as the information saturation degree change amount of the parent neuron, and if there is no parent neuron, it is set as the maximum value S r / 4 of the information saturation degree change amount of the immature neuron, wherein S r ∈(0, 1) is the reset threshold of the information saturation degree; the stage from the generation of the neuron to the information saturation degree reaching the reset threshold is the immature stage, and the immature neuron inherits the class representation of the parent neuron; when the information saturation degree exceeds the reset threshold, it is determined that the neuron enters the mature stage, and the neuron represents an independent class after maturation; when the information saturation degree reaches the saturation threshold, the winning neuron splits to generate a new neuron, the winning neuron is the parent neuron of the new neuron, and the new neuron is the child neuron of the winning neuron, at this time the winning neuron enters the parent stage.

[0017] Further, the winning neuron n k The information saturation degree change amount Δs i of the input signal x k,i is determined according to the similarity o i between the input signal x k and the winning neuron n k,j , and the similarity o k,j is calculated by using a Gaussian radial basis function, that is:

[0018]

[0019] Wherein, d() is the calculation of the input signal x i and the current winning neuron n kthe center vector μ of k the distance function σ k the current neuron n k the coverage parameter of.

[0020] Further, the information saturation s k The update calculation formula is:

[0021]

[0022] s k ′=s k +Δs k,i

[0023] Wherein, s k ′ is the updated information saturation, s k is the information saturation before updating, S r is the reset threshold, and a=(1-S r ) is the proportional coefficient, a and S r are negatively correlated, and the information saturation change amount Δs k,i is positively correlated with S r when the neuron is immature, and Δs k,i is negatively correlated with S r when mature.

[0024] Further, in the structural development neural network, the new neuron is centered on the current input signal, when the information saturation of the winning neuron reaches the saturation threshold, the new neuron is split from the winning neuron, the coverage parameter of the new neuron is initialized as one third of the distance between the center vectors of the child and parent neurons, and the information saturation is initialized as the information saturation change amount of the parent neuron; when the input signal does not fall into the coverage domain of any neuron, a new root neuron is generated from the input signal, the coverage parameter of the generated new neuron is initialized as one third of the distance from the center vector to the center vector of the nearest neighbor neuron, and the information saturation is initialized as S r / 4, if it is the first generated neuron, it is initialized as one third of the distance from the center vector to the origin of the sample space, and the information saturation is initialized as S r / 4.

[0025] Further, the structural development neural network further includes, during the structural development process, directly decaying to the reset threshold S r when the information saturation of the neuron reaches the saturation threshold, to realize the information saturation decay forgetting mechanism, wherein the information saturation of the mature neuron except the winning neuron and the parent neuron of the branch where the winning neuron is located decays towards the reset threshold, and the decay process is:

[0026] s j′ = s j -τ(s j -S r ),S r ≤s j ,τ∈(0,1)

[0027] wherein τ is information saturation decay rate, s j ′ is decayed information saturation, s j is pre-decayed information saturation.

[0028] Further, the structural development type neural network discriminates whether to generate a new neuron in the structural development process according to the following rule: when the distance between the center vector of a sibling neuron and the center vector of the neuron to be generated is less than the coverage domain radius of one of the neurons, i.e. the center vector of the pre-generated neuron or the center vector of the existing sibling neuron falls into the coverage domain of the other, the current neuron is not generated.

[0029] Further, the center vector and the coverage domain parameter in the structural development type neural network are updated according to the following formula:

[0030]

[0031] wherein μ k and σ k represent the center vector and the coverage domain parameter before updating, μ p represents the center vector of the parent neuron, μ k ′, σ k ′ and s k ′ represent the center vector, the coverage domain parameter and the information saturation after updating respectively, Δs k,i is the information saturation change amount, lay k is the layer number in the parent-child relationship tree; MT k is the maturation mark, taking the value of 0 or 1, and the updating formula is:

[0032]

[0033] Further, in the structural development type neural network, the parent-child neuron connection updating mechanism is used for updating, when the neuron n j is mature, if the center vector moves out of the coverage domain of the parent neuron, i.e. d(μ j ,μ p )>r p , wherein μ j is the center vector of the neuron n j , μ p is the center vector of the parent neuron, and r p is the coverage domain radius of the parent neuron, it is determined that the neuron n jIf the neuron cannot become a subdivision of the class represented by the current parent neuron, i.e. is no longer a child of the current parent neuron, the current parent-child connection is deleted, the neuron n j is promoted to the next layer of the cluster, and a new parent neuron is sought in the parent layer of the current layer, and if no new parent neuron is found, the neuron is promoted to the next layer, until a new parent neuron is found or the neuron becomes the root neuron of the first layer; the judgment whether the neuron is a parent neuron is based on whether the center vector μ j is within the coverage domain of the target neuron for which the judgment is required, and if yes, the target neuron is determined to be the new parent neuron sought.

[0034] Compared with the prior art, the present application has the following advantages:

[0035] 1. The present application realizes the optimization of the structural development type neural network by adopting the new neuron generation rule based on the coverage domain and the information saturation degree, uses the information saturation degree of the neuron and the maturation stage of the neuron based on the information saturation degree, simulates the long-term and short-term memory of the human brain, so that the network model has plasticity and stability, the parent-child relationship generated by the splitting of the neuron makes the network can subdivide the original categories as the number of learning samples increases, simulates the gradual fine cognitive process of human understanding of things; at the same time, the inhibition mechanism between the brother neurons is used to limit the excessive growth of the neurons, so that the size of the network can be controlled, and the network can develop incrementally, so that it can adapt to the continuous learning scenarios of the instance increment type, the class increment type and the task increment type.

[0036] 2. The present application further sets an information saturation degree decay forgetting mechanism in the network development process, directly decays the information saturation degree of the neuron to the reset threshold when the information saturation degree of the neuron reaches the saturation threshold, and the information saturation degree of the mature neurons except the winning neuron and the parent neuron of the branch where the winning neuron is located will decay towards the reset threshold, which can prevent the frequent splitting of the parent neuron and further improve the stability of the model.

[0037] 3. The present application further introduces the number of cluster layers, so that the user can independently select the required degree of category subdivision, and the network can subdivide as the number of learning samples increases, so that the recognition requirements of different degrees of category subdivision can be met.

[0038] 4. The present application further sets the updating mode of the center vector, so that the input signals with different distances from the center vector of the neuron have different contributions to determining the internal attributes of the neuron, in the immature stage of the neuron, the signals close to or far from the center vector of the neuron have small contributions, and vice versa, in the mature stage, the signals far from the center vector of the neuron have large contributions, which promotes the splitting of the neuron, and at the same time, as the information saturation degree increases, the adjustment amplitude of the internal attributes of the neuron gradually decreases, so that the neuron tends to be stable, effectively simulating the transformation of the short-term memory to the long-term memory in the human brain.

[0039] 5、The application further sets the coverage domain parameter updating mode, the coverage domain parameter keeps the initial state unchanged, when the neuron enters the mature stage from the immature stage, the coverage domain parameter is updated to one third of the distance between the center vector and the parent neuron center vector, and keeps unchanged in the subsequent network operation process, which can avoid the instability caused by the dramatic oscillation of the coverage domain radius, enhance the stability of the neuron, and reasonably divide the local information of the parent neuron and the child neuron. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 is the implementation flowchart of the structure development type neural network optimization method of the open environment pattern recognition of the embodiment.

[0041] Figure 2 is the input signal and neuron center vector similarity degree and information saturation increment curve obtained in the embodiment.

[0042] Figure 3 is the output layer neuron category label principle diagram of the embodiment.

[0043] Figure 4 is the input sample diagram obtained in the specific application embodiment.

[0044] Figure 5 is the principle diagram of the error parent-child relationship caused by different sample input sequences in the specific application embodiment.

[0045] Figure 6 is the neuron number result diagram generated by the ESDNNIS of the application in the instance increment (corresponding to (a)) and class increment (corresponding to (b)) learning scene in the specific application embodiment.

[0046] Figure 7 is the original artificial data set diagram used in the specific application embodiment.

[0047] Figure 8 is the category subdivision result diagram of the ESDNNIS of the application in the specific application embodiment.

[0048] Figure 9 is the subdivision degree result diagram of the clustering layer number in the specific application embodiment.

[0049] Figure 10 is the neuron number and recognition clustering number change result diagram of the network in the instance increment and class increment learning scene in the development process in the specific application embodiment.

[0050] Figure 11Fig. 1 is a schematic diagram of the clustering accuracy results obtained in a specific application example. DETAILED DESCRIPTION

[0051] The present application is further described below in conjunction with the accompanying drawings and specific preferred embodiments, but the scope of the present application is not limited thereby.

[0052] As shown in Fig. 1, the steps of the structural development type neural network optimization method for open environment pattern recognition in the present embodiment include: Figure 1

[0053] S01. Inputting an image or text data stream to be recognized into a structural development type neural network for learning and classification, wherein each neuron in the structural development type neural network is provided with an information saturation attribute;

[0054] S02. In the structural development and parameter adjustment of the structural development type neural network, the information saturation of the winning neuron is used to control the neuron parameters and the splitting timing, and the information saturation of the winning neuron is continuously adjusted. The information saturation is calculated according to the winning frequency and the information carrying capacity of the neuron. When the information saturation of the neuron reaches a preset saturation threshold, a new neuron is split;

[0055] S03. In the structural development process of the structural development type neural network, an inhibition mechanism for neuron splitting is set. After the information saturation of the neuron reaches saturation, it is determined whether to generate a new neuron. If the neuron generation is unsuccessful, the target neuron that causes inhibition of neuron generation is taken as the winning neuron, and the attributes of the target neuron are adjusted.

[0056] ​The embodiment configures an enhanced structural developmental neural network with information saturation (ESDNNIS) for open environment pattern recognition. The network can adaptively adjust the network topology to adapt to unsupervised continuous learning of non-steady state data flow based on competitive learning and neuron splitting mechanism. The splitting mechanism of information saturation enables the structural developmental neural network to split new neurons when needed without relying on prior knowledge of periodic parameter addition or pruning neurons, which can improve the stability and real-time performance of the model. Based on the parent-child relationship generated by neuron splitting, the model can also simulate the human cognitive process to subdivide classes during continuous learning, enabling the network to continuously subdivide learning results as the amount of input information increases. The network can adaptively adjust the network topology to adapt to unsupervised continuous learning of non-steady state data flow. On the basis of the above structural developmental neural network, the neuron splitting inhibition mechanism is used. When the information saturation of the neuron reaches saturation but the generation of new neurons is unsuccessful, the neuron that inhibits the generation of new neurons is taken as the winning neuron, which can limit the excessive growth of neurons and avoid the uncontrolled splitting of corresponding neurons when the same type of sample frequently appears, thereby ensuring the stability of network development.

[0057] The embodiment adds information saturation attribute to neurons in ESDNNIS to represent the amount of information carried by neurons and determine the development stage of individual neurons and whether to split. Specifically, the information saturation of the winning neuron increases with the amount of information brought by input data, the neuron matures gradually, the adjustment amplitude becomes smaller and smaller, and tends to be stable. This process simulates the long and short term memory mechanism of human brain and can improve the stability of neural network model. When the information saturation reaches the splitting threshold, it means that the information carried by the neuron is already rich enough, and the neuron splits to generate new neurons, which can simulate the subdivision process in human cognition and increase the plasticity of the model.

[0058] ESDNNIS is a growing competitive neural network composed of an input layer and an output layer. The input layer accepts the feature vector of the recognition target, and the output layer develops from zero. Each neuron has three internal attributes: a center vector, a coverage domain parameter, and an information saturation degree. The neuron uses a Gaussian function as its response function, and the mean and standard deviation of the Gaussian function are the center vector and coverage domain parameter of the neuron, which change together with the adjustment of the neuron. ESDNNIS learns and classifies the input data stream in an incremental competitive learning manner. When an input signal is transmitted to the input layer, only the neurons in the output layer that fall within their coverage domain respond to the input signal. The neurons that respond to the input signal compete with each other according to the output value of their response function to produce a winning neuron. The network output is the class to which the winning neuron belongs. If the input signal does not fall within the coverage domain of any neuron, a new neuron is generated, achieving abnormal detection or obtaining new cognition.

[0059] After classifying the signal, ESDNNIS enters the update phase of neuron attributes and lateral connections. The attributes and lateral connections of the winning neuron will be adjusted according to the development stage and the input signal. The development stage of the neuron is determined by the information saturation degree. The larger the information saturation degree, the more mature the neuron, the smaller the update amplitude, and the more stable the neuron. This effectively simulates the long-term and short-term memory mechanism in the human learning process. When the information saturation degree of a neuron reaches the saturation threshold, it means that the neuron has carried enough information, or the frequency of sample features in the neuron's coverage domain is relatively large, causing the neuron to split into a new neuron. The new neuron has a tendency to subdivide the original class, effectively simulating the process of human cognition becoming more and more refined with the increase of knowledge. This embodiment further sets up a forgetting mechanism for the neuron to avoid frequent splitting of the neuron and ensure the stability of the network model.

[0060] For convenience of description, this embodiment first gives the following symbols and definitions: input signal x i =(x i1 ,x i2 ,...,x id ) represents the i-th signal input to the network in the sample space χ, with d-dimensional features. The output layer is a set of neurons , which is described as follows: where i represents the number of signals that the model has processed, and element n j represents the j-th neuron, and subscript k i represents the number of neurons generated after the algorithm processes the i-th information. Each neuron is a three-tuple n j =(s j ,μ j ,σ j ), where s j represents the information saturation degree of the neuron, μ jInformation saturation, μ j n j The center vector, σ j n j The coverage area parameter. The coverage area radius is denoted as r. j The coverage area D of the neuron j ={x|d(x,μ j )≤r j}, where d() is the distance function. Information saturation involves resetting the threshold S. r The neuronal division mechanism leads to parent-child relationships between neurons, thus forming a hierarchical tree structure T. ESDNNIS clusters neurons based on T and the user-defined number of clustering layers C, where n... j The category label is denoted as l k .

[0061] When an input signal is passed to the input layer, neurons in the output layer only respond to input signals that fall within their coverage area. If the input signal does not fall within the coverage area of ​​any neuron (i.e., the current output layer has no neuron capable of representing the input), a new neuron is generated directly with the input signal as its center vector. The neuron with the largest local response wins, and the network outputs its index as the category of the current input signal, then proceeds to the update phase. Each time a neuron responds to an input signal and becomes the winning neuron, the "information" carried by that input causes an increase in the information saturation of the winning neuron. Winning neuron n k Regarding the input signal x i The change in information saturation is denoted as Δs k,i Its magnitude depends on the input signal x i With the winning neuron n k Similarity o k,j In this embodiment, the Gaussian radial basis function is used for calculation, that is:

[0062]

[0063] The change in information saturation Δs at different developmental stages k,iThe calculation method of the information saturation degree s is different. In the immature stage, the input signal too close to the center vector of the winning neuron has too little difference with the center vector of the neuron, which leads to little new information being introduced. In the extreme case, when the input signal coincides with the center vector of the neuron, the amount of information introduced is 0. The input signal too far from the center vector of the winning neuron has a greater possibility of being represented by other potential neurons, and has little effect on the development of the winning neuron, so the adjustment of the information saturation degree should be minimal. In the mature stage, the neuron can better represent the local information, so the input signal closer to the center vector of the neuron has less “amount of information”, and the input signal farther from the center vector of the neuron introduces information with greater difference, leading to a greater change in the information saturation degree, prompting the neuron to split new neurons to represent potential new categories. In summary, in the embodiment, the information saturation degree s k The update calculation formula is as follows:

[0064]

[0065] s k ′=s k +Δs k,i (3)

[0066] wherein s k ′ is the updated information saturation degree, S r is the reset threshold, and a = (1-S r ) is the proportional coefficient.

[0067] The above a and S r are negatively correlated, and further, the change amount Δs k,i of the information saturation degree when the neuron is immature is positively correlated with S r , and when the neuron is mature, Δs k,i is negatively correlated with S r , which can reduce the sensitivity of the model classification effect to the adjustable parameter S r and improve the robustness of the ESDNNIS. When S r is too large, the information saturation degree of the neuron in the immature stage is more difficult to reach S r and further difficult to mature, and the information saturation degree of the neuron in the mature stage is more likely to reach the saturation threshold and further more likely to split. Through the setting of the proportional coefficient a, when S r is too large, the information saturation degree in the immature stage grows faster and in the mature stage grows slower, and vice versa, thereby to a certain extent, offsetting the influence of S r .

[0068] In this embodiment, the developmental stages of neurons are specifically divided into immature stage, mature stage, and parent stage based on information saturation. When a neuron is first generated, it has a certain initial saturation, which is considered to be the amount of information in the input signal that causes the generation of the new neuron. The initial saturation is set as the change in information saturation of the parent (as shown in equation (2)). If there is no parent, it is set as the maximum value S of the change in information saturation of the immature neuron. r / 4, where S r ∈(0,1) is the reset threshold for information saturation, an adjustable constant parameter. The stage from neuron generation to the information saturation reaching the reset threshold is the immature stage, where immature neurons inherit the category representation from their parent neurons. When information saturation exceeds the reset threshold, the neuron enters the mature stage, and the mature neuron independently represents a category. When information saturation reaches the saturation threshold (value 1), the winning neuron splits to generate a new neuron; that is, the winning neuron becomes the parent of the new neuron, and the new neuron becomes the child of the winning neuron. At this point, the winning neuron enters the parent stage.

[0069] To better understand the adjustment mechanism of information saturation at different stages, let's take a one-dimensional input as an example, such as... Figure 2 As shown, in the immature stage, the closer the input signal x is to the neuron's center vector, the closer the similarity is to 1, indicating that the neuron can represent the current input well, and Δs tends to 0, meaning the information saturation increase is small. If the input signal is far from the center weight, the similarity is close to 0, indicating that the neuron cannot represent the input well, and Δs tends to 0. Furthermore, only when the input signal is below the double peak of the similarity curve in the figure is it considered that the input signal can better reflect local information in immature neurons, providing a larger change in information saturation and promoting neuron maturation. In the mature stage, the change in information saturation introduced by input signals closer to the center vector tends to 0, while input signals farther from the center vector introduce more changes in information saturation, thus promoting neuron splitting.

[0070] This embodiment also includes setting an information saturation decay and forgetting mechanism during network development, which directly decays the information saturation of neurons to a reset threshold S when the information saturation reaches a saturation threshold. r The information saturation of mature neurons, excluding the winning neuron and its parent neurons in the branch, will decay towards the reset threshold. The decay process is as follows:

[0071] s j ′=s j -τ(s j -S r ), S r ≤s j , τ∈(0,1) (4)

[0072] where τ is the information saturation decay rate, s j is the decayed information saturation, s j is the pre-decayed information saturation. The neuron forgetting mechanism prevents frequent division of parent neurons, which can further improve the stability of the model.

[0073] In order to adapt to the dynamic changes of input signals, the ESDNNIS dynamically adds or adjusts neurons according to the input signals and the information saturation of the winning neuron. In this embodiment, there are two cases for the generation of new neurons. One is that when the information saturation of the winning neuron reaches the saturation threshold (the value is 1), the winning neuron divides into a new neuron. The other is that when the input signal does not fall into the coverage domain of any neuron, a new neuron is generated from the input signal. This kind of neuron generated in this way is called root neuron, and the immature root neuron is called noise neuron. In both cases, the new neuron is centered on the current input signal as the center vector, while the initialization of the coverage domain parameter and the information saturation is different. The coverage domain parameter of the new neuron divided from the winning neuron is initialized as one third of the distance between the center vectors of the child and parent neurons, and the information saturation is initialized as the change of the information saturation of the parent neuron. In the latter case, the coverage domain parameter of the new neuron generated is initialized as one third of the distance from the center vector to the center vector of the nearest neighbor neuron, and the information saturation is initialized as r / 4, especially if it is the first generated neuron, the information saturation is initialized as one third of the distance from the center vector to the origin of the sample space, and the information saturation is initialized as. The information saturation is initialized as the maximum value of the change of the information saturation of the immature neuron. Therefore, the initialization formula of the new neuron is:

[0074] n1 = {S r / 4, x i , d(x i ,0) / 3} (5)

[0075]

[0076] At the same time, the properties of the parent neuron in this embodiment are updated in the following way: the center vector and the coverage domain parameter remain unchanged, and the information saturation is directly decayed to the reset threshold.

[0077] The ESDNNIS of the embodiment can have the following advantages by using the neuron splitting mechanism: first, the model has the ability to discover potential new classes and subdivide classes with higher occurrence frequency; second, compared to adding neurons near the neuron with the largest accumulated local error, which involves global calculation, the splitting mechanism based on information saturation is the behavior of a single neuron, which improves the parallelism of the ESDNNIS operation and thus improves the operation efficiency; third, the ESDNNIS splits new neurons when needed, instead of adding new neurons at places with large accumulated error by using a periodic parameter that requires certain prior knowledge, thus improving the real-time performance of the model.

[0078] The embodiment further sets an inhibition mechanism for neuron splitting during the development of the network to prevent overgrowth of neurons. After the information saturation of a neuron reaches saturation, it is determined whether to generate a new neuron. The determination rule is: when the distance between the center vector of a sibling neuron and the center vector of the neuron to be generated is less than the coverage domain radius of one of the neurons, i.e., the center vector of the pre-generated neuron or the center vector of the existing sibling neuron falls within the coverage domain of the other, the neuron is not generated. If the neuron generation is unsuccessful, the neuron that inhibits the generation of the neuron is taken as the winning neuron, and the related attributes of the neuron are further adjusted.

[0079] In the embodiment, when the input signal is x i , and the winning neuron is n k , the updating method of the center vector and the coverage domain parameter is as follows:

[0080]

[0081]

[0082] wherein μ k and σ k represent the center vector and the coverage domain parameter before updating, μ p represents the center vector of the parent neuron, μ k ', σ k ', and s k ' represent the center vector, the coverage domain parameter, and the information saturation after updating, Δs k,i is the change amount of the information saturation, lay k is the layer number in the parent-child relationship tree; MT k is a maturity marker, taking values of 0 or 1, and the updating formula is:

[0083]

[0084] The value of μ k above is equivalent to triggering the neuron n kThe weighted average of the winning input signals approximates the neuron n using the sample mean k The estimation of the expectation of the represented local information records the local first moment information of the neuron, but the ratio of the change of information saturation to the total is introduced as the weight instead of the average weight, so that the input signals with different distances to the center vector of the neuron have different contributions to the determination of the internal properties of the neuron. In the immature stage of the neuron, the signals close to or far from the center vector of the neuron have small contributions, and vice versa. In the mature stage, the farther the signal is from the center vector of the neuron, the greater the contribution, which promotes the splitting of the neuron. Moreover, as the information saturation increases, the adjustment range of the internal properties of the neuron gradually decreases, and the neuron tends to be stable, simulating the transformation of short-term memory to long-term memory in the human brain. For the coverage domain parameter, since the neurons in the immature stage carry less information, they cannot well represent the local information. In order to avoid the instability caused by the dramatic oscillation of the coverage domain radius, the coverage domain parameter remains unchanged in the initial state. When the neuron enters the mature stage from the immature stage, MT changes from 0 to 1, and the coverage domain parameter is updated to one third of the distance from the center vector to the center vector of the parent neuron, and remains unchanged in the subsequent network operation. Through the above way, the stability of the neuron can be enhanced, and the parent and child neurons can be reasonably divided according to the 3-sigma rule.

[0085] The center vector of each neuron can be used as a prototype in clustering, and the number of prototypes does not need to be specified in advance but is dynamically adjusted according to the input signal. The parent-child relationship formed by the splitting of the neuron establishes a tree-type hierarchical structure, and the child neurons subdivide the class represented by the parent neuron. The user can set the number of clustering layers according to the required clustering accuracy. As shown in Figure 3 The root node of each tree is the root neuron, which is the first layer of hierarchical clustering. The child neurons of the root neuron are the second layer, and so on. Assuming that the user sets the number of clustering layers to C, each mature neuron at the layer less than C represents a class, and the class label is the neuron subscript. The center vector serves as the prototype of the class, and the class label of the immature neuron and the neuron at the layer greater than l directly inherits the parent label. The class label of the neuron generated by the input signal that does not fall within the coverage domain of any neuron, i.e., the noise neuron, is the abnormal label -1, meaning that there is no neuron in the current output layer that can well represent the input. This input is considered as an outlier (new knowledge) by the ESDNNIS, so that the ESDNNIS has the function of anomaly detection (discovering new knowledge).

[0086] The embodiment further adopts an updating mechanism for the connection of parent and child neurons to alleviate the problem that the randomness of the sample input order in continuous learning may lead to incorrect parent-child relationship. Taking two-dimensional two-class samples as an example, the example sample set is as follows: Figure 4As shown, there is a slight overlap between the two classes of samples, and the input order is A class first and B class second. Different input orders of B class samples will lead to different parent-child neuron relationships, and in turn, different clustering results. For example, Figure 5 As shown, where (a) corresponds to MT b = 0, 1 b = -1, (b) corresponds to MT b = 1, 1 b = b, (c) corresponds to MT b = 0, 1 b = a, and (d) corresponds to MT b = 1, 1 b = a. If n b is generated by a sample point outside the covering domain of n a , then n b has no parent-child relationship with n a , and the class label of n b is initialized to -1, which is a noise node (e.g., (a) in Figure 5 ). When n b is mature, the class label is updated to its own subscript (e.g., (b) in Figure 5 ). If n b is generated by a sample located in the covering domain of n a , then n b is a child neuron of n a , and the label is inherited from the parent neuron (e.g., (c) in Figure 5 ). Since the number of clustering layers is 1, the class label of n b after maturation is still the class label of the parent neuron, i.e., A class and B class are treated as one class (e.g., (d) in Figure 5 ). In summary, different input orders lead to different parent-child relationships, which in turn lead to different class labels of n b after maturation, and thus different clustering results.

[0087] To alleviate the confusion caused by the randomness of the input order, the present embodiment further employs an updating mechanism for the parent-child neuron connection. When neuron n j is mature, if the center vector moves out of the covering domain of the parent neuron, i.e., d(μ j , μ p ) > r p , where μ p is the center vector of the parent neuron and r p is the covering domain radius of the parent neuron, it means that this neuron cannot become a subdivision of the class represented by the parent neuron, i.e., it is no longer a child neuron of the parent neuron, then the current parent-child connection is deleted, the clustering layer of this neuron is promoted by one, and a new parent neuron is found in the parent neuron layer of this layer. The criterion for whether to be a parent neuron is d(μ jwhether in the coverage domain of the neuron. If no new parent is found, then promote one level again until a new parent is found or become the root neuron of the first level. The judgment basis for whether it is a parent neuron when searching for a new parent in the parent layer of the current layer is the center vector μ j whether in the coverage domain of the target neuron for which the judgment is required, and if so, determine that the target neuron is the new parent to be found.

[0088] To verify the effectiveness of the present application, the effectiveness of the ESDNNIS of the present application is verified on an artificial image dataset and a real-world dataset, respectively, and compared with typical structural development neural networks GNG, GWR and SOINN. The three global parameters in the ESDNNIS of the present application are reset threshold S r , information saturation decay rate τ and clustering layer number C, S r is used to reflect the winning frequency required for a neuron to reach the mature stage and define the callback value of information saturation after splitting, S r is larger, the center vector of the neuron is relatively stable in the mature stage but is prone to splitting; τ is used to define the information saturation decline rate of the mature neuron, and when τ is relatively large, the information saturation declines relatively fast, and the neuron is relatively not prone to splitting; C is used to determine the subdivision degree of the clustering result, and when C is relatively large, the subdivision degree of the clustering result is relatively high. Since the influence of τ on the ESDNNIS is very simple and intuitive, below, the influence of S r and C on the network size and the clustering result will be tested respectively.

[0089] In this embodiment, the test dataset is divided into five groups according to the sample quantity, each group consisting of five clusters obeying different normal distributions, and the sample quantity of each cluster in each group is equal, being 100, 200, 400, 800 and 1600 respectively. According to the definition of the reset threshold, the neurons with information saturation greater than the reset threshold tend to be stable and cannot exceed the saturation threshold 1, so the influence of S r ∈[0.55, 0.95] on the neuron quantity of the ESDNNIS is tested under the two learning scenarios of instance increment and class increment. Since the ESDNNIS is sensitive to the sample input order, each group is tested 10 times according to different sample input orders, and the final result of each group is the average value of the 10 test results. Figure 6 To generate the neuron quantity average value of the ESDNNIS with the change of the reset threshold under the instance increment (corresponding to (a) in Figure 6 ) and class increment (such as (b) in Figure 6 ), the shadow area represents the standard deviation, and there are curves with similar shapes under the two learning scenarios of instance increment and class increment, that is, high in the middle and low at both ends, when S rwhen ∈ [0.7, 0.85], the number of neurons is small and enough to represent the test dataset, and in this interval, the number of neurons generated is relatively insensitive to the setting of S r , while at the two ends of the curve, the opposite is true. Therefore, too large and too small reset thresholds will cause frequent splitting of neurons, leading to instability of the network. The above verifies that the number of neurons generated by the ESDNNIS under different reset thresholds S r , to some extent, can reflect the influence of the reset threshold S r on the ESDNNIS.

[0090] In the following experiments, the value of S r is set to 0.85. The splitting characteristics of neurons enable the ESDNNIS to have the ability to subdivide classes. In practical applications, users require different degrees of clustering subdivision, and the degree of subdivision of the clustering result can be determined by adjusting the number of clustering layers C. The subdivision ability of the ESDNNIS is shown in Figure 5 , Figure 7 The original artificial dataset is composed of 5 clusters obeying normal distribution, and there is overlap between clusters. Overlapping clusters can be regarded as constituting a large cluster, so this dataset can be regarded as composed of two large clusters, and the two large clusters are composed of 2 and 3 sub-clusters respectively. The learning mode is instance incremental. In the initial stage of learning, when the number of input samples is small, the number of generated neurons is small, and the granularity of classification is coarse, as shown in Figure 8 (a), when the number of input samples is 8, only one class can be recognized. With the increase of the number of learning samples, when the information saturation degree of neurons in the ESDNNIS reaches the saturation threshold, new neurons will be split, leading to subdivision of the original class. The newly generated neurons can represent more fine-grained classes, simulating the human cognitive process, as shown in Figure 8 (b) and (c). When the number of learning samples is 180, the ESDNNIS can recognize two large classes, and when the number of learning samples grows to 600, the ESDNNIS can recognize the sub-classes in the large classes.

[0091] The influence of the number of clustering layers C is shown in Figure 9 , which shows the clustering results when C is 1, 2, and 3 respectively. The test dataset has the same distribution as Figure 6 , except that the number of samples in each cluster is 400. As shown in Figure 3 , when C = 1, the clustering result is one class, when C = 2, the clustering result is three classes, and when C = 3, the clustering result has seven classes (including parent classes). It can be intuitively seen that using a larger number of clustering layers can obtain a more fine-grained clustering result, and the specific number of clustering layers can be set according to actual needs. In addition, from Figure 9It can be seen that ESDNNIS can well reflect the categories contained in the learning samples at different levels of subdivision, which conforms to the intuitive feeling of the human brain.

[0092] This embodiment further tests the performance of ESDNNIS on artificial data sets. Since two-dimensional data has the advantages of visibility and perceptibility, and normal distribution is the most common distribution in nature, an artificial data set composed of 5 clusters obeying two-dimensional normal distribution is constructed. Each cluster has different size and density and the distance between clusters (A and B clusters have slight overlap), which can better represent the complexity of sample distribution in open environment. The variance ratio of clusters A, B, C, D and E is 3:5:4:4:6, and the sample quantity ratio is 2:2:2:1:2. It can be seen that C and D have the same variance but different sample quantities, D cluster has small size, C and E have the same sample quantity but different variances, and E cluster has more sparse sample distribution. The samples in each cluster are divided into training set and test set according to the ratio of 4:1. The artificial data set experiment is divided into 6 groups according to 3 learning sample quantities (represented by the sample quantity of A cluster, 50, 200 and 800) and 2 instance increment methods (instance increment and class increment). Since the structured developmental neural network is sensitive to sample input order, 10 experiments are performed for each group with different sample input orders, and the final result is the average value of the evaluation index on the test set. This embodiment uses external evaluation index ACC and internal evaluation index DBI to evaluate the clustering effect. ACC (Accuracy) represents the proportion of correct clustering, and the calculation formula is:

[0093]

[0094] wherein r i ,s i represent the predicted label and the true label respectively, n is the total number of data, and map() is the Hungarian algorithm function to realize the best allocation of labels. The value range of ACC is [0, 1], and the larger the value is, the higher the proportion of correct clustering result is. DBI, also known as classification accuracy index, has the calculation formula:

[0095]

[0096] wherein avg(C) corresponds to the average distance between samples in cluster C, d cen (C i ,C j ) corresponds to the distance between the center points of cluster C i and cluster C j . Obviously, the smaller the value of DBI is, the better the clustering effect is. In this experiment, the hyperparameters of ESDNNIS are taken as S r= 0.85, τ = 0.05, C = 3 for instance incremental learning scenario and C = 1 for class incremental learning scenario. The GNG, GWR and SOINN are tested on the same dataset as a comparison. The maximum age parameter a max = 5, the node insertion period parameter λ = 50 for GNG and SOINN are the same. The clustering accuracy results of the ESDNNIS and the Benchmark on the artificial dataset are shown in Table 1, and the neuron growth and clustering of each structural plasticity neural network on the training set (the sample amount of cluster A is 640) are shown in Fig. 1, where (a) corresponds to the neuron number change in the sample incremental learning scenario, (b) corresponds to the neuron number change in the class incremental learning scenario, (c) corresponds to the cluster number change in the sample incremental learning scenario, and (d) corresponds to the cluster number change in the class incremental learning scenario. Figure 10

[0097] Table 1: Performance comparison of structural plasticity neural networks under different incremental learning scenarios and different input amounts

[0098]

[0099]

[0100] As shown in Table 1, on the artificial dataset with uneven sample distribution and some overlapping clusters, the ESDNNIS proposed in this embodiment achieves the highest accuracy under different sample input amounts and learning scenarios, and has the smallest DBI value and the best clustering effect in the class incremental learning scenario. The other three structural plasticity neural networks perform poorly on this dataset, especially when the sample amount is small, they can only recognize one class and cannot calculate the DBI value. The main reasons are two-fold. First, the connections between neurons in the other three networks are pruned according to the "age". When the age reaches a specified threshold, the connection is removed. When the sample amount is small or the distance between real classes is small, the connections between classes may not be pruned in time, resulting in different classes that cannot be effectively distinguished. Second, GNG, which relies on a new node insertion period, cannot discover new classes in time when the data amount is small. The ESDNNIS does not rely on input samples to connect the nearest neighbor neuron and the second nearest neighbor neuron to represent the same class, but each neuron represents a class. When the neuron information saturation degree reaches saturation or the input sample is outside the coverage domain of all neurons, a new neuron is generated to represent a potential new class, and the neurons are connected according to the parent-child relationship to represent the subdivision of classes. This avoids the problem of not being able to prune the connections between classes in time, making the ESDNNIS real-time and obtaining results that conform to the actual situation, and eliminating the need to set the maximum age parameter, which requires prior knowledge, making the ESDNNIS more suitable for unknown open application scenarios.​

[0101] Figure 10 This reflects the changes in the number of neurons and the number of clusters recognized as a function of the number of input samples for four structural developmental neural networks under two learning scenarios. In the instance-incremental learning scenario, such as... Figure 10 As shown in (a), GNG exhibits a roughly linear increase in the number of neurons without a maximum neuron limit, while GWR and SOINN show a plateauing trend in neuron number growth after reaching a certain level, exhibiting indirect small fluctuations. The ESDNNIS of this invention can rapidly converge to a number consistent with the actual number of classes when the input sample size is small, and remains stable during subsequent sample input. The rapid convergence to the actual number of classes is attributed to the information saturation growth mechanism and the two neuron generation mechanisms, while stability is attributed to the neuron forgetting mechanism. In a class increment learning scenario (one class by one, classes A, B, C, and E contain 640 samples, and class D contains 320 samples), as... Figure 10 As shown in (b), the number of neurons in GNG still shows a linear growth, while the growth curves of the number of neurons in GWR, SOINN, and ESDNNIS are all close to a step shape. When a new class is input, the number of neurons increases and then tends to plateau. The number of neurons and the connections between neurons affect the clustering results, such as... Figure 10 As shown in (c) and (d), fluctuations in the number of neurons lead to oscillations in the number of clusters. GNG, GWR, and SOINN exhibit significant fluctuations in the number of clusters during network development, resulting in unstable clustering results and failing to meet the real-time requirements of some open application scenarios. The ESDNNIS of this invention can obtain realistic clustering results when the number of samples is small during development and remains stable in subsequent development processes. In incremental class learning scenarios, neuron splitting is relatively more frequent, and the number of neurons is unstable. However, the clustering mechanism based on parent-child relationships and the number of clustering layers allows ESDNNIS to still obtain stable clustering results in incremental class learning scenarios.

[0102] The incremental clustering effect of the ESDNNIS of the present application is further tested in the ATT_FACE dataset, and is compared with typical structural developmental neural networks GNG, GWR and SOINN. The ATT_FACE dataset contains 40 different face objects, each object having 10 different pictures. The pictures are taken at different times, different light, facial expression (open / close eyes, smile / not smile) and facial details (wear glasses / not wear glasses). The objects are in an upright frontal position (allowing some side movement). The size of each image is 92x112 pixels, and each pixel has 256 gray levels. 15 face objects are selected from the dataset, and they are divided into two groups A and B, group A containing 10 objects and group B containing 5. The purpose of dividing into two groups is to simulate two different environments in an open environment, and to test the incremental clustering effect of the structural developmental neural network under the condition of environmental change. The image feature vector extraction method is as follows: the bilinear interpolation method is used to adjust the image size to 23x28, and then the PCA method is used to reduce the dimension of the image after size adjustment to obtain a 10-dimensional feature vector. The test is divided into two tasks, each task being divided into a learning stage and an evaluation stage. The first task learning stage training set is the A group training set (10 classes, 10 samples per class), the learning method is class incremental, and Gaussian noise (σ=0.005) is added to the pictures of the A group face objects as the test set in the evaluation stage, and the input order is random input. The training set of the second task learning stage is the B group training set (5 classes, 10 samples per class), and the learning method is instance incremental. In the evaluation stage of the second task, not only the clustering results of the B group by the ESDNNIS are tested, but also whether the learning achievements of the first task are destroyed after learning the B group data. Therefore, the test set in the evaluation stage of the second task is the pictures of the A and B groups of face objects with Gaussian noise (σ=0.005) added, and the input order is random input. Since the structural developmental neural network is sensitive to the input order of the samples, the test is repeated ten times according to different input orders. The average value of the clustering accuracy is taken as the performance indicator.

[0103] The hyperparameter settings of the network in this embodiment are the same as in the artificial dataset experiment. Both the ESDNNIS and the SOINN of the present application have achieved good accuracy (as shown in Figure 11 In the test results of the second task, the clustering accuracy of the ESDNNIS of the present application on the A group objects is almost the same as the test results in the first task, that is, the learning achievements of the ESDNNIS of the present application on the first task are not destroyed by the second learning task. In addition, the traditional GNG and GWR cannot effectively separate each object, so GNG and GWR are not suitable for this task. The experimental results show that the ESDNNIS of the present application can perform continuous unsupervised learning on real datasets.

[0104] The application realizes the optimization of the structural development type neural network by adopting the new neuron generation rules based on the coverage domain and the information saturation degree. First, by using the neuron information saturation degree and the neuron maturation stage based on the information saturation degree, the long and short term memory of the human brain is simulated, so that the network model has plasticity and stability. At the same time, based on the parent-child relationship generated by the splitting of the neuron, the network can subdivide the original category as the number of learning samples increases, simulate the gradual fine cognitive process of human understanding of things, and introduce the clustering layer number, so that the user can independently select the required category subdivision degree. Then, the inhibition mechanism between the brother neurons and the information saturation forgetting mechanism are used to limit the excessive growth of the neurons and control the size of the network, so that the network can incrementally develop and adapt to the incremental instance type, the incremental class type and the incremental task type of the continuous learning scene.

[0105] The above is only the preferred embodiment of the present application, and does not limit the present application in any form. Although the present application has been disclosed as above with the preferred embodiment, it is not intended to limit the present application. Therefore, any simple modification, equivalent change and modification of the above embodiment made without departing from the technical solution of the present application, according to the technical essence of the present application, shall fall within the scope of protection of the technical solution of the present application.

Claims

1. An architecture development type neural network optimization method for open environment pattern recognition, characterized by the steps of The application relates to a structure-developing neural network and a method for training the structure-developing neural network. The structure-developing neural network adjusts the information saturation degree of the winning neuron during the structure development and parameter adjustment, and continuously adjusts the information saturation degree of the winning neuron, wherein the information saturation degree is calculated according to the winning frequency and the information carrying capacity of the neuron, and a new neuron is split when the information saturation degree of the neuron reaches a preset saturation threshold. During the structure development of the structure-developing neural network, an inhibition mechanism for neuron splitting is set, and after the information saturation degree of the neuron reaches the saturation, it is determined whether a new neuron is generated; if the neuron generation is unsuccessful, a target neuron which inhibits the generation of the new neuron is taken as the winning neuron, and the attribute of the target neuron is adjusted. During the development of the structure-developing neural network, the attribute and the transverse connection of the winning neuron are adjusted according to the development stage and the input signal, wherein the development stage of the neuron is determined according to the information saturation degree, and the development stage of the neuron is divided into an immature stage, a mature stage and a parent stage according to the information saturation degree. The attenuation process is as follows: When the neuron is just generated, the initial saturation is set to the information saturation change of the parent neuron, and if there is no parent neuron, it is set to the maximum value S of the information saturation change of the immature neuron r / 4, wherein S r ∈(0, 1) is the reset threshold of the information saturation; the stage from the generation of the neuron to the information saturation reaching the reset threshold is the immature stage, and the immature neuron inherits the class representation of the parent neuron; when the information saturation exceeds the reset threshold, it is determined that the neuron enters the mature stage, and the neuron represents an independent class after maturation; when the information saturation reaches the saturation threshold, the winning neuron splits to generate a new neuron, the winning neuron is the parent neuron of the new neuron, and the new neuron is the child neuron of the winning neuron, at which time the winning neuron enters the parent stage; The structural development type neural network further comprises, in the structural development process, directly attenuating to a reset threshold S when the information saturation degree of a neuron reaches a saturation threshold r to realize an information saturation degree attenuation forgetting mechanism, wherein the information saturation degrees of mature neurons other than the winning neuron and the parent neurons of the branch where the winning neuron is located are attenuated toward the reset threshold.

2. The structural development of neural network optimization method for open environment pattern recognition according to claim 1, characterized in that, winning neuron n k about the input signal x i the amount of change in information saturation Δs k,i according to the input signal x i the similarity o k with the winning neuron n k,j determined, the similarity o k,j is calculated by using a Gaussian radial basis function, that is: where d() is a distance function of the input signal x i the center vector μ k of the current winning neuron n k the distance function, σ k is the coverage parameter of the current neuron n k .

3. The structural development of neural network optimization method for open environment pattern recognition according to claim 2, characterized in that, Information saturation s k The update calculation formula is: s k ′=s k +Δs k,i where s k is the updated information saturation, s k is the information saturation before updating, S r is the reset threshold, and a = (1 - S r ) is a proportional coefficient, a and S r are negatively correlated, and the information saturation change amount Δs k,i is positively correlated with S r when the neuron is immature, and Δs k,i is negatively correlated with S r when the neuron is mature.

4. The structural development of neural network optimization method for open environment pattern recognition according to claim 1, characterized in that, In the structural development neural network, the new neuron is centered on the current input signal as a vector, when the information saturation of the winning neuron reaches the saturation threshold, the new neuron is split from the winning neuron, the coverage domain parameter of the new neuron is initialized as one third of the distance between the child and parent center vectors, and the information saturation is initialized as the change of the information saturation of the parent; when a new root neuron is generated from an input signal that does not fall into the coverage domain of any neuron, the coverage domain parameter of the generated new neuron is initialized as one third of the distance from the center vector to the center vector of the nearest neighbor neuron, and the information saturation is initialized as S r / 4, if it is the first generated neuron, the distance from the center vector to the origin of the sample space is one third, and the information saturation is initialized as S r / 4.

5. The structure development neural network optimization method for open environment pattern recognition according to any one of claims 1 to 4, characterized by, During the structure development of the structure-developing neural network, the rule for determining whether a new neuron is generated is that when the distance between the center vector of a brother neuron and the center vector of the neuron to be generated is less than the coverage domain radius of one of the neurons, that is, the center vector of the pre-generated neuron or the center vector of the existing brother neuron falls in the coverage domain of the other, the current neuron is not generated. s j ′=s j -τ(s j -S r ),S r ≤s j ,τ∈(0,1) where τ is the information saturation decay rate, s j is the decayed information saturation, s j is the un-decayed information saturation.

6. The structure development neural network optimization method for open environment pattern recognition according to any one of claims 1 to 4, characterized by, The center vector and the coverage domain parameter in the structure-developing neural network are updated according to the following formula.

7. The structure development neural network optimization method for open environment pattern recognition according to any one of claims 1 to 4, characterized by, ​ wherein μ k and σ k represent the center vector and the coverage domain parameter before updating, μ p represents the center vector of the parent meta, μ k ', σ k ' and s k ' represent the center vector, the coverage domain parameter and the information saturation after updating, respectively, Δs k,i is the information saturation change amount, lay k is the layer number in the parent-child relationship tree; MT k is the maturation mark, taking values of 0 or 1, and the updating formula is:

8. The structure development neural network optimization method for open environment pattern recognition according to any one of claims 1 to 4, characterized by, In the structural development neural network, the update mechanism of parent-child neuron connection is used for update. When the neuron n j is mature, if the center vector μ j of the neuron n j moves out of the covering domain of the parent neuron, i.e. d(μ j ,μ p )>r p , where μ p is the center vector of the neuron n j , μ p is the center vector of the parent neuron, and r p is the covering domain radius of the parent neuron, it is determined that the neuron n j cannot be a subdivision of the class represented by the current parent neuron, i.e. it is no longer a child neuron of the current parent neuron, and then the current parent-child connection is deleted, the cluster level of the neuron n j is promoted by one level, and a new parent neuron is searched in the parent neuron layer of the current level. If a new parent neuron is not found, the level is promoted by one level again, until a new parent neuron is found or becomes the root neuron of the first level. When searching for the new parent neuron in the parent neuron layer of the current level, the judgment basis for whether it is a parent neuron is whether the center vector μ j is in the covering domain of the target neuron required to be judged. If yes, it is determined that the target neuron is the new parent neuron required to be searched.

Images