Campus public opinion monitoring-oriented incremental learning neural network model continuous optimization method

Abstract

The application discloses a campus public opinion monitoring-oriented incremental learning type neural network model continuous optimization method, and relates to the technical field of neural networks. In the method, the incremental learning type neural network model to be optimized comprises at least four layers, namely an input layer, two hidden layers and an output layer, each hidden layer comprises at least two neurons, and is divided into shallow feature extraction neurons and deep monitoring neurons. In the incremental learning process, a network model for neuron state recognition is established, a fully-connected deep neural network is adopted as a basic framework, and a total loss function of the network model is composed of three parts, namely a regularization loss term, a task loss term and a weight recovery loss term. Through weight reconstruction, bias correction and adaptive regularization constraint, model parameters are iteratively optimized, and the capacity protection and self-healing ability of the network topology structure are realized.

Description

Technical Field

[0001] This invention relates to the field of neural network technology, specifically to a method for continuous optimization of incremental learning neural network models for campus public opinion monitoring. Background Technology

[0002] In the process of continuously optimizing incremental learning neural network models for campus public opinion monitoring, incremental learning neural networks refer to a class of neural network models that can retain knowledge of old tasks while learning new tasks. Their core challenge lies in mitigating catastrophic forgetting. Existing methods mainly include elastic weight consolidation and memory replay, which constrain parameter updates by introducing regularization terms into the loss function or retaining a subset of old samples.

[0003] In existing incremental learning neural network models, the models are prone to catastrophic forgetting, making it difficult to learn both historical public opinion knowledge and newly added public opinion features; neurons are prone to silent inactivation, leading to model structure degradation; regularization constraints lack specificity and are difficult to dynamically adjust according to neuron state, with excessive constraints leading to insufficient model plasticity or insufficient constraints leading to structural damage, ultimately affecting the accuracy and real-time performance of public opinion monitoring. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a method for continuous optimization of incremental learning neural network models for campus public opinion monitoring, thus solving the problems mentioned in the background technology.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] Methods for continuous optimization of incremental learning neural network models for campus public opinion monitoring include:

[0007] The incremental learning neural network model to be optimized includes at least four network layers. Each of the four network layers includes at least one input layer, two hidden layers, and one output layer. Each hidden layer contains at least two neurons, including deep neurons and shallow neurons.

[0008] In the incremental learning process, an incremental learning neural network model for neuron state recognition is established;

[0009] Design an incremental learning neural network model, including: selecting a fully connected deep neural network as the network architecture of the incremental learning neural network model to be optimized, and using a total loss function of the fully connected deep neural network that includes a regularization loss term, a task loss term, and a weight recovery loss term;

[0010] By refactoring weights, correcting biases, and applying adaptive regularization constraints, the model parameters within the incremental learning neural network model to be optimized are adaptively and iteratively optimized, thereby achieving capacity protection and self-healing capabilities of the neural network topology.

[0011] Preferably, the incremental learning neural network model is designed, including:

[0012] Based on the trainable parameters of the incremental learning neural network model to be optimized, the connection weights of all neurons in each network layer are obtained, and the weight occupancy data is acquired.

[0013] The incremental task to be learned is input into the incremental learning neural network model to be optimized, and forward propagation is performed. The activation distribution of neurons is obtained through the first few feature extraction layers to perform association level division on the incremental task to be learned. During the incremental learning process, the activation occupancy ratio is obtained based on the activation value output by each shallow neuron in the forward propagation. Combined with the weight occupancy data, the activation data is obtained.

[0014] Based on activation data and association levels, the gradient value of the activation value of each deep neuron relative to each of its connection weights is calculated. Combined with the neuron activation distribution, the sensitivity of deep neurons to the output of different neurons in the feature extraction layer is analyzed, and a neuron response function for dynamically adjusting the neuron protection strength is obtained.

[0015] Preferably, the design of the incremental learning neural network model further includes:

[0016] After the forward propagation of each training batch is completed, the average neuron activation state of all samples in the current batch on the feature extraction layer is used as the average embedding vector, and the theoretical expected output of each neuron is obtained for the neuron response function of each neuron.

[0017] A sliding window is set to obtain the average output of each deep neuron and compared with the theoretical expected output. If the average output is lower than the theoretical expected output, the average output is considered lower. If the value is doubled, it is marked as a potentially abnormal neuron, where The preset deviation threshold;

[0018] For potentially abnormal neurons, a cumulative threshold detection method is used to compare each output mean with the historical activation baseline, and the threshold is adjusted by weighting the neuron response function to determine the neuron node.

[0019] Based on the gradient values, activation values, and weight occupancy data of neuron nodes, the quiescent state of the current neuron node is analyzed to identify the neuron node type.

[0020] Based on neuron nodes, we analyze the structural features of the neural network to obtain the final regularization strength for neuron structure repair and the total loss function for backpropagation and gradient update.

[0021] Preferably, based on neuron nodes, the structural features of the neural network are analyzed to obtain the final regularization strength for neuron structure repair and the total loss function for backpropagation and gradient update, including:

[0022] Based on the association level and activation occupancy ratio of each neuron node, the intensity of the differentiated basic constraint is dynamically allocated to obtain the regularized baseline intensity for smooth transition.

[0023] The structural importance of each neuron node is analyzed within the network parameter structure space. The structural importance is corrected by combining the neuron weight distribution structure and the update range structure. The multi-task importance distribution is fused by decay accumulation to form a comprehensive importance representation of the internal structural state of the adaptive incremental learning neural network model.

[0024] During the training of the incremental learning neural network model, a differentiated regularization constraint is applied to the regularization baseline strength of each neuron node based on its comprehensive importance. An appropriate constraint strength is assigned to neuron nodes with different degrees of structural degradation to obtain the final regularization strength corresponding to each neuron node. A total loss function for backpropagation and gradient update is then constructed.

[0025] Preferably, the process of constructing the total loss function includes:

[0026] Based on the final regularization strength and overall importance of each neuron node, we analyze the magnitude of the change in the current connection weights relative to the historical best weights, and construct a regularization loss term to constrain parameter updates and protect the historical topology of the neural network.

[0027] Preferably, the process of constructing the total loss function further includes:

[0028] Based on the current task, calculate the prediction error of the incremental learning neural network model on the task to obtain the task loss term;

[0029] Based on the structural difference between the data occupied by neuron node weights and the current weights, the degree of weight dispersion is analyzed to obtain the weight recovery loss term used to guide the recovery of neuron node structure;

[0030] The regularization loss term, task loss term, and weighted recovery loss term are weighted and merged to construct the total loss function.

[0031] Preferably, the model parameters within the incremental learning neural network model to be optimized are adaptively iteratively optimized through weight reconstruction, bias correction, and adaptive regularization constraints, including:

[0032] In backpropagation, the gradient value of each neuron node is differentially scaled based on the final regularization strength. Combined with the total loss function, the differentially scaled gradient is obtained and used for connection weight updates.

[0033] Based on the distribution characteristics of the connection weights of the current neuron nodes in the incremental learning neural network model and their activation occupancy ratio under task-driven conditions, the validity of neuron connections is determined, and selective pruning and re-initialization are implemented according to the determination results to adjust the bias and achieve optimization of neural network parameter update strategy for continuous learning.

[0034] Based on the current activation value of each neuron node, the network topology is dynamically adjusted to dynamically update the number of neuron nodes.

[0035] The updated sets of neurons are located after the feature extraction layer in the incremental learning neural network model, thereby achieving capacity protection and self-healing capabilities of the neural network topology.

[0036] The above-described solution of the present invention has at least the following beneficial effects:

[0037] This method addresses the shortcomings of traditional incremental learning models, such as blind parameter updates, insufficient preservation of historical knowledge, and difficulty in activating silent nodes. Through neuron node identification, differentiated regularization constraints, and dynamic parameter adjustment, it achieves synergistic advancement of model learning and knowledge retention. Compared to traditional incremental learning methods, this method effectively avoids catastrophic forgetting. Furthermore, through targeted parameter optimization and structural adjustments, it ensures that the model can efficiently acquire new task knowledge while fully retaining historical learning outcomes during continuous learning. This significantly improves the model's learning stability and parameter adaptability, solving the problems of untargeted parameter adjustments and structural degradation in traditional models, and providing a reliable guarantee for the long-term stable operation of the model. Attached Figure Description

[0038] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0039] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0040] like Figure 1As shown, embodiments of the present invention provide a method for continuous optimization of an incremental learning neural network model for campus public opinion monitoring, including:

[0041] The incremental learning neural network model to be optimized includes at least four network layers. Each of the four network layers contains at least one input layer, two hidden layers, and one output layer. Each hidden layer contains at least two neurons, including deep neurons and shallow neurons. Deep neurons are located in network layers after the feature extraction layers, such as hidden layers and output layers. Shallow neurons are located in the first few feature extraction layers.

[0042] In the incremental learning process, an incremental learning neural network model for neuron state recognition is established;

[0043] Design an incremental learning neural network model, including: selecting a fully connected deep neural network as the network architecture of the incremental learning neural network model to be optimized, and using a total loss function of the fully connected deep neural network that includes a regularization loss term, a task loss term, and a weight recovery loss term; in the fully connected deep neural network, each neuron between two adjacent layers is connected to each neuron in the next layer;

[0044] By refactoring weights, correcting biases, and applying adaptive regularization constraints, the model parameters within the incremental learning neural network model to be optimized are adaptively and iteratively optimized, thereby achieving capacity protection and self-healing capabilities of the neural network topology.

[0045] In this embodiment, by structurally designing and dynamically optimizing the campus public opinion monitoring neural network in an incremental learning scenario, the model no longer relies on a single fixed structure for learning. Instead, the network is divided into an input layer, a double hidden layer, and an output layer, and shallow and deep neurons are further distinguished. This allows different layers to perform differentiated functions in feature extraction and semantic decision-making, providing a structural foundation for hierarchical memory and hierarchical expression. Simultaneously, a composite loss function including regularization loss, task loss, and weight recovery loss is introduced during training. This ensures that when learning new public opinion tasks, the model not only focuses on minimizing the error of the current task but also simultaneously constrains historical knowledge from being destroyed and structurally repairs degraded or inefficient parameters. Furthermore, through a joint iterative mechanism of weight reconstruction, bias correction, and adaptive regularization constraints, the neural network can automatically identify which neurons still carry out effective semantic expression and which neurons have gradually become inactive or have weakened in function when facing continuously input new tasks. It then adjusts the intensity of parameter updates accordingly, thereby achieving a dynamic balance of strengthening where needed, repairing where needed, and protecting where needed.

[0046] By constructing a fully connected deep network containing an input layer, two hidden layers, and an output layer, and distinguishing the functional distribution of shallow and deep neurons, the feature extraction and semantic decision-making processes are structurally hierarchically carried out.

[0047] Based on this, a joint constraint of regularization loss term, task loss term and weight recovery loss term is introduced, so that when the model faces new public opinion tasks, it no longer relies solely on a single error to drive updates, but simultaneously takes into account the balance between current task fitting, historical knowledge preservation and degradation parameter repair.

[0048] By further combining weight reconstruction, bias correction and adaptive regularization mechanisms, the network can differentiate the state of different neurons during incremental learning, so that neurons that still have expressive ability can be stably updated, while neurons that show signs of degeneration or quiescence can be structurally repaired or reactivated, thus forming a dynamic closed-loop control logic of preservation, inhibition and repair at the parameter level.

[0049] Based on the above mechanism, when campus public opinion data is continuously input and causes changes in task distribution, the network can continuously absorb new information without destroying the original knowledge structure. At the same time, it avoids the catastrophic forgetting and structural rigidity problems common in traditional incremental learning, so that the model as a whole can maintain the unity of topological stability and functional plasticity in long-term operation, and achieve synergistic enhancement of continuous learning ability and adaptive repair ability in complex public opinion evolution scenarios.

[0050] In a preferred embodiment of the present invention, an incremental learning neural network model is designed, comprising:

[0051] At the beginning of the incremental learning task, based on the trainable parameters of the incremental learning neural network model to be optimized, the connection weights of all neurons in each network layer and the activation function type parameters corresponding to each neuron, such as ReLU, Sigmoid, Tanh, etc., are obtained. During the loading process, the historical task dataset corresponding to the neural network in the initial training stage is also acquired. This dataset contains the feature vectors and class labels of each task sample, which are used for the subsequent calculation of activation occupancy ratio. Based on the distribution characteristics and variation range of the connection weights, the weight occupancy data is obtained, thereby obtaining the static structural feature mapping of the neurons in the initial stage, providing an initial reference for subsequent silent state judgment.

[0052] In neural networks, trainable parameters refer to the internal variables that are continuously updated and optimized through the backpropagation algorithm during training, including connection weights and biases.

[0053] Based on the global histogram, all weight values ​​of each network layer are divided into multiple equal-width intervals, and the number of weights contained in each interval is counted to form a histogram to obtain the frequency of each connection weight in its respective interval, which is used as the distribution density of each connection weight. When the value is close to 1, it indicates that the weight value of the neuron is highly concentrated in a specific value interval and has low plasticity; when it is close to 0, it indicates that the weight value is evenly distributed in the entire value space and has high plasticity.

[0054] Simultaneously, based on historical training records, during incremental learning, the values ​​of each weight are recorded after each update, and the difference between its historical maximum and minimum values ​​is calculated to obtain the range of change for each connection weight. In the initial stage, the theoretical range is determined according to the initialization method. For example, in Xavier uniform initialization, the range size is 2×limit; limit is the uniform distribution boundary. This range reflects the length of the interval in which the weight is allowed to change. In subsequent incremental learning, if the actual change of a weight is much smaller than its theoretical range, it indicates that the weight has not been effectively updated for a long time and may be in a frozen state.

[0055] By combining the distribution density and the range of variation, we obtain the weight occupancy data, which is used to reflect the parameter distribution characteristics of each neuron in each layer of the neural network in the initial state.

[0056] The incremental task to be learned is input into the incremental learning neural network model to be optimized, and forward propagation is performed. The activation distribution of neurons, i.e., the embedding vector, is obtained through the first few feature extraction layers. The association level is divided according to the incremental task to be learned. During the incremental learning process, based on the activation value output by each shallow neuron in the forward propagation, the proportion of samples that exceed the preset output threshold is calculated to obtain the activation occupancy ratio. Combined with the weight occupancy data, the activation data is obtained. This data records the correspondence between the static structural features and dynamic functional features of each neuron.

[0057] In this embodiment, for each task in the incremental task sequence to be learned, samples from its dataset are sequentially input into the incremental learning neural network model to be optimized. The first few feature extraction layers of the neural network are selected, typically the first or second hidden layers. For each input sample, the output activation values ​​of all neurons in that feature extraction layer are recorded and arranged in order of neuron index to form the embedding vector of that sample. For each task, the embedding vectors of all samples under that task are summarized, and the mean of the activation values ​​in each feature dimension is calculated to obtain the feature distribution center vector of that task. Simultaneously, the Euclidean distance from the embedding vector of each sample to this center vector is calculated, and the average of all distances is taken to obtain the distribution radius of that task.

[0058] Based on the feature distribution center and distribution radius of each task, the overlap ratio between the distribution radii of each pair of adjacent tasks in the task sequence is calculated one by one. That is, the intersection length of the two distribution intervals divided by the union length. The higher the overlap ratio, the more overlapping the data distribution coverage of the two tasks is. The overlap ratio is used as the similarity of each pair of adjacent tasks.

[0059] When the similarity exceeds a preset similarity threshold, it is recorded as a highly associated task pair; otherwise, it is recorded as a low-association task pair. The number of samples whose activation values ​​exceed a preset output threshold is also recorded, for example, 0, which means the output is greater than 0 for the ReLU activation function. Dividing this number by the total number of samples for the task yields the activation ratio of the neuron for that task. This ratio reflects the degree to which the task depends on the neuron; the higher the ratio, the more the task activates the neuron during training and inference, and the more important the neuron plays in the functional implementation of the task.

[0060] Subsequently, the activation occupancy ratio of each neuron for each task is associated with and stored with the weight occupancy data. The weight occupancy data records the weight distribution density and variation range of the input connections of each neuron, which belongs to the static structural features of the neuron; the activation occupancy ratio records the response intensity of each neuron to different tasks, which belongs to the dynamic functional features of the neuron.

[0061] Based on activation data and association levels, the gradient value of the activation value of each deep neuron relative to each of its connection weights is calculated. Combined with the neuron activation distribution, the sensitivity of deep neurons to the output of different neurons in the feature extraction layer is analyzed, that is, the sensitivity of deep neurons to different feature dimensions, and a neuron response function for dynamically adjusting the neuron protection strength is obtained.

[0062] In this embodiment, a subset of samples corresponding to tasks deemed highly correlated is selected from the incremental task sequence to be learned. For each neuron in the neural network, each sample from the selected subset is sequentially input into the network, and forward propagation is performed to obtain the neuron's output value, i.e., the activation value. Then, backpropagation is performed to calculate the partial derivative of the neuron's output value with respect to each of its input connection weights, i.e., the gradient value on each connection.

[0063] For each sample, the absolute values ​​of the partial derivatives of all connection weights of the neuron are summed to obtain the gradient response value of the neuron for that sample. After traversing all selected samples, the mean and variance of the gradient response values ​​are calculated according to the category to which the sample belongs. Then, the global gradient response variance and mean are calculated for all categories, which are used to reflect the distribution of the gradient response intensity of the neuron under different input samples.

[0064] The activation distribution is used as the original embedding vector for each sample and normalized. Combined with the gradient response value, a neuron response function is constructed to obtain the neuron response matrix. The rows of this matrix correspond to each neuron in the neural network, and the columns correspond to each dimension of the normalized embedding vector, i.e., each neuron in the feature extraction layer. For the i-th neuron and the d-th feature dimension, the gradient response value of the corresponding neuron on all samples is first calculated as the product of the normalized embedding value of the corresponding sample in dimension d. Then, the arithmetic mean of these products is calculated, and finally, the global gradient response variance of the neuron is multiplied to obtain the corresponding neuron response function, i.e., the neuron responsiveness of the i-th neuron in the matrix to the d-th feature dimension. The larger the value, the more sensitive the output of the neuron is to the change of the input connection weight corresponding to that dimension when it receives input with feature information of that dimension.

[0065] Feature dimension refers to the number of neurons in the feature extraction layer of a neural network; that is, each neuron in the layer corresponds to an independent feature dimension.

[0066] In this embodiment of the invention, the trainable parameters of the neural network are first decomposed in a structured manner in the initial stage of incremental learning, so that the connection weights, biases and activation function types are uniformly included in the parameter mapping space. Combined with the feature vectors and class labels in the historical task dataset, the embedding representation of the shallow feature extraction layer is constructed in the forward propagation process, thereby projecting the static weight structure and dynamic task behavior into the same analysis framework at the same time.

[0067] Furthermore, the weights are divided into equal-width intervals and their distribution density is statistically analyzed using a global histogram. At the same time, the range of weight changes is calculated by combining historical update records. The degree of distribution concentration and the updatable spatial scale are integrated to form the weight occupancy data, so that each neuron is no longer described by a single numerical value, but has a dual characterization dimension of structural stability and plasticity.

[0068] On the task input side, the task distribution center and distribution radius are constructed by embedding vector mean and Euclidean distance, and the degree of correlation between tasks is quantified by the overlap ratio of adjacent task distributions, thereby forming a hierarchical structure of correlation at the task sequence level and avoiding feature drift caused by disordered superposition of incremental tasks.

[0069] At the same time, the activation occupancy ratio is used to characterize the dependence of neurons on different tasks, so that the functional contribution of each neuron is mapped to the specific task dimension and associated with the weight occupancy data, thereby realizing the unified binding of static structural features and dynamic functional response.

[0070] Building upon this foundation, backpropagation is performed on a subset of highly correlated task samples to obtain the neuron gradient response distribution. This is then combined with normalized embedding vectors to calculate the neuron response function matrix, enabling the quantification of each neuron's sensitivity to different feature dimensions. This further establishes a computable coupled mapping relationship between parameter distribution, task dependencies, and gradient response intensity. Logically, this method continuously tracks the evolutionary trajectory of neuron structures when faced with continuous task inputs. By jointly characterizing weight occupancy and activation responses, it reduces the blind structural update problem caused by relying solely on local gradient updates, allowing the neural network to maintain the stability of historical knowledge while possessing a more granular response differentiation capability for new tasks.

[0071] In a preferred embodiment of the present invention, the design of the incremental learning neural network model further includes:

[0072] After the forward propagation of each training batch is completed, the average neuron activation state of all samples in the current batch on the feature extraction layer is used as the average embedding vector, and the theoretical expected output of each neuron is obtained for the neuron response function of each neuron.

[0073] The average embedding vector is a vector obtained by averaging the output activation values ​​of each neuron in the feature extraction layer of a neural network across the sample dimension after all input samples have passed through the feature extraction layer of the neural network in the current training batch. It is used to characterize the overall activation distribution of the input samples in the current batch on the feature extraction layer of the neural network.

[0074] After obtaining the neuron response function for each deep neuron and calculating the average embedding vector of the feature extraction layer in the current training batch, the theoretical expected output of the deep neuron under the current batch input is obtained by performing a dot product operation between the neuron response function and the average embedding vector. For example, for the i-th deep neuron, its response sensitivity to each neuron in the feature extraction layer is multiplied by the average embedding vector component at the corresponding position, and then all these product results are summed to finally obtain the theoretical expected output of the neuron. This expected output is used to reflect the average activation level that the neuron should produce in a healthy state under the feature distribution of the current batch input, based on the inherent response characteristics of the neuron.

[0075] A sliding window is set to obtain the average output of each deep neuron and compared with the theoretical expected output. If the average output is lower than the theoretical expected output, the average output is considered lower. If the value is more than twice the normal value, it indicates that the neuron's responsiveness has deteriorated and there is a risk of silent neuron activity; this neuron is marked as a potentially abnormal neuron. This step is used to detect transient response abnormalities. The preset deviation threshold is 0.3; the theoretically expected output is... This is the minimum acceptable response level;

[0076] For potentially abnormal neurons, the cumulative threshold detection method is used to compare each output mean with the historical activation baseline, and the threshold is adjusted by weighting the neuron response function to determine the neuron node, i.e. the risk neuron.

[0077] In this embodiment, for each neuron, through The dynamic lower bound of its silent judgment is dynamically adjusted, whereby... For neurons The dynamic lower bound is used to determine whether the current output of a neuron is within an acceptable healthy range. When the neuron has a higher responsiveness, its dynamic lower bound is adjusted to be higher, and therefore it is easier to be judged as having a quiescent risk.

[0078] For neurons The historical activation baseline is used to describe the average output value of the neuron itself during the initial training phase, and is the historical response level of the neuron. This is a preset baseline tolerance coefficient used to control the rate of decrease in the dynamic tolerance lower limit relative to the historical active baseline. Its value typically ranges from 0.1 to 1, and it can be tested on the validation set using a grid search method. The value is selected to maximize the accuracy of silent detection.

[0079] For neurons Mean neuron responsivity across all feature dimensions The highest neuronal responsivity among all neurons in the network;

[0080] When the average output exceeds the dynamic tolerance limit, it is considered normal and no intervention is performed; otherwise, it is considered a low-response batch and needs to be accumulated. When the accumulated batch exceeds the preset accumulation threshold, it is determined that the neuron has a quiescent trend and is recorded as a risk neuron.

[0081] Based on the gradient values, activation values, and weight occupancy data of neuron nodes, and using a Naive Bayes model pre-trained with healthy neurons and artificially silenced samples to calculate the posterior probability, the current silence state of neuron nodes is analyzed to identify the neuron node type, which includes passively suppressed silenced neurons, actively inactivated silenced neurons, and hybrid progressive silenced neurons.

[0082] In this embodiment, for each neuron node, an exponential fitting is performed based on the gradient magnitude sequence of its K consecutive training batches to obtain the gradient temporal decay coefficient, which is used to characterize the decay trend of the gradient as the training process progresses. The closer the value is to 0, the more it tends to be passively suppressed; the closer it is to 1, the more it tends to be actively deactivated. The gradient magnitude sequence is a sequence composed of the average gradient magnitude of a neuron over K consecutive training batches.

[0083] Based on the gradient magnitude sequence and activation output sequence within the same sliding window, the mutual information between the gradient and activation is calculated to quantify the statistical correlation between gradient changes and neuron responses. The mutual information of passively suppressed neurons is significantly higher, while that of actively inactivated neurons is close to 0. The activation output sequence is a sequence composed of the average output activation values ​​of the neuron within the same K training batches.

[0084] Based on the distribution density of the weights in the neuron node connections in the data, the information entropy is calculated and inter-layer normalization is performed to obtain the normalized weight distribution entropy, which is used to reflect the degree of dispersion of the weight distribution. The entropy value of the active inactivation type is extremely low, while the passive type maintains a medium or higher level.

[0085] Based on the initial range of weight changes and the current actual range of weight changes recorded in the weight occupancy data, a non-negative truncation calculation is performed. The shrinkage rate of the weight change range for each neuron is obtained to reflect the degree of narrowing of the weight update space; whereby... For neurons The shrinkage rate of the weight change range is as follows: the closer it is to 1, the smaller the current change range of the weight is compared to the initial range, the weight is hardly being updated anymore, and it tends to be an actively inactive silent neuron. Conversely, the closer it is to 0, the better the weight change range is maintained and it is still updating normally, and it tends to be a passively suppressed neuron. For neurons The total number of input connections. For joining weighted indexes, For the first The current actual range of change of each connection weight For the first The theoretical range of change for each connection weight during initialization. To obtain the maximum value;

[0086] The gradient time decay coefficient, mutual information, normalized weight distribution entropy, and weight change range shrinkage rate are input into the Naive Bayes model. The class to be distinguished is set, and the posterior probability of different features under each class is obtained by Bayes' theorem. The posterior probabilities of different features under each class are multiplied and calculated, and then multiplied by the prior probability of the corresponding type of neuron to obtain the comprehensive probability of the three types. The class with the largest comprehensive probability is extracted and determined as the final quiescent type of the current neuron.

[0087] By combining cumulative threshold detection and multi-feature Bayesian classification to accurately identify the quiescent type of neurons, and implementing differentiated regularization constraints, gradient scaling, selective pruning, and reinitialization for different types, the process of neurons progressing from sub-health to irreversible quiescent death is effectively suppressed. Furthermore, by introducing weight recovery loss to guide the regression of degenerate weights to the initial distribution, as well as dynamic topology expansion and capacity compensation, the model avoids sudden performance collapse during long-term incremental learning and possesses robust adaptive capabilities for arbitrary task order, achieving a triple synergistic optimization of task accuracy, historical structure protection, and neuronal structure repair.

[0088] Based on neuron nodes, we analyze the structural features of the neural network to obtain the final regularization strength for neuron structure repair and the total loss function for backpropagation and gradient update.

[0089] Based on neuron nodes, the structural features of the neural network are analyzed to obtain the final regularization strength for neuron structure repair and the total loss function for backpropagation and gradient update, including:

[0090] Based on the association level and activation occupancy ratio of each neuron node, a differentiated basic constraint strength is dynamically allocated through a Sigmoid nonlinear mapping to obtain a regularized baseline strength for smooth transition. This avoids training fluctuations caused by hard thresholds and provides a benchmark for subsequent individualized regularization calculations. For neurons In the current task The baseline regularization strength is used to automatically and smoothly allocate the baseline regularization strength based on the degree of activation contribution of neurons to the current task. This is the base strength coefficient corresponding to the task association level, representing the basic constraint strength at the task level. Corresponding to high and low task association levels; The sigmoid nonlinear mapping function maps any real number to the interval (0,1). For neurons In the current task Activation occupancy rate It stretches the values ​​that were originally in the [0,1] interval to [-1,1] so that the Sigmoid function can work in steep, sensitive, and distinguishable intervals. For task indexing; This represents the degree to which a neuron is irrelevant to the task. The lower the activation occupancy rate, the less important the neuron is to the current task, and the larger this value is.

[0091] Based on the difference in the distribution of the current task and the historical task in the feature space, the Jensen-Shannon divergence in information theory is used to quantify the degree of difference between tasks, and the difference value is converted into the basic constraint strength coefficient through linear mapping, with the value ranging from 0 to 1.

[0092] By extracting the differences in feature distribution between tasks and classifying the association levels, the regularization baseline strength of each neuron node is adaptively adjusted under the condition that the arrival order of tasks is uncontrollable. High-strength constraints are used for low-association tasks to protect old knowledge, while low-strength constraints are used for high-association tasks to retain plasticity. This significantly reduces the model's dependence on task order, and can maintain stable prediction performance even if tasks with large feature jumps arrive consecutively.

[0093] This study analyzes the structural importance of each neuron node within the network parameter structure space, corrects the structural importance by combining the neuron weight distribution structure and update range structure, and fuses the multi-task importance distribution through decay accumulation to form a comprehensive importance representation of the internal structural state of an incremental learning neural network model; this approach preserves the importance of historical knowledge while avoiding overprotection of degenerating neurons. The network parameter structure space refers to a high-dimensional abstract space formed by organizing all trainable parameters in a neural network according to their topological position and connectivity within the network.

[0094] In this embodiment, based on the parameter structure state after the incremental learning neural network has completed training for historical tasks, all connection weight parameters corresponding to each neuron node are extracted, and a correlation structure graph of neurons and weights is constructed to clarify the position and connection topology of each neuron in the network parameter structure space.

[0095] For each historical task, within the network parameter structure space, the gradient distribution characteristics of the corresponding connection weights of each neuron node are statistically analyzed, and the diagonal element of the Fisher information of a single weight parameter is calculated to quantify the structural sensitivity of the weight during task execution. Subsequently, the Fisher information of all connection weights of a neuron node is aggregated by mean to obtain the initial structural importance at the neuron level. The larger this value is, the higher the sensitivity and the stronger the structural importance of the neuron as a key node in the network structure of the current task.

[0096] Calculate the information entropy of the connection weights of each neuron node to quantify the dispersion of the weight distribution. The higher the entropy value, the more uniform the weight distribution and the more stable the structure; the lower the entropy value, the more concentrated the weight distribution and the more degenerate the structure. Calculate the shrinkage rate of the change range of the connection weights of each neuron node to quantify the narrowing of the weight update space. The closer the shrinkage rate is to 1, the more rigid the weight update space and the lower the structural activity; the closer the shrinkage rate is to 0, the more sufficient the weight update space and the higher the structural activity.

[0097] The original Fisher importance of a neuron is multiplied by the normalized weight distribution entropy, and then multiplied by the structural activity coefficient to obtain the structurally corrected Fisher importance of the neuron. The structural activity coefficient is 1 minus the shrinkage rate of the weight change range, which suppresses the importance of rigid neurons, avoids over-regularization protection, and leaves room for possible recovery. The higher the shrinkage rate, the smaller the structural activity coefficient, and the lower the importance is suppressed. This results in weaker constraints on such neurons in subsequent regularization, avoiding over-protection of degenerated structures.

[0098] The multi-task structural importance is iteratively fused using a decay-accumulation method. When learning each task, the historical comprehensive importance is decayed by a historical importance decay coefficient and then superimposed with the importance of the current task after structural correction. This gradually forms a comprehensive neuron importance that integrates the historical structural contribution and the current structural state. This is used to precisely control the regularization strength of each neuron in incremental learning. The more important the neuron, the stronger the constraint, and the less important or more rigid the neuron, the weaker the constraint. This protects the historical structure of the network while leaving room for the revival and renewal of dormant neurons.

[0099] In the training process of incremental learning neural network models, the regularization baseline strength of each neuron node is subject to differentiated regularization constraints based on comprehensive importance. Adaptive constraint strengths are assigned to neuron nodes with different degrees of structural degradation to obtain the final regularization strength corresponding to each neuron node. The passive type reduces the regularization constraint, the active type temporarily clears the regularization during the reinitialization phase, and the hybrid type adopts a compromise coefficient. A total loss function is constructed for backpropagation and gradient update.

[0100] For passively suppressed silent neurons, since the neuron's structural characteristics include severe gradient decay, a basically healthy weight distribution, and an incompletely rigid update space, the regularization constraint strength is reduced to obtain the final regularization strength = regularization baseline strength × comprehensive importance ratio × passive modulation coefficient. This preserves the neuron's structural plasticity and creates conditions for gradient enhancement and parameter updates. The comprehensive importance ratio is the ratio of the neuron's comprehensive importance to the maximum comprehensive importance of all neurons in the network. This ensures that neurons with higher structural importance can retain basic protection even with reduced constraints, preventing over-updating of critical structures. The passive modulation coefficient is used to weaken the constraints, allowing the neuron to regain update space and overcome the gradient suppression caused by excessively high regularization. Its value is between (0,1), with a commonly used range of 0.2 to 0.6.

[0101] For actively deactivated silent neurons, due to their structural characteristics of essentially vanished gradients, highly degraded weight distribution, and extremely rigid update space, the regularization constraints are temporarily cleared during the reinitialization phase, i.e., the final regularization strength is 0, to avoid regularization hindering weight reconstruction. After reinitialization, the regularization constraints are gradually restored. At this point, the final regularization strength = regularization baseline strength × comprehensive importance ratio × active recovery coefficient. The reinitialization phase is a dedicated training phase for actively deactivated neurons to perform weight reconstruction and awaken structural activity. The connection weights of the neuron are reinitialized and locally fine-tuned based on a healthy distribution, so that the weight distribution returns from the degraded and rigid state to a reasonable distribution range, allowing the activation input to fall back into the ReLU effective linear region, thus deactivating the neuron from a structural level. The active recovery coefficient is used to achieve a smooth transition, and its value is between [0,1]. It is fixed at 0 during the reinitialization phase and linearly increases from 0 to 1 after the repair is completed, achieving a smooth recovery of the regularization constraints.

[0102] For hybrid progressively quiescent neurons, whose structural characteristics include both weight degradation and gradient suppression, and whose structural state lies between passive and active types, a compromise coefficient is used to modulate the regularization strength. The final regularization strength is calculated as: Regularization baseline strength × Overall importance ratio × Hybrid compromise coefficient. This approach retains some regularization constraints to protect incompletely degenerated structures while releasing some update space to promote recovery. The hybrid compromise coefficient balances structural protection and recovery space, and its value is between the passive modulation coefficient and 1, with a commonly used range of 0.5 to 0.8.

[0103] In this embodiment of the invention, a four-dimensional feature is constructed by fusing gradient temporal decay coefficient, gradient and activation mutual information, normalized weight distribution entropy and weight change range shrinkage rate. The Naive Bayes model is used to accurately distinguish between passive suppression type, active deactivation type and hybrid progressive silencing, avoiding invalid reset or excessive intervention caused by confusion of silencing causes in traditional methods.

[0104] Based on this, different types of neurons are subjected to different methods, such as reducing the regularization intensity, temporarily clearing the regularization and combining it with weight reconstruction, and adjusting the compromise coefficient. This allows passive neurons to break through the regularization suppression and restore plasticity, active neurons to remove ReLU inactivation from the root of weight distribution, and hybrid neurons to take into account both structural protection and recovery space.

[0105] Meanwhile, by combining task association level and activation occupancy ratio, the baseline regularization strength is dynamically allocated through Sigmoid mapping, and the Jensen-Shannon divergence is used to quantify task differences to automatically adjust the baseline strength coefficient. Then, through the comprehensive importance representation of Fisher information, weight distribution entropy, and shrinkage rate correction, the adaptive regulation of neuron-level regularization strength is achieved. This not only protects the key structures in historical knowledge, but also preserves a precise recovery channel for silent or degenerated neurons. Ultimately, it significantly inhibits the cumulative death of silent neurons during long-term incremental learning, reduces the model's sensitivity to task order, and endows the network structure with self-healing and capacity maintenance capabilities.

[0106] In a preferred embodiment of the present invention, the process of constructing the total loss function includes:

[0107] Based on the final regularization strength and overall importance of each neuron node, the variation of the current connection weights relative to the historical best weights is analyzed, and a regularization loss term is constructed to constrain parameter updates and protect the historical topology of the neural network. This refers to a regularization term based on elastic weight consolidation, used to constrain important parameters from deviating from historical values, thereby preventing catastrophic forgetting; where... This is the regularization loss term. This represents the total number of neuron nodes. This is the final regularization strength, used to control the penalty intensity in the loss function; To consider its overall importance, This is the connection weight vector of the current neuron node. This is the connection weight vector of the historical neuron nodes;

[0108] The historical optimal weights are the set of weight parameters that the neural network retains after learning all the previous old tasks, and which have the best performance. They are derived from the average optimal parameters of multiple historical tasks.

[0109] Based on the current task, calculate the prediction error of the incremental learning neural network model on the task to obtain the task loss term, which reflects the prediction error of the model on the current task. For classification tasks, it is usually the cross-entropy loss; for regression tasks, it is usually the mean squared error loss. This loss term is used to guide the model to learn new knowledge.

[0110] Based on the structural difference between the data occupied by neuron node weights and the current weights, the degree of weight dispersion is analyzed to obtain a weight recovery loss term used to guide the recovery of neuron node structure. This term is used to drive the degraded weights back to a healthy structure in the total loss.

[0111] The weight recovery loss is obtained by multiplying the recovery loss weights by the KL divergence between the current weight distribution and the initial weight distribution. This loss is accumulated only for passively suppressed neurons and is used to guide their weight distribution back to the initial state. Through backpropagation, the recovery loss generates additional gradient components on the input connection weights of passively suppressed neurons, so that these weights are subjected to a pull towards the initial weight values ​​with each parameter update, thereby gradually restoring the diversity and plasticity of their distribution. The further the current weight distribution deviates from the initial distribution, the larger the loss value, thus guiding the neuron's weight distribution back to the initial distribution through the gradient descent process.

[0112] The recovery loss weight is used to control the strength of the recovery loss. It is obtained by combining the quiescence type of the neuron and the degree of distribution degradation. Specifically, the basic weight is determined according to the quiescence type, such as 0.8 for active inactivation, 0.6 for mixed type, and 0.2 for passive suppression type. Then, the recovery loss weight is obtained by an adaptive weight allocation method that dynamically corrects the weight based on KL divergence, so as to ensure that the more severely degenerated the neuron, the higher the weight.

[0113] The current weight distribution is the actual weight distribution at the current iteration after multiple rounds of incremental training of the model, and it is used to measure the degree of parameter shift.

[0114] The regularization loss term, the task loss term, and the weighted recovery loss term are weighted and fused together to construct the total loss function;

[0115] By minimizing the total loss function to update model parameters, including connection weights and biases, the system can maintain task accuracy while also protecting historical structures and optimizing the distribution of silent neuron weights.

[0116] By supervising the regression of the weight distribution of degenerated neurons to their initial healthy state through the weight recovery loss term, the differential regularization constraint and weight structure repair form a synergistic effect. This not only avoids the destruction of key structures but also promotes the structural recovery of quiescent or degenerated neurons, achieving a triple optimization of task accuracy, historical structure protection, and neuronal structure repair.

[0117] In this embodiment of the invention, since traditional elastic weight consolidation relies solely on Fisher information constraint parameters, it is easy to impose excessive protection on degenerated neurons and hinder their recovery. This invention introduces final regularization strength and comprehensive importance into the regularization loss term, which can dynamically adjust the penalty intensity according to the neuron quiescent type, so that the constraint of passively suppressed neurons is weakened and the constraint of actively inactivated neurons is completely removed during the reinitialization stage, thereby leaving differentiated update space for different types of quiescent neurons.

[0118] Meanwhile, the weight recovery loss term guides the weight distribution of passively suppressed neurons back to its initial healthy state through KL divergence. Furthermore, the recovery loss weights are adaptively allocated based on the quiescence type and the degree of distribution degradation, ensuring that neurons with more severe degradation receive stronger repair traction. After weighted fusion of regularization loss, task loss, and weight recovery loss, the model, while minimizing the total loss function, simultaneously ensures that the key structures of historical tasks are not destroyed, new knowledge for the current task is accurately learned, and the weight distribution degradation of quiescent neurons is actively repaired. This collaborative optimization mechanism effectively resolves the conflict between protecting old knowledge and restoring neuronal plasticity in traditional methods. It enables the neural network to both suppress catastrophic forgetting and maintain the healthy activity of its internal neuronal population during long-term incremental learning, thereby avoiding sudden collapse caused by cumulative neuronal quiescence.

[0119] In a preferred embodiment of the present invention, adaptive iterative optimization of the model parameters within the incremental learning neural network model to be optimized is performed through weight reconstruction, bias correction, and adaptive regularization constraints, including:

[0120] Although the total loss function already includes the recovery loss and regularization term, the magnitude of the gradient may still be insufficient for the passively suppressed neuron to update effectively. This is because the gradient may still be very small after the regularization coefficient is reduced. Therefore, gradient scaling is required to make personalized adjustments for the silent type.

[0121] In backpropagation, the gradient value of each neuron node is differentially scaled based on the final regularization strength. For example, a gradient boosting coefficient greater than 1 is applied to the nodes that need to enhance plasticity to amplify the gradient, a gradient boosting coefficient equal to 1 is applied to the nodes that need to maintain the normal update rhythm, and a gradient boosting coefficient equal to 0 is applied to the nodes that need to pause updates. Combined with the total loss function, the differentially scaled gradient is obtained and used to update the connection weights.

[0122] Based on the neuron quiescence type, the base coefficients for gradient enhancement are determined. Higher base coefficients are used for actively inactivated neurons to enhance gradient repair, medium coefficients are used for hybrid neurons, and lower coefficients are used for passively suppressed neurons. Then, weighted corrections are performed using the final regularization strength as a constraint. The greater the regularization strength, the smaller the gradient enhancement coefficient, thus achieving adaptive control of the parameter update amplitude. Finally, the calculation results are mapped to a reasonable range through the Sigmoid function to obtain stable and smooth gradient enhancement coefficients, which are used for gradient control in subsequent backpropagation.

[0123] The formula for updating the connection weights in this iteration is: New weight = Neuron weight value after the previous iteration - Learning rate × Differentially scaled gradient; where the neuron weight value after the previous iteration is the base value for this update; the learning rate is the parameter update step size control coefficient preset by the model; and the differentially scaled gradient reflects the direction and magnitude of the weight adjustment.

[0124] The original gradient is obtained by differentiating the total loss function with respect to the weights. This original gradient is then multiplied by the gradient boosting coefficient to obtain a gradient that has been differentially scaled according to the neuron type and importance. The gradient boosting coefficient is obtained based on the neuron quiescent type and the final regularization strength and is used to adaptively amplify or suppress the gradient.

[0125] Based on the distribution characteristics of the connection weights of the current neuron nodes in the incremental learning neural network model and their activation occupancy ratio under task-driven conditions, the validity of neuron connections is determined, and selective pruning and re-initialization are implemented according to the determination results to adjust the bias and achieve optimization of neural network parameter update strategy for continuous learning.

[0126] In practice, because actively deactivated neurons suffer from weight degradation and abnormal activation, their weight distribution deviates from a healthy state and their gradient signals are weak. Conventional weight updates cannot restore their activity. Therefore, the neurons mentioned here refer to actively deactivated neurons to ensure targeted repair and avoid interference with irrelevant neurons.

[0127] For each input connection of an actively deactivated neuron, if the absolute value of the connection weight is less than a preset weight threshold and the historical maximum activation occupancy ratio is less than a preset activation threshold, the connection is marked as a state to be pruned, and a binary pruning mask with the same dimension as the weight matrix is ​​generated, where the position to be pruned is marked as 1 and the position to be retained is marked as 0, and the mask is passed to the re-initialization stage.

[0128] For connections marked as retained, their weight values ​​remain unchanged and are directly used as the weights output after reconstruction.

[0129] For connections to be pruned, a weighted mixed distribution is applied. Resampling is performed to generate reconstructed connection weights, where For the reconstructed connection weights, For Xavier uniform distribution, it means that in the interval [ , The random variables sampled uniformly within the network are used to ensure that the initial output variance of neurons remains stable between network layers, avoiding gradient vanishing or exploding. The boundary is uniformly distributed, determined by the input and output dimensions of the layer containing the neuron;

[0130] The distribution of healthy neurons represents a normal distribution with the mean and standard deviation of the weights of healthy neurons in the same layer as parameters. It is used to align the statistical characteristics of the new weights with the neurons that are still working normally in the current network, so that the repaired neurons can be smoothly integrated into the existing network structure and reduce the impact on the model output.

[0131] Healthy neurons are those that are not marked as having any type of quiescent risk;

[0132] The mixing ratio is used to preserve the diversity of Xavier initialization while incorporating the statistical characteristics of healthy neurons, so that the repaired neurons can be smoothly integrated into the network, reducing the impact on the model output. It is usually taken as 0.2 to 0.5, and can be determined by maximum likelihood estimation, analytic hierarchy process or online adjustment based on recovery effect.

[0133] and The mean and standard deviation of the weights of healthy neurons represent the average and standard deviation of the input connection weights of all healthy neurons in the same network layer, reflecting the dispersion of the weights in the current layer.

[0134] Next, the newly generated weight values ​​replace the corresponding positions in the original weight matrix and are passed to the bias adjustment stage, using the formula... The corrected bias value of the actively deactivated neuron is calculated, and the maximum value is used to ensure that the bias is not lower than a preset lower limit. This is the corrected bias value, used to make it easier for the weighted input of the neuron to enter the positive region of ReLU; For the initial bias, For neurons The historical activation baseline is used to describe the average output value of a neuron during the initial training phase. For neurons Mean neuron responsivity across all feature dimensions The highest neuronal responsivity among all neurons in the network; This is the lower limit threshold for the bias, used to prevent the corrected bias from becoming too negative, which could lead to the continuous inactivation of neurons. To add a positive offset to the initial bias, the offset is larger for neurons with higher responsivity, in order to ensure that they can be fully activated; This is a lower bound constraint used to prevent the bias from being too negative, causing the weighted input of the neuron to always be negative, thus making it impossible to recover its activity; The offset is the average output value of a neuron in the initial stage, which is then weighted and scaled according to the relative importance of its responsivity in the network. The purpose is to add a positive offset to actively inactive neurons that need to be reinitialized, making it easier for their weighted input to enter the positive region of ReLU, thereby solving the dead ReLU problem.

[0135] In this invention, all parameters are dimensionless by using dimensionless processing technology to remove their dimensions, and all thresholds can be obtained by the mean-standard deviation method.

[0136] Based on the current activation value behavior of each neuron node after execution, the network topology is dynamically adjusted to dynamically update the number of neuron nodes.

[0137] Specifically, the silent type determination is repeated. If the neuron node is no longer marked as a risk neuron, it is restored to normal monitoring status. That is, in subsequent tasks, the neuron is monitored according to the strategy of a healthy neuron, using standard regularization strength, the gradient boosting coefficient is restored to 1, and the re-initialization process is not executed. If it is still marked as a risk neuron and the preset maximum number of interventions has not been reached, such as 2 times, differential gradient scaling and selective re-initialization are executed again. If it is still a risk neuron and the maximum number of interventions has been reached, it is marked as an unrecoverable neuron, and structured isolation and network capacity compensation are performed.

[0138] The structured isolation and network capacity compensation process includes: judging the functional redundancy of neurons based on their historical activation occupancy ratio. Functional redundancy reflects the degree to which the function performed by a neuron in the current network can be replaced by other neurons. The higher the redundancy, the more unique and indispensable the neuron's contribution to the overall network performance.

[0139] When the historical maximum activation occupancy rate is lower than the preset occupancy threshold, it means that the corresponding neuron is rarely activated in all historical tasks, and its output contributes very little to the network decision. At this time, a soft isolation mechanism will be adopted to multiply the forward propagation activation value of the neuron by 0, thereby reducing the consumption of computing resources.

[0140] Conversely, if the historical maximum activation occupancy ratio is not lower than the preset occupancy threshold, then in the same network layer, healthy neurons with activation patterns similar to those of irreversible neurons are selected. For example, by calculating the cosine similarity of the historical activation occupancy ratio, healthy neurons with a cosine similarity exceeding the preset similarity threshold are selected. If the similarity is insufficient, then several groups of healthy neurons with larger output weights are selected.

[0141] By fine-tuning the weights of several groups of healthy neurons, they gradually absorb the feature detection function originally undertaken by the irreversible neurons. That is, each connection weight of the healthy neurons is multiplied by 1+ϵ, where ϵ is a value randomly sampled from a normal distribution. Finally, the bias of the irreversible neurons is divided equally according to the number of selected healthy neurons to correct the bias of each healthy neuron.

[0142] The updated sets of neurons are located after the feature extraction layer in the incremental learning neural network model, thereby achieving capacity protection and self-healing capabilities of the neural network topology.

[0143] Meanwhile, by calculating the proportion of newly added neuron nodes after the completion of the corresponding task to the total number of neurons involved in the task, the silencing rate is obtained, which reflects the degree of damage caused to the neural network by the current task. When the silencing rate is higher than the preset occurrence threshold, it indicates that the base strength coefficient corresponding to the current level is insufficient to protect the neurons, and the coefficient should be increased, such as by 5%. Conversely, it indicates that the regularization strength may be too high, limiting the plasticity of the model, and the coefficient should be appropriately reduced, such as by 5%. The adjusted base strength coefficient will be used as the base strength coefficient for the next task of the same related level to calculate the final regularization strength of each neuron.

[0144] In the backpropagation stage of incremental learning, this embodiment of the invention does not use the traditional uniform gradient update method. Instead, it first performs differential scaling on the original gradient based on the neuron quiescent type and the final regularization strength, so that the same loss function presents different update driving forces on different neurons. Furthermore, it achieves smooth constraint of gradient enhancement coefficient through Sigmoid mapping, thereby avoiding training instability caused by gradient mutation at the numerical level.

[0145] Furthermore, a gradient, type, and importance linkage mechanism is introduced in the weight update process, so that actively deactivated neurons obtain stronger repair gradients, passively suppressed neurons maintain low-intensity updates, and hybrid neurons adapt to the transition between the two, so that parameter updates are no longer a uniform convergence path, but have hierarchical repair and hierarchical evolution characteristics.

[0146] Meanwhile, by performing connection validity determination and pruning mask generation on actively inactivated neurons, inefficient connections are explicitly stripped away. Combined with a weighted hybrid resampling mechanism of Xavier uniform distribution and statistical distribution of healthy neurons, the weights are directionally transferred from the structural degradation space to the healthy distribution space, avoiding the distribution break caused by simple random reset.

[0147] Subsequently, by introducing the ratio of historical activation baseline to responsivity through bias correction, neurons are more likely to re-enter the ReLU effective activation range, structurally solving the problem of continuous inactivation caused by long-term negative bias. Combined with the dynamic topology adjustment mechanism, soft isolation and functional transfer are performed on neurons that cannot be recovered after multiple rounds of intervention, so that their computational overhead is suppressed while their original feature expression ability is transferred to a highly similar healthy neuron population, thereby realizing a structural self-healing path that is not polluted by failure and has redundancy that can be replaced.

[0148] Finally, the basic strength coefficient is corrected online through the silent occurrence rate feedback mechanism, so that the system can adaptively adjust the protection strength according to the impact of the current task on the network structure. In this way, the plasticity of the model and the stability of the structure can be maintained at the same time during long-term learning, reducing the risk of silent neurons accumulating and improving the stable upper limit of continuous learning ability.

[0149] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for continuous optimization of an incremental learning neural network model for campus public opinion monitoring, characterized in that, The method includes: The incremental learning neural network model to be optimized includes at least four network layers. Each of the four network layers includes at least one input layer, two hidden layers, and one output layer. Each hidden layer contains at least two neurons, including deep neurons and shallow neurons. In the incremental learning process, an incremental learning neural network model for neuron state recognition is established; Design an incremental learning neural network model, including: selecting a fully connected deep neural network as the network architecture of the incremental learning neural network model to be optimized, and using a total loss function of the fully connected deep neural network that includes a regularization loss term, a task loss term, and a weight recovery loss term; By refactoring weights, correcting biases, and applying adaptive regularization constraints, the model parameters within the incremental learning neural network model to be optimized are adaptively and iteratively optimized, thereby achieving capacity protection and self-healing capabilities of the neural network topology.

2. The method for continuous optimization of an incremental learning neural network model for campus public opinion monitoring according to claim 1, characterized in that, Design incremental learning neural network models, including: Based on the trainable parameters of the incremental learning neural network model to be optimized, the connection weights of all neurons in each network layer are obtained, and the weight occupancy data is acquired. The incremental task to be learned is input into the incremental learning neural network model to be optimized, and forward propagation is performed. The activation distribution of neurons is obtained through the first few feature extraction layers to perform association level division on the incremental task to be learned. During the incremental learning process, the activation occupancy ratio is obtained based on the activation value output by each shallow neuron in the forward propagation. Combined with the weight occupancy data, the activation data is obtained. Based on activation data and association levels, the gradient value of the activation value of each deep neuron relative to each of its connection weights is calculated. Combined with the neuron activation distribution, the sensitivity of deep neurons to the output of different neurons in the feature extraction layer is analyzed, and a neuron response function for dynamically adjusting the neuron protection strength is obtained.

3. The method for continuous optimization of an incremental learning neural network model for campus public opinion monitoring according to claim 2, characterized in that, Designing incremental learning neural network models also includes: After the forward propagation of each training batch is completed, the average neuron activation state of all samples in the current batch on the feature extraction layer is used as the average embedding vector, and the theoretical expected output of each neuron is obtained for the neuron response function of each neuron. A sliding window is set to obtain the average output of each deep neuron and compared with the theoretical expected output. If the average output is lower than the theoretical expected output, the average output is considered lower. If the value is doubled, it is marked as a potentially abnormal neuron, where The preset deviation threshold; For potentially abnormal neurons, a cumulative threshold detection method is used to compare each output mean with the historical activation baseline, and the threshold is adjusted by weighting the neuron response function to determine the neuron node. Based on the gradient values, activation values, and weight occupancy data of neuron nodes, the quiescent state of the current neuron node is analyzed to identify the neuron node type. Based on neuron nodes, we analyze the structural features of the neural network to obtain the final regularization strength for neuron structure repair and the total loss function for backpropagation and gradient update.

4. The method for continuous optimization of an incremental learning neural network model for campus public opinion monitoring according to claim 3, characterized in that, Based on neuron nodes, the structural features of the neural network are analyzed to obtain the final regularization strength for neuron structure repair and the total loss function for backpropagation and gradient update, including: Based on the association level and activation occupancy ratio of each neuron node, the intensity of the differentiated basic constraint is dynamically allocated to obtain the regularized baseline intensity for smooth transition. The structural importance of each neuron node is analyzed within the network parameter structure space. The structural importance is corrected by combining the neuron weight distribution structure and the update range structure. The multi-task importance distribution is fused by decay accumulation to form a comprehensive importance representation of the internal structural state of the adaptive incremental learning neural network model. During the training of the incremental learning neural network model, a differentiated regularization constraint is applied to the regularization baseline strength of each neuron node based on its comprehensive importance. An appropriate constraint strength is assigned to neuron nodes with different degrees of structural degradation to obtain the final regularization strength corresponding to each neuron node. A total loss function for backpropagation and gradient update is then constructed.

5. The method for continuous optimization of an incremental learning neural network model for campus public opinion monitoring according to claim 4, characterized in that, The process of constructing the total loss function includes: Based on the final regularization strength and overall importance of each neuron node, we analyze the magnitude of the change in the current connection weights relative to the historical best weights, and construct a regularization loss term to constrain parameter updates and protect the historical topology of the neural network.

6. The method for continuous optimization of an incremental learning neural network model for campus public opinion monitoring according to claim 5, characterized in that, The process of constructing the total loss function also includes: Based on the current task, calculate the prediction error of the incremental learning neural network model on the task to obtain the task loss term; Based on the structural difference between the data occupied by neuron node weights and the current weights, the degree of weight dispersion is analyzed to obtain the weight recovery loss term used to guide the recovery of neuron node structure; The regularization loss term, task loss term, and weighted recovery loss term are weighted and merged to construct the total loss function.

7. The method for continuous optimization of an incremental learning neural network model for campus public opinion monitoring according to claim 6, characterized in that, Through weight reconstruction, bias correction, and adaptive regularization constraints, the model parameters within the incremental learning neural network model to be optimized are adaptively and iteratively optimized, including: In backpropagation, the gradient value of each neuron node is differentially scaled based on the final regularization strength. Combined with the total loss function, the differentially scaled gradient is obtained and used for connection weight updates. Based on the distribution characteristics of the connection weights of the current neuron nodes in the incremental learning neural network model and their activation occupancy ratio under task-driven conditions, the validity of neuron connections is determined, and selective pruning and reinitialization are implemented according to the determination results to adjust the bias and achieve optimization of neural network parameter update strategy for continuous learning.

8. The method for continuous optimization of an incremental learning neural network model for campus public opinion monitoring according to claim 7, characterized in that, Through weight reconstruction, bias correction, and adaptive regularization constraints, the model parameters within the incremental learning neural network model to be optimized are adaptively and iteratively optimized. This also includes: Based on the current activation value of each neuron node, the network topology is dynamically adjusted to dynamically update the number of neuron nodes. The updated sets of neurons are located after the feature extraction layer in the incremental learning neural network model, thereby achieving capacity protection and self-healing capabilities of the neural network topology.

Images