Sentiment analysis methods, equipment, media and program products

Abstract

The present invention relates to the field of computer technology, and discloses a sentiment analysis method, device, medium and program product, including: dividing a data set containing multiple sentiment categories into different subtasks according to set regression categories; traversing each subtask, and when using the current subtask to train the pulse neural network, randomly selecting a set number of samples in the current subtask, and calculating the weight importance corresponding to each sample; forming a fuzzy set containing membership according to the weight importance, and using the fuzzy set to selectively perform gradient shielding on some neurons in the fully connected layer and some convolution kernels in the convolution structure; after traversing all subtasks, obtaining an observation model for sentiment analysis, and using the observation model to perform sentiment analysis on the data to be tested. The gradient shielding method based on fuzzy set theory of the present invention can improve the ability of continuous learning in sentiment analysis tasks, reduce the amount of data transmission, improve the hardware processing speed, and thus increase the accuracy of sentiment prediction.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a sentiment analysis method, device, medium and program product. Background Art

[0002] The pulse neural network is based on the signal transmission and processing mode of simulating brain electrical pulses, which makes it have the dual advantages of biological similarity and low energy consumption. When faced with new tasks, organisms can quickly use the skills acquired from previous knowledge to acquire new knowledge, while basically maintaining the original cognition of old knowledge. In the relevant technical scheme, the pulse neural network usually performs task-specific activation at different time steps of the pulse neurons during continuous learning. This method expands the application scenarios of the pulse neural network, but the activation of different tasks according to the time step is inconsistent with the emotional ambiguity in the sentiment analysis task, resulting in low accuracy of sentiment prediction. Therefore, how to solve the problem that the above method cannot effectively match the ambiguity of the emotion label, resulting in the low final accuracy of the emotion prediction, is a technical problem that needs to be solved urgently by technical personnel in this field. Summary of the invention

[0003] The purpose of the present invention is to provide a sentiment analysis method, device, medium and program product, which can use pulse neural network to model the problem of human subjective sentiment discrimination, improve the ability of continuous learning in sentiment analysis tasks based on the gradient shielding method of fuzzy set theory, reduce the amount of data transmission, increase the hardware processing speed, and thus improve the accuracy of sentiment prediction.

[0004] In order to solve the above technical problems, the present invention provides a sentiment analysis method, which comprises:

[0005] Divide the dataset containing multiple emotion categories into different subtasks according to the set regression categories;

[0006] Traversing each subtask, when using the current subtask to train the spiking neural network, randomly selecting a set number of samples in the current subtask, and calculating the weight importance of the spiking neural network corresponding to each sample; the backbone network of the spiking neural network includes a convolutional structure and a fully connected layer;

[0007] Forming a fuzzy set including a degree of membership according to the weight importance, and using the fuzzy set to selectively perform gradient shielding on some neurons in the fully connected layer and some convolution kernels in the convolution structure;

[0008] After traversing all subtasks, an observation model for sentiment analysis is obtained, and sentiment analysis is performed on the data to be tested using the observation model.

[0009] In a first aspect, in the above-mentioned sentiment analysis method provided by the present invention, calculating the weight importance of the spiking neural network corresponding to each sample includes:

[0010] Perform forward reasoning and back propagation on randomly selected samples to obtain the weights and gradients of the spiking neural network corresponding to each sample, as well as the output pulse emission frequency;

[0011] The weight importance of the spiking neural network corresponding to each sample is calculated based on the obtained absolute value of the weight, the absolute value of the gradient and the pulse emission frequency.

[0012] On the other hand, in the above sentiment analysis method provided by the present invention, a fuzzy set including membership is formed according to the weight importance, including:

[0013] For the fully connected layer, the importance of each pre- and post-synaptic weight connected to the current spiking neuron is obtained;

[0014] Calculate the average importance of the synaptic weights before and after the current pulse neuron to obtain the importance measure of the neuron;

[0015] Taking the importance measure of the neuron as the degree of membership in the first fuzzy set corresponding to the fully connected layer;

[0016] For the convolution structure, calculating the sum of the importance of all weights on the convolution kernel;

[0017] The sum of the importance of all weights on the convolution kernel is taken as the membership degree in the second fuzzy set corresponding to the convolution structure.

[0018] On the other hand, in the above sentiment analysis method provided by the present invention, the fuzzy set is used to selectively perform gradient shielding on some neurons in the fully connected layer and some convolution kernels in the convolution structure, including:

[0019] For the fully connected layer, the median of the membership degrees of all neurons in the first fuzzy set is used as a distinguishing point, and the neurons whose membership degrees are greater than the median are gradient shielded;

[0020] For the convolution structure, gradient masking is performed at the level of the convolution kernel.

[0021] On the other hand, in the sentiment analysis method provided by the present invention, obtaining the weight of the pulse neural network corresponding to each sample includes:

[0022] During the back propagation calculation, the weight of the spiking neural network corresponding to each sample is obtained by using the replacement gradient.

[0023] On the other hand, in the above sentiment analysis method provided by the present invention, after the current subtask training is completed, it also includes:

[0024] When the spiking neural network is trained using the next subtask, the weights of the unshielded neurons are updated, and new fuzzy sets are formed based on the updated weights and gradients.

[0025] On the other hand, in the above sentiment analysis method provided by the present invention, before obtaining the observation model for sentiment analysis, it also includes:

[0026] Select representative samples from each subtask and put them into a representative sample set with corresponding labels;

[0027] The representative sample set is used to fine-tune the classifier of the spiking neural network every preset number of subtasks.

[0028] On the other hand, in the sentiment analysis method provided by the present invention, representative samples are selected from each subtask, including:

[0029] Use a variety of data enhancement methods to process samples in each subtask and obtain the regression results corresponding to each data enhancement method;

[0030] Calculating and sorting the variances of the regression results, and extracting first samples corresponding to a first set number of variances starting from the maximum variance;

[0031] Calculate the information entropy of the samples in each subtask to obtain the output vector entropy value and sort them, and extract the second samples corresponding to the second set number of output vector entropy values ​​starting from the maximum output vector entropy value;

[0032] The first sample and the second sample are combined and deduplicated to obtain a representative sample.

[0033] On the other hand, in the above sentiment analysis method provided by the present invention, a data set containing multiple sentiment categories is divided into different subtasks according to set regression categories, including:

[0034] Divide the data set containing multiple emotion categories according to the set regression categories to obtain sample sets of different regression categories;

[0035] The sample sets of different regression categories are combined in pairs to form corresponding subtasks.

[0036] On the other hand, in the above sentiment analysis method provided by the present invention, a data set containing multiple sentiment categories is divided according to set regression categories to obtain sample sets of different regression categories, including:

[0037] According to the set regression categories including sad category, relaxed category, happy category, surprised category, fear category, angry category, disgust category, and neutral category, the data set containing multiple emotion categories is divided to obtain sad category sample set, relaxed category sample set, happy category sample set, surprised category sample set, fear category sample set, angry category sample set, disgust category sample set, and neutral category sample set;

[0038] The sample sets of different regression categories are combined in pairs to form corresponding subtasks, including:

[0039] The sad category sample set and the relaxed category sample set are combined into the first subtask;

[0040] The happy category sample set and the surprised category sample set are combined into the second subtask;

[0041] The fear category sample set and the anger category sample set are combined into the third subtask;

[0042] The disgust category sample set and the neutral category sample set are combined into the fourth subtask;

[0043] Among them, the first subtask, the second subtask, the third subtask and the fourth subtask train the pulse neural network in sequence.

[0044] On the other hand, in the sentiment analysis method provided by the present invention, using the observation model to perform sentiment analysis on the data to be tested includes:

[0045] The label of the entire video to be tested is copied and spread to each video frame of the entire video sequence to be tested;

[0046] The observation model is used to perform sentiment analysis on each video frame and output observation results.

[0047] On the other hand, in the above sentiment analysis method provided by the present invention, after outputting the observation results, it also includes:

[0048] Adding Gaussian noise to the observation results using a dynamic model to obtain an emotional state space vector;

[0049] The observation result and the emotion state space vector are fused by using Kalman filtering to obtain the final output result of each video frame.

[0050] In order to solve the above technical problems, the present invention further provides a sentiment analysis device, the device comprising:

[0051] Memory for storing computer programs;

[0052] A processor is used to implement the steps of the above-mentioned sentiment analysis method when executing the computer program.

[0053] In order to solve the above technical problem, the present invention also provides a non-volatile storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above sentiment analysis method are implemented.

[0054] In order to solve the above technical problem, the present invention also provides a computer program product, including a computer program / instruction, which implements the steps of the above sentiment analysis method when executed by a processor.

[0055] It can be seen from the above technical scheme that the present invention provides a sentiment analysis method, which includes: dividing a data set containing multiple sentiment categories into different subtasks according to set regression categories; traversing each subtask, and when using the current subtask to train the pulse neural network, randomly selecting a set number of samples in the current subtask, and calculating the weight importance of the pulse neural network corresponding to each sample; the backbone network of the pulse neural network includes a convolution structure and a fully connected layer; according to the weight importance, a fuzzy set containing membership is formed, and the fuzzy set is used to selectively perform gradient shielding on some neurons in the fully connected layer and some convolution kernels in the convolution structure; after traversing all subtasks, an observation model for sentiment analysis is obtained, and the observation model is used to perform sentiment analysis on the data to be tested.

[0056] The beneficial effect of the present invention lies in that the above-mentioned sentiment analysis method provided by the present invention first divides the data set containing multiple sentiment categories into different subtasks according to the set regression categories, and the subtasks can train the pulse neural network in turn, and the backbone network of the pulse neural network includes a convolution structure and a fully connected layer; then traverse each subtask, and when the current subtask is used for training, randomly select a set number of samples in the current subtask, and calculate the importance of the weight of the pulse neural network corresponding to each sample; according to the importance of the calculated weight, a fuzzy set containing membership is formed, and the fuzzy set is used to selectively perform gradient shielding on some neurons in the fully connected layer and some convolution kernels in the convolution structure; after traversing all subtasks, an observation model for sentiment analysis is obtained; finally, the observation model is used to perform sentiment analysis on the data to be tested. The present invention uses a biologically interpretable spiking neural network to model the problem of human subjective emotion discrimination, which is consistent with the emotion ambiguity in the emotion analysis task. The gradient shielding method of some spiking neurons and some convolution kernels based on fuzzy set theory is used to improve the ability of continuous learning in the emotion analysis task, so that the model has the ability of lifelong learning, which solves the problem that traditional continuous learning methods cannot effectively match the ambiguity of emotion labels, resulting in low accuracy of overall prediction results in the field of emotion analysis. It can also reduce the amount of data transmission, improve the hardware processing speed, and thus improve the accuracy of emotion prediction.

[0057] In addition, the present invention also provides corresponding sentiment analysis equipment, non-volatile storage medium and program products for the sentiment analysis method, which have the same or corresponding technical features as the above-mentioned sentiment analysis method and have the same effects as above. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0059] Figure 1 A flowchart of a sentiment analysis method provided by an embodiment of the present invention;

[0060] Figure 2 A schematic diagram of a continuous learning process for sentiment analysis in a sentiment analysis method provided in an embodiment of the present invention;

[0061] Figure 3 A schematic diagram of the structure of a spiking neuron provided in an embodiment of the present invention;

[0062] Figure 4 A schematic diagram of a convolution structure selection process provided by an embodiment of the present invention;

[0063] Figure 5 A schematic diagram of the position of fuzzy sets in the learning process provided by an embodiment of the present invention;

[0064] Figure 6 A schematic diagram of joint fine-tuning of representative samples accumulated across tasks provided by an embodiment of the present invention;

[0065] Figure 7 A schematic diagram of a representative sample selection method provided in an embodiment of the present invention;

[0066] Figure 8 A schematic diagram of emotion enhancement reasoning based on Kalman filtering provided by an embodiment of the present invention;

[0067] Fig. 9 A schematic diagram of the overall framework of the sentiment analysis system provided by an embodiment of the present invention;

[0068] Fig.10 A schematic diagram of the structure of a sentiment analysis device provided by an embodiment of the present invention;

[0069] Fig.11 A schematic diagram of the structure of a sentiment analysis device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0070] The Spiking Neural Network (SNN) is a neural network model inspired by biology. It belongs to the category of deep learning and has a strong biological basis. When faced with new tasks, organisms can quickly use the skills acquired from previous knowledge to acquire new knowledge, while basically maintaining the original cognition of old knowledge. This is very different from the neural network method based on transfer learning: the artificial neural network method is basically linked to the task. A specific model needs to be trained for a specific task. If the task is modified, the network needs to be retrained, and the performance loss on the original task is extremely large. This makes the transfer learning technology only able to migrate from the original task to the existing task, but there is no way to maintain the performance of the original task. In order to solve this problem, tasks similar to continuous learning of the brain have begun to become the focus of researchers.

[0071] Continuous learning methods can be divided into three categories at a macro level: regularization-based methods, replay-based methods, and architecture-based methods. Regularization-based methods control the update process of the model by limiting the changes in weights that are closely related to the old tasks, so that the performance on the new task is maximized while optimizing the objective function. This method regularizes at the synaptic level and ignores the characteristics of functional areas in the brain. Replay-based methods start from the perspective of data. When training a new task, data related to the old task is added to the training data of the new task, so that the model can take into account two different tasks at the same time. This method is heavily dependent on data and requires more resources to store data and train the network. Architecture-based methods directly split the model and divide different sub-networks to complete different sub-tasks. This greatly increases the original network and requires prior task information to select tasks.

[0072] In the related technical solutions, the method of performing task-specific activation at different time steps of spiking neurons has expanded the application scenarios of spiking neural networks. However, the activation of different tasks according to the time step does not match the emotional ambiguity in the sentiment analysis task, resulting in low performance. In order to solve the problem that the above method cannot effectively match the ambiguity of the sentiment label, resulting in low final accuracy of sentiment prediction, the present invention provides a sentiment analysis method.

[0073] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0074] In order to enable those skilled in the art to better understand the solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Figure 1 A flowchart of a sentiment analysis method provided by an embodiment of the present invention, such as Figure 1 As shown, the method includes:

[0075] S101. Divide a data set containing multiple emotion categories into different subtasks according to set regression categories.

[0076] It should be noted that sentiment analysis itself is ambiguous. Table 1 shows the ambiguity display table of sentiment.

[0077] Table 1 Display of emotional ambiguity

[0078]

[0079] From Table 1, we can see that the left side represents different sample numbers, the first row represents different categories, and the numbers in the table represent the degree of different emotion categories, where 1 represents no corresponding emotion, and 5 represents the highest degree of corresponding emotion. Through analysis, we can find that for the same sample, similar emotional experiences will be stimulated. For example, for sample 1, both relaxation and happiness are stimulated, and the emotion of relaxation is stronger than that of happiness. Samples 3, 6, 12, and 16 also stimulate these two emotions, among which sample 12 also contains the emotional component of surprise, that is, surprise, a mixed emotion. Sample 7, which also contains the emotion of surprise, presents a mixed emotion of panic including anger, disgust, and fear. From the above analysis, we can see that the classification of emotions itself has a certain mixed type. For a specific emotional stimulation process, different people have different subjective judgment criteria, which will cause blurred boundaries, which is in line with human emotional reality.

[0080] In the present invention, in order to model the above process, the sentiment analysis process is modeled as a regression task, and the overall regression category can include multiple types. When executing step S101, the data set containing multiple sentiment categories can be divided into different subtasks according to the set regression category. The present invention can use each subtask to train the pulse neural network with continuous learning ability in turn to complete the continuous learning of sentiment analysis.

[0081] S102, traversing each subtask, when using the current subtask to train the pulse neural network, randomly selecting a set number of samples in the current subtask, and calculating the weight importance of the pulse neural network corresponding to each sample; the backbone network of the pulse neural network includes a convolutional structure and a fully connected layer.

[0082] In practice, the present invention uses a spiking neural network for modeling because the information processing method of spiking neurons is closer to that of the brain, both of which rely on pulses to transmit information. When the present invention uses the current subtask to train the spiking neural network, a set number (such as 10) of samples can be randomly selected to calculate the importance of the weight corresponding to each sample.

[0083] S103. A fuzzy set including a degree of membership is formed according to the weight importance, and the fuzzy set is used to selectively perform gradient shielding on some neurons in the fully connected layer and some convolution kernels in the convolution structure.

[0084] It should be pointed out that the structure of the pulse neural network backbone network of the present invention includes convolution and fully connected layers, and these two types of structures need to consider their roles in continuous learning respectively. The present invention selects different neuron clusters to complete corresponding tasks by utilizing certain fuzzy rules, but in actual operation, the consideration of the importance of neurons is attributed to the numerical value of the weight of the neural network connected to it. By selectively setting the weight of some synapses to 0 according to the pre-set selection conditions, the shielding of the local functions of neurons is achieved, so that different neuron clusters complete different tasks and overlap with each other.

[0085] S104: After traversing all subtasks, an observation model for sentiment analysis is obtained, and sentiment analysis is performed on the data to be tested using the observation model.

[0086] In the above-mentioned sentiment analysis method provided by an embodiment of the present invention, first, a data set containing multiple sentiment categories is divided into different subtasks according to set regression categories, and the subtasks can train the pulse neural network in turn, and the backbone network of the pulse neural network includes a convolution structure and a fully connected layer; then, each subtask is traversed, and when the current subtask is used for training, a set number of samples in the current subtask are randomly selected, and the importance of the weights of the pulse neural network corresponding to each sample is calculated; a fuzzy set containing membership is formed according to the importance of the calculated weights, and the fuzzy set is used to selectively perform gradient shielding on some neurons in the fully connected layer and some convolution kernels in the convolution structure; after traversing all subtasks, an observation model for sentiment analysis is obtained; finally, the observation model is used to perform sentiment analysis on the data to be tested. The present invention uses a biologically interpretable spiking neural network to model the problem of human subjective emotion discrimination, which is consistent with the emotion ambiguity in the emotion analysis task. The gradient shielding method of some spiking neurons and some convolution kernels based on fuzzy set theory is used to improve the ability of continuous learning in the emotion analysis task, so that the model has the ability of lifelong learning, which solves the problem that traditional continuous learning methods cannot effectively match the ambiguity of emotion labels, resulting in low accuracy of overall prediction results in the field of emotion analysis. It can also reduce the amount of data transmission, improve the hardware processing speed, and thus improve the accuracy of emotion prediction.

[0087] Furthermore, in the specific implementation, in the above-mentioned sentiment analysis method provided in the embodiment of the present invention, step S101 divides the data set containing multiple sentiment categories into different subtasks according to the set regression categories, which may specifically include: dividing the data set containing multiple sentiment categories according to the set regression categories to obtain sample sets of different regression categories; combining the sample sets of different regression categories in pairs to form corresponding subtasks.

[0088] In implementation, the present invention can divide the data set containing multiple emotion categories according to the set regression categories to obtain multiple regression category sample sets. In order to build a continuous learning scenario, the whole can be divided into two categories, and the network training can be carried out in two categories in turn, which requires multiple stages, and different stages represent new and old subtasks.

[0089] Furthermore, in the specific implementation, in the above steps, the data set containing multiple emotion categories is divided according to the set regression categories to obtain sample sets of different regression categories, which can specifically include: according to the set regression categories including sad category, relaxation category, happy category, surprise category, fear category, anger category, disgust category, and neutral category, the data set containing multiple emotion categories is divided to obtain sad category sample set, relaxation category sample set, happy category sample set, surprise category sample set, fear category sample set, anger category sample set, disgust category sample set, and neutral category sample set.

[0090] Accordingly, sample sets of different regression categories are combined in pairs to form corresponding subtasks, which may specifically include: combining the sad category sample set and the relaxation category sample set to form a first subtask; combining the happy category sample set and the surprised category sample set to form a second subtask; combining the fear category sample set and the angry category sample set to form a third subtask; combining the disgust category sample set and the neutral category sample set to form a fourth subtask; wherein the first subtask, the second subtask, the third subtask and the fourth subtask train the pulse neural network in turn.

[0091] In implementation, the regression categories may include eight categories, namely, sad category, relaxed category, happy category, surprised category, fear category, angry category, disgust category, and neutral category. The present invention may require a total of eight stages in continuous learning.

[0092] The goal of continuous learning is to minimize the performance loss from the first stage to the third stage when testing the whole after multiple stages of training. The corresponding formula is as follows:

[0093] ;

[0094] in, Represents the input data, Represents the system's processing flow, the final result is the prediction result of the observation model. As mentioned above, it is divided into four stages for continuous learning: first, learning regression sentiment analysis sadness and relaxation, followed by the tasks in the order of happiness and surprise, fear and anger, disgust and neutrality. Figure 2 A schematic diagram of the continuous learning process of sentiment analysis in the sentiment analysis method provided in an embodiment of the present invention. Figure 2 As shown in Figure 1, the spiking neural network includes a backbone network and a regression head. A baseline comparison is the model obtained by training all the data at the same time. Generally speaking, the model finally obtained by continuous learning should be as close to this model as possible.

[0095] Furthermore, in a specific implementation, in the above-mentioned sentiment analysis method provided in an embodiment of the present invention, step S102 calculates the weight importance of the pulse neural network corresponding to each sample, which may specifically include: performing forward reasoning and back propagation on randomly selected samples to obtain the weights and gradients of the pulse neural network corresponding to each sample, as well as the output pulse emission frequency; and calculating the weight importance of the pulse neural network corresponding to each sample based on the absolute value of the obtained weight, the absolute value of the gradient and the pulse emission frequency.

[0096] In the above steps, obtaining the weight of the spiking neural network corresponding to each sample may specifically include: using a replacement gradient to obtain the weight of the spiking neural network corresponding to each sample during back propagation calculation.

[0097] Figure 3 Schematic diagram of the structure of a spiking neuron provided by an embodiment of the present invention. Figure 3 As shown, the structure of the pulse neuron mainly includes three parts: dendrite input, cell body processing and axon output. The function of the dendrite is to process the input. Each branch of the dendrite has a corresponding weight. The input pulse will pass through the corresponding weight, and then the result will be accumulated to the variable membrane voltage V of the cell body. When the voltage of V exceeds the threshold corresponding to the cell body, a pulse will be released through the axon, and at the same time, the membrane voltage will also be reset. The pulse neuron transmits information by repeating this pattern. The jumping characteristics of the pulse neural network make it impossible to directly use the back propagation method for weight training. The present invention uses a substitute gradient, that is, a differentiable curve is used to approximate this jump process. The normal step function is used during forward propagation, but the gradient used for weight update in the back propagation calculation is trained by finding an approximate substitute gradient. This method solves the problem that the step function cannot be differentiable during the training process of the pulse neural network.

[0098] The present invention can obtain the weight of the pulse neural network corresponding to the current subtask (such as the first subtask consisting of the sad category sample set and the relaxed category sample set) in an alternative gradient training method, wherein the selection rule of the pulse neuron is to first determine the importance of a single weight and then determine the importance of the neuron.

[0099] In practice, in the process of determining the importance of a single weight, the present invention can first randomly select 10 samples in the data set for forward reasoning and back propagation. The 10 here is only used as an example. In actual operation, this number can be adjusted and set as long as there are samples with a corresponding amount of data. Then, the classification results and corresponding gradient weights corresponding to the 10 samples are obtained, and they are assumed to be recorded as input samples , the network weight is , corresponding to the pulse emission frequency of the output category , and the corresponding gradients are . The gradient here is only used to illustrate the principle of screening, because in the actual operation process, each pulse neural network weight corresponding to the back propagation of each sample has a corresponding gradient. Since the firing frequency of the pulse neural network is between 0 and 1, the closer the pulse firing frequency is to 1, the closer the prediction corresponding to the class is to the correct label, which means that the credibility of the gradient calculated for the sample is higher. For the network weight, the present invention also uses the same method for measurement. In order to enhance the contribution of the corresponding sample gradient close to 1, the present invention calculates the overall weight importance in the following way:

[0100] ;

[0101] in, It focuses on the importance of a specific weight. In order to avoid the cancellation of positive and negative gradients of different samples, the absolute value method is used for the numerator. That is to say, the present invention only looks at the influence of the weight on the result, without considering the direction of the gradient. This can reduce the analysis dimension and the amount of calculation, and there is no need to consider the complex changes brought about by the gradient direction. The speed of model training is improved, the model is more stable during the training process, and exhibits better robustness.

[0102] Furthermore, in a specific implementation, in the above-mentioned sentiment analysis method provided by an embodiment of the present invention, step S103 forms a fuzzy set including membership according to the importance of weights, which may specifically include: for the fully connected layer, obtaining the importance of each pre- and post-synaptic weight connected to the current pulse neuron; calculating the average of the importance of the pre- and post-synaptic weights connected to the current pulse neuron to obtain the importance measure of the neuron; using the importance measure of the neuron as the membership in the first fuzzy set corresponding to the fully connected layer; for the convolution structure, calculating the sum of the importance of all weights on the convolution kernel; using the sum of the importance of all weights on the convolution kernel as the membership in the second fuzzy set corresponding to the convolution structure.

[0103] In implementation, the present invention can determine the importance of neurons according to the importance of the weights connected to the neurons. In the process of determining the importance of neurons, according to the structure of the fully connected layer, the present invention can determine the importance of the current neuron by calculating the importance of each pre / post synaptic weight connected to the current pulse neuron. The output layer and the input layer are fixed, and the neurons cannot be cut. For the output layer, different tasks are assigned to different decision neurons, which already have the selection characteristics of continuous learning: for the number of neurons in the output layer, although the setting of continuous learning is that two categories are learned separately together, the overall neurons have been reserved in advance, that is, the neurons in the output layer are based on Figure 2In the example given, the number of neurons is 8, but in the first task, that is, when classifying sad / relax, only two of them are used (without loss of generality, the first two are generally selected). Assume that the second-to-last fully connected layer has a total of 64 neurons, and the third-to-last layer has a total of 128 neurons. For each neuron in the second-to-last layer, the number of connection weights associated with it is 128+64=192 in total, and each weight has a corresponding , the present invention uses 192 The average value of is used as the importance measure of the neuron, denoted as .

[0104] It should be noted that the method is inspired by the uniqueness and sharing between different neuron subgroups in the brain. The uniqueness indicates that different neuron subgroups perform different functions, while the sharing indicates that in order to reduce the overall brain load, reusable neurons will be reused for different functions. For each task in continuous learning, the present invention constructs a fuzzy set for all neurons based on the neuron weights obtained in the previous step. The elements in this fuzzy set are the corresponding specific neuron IDs, and the neuron membership is expressed in importance. To indicate that, due to the need to reflect the difference in id, without loss of generality, a superscript is added to the importance here , the above process can be expressed as:

[0105] ;

[0106] in, represents a spiking neuron in a spiking neural network, It's the degree of membership.

[0107] Furthermore, in a specific implementation, in the above-mentioned sentiment analysis method provided in an embodiment of the present invention, step S103 uses fuzzy sets to selectively perform gradient shielding on some neurons in the fully connected layer and some convolution kernels in the convolution structure, which may specifically include: for the fully connected layer, taking the median of the membership degrees of all neurons in the first fuzzy set as the distinguishing point, and performing gradient shielding on neurons whose membership degrees are greater than the median; for the convolution structure, performing gradient shielding at the level of the convolution kernel.

[0108] In the implementation, for the structure of the fully connected layer, the The median of is used as the distinguishing point for gradient shielding. If it is greater than the median, gradient shielding is performed because it proves that these neurons contribute more to the previous task.

[0109] Figure 4Schematic diagram of the selection process of the convolution structure provided by the embodiment of the present invention. For the convolution structure, since the number of neurons contained in the feature map is large, for the convolution structure, such as Figure 4 As shown, the present invention can directly perform gradient shielding at the convolution kernel level. It only needs to change the importance of the neurons in the fully connected layer to the importance of the corresponding synaptic weights. The construction process of the fuzzy set is similar. The membership of the element (i.e., the screening of the convolution kernel) is expressed by the sum of the importance of all the weights corresponding to the convolution kernel.

[0110] Furthermore, in a specific implementation, in the above-mentioned sentiment analysis method provided in an embodiment of the present invention, after the training of the current subtask is completed, it may also include: when the pulse neural network is trained using the next subtask, the weights of the unshielded neurons are updated, and a new fuzzy set is formed based on the updated weights and gradients.

[0111] In implementation, after membership screening, all remaining neurons will be used as trainable structures for training the next subtask. If the current subtask is the first subtask, the next task is the second task, and the next subtask is the third subtask. It should be noted that when screening neurons for the next subtask, the newly constructed fuzzy set needs to be re-screened according to the importance of all neurons.

[0112] The present invention divides the pulse neurons into fuzzy sets so that different neuron subsets (which can be regarded as neuron subgroups) can perform different functions, and at the same time, they can be shared with each other to enhance the reuse rate of neurons.

[0113] Figure 5 Schematic diagram of the position of fuzzy sets in the learning process provided by the embodiment of the present invention. It should be noted that the fuzzy set strategy adopted by the present invention plays a role in maintaining the significance weight of the subtask in the previous stage during the continuous learning process. By maintaining the original value of the synaptic weight with higher importance, the performance loss between two adjacent subtasks can be minimized. The principle of screening based on the value of the membership degree in the fuzzy set plays a role in the entire learning process. Figure 5 As shown, each screening strategy is based on the weights obtained after the training of the closest subtask in the previous step to select the importance of neurons and convolution kernels, and after the selection, the present invention does not abandon the original weights, but only keeps the original weights in the calculation of the new task during the training process of the current subtask. This ensures the coupling of weights in two adjacent subtasks, but a problem that comes with it is that as the learning subtasks accumulate, the effective weights at the beginning may be modified, thereby losing the effectiveness of the early tasks.

[0114] Furthermore, in specific implementation, in the above-mentioned sentiment analysis method provided in the embodiment of the present invention, before obtaining the observation model for sentiment analysis, it can also include: selecting representative samples from each subtask, and putting them into a representative sample set in combination with the corresponding labels; and fine-tuning the classifier of the pulse neural network using the representative sample set for every preset number of subtasks.

[0115] In implementation, the present invention can fine-tune the model using samples from a representative sample set every two subtasks. In order to keep the model effective for premature tasks, the present invention selects the above method for joint training from a data perspective. In order to better maintain the continuous effectiveness of continuous learning, the present invention can start from the second subtask. After training the second subtask using the above method, fine-tune using the selected representative samples of the first three subtasks. Although this increases the complexity of the learning process to a certain extent, even in people's continuous learning, there is a situation of reviewing classic cases. Therefore, selecting representative samples for use in the later fine-tuning process can maintain the classification performance of the original task from a data perspective.

[0116] It should be noted that the above fine-tuning does not adjust all the weights of the network. On the contrary, the present invention only fine-tunes the final classifier. Because the training method based on the fuzzy set principle screening proposed in the present invention has preserved the important information in the backbone network, the performance loss in the two-step continuous learning process can be compensated to a certain extent by simply adjusting the classifier.

[0117] Furthermore, in specific implementation, in the above-mentioned sentiment analysis method provided in an embodiment of the present invention, representative samples are selected from each subtask, which may specifically include: using multiple data enhancement methods to process the samples in each subtask to obtain the regression results corresponding to each data enhancement method; calculating and sorting the variance of the regression results, starting from the maximum variance, extracting the first samples corresponding to the first set number of variances; calculating the information entropy of the samples in each subtask to obtain the output vector entropy value and sorting it, starting from the maximum output vector entropy value, extracting the second samples corresponding to the second set number of output vector entropy values; merging the first sample and the second sample to remove duplicates to obtain a representative sample.

[0118] In practice, in order to obtain a small number of representative task-related samples, it is necessary to select samples with the most class characteristics; such samples are generally obtained by clustering feature vectors, and such an operation method has a large amount of calculation; the present invention is inspired by the support vector of the support vector machine and tends to find samples at the boundary, because such samples can not only represent the essential characteristics of this category, but also have the property of distinguishing from other categories. In order to make the obtained samples diverse, the present invention combines two different representative sample selection methods. Figure 7 A schematic diagram of a representative sample selection method provided in an embodiment of the present invention. Figure 7 As shown, the present invention first adopts a variance-based method to obtain representative samples. The idea of ​​this method is that for different data enhancement methods, if the variance of the corresponding output result is larger, it means that the sample is more likely to be located at the boundary. At the same time, a method based on information entropy is used. Information entropy calculates the uncertainty of the final output vector. The larger the uncertainty, the better the boundary support of the sample for classification. Finally, the representative samples obtained by the two methods are merged and deduplicated to obtain the samples needed for fine-tuning in subsequent steps. These samples are put into a set with joint labels. This set will continue to increase with the increase of subtasks, which is equivalent to the present invention using a certain storage space plus subsequent fine-tuning to compensate for the performance decline of the overall model for previous learning tasks in the continuous learning mode.

[0119] Furthermore, in specific implementation, in the above-mentioned sentiment analysis method provided in an embodiment of the present invention, sentiment analysis is performed on the test data using an observation model, which may specifically include: copying the label of the entire test video and spreading it to each video frame in the entire test video sequence; performing sentiment analysis on each video frame using the observation model and outputting the observation results.

[0120] In practice, since it is very difficult to obtain fine-grained emotional labels, the present invention copies the labels of the entire video and spreads them to each frame of the entire video sequence. The observation model is the pulse neural network structure with continuous learning ability trained above. The present invention uses the observation model to perform emotional analysis on each video frame and output the observation results.

[0121] Furthermore, in a specific implementation, in the above-mentioned sentiment analysis method provided in an embodiment of the present invention, after outputting the observation results, it may also include: using a dynamic model to add Gaussian noise to the observation results to obtain an emotion state space vector; using Kalman filtering to fuse the observation results and the emotion state space vector to obtain the final output result of each video frame.

[0122] In implementation, in order to further improve the performance of the learning system in the reasoning process through lower computational cost, the present invention adopts Kalman filtering to enhance the prediction results of this sequence. Figure 8 A schematic diagram of emotion enhancement reasoning based on Kalman filtering provided by an embodiment of the present invention. Figure 8 As shown, based on the prior assumption that a person's emotions are relatively stable, the calculation rule of the dynamic model of the present invention is to add a certain degree of Gaussian noise on the original basis to obtain a new emotional state space vector. The observation results of each step and the prediction results of the dynamic model are fused using the Kalman gain method to obtain the output of each video frame, and finally the final output of the model is selected as the fuzzy output of the system, which further enhances the accuracy of the results in the reasoning process.

[0123] Fig. 9 The overall framework diagram of the sentiment analysis system provided by the embodiment of the present invention. In order to accelerate the training and reasoning process, the present invention can adopt the computing architecture of the host computer and the programmable logic device. The task starts from the task allocation of the host computer, and the organized sample data information and other data are sent to the programmable logic device through the bus. In the training stage, the fuzzy set rule processing module plays a role. When the fixed task is reached, the neurons and weights are screened based on the fuzzy set rules, and the weight update is also completed on the programmable logic device. In the reasoning stage, the present invention uses a dynamic model constructed using prior knowledge. The model uses the prediction results of the observation model as initialization at the beginning. Since the vector represented by the observation result is very small, even if it is transmitted through the bus, it will not cause too much data congestion. In the subsequent reasoning process, the results of the observation model and the dynamic model are sent to the Kalman filter processing module on the programmable logic device to update the output result until the final reasoning result is returned after the last frame of the video is processed.

[0124] The sentiment analysis method provided by the embodiment of the present invention is described below by way of example, and the specific steps are as follows:

[0125] Step 1: Follow Figure 2 In this way, a dataset containing multiple different categories is divided into different subtasks according to agreed categories.

[0126] Step 2: Construct a pulse neural network backbone network containing convolutional structure and fully connected structure. Here the given input is , where 20 represents the number of video frames input, that is, each step input is a two-dimensional 3-channel matrix The network architecture is conv3X3-avgpoool2X2-conv3X3-avgpoool2X2-conv3X3-avgpoool2X2-conv3X3-avgpoool2X2-conv3X3-avgpoool2X2-conv3X3-avgpoool2X2-Linear80-avgpool1d10, where conv3X3 represents The convolution kernel has 20 channels, avgpool2X2 represents two-dimensional average pooling, Linear80 represents a fully connected layer with 80 neurons, and avgpool1d10 represents a one-dimensional average pooling output that outputs the average firing rate of each category to represent the regression results of each category.

[0127] Step 3: Traverse all current subtasks and execute the operations from step 4 to step 10.

[0128] Step 4: Select all samples of the current subtask (such as sad / relaxed) category for training and testing, and use the first two of the final eight output results as the output results of the current subtask (sad / relaxed). Use the proxy gradient method based on the data to obtain the convergence weight of the spiking neural network at this time.

[0129] Step 5: Randomly select 10 samples from the training samples to calculate the corresponding weight importance, and selectively shield the weights of some neurons in the Linear80 layer according to the fuzzy set membership rule neurons.

[0130] Step 6: For the convolution structure, selectively shield the gradient update of some convolution kernels according to the above fuzzy set membership rule.

[0131] Step 7: Select the representative sample of the current subtask according to the representative sample selection method and add it to the representative sample set.

[0132] Step 8: According to Figure 6 In the process, the next subtask (such as happiness / surprise) is selected for training, and only the weights that are not blocked are updated.

[0133] Step 9: Reconstruct the fuzzy set rules according to the weight and gradient of the next subtask for subsequent calculations, and add the representative samples of the next subtask to the representative sample set.

[0134] Step 10: Fine-tune all weights of the current model using samples from the representative sample set.

[0135] Step 11. After the traversal of step 3 is completed, the final fixed weight model is obtained. In the inference stage, use Figure 8 The reasoning results are enhanced in this way.

[0136] In the above embodiments, the sentiment analysis method is described in detail, and the present invention also provides corresponding embodiments of the sentiment analysis device and the sentiment analysis equipment. It should be noted that the present invention describes the embodiments of the device part from two perspectives, one is based on the functional module perspective, and the other is based on the hardware perspective.

[0137] Fig.10 This is a schematic diagram of the structure of the sentiment analysis device provided by an embodiment of the present invention. This embodiment is based on the perspective of functional modules, such as Fig.10 As shown, the device comprises:

[0138] A task division module 10 is used to divide a data set containing multiple emotion categories into different subtasks according to set regression categories;

[0139] The network training module 11 is used to traverse each subtask, and when the current subtask is used to train the pulse neural network, randomly select a set number of samples in the current subtask, and calculate the weight importance of the pulse neural network corresponding to each sample; the backbone network of the pulse neural network includes a convolution structure and a fully connected layer; a fuzzy set containing membership is formed according to the weight importance, and the fuzzy set is used to selectively perform gradient shielding on some neurons in the fully connected layer and some convolution kernels in the convolution structure;

[0140] The model reasoning module 12 is used to obtain the observation model for sentiment analysis after traversing all subtasks, and use the observation model to perform sentiment analysis on the data to be tested.

[0141] In the above-mentioned sentiment analysis device provided by the embodiment of the present invention, the interaction of the above-mentioned three modules can be used to use a biologically interpretable pulse neural network to model the problem of human subjective sentiment discrimination, which is consistent with the sentiment ambiguity in the sentiment analysis task. The gradient shielding method of some pulse neurons and some convolution kernels based on fuzzy set theory can be used to improve the ability of continuous learning in the sentiment analysis task, so that the model has the ability of lifelong learning, which solves the problem that traditional continuous learning methods cannot effectively match the ambiguity of sentiment labels, resulting in low accuracy of overall prediction results in the field of sentiment analysis, and can reduce the amount of data transmission, improve hardware processing speed, and thus improve the accuracy of sentiment prediction.

[0142] Since the embodiments of the device part correspond to the embodiments of the method part, the embodiments of the device part refer to the description of the embodiments of the method part, which will not be described here. And it has the same beneficial effects as the above-mentioned sentiment analysis method.

[0143] Furthermore, in a specific implementation, in the above-mentioned sentiment analysis device provided in an embodiment of the present invention, the task division module 10 can be specifically used to divide a data set containing multiple sentiment categories according to set regression categories to obtain sample sets of different regression categories; and the sample sets of different regression categories are combined in pairs to form corresponding subtasks.

[0144] In implementation, the task division module 10 can first divide the data set containing multiple emotion categories according to the set regression categories including sadness category, relaxation category, happiness category, surprise category, fear category, anger category, disgust category, and neutral category to obtain sadness category sample set, relaxation category sample set, happy category sample set, surprise category sample set, fear category sample set, anger category sample set, disgust category sample set, and neutral category sample set, and then combine the sadness category sample set and the relaxation category sample set into the first subtask; combine the happy category sample set and the surprise category sample set into the second subtask; combine the fear category sample set and the anger category sample set into the third subtask; and combine the disgust category sample set and the neutral category sample set into the fourth subtask.

[0145] Furthermore, in a specific implementation, in the above-mentioned sentiment analysis device provided in an embodiment of the present invention, the network training module 11 may include: a first computing unit, used to perform forward reasoning and back propagation on randomly selected samples to obtain the weights and gradients of the pulse neural network corresponding to each sample, and the output pulse emission frequency; based on the absolute value of the obtained weight, the absolute value of the gradient and the pulse emission frequency, the weight importance of the pulse neural network corresponding to each sample is calculated.

[0146] Furthermore, in a specific implementation, in the above-mentioned sentiment analysis device provided by an embodiment of the present invention, the network training module 11 may also include: a second calculation unit, for obtaining, for the fully connected layer, the importance of each pre- and post-synaptic weight connected to the current pulse neuron; calculating the average value of the importance of the pre- and post-synaptic weights connected to the current pulse neuron to obtain the importance measure of the neuron; using the importance measure of the neuron as the membership in the first fuzzy set corresponding to the fully connected layer; for the convolution structure, calculating the sum of the importance of all weights on the convolution kernel; using the sum of the importance of all weights on the convolution kernel as the membership in the second fuzzy set corresponding to the convolution structure.

[0147] Furthermore, in a specific implementation, in the above-mentioned sentiment analysis device provided by an embodiment of the present invention, the network training module 11 may also include: a gradient shielding unit, which is used for, for the fully connected layer, taking the median of the membership of all neurons in the first fuzzy set as a distinguishing point, and performing gradient shielding on neurons whose values ​​are greater than the median; for the convolution structure, gradient shielding is performed at the level of the convolution kernel.

[0148] Furthermore, in a specific implementation, in the above-mentioned sentiment analysis device provided in an embodiment of the present invention, the network training module 11 can also be used to update the weights of unshielded neurons when the pulse neural network is trained using the next subtask, and to form a new fuzzy set based on the updated weights and gradients.

[0149] Furthermore, in a specific implementation, the sentiment analysis device provided in the embodiment of the present invention may also include: a model fine-tuning module, which is used to select representative samples from each subtask and put them into a representative sample set in combination with corresponding labels; and fine-tune the classifier of the pulse neural network using the representative sample set every preset number of subtasks.

[0150] Furthermore, in specific implementation, in the above-mentioned sentiment analysis method provided in an embodiment of the present invention, the model inference module 12 can be specifically used to copy the label of the entire video to be tested, and diffuse it to each video frame on the entire video sequence to be tested; use the observation model to perform sentiment analysis on each video frame, and output the observation result; it can also be specifically used to use the dynamic model to add Gaussian noise to the observation result to obtain the sentiment state space vector; the observation result and the sentiment state space vector are fused using Kalman filtering to obtain the final output result of each video frame.

[0151] Fig.11 This is a schematic diagram of the structure of the sentiment analysis device provided by the embodiment of the present invention. This embodiment is based on the hardware perspective, such as Fig.11 As shown, the sentiment analysis device includes:

[0152] A memory 20, used for storing computer programs;

[0153] The processor 21 is used to implement the steps of the sentiment analysis method mentioned in the above embodiment when executing the computer program.

[0154] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one hardware form of a digital signal processor (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU; the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 21 may be integrated with a graphics processing unit (GPU), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.

[0155] The memory 20 may include one or more non-volatile storage media, which may be non-transitory. The memory 20 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 20 is at least used to store the following computer program 201, wherein, after the computer program is loaded and executed by the processor 21, it can implement the relevant steps of the sentiment analysis method disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 20 may also include an operating system 202 and data 203, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 202 may include Windows, Unix, Linux, etc. The data 203 may include, but is not limited to, the data involved in the sentiment analysis method mentioned above.

[0156] In some embodiments, the sentiment analysis device may further include a display screen 22, an input / output interface 23, a communication interface 24, a power supply 25, and a communication bus 26. Those skilled in the art will appreciate that Fig.11 The structure shown in does not constitute a limitation on the sentiment analysis device, and may include more or fewer components than shown in the figure. The sentiment analysis device provided by the embodiment of the present invention includes a memory and a processor. When the processor executes the program stored in the memory, it can implement the following method: sentiment analysis method, the effect is the same as above.

[0157] Finally, the present invention also provides an embodiment corresponding to a non-volatile storage medium. The non-volatile storage medium stores a computer program, and when the computer program is executed by a processor, the steps recorded in the above method embodiment are implemented.

[0158] It is understandable that if the method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or part of the contribution to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and executes all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code. The non-volatile storage medium provided by the present invention can implement the above-mentioned sentiment analysis method, and the effect is the same as above.

[0159] Finally, the present invention also provides an embodiment corresponding to a computer program product. The computer program product includes a computer program / instruction, and when the computer program / instruction is executed by a processor, the steps described in the above-mentioned sentiment analysis method embodiment are implemented. The computer program product provided by the present invention can implement the above-mentioned sentiment analysis method, and the effect is the same as above.

[0160] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0161] The above is a detailed introduction to the sentiment analysis method, device, medium and program product provided by the present invention. The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the embodiments can be referenced to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principle of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the scope of protection of the present invention.

Claims

1. A sentiment analysis method, characterized in that: The method comprises: Divide the dataset containing multiple emotion categories into different subtasks according to the set regression categories; Traversing each subtask, when using the current subtask to train the spiking neural network, randomly selecting a set number of samples in the current subtask, and calculating the weight importance of the spiking neural network corresponding to each sample; the backbone network of the spiking neural network includes a convolutional structure and a fully connected layer; Forming a fuzzy set including a degree of membership according to the weight importance, and using the fuzzy set to selectively perform gradient shielding on some neurons in the fully connected layer and some convolution kernels in the convolution structure; After traversing all subtasks, an observation model for sentiment analysis is obtained, and sentiment analysis is performed on the data to be tested using the observation model.

2. The sentiment analysis method according to claim 1, characterized in that: Calculating the weight importance of the spiking neural network corresponding to each sample, including: Perform forward reasoning and back propagation on randomly selected samples to obtain the weights and gradients of the spiking neural network corresponding to each sample, as well as the output pulse emission frequency; The weight importance of the spiking neural network corresponding to each sample is calculated based on the obtained absolute value of the weight, the absolute value of the gradient and the pulse emission frequency.

3. The sentiment analysis method according to claim 1, characterized in that: According to the weight importance, a fuzzy set containing membership is formed, including: For the fully connected layer, the importance of each pre- and post-synaptic weight connected to the current spiking neuron is obtained; Calculate the average importance of the synaptic weights before and after the current pulse neuron to obtain the importance measure of the neuron; Taking the importance measure of the neuron as the degree of membership in the first fuzzy set corresponding to the fully connected layer; For the convolution structure, calculating the sum of the importance of all weights on the convolution kernel; The sum of the importance of all weights on the convolution kernel is taken as the membership degree in the second fuzzy set corresponding to the convolution structure.

4. The sentiment analysis method according to claim 3, characterized in that: Selectively performing gradient shielding on some neurons in the fully connected layer and some convolution kernels in the convolution structure by using the fuzzy set, including: For the fully connected layer, the median of the membership degrees of all neurons in the first fuzzy set is used as a distinguishing point, and the neurons whose membership degrees are greater than the median are gradient shielded; For the convolution structure, gradient masking is performed at the level of the convolution kernel.

5. The sentiment analysis method according to claim 2, characterized in that: Obtaining the weight of the spiking neural network corresponding to each sample includes: During the back propagation calculation, the weight of the spiking neural network corresponding to each sample is obtained by using the replacement gradient.

6. The sentiment analysis method according to claim 1, characterized in that: After the current subtask training is completed, it also includes: When the spiking neural network is trained using the next subtask, the weights of the unshielded neurons are updated, and new fuzzy sets are formed based on the updated weights and gradients.

7. The sentiment analysis method according to claim 1, characterized in that: Before obtaining the observation model for sentiment analysis, it also includes: Select representative samples from each subtask and put them into a representative sample set with corresponding labels; The representative sample set is used to fine-tune the classifier of the spiking neural network every preset number of subtasks.

8. The sentiment analysis method according to claim 7, characterized in that: Select representative samples from each subtask, including: Use a variety of data enhancement methods to process samples in each subtask and obtain the regression results corresponding to each data enhancement method; Calculating and sorting the variances of the regression results, and extracting first samples corresponding to a first set number of variances starting from the maximum variance; Calculate the information entropy of the samples in each subtask to obtain the output vector entropy value and sort them, and extract the second samples corresponding to the second set number of output vector entropy values ​​starting from the maximum output vector entropy value; The first sample and the second sample are combined and deduplicated to obtain a representative sample.

9. The sentiment analysis method according to claim 1, characterized in that: The dataset containing multiple emotion categories is divided into different subtasks according to the set regression categories, including: Divide the data set containing multiple emotion categories according to the set regression categories to obtain sample sets of different regression categories; The sample sets of different regression categories are combined in pairs to form corresponding subtasks.

10. The sentiment analysis method according to claim 9, characterized in that: The dataset containing multiple emotion categories is divided according to the set regression categories to obtain sample sets of different regression categories, including: According to the set regression categories including sad category, relaxed category, happy category, surprised category, fear category, angry category, disgust category, and neutral category, the data set containing multiple emotion categories is divided to obtain sad category sample set, relaxed category sample set, happy category sample set, surprised category sample set, fear category sample set, angry category sample set, disgust category sample set, and neutral category sample set; The sample sets of different regression categories are combined in pairs to form corresponding subtasks, including: The sad category sample set and the relaxed category sample set are combined into the first subtask; The happy category sample set and the surprised category sample set are combined into the second subtask; The fear category sample set and the anger category sample set are combined into the third subtask; The disgust category sample set and the neutral category sample set are combined into the fourth subtask; Among them, the first subtask, the second subtask, the third subtask and the fourth subtask train the pulse neural network in sequence.

11. The sentiment analysis method according to claim 1, characterized in that: Using the observation model to perform sentiment analysis on the data to be tested includes: The label of the entire video to be tested is copied and spread to each video frame of the entire video sequence to be tested; The observation model is used to perform sentiment analysis on each video frame and output observation results.

12. The sentiment analysis method according to claim 11, characterized in that: After outputting the observations, it also includes: Adding Gaussian noise to the observation results using a dynamic model to obtain an emotional state space vector; The observation result and the emotion state space vector are fused by using Kalman filtering to obtain the final output result of each video frame.

13. A sentiment analysis device, characterized in that: The device comprises: Memory for storing computer programs; A processor, configured to implement the steps of the sentiment analysis method according to any one of claims 1 to 12 when executing the computer program.

14. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the sentiment analysis method according to any one of claims 1 to 12 are implemented.

15. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the sentiment analysis method according to any one of claims 1 to 12 are implemented.

Images