Emotion prediction method and system based on KAN network, and storage medium

Abstract

The invention relates to the technical field of image prediction, and discloses an emotion prediction method and system based on a KAN network, and a storage medium, and the method comprises the steps: collecting a data set, and carrying out the preprocessing of the data set; constructing an emotion prediction model based on a KAN network, training the emotion prediction model by using the training set, and adjusting parameters of the emotion prediction model by using a cross entropy loss function in the training process to obtain a trained emotion prediction model; evaluating the performance of the trained emotion prediction model by using the verification set; and inputting a to-be-predicted image into the trained emotion prediction model, and outputting an emotion prediction result. According to the method, the Kolmogorov-Arnold network is used for replacing a traditional multi-layer perceptron for image sentiment analysis, the KAN network has remarkable advantages in the aspects of complex relation processing and logical reasoning, the sentiment prediction accuracy can be improved, training data are obtained by combining network data collection and a text generation picture model, and the method is high in practicability and high in practicability. And the workload of manual labeling is effectively reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of image prediction, and in particular to a KAN network-based emotion prediction method, system and storage medium. Background Art

[0002] Sentiment prediction refers to the process of analyzing a piece of data to determine whether it expresses positive, negative or neutral emotions. Due to the advancement of computer network technology, people are not only expressing their opinions through text, but more and more people are choosing to actively express their views and opinions through the combination of visual images and text in social media. Sentiment prediction currently mainly includes the following methods: Rule-based sentiment prediction - In rule-based sentiment prediction, the software is trained to classify sentiment based on keywords in the text. For example, if you encounter keywords such as "happy" and "beautiful", you can determine that it is a positive part of speech, and if you encounter "unlucky" and "bad", you can determine that it is a negative magnetism; sentiment prediction based on machine learning - through machine learning methods, developers use sentiment prediction algorithms to train software to recognize emotions in text like humans, including but not limited to linear regression, support vector machines, deep learning and other methods.

[0003] In the field of emotion prediction, the existing technology generally trains from the supervised direction. By collecting multiple pictures and corresponding emotion labels to train the corresponding model, the model can predict the emotions contained in other unseen photos. In the early days, pictures were collected by manual annotation for model training, but the manpower cost was very high. It is usually not only time-consuming but also laborious to annotate a large number of data sets. Later, with the rapid development of artificial intelligence models, the training set annotated by a single group is far from the number of training sets required for large models. Therefore, now almost all work is to obtain data sets by crawling data online or collecting data from special data websites. However, due to the uneven quality of photos online, some photos have emotional gray areas, which can represent both positive and negative parts of speech at the same time, and the size is different. This requires researchers to process the obtained data, and due to the large amount of data, it is inevitable that some low-quality data will not be detected, which easily leads to unsatisfactory results of model training. Summary of the invention

[0004] The primary purpose of the present invention is to overcome the problems existing in the prior art and provide a method for emotion prediction based on a KAN network. The present invention can improve the accuracy of emotion prediction and reduce labor costs.

[0005] As another object of the present invention, a system adapted to the method based on the aforementioned object is provided.

[0006] As another object of the present invention, a non-volatile storage medium suitable for storing a computer program implemented according to the method described is provided.

[0007] In order to achieve the above object, the present invention provides a KAN network-based emotion prediction method, comprising: Collecting a data set, preprocessing the data set, and dividing the preprocessed data set into a training set and a validation set in proportion; Constructing a KAN network-based emotion prediction model, using the training set to train the emotion prediction model, and using a cross entropy loss function during the training process to adjust the parameters of the emotion prediction model to obtain a trained emotion prediction model; Using the validation set to evaluate the performance of the trained emotion prediction model; The image to be predicted is input into the trained emotion prediction model, and the emotion prediction result is output.

[0008] Furthermore, the collecting of the data set and preprocessing the data set, and dividing the preprocessed data set into a training set and a validation set in proportion, specifically includes: defining emotion categories, the emotion categories including funny, excited, satisfied, respectful, disgusted, angry, fearful, and sad; Based on the emotion category, images corresponding to the emotion category are collected from a public emotion classification data set and labeled to obtain a first image set, images corresponding to the emotion category are generated using a text-generated image model and labeled to obtain a second image set, and the first image set and the second image set are combined to obtain a data set; Normalizing the images in the data set and performing one-hot encoding on the labels in the data set to obtain a preprocessed data set; The preprocessed data set is divided into a training set and a validation set in a ratio of 7:3.

[0009] Furthermore, the emotion prediction model based on the KAN network specifically includes: a convolutional layer module, a KAN network, a first self-attention module, a second self-attention module, a residual branch, a global average pooling layer and a fully connected layer. The convolutional layer module, the KAN network, the first self-attention module, the second self-attention module, the global average pooling layer and the fully connected layer are connected in sequence, one end of the residual branch is connected to the input of the first self-attention module, and the other end of the residual branch is connected to the output of the second self-attention module.

[0010] Furthermore, the emotion prediction model based on the KAN network also includes a filling module, which is located between the convolutional layer module and the KAN network, and is used to fill the feature map output by the convolutional layer module.

[0011] Furthermore, the convolutional layer module includes five convolutional layers connected in sequence, namely a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer and a fifth convolutional layer. The parameters of the first convolutional layer are: , stride is 2, output channels are 64; The parameter configuration of the second convolutional layer is: convolution kernel , stride is 2, output channels are 128; The parameter configuration of the third convolutional layer is: convolution kernel , the stride is 2, and the output channel is 256; The parameter configuration of the fourth convolutional layer is: convolution kernel , stride is 2, output channels are 512; The parameter configuration of the fifth convolutional layer is: convolution kernel , the stride is 2, the output channel is 1024; the activation function of all convolutional layers is ReLU.

[0012] Furthermore, the KAN network is a convolutional network, and the convolution kernel of the convolutional network is:

[0013] in, , , , and is a learnable parameter, for each and Take a normally distributed random value, , , is defined as follows:

[0014] For control The lines, is the location of the node.

[0015] Furthermore, the emotion prediction model is trained using the training set, and the parameters of the emotion prediction model are adjusted using a cross-entropy loss function during the training process to obtain a trained emotion prediction model, and each category of emotion image in the training set is input into the emotion prediction model to output an emotion prediction result; based on the emotion prediction result and the labels corresponding to the each category of emotion image, a cross-entropy loss function is used to calculate a loss value; with minimizing the loss value as the optimization goal, the parameters of the emotion prediction model are optimized and adjusted using a back propagation method; the above-mentioned iterative training process is continuously repeated until a preset number of iterative training times is reached or the loss value converges to obtain a trained emotion prediction model.

[0016] Furthermore, the expression of the cross entropy loss function is as follows:

[0017] in, is the total number of emotion categories, is the one-hot encoding of the true label, is the corresponding one-hot encoding of the model prediction.

[0018] In order to achieve another object of the present invention, the present invention further provides a KAN network-based emotion prediction system, the system is based on the KAN network-based emotion prediction method, comprising: Acquisition module: used to collect data sets, preprocess the data sets, and divide the preprocessed data sets into training sets and validation sets in proportion; Training module: used to build a KAN network-based emotion prediction model, use the training set to train the emotion prediction model, and use the cross entropy loss function to adjust the parameters of the emotion prediction model during the training process to obtain a trained emotion prediction model; Verification module: used to evaluate the performance of the trained emotion prediction model using the verification set; Prediction module: used to input the image to be predicted into the trained emotion prediction model and output the emotion prediction result.

[0019] In order to achieve another purpose of the present invention, the present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer-stored program is executed by a processor, the emotion prediction method based on the KAN network is implemented.

[0020] Compared with the prior art, the present invention has the following beneficial effects: The present invention constructs an emotion recognition model based on a KAN network, which includes a convolutional layer module, a KAN network, a first self-attention module, a second self-attention module, a residual branch, a global average pooling layer and a fully connected layer. The KAN network has significant advantages over MLP in processing complex relationships and logical reasoning, and can effectively improve the accuracy of emotion prediction. The present invention also obtains training data by combining network data collection and a text-generated image model. The text-generated image model can be used to directly generate images of uniform specifications, which reduces the need for a large amount of preprocessing of images from different sources in traditional methods, which can not only save a lot of human resources, but also speed up data collection. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a flow chart of a KAN network-based emotion prediction method according to Embodiment 1 of the present invention; Figure 2 is a block diagram of an emotion prediction system based on a KAN network according to Embodiment 2 of the present invention; Figure 3 This is a structural diagram of a KAN network-based emotion prediction model according to Embodiment 1 of the present invention; Figure 4 This is a data collection block diagram of Example 1 of the present invention. DETAILED DESCRIPTION

[0022] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0023] In the description of the present invention, it should be noted that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance.

[0024] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0025] Furthermore, in the description of the present invention, unless otherwise specified, “plurality” means two or more.

[0026] Example 1 like Figure 1 As shown, a KAN network-based emotion prediction method according to a preferred embodiment of the present invention comprises the following steps: S1: Collecting a data set, preprocessing the data set, and dividing the preprocessed data set into a training set and a validation set in proportion; In a feasible embodiment, S1 specifically includes: S1.1: Define emotion categories, including 1. entertainment; 2. excitement; 3. satisfaction; 4. awe; 5. disgust; 6. anger; 7. fear; and 8. sadness; S1.2: Based on the emotion category, collect images of the corresponding emotion category from the public emotion classification dataset and label them to obtain a first image set , use the text-generated image model to generate images corresponding to the emotion category and label them to obtain the second image set , combining the first image set and the second image set to obtain a data set; the public data sets include ArtPhoto: an emotional image data set designed based on psychology and art theory; Emotion6 (EmotionROI): an image data set containing 6 emotional labels, suitable for predicting emotional distribution; IESN: a data set for personalized emotional perception in social networks; Furthermore, the text-generated image model generates images corresponding to the emotion category, including text acquisition: this embodiment obtains relevant sentences expressing the above emotions through search engines such as Google and Baidu, or can obtain sentences corresponding to specific emotions through the text generation model, pre-process the collected data to exclude sentences that do not meet the corresponding emotions or cannot generate picture associations, summarize the processed sentences to obtain a total sentence library S, and classify the sentence library by emotion, respectively. , , , , , , ,

[0027] Add conditions to the statements in the above statement library, for example: "The image size is 226*226, and we hope to strongly show the emotions expressed in the text" to standardize the format and quality of the generated images. Resynthesize the statements with fixed conditions and put them in the statement library In the same way, they are divided into , , , , , , , . Select a pre-trained text-to-image model, such as OpenFULX.1, stable-diffusion, Hyper-SD, etc. Put all the statements in the model into the image generation to get the generated image , according to the specific classification label in the statement library where the specific generated statement is located , combine the two to get a corresponding data set =( , ), in this way, all the images corresponding to the sentences in the sentence library are obtained, and finally the second image set is obtained . See Figure 4 .

[0028] S1.3: performing normalization processing on the images in the data set and performing one-hot encoding processing on the labels in the data set to obtain the preprocessed data set; Normalization is to transform the first image set and the second image set The image processing in , perform one-hot encoding on the labels in the dataset, specifically: label 1 becomes [1, 0, 0, 0, 0, 0, 0, 0], label 2 becomes [0, 1, 0, 0, 0, 0, 0, 0], label 3 becomes [0, 0, 1, 0, 0, 0, 0, 0], label 4 becomes [0, 0,0, 1, 0, 0, 0, 0], label 5 becomes [0, 0, 0, 0, 1, 0, 0, 0], label 6 becomes [0, 0, 0, 0, 0,1, 0, 0], label 7 becomes [0, 0, 0, 0, 0, 0, 1, 0], label 8 becomes [0, 0, 0, 0, 0, 0, 0, 1], so that the probability distribution of subsequent model outputs can be compared with the one-hot label, which is convenient for subsequent training.

[0029] S1.4: Divide the preprocessed data set into a training set and a validation set in a ratio of 7:3.

[0030] S2: constructing a KAN network-based emotion prediction model, using the training set to train the emotion prediction model, and using a cross entropy loss function during the training process to adjust the parameters of the emotion prediction model to obtain a trained emotion prediction model; In a feasible embodiment, the emotion prediction model based on the KAN network specifically includes: a convolutional layer module, a KAN network, a first self-attention module, a second self-attention module, a residual branch, a global average pooling layer and a fully connected layer, wherein the convolutional layer module, the KAN network, the first self-attention module, the second self-attention module, the global average pooling layer and the fully connected layer are connected in sequence, one end of the residual branch is connected to the input of the first self-attention module, and the other end of the residual branch is connected to the output of the second self-attention module, specifically see Figure 3 , where the convolutional layer module includes five convolutional layers connected in sequence, namely the first convolutional layer, the second convolutional layer, the third convolutional layer, the fourth convolutional layer and the fifth convolutional layer. The parameters of the first convolutional layer are: , the stride is 2, the output channel is 64, and the input of the first convolutional layer is , the output is ; The parameter configuration of the second convolutional layer is: convolution kernel , the stride is 2, the output channel is 128, and the second convolutional layer input is , the output is ; The parameter configuration of the third convolutional layer is: convolution kernel , the stride is 2, the output channel is 256, and the input of the third convolutional layer is , the output is ; The parameter configuration of the fourth convolutional layer is: convolution kernel , the stride is 2, the output channel is 512, and the input of the fourth convolutional layer is , the output is ; The parameter configuration of the fifth convolutional layer is: convolution kernel , the stride is 2, the output channel is 1024, and the input of the fifth convolutional layer is , the output is ; The activation function of all convolutional layers is ReLU. The convolutional layer module is used to extract image information.

[0031] It should be noted that the KAN network is a convolutional network, and the convolution kernel of the convolutional network is:

[0032] in, , , , and is a learnable parameter, for each and Take a normally distributed random value, , , is defined as follows:

[0033] For control The lines, is the position of the node. The KAN network obtains high-level features of the image based on the information extracted by the convolutional layer module; the self-attention module is used to focus on the image area related to emotion, and the input is , the output is also The global average pooling layer is used to compress the input feature map into , global average pooling compresses each feature map into a single value by averaging the feature map, thereby effectively reducing the input dimension of the subsequent fully connected layers. This not only reduces the number of parameters and the complexity of the model, but also helps prevent overfitting, because fewer parameters mean fewer degrees of freedom and lower risk of overfitting; the fully connected layer first converts the input into 1024 through a tiling operation 1, then 1024 256, 256 64, 64 16, 16 The matrix product of 8 is calculated to get 8 The size of 1 is taken, and the coordinate of the array where the maximum value in the output result is the classification result of the model for the image emotion; the residual branch maintains the gradient flow to avoid gradient disappearance.

[0034] Furthermore, the emotion prediction model based on the KAN network also includes a filling module, which is located between the convolutional layer module and the KAN network. The filling module is used to fill the feature map output by the convolutional layer module with 1, thereby effectively preventing the problem of inaccurate emotion classification caused by the loss of edge information.

[0035] Furthermore, the emotion prediction model is trained using the training set, and the parameters of the emotion prediction model are adjusted using a cross-entropy loss function during the training process to obtain a trained emotion prediction model, and each category of emotion image in the training set is input into the emotion prediction model to output an emotion prediction result; based on the emotion prediction result and the labels corresponding to the each category of emotion image, a cross-entropy loss function is used to calculate a loss value; with minimizing the loss value as the optimization goal, the parameters of the emotion prediction model are optimized and adjusted using a back propagation method; the above-mentioned iterative training process is continuously repeated until a preset number of iterative training times is reached or the loss value converges to obtain a trained emotion prediction model.

[0036] Furthermore, the expression of the cross entropy loss function is as follows:

[0037] in, is the total number of emotion categories, is the one-hot encoding of the true label, is the corresponding one-hot encoding of the model prediction.

[0038] S3: Evaluate the performance of the trained emotion prediction model using the validation set; S4: Input the image to be predicted into the trained emotion prediction model, and output the emotion prediction result.

[0039] This embodiment constructs an emotion recognition model based on the KAN network, which includes a convolutional layer module, a KAN network, a first self-attention module, a second self-attention module, a residual branch, a global average pooling layer and a fully connected layer. The KAN network has significant advantages over MLP in processing complex relationships and logical reasoning, and can effectively improve the accuracy of emotion prediction. Training data is obtained by combining network data collection and text generation picture model. The text generation picture model can directly generate pictures of uniform specifications, which reduces the need for a large amount of preprocessing of pictures from different sources in traditional methods, which can not only save a lot of human resources, but also speed up data collection.

[0040] Example 2 like Figure 2 As shown, an emotion prediction system based on a KAN network according to an embodiment of the present invention includes: Acquisition module: used to collect data sets, preprocess the data sets, and divide the preprocessed data sets into training sets and validation sets in proportion; Training module: used to build a KAN network-based emotion prediction model, use the training set to train the emotion prediction model, and use the cross entropy loss function to adjust the parameters of the emotion prediction model during the training process to obtain a trained emotion prediction model; Verification module: used to evaluate the performance of the trained emotion prediction model using the verification set; Prediction module: used to input the image to be predicted into the trained emotion prediction model and output the emotion prediction result.

[0041] The system proposed in this embodiment is based on the emotion prediction method based on the KAN network proposed in Embodiment 1. Therefore, the options proposed in Embodiment 1 are also applicable to this embodiment and will not be described in detail here.

[0042] Example 3 An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer-stored program is executed by a processor, the emotion prediction method based on the KAN network is implemented.

[0043] In summary, the embodiment of the present invention provides a method, system and storage medium for emotion prediction based on a KAN network, which constructs an emotion recognition model based on a KAN network, the model includes a convolutional layer module, a KAN network, a first self-attention module, a second self-attention module, a residual branch, a global average pooling layer and a fully connected layer, wherein the KAN network has significant advantages over MLP in processing complex relationships and logical reasoning, and can effectively improve the accuracy of emotion prediction; training data is obtained by combining network data collection and a text-generated picture model, and the text-generated picture model can be used to directly generate pictures of uniform specifications, which reduces the need for a large amount of preprocessing of pictures from different sources in traditional methods, which can not only save a lot of human resources, but also speed up data collection. .

[0044] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and substitutions can be made without departing from the technical principles of the present invention. These improvements and substitutions should also be regarded as the scope of protection of the present invention.

Claims

1. A KAN network-based emotion prediction method, characterized in that: The method comprises the following steps: Collecting a data set, preprocessing the data set, and dividing the preprocessed data set into a training set and a validation set in proportion; Constructing a KAN network-based emotion prediction model, using the training set to train the emotion prediction model, and using a cross entropy loss function during the training process to adjust the parameters of the emotion prediction model to obtain a trained emotion prediction model; Using the validation set to evaluate the performance of the trained emotion prediction model; The image to be predicted is input into the trained emotion prediction model, and the emotion prediction result is output.

2. The emotion prediction method based on KAN network according to claim 1, characterized in that: The collecting of the data set, preprocessing the data set, and dividing the preprocessed data set into a training set and a validation set in proportion, specifically includes: defining emotion categories, the emotion categories including funny, excited, satisfied, respectful, disgusted, angry, fearful, and sad; Based on the emotion category, images corresponding to the emotion category are collected from a public emotion classification data set and labeled to obtain a first image set, images corresponding to the emotion category are generated using a text-generated image model and labeled to obtain a second image set, and the first image set and the second image set are combined to obtain a data set; Normalizing the images in the data set and performing one-hot encoding on the labels in the data set to obtain a preprocessed data set; The preprocessed data set is divided into a training set and a validation set in a ratio of 7:

3.

3. The emotion prediction method based on KAN network according to claim 1, characterized in that: The emotion prediction model based on the KAN network specifically includes: a convolutional layer module, a KAN network, a first self-attention module, a second self-attention module, a residual branch, a global average pooling layer and a fully connected layer. The convolutional layer module, the KAN network, the first self-attention module, the second self-attention module, the global average pooling layer and the fully connected layer are connected in sequence, one end of the residual branch is connected to the input of the first self-attention module, and the other end of the residual branch is connected to the output of the second self-attention module.

4. The emotion prediction method based on KAN network according to claim 3, characterized in that: The KAN network-based emotion prediction model also includes a filling module, which is located between the convolutional layer module and the KAN network, and is used to fill the feature map output by the convolutional layer module.

5. The emotion prediction method based on KAN network according to claim 3, characterized in that: The convolutional layer module includes five convolutional layers connected in sequence, namely the first convolutional layer, the second convolutional layer, the third convolutional layer, the fourth convolutional layer and the fifth convolutional layer. The parameters of the first convolutional layer are: , the stride is 2, and the output channels are 64; The parameter configuration of the second convolutional layer is: convolution kernel , stride is 2, output channels are 128; The parameter configuration of the third convolutional layer is: convolution kernel , the stride is 2, and the output channel is 256; The parameter configuration of the fourth convolutional layer is: convolution kernel , stride is 2, output channels are 512; The parameter configuration of the fifth convolutional layer is: convolution kernel , the stride is 2, the output channel is 1024; the activation function of all convolutional layers is ReLU.

6. The emotion prediction method based on KAN network according to claim 3, characterized in that: The KAN network is a convolutional network, and the convolution kernel of the convolutional network is: in, , , , and is a learnable parameter, for each and Take a normally distributed random value, , , is defined as follows: For control The lines, is the location of the node.

7. The emotion prediction method based on KAN network according to claim 1, characterized in that: The emotion prediction model is trained using the training set, and the parameters of the emotion prediction model are adjusted using a cross entropy loss function during the training process to obtain a trained emotion prediction model, and each category of emotion image in the training set is input into the emotion prediction model to output an emotion prediction result; based on the emotion prediction result and the labels corresponding to each category of emotion image, a loss value is calculated using a cross entropy loss function; With minimizing the loss value as the optimization goal, the back propagation method is used to optimize and adjust the parameters of the emotion prediction model; the above iterative training process is repeated until the preset number of iterative training times is reached or the loss value converges, and a trained emotion prediction model is obtained.

8. The emotion prediction method based on KAN network according to claim 1, characterized in that: The expression of the cross entropy loss function is as follows: in, is the total number of emotion categories, is the one-hot encoding of the true label, is the corresponding one-hot encoding of the model prediction.

9. A KAN network-based emotion prediction system, the system being based on the KAN network-based emotion prediction method according to any one of claims 1 to 8, comprising: Acquisition module: used to collect data sets, preprocess the data sets, and divide the preprocessed data sets into training sets and validation sets in proportion; Training module: used to build a KAN network-based emotion prediction model, use the training set to train the emotion prediction model, and use the cross entropy loss function to adjust the parameters of the emotion prediction model during the training process to obtain a trained emotion prediction model; Verification module: used to evaluate the performance of the trained emotion prediction model using the verification set; Prediction module: used to input the image to be predicted into the trained emotion prediction model and output the emotion prediction result.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer stored program is executed by the processor, the emotion prediction method based on the KAN network described in any one of claims 1 to 8 is implemented.

Images