Image emotion prediction method based on double attention and diversified knowledge distillation

Abstract

The invention provides an image emotion prediction method based on double attention and diversified knowledge distillation, and the method comprises the steps: constructing a channel attention module based on a channel attention mechanism, constructing a feature fusion module based on a space attention mechanism, obtaining an image emotion data set, and carrying out the image emotion prediction. The emotion image in the image emotion data set is preprocessed to obtain a preprocessed emotion image, and the preprocessed emotion image is processed through ConvNeXt, a channel attention module and a feature fusion module in sequence to obtain fused multi-scale features; and processing the fused multi-scale features through an improved Transform encoder, average pooling, a full connection layer and a classifier in sequence to obtain probability distribution output by the classifier. According to the method, the characterization capability of the multi-scale features is further enhanced by increasing channel attention and space attention, semantic gaps among the multi-scale features are reduced, key local information is reserved, and multi-scale context information is fully mined.

Description

Technical Field

[0001] This invention relates to the field of computer image processing technology, and in particular to an image sentiment prediction method based on dual attention and diverse knowledge distillation. Background Technology

[0002] Sentiment analysis originates from an important branch of natural language processing. However, due to the vast diversity of languages ​​globally, building a text-based, universal, and effective sentiment analysis model is challenging. In recent years, with the continuous development of computer technology, more and more people enjoy recording their daily lives on social media platforms to better express their private emotions. Because acquiring images is easier than ever before, the form of recorded information is no longer limited to text, but increasingly includes images, animations, and videos. Related organizations have developed numerous applications using this multimedia data, including opinion mining, business intelligence, mental health, and entertainment assistance.

[0003] Currently, existing image sentiment prediction methods mainly suffer from the following problems: 1. Image sentiment analysis datasets are mostly manually labeled, which is highly subjective. Therefore, predicting sentiment categories requires attention to both the global image and local regions, which limits the receptive field. 2. Due to the complexity and diversity of image content, current methods often choose to build networks with multiple branches to learn different but complementary features in a targeted manner. However, this requires prior identification of which prior knowledge is beneficial to image sentiment analysis. 3. When fusing features from different branches, the inherent relationships between features are easily overlooked. Furthermore, multiple branches increase model complexity and training burden. 4. Currently, most image sentiment analysis tasks use supervised learning methods, and the constructed image sentiment analysis models rely excessively on manual annotation. The models cannot actively adjust themselves and fail to extract valuable feature information from the models themselves. Summary of the Invention

[0004] In view of the above, the main objective of this invention is to propose an image sentiment prediction method and system based on dual attention and diversified knowledge distillation to solve the aforementioned technical problems.

[0005] This invention proposes an image sentiment prediction method based on dual attention and diverse knowledge distillation, the method comprising the following steps: Step 1: Construct a channel attention module based on the channel attention mechanism, a feature fusion module based on the spatial attention mechanism, and an improved Transformer encoder based on the probabilistic sparse self-attention mechanism; based on the self-distillation technique, construct a teacher model using the channel attention module, feature fusion module, improved Transformer encoder, ConvNeXt, and classifier; construct a student model using the classifier and ConvNeXt. Step 2: Obtain the image sentiment dataset, preprocess the sentiment images in the image sentiment dataset, and input them into ConvNeXt for feature extraction to obtain the downsampled feature map, the feature map generated by the second layer ConvNeXt, and the upsampled feature map respectively. Step 3: Input the downsampled feature map, the feature map generated by the second layer ConvNeXt, and the upsampled feature map into the channel attention module for processing to obtain the first weighted feature map, the second weighted feature map, and the third weighted feature map, respectively. Step 4: Input the first weighted feature map, the second weighted feature map, and the third weighted feature map into the feature fusion module for feature extraction and fusion to obtain the fused multi-scale features.

[0006] Step 5: The fused multi-scale features are sequentially processed through an improved Transformer encoder, average pooling, a fully connected layer, and a classifier to obtain the probability distribution output by the classifier. Based on the probability distribution output by the classifier, cross-entropy loss function and relative entropy loss function are constructed respectively. The classifier is trained using the cross-entropy loss function and relative entropy loss function to obtain the trained classifier. Based on the feature map generated by ConvNeXt, a feature distillation loss function is constructed. ConvNeXt is trained using the feature distillation loss function to obtain the trained ConvNeXt. The trained classifier and the trained ConvNeXt constitute the trained student model. The sentiment image is input into the trained student model to obtain the prediction result.

[0007] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention extracts multi-scale features, enabling the model to acquire sufficient global information while also focusing on as much fine-grained information as possible, thereby enhancing the receptive field and accurately describing the emotional semantics contained in the image, achieving objective and accurate image sentiment prediction. 2. This invention enhances the representational ability of multi-scale features by adding channel attention and spatial attention, reduces the semantic gap between multi-scale features, retains key local information, and fully explores multi-scale contextual information. 3. By introducing the concept of self-distillation to construct a closed-loop knowledge iteration mechanism, the real labels guide the results of all classifiers, the deepest classifier Logits layer guides the shallow classifier Logits layer, and the deepest features of the improved Transformer encoder guide the shallow features at different scales, thus realizing efficient interaction between deep and shallow modules, as well as interaction between CNN and Transformer encoder, and comprehensively improving model performance.

[0008] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by means of embodiments of the invention. Attached Figure Description

[0009] Figure 1 This is a flowchart illustrating the steps of an image sentiment prediction method based on dual attention and diverse knowledge distillation proposed in this invention. Figure 2 This is a diagram illustrating the overall framework of an image sentiment prediction method based on dual attention and diverse knowledge distillation proposed in this invention. Figure 3 This is a structural diagram of the channel attention module proposed in this invention; Figure 4 This is a structural diagram of the feature fusion module proposed in this invention. Detailed Implementation

[0010] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0011] These and other aspects of the embodiments of the present invention will become clear from the following description and accompanying drawings. In these descriptions and drawings, some specific embodiments of the present invention are specifically disclosed to illustrate some ways of implementing the principles of the embodiments of the present invention; however, it should be understood that the scope of the embodiments of the present invention is not limited thereto.

[0012] Please see Figure 1 This embodiment provides an image sentiment prediction method based on dual attention and diverse knowledge distillation, the method including the following steps: Step 1: Construct a channel attention module based on the channel attention mechanism, a feature fusion module based on the spatial attention mechanism, and an improved Transformer encoder based on the probabilistic sparse self-attention mechanism; based on the self-distillation technique, use the channel attention module, feature fusion module, improved Transformer encoder, ConvNeXt, and classifier to form a teacher model; use the classifier and ConvNeXt to form a student model.

[0013] Step 2: Obtain the image sentiment dataset, preprocess the sentiment images in the image sentiment dataset, and input them into ConvNeXt for feature extraction to obtain the downsampled feature map, the feature map generated by the second layer ConvNeXt, and the upsampled feature map, respectively.

[0014] Please see Figure 2 In step 2, an image sentiment dataset is obtained, the sentiment images in the dataset are preprocessed, and then input into ConvNeXt for feature extraction to obtain the downsampled feature map, the feature map generated by the second layer ConvNeXt, and the upsampled feature map, respectively. Specifically, this includes the following sub-steps: Obtain the image sentiment dataset, preprocess the sentiment images in the image sentiment dataset, and obtain the preprocessed sentiment images; The preprocessed emotion image is input into ConvNeXt for feature extraction to obtain feature maps generated by the first layer of ConvNeXt, the second layer of ConvNeXt, and the third layer of ConvNeXt, respectively. The feature map generated by the first layer ConvNeXt is downsampled to obtain the downsampled feature map. The following relationship exists in the corresponding process: ; in, This represents the feature map after downsampling. This indicates that a downsampling operation has been performed. This represents the feature map generated by the first layer of ConvNeXt; Upsampling is performed on the feature map generated by the third layer ConvNeXt to obtain the upsampled feature map. The following relationship exists in the corresponding process: ; in, This represents the feature map after upsampling. This indicates that an upsampling operation has been performed. This represents the feature map generated by the third layer ConvNeXt.

[0015] Furthermore, the emotional images in the image emotion dataset are preprocessed, specifically including the following sub-steps: All emotional images in the image emotion dataset were normalized, with the mean set to [0.485, 0.456, 0.406] and the standard deviation set to [0.229, 0.224, 0.225]. For the training set images, each image is randomly cropped to a resolution of 224×224 and then randomly flipped horizontally with a probability of 0.5. For the test set images, each image was scaled to a resolution of 256×256, and then cropped from the center to a resolution of 224×224.

[0016] Step 3: Input the downsampled feature map, the feature map generated by the second layer ConvNeXt, and the upsampled feature map into the channel attention module for processing to obtain the first weighted feature map, the second weighted feature map, and the third weighted feature map, respectively.

[0017] Please see Figure 3 In step 3, the downsampled feature map, the feature map generated by the second layer ConvNeXt, and the upsampled feature map are respectively input into the channel attention module for processing to obtain the first weighted feature map, the second weighted feature map, and the third weighted feature map, which specifically includes the following sub-steps: S301. Perform average pooling on the downsampled feature map to obtain the first average pooled feature. The following relationship exists in the corresponding process: ; in, This represents the first average pooling characteristic. This indicates that the average pooling process has been performed. S302. Perform max pooling on the downsampled feature map to obtain the first max pooled feature. The following relationship exists in the corresponding process: ; in, This represents the first max pooling feature. This indicates that it has undergone max pooling. S303. The first average pooling feature and the first max pooling feature are respectively input into a multilayer perceptron for processing, then added together, and processed using the Sigmoid function to obtain the attention value of the first channel. The following relationship exists in the corresponding process: ; in, This represents the attention value of the first channel. This indicates that the signal has been processed by the Sigmoid function. This indicates that the data has been processed by a multilayer perceptron. S304. Multiply the attention value of the first channel with the downsampled feature map to obtain the first weighted feature map. The following relationship exists in the corresponding process: ; in, This represents the first weighted feature map; Repeat steps S301 to S304 for the feature map generated by the second layer ConvNeXt and the upsampled feature map respectively to obtain the second weighted feature map and the third weighted feature map respectively.

[0018] Step 4: Input the first weighted feature map, the second weighted feature map, and the third weighted feature map into the feature fusion module for feature extraction and fusion to obtain the fused multi-scale features.

[0019] Please see Figure 4 In step 4, the first weighted feature map, the second weighted feature map, and the third weighted feature map are respectively input into the feature fusion module for feature extraction and fusion to obtain the fused multi-scale features. Specifically, this includes the following sub-steps: The first weighted feature map, the second weighted feature map, and the third weighted feature map are reshaped to obtain the first feature map, the second feature map, and the third feature map, respectively. The first feature map is multiplied by the transpose of the second feature map, and then processed using the Softmax function to obtain the first spatial attention map. The following relationship exists in the corresponding process: ; in, This represents the first spatial attention map. Represents the first feature map. Represents matrix multiplication. Represents the second feature map; Multiplying the second feature map with the first spatial attention map by matrix multiplication yields the key feature map. The following relationship exists in the correspondence process: ; in, Represents the key feature map; The key feature map is multiplied by the transpose of the second feature map, and then processed using the Softmax function to obtain the second spatial attention map. The following relationship exists in the corresponding process: ; in, This represents the second spatial attention map. Represents the third feature map; The third feature map is multiplied by the second spatial attention map to obtain the final feature map. The following relationship exists in the corresponding process: ; in, This represents the final feature map; The final feature map is reshaped and then residually connected to the upsampled feature map to obtain the fused multi-scale features. ; in, This represents the multi-scale features after fusion.

[0020] It should be noted that, in Figure 4 middle, This represents the second weighted feature map. This represents the third weighted feature map. This indicates a residual connection.

[0021] Step 5: The fused multi-scale features are sequentially processed through an improved Transformer encoder, average pooling, a fully connected layer, and a classifier to obtain the probability distribution output by the classifier. Based on the probability distribution output by the classifier, cross-entropy loss function and relative entropy loss function are constructed respectively. The classifier is trained using the cross-entropy loss function and relative entropy loss function to obtain the trained classifier. Based on the feature map generated by ConvNeXt, a feature distillation loss function is constructed. ConvNeXt is trained using the feature distillation loss function to obtain the trained ConvNeXt. The trained classifier and the trained ConvNeXt constitute the trained student model. The sentiment image is input into the trained student model to obtain the prediction result.

[0022] In step 5, the fused multi-scale features are sequentially processed through an improved Transformer encoder, average pooling, a fully connected layer, and a classifier to obtain the probability distribution output by the classifier. Based on this probability distribution, a cross-entropy loss function and a relative entropy loss function are constructed. These functions are then used to train the classifier, resulting in a trained classifier. A feature distillation loss function is constructed based on the feature maps generated by ConvNeXt. This function is then used to train ConvNeXt, resulting in a trained ConvNeXt. The trained classifier and the trained ConvNeXt constitute the trained student model. The sentiment image is then input into the trained student model to obtain the prediction result. This process includes the following sub-steps: The fused multi-scale features are sequentially processed through an improved Transformer encoder, average pooling, and a fully connected layer to obtain the output features of the fully connected layer. The following relationship exists in the corresponding process: ; in, Indicates the first Output characteristics of fully connected layers Represents the weight vector. This indicates that the code has been processed using an improved Transformer encoder. Indicates the bias term; The output features of the fully connected layer are input into the classifier for processing to obtain the probability distribution of the classifier output. The following relationship exists in the corresponding process: ; in, Indicates the first The probability distribution output by the layer classifier. Indicates the first Layer classifier, This represents a temperature parameter; the higher the temperature, the smoother the probability distribution. Based on the probability distribution of the classifier output, the cross-entropy loss function and the relative entropy loss function are constructed respectively. The classifier is trained using the cross-entropy loss function and the relative entropy loss function to obtain the trained classifier. The expression for the cross-entropy loss function is as follows: ; in, Represents cross-entropy loss, Indicates the weighting coefficient. This represents the total number of classifiers. Represents the cross-entropy loss function. Indicates the true label; The expression for the relative entropy loss function is as follows: ; in, Represents relative entropy loss. Represents relative entropy. Represents the last layer classifier The probability distribution of the output; The feature distillation loss function is constructed based on the feature map generated by ConvNeXt. ConvNeXt is then trained using the feature distillation loss function to obtain the trained ConvNeXt. The following relationship exists in the corresponding process: ; in, Indicates characteristic distillation loss, Indicates hyperparameters, Indicates the first Feature maps generated by layer ConvNeXt, This represents the features extracted by the last layer of the improved Transformer encoder. This indicates taking the 2-norm; The trained classifier and the trained ConvNeXt together constitute the trained student model. The sentiment image is input into the trained student model to obtain the prediction result.

[0023] Furthermore, the total training loss function is: ; in, This represents the total training loss.

[0024] It should be understood that although the steps in the flowcharts of the various embodiments of the present invention are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the various embodiments may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0025] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0026] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0027] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. An image sentiment prediction method based on dual attention and diverse knowledge distillation, characterized in that, The method includes the following steps: Step 1: Construct a channel attention module based on the channel attention mechanism, a feature fusion module based on the spatial attention mechanism, and an improved Transformer encoder based on the probabilistic sparse self-attention mechanism; based on the self-distillation technique, construct a teacher model using the channel attention module, feature fusion module, improved Transformer encoder, ConvNeXt, and classifier; construct a student model using the classifier and ConvNeXt. Step 2: Obtain the image sentiment dataset, preprocess the sentiment images in the image sentiment dataset, and input them into ConvNeXt for feature extraction to obtain the downsampled feature map, the feature map generated by the second layer ConvNeXt, and the upsampled feature map, respectively. Step 3: Input the downsampled feature map, the feature map generated by the second layer ConvNeXt, and the upsampled feature map into the channel attention module for processing to obtain the first weighted feature map, the second weighted feature map, and the third weighted feature map, respectively. Step 4: Input the first weighted feature map, the second weighted feature map, and the third weighted feature map into the feature fusion module for feature extraction and fusion to obtain the fused multi-scale features; Step 5: The fused multi-scale features are sequentially processed through an improved Transformer encoder, average pooling, a fully connected layer, and a classifier to obtain the probability distribution output by the classifier. Based on the probability distribution output by the classifier, cross-entropy loss function and relative entropy loss function are constructed respectively. The classifier is trained using the cross-entropy loss function and relative entropy loss function to obtain the trained classifier. Based on the feature map generated by ConvNeXt, a feature distillation loss function is constructed. ConvNeXt is trained using the feature distillation loss function to obtain the trained ConvNeXt. The trained classifier and the trained ConvNeXt constitute the trained student model. The sentiment image is input into the trained student model to obtain the prediction result.

2. The image sentiment prediction method based on dual attention and diverse knowledge distillation according to claim 1, characterized in that, In step 2, an image sentiment dataset is acquired, the sentiment images in the dataset are preprocessed, and then input into ConvNeXt for feature extraction to obtain the downsampled feature map, the feature map generated by the second layer of ConvNeXt, and the upsampled feature map, respectively. Specifically, this includes the following sub-steps: Obtain the image sentiment dataset, preprocess the sentiment images in the image sentiment dataset, and obtain the preprocessed sentiment images; The preprocessed emotion image is input into ConvNeXt for feature extraction to obtain feature maps generated by the first layer of ConvNeXt, the second layer of ConvNeXt, and the third layer of ConvNeXt, respectively. The feature map generated by the first layer ConvNeXt is downsampled to obtain the downsampled feature map. An upsampling operation is performed on the feature map generated by the third layer ConvNeXt to obtain the upsampled feature map.

3. The image sentiment prediction method based on dual attention and diverse knowledge distillation according to claim 2, characterized in that, In the step of downsampling the feature map generated by the first layer ConvNeXt to obtain the downsampled feature map, the following relationship exists: ； in, This represents the feature map after downsampling. This indicates that a downsampling operation has been performed. This represents the feature map generated by the first layer of ConvNeXt; In the step of upsampling the feature map generated by the third layer ConvNeXt to obtain the upsampled feature map, the following relationship exists: ； in, This represents the feature map after upsampling. This indicates that an upsampling operation has been performed. This represents the feature map generated by the third layer ConvNeXt.

4. The image sentiment prediction method based on dual attention and diverse knowledge distillation according to claim 3, characterized in that, In step 3, the downsampled feature map, the feature map generated by the second layer ConvNeXt, and the upsampled feature map are respectively input into the channel attention module for processing to obtain the first weighted feature map, the second weighted feature map, and the third weighted feature map, which specifically includes the following sub-steps: S301. Perform average pooling on the downsampled feature map to obtain the first average pooled feature; S302. Perform max pooling on the downsampled feature map to obtain the first max pooled feature; S303. The first average pooling feature and the first max pooling feature are respectively input into the multilayer perceptron for processing, then added together, and processed using the Sigmoid function to obtain the first channel attention value. S304. Multiply the attention value of the first channel with the downsampled feature map to obtain the first weighted feature map; Repeat steps S301 to S304 for the feature map generated by the second layer ConvNeXt and the upsampled feature map respectively to obtain the second weighted feature map and the third weighted feature map respectively.

5. The image sentiment prediction method based on dual attention and diverse knowledge distillation according to claim 4, characterized in that, In the step of performing average pooling on the downsampled feature map to obtain the first average pooled feature, the following relationship exists: ； in, This represents the first average pooling characteristic. This indicates that the average pooling process has been performed. In the step of performing max pooling on the downsampled feature map to obtain the first max pooled feature, the following relationship exists: ； in, This represents the first max pooling feature. This indicates that it has undergone max pooling. In the step of inputting the first average pooling feature and the first max pooling feature into the multilayer perceptron for processing, then adding them together and processing them using the Sigmoid function to obtain the attention value of the first channel, the following relationship exists: ； in, This represents the attention value of the first channel. This indicates that the signal has been processed by the Sigmoid function. This indicates that the data has been processed by a multilayer perceptron. In the step of multiplying the first channel attention value with the downsampled feature map to obtain the first weighted feature map, the following relationship exists: ； in, This represents the first weighted feature map.

6. The image sentiment prediction method based on dual attention and diverse knowledge distillation according to claim 5, characterized in that, In step 4, the first weighted feature map, the second weighted feature map, and the third weighted feature map are respectively input into the feature fusion module for feature extraction and fusion to obtain fused multi-scale features. Specifically, this includes the following sub-steps: The first weighted feature map, the second weighted feature map, and the third weighted feature map are reshaped to obtain the first feature map, the second feature map, and the third feature map, respectively. The first feature map and the transpose of the second feature map are multiplied by a matrix and then processed using the Softmax function to obtain the first spatial attention map. The second feature map is multiplied by the first spatial attention map to obtain the key feature map; The key feature map is multiplied by the transpose of the second feature map, and then processed using the Softmax function to obtain the second spatial attention map. The third feature map is multiplied by the second spatial attention map to obtain the final feature map; The final feature map is reshaped and then residually connected to the upsampled feature map to obtain the fused multi-scale features. ; in, This represents the multi-scale features after fusion.

7. The image sentiment prediction method based on dual attention and diverse knowledge distillation according to claim 6, characterized in that, In the step of multiplying the transposes of the first feature map and the second feature map by matrix multiplication and processing them using the Softmax function to obtain the first spatial attention map, the following relationship exists: ； in, This represents the first spatial attention map. Represents the first feature map. Represents matrix multiplication. Represents the second feature map; In the step of performing matrix multiplication between the second feature map and the first spatial attention map to obtain the key feature map, the following relationship exists: ； in, Represents the key feature map; In the step of multiplying the key feature map with the transpose of the second feature map and then processing it using the Softmax function to obtain the second spatial attention map, the following relationship exists: ； in, This represents the second spatial attention map. Represents the third feature map; In the step of performing matrix multiplication between the third feature map and the second spatial attention map to obtain the final feature map, the following relationship exists: ； in, This represents the final feature map.

8. The image sentiment prediction method based on dual attention and diverse knowledge distillation according to claim 7, characterized in that, In step 5, the fused multi-scale features are sequentially processed through an improved Transformer encoder, average pooling, a fully connected layer, and a classifier to obtain the probability distribution output by the classifier. This process includes the following sub-steps: The fused multi-scale features are sequentially processed through an improved Transformer encoder, average pooling, and a fully connected layer to obtain the output features of the fully connected layer. The following relationship exists in the corresponding process: ； in, Indicates the first Output characteristics of fully connected layers Represents the weight vector. This indicates that the code has been processed using an improved Transformer encoder. Indicates the bias term; The output features of the fully connected layer are input into the classifier for processing to obtain the probability distribution of the classifier output. The following relationship exists in the corresponding process: ； in, Indicates the first The probability distribution output by the layer classifier. Indicates the first Layer classifier, This represents the temperature parameter.

9. The image sentiment prediction method based on dual attention and diverse knowledge distillation according to claim 8, characterized in that, In step 5, a cross-entropy loss function and a relative entropy loss function are constructed based on the probability distribution of the classifier output. The classifier is then trained using these two functions to obtain the trained classifier. A feature distillation loss function is constructed based on the feature map generated by ConvNeXt. ConvNeXt is then trained using this feature distillation loss function to obtain the trained ConvNeXt. The expression for the cross-entropy loss function is as follows: ； in, Represents cross-entropy loss, Indicates the weighting coefficient. This represents the total number of classifiers. Represents the cross-entropy loss function. Indicates the true label; The expression for the relative entropy loss function is as follows: ； in, Represents relative entropy loss. Represents relative entropy. Represents the last layer classifier The probability distribution of the output; The expression for the characteristic distillation loss function is as follows: ； in, Indicates characteristic distillation loss, Indicates hyperparameters, Indicates the first Feature maps generated by layer ConvNeXt, This represents the features extracted by the last layer of the improved Transformer encoder. This indicates taking the 2-norm.

Images