Multi-modal social media sentiment analysis method based on adaptive sentiment features

Abstract

The invention relates to the field of image processing, and provides a multi-modal social media sentiment analysis method based on self-adaptive sentiment features, which performs sentiment tendency prediction on multi-modal social media features through a sentiment knowledge distillation algorithm to obtain sentiment polarity scores so as to fully pay attention to explicit sentiment polarity in image-text features, and improve the sentiment tendency of the multi-modal social media features. Therefore, comprehensive capture of emotion information is achieved, a clear emotion polarity signal is directly extracted from a text and utilized, direct emotion context guidance is provided for subsequent feature processing, the problem that emotion information is not comprehensive or meticulous is avoided, and a dynamic emotion polarity modulation factor is designed, so that the emotion information can be comprehensively captured. Therefore, the importance of the image and text features can be dynamically adjusted according to the extracted explicit emotion polarity so as to better adapt to the difference of emotion expression modes and intensities among different image-text samples, and the accuracy and comprehensiveness of the multi-modal social media emotion analysis method are improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a multimodal social media sentiment analysis method based on adaptive sentiment features. Background Art

[0002] With the rapid development of the Internet industry, the comprehensiveness and diversity of information in social media have gradually become the mainstream trend, in order to meet the needs of users to express their personal emotions and opinions. In particular, multimodal content containing images and text has increased dramatically. Therefore, when conducting user analysis in social media, multimodal information, as an important carrier of user emotional dynamics in the digital environment, has become a crucial part of the analysis process.

[0003] In the existing technology, when conducting social media sentiment analysis, multimodal information is often isolated and the feature information of text and image is analyzed separately to obtain analysis results. This leads to the neglect of the correlation between image and text combinations, lacks the ability to adaptively recognize the detailed emotional information of image and text combinations, and makes it difficult to distinguish the significant differences in emotional intensity between different image and text combinations. At the same time, existing methods often only focus on the implicit emotional information in image and text features, and do not fully utilize the explicit emotional polarity (such as positive, negative, and neutral), resulting in incomplete capture of emotional information, which further causes inaccurate, incomplete and incomplete sentiment analysis.

[0004] Therefore, how to design a social media sentiment analysis method to improve the accuracy and comprehensiveness of sentiment analysis has become an urgent problem to be solved. Summary of the Invention

[0005] Based on this, the present invention proposes a multimodal social media sentiment analysis method based on adaptive sentiment features, which predicts the sentiment tendency of multimodal social media features through a sentiment knowledge distillation algorithm to obtain a sentiment polarity score, so as to fully focus on the explicit sentiment polarity in the image and text features, thereby achieving comprehensive capture of sentiment information, and directly extracting and utilizing clear sentiment polarity signals from the text, providing direct sentiment context guidance for subsequent feature processing, avoiding the problem of incomplete and incomplete sentiment information, and then designing a dynamic sentiment polarity modulation factor, so that the importance of image and text features can be dynamically adjusted according to the extracted explicit sentiment polarity, so as to better adapt to the differences in sentiment expression methods and intensity between different image and text samples. The present invention improves the accuracy and comprehensiveness of the multimodal social media sentiment analysis method.

[0006] The present invention proposes a multimodal social media sentiment analysis method based on adaptive sentiment features, comprising: Acquiring multimodal social media information and performing preprocessing to extract multimodal social media features, wherein the multimodal social media features include multimodal text features and multimodal image features, and the multimodal social media features are extracted based on a multimodal information model; Performing sentiment tendency prediction on the multimodal social media features based on a sentiment knowledge distillation algorithm to obtain a sentiment polarity score, wherein the sentiment tendency prediction is obtained based on a sentiment knowledge source, which is a pre-trained text sentiment analysis model, and the explicit sentiment polarity score is an explicit numerical value of the predicted sentiment category; Adaptively enhancing the multimodal social media features according to the sentiment polarity scores to obtain multimodal adaptively enhanced sentiment features, wherein the adaptive enhancement is based on a dynamic sentiment polarity modulation factor, wherein the dynamic sentiment polarity modulation factor is used to dynamically adjust feature importance according to an explicit sentiment signal; Performing multimodal feature fusion processing on the multimodal adaptive enhanced emotional features based on an attention mechanism to obtain a multimodal fusion emotional feature; Emotion classification is performed based on the multimodal fusion emotion features to obtain the final emotion analysis result.

[0007] In summary, according to the above-mentioned multimodal social media sentiment analysis method based on adaptive emotional features, the emotional tendency of multimodal social media features is predicted by the emotional knowledge distillation algorithm to obtain the emotional polarity score, so as to fully focus on the explicit emotional polarity in the image and text features, thereby achieving comprehensive capture of emotional information, and directly extracting and utilizing clear emotional polarity signals from the text, providing direct emotional context guidance for subsequent feature processing, avoiding the problem of incomplete and incomplete emotional information, and then designing a dynamic emotional polarity modulation factor, so that the importance of image and text features can be dynamically adjusted according to the extracted explicit emotional polarity, so as to better adapt to the differences in emotional expression methods and intensity between different image and text samples. The present invention improves the accuracy and comprehensiveness of the multimodal social media sentiment analysis method. Specifically, multimodal social media information is obtained and preprocessed to extract multimodal social media features, which include multimodal text features and multimodal image features. The multimodal social media features are extracted based on a multimodal information model. For each image-text pair of multimodal social media information, a personalized feature representation based on a specific emotional context can be generated to capture more detailed and deeper emotional patterns. The emotional tendency of the multimodal social media features is predicted based on the emotional knowledge distillation algorithm to obtain an emotional polarity score. The emotional tendency prediction is based on an emotional knowledge source, which is a pre-trained text emotional analysis model. The explicit emotional polarity score is an explicit numerical value for predicting emotional categories, so as to fully focus on the explicit emotional polarity in the image-text features, thereby achieving comprehensive capture of emotional information, directly extracting and utilizing clear emotional polarity signals from the text, and providing information for subsequent features. The processing provides direct emotional context guidance, avoiding the problem of incomplete and indetailed emotional information, and adaptively enhances the multimodal social media features according to the emotional polarity score to obtain multimodal adaptively enhanced emotional features. The adaptive enhancement processing is based on a dynamic emotional polarity modulation factor, and the dynamic emotional polarity modulation factor is used to dynamically adjust the feature importance according to the explicit emotional signal, so that the importance of image and text features can be dynamically adjusted according to the extracted explicit emotional polarity to better adapt to the differences in emotional expression and intensity between different image and text samples. Based on the attention mechanism, the multimodal adaptively enhanced emotional features are subjected to multimodal feature fusion processing to obtain multimodal fused emotional features, and emotional classification is performed according to the multimodal fused emotional features to obtain the final emotional analysis results. The present invention improves the accuracy and comprehensiveness of the multimodal social media sentiment analysis method.

[0008] Furthermore, the step of obtaining multimodal social media information and preprocessing it to extract multimodal social media features specifically includes: Multimodal social media information is obtained and preprocessed, and features are extracted from the multimodal social media information based on a multimodal information model. The specific algorithm for feature extraction is as follows: , , in, and Represent multimodal image features and multimodal text features respectively, represents the L2 normalization operation, and represent the image encoder and text encoder of the multimodal information model respectively, and They represent image information and text information in multimodal social media information respectively.

[0009] Furthermore, the step of predicting the sentiment tendency of the multimodal social media features based on the sentiment knowledge distillation algorithm to obtain the sentiment polarity score specifically includes: Acquiring a sentiment knowledge source based on a pre-trained text sentiment analysis model, wherein the text sentiment analysis model is a Twitter-RoBERTa model; The multimodal text features in the multimodal social media features are input into the sentiment knowledge source, and the sentiment knowledge source performs sentiment tendency prediction to obtain an original sentiment score. The specific algorithm for obtaining the original sentiment score is as follows: , in, represents the original sentiment score, represents the source of emotional knowledge, Represent multimodal text features; Emotion category prediction is performed based on the original emotion score to obtain an emotion polarity score.

[0010] Furthermore, the step of performing sentiment category prediction based on the original sentiment score to obtain a sentiment polarity score specifically includes: Calculate the original emotion score to obtain the predicted emotion category. The specific algorithm for obtaining the predicted emotion category is as follows: , in, Represents the predicted sentiment category, represents the maximum function, represents the raw sentiment score; The predicted emotion category is explicitly converted into a numerical value according to the mapping matrix to obtain an emotion polarity score. The specific algorithm for obtaining the emotion polarity score is as follows: , in, represents the sentiment polarity score, represents the mapping matrix, represents the Softmax function, Represents the predicted sentiment category.

[0011] Furthermore, the step of adaptively enhancing the multimodal social media features according to the sentiment polarity scores to obtain multimodal adaptively enhanced sentiment features specifically includes: The multimodal social media features are processed according to the sentiment polarity scores to obtain sentiment polarity fusion features. The specific algorithm for obtaining the sentiment polarity fusion features is as follows: , in, Indicates the emotional polarity fusion feature, represents a nonlinear activation function, and represents the learnable parameters, and Represent multimodal image features and multimodal text features respectively, represents the sentiment polarity score, Represents vector concatenation; A linear transformation is performed according to the emotion polarity fusion feature to obtain a dynamic emotion polarity modulation factor. The specific algorithm for obtaining the dynamic emotion polarity modulation factor is as follows: , in, represents the dynamic emotion polarity modulation factor, represents the Sigmoid activation function, and represents the learnable parameters, Indicates the emotional polarity fusion feature; The multimodal social media features are element-wise multiplied according to the dynamic emotion polarity modulation factor to obtain a multimodal adaptive enhanced emotion feature. The specific algorithm for obtaining the multimodal adaptive enhanced emotion feature is as follows: , , in, and They represent the image adaptive enhanced emotional features and text adaptive enhanced emotional features in the multimodal adaptive enhanced emotional features, and Represent multimodal image features and multimodal text features respectively, represents the dynamic emotion polarity modulation factor, Represents element-wise multiplication.

[0012] Furthermore, the step of performing multimodal feature fusion processing on the multimodal adaptive enhanced emotion feature based on the attention mechanism to obtain a multimodal fusion emotion feature specifically includes: The multimodal adaptive enhanced emotion features are subjected to feature fusion processing to obtain multimodal fusion features. The specific algorithm for obtaining the multimodal fusion features is as follows: , in, represents multimodal fusion features, and They represent the image adaptive enhanced emotional features and text adaptive enhanced emotional features in the multimodal adaptive enhanced emotional features, Represents vector concatenation; The multimodal fusion feature is enhanced according to the multi-head attention mechanism to obtain the multimodal fusion emotion feature. The specific algorithm for obtaining the multimodal fusion emotion feature is as follows: , in, Represents multimodal fusion sentiment features, represents the multi-head attention mechanism, Represents multimodal fusion features.

[0013] Furthermore, the step of performing sentiment classification based on the multimodal fusion sentiment features to obtain a final sentiment analysis result specifically includes: Inputting the multimodal fusion emotion features into an emotion classifier, wherein the emotion classifier is a multilayer perceptron including a Gelu nonlinear activation function; The emotion classifier calculates the emotion category probability distribution, and the specific algorithm of the emotion category probability distribution is as follows: , in, represents the probability distribution of sentiment categories, represents the Softmax function, represents the Gelu nonlinear activation function, represents a multilayer perceptron, Represents multimodal fusion emotional features; The final sentiment analysis result is obtained according to the sentiment category probability distribution.

[0014] The present invention proposes a multimodal social media sentiment analysis system based on adaptive sentiment features, comprising: A feature extraction module, configured to obtain and preprocess multimodal social media information to extract multimodal social media features, wherein the multimodal social media features include multimodal text features and multimodal image features, and the multimodal social media features are extracted based on a multimodal information model; A sentiment polarity score calculation module is used to predict the sentiment tendency of the multimodal social media features based on the sentiment knowledge distillation algorithm to obtain a sentiment polarity score. The sentiment tendency prediction is obtained based on a sentiment knowledge source, which is a pre-trained text sentiment analysis model. The explicit sentiment polarity score is an explicit numerical value of the predicted sentiment category. an adaptive enhancement module, configured to adaptively enhance the multimodal social media features according to the sentiment polarity scores to obtain multimodal adaptively enhanced sentiment features, wherein the adaptive enhancement is based on a dynamic sentiment polarity modulation factor, wherein the dynamic sentiment polarity modulation factor is used to dynamically adjust feature importance according to an explicit sentiment signal; A fusion module, configured to perform multimodal feature fusion processing on the multimodal adaptive enhanced emotion feature based on an attention mechanism to obtain a multimodal fused emotion feature; The classification module is used to perform sentiment classification based on the multimodal fusion sentiment features to obtain the final sentiment analysis result.

[0015] The present invention also provides a storage medium storing one or more programs, which, when executed by a processor, implement the multimodal social media sentiment analysis method based on adaptive sentiment features as described above.

[0016] The present invention further provides a computer device, comprising a memory and a processor, wherein: The memory is used to store computer programs; When the processor is used to execute the computer program stored in the memory, it implements the multimodal social media sentiment analysis method based on adaptive sentiment features as described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a flowchart of the multimodal social media sentiment analysis method based on adaptive sentiment features proposed in the first embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a multimodal social media sentiment analysis system based on adaptive sentiment features proposed in the second embodiment of the present invention.

[0018] The following specific embodiments will further illustrate the present invention in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0019] To facilitate understanding of the present invention, the present invention will be described more fully below with reference to the accompanying drawings. The drawings illustrate several embodiments of the present invention. However, the present invention may be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and comprehensive understanding of the present invention.

[0020] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be an intermediate element. When an element is referred to as being "connected to" another element, it may be directly connected to the other element or there may be an intermediate element. The terms "vertical," "horizontal," "left," "right," and similar expressions used herein are for illustrative purposes only.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one skilled in the art to which this invention pertains. The terms used in this specification of the present invention are for the purpose of describing specific embodiments only and are not intended to limit the present invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0022] See also Figure 1 , shown is a flowchart of a multimodal social media sentiment analysis method based on adaptive sentiment features proposed in a first embodiment of the present invention. The multimodal social media sentiment analysis method based on adaptive sentiment features includes steps S01 to S05, wherein: Step S01: Acquire multimodal social media information and perform preprocessing to extract multimodal social media features; It should be noted that in this embodiment, the multimodal social media features include multimodal text features and multimodal image features. The multimodal social media features are extracted based on a multimodal information model. Multimodal social media information is obtained and preprocessed, and feature extraction is performed on the multimodal social media information based on the multimodal information model. The specific algorithm for feature extraction is as follows: , , in, and Represent multimodal image features and multimodal text features respectively, represents the L2 normalization operation, and represent the image encoder and text encoder of the multimodal information model respectively, and They represent image information and text information in multimodal social media information respectively.

[0023] Step S02: Predicting the sentiment tendency of multimodal social media features based on the sentiment knowledge distillation algorithm to obtain a sentiment polarity score; It should be noted that in this embodiment, the sentiment tendency prediction is based on the acquisition of a sentiment knowledge source, which is a pre-trained text sentiment analysis model. The explicit sentiment polarity score is an explicit numerical value of the predicted sentiment category. The sentiment knowledge source is obtained according to the pre-trained text sentiment analysis model, which is the Twitter-RoBERTa model. The multimodal text features in the multimodal social media features are input into the sentiment knowledge source, and the sentiment knowledge source performs sentiment tendency prediction to obtain an original sentiment score. The specific algorithm for obtaining the original sentiment score is as follows: , in, represents the original sentiment score, represents the source of emotional knowledge, Represent multimodal text features; Performing sentiment category prediction based on the original sentiment score to obtain a sentiment polarity score; The step of performing sentiment category prediction based on the original sentiment score to obtain a sentiment polarity score specifically includes: Calculate the original emotion score to obtain the predicted emotion category. The specific algorithm for obtaining the predicted emotion category is as follows: , in, Represents the predicted sentiment category, represents the maximum function, represents the raw sentiment score; The predicted emotion category is explicitly converted into a numerical value according to the mapping matrix to obtain an emotion polarity score. The specific algorithm for obtaining the emotion polarity score is as follows: , in, represents the sentiment polarity score, represents the mapping matrix, represents the Softmax function, Represents the predicted sentiment category.

[0024] Step S03: Adaptively enhance the multimodal social media features according to the sentiment polarity scores to obtain multimodal adaptively enhanced sentiment features; It should be noted that in this embodiment, the adaptive enhancement processing is based on a dynamic emotion polarity modulation factor, which is used to dynamically adjust the feature importance according to the explicit emotion signal, and perform feature processing on the multimodal social media features according to the emotion polarity score to obtain the emotion polarity fusion feature. The specific algorithm for obtaining the emotion polarity fusion feature is as follows: , in, Indicates the emotional polarity fusion feature, represents a nonlinear activation function, and represents the learnable parameters, and Represent multimodal image features and multimodal text features respectively, represents the sentiment polarity score, Represents vector concatenation; A linear transformation is performed according to the emotion polarity fusion feature to obtain a dynamic emotion polarity modulation factor. The specific algorithm for obtaining the dynamic emotion polarity modulation factor is as follows: , in, represents the dynamic emotion polarity modulation factor, represents the Sigmoid activation function, and represents the learnable parameters, Indicates the emotional polarity fusion feature; The multimodal social media features are element-wise multiplied according to the dynamic emotion polarity modulation factor to obtain a multimodal adaptive enhanced emotion feature. The specific algorithm for obtaining the multimodal adaptive enhanced emotion feature is as follows: , , in, and They represent the image adaptive enhanced emotional features and text adaptive enhanced emotional features in the multimodal adaptive enhanced emotional features, and Represent multimodal image features and multimodal text features respectively, represents the dynamic emotion polarity modulation factor, Represents element-wise multiplication.

[0025] Step S04: performing multimodal feature fusion processing on the multimodal adaptive enhanced emotion feature based on the attention mechanism to obtain a multimodal fusion emotion feature; It should be noted that in this embodiment, feature fusion processing is performed on the multimodal adaptive enhanced emotion feature to obtain a multimodal fusion feature. The specific algorithm for obtaining the multimodal fusion feature is as follows: , in, represents multimodal fusion features, and They represent the image adaptive enhanced emotional features and text adaptive enhanced emotional features in the multimodal adaptive enhanced emotional features, Represents vector concatenation; The multimodal fusion feature is enhanced according to the multi-head attention mechanism to obtain the multimodal fusion emotion feature. The specific algorithm for obtaining the multimodal fusion emotion feature is as follows: , in, Represents multimodal fusion sentiment features, represents the multi-head attention mechanism, Represents multimodal fusion features.

[0026] Step S05: performing sentiment classification based on the multimodal fusion sentiment features to obtain the final sentiment analysis result; It should be noted that in this embodiment, the multimodal fusion emotion features are input into the emotion classifier, and the emotion classifier is a multi-layer perceptron, and the multi-layer perceptron includes a Gelu nonlinear activation function; The emotion classifier calculates the emotion category probability distribution, and the specific algorithm of the emotion category probability distribution is as follows: , in, represents the probability distribution of sentiment categories, represents the Softmax function, represents the Gelu nonlinear activation function, represents a multilayer perceptron, Represents multimodal fusion emotional features; The final sentiment analysis result is obtained according to the sentiment category probability distribution.

[0027] In summary, according to the above-mentioned multimodal social media sentiment analysis method based on adaptive emotional features, the emotional tendency of multimodal social media features is predicted by the emotional knowledge distillation algorithm to obtain the emotional polarity score, so as to fully focus on the explicit emotional polarity in the image and text features, thereby achieving comprehensive capture of emotional information, and directly extracting and utilizing clear emotional polarity signals from the text, providing direct emotional context guidance for subsequent feature processing, avoiding the problem of incomplete and incomplete emotional information, and then designing a dynamic emotional polarity modulation factor, so that the importance of image and text features can be dynamically adjusted according to the extracted explicit emotional polarity, so as to better adapt to the differences in emotional expression methods and intensity between different image and text samples. The present invention improves the accuracy and comprehensiveness of the multimodal social media sentiment analysis method. Specifically, multimodal social media information is obtained and preprocessed to extract multimodal social media features, which include multimodal text features and multimodal image features. The multimodal social media features are extracted based on a multimodal information model. For each image-text pair of multimodal social media information, a personalized feature representation based on a specific emotional context can be generated to capture more detailed and deeper emotional patterns. The emotional tendency of the multimodal social media features is predicted based on the emotional knowledge distillation algorithm to obtain an emotional polarity score. The emotional tendency prediction is based on an emotional knowledge source, which is a pre-trained text emotional analysis model. The explicit emotional polarity score is an explicit numerical value for predicting emotional categories, so as to fully focus on the explicit emotional polarity in the image-text features, thereby achieving comprehensive capture of emotional information, directly extracting and utilizing clear emotional polarity signals from the text, and providing information for subsequent features. The processing provides direct emotional context guidance, avoiding the problem of incomplete and indetailed emotional information, and adaptively enhances the multimodal social media features according to the emotional polarity score to obtain multimodal adaptively enhanced emotional features. The adaptive enhancement processing is based on a dynamic emotional polarity modulation factor, and the dynamic emotional polarity modulation factor is used to dynamically adjust the feature importance according to the explicit emotional signal, so that the importance of image and text features can be dynamically adjusted according to the extracted explicit emotional polarity to better adapt to the differences in emotional expression and intensity between different image and text samples. Based on the attention mechanism, the multimodal adaptively enhanced emotional features are subjected to multimodal feature fusion processing to obtain multimodal fused emotional features, and emotional classification is performed according to the multimodal fused emotional features to obtain the final emotional analysis results. The present invention improves the accuracy and comprehensiveness of the multimodal social media sentiment analysis method.

[0028] See also Figure 2 , which is a schematic diagram of the structure of a multimodal social media sentiment analysis system based on adaptive sentiment features proposed in the second embodiment of the present invention, the system includes: A feature extraction module 10 is configured to obtain and preprocess multimodal social media information to extract multimodal social media features, wherein the multimodal social media features include multimodal text features and multimodal image features, and the multimodal social media features are extracted based on a multimodal information model; A sentiment polarity score calculation module 20 is configured to perform sentiment tendency prediction on the multimodal social media features based on a sentiment knowledge distillation algorithm to obtain a sentiment polarity score, wherein the sentiment tendency prediction is obtained based on a sentiment knowledge source, which is a pre-trained text sentiment analysis model. The explicit sentiment polarity score is an explicit numerical value of the predicted sentiment category. an adaptive enhancement module 30 for adaptively enhancing the multimodal social media features according to the sentiment polarity scores to obtain multimodal adaptively enhanced sentiment features, wherein the adaptive enhancement is based on a dynamic sentiment polarity modulation factor, wherein the dynamic sentiment polarity modulation factor is used to dynamically adjust feature importance according to an explicit sentiment signal; A fusion module 40 is configured to perform multimodal feature fusion processing on the multimodal adaptive enhanced emotion feature based on an attention mechanism to obtain a multimodal fused emotion feature; The classification module 50 is used to perform sentiment classification based on the multimodal fusion sentiment features to obtain a final sentiment analysis result.

[0029] The present invention also proposes a computer storage medium having one or more programs stored thereon, which, when executed by a processor, implements the above-mentioned multimodal social media sentiment analysis method based on adaptive sentiment features.

[0030] The present invention also proposes a computer device comprising a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to implement the above-mentioned multimodal social media sentiment analysis method based on adaptive sentiment features.

[0031] Those skilled in the art will appreciate that the logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device), or in conjunction with such instruction execution system, apparatus, or device. For purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transmit a program for use by an instruction execution system, apparatus, or device, or in conjunction with such instruction execution system, apparatus, or device.

[0032] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting, or processing it in another suitable manner as necessary, and then storing it in a computer memory.

[0033] It should be understood that various components of the present invention may be implemented using hardware, software, firmware, or a combination thereof. In the aforementioned embodiments, multiple steps or methods may be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one or a combination of the following technologies known in the art may be used: a discrete logic circuit having logic gate circuits for implementing logic functions on data signals, an application-specific integrated circuit having suitable combinational logic gate circuits, a programmable gate array (PGA), a field-programmable gate array (FPGA), etc.

[0034] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0035] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A multimodal social media sentiment analysis method based on adaptive sentiment features, characterized in that: include: Acquiring multimodal social media information and performing preprocessing to extract multimodal social media features, wherein the multimodal social media features include multimodal text features and multimodal image features, and the multimodal social media features are extracted based on a multimodal information model; Performing sentiment tendency prediction on the multimodal social media features based on a sentiment knowledge distillation algorithm to obtain a sentiment polarity score, wherein the sentiment tendency prediction is obtained based on a sentiment knowledge source, which is a pre-trained text sentiment analysis model, and the explicit sentiment polarity score is an explicit numerical value of the predicted sentiment category; Adaptively enhancing the multimodal social media features according to the sentiment polarity scores to obtain multimodal adaptively enhanced sentiment features, wherein the adaptive enhancement is based on a dynamic sentiment polarity modulation factor, wherein the dynamic sentiment polarity modulation factor is used to dynamically adjust feature importance according to an explicit sentiment signal; Performing multimodal feature fusion processing on the multimodal adaptive enhanced emotional features based on an attention mechanism to obtain a multimodal fusion emotional feature; Emotion classification is performed based on the multimodal fusion emotion features to obtain the final emotion analysis result.

2. The multimodal social media sentiment analysis method based on adaptive sentiment features according to claim 1 is characterized in that: The step of obtaining multimodal social media information and performing preprocessing to extract multimodal social media features specifically includes: Multimodal social media information is obtained and preprocessed, and features are extracted from the multimodal social media information based on a multimodal information model. The specific algorithm for feature extraction is as follows: ， ， in, and Represent multimodal image features and multimodal text features respectively, represents the L2 normalization operation, and represent the image encoder and text encoder of the multimodal information model respectively, and They represent image information and text information in multimodal social media information respectively.

3. The multimodal social media sentiment analysis method based on adaptive sentiment features according to claim 1 is characterized in that: The step of predicting the sentiment tendency of the multimodal social media features based on the sentiment knowledge distillation algorithm to obtain the sentiment polarity score specifically includes: Acquiring a sentiment knowledge source based on a pre-trained text sentiment analysis model, wherein the text sentiment analysis model is a Twitter-RoBERTa model; The multimodal text features in the multimodal social media features are input into the sentiment knowledge source, and the sentiment knowledge source performs sentiment tendency prediction to obtain an original sentiment score. The specific algorithm for obtaining the original sentiment score is as follows: ， in, represents the original sentiment score, represents the source of emotional knowledge, Represent multimodal text features; Emotion category prediction is performed based on the original emotion score to obtain an emotion polarity score.

4. The multimodal social media sentiment analysis method based on adaptive sentiment features according to claim 3 is characterized in that: The step of performing sentiment category prediction based on the original sentiment score to obtain a sentiment polarity score specifically includes: Calculate the original emotion score to obtain the predicted emotion category. The specific algorithm for obtaining the predicted emotion category is as follows: ， in, Represents the predicted sentiment category, represents the maximum function, represents the raw sentiment score; The predicted emotion category is explicitly converted into a numerical value according to the mapping matrix to obtain an emotion polarity score. The specific algorithm for obtaining the emotion polarity score is as follows: ， in, represents the sentiment polarity score, represents the mapping matrix, represents the Softmax function, Represents the predicted sentiment category.

5. The multimodal social media sentiment analysis method based on adaptive sentiment features according to claim 1 is characterized in that: The step of adaptively enhancing the multimodal social media features according to the sentiment polarity scores to obtain multimodal adaptively enhanced sentiment features specifically includes: The multimodal social media features are processed according to the sentiment polarity scores to obtain sentiment polarity fusion features. The specific algorithm for obtaining the sentiment polarity fusion features is as follows: ， in, Indicates the emotional polarity fusion feature, represents a nonlinear activation function, and represents the learnable parameters, and Represent multimodal image features and multimodal text features respectively, represents the sentiment polarity score, Represents vector concatenation; A linear transformation is performed according to the emotion polarity fusion feature to obtain a dynamic emotion polarity modulation factor. The specific algorithm for obtaining the dynamic emotion polarity modulation factor is as follows: ， in, represents the dynamic emotion polarity modulation factor, represents the Sigmoid activation function, and represents the learnable parameters, Indicates the emotional polarity fusion feature; The multimodal social media features are element-wise multiplied according to the dynamic emotion polarity modulation factor to obtain a multimodal adaptive enhanced emotion feature. The specific algorithm for obtaining the multimodal adaptive enhanced emotion feature is as follows: ， ， in, and They represent the image adaptive enhanced emotional features and text adaptive enhanced emotional features in the multimodal adaptive enhanced emotional features, and Represent multimodal image features and multimodal text features respectively, represents the dynamic emotion polarity modulation factor, Represents element-wise multiplication.

6. The multimodal social media sentiment analysis method based on adaptive sentiment features according to claim 1 is characterized in that: The step of performing multimodal feature fusion processing on the multimodal adaptive enhanced emotion feature based on the attention mechanism to obtain the multimodal fusion emotion feature specifically includes: The multimodal adaptive enhanced emotion features are subjected to feature fusion processing to obtain multimodal fusion features. The specific algorithm for obtaining the multimodal fusion features is as follows: ， in, represents multimodal fusion features, and They represent the image adaptive enhanced emotional features and text adaptive enhanced emotional features in the multimodal adaptive enhanced emotional features, Represents vector concatenation; The multimodal fusion feature is enhanced according to the multi-head attention mechanism to obtain the multimodal fusion emotion feature. The specific algorithm for obtaining the multimodal fusion emotion feature is as follows: ， in, Represents multimodal fusion sentiment features, represents the multi-head attention mechanism, Represents multimodal fusion features.

7. The multimodal social media sentiment analysis method based on adaptive sentiment features according to claim 1 is characterized in that: The step of performing sentiment classification according to the multimodal fusion sentiment features to obtain a final sentiment analysis result specifically includes: Inputting the multimodal fusion emotion features into an emotion classifier, wherein the emotion classifier is a multilayer perceptron including a Gelu nonlinear activation function; The emotion classifier calculates the emotion category probability distribution, and the specific algorithm of the emotion category probability distribution is as follows: ， in, represents the probability distribution of sentiment categories, represents the Softmax function, represents the Gelu nonlinear activation function, represents a multilayer perceptron, Represents multimodal fusion emotional features; The final sentiment analysis result is obtained according to the sentiment category probability distribution.

8. A multimodal social media sentiment analysis system based on adaptive sentiment features, characterized in that: include: A feature extraction module, configured to obtain and preprocess multimodal social media information to extract multimodal social media features, wherein the multimodal social media features include multimodal text features and multimodal image features, and the multimodal social media features are extracted based on a multimodal information model; A sentiment polarity score calculation module is used to predict the sentiment tendency of the multimodal social media features based on the sentiment knowledge distillation algorithm to obtain a sentiment polarity score. The sentiment tendency prediction is obtained based on a sentiment knowledge source, which is a pre-trained text sentiment analysis model. The explicit sentiment polarity score is an explicit numerical value of the predicted sentiment category. an adaptive enhancement module, configured to adaptively enhance the multimodal social media features according to the sentiment polarity scores to obtain multimodal adaptively enhanced sentiment features, wherein the adaptive enhancement is based on a dynamic sentiment polarity modulation factor, wherein the dynamic sentiment polarity modulation factor is used to dynamically adjust feature importance according to an explicit sentiment signal; A fusion module, configured to perform multimodal feature fusion processing on the multimodal adaptive enhanced emotion feature based on an attention mechanism to obtain a multimodal fused emotion feature; The classification module is used to perform sentiment classification based on the multimodal fusion sentiment features to obtain the final sentiment analysis result.

9. A storage medium, characterized in that: The storage medium stores one or more programs, which, when executed by a processor, implement the multimodal social media sentiment analysis method based on adaptive sentiment features as described in any one of claims 1 to 7.

10. A computer device, characterized in that: The computer device comprises a memory and a processor, wherein: The memory is used to store computer programs; When the processor is used to execute the computer program stored in the memory, it implements the multimodal social media sentiment analysis method based on adaptive sentiment features as described in any one of claims 1 to 7.

Images