Subjective visual emotion interpretation generation method and system

Abstract

The invention relates to the technical field of artificial intelligence, and discloses a subjective visual emotion interpretation generation method and system. The construction and training process of the emotion interpretation model comprises the following steps of: counting the normalization frequency of each emotion label to obtain a consensus ratio, and mapping the consensus ratio into a plurality of consensus levels; extracting global semantic features and local patch features of the image; encoding the consensus level into a bucket prompt, splicing the bucket prompt with a global semantic feature, inputting the bucket prompt into a visual emotion encoder to generate a visual emotion feature, and fusing the visual emotion feature with a local patch feature to form double-space visual representation; the barrel prompt and the prompt word text are connected to form a consensus prompt, the consensus prompt and the double-space visual representation are jointly input into a large language model, and an emotion explanation text is output; according to the method, modeling is carried out on emotion consensus degrees of different observers, so that the system can capture diversified subjective emotion reactions caused by visual contents, and the perception ability of the model on emotion subjectivity and diversity is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method and system for generating subjective visual emotion interpretation. Background Art

[0002] The rapid development of artificial intelligence in recent years has increased the demand for emotional interaction. For example, companion robots that can recognize and express emotions can provide more harmonious companionship, especially for the elderly and children. To endow machines with human-like emotions, understanding the human emotional experience of different visual stimuli is crucial. Visual emotion interpretation generation focuses on understanding the emotions perceived by viewers from images, particularly the reasons behind these emotions. This task can infer and explain human emotional states, laying the foundation for applications such as mental health monitoring, personalized emotional intervention, and empathetic virtual agents.

[0003] Recently, thanks to the development of large language models (LLMs), this field has made initial progress. By combining an image encoder based on the CLIP model with large language models such as GPT, sentiment explanation generation can be transformed into a text generation task. After further fine-tuning on sentiment datasets, LLMs are able to predict the emotion category from images and generate explanatory sentences. However, existing methods still face challenges in providing fair and diverse image-induced emotion understanding. Psychological research shows that human emotion perception is influenced by multiple factors, including neurophysiology, cultural background, personal past experiences, and social and environmental context. These factors contribute to the inherent subjectivity challenges in visual emotion interpretation tasks. Existing methods directly optimize for fitting the true labels, biasing the model towards common dominant emotions while neglecting less common but equally important emotional perspectives. This leads to two major problems: 1) the explanations provided by humans are clearly emotionally unique, making existing models poorly able to learn less frequent emotions; and 2) many images lack a consensus on the dominant emotion within the group, making it difficult for the model to determine the dominant emotion. Summary of the Invention

[0004] To solve the above technical problems, the present invention provides a method and system for generating subjective visual emotion interpretation based on consensus prompts and a large language model, which solves this problem by introducing an additional control signal (i.e., the emotion consensus level).

[0005] The proposed method calculates the normalized frequency of sentiment labels for each "image-sentiment label-explanatory text" data set as the consensus rate, which is then discretized into predefined consensus levels and used as a control signal during training. This allows the model to learn to generate diverse sentiment interpretations at different consensus levels, balancing mainstream and minority viewpoints.

[0006] To bridge the emotional gap between visual content and abstract emotions, this paper introduces an emotional space that organizes features by emotion category, promoting consistency among features of the same emotion while enhancing the distinction between features of different emotions. A dual-space visual representation method is proposed that integrates the emotional and semantic features of an image, providing effective emotional cues and visual evidence for generating emotional interpretations.

[0007] In order to solve the above technical problems, the present invention adopts the following technical solutions: The present invention provides a method for generating subjective visual emotion interpretation, wherein the construction and training process of the emotion interpretation model adopted includes: Construct an image dataset, where each image sample in the image dataset includes an image, a corresponding emotion label, and an explanation text; Based on multiple emotion labels of the same image, the normalized frequency of each emotion label is counted to obtain the consensus ratio, and the consensus ratio is mapped to multiple predefined consensus levels through a discretization function. ; Extracting images The consensus level is encoded as a bucket prompt, which is concatenated with the global semantic features and then input into the visual emotion encoder to generate visual emotion features, which are then fused with the local patch features to form a dual-space visual representation. The bucket prompt and the prompt word text are connected to form a consensus prompt. The consensus prompt and the dual-space visual representation are input into the decoder of the large language model. The decoder of the large language model autoregressively outputs the sentiment explanation text containing the sentiment category and the explanation text. The sentiment interpretation model is optimized by jointly combining the language generation loss based on autoregressive prediction of the target interpretation text, the sentiment feature alignment loss based on cross-modal sentiment space feature matching, and the sentiment classification loss based on real sentiment label classification supervision.

[0008] In one embodiment, the method is to calculate the normalized frequency of each emotion tag based on the same image to obtain a consensus ratio, and then map the consensus ratio to a plurality of predefined consensus levels through a discretization function. , specifically including: For a given image And the corresponding Emotional interpretation of text by different audiences , computing emotions Consensus Ratio : ; in, represents the emotional label given by the nth viewer to the image, express Corresponding explanatory text; is an indicator function, when When 1, Time is 0; Using bucketing technology Convert to Discrete consensus levels : ; in, It is a Bucketization function with equally spaced boundaries.

[0009] In one embodiment, the extracted image The global semantic features and local patch features of , specifically including: The image Input to image encoder , extracting global-level and patch-level semantics: ; is the global semantic feature of the image, represents the Mth local patch feature of the image, and M represents the total number of local patches in the image.

[0010] In one embodiment, encoding the consensus level into a bucket prompt, concatenating it with the global semantic features, and then inputting it into a visual emotion encoder to generate a visual emotion feature, which is then fused with the local patch features to form a dual-space visual representation, specifically includes: Using a learnable bucket embedding layer Consensus Level Encode and get the bucket prompt ; The bucket prompt and global semantic features Connect them and input into the visual emotion encoder implemented based on the linear layer To obtain visual emotion features : ; Indicates column-level splicing; Visual emotional features Combined with local patch features, a dual-space visual representation is obtained : ; Indicates row connection; represents the Mth local patch feature of the image, and M represents the total number of local patches in the image.

[0011] In one embodiment, the step of connecting the bucket prompt with the prompt word text to form a consensus prompt specifically includes: The bucket prompt Concatenate it with the word embedding of the prompt word text to get the consensus prompt T: ; in Indicates row connection. represents the word embedding layer, is the Lth word segmentation of the prompt word text, and L is the total number of word segments in the prompt word text.

[0012] In one embodiment, the step of inputting the consensus prompt and the dual-space visual representation into a decoder of a large language model, and outputting an emotional interpretation text including an emotional category and an interpretation text through autoregression of the decoder of the large language model, specifically includes: Feed the consensus cue T and the dual-space visual representation V into the decoder of the large language model To predict sentiment categories and explanations: ; in, and are the hidden states of sentiment category and explanation text respectively; and The input is fed into the linear layer and the softmax activation function to derive the probability distribution over the vocabulary, and after passing through the argmax function, the emotional interpretation text of the image is predicted; the emotional interpretation text includes the emotional category and the interpretation text.

[0013] In one embodiment, the language generation loss based on the target interpretation text autoregressive prediction specifically includes: Language Generation Loss for: ; is used to predict the sentiment interpretation text The probability distribution of target words, Represents the predicted sentiment interpretation text target words, Represents the first t-1 target words of the predicted sentiment explanation text.

[0014] In one embodiment, the emotion feature alignment loss based on cross-modal emotion space feature matching specifically includes: Sentiment feature alignment loss for: ; is the visual emotion feature, Indicates the emotional characteristics of the interpreted text; ; The hidden state of the interpretation text output by the decoder autoregression of the large language model, Represents the semantic features of the interpreted text, represents average pooling, Tip for the bucket, Indicates column-level splicing, Represents a text sentiment encoder implemented based on a linear layer.

[0015] In one embodiment, the sentiment classification loss based on real sentiment label classification supervision specifically includes: Sentiment classification loss : ; in, and Represented by the learnable visual emotion classifier and text sentiment classifier Predicted sentiment distribution, is the visual emotion feature, Indicates the emotional characteristics of the interpretation text, is the sentiment label of the image sample.

[0016] In a second aspect, the present invention provides a computer system comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method of any one embodiment of the first aspect when executing the computer program.

[0017] This paper introduces a consensus guidance mechanism and combines visual emotion recognition with a language generation model to achieve subjective modeling and accurate interpretation of visually induced emotions, with the following technical effects: 1. Enhanced subjective emotion understanding: By modeling the degree of emotional consensus among different observers, the system can capture the diverse subjective emotional responses elicited by visual content, improving the model's ability to perceive the subjectivity and diversity of emotions.

[0018] 2. Achieve controllable emotional interpretation generation: Use the consensus level as a control signal to guide the system to generate interpretation text consistent with the specific emotional cognitive level, thereby achieving fine-grained regulation and diversified output of emotional expression.

[0019] 3. Improving emotional consistency and semantic credibility: By designing emotional consistency constraints at the feature and label levels, the generated text is made more consistent with the real emotions inspired by the input image in terms of content, enhancing the semantic accuracy and emotional credibility of the interpreted text.

[0020] 4. Good application scalability: The method of the present invention can be widely used in fields such as emotional health assessment, educational companionship, virtual human interaction, and digital art understanding, providing effective technical support for building more humane and emotionally interactive intelligent systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 4 is a flow chart of a method in an embodiment of the present invention.

[0022] Figure 2 It is a schematic diagram of the framework of the present invention. DETAILED DESCRIPTION

[0023] A preferred embodiment of the present invention will be described in detail below with reference to the accompanying drawings.

[0024] like Figure 1 As shown, the present invention provides a method for generating subjective visual emotion interpretation, and the construction and training process of the emotion interpretation model adopted includes the following steps: S1, construct an image dataset, each image sample in the image dataset includes an image and the corresponding emotional label and explanation text; S2, based on multiple emotion labels of the same image, count the normalized frequencies of each emotion label to obtain the consensus ratio, and map the consensus ratio to multiple predefined consensus levels through a discretization function. ; S3, extract image The consensus level is encoded as a bucket prompt, which is concatenated with the global semantic features and then input into the visual emotion encoder to generate visual emotion features, which are then fused with the local patch features to form a dual-space visual representation. S4, connects the bucket prompt and the prompt word text to form a consensus prompt, inputs the consensus prompt and the dual-space visual representation into the decoder of the large language model, and outputs the emotional explanation text containing the emotional category and the explanation text through the autoregressive process of the decoder of the large language model; S5, jointly optimizes the sentiment interpretation model by combining the language generation loss based on the autoregressive prediction of the target interpretation text, the sentiment feature alignment loss based on cross-modal sentiment space feature matching, and the sentiment classification loss based on the real sentiment label classification supervision.

[0025] Through the above technical solutions, the present invention realizes the integrated processing of recognition and interpretation of visually induced emotions, breaking through the problem that traditional visual emotion analysis focuses on classification while ignoring interpretation, and provides a theoretical basis and technical support for building an emotional intelligence system with greater human-computer interaction value.

[0026] The purpose of this invention is to provide a method for identifying and interpreting visually evoked emotions, addressing the inability of existing methods to effectively model the subjectivity of emotions and the lack of high-quality emotional interpretation. By introducing a consensus-based control mechanism, a dual-space visual representation, and an emotion consistency learning strategy, we achieve accurate recognition and interpretation of visual emotions.

[0027] The present invention introduces a subjective modeling mechanism based on consensus level: the emotional response of the group is discretized and graded according to the degree of consensus, and used as a control signal to guide the generation of emotional interpretation, thereby improving the model's ability to model subjectivity and diversity, and supporting the controllable generation of emotional interpretation.

[0028] The present invention enhances semantic emotion perception capabilities by constructing a dual-space visual representation: integrating visual semantic features with subjective emotional features, and constructing a unified dual-space visual representation structure, so that the model can simultaneously perceive the objective attributes of image content and the subjective emotions induced, thereby improving the accuracy of emotional interpretation and the consistency of human cognition.

[0029] The present invention designs an emotional consistency learning mechanism to ensure the reliability of emotional expression: consistency constraints are introduced at the feature level and label level to ensure that the generated text interpretation is consistent with the image content in terms of semantic expression and emotional attribution, effectively improving the emotional fidelity and semantic relevance of the emotional interpretation.

[0030] The present invention is designed to have strong generation capabilities and scalability: combining a large-scale pre-trained language model with a controllable generation strategy, the method of the present invention not only achieves good results in sentiment classification accuracy, but also has the ability to generate natural, fluent, and emotionally appropriate interpretations, and has broad application potential.

[0031] In one embodiment, constructing the image dataset in step S1 specifically includes: Obtain an input image database. This image database uses two art image datasets, ArtEmisv1.0 and ArtEmisv2.0, and the Affection real-world image dataset, which contains image works, their corresponding emotion labels, and explanatory text. The Affection dataset is constructed using real-world images from the MS-COCO dataset, the Flickr30k Entities dataset, the Visual Genome dataset, and the FI dataset.

[0032] For the ArtEmisv1.0 and ArtEmisv2.0 datasets, the standard preprocessing process provided by the official repository was followed. To comprehensively evaluate our model, we defined five test set splits based on the image's emotional dominance and diversity. Emotional dominance includes test images where a single emotion dominates the majority of their annotators, while other test images are classified as non-dominant. Regarding emotional diversity, the test images in the dataset were categorized as containing a single emotion, consistent emotions, or conflicting emotions.

[0033] In one embodiment, the extracted image in step S3 The global semantic features and local patch features of , specifically including: The present invention adopts the CLIP model image encoder, which aims to align image and text modalities, and freezes the parameters. In the CLIP space, semantically related features are close to each other. Figure 2 As shown, given an image , the present invention extracts image-level and patch-level semantics as follows: ; in, is the global semantic feature of the image, represents the visual feature dimension, represents the Mth local patch feature of the image, and M represents the total number of local patches in the image.

[0034] This paper achieves fair and diverse visual emotion reasoning by explicitly modeling the subjectivity of emotional experiences across different groups of people. It also proposes a consensus-based emotion modeling approach to guide the model to adapt to different emotional perspectives.

[0035] In one embodiment, the normalized frequencies of the emotion tags in step S2 are counted to obtain a consensus ratio, and the consensus ratio is mapped to a plurality of predefined consensus levels through a discretization function, specifically including: The consensus level is approximated by the normalized frequency of each sentiment label in the image, which is called the consensus ratio. For a given image in the image dataset, and from Emotional interpretation of text by different audiences , computing emotions Consensus Ratio , the formula is as follows: ; in, is an indicator function, when is equal to 1 when , otherwise it is 0.

[0036] In order to obtain the consensus ratio of all training samples , the present invention constructs buckets to capture different samples, Convert to Discrete consensus levels , the specific method is to use bucketing technology, the formula is as follows: .

[0037] in, It is a This bucketing allows training triplets with low overlap to be redistributed into buckets with lower consensus (e.g. ), while training triplets with high overlap are grouped into buckets with higher consensus (e.g. ).

[0038] In one embodiment, step S3 encodes the consensus level into a bucket prompt, concatenates it with the global semantic feature, and then inputs it into the visual emotion encoder to generate a visual emotion feature, which is then fused with the local patch feature to form a dual-space visual representation. Specifically, the following steps are performed: To generate sentiment explanations, models require a comprehensive visual understanding that captures both semantic clarity and sentiment accuracy. While traditional visual encoders are effective at extracting scene semantics, they struggle to model subtle sentiment relationships. To bridge this gap, we propose a joint approach using a semantic encoder to extract objective semantic features and a visual sentiment encoder to capture sentiment nuances.

[0039] The original visual features given by the image encoder of the CLIP model do not contain emotional meaning. Therefore, this paper introduces a visual emotion encoder to construct an emotion representation space, clustering similar emotions together and separating dissimilar emotions. Based on the fact that emotions are derived from the overall visual environment, this paper uses global semantic features Initialize the emotion encoding. However, in emotion interpretation, the same image can trigger multiple emotions depending on personal experience, which means that visual features alone cannot fully capture the different perceptions of emotions by different people. To address this, the present invention explicitly conditions the emotion representation on the consistency level of a given image-emotion label-explanation text triple (image sample). The present invention uses the bucket prompt and global semantic features Concatenate them and feed them into the visual emotion encoder to obtain visual emotion features: ; in Indicates column-level splicing. Represents the visual emotion encoder, implemented by a linear layer.

[0040] Visual emotional features Combined with features from local patches, we get a dual-space visual representation : .

[0041] The present invention believes that Captures the emotional tone of the image, while local patch features Provides the fine-grained visual evidence needed to generate plausible explanations.

[0042] In one embodiment, S4 connects the bucket prompt and the prompt word text to form a consensus prompt, inputs the consensus prompt and the dual-space visual representation into the decoder of the large language model, and outputs the sentiment category and explanation text through the autoregressive decoder of the large language model, specifically including: In order to enable the model to dynamically adjust sentiment reasoning according to different consensus levels, this paper introduces consensus-based bucket hints as control signals. Using a learnable bucket embedding layer Consensus Level Encode and prompt the bucket Connect with the prompt word text to get the consensus prompt : ; in, The dimension representing the word embedding can be set to 768 in a preferred embodiment; Indicates row connection. represents the word embedding layer, is the Lth word segment of the prompt text. With the help of the prompt text, the GPT decoder is guided to generate explanations that match different consensus levels. This controllability allows the model of the present invention to act as an implicit mixture of experts (but sharing parameters), where one expert (e.g., with The model) focuses on a set of high-consensus image-sentiment label-explanation text triplets (image samples), while parameter sharing among experts allows the learned knowledge to be shared among all experts to cover rich sentiment perspectives.

[0043] Consensus Tip and dual-space visual representation Input into the GPT-based decoder, specifically the GPT2 decoder , to predict sentiment categories and interpret text: ; in, and They are the hidden states of the sentiment category and the explanation text, respectively. By using a linear layer and then inputting it into the softmax activation function, the probability distribution over the vocabulary can be derived from the hidden state .

[0044] In one embodiment, step S5 optimizes the sentiment interpretation model by jointly optimizing the language generation loss based on the autoregressive prediction of the target interpretation text, the sentiment feature alignment loss based on cross-modal sentiment space feature matching, and the sentiment classification loss based on the real sentiment label classification supervision, specifically including: The text generated by the sentiment interpretation model should indicate the expected sentiment category and provide a grounded interpretation of the image that is consistent with the sentiment. To achieve this goal, the present invention first applies a language generation loss To encourage fluent and meaningful sentence generation. Different from the typical loss function, this paper proposes a sentiment interpretation model based on sentiment consensus level, which generates a new objective as follows: ; in, is the predicted sentiment explanation text target word. However, There is no explicit guarantee that the interpretation is consistent with the target emotion, so the present invention proposes sentiment consistency constraints involving feature-level and label-level regularization.

[0045] Since emotions are inherently subjective and not explicitly encoded in raw visual features, the feature-level emotion consistency constraint is implemented by minimizing the distance between two spatial feature representations: ; and All are located in the same sentiment space, allowing feature-level alignment. This paper achieves this sentiment consistency through mean square error loss, obtaining the sentiment feature alignment loss : .

[0046] Unlike direct feature matching, alignment in the sentiment space ensures that the generated explanation preserves the emotional essence of the image rather than just reflecting semantic similarity, while tying the explanation to the visual input also prevents it from drifting towards generic emotional expressions.

[0047] In order to ensure that the emotional features can well represent different emotional categories, the present invention designs a label-level emotional consistency constraint. Unrestricted emotional space may become chaotic, and semantically similar emotions may overlap, resulting in ambiguous emotional representation. To solve this problem, the present invention introduces a and Sentiment classification loss , to enhance the uniqueness of the learned emotional space. Specifically, Encourage and The predicted sentiment categories match the true labels: ; in, and Represented by the learnable visual emotion classifier and text sentiment classifier Predicted sentiment distribution. Each sentiment classifier is a combination of ReLU layers, linear layers, and softmax functions. is the sentiment label in the input triplet (image sample). By minimizing , the model learns to maintain clear sentiment boundaries and generate explanations that are consistent with their sentiment labels.

[0048] In order to train the model in an end-to-end manner, the ultimate goal of this invention is to combine all losses to obtain the total loss function : ; in, and is a hyperparameter used to balance the loss.

[0049] The present invention uses the AdamW optimizer for end-to-end training. If the BLEU-4 index does not improve within 10 consecutive rounds on the validation set, the model is judged to have converged and training is stopped.

[0050] In the inference stage, given the input image and consensus level, the sentiment interpretation model autoregressively generates sentiment interpretation text, which includes sentiment category and interpretation text. The output format is "sentiment category because interpretation text".

[0051] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this application, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times. The order of execution of these steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the steps or stages in other steps.

[0052] In one embodiment, the present invention provides a computer system. The computer system includes a processor, memory, and a network interface connected via a system bus. The processor of the computer system is used to provide computing and control capabilities. The memory of the computer system includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer system is used to store data used in the above-mentioned method. The network interface of the computer system is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, the above-mentioned method is implemented.

[0053] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0054] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. It is intended that all variations within the meaning and range of equivalents of the claims be embraced herein, and any reference signs in the claims should not be construed as limiting the claims to which they relate.

[0055] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. A method for generating subjective visual emotion interpretation, characterized in that: The construction and training process of the adopted sentiment interpretation model includes: Construct an image dataset, where each image sample in the image dataset includes an image, a corresponding emotion label, and an explanation text; Based on multiple emotion labels of the same image, the normalized frequency of each emotion label is counted to obtain the consensus ratio, and the consensus ratio is mapped to multiple predefined consensus levels through a discretization function. ; Extracting images The consensus level is encoded as a bucket prompt, which is concatenated with the global semantic features and then input into the visual emotion encoder to generate visual emotion features, which are then fused with the local patch features to form a dual-space visual representation. The bucket prompt and the prompt word text are connected to form a consensus prompt. The consensus prompt and the dual-space visual representation are input into the decoder of the large language model. The decoder of the large language model autoregressively outputs the sentiment explanation text containing the sentiment category and the explanation text. The sentiment interpretation model is optimized by jointly combining the language generation loss based on autoregressive prediction of the target interpretation text, the sentiment feature alignment loss based on cross-modal sentiment space feature matching, and the sentiment classification loss based on real sentiment label classification supervision.

2. A method for generating subjective visual emotion interpretation according to claim 1, characterized in that: The method is based on multiple emotion labels of the same image, and the normalized frequency of each emotion label is counted to obtain a consensus ratio, and the consensus ratio is mapped to a plurality of predefined consensus levels through a discretization function. , specifically including: For a given image And the corresponding Emotional interpretation of text by different audiences , computing emotions Consensus Ratio : ； in, represents the emotional label given by the nth viewer to the image, express Corresponding explanatory text; is an indicator function, when When 1, Time is 0; Using bucketing technology Convert to Discrete consensus levels : ； in, It is a Bucketization function with equally spaced boundaries.

3. A subjective visual emotion interpretation generation method according to claim 1, characterized in that: The extracted image The global semantic features and local patch features of , specifically including: The image Input to image encoder , extracting global-level and patch-level semantics: ； is the global semantic feature of the image, represents the Mth local patch feature of the image, and M represents the total number of local patches in the image.

4. A method for generating subjective visual emotion interpretation according to claim 1, characterized in that: The consensus level is encoded as a bucket prompt, which is then concatenated with the global semantic features and input into the visual emotion encoder to generate visual emotion features, which are then fused with the local patch features to form a dual-space visual representation. Specifically, the following steps are performed: Using a learnable bucket embedding layer Consensus Level Encode and get the bucket prompt ; The bucket prompt and global semantic features Connect them and input into the visual emotion encoder implemented based on the linear layer To obtain visual emotion features : ； Indicates column-level splicing; Visual emotional features Combined with local patch features, a dual-space visual representation is obtained : ； Indicates row connection; represents the Mth local patch feature of the image, and M represents the total number of local patches in the image.

5. A method for generating subjective visual emotion interpretation according to claim 1, characterized in that: The step of connecting the bucket prompt and the prompt word text to form a consensus prompt specifically includes: The bucket prompt Concatenate it with the word embedding of the prompt word text to get the consensus prompt T: ； in Indicates row connection. represents the word embedding layer, is the Lth word segmentation of the prompt word text, and L is the total number of word segments in the prompt word text.

6. A method for generating subjective visual emotion interpretation according to claim 1, characterized in that: The consensus prompt and the dual-space visual representation are inputted into the decoder of the large language model, and the decoder of the large language model autoregressively outputs the emotional interpretation text containing the emotional category and the interpretation text, specifically including: Feed the consensus cue T and the dual-space visual representation V into the decoder of the large language model To predict sentiment categories and explanations: ； in, and are the hidden states of sentiment category and explanation text respectively; and The input is fed into the linear layer and the softmax activation function to derive the probability distribution over the vocabulary, and after passing through the argmax function, the emotional explanation text of the image is predicted; the emotional explanation text includes the emotional category and the explanation text.

7. A method for generating subjective visual emotion interpretation according to claim 1, characterized in that: The language generation loss based on the target interpretation text autoregressive prediction specifically includes: Language Generation Loss for: ； is used to predict the sentiment interpretation text The probability distribution of target words, Represents the predicted sentiment interpretation text target words, Represents the first t-1 target words of the predicted sentiment explanation text.

8. A method for generating subjective visual emotion interpretation according to claim 1, characterized in that: The emotional feature alignment loss based on cross-modal emotional space feature matching is specifically include: Sentiment feature alignment loss for: ； is the visual emotion feature, Indicates the emotional characteristics of the interpreted text; ； The hidden state of the interpretation text output by the decoder autoregression of the large language model, Represents the semantic features of the interpreted text, represents average pooling, Tip for the bucket, Indicates column-level splicing, Represents a text sentiment encoder implemented based on a linear layer.

9. A method for generating subjective visual emotion interpretation according to claim 1, characterized in that: The sentiment classification loss based on real sentiment label classification supervision specifically includes: Sentiment classification loss : ； in, and Represented by the learnable visual emotion classifier and text sentiment classifier Predicted sentiment distribution, is the visual emotion feature, Indicates the emotional characteristics of the interpretation text, is the sentiment label of the image sample.

10. A computer system comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.

Images