A method, device, equipment and storage medium for processing emoticon package images

Abstract

The invention discloses an emoticon package image processing method, device, equipment and storage medium, belonging to the field of artificial intelligence technology. The method comprises: obtaining an emoticon package image to be processed, and extracting auxiliary text from the context text of the emoticon package image to be processed; based on a trained emotion classification model, according to the auxiliary text, performing emotion type recognition on the emoticon package image to be processed, and determining the target emotion type to which the emoticon package image to be processed belongs; the emotion classification model is trained according to the contrast loss of sample image-text pairs and the entropy of predicted probability distribution; text extraction is performed on the emoticon package image to be processed, and the target text content corresponding to the emoticon package image to be processed is obtained; based on an emoticon package reply prompt template, according to the target emotion type and the target text content, an emoticon package prompt text is generated; the emoticon package prompt text is input into a large language model, and a text reply content for replying to the emoticon package image to be processed is generated.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an emoticon package image processing method, device, equipment and storage medium. Background Art

[0002] With the continuous development of multimodal learning, some progress has been made in the field of sentiment analysis for image and text content. However, there are still obvious limitations in the processing of emoticon images.

[0003] As one of the main forms of communication on social media today, the meanings contained in emoticons often have multiple layers of emotions and intentions, such as sarcasm, puns, or self-mockery. However, these complex emotions are often difficult to effectively identify and interpret, resulting in a large deviation between the generated replies and the user's actual intentions. Summary of the invention

[0004] The present invention provides an emoticon package picture processing method, device, equipment and storage medium to improve the accuracy of sentiment analysis on emoticon package pictures and improve the accuracy of reply content in replying to emoticon package pictures.

[0005] According to one aspect of the present invention, a method for processing an emoticon package image is provided, the method comprising:

[0006] Obtain an emoticon package image to be processed, and extract auxiliary text from the context text of the emoticon package image to be processed;

[0007] Based on the trained sentiment classification model, according to the auxiliary text, the sentiment type of the emoticon package picture to be processed is identified to determine the target sentiment type of the emoticon package picture to be processed; wherein the sentiment classification model is trained based on the contrast loss of the sample image-text pair and the entropy of the predicted probability distribution; the sample image-text pair includes the sample emoticon package picture and the annotated sentiment type;

[0008] Perform text extraction on the emoticon package image to be processed to obtain the target text content corresponding to the emoticon package image to be processed;

[0009] Based on the emoji reply prompt template, generate emoji prompt text according to the target emotion type and target text content;

[0010] The emoji prompt text is input into the large language model to generate text reply content to reply to the emoji image to be processed.

[0011] According to another aspect of the present invention, there is provided an emoticon package image processing device, the device comprising:

[0012] An auxiliary text extraction module is used to obtain the emoticon package image to be processed and extract auxiliary text from the context text of the emoticon package image to be processed;

[0013] The target emotion type determination module is used to identify the emotion type of the emoticon package image to be processed based on the trained emotion classification model and the auxiliary text, and determine the target emotion type of the emoticon package image to be processed; wherein the emotion classification model is trained based on the contrast loss of the sample image-text pair and the entropy of the predicted probability distribution; the sample image-text pair includes the sample emoticon package image and the annotated emotion type;

[0014] A target text content determination module is used to extract text from the emoticon package image to be processed and obtain the target text content corresponding to the emoticon package image to be processed;

[0015] An emoticon prompt text generation module is used to generate an emoticon prompt text based on an emoticon reply prompt template, according to a target emotion type and a target text content;

[0016] The text reply content generation module is used to input the emoticon prompt text into the large language model to generate text reply content for replying to the emoticon package image to be processed.

[0017] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0018] at least one processor; and

[0019] a memory communicatively connected to at least one processor; wherein,

[0020] The memory stores a computer program that can be executed by at least one processor, and the computer program is executed by at least one processor so that the at least one processor can execute the emoticon package image processing method of any embodiment of the present invention.

[0021] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the emoticon package image processing method of any embodiment of the present invention when executed.

[0022] According to another aspect of the present invention, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the computer program implements the emoticon package image processing method of any embodiment of the present invention.

[0023] The technical solution of the embodiment of the present invention is to obtain an emoticon package image to be processed and extract auxiliary text from the context text of the emoticon package image to be processed; based on a trained emotion classification model, the emotion type of the emoticon package image to be processed is identified according to the auxiliary text, and the target emotion type to which the emoticon package image to be processed belongs is determined; wherein the emotion classification model is trained according to the contrast loss of sample image-text pairs and the entropy of the predicted probability distribution; the sample image-text pairs include sample emoticon package images and annotated emotion types; text is extracted from the emoticon package image to be processed to obtain the target text content corresponding to the emoticon package image to be processed; based on an emoticon package reply prompt template, an emoticon package prompt text is generated according to the target emotion type and the target text content; the emoticon package prompt text is input into a large language model to generate text reply content for replying to the emoticon package image to be processed. The above technical solution, with the help of a trained sentiment classification model, identifies the sentiment type of the emoticon package image to be processed according to the auxiliary text, so that the obtained target sentiment category is more accurate, and the accuracy of the sentiment analysis of the emoticon package image is improved, thereby making the subsequently generated emoticon package prompt text more accurate, and thus making the text reply content of the emoticon package image to be processed by the large language model more in line with the actual context, more natural and accurate, and easier to arouse the user's emotional resonance, thereby improving the accuracy of the reply content to the emoticon package image, thereby enhancing the interactivity and stickiness between users and the social media platform, and improving user satisfaction with the social media platform.

[0024] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 is a flow chart of an emoticon package image processing method provided according to the first embodiment of the present invention;

[0027] Figure 2 is a flow chart of an emoticon package image processing method provided according to Embodiment 2 of the present invention;

[0028] Figure 3 2 is a schematic diagram of the structure of an emoticon package image processing device provided according to Embodiment 3 of the present invention;

[0029] Figure 4 It is a structural schematic diagram of an electronic device for implementing the emoticon package picture processing method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "target", "first" and "second" in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] In addition, it should be noted that in the technical scheme of the present invention, the collection, storage, use, processing, transmission, provision and disclosure of the context text of the emoticon images to be processed, sample image-text pairs, and emoticon reply prompt templates, etc., all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0033] Embodiment 1

[0034] Figure 1 This is a flow chart of an emoticon image processing method provided in the first embodiment of the present invention. This embodiment can be applied to the case of processing emoticon images used in social media communication. The method can be executed by an emoticon image processing device, which can be implemented in the form of hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the method includes:

[0035] S101, obtaining an emoticon package image to be processed, and extracting auxiliary text from the context text of the emoticon package image to be processed.

[0036] The emoticon package images to be processed refer to emoticon package images that need to be processed. The auxiliary text refers to text used to assist in analyzing the emotions to be expressed by the emoticon package images to be processed.

[0037] Specifically, an emoticon package image to be processed is obtained from an interactive interface provided to the user; at the same time, the text sent by the user before or after uploading the emoticon package image to be processed is obtained from the interactive interface as the context text of the emoticon package image to be processed; and a preset text extraction algorithm is used to extract auxiliary text from the context text. It should be noted that the text extraction algorithm can be pre-selected according to actual business needs, and the embodiment of the present invention does not specifically limit it.

[0038] It can be understood that using the text extracted from the context text of the emoticon package image to be processed as auxiliary text can help improve the trained sentiment classification model's ability to understand the emoticon package image to be processed, thereby improving the accuracy of subsequent target sentiment types.

[0039] Optionally, if the context text of the emoticon package image to be processed cannot be obtained, the preset text is used as the auxiliary text. The preset text can be preset according to actual business requirements, and the embodiment of the present invention does not specifically limit it.

[0040] S102: Based on the trained emotion classification model and the auxiliary text, the emotion type of the emoticon package image to be processed is identified to determine the target emotion type of the emoticon package image to be processed.

[0041] The sentiment classification model is trained based on the contrast loss of sample image-text pairs and the entropy of the predicted probability distribution; the sample image-text pairs include sample emoticon images and labeled emotion types. The labeled emotion types include at least one of the following: wry smile, sarcasm, sadness, anger, fear, and surprise. The sample emoticon images refer to the emoticon images required for training the sentiment classification model. It should be noted that for each sample emoticon image, the emotion actually expressed by the sample emoticon image is one of wry smile, sarcasm, sadness, anger, fear, and surprise. The target emotion type refers to the type used to characterize the emotion actually expressed by the emoticon image to be processed.

[0042] Among them, the contrast loss of the sample image and text pairs is as follows:

[0043]

[0044] Among them, L Contrastive represents the contrast loss of the sample image-text pair, N represents the total number of sample image-text pairs, i represents the i-th sample image-text pair, y i represents the sample label corresponding to the i-th sample image-text pair, m represents the boundary parameter, d irepresents the Euclidean distance between the sample emoticon image and the labeled emotion type in the i-th sample image-text pair. It should be noted that if the i-th sample image-text pair is a positive sample pair, then y i =0; if the i-th sample image-text pair is a negative sample pair, then y i =1.

[0045] Specifically, the emoticon package image to be processed and the auxiliary text are input into the trained sentiment classification model, and the trained sentiment classification model identifies the sentiment type of the emoticon package image to be processed according to the auxiliary text to obtain the target sentiment type to which the emoticon package image to be processed belongs.

[0046] Optionally, the sentiment classification model is trained based on the contrast loss of the sample image-text pairs and the entropy of the predicted probability distribution. Specifically, the target loss function of the model training is determined by the following formula based on the contrast loss of the sample image-text pairs and the entropy of the predicted probability distribution:

[0047]

[0048] Among them, L represents the target loss function, β represents the regularization parameter, and p i represents the predicted probability distribution of the i-th sample image-text pair, H(p i ) represents the entropy of the predicted probability distribution of the i-th sample image-text pair;

[0049] At least one sample auxiliary text is generated according to the labeled emotion type and the auxiliary text template; the emotion classification model is trained according to the target loss function, the sample emoticon package image, and the at least one sample auxiliary text to obtain a trained emotion classification model.

[0050] Among them, H(p i )=-∑(y i ·logp i ), y i Represents the sample label corresponding to the i-th sample image-text pair. Sample auxiliary text refers to text used to assist in analyzing the emotion to be expressed by the sample emoticon package image. The auxiliary text template can be pre-set according to actual business needs. For example, the auxiliary text template can be "This emoticon package embodies an emotion of [CLS]", which is not specifically limited in the embodiment of the present invention. Among them, [CLS] is a placeholder. It should be noted that in order to enable the trained sentiment classification model to better process the emoticon package images, 6 different labeled emotion types are used when training the sentiment classification model, namely wry smile, sarcasm, sadness, anger, fear and surprise.

[0051] At least one sample auxiliary text is generated according to the marked emotion type and the auxiliary text template. Specifically, six different marked emotion types are used to replace the placeholders in the auxiliary text template, thereby obtaining six sample auxiliary texts.

[0052] Among them, the sentiment classification model is trained according to the target loss function, the sample emoticon package picture, and at least one sample auxiliary text to obtain the trained sentiment classification model, which can be specifically: the sample emoticon package picture and at least one sample auxiliary text are encoded respectively to obtain image encoding features and text encoding features; according to the feature similarity between the image encoding features and each text encoding feature, the sample emotion type to which the sample emoticon package picture belongs is determined; according to the target loss function and the sample emotion type, the training loss is determined; according to the training loss, the sentiment classification model is trained to obtain the trained sentiment classification model.

[0053] Among them, the image coding feature refers to the feature obtained after the sample emoticon package picture is encoded by the image. The text coding feature refers to the feature obtained after the sample auxiliary text is encoded by the text. The feature similarity refers to the cosine similarity between the image coding feature and the text coding feature. The sample emotion type refers to the type used to characterize the emotion actually expressed by the sample emoticon package picture.

[0054] More specifically, an image encoder is used to encode the sample emoticon package image to obtain the image encoding feature, which is recorded as I1; for each sample auxiliary text, a text encoder is used to encode the sample auxiliary text to obtain the text encoding feature corresponding to the sample auxiliary text, thereby obtaining the text encoding feature corresponding to the sample auxiliary text, which is recorded as {T1, T2, …, T N}; where N represents the total number of sample auxiliary texts; based on the cosine similarity algorithm, the similarity between the image encoding feature and each text encoding feature is calculated to obtain multiple feature similarities, which are recorded as {s1, s2, …, s n}; Extract the maximum feature similarity from the feature similarities between the image encoding features and each text encoding feature; Decode the text encoding feature corresponding to the maximum feature similarity to obtain the sample emotion type to which the sample emoticon package image belongs; Input the sample emotion type into the target loss function, and run the target loss function to obtain the training loss; Monitor the training loss until the training loss reaches the preset loss threshold, stop training the emotion classification model, and obtain a trained emotion classification model.

[0055] Among them, the text encoding feature corresponding to the maximum feature similarity is decoded to obtain the sample emotion to which the sample emoticon package image belongs. Specifically, it can be: the text encoding feature corresponding to the maximum feature similarity is decoded to obtain the sample auxiliary text corresponding to the text encoding feature; the emotion type at the placeholder in the sample auxiliary text is used as the sample emotion type to which the sample emoticon package image belongs.

[0056] It can be understood that the sentiment classification model is trained according to the contrast loss of the sample image-text pairs and the entropy of the predicted probability distribution, so that the trained sentiment classification model has better generalization ability, and enhances the trained sentiment classification model's ability to analyze the sentiment of emoticon pictures of different styles and text backgrounds, thereby improving the accuracy of the model classification results and making the analyzed target sentiment type more accurate.

[0057] S103, extracting text from the emoticon package image to be processed to obtain target text content corresponding to the emoticon package image to be processed.

[0058] The target text content refers to the actual text content on the emoticon package image to be processed.

[0059] Specifically, an OCR recognition model can be used to extract text from the emoticon package image to be processed to obtain the target text content corresponding to the emoticon package image to be processed. Among them, the OCR (Optical Character Recognition) recognition model can determine the shape of the characters in the image by detecting dark or light patterns, and then use the character recognition method to translate the character shape into text.

[0060] S104, based on the emoticon reply prompt template, generate an emoticon prompt text according to the target emotion type and the target text content.

[0061] The emoticon reply prompt template includes an emotion type placeholder and a text content placeholder; optionally, the emoticon reply prompt template can be pre-set according to actual business needs. For example, the emoticon reply prompt template can be "This emoticon expresses [Emotion], and the text content is [Text]", which is not specifically limited in the embodiment of the present invention. [Emotion] is the emotion type placeholder, and [Text] is the text content placeholder. The emoticon prompt text refers to the text used to prompt the large language model of the meaning contained in the emoticon image to be processed.

[0062] Specifically, the target emotion type is filled into the emotion type placeholder in the emoticon reply prompt template, and the target text content is filled into the text content placeholder in the emoticon reply prompt template, so as to obtain the emoticon prompt text.

[0063] S105: Input the emoticon prompt text into the large language model to generate text reply content for replying to the emoticon image to be processed.

[0064] Among them, the large language model can be pre-set according to actual business needs or the experience of technical personnel in this field. For example, the large language model can be ChatGPT (Chat Generative Pre-trained Transformer), which is not limited in the embodiments of the present invention.

[0065] Exemplarily, the emoticon prompt text is input into ChatGPT, and after being processed by ChatGPT, a text reply content for replying to the emoticon picture to be processed is obtained.

[0066] The technical solution of the embodiment of the present invention is to obtain an emoticon package image to be processed and extract auxiliary text from the context text of the emoticon package image to be processed; based on a trained emotion classification model, the emotion type of the emoticon package image to be processed is identified according to the auxiliary text, and the target emotion type to which the emoticon package image to be processed belongs is determined; wherein the emotion classification model is trained according to the contrast loss of sample image-text pairs and the entropy of the predicted probability distribution; the sample image-text pairs include sample emoticon package images and annotated emotion types; text is extracted from the emoticon package image to be processed to obtain the target text content corresponding to the emoticon package image to be processed; based on an emoticon package reply prompt template, an emoticon package prompt text is generated according to the target emotion type and the target text content; the emoticon package prompt text is input into a large language model to generate text reply content for replying to the emoticon package image to be processed. The above technical solution, with the help of a trained sentiment classification model, identifies the sentiment type of the emoticon package image to be processed according to the auxiliary text, so that the obtained target sentiment category is more accurate, and the accuracy of the sentiment analysis of the emoticon package image is improved, thereby making the subsequently generated emoticon package prompt text more accurate, and thus making the text reply content of the emoticon package image to be processed by the large language model more in line with the actual context, more natural and accurate, and easier to arouse the user's emotional resonance, thereby improving the accuracy of the reply content to the emoticon package image, thereby enhancing the interactivity and stickiness between users and the social media platform, and improving user satisfaction with the social media platform.

[0067] Embodiment 2

[0068] Figure 2 This is a flowchart of an emoticon image processing method provided in Example 2 of the present invention. Based on the above example, this example further optimizes "text extraction from the emoticon image to be processed to obtain the target text content corresponding to the emoticon image to be processed" and provides an optional implementation scheme. It should be noted that for the parts not described in detail in the examples of the present invention, reference can be made to the relevant descriptions of other examples. Figure 2 As shown, the method includes:

[0069] S201, obtaining an emoticon package image to be processed, and extracting auxiliary text from the context text of the emoticon package image to be processed.

[0070] S202: Based on the trained sentiment classification model and the auxiliary text, the sentiment type of the emoticon package image to be processed is identified to determine the target sentiment type of the emoticon package image to be processed.

[0071] Among them, the sentiment classification model is trained based on the contrast loss of sample image-text pairs and the entropy of the predicted probability distribution; the sample image-text pairs include sample emoticon images and annotated sentiment types.

[0072] It should be noted that before training the sentiment classification model, it is necessary to preprocess the emoticon training data consisting of sample emoticon pictures to obtain the preprocessed emoticon training data, so as to enrich the visual information in the emoticon training data and improve the data quality of the emoticon training data, so as to facilitate the subsequent training of the sentiment classification model, so that the trained sentiment classification model can more accurately analyze the implicit and complex emotions contained in the emoticon pictures.

[0073] Exemplarily, before training the sentiment classification model, the following non-local mean denoising algorithm may be used to denoise each sample emoticon package image in the emoticon package training data, thereby obtaining the first intermediate training data:

[0074]

[0075] Among them, NL[v](i) represents the pixel value of pixel i in the sample expression package image after denoising, I represents the total number of pixels in the sample expression package image; w(i,j) is a weight used to represent the similarity between pixel i and pixel j in the sample expression package image; v(j) represents the pixel value of pixel j in the sample expression package image; Represents a normalization factor, which is used to ensure that the sum of all weights is 1. It should be noted that the core idea of ​​the non-local means denoising algorithm is to use repeated textures and structures in the image to restore the pixel values ​​of the noisy area.

[0076] Afterwards, the image sizes of all sample emoticon package images in the first intermediate training data are adjusted to a pixel format of 224×224, thereby obtaining the second intermediate training data, so as to ensure the consistency of the sizes of the sample emoticon package images in the emoticon package training data.

[0077] Afterwards, the following bicubic interpolation method is used to perform edge smoothing on each sample emoticon package image in the second intermediate training data, thereby obtaining the third intermediate training data:

[0078]

[0079] Among them, f(x,y) represents the pixel value obtained after edge smoothing of the interpolation point (x,y) in the sample emoticon package image, and a ij is a coefficient, i represents the set of pixels around the horizontal coordinate of the interpolation point (x, y), and j represents the set of pixels around the vertical coordinate of the interpolation point (x, y).

[0080] Afterwards, the following Gaussian blur algorithm is used to perform image smoothing processing on each sample expression package image in the third intermediate training data, thereby obtaining the fourth intermediate training data:

[0081]

[0082] Among them, σ represents the standard deviation, and G(x,y) represents the Gaussian blur result of the pixel at the coordinate (x,y) in the sample emoticon package image.

[0083] Afterwards, a series of image enhancement processes are performed on the sample emoticon package images in the fourth intermediate training data, such as random cropping, random horizontal flipping, random translation and color jittering, so as to obtain the fifth intermediate training data; afterward, each sample emoticon package image in the fifth intermediate training data is normalized using the following normalization algorithm, so as to obtain the pre-processed emoticon package training data:

[0084]

[0085] Where I represents the original pixel value of the sample emoticon package image, μ represents the mean value of the pixel value of the pixel point in the sample emoticon package image, σ represents the standard deviation of the pixel value of the pixel point in the sample emoticon package image, and I norm Indicates the pixel value of the sample emoticon package image after normalization.

[0086] S203, performing image segmentation on the emoticon package image to be processed to obtain at least two candidate regions.

[0087] Specifically, the emoticon package image to be processed is segmented by a selective search algorithm to obtain at least two candidate regions. It should be noted that the region size, region shape, region texture and region text content of different candidate regions are all different.

[0088] S204: Determine the region similarity between any two candidate regions based on the region information of the candidate regions.

[0089] The region information includes, but is not limited to, region color information, region texture information, region area, and region text information, wherein the region text information is specifically manifested as text features.

[0090] Specifically, for any two candidate regions, one of them is called the first candidate region, and the other is called the second candidate region; the content similarity between the first candidate region and the second candidate region can be determined based on the first text feature of the first candidate region and the second text feature of the second candidate region; the size similarity between the first candidate region and the second candidate region can be determined based on the first region area of ​​the first candidate region and the second region area of ​​the second candidate region; the texture similarity between the first candidate region and the second candidate region can be determined based on the texture influence factor and the Euclidean distance between the first candidate region and the second candidate region; the color similarity between the first candidate region and the second candidate region can be determined based on the first color information of the first candidate region and the second color information of the second candidate region; the content similarity, size similarity, texture similarity and color similarity are weighted and summed to obtain the region similarity between the first candidate region and the second candidate region.

[0091] Optionally, the content similarity between the first candidate region and the second candidate region is determined according to the first text feature of the first candidate region and the second text feature of the second candidate region, which can be specifically expressed as follows:

[0092]

[0093] Among them, A represents the first candidate region, B represents the second candidate region, sim-context(A, B represents the content similarity between the first candidate region and the second candidate region, v A represents the first text feature, v B Represents the second text feature.

[0094] Optionally, the size similarity between the first candidate region and the second candidate region is determined according to the first region area of ​​the first candidate region and the second region area of ​​the second candidate region, which may be specifically expressed as follows:

[0095]

[0096] Among them, A represents the first candidate area, B represents the second candidate area, S A represents the area of ​​the first region, S B represents the area of ​​the second region, and sim_size(A,B) represents the size similarity between the first candidate region and the second candidate region.

[0097] Optionally, the texture similarity between the first candidate region and the second candidate region is determined according to the texture influence factor and the Euclidean distance between the first candidate region and the second candidate region, which can be specifically expressed as follows:

[0098] sim_texture(A,B)=e -δ·texture_distance(A,B) ;

[0099] Wherein, A represents the first candidate region, B represents the second candidate region, sim_texture(A,B represents the texture similarity between the first candidate region and the second candidate region, δ represents the texture influence factor, and texture_distance(A,B) represents the Euclidean distance between the first candidate region and the second candidate region.

[0100] Optionally, according to the first color information of the first candidate region and the second color information of the second candidate region, the color similarity between the first candidate region and the second candidate region is determined, which can be specifically expressed as follows:

[0101]

[0102] Where A represents the first candidate region, B represents the second candidate region, sim_color(A,B) represents the color similarity between the first candidate region and the second candidate region, and H A (i) Histogram value of the first candidate region in the i-th color region, H B (i) represents the histogram value of the second candidate region in the i-th color region.

[0103] Optionally, a weighted sum process is performed on the content similarity, size similarity, texture similarity and color similarity to obtain the region similarity between the first candidate region and the second candidate region, which can be specifically expressed as follows:

[0104] Sim(A,B)=α·sim_color(A,B)+β·sim_texture(A,B)+γ·sim_size(A,B)+δ·sim_context(A,B);

[0105] Among them, A represents the first candidate region, B represents the second candidate region, Sim(A,B) represents the regional similarity between the first candidate region and the second candidate region, α represents the weight coefficient of color similarity, β represents the weight coefficient of texture similarity, γ represents the weight coefficient of size similarity, and δ represents the weight coefficient of content similarity.

[0106] It is understandable that in the scenario of processing emoticon images, in order to better parse the text content in the emoticon images, when determining the regional similarity between pairwise candidate regions, the influence of regional color information, regional texture information, regional area and regional text information on determining the regional similarity between pairwise candidate regions is considered, thereby making the obtained regional similarity more accurate, facilitating a more precise merging of candidate regions based on the determined regional similarity, thereby avoiding the situation where a character in the emoticon image is incorrectly divided into multiple parts, thereby improving the accuracy of the subsequent target text content.

[0107] S205 . Perform region merging processing on at least two candidate regions according to the region similarity between any two candidate regions, to obtain at least one target text region.

[0108] The target text area refers to the area obtained after the candidate areas are merged.

[0109] Specifically, if there are M candidate regions, namely, region 1, region 2, region 3, ..., region M, then it is detected whether there is a region similarity between region 1 and other regions that reaches a preset threshold; if so, region 1 is merged with other regions whose region similarities reach the preset threshold to obtain a target text region. Exemplarily, if it is detected that the region similarity between region 1 and region 3 reaches the preset threshold, region 1 is merged with region 3 to obtain a target text region.

[0110] It can be understood that merging candidate areas with high regional similarity to obtain the target text area avoids the situation where a character in the emoticon package image to be processed is incorrectly divided into multiple parts, thereby improving the integrity of the text in the text recognition process of the emoticon package image to be processed.

[0111] S206: Extract features from the target text area to obtain key text features.

[0112] Among them, the key text features refer to the text features in the target text area.

[0113] Specifically, based on a preset text feature extraction algorithm, feature extraction is performed on each target text region to obtain key text features corresponding to each target text region.

[0114] S207, based on the long short-term memory network, character analysis is performed on the key text features corresponding to each target text area to obtain the target text content corresponding to the emoticon package image to be processed.

[0115] Specifically, the key text features corresponding to each target text area are input into the long short-term memory network, and the target text content corresponding to the emoticon package image to be processed is obtained after being processed by the long short-term memory network.

[0116] It is understandable that the gating mechanism in the long short-term memory network can filter out irrelevant features in the input key text features, thereby making the obtained target text content more accurate.

[0117] S208. Generate an emoticon package prompt text based on the emoticon package reply prompt template and according to the target emotion type and the target text content.

[0118] S209: Input the emoticon prompt text into the large language model to generate text reply content for replying to the emoticon image to be processed.

[0119] The technical solution of the embodiment of the present invention is as follows: by obtaining an emoticon package image to be processed and extracting auxiliary text from the context text of the emoticon package image to be processed; based on a trained emotion classification model, according to the auxiliary text, the emotion type of the emoticon package image to be processed is identified to determine the target emotion type to which the emoticon package image to be processed belongs; region extraction is performed on the emoticon package image to be processed to obtain at least two candidate regions; according to the region information of the candidate regions, the region similarity between two candidate regions is determined; according to the region similarity between two candidate regions, at least two candidate regions are region merged to obtain at least one target text region; feature extraction is performed on the target text region to obtain key text features; based on a long short-term memory network, character analysis is performed on the key text features corresponding to each target text region to obtain the target text content corresponding to the emoticon package image to be processed; based on an emoticon package reply prompt template, an emoticon package prompt text is generated according to the target emotion type and the target text content; the emoticon package prompt text is input into a large language model to generate text reply content for replying to the emoticon package image to be processed. The above technical solution, with the help of a trained sentiment classification model, identifies the sentiment type of the emoticon package image to be processed according to the auxiliary text, so that the obtained target sentiment category is more accurate, and the accuracy of sentiment analysis of the emoticon package image is improved; secondly, according to the regional information of the candidate area, that is, the regional color information, regional texture information, regional area and regional text information, the regional similarity between the candidate areas is determined, which can make the obtained regional similarity between the candidate areas more accurate, and facilitate the subsequent merging of more similar candidate areas, so as to obtain multiple target text areas with more complete regional information, thereby making the target text content obtained subsequently more accurate; based on more accurate target sentiment categories and target text content, the emoticon package prompt text generated subsequently is more accurate, and the text reply content of the emoticon package image to be processed by the large language model is more in line with the actual context, more natural and accurate, and easier to arouse the user's emotional resonance, thereby improving the accuracy of the reply content to the emoticon package image, thereby enhancing the interactivity and stickiness between the user and the social media platform, and improving the user's satisfaction with the social media platform.

[0120] Embodiment 3

[0121] Figure 3 This is a schematic diagram of the structure of an emoticon image processing device provided in the third embodiment of the present invention. This embodiment can be applied to the case of processing emoticon images used in social media communication. The device can be implemented in the form of hardware and / or software and can be configured in an electronic device. Figure 3 As shown, the device comprises:

[0122] The auxiliary text extraction module 301 is used to obtain the emoticon package picture to be processed and extract the auxiliary text from the context text of the emoticon package picture to be processed;

[0123] The target emotion type determination module 302 is used to identify the emotion type of the emoticon package picture to be processed based on the trained emotion classification model and the auxiliary text, and determine the target emotion type to which the emoticon package picture to be processed belongs; wherein the emotion classification model is trained based on the contrast loss of the sample image-text pair and the entropy of the predicted probability distribution; the sample image-text pair includes a sample emoticon package picture and annotated emotion type;

[0124] The target text content determination module 303 is used to extract text from the emoticon package image to be processed to obtain the target text content corresponding to the emoticon package image to be processed;

[0125] The emoticon package prompt text generation module 304 is used to generate the emoticon package prompt text based on the emoticon package reply prompt template, according to the target emotion type and the target text content;

[0126] The text reply content generation module 305 is used to input the emoticon package prompt text into the large language model to generate text reply content for replying to the emoticon package image to be processed.

[0127] The technical solution of the embodiment of the present invention is to obtain an emoticon package image to be processed and extract auxiliary text from the context text of the emoticon package image to be processed; based on a trained emotion classification model, the emotion type of the emoticon package image to be processed is identified according to the auxiliary text, and the target emotion type to which the emoticon package image to be processed belongs is determined; wherein the emotion classification model is trained according to the contrast loss of sample image-text pairs and the entropy of the predicted probability distribution; the sample image-text pairs include sample emoticon package images and annotated emotion types; text is extracted from the emoticon package image to be processed to obtain the target text content corresponding to the emoticon package image to be processed; based on an emoticon package reply prompt template, an emoticon package prompt text is generated according to the target emotion type and the target text content; the emoticon package prompt text is input into a large language model to generate text reply content for replying to the emoticon package image to be processed. The above technical solution, with the help of a trained sentiment classification model, identifies the sentiment type of the emoticon package image to be processed according to the auxiliary text, so that the obtained target sentiment category is more accurate, and the accuracy of the sentiment analysis of the emoticon package image is improved, thereby making the subsequently generated emoticon package prompt text more accurate, and thus making the text reply content of the emoticon package image to be processed by the large language model more in line with the actual context, more natural and accurate, and easier to arouse the user's emotional resonance, thereby improving the accuracy of the reply content to the emoticon package image, thereby enhancing the interactivity and stickiness between users and the social media platform, and improving user satisfaction with the social media platform.

[0128] Optionally, the labeled emotion type includes at least one of the following: wry smile, sarcasm, sadness, anger, fear, and surprise; the contrast loss of the sample image-text pair is as follows:

[0129]

[0130] Among them, L Contrastive represents the contrast loss of the sample image-text pair, N represents the total number of sample image-text pairs, i represents the i-th sample image-text pair, y i represents the sample label corresponding to the i-th sample image-text pair, m represents the boundary parameter, d i Represents the Euclidean distance between the sample emoticon image and the annotated emotion type in the i-th sample image-text pair.

[0131] Optionally, the target emotion type determination module 302 includes a model training submodule, and the model training submodule includes:

[0132] The target loss function determination unit is used to determine the target loss function of the model training according to the contrast loss of the sample image-text pair and the entropy of the predicted probability distribution through the following formula:

[0133]

[0134] Among them, L represents the target loss function, β represents the regularization parameter, and p i represents the predicted probability distribution of the i-th sample image-text pair, H(p i ) represents the entropy of the predicted probability distribution of the i-th sample image-text pair;

[0135] A sample auxiliary text generating unit, used to generate at least one sample auxiliary text according to the marked emotion type and the auxiliary text template;

[0136] The model training unit is used to train the sentiment classification model according to the target loss function, the sample emoticon package image, and at least one sample auxiliary text to obtain a trained sentiment classification model.

[0137] Optional, model training unit, including:

[0138] A coding feature determination subunit, used to perform coding processing on the sample emoticon package image and at least one sample auxiliary text respectively to obtain an image coding feature and at least one text coding feature;

[0139] A sample emotion type determination subunit is used to determine the sample emotion type to which the sample emoticon package picture belongs according to the feature similarity between the image coding feature and each text coding feature;

[0140] A training loss determination subunit is used to determine the training loss according to the target loss function and the sample sentiment type;

[0141] The model determination subunit is used to train the sentiment classification model according to the training loss to obtain a trained sentiment classification model.

[0142] Optionally, a sample emotion type determination subunit is used to:

[0143] Extract the maximum feature similarity from the feature similarities between the image encoding feature and each text encoding feature;

[0144] The text encoding features corresponding to the maximum feature similarity are decoded to obtain the sample emotions corresponding to the sample emoticon package pictures.

[0145] Optionally, the target text content determination module 303 includes:

[0146] A candidate region determination unit, used for performing image segmentation on the emoticon package image to be processed to obtain at least two candidate regions;

[0147] A region similarity determination unit, used to determine the region similarity between two candidate regions according to the region information of the candidate regions;

[0148] A target text region determination unit, configured to perform region merging processing on at least two candidate regions according to the region similarity between any two candidate regions, to obtain at least one target text region;

[0149] A key text feature determination unit is used to extract features from a target text area to obtain key text features;

[0150] The target text content determination unit is used to perform character analysis on the key text features corresponding to each target text area based on the long short-term memory network to obtain the target text content corresponding to the emoticon package image to be processed.

[0151] Optionally, the region similarity determination unit is specifically used to:

[0152] Determine the content similarity between the first candidate region and the second candidate region according to the first text feature of the first candidate region and the second text feature of the second candidate region;

[0153] Determine the size similarity between the first candidate region and the second candidate region according to the first region area of ​​the first candidate region and the second region area of ​​the second candidate region;

[0154] Determining the texture similarity between the first candidate region and the second candidate region according to the texture influence factor and the Euclidean distance between the first candidate region and the second candidate region;

[0155] Determine the color similarity between the first candidate region and the second candidate region according to the first color information of the first candidate region and the second color information of the second candidate region;

[0156] The content similarity, size similarity, texture similarity and color similarity are weightedly summed to obtain the region similarity between the first candidate region and the second candidate region.

[0157] The emoticon package image processing device provided in the embodiment of the present invention can execute the emoticon package image processing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing each emoticon package image processing method.

[0158] According to an embodiment of the present invention, the present invention also provides an electronic device, a readable storage medium and a computer program product.

[0159] Embodiment 4

[0160] Figure 4 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0161] like Figure 4 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, ROM 12 and RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0162] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0163] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as the emoticon package image processing method.

[0164] In some embodiments, the emoticon image processing method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the emoticon image processing method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the emoticon image processing method in any other appropriate manner (e.g., by means of firmware).

[0165] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0166] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0167] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0168] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0169] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0170] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0171] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0172] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for processing emoticon images, characterized in that: include: Obtain an emoticon package image to be processed, and extract auxiliary text from the context text of the emoticon package image to be processed; Based on the trained sentiment classification model, the sentiment type of the emoticon package picture to be processed is identified according to the auxiliary text, and the target sentiment type of the emoticon package picture to be processed is determined; wherein the sentiment classification model is trained according to the contrast loss of the sample image-text pair and the entropy of the predicted probability distribution; the sample image-text pair includes a sample emoticon package picture and annotated sentiment type; the contrast loss of the sample image-text pair is as follows: Among them, L Contrastive represents the contrast loss of the sample image-text pair, N represents the total number of sample image-text pairs, i represents the i-th sample image-text pair, y i represents the sample label corresponding to the i-th sample image-text pair, m represents the boundary parameter, d i represents the Euclidean distance between the sample emoticon image and the labeled emotion type in the i-th sample image-text pair; if the i-th sample image-text pair is a positive sample pair, then y i =0; if the i-th sample image-text pair is a negative sample pair, then y i =1; Performing text extraction on the emoticon package image to be processed to obtain target text content corresponding to the emoticon package image to be processed; Based on the emoji reply prompt template, generating an emoji prompt text according to the target emotion type and the target text content; Inputting the emoticon package prompt text into a large language model to generate text reply content for replying to the emoticon package image to be processed; The sentiment classification model is trained based on the contrast loss of sample image-text pairs and the entropy of predicted probability distribution, including: According to the contrast loss of the sample image-text pair and the entropy of the predicted probability distribution, the target loss function of the model training is determined by the following formula: Among them, L represents the target loss function, β represents the regularization parameter, and p i represents the predicted probability distribution of the i-th sample image-text pair, H(p i ) represents the entropy of the predicted probability distribution of the i-th sample image-text pair; Generate at least one sample auxiliary text according to the marked emotion type and the auxiliary text template; The sentiment classification model is trained according to the target loss function, the sample emoticon package image, and the at least one sample auxiliary text to obtain a trained sentiment classification model.

2. The method according to claim 1, characterized in that The labeled emotion type includes at least one of the following: wry smile, sarcasm, sadness, anger, fear and surprise.

3. The method according to claim 1, characterized in that The step of training the sentiment classification model according to the target loss function, the sample emoticon package image, and the at least one sample auxiliary text to obtain a trained sentiment classification model includes: Encoding the sample emoticon package image and the at least one sample auxiliary text respectively to obtain an image encoding feature and at least one text encoding feature; Determining the sample emotion type to which the sample emoticon package picture belongs according to the feature similarity between the image coding feature and each text coding feature; Determining a training loss according to the target loss function and the sample emotion type; The sentiment classification model is trained according to the training loss to obtain a trained sentiment classification model.

4. The method according to claim 3, characterized in that The step of determining the sample emotion type to which the sample emoticon package picture belongs according to the feature similarity between the image encoding feature and each text encoding feature includes: Extracting the maximum feature similarity from the feature similarities between the image encoding feature and each text encoding feature; The text encoding feature corresponding to the maximum feature similarity is decoded to obtain the sample emotion type to which the sample emoticon package picture belongs.

5. The method according to claim 1, characterized in that The step of extracting text from the emoticon package image to be processed to obtain target text content corresponding to the emoticon package image to be processed includes: Performing image segmentation on the emoticon package picture to be processed to obtain at least two candidate regions; Determine the regional similarity between any two candidate regions according to the regional information of the candidate regions; According to the region similarity between the two candidate regions, performing region merging processing on the at least two candidate regions to obtain at least one target text region; Extracting features from the target text area to obtain key text features; Based on the long short-term memory network, character analysis is performed on the key text features corresponding to each target text area to obtain the target text content corresponding to the emoticon package image to be processed.

6. The method according to claim 5, characterized in that The determining, based on the region information of the candidate regions, the region similarity between any two candidate regions comprises: Determine the content similarity between the first candidate region and the second candidate region according to the first text feature of the first candidate region and the second text feature of the second candidate region; Determining a size similarity between the first candidate region and the second candidate region according to a first region area of ​​the first candidate region and a second region area of ​​the second candidate region; determining a texture similarity between the first candidate region and the second candidate region according to a texture influence factor and a Euclidean distance between the first candidate region and the second candidate region; determining, according to first color information of the first candidate region and second color information of the second candidate region, a color similarity between the first candidate region and the second candidate region; A weighted sum process is performed on the content similarity, the size similarity, the texture similarity, and the color similarity to obtain a region similarity between the first candidate region and the second candidate region.

7. An emoticon package image processing device, characterized in that: include: An auxiliary text extraction module is used to obtain an emoticon package image to be processed and extract auxiliary text from the context text of the emoticon package image to be processed; The target emotion type determination module is used to identify the emotion type of the emoticon package picture to be processed based on the trained emotion classification model and the auxiliary text, and determine the target emotion type to which the emoticon package picture to be processed belongs; wherein the emotion classification model is trained based on the contrast loss of the sample image-text pair and the entropy of the predicted probability distribution; the sample image-text pair includes a sample emoticon package picture and annotated emotion type; the contrast loss of the sample image-text pair is as follows: Among them, L Contrastive represents the contrast loss of the sample image-text pair, N represents the total number of sample image-text pairs, i represents the i-th sample image-text pair, y i represents the sample label corresponding to the i-th sample image-text pair, m represents the boundary parameter, d i represents the Euclidean distance between the sample emoticon image and the labeled emotion type in the i-th sample image-text pair; if the i-th sample image-text pair is a positive sample pair, then y i =0; if the i-th sample image-text pair is a negative sample pair, then y i =1; A target text content determination module is used to extract text from the emoticon package image to be processed to obtain the target text content corresponding to the emoticon package image to be processed; An emoticon prompt text generation module, used to generate an emoticon prompt text based on an emoticon reply prompt template, according to the target emotion type and the target text content; A text reply content generation module, used for inputting the emoticon package prompt text into a large language model to generate text reply content for replying to the emoticon package image to be processed; The target emotion type determination module includes a model training submodule, and the model training submodule includes: The target loss function determination unit is used to determine the target loss function of the model training according to the contrast loss of the sample image-text pair and the entropy of the predicted probability distribution through the following formula: Among them, L represents the target loss function, β represents the regularization parameter, and p i represents the predicted probability distribution of the i-th sample image-text pair, H(p i ) represents the entropy of the predicted probability distribution of the i-th sample image-text pair; A sample auxiliary text generating unit, used to generate at least one sample auxiliary text according to the marked emotion type and the auxiliary text template; The model training unit is used to train the sentiment classification model according to the target loss function, the sample emoticon package image, and the at least one sample auxiliary text to obtain a trained sentiment classification model.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the emoticon package image processing method described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the emoticon package image processing method according to any one of claims 1 to 6 when executed.

Images