Aspect-level cross-language sentiment analysis framework, product, medium and equipment

Abstract

The invention discloses an aspect-level cross-language sentiment analysis framework, a product, a medium and equipment, and belongs to the technical field of artificial intelligence. The framework is an improved MIXER framework, and the improvement of a code hybrid context demonstration module comprises the following steps: constructing a multi-language data set based on a source language data set and a target language data set so as to enrich semantic information; semantic alignment optimization is carried out by introducing a similarity scoring mechanism, so that the model capture capability is improved; the improvement of the language specific tag module comprises the following steps: designing a language category tag for the constructed multi-language data set; taking the language category label as a part of model input, and guiding the model to pay attention to language differences; and the high-efficiency fine tuning module is used for finely tuning the optimized parameters so as to support high-efficiency adaptation of the whole frame on a large-scale pre-training model. According to the method, the performance of cross-language sentiment analysis can be improved, and the multi-language adaptive capacity of the model is enhanced while dependence on target language annotation data is reduced.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and specifically relates to an aspect-level cross-language sentiment analysis framework, product, medium and device based on code hybrid context learning and language-specific tag enhancement. Background Art

[0002] Cross-lingual aspect-level sentiment analysis (XABSA) aims to extract aspect-level sentiment across multiple languages. This task typically relies on source language data to train models and transfer them to the target language, thus facing significant challenges such as data scarcity and language differences.

[0003] When performing cross-language aspect-level sentiment analysis, existing technologies typically use machine translation tools to align sentiment annotation data in the source language to the target language, addressing language differences and data scarcity issues at the data level. When aligning pre-trained models, pre-trained language models (such as mBERT and XLM-R) or contrastive learning techniques are used to align sentiment semantic spaces across languages. In multilingual contextual learning (ICL), context is leveraged, relying on a small number of examples and prompts to improve the model's cross-language transfer capabilities. Furthermore, heuristic methods may be used to select semantics and labels to optimize sample selection, and supervised learning may be used to train retrievers to improve relevance. The main drawbacks of existing technologies include:

[0004] 1. Although pre-trained models cover multiple languages, they still face performance limitations when handling specific tasks, mainly due to insufficient model size and ability to capture contextual nuances.

[0005] 2. Dependence on target language data: Most methods still require a certain amount of target language annotated data to improve performance.

[0006] 3. Insufficient multilingual capabilities: Existing methods perform unstable in multilingual environments and find it difficult to fully utilize multilingual knowledge.

[0007] 4. Cultural and semantic differences: Different languages express emotions differently, and existing methods are difficult to fully align. Summary of the Invention

[0008] The present invention aims to solve one of the technical problems in the above-mentioned related art at least to a certain extent.

[0009] To this end, the purpose of the present invention is to provide an aspect-level cross-language sentiment analysis framework based on code hybrid context learning and language-specific tag enhancement, which can not only improve the sentiment analysis capabilities of low-resource languages, but also achieve efficient migration of sentiment analysis models between different languages and reduce the annotation cost of new languages.

[0010] In order to solve the above-mentioned technical problems, the present invention is achieved as follows:

[0011] An embodiment of the present invention provides an aspect-level cross-language sentiment analysis framework, which is an improved MIXER framework and includes: a code mixing context presentation module and a language-specific labeling module;

[0012] Improvements to the code mixed context demonstration module include: first, constructing a multilingual dataset based on the source and target language datasets to enrich semantic information; second, introducing a similarity scoring mechanism to optimize semantic alignment to enhance the model's ability to capture cross-language semantic differences;

[0013] Improvements to the language-specific labeling module include: first, designing language category labels for the constructed multilingual dataset to mark the example language type to clarify the language identification; second, using language category labels as part of the model input to guide the model to pay attention to language differences and optimize the model's learning of emotional expressions in different languages.

[0014] In addition, the aspect-level cross-language sentiment analysis framework according to the present invention may also have the following additional technical features:

[0015] In some embodiments, the framework further includes an efficient fine-tuning module for fine-tuning optimization parameters to support efficient adaptation of the overall framework on large-scale pre-trained models.

[0016] In some of the embodiments, the specific content of the first aspect of the improvement of the code mixing context demonstration module includes: based on the source language dataset and the target language dataset, generating a source-target dataset and a target-source dataset through cross-replacement of aspect words, thereby obtaining four datasets of source language, target language, source-target and target-source, so as to enrich semantic information and realize cross-language knowledge fusion.

[0017] In some embodiments, the second aspect of the improvement of the code mixed context presentation module includes: introducing a similarity scoring mechanism to perform example retrieval based on the semantic relevance between the query and the candidate sentences to enhance the model's ability to capture cross-language semantic differences.

[0018] In some embodiments, the similarity scoring mechanism is a cosine similarity scoring function.

[0019] In some embodiments, the efficient fine-tuning module uses LoRA technology for fine-tuning, updating a small number of parameters to improve the performance of specific tasks while maintaining model stability.

[0020] In some embodiments, the specific content of example retrieval based on the semantic relevance between the query and the candidate sentence includes: based on the calculated similarity score, selecting several most similar samples from the context retrieval dataset, and constructing a code hybrid demonstration for the current query.

[0021] An embodiment of the present invention further provides a computer program product, including a computer program, characterized in that when the computer program is executed by a processor, the content of the aspect-level cross-language sentiment analysis framework as described in any one of the above items is implemented.

[0022] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the content of the aspect-level cross-language sentiment analysis framework as described in any one of the above items.

[0023] An embodiment of the present invention also provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the content of the aspect-level cross-language sentiment analysis framework as described in any one of the above items.

[0024] Compared with the prior art, the present invention has at least the following beneficial effects:

[0025] In the embodiments of the present invention, the aspect-level cross-language sentiment analysis framework based on code mixing context learning and language-specific label enhancement does not directly rely on target language data. Instead, it analyzes aspect-level sentiment in multilingual contexts by constructing multiple retrieval libraries (such as source language, target language, source-target language hybrid, etc.);

[0026] In an embodiment of the present invention, an aspect-level cross-language sentiment analysis framework based on code hybrid context learning and language-specific tag enhancement is provided, which integrates cross-language features into the context to enhance the language alignment capability of the model;

[0027] In an embodiment of the present invention, an aspect-level cross-language sentiment analysis framework based on code mixing context learning and language-specific tag enhancement is provided. Through improved language-specific tags, the model helps the model quickly identify language types in a multilingual environment and avoid confusion.

[0028] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A model framework diagram disclosed in one embodiment of the present invention;

[0030] Figure 2A code mixing context demonstration diagram disclosed in an embodiment of the present invention;

[0031] Figure 3 The results of using different numbers of demonstrations on Mistral are disclosed for one embodiment of the present invention;

[0032] Figure 4 The results of using different numbers of demonstrations on Llama3 are disclosed for one embodiment of the present invention. DETAILED DESCRIPTION

[0033] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0034] The embodiments of the present invention are described in detail below through specific embodiments and application scenarios with reference to the accompanying drawings.

[0035] See also Figure 1 As shown, some embodiments of the present invention provide an aspect-level cross-language sentiment analysis framework based on code mixed context learning and language-specific label enhancement, which is mainly implemented through the MIXER framework. The MIXER framework mainly includes a code mixed context demonstration module and a language-specific label module.

[0036] The core purpose of the MIXER framework is to address data scarcity, language diversity, and insufficient multilingual model capabilities in cross-lingual sentiment analysis through code mixing context learning. MIXER primarily consists of Code Mixed Context Demonstration (CICD) and Language Specific Tags (LST). It is independent of target language data and enhances prompt diversity through code mixing mechanisms.

[0037] To address data scarcity in cross-lingual sentiment analysis, MIXER, based on a code-mixing mechanism, uses the source dataset SemEval-2016 to construct four demonstration retrieval libraries containing target language knowledge. Specifically, a multilingual translator is first used to expand the source language dataset into the target language. Then, aspect terms are cross-substituted based on the source and target language datasets to construct a source language data retrieval library, a target language data retrieval library, a source-target code hybrid data retrieval library, and a target-source code hybrid database. This enriches the model's language input and alleviates reliance on target language data. MIXER uses these multilingual sentiment classification examples to improve the cross-lingual transfer capabilities of LLMs.

[0038] To address issues such as language differences and insufficient multilingual model capabilities, MIXER can achieve deeper semantic alignment between the source and target languages by introducing code mixing context demonstrations. For example, by scoring the semantic relevance between the query sentence and the candidate sentences, the top k most relevant samples are selected as code mixing demonstrations for the current query. This not only ensures that the examples input to the model are highly relevant to the task, but also enhances the model's ability to understand different languages and its cross-language reasoning capabilities. The introduction of specific language tags helps the model more accurately identify example types in different language contexts, avoiding performance degradation due to language confusion.

[0039] In some embodiments of the present invention, within the MIXER framework, the code mixing context demonstration module utilizes the code mixing mechanism to construct code mixing data, translates the source language data into the target language data, and further performs cross-replacement of aspect terms based on the source language data and the translated target language data. Similar examples are selected from four types of code mixing databases as input context. Next, the language-specific labeling module designs and populates task-related prompt templates, adding specific language tags to each example to ensure that the model can correctly identify different language environments. Its working principles include:

[0040] Task definition: XABSA is considered as a hybrid task involving extraction and classification. Given a training dataset of source language sentences X S Pairing with aspect and sentiment labels The goal of XABSA is to enable the model M to learn sentences X from the target language set. T Extract all included aspect items and sentiment pairs Formally, a represents an entity or aspect in a sentence related to sentiment, or a sentiment polarity e∈{positive,negative,neutral}.

[0041] 1. Code Mixing Context Demonstration Module

[0042] This module mainly achieves deep semantic alignment of the source language and target language in the demonstration through the code hybrid retrieval library design mechanism, and enhances the cross-language reasoning ability of the model.

[0043] First, use a multilingual translator to convert the source data Extension to target language set In order to further integrate more target language knowledge, the aspect item A is replaced based on the above two datasets. This will generate two types of data: source-target (S T ) Language code mixed data The aspect items in the source language data are replaced by the corresponding items in the target language; target-source (T S ) Language code mixed data The aspect terms in the target language data are replaced with the corresponding terms in the source language. Based on the above data, vector encoding is performed to obtain the following demonstration retrieval library:

[0044]

[0045] Among them, σ represents different types of retrieval libraries, σ∈(S, T, S T , T S ). Encoder(.) represents vector encoding.

[0046] Although the construction of the demonstration retrieval library is completely independent of the target language, it contains rich target language knowledge. Therefore, the query set is introduced to implement a selection strategy based on the relevance of the input information. The present invention defines a scoring function, where q i Denotes the i-th query sentence, and d is the candidate vector. The scoring function is based on the semantic relevance between the query and the candidate sentence. Based on these technologies, the present invention constructs a similarity-based calculation method, using cosine similarity to evaluate the semantic proximity between texts. The specific scoring function is as follows:

[0047]

[0048] Score(.) indicates vectorization The cosine score between d and d. Given that selecting contextual examples that are semantically similar to the test sample can better stimulate model performance, the top-k samples are selected from the context retrieval library based on the relevance score to construct a code hybrid demonstration for the current query Q. Each group is represented as follows:

[0049]

[0050] 2. Language-specific tag module

[0051] In order to enhance the model's understanding of different data categories in context and prevent the model from misidentifying language types in a multilingual environment, this paper introduces language specific tags (LST), which include four category tags flags: σ (σ∈(S,T,S T , T S )). These labels will be clearly marked before each example. The present invention uniformly labels the data generated by the code mixing mechanism as "code mixing demonstration". The specific category description and context demonstration are as follows:

[0052]

[0053] D σ Represents the final generated code hybrid demo.

[0054] In some embodiments of the present invention, in order to improve the performance of the model on a specific task, efficient fine-tuning is introduced. Based on the selected demonstration, the present invention constructs a complete prompt, prefixing the aspect word definition, task definition, and output format definition. For LLMs, the present invention adopts supervised fine-tuning (SFT) and introduces an efficient fine-tuning method called low-rank adaptation (LoRA). Unlike traditional methods that require updating all model parameters, LoRA uses low-rank matrix decomposition to update only a small number of parameters, thereby achieving efficient parameter adjustment. Specifically, the original parameter matrix W of the large model is kept frozen unchanged, and the decomposition of the low-rank matrices A and B is used to update the parameters in a specific layer (such as the Transformer layer) to obtain a new fine-tuning weight matrix W′:

[0055] W′=W+AB (6)

[0056] A∈R d ×r and B∈R r ×d is a low-rank matrix with a rank of r. The dimension of the original model parameter matrix is d, and r < < d. While preserving the emotional characteristics of the source language, LoRA helps the model quickly adapt to the target language task by updating a small number of parameters.

[0057] Through the above structure and methods, MIXER can improve the performance of cross-language aspect-level sentiment analysis, while reducing the dependence on target language annotated data and enhancing the model's multilingual adaptability.

[0058] In some embodiments of the present invention, the improvement of the code mixed context demonstration module includes: MIXER introduces code mixed context demonstration, and the construction method of the demonstration is as follows: Figure 2 As shown in the figure, by mixing the source and target languages, richer semantic information is generated. This structure not only preserves the knowledge of the source language but also enhances semantic alignment with the target language. Specifically, code-mixed data combines features from both the source and target languages, thereby enhancing semantic richness and enabling the model to learn richer semantic information. Furthermore, by introducing code information, the semantic differences between the source and target languages are effectively bridged, improving the model's cross-language reasoning capabilities.

[0059] In some embodiments of the present invention, improvements to the language-specific labeling module include: MIXER introduces language-specific labels to clearly mark the language type of each example (source language, target language, or code mix). This structure enables the model to accurately identify and understand the context of different languages in a multilingual environment. Specifically, language-specific labels help the model quickly identify the language type in a multilingual environment and avoid confusion. By clearly marking, the model can better understand the context of different languages and improve the accuracy of cross-language analysis.

[0060] To analyze the advantages of the proposed method, experiments were conducted. The following describes the experimental dataset, experimental setup, main experimental results, ablation experimental results, and other experimental aspects.

[0061] Experimental Dataset: The experiment uses the SemEval-2016 dataset, which contains restaurant user reviews in multiple languages, including English (EN), French (FR), Spanish (ES), Dutch (NL), and Russian (RU). Previous studies excluded Turkish from the experimental analysis due to data size limitations. Following this approach, Turkish was omitted from the experiment. In addition, the source language dataset was preprocessed with code mixing to generate three different datasets: the target language dataset D T , target language dataset D with source language aspects TS and a source language dataset D with target language aspects ST Table 1 shows the specific distribution of the dataset.

[0062] Experimental settings: Experiments were conducted on Llama3-8B and Mistral-7B. All experiments were performed on NVIDIA A40GPU (40GB) environment, and the learning rate was set to 1×10 -4 , using F1 score as the evaluation metric. The following three methods are comparative methods for comprehensively evaluating model performance:

[0063] Equi-XABSA: proposes dynamic weighted loss and anti-decoupling technology to solve the problem of dataset imbalance and reduce semantic gap;

[0064] LLM-SFT: Directly fine-tunes the pre-trained LLM without providing additional hints or examples during training, relying primarily on input data for learning;

[0065] LLM-random: Randomly select k examples as context and incorporate them into the input of the model to guide generation or decision making.

[0066] Main experimental results: The main results shown in Table 2 show that the proposed MIXER model generally outperforms the baseline Equi-XABSA in two PLMs (i.e., XLM-R and mBERT), especially showing significant improvements in ES and NL. Among the two public LLMs, MIXER continues to lead LLM-SFT and LLM-random. Specifically, the MIXER model performs well on the XABSA task, with an average F1 score improvement of 1.59% and 1.44% on Mistral and Llama3, respectively. Overall, MIXER outperforms the baseline, showing its superiority over XABSA.

[0067] Ablation Experiment Results: To verify the effectiveness of the various components, namely, Code Mixed Context Demonstration (CICD) and Language Specific Tags (LST), we conducted a series of ablation experiments using Word2vec (V1) and TF-IDF (V2) as retrieval benchmarks. The results (Table 3) show that removing the entire CICD results in a decrease in average performance. Across the different retrieval benchmarks, the F1 score decreases the most, by 2% on Mistral-V2.

[0068] In addition, removing LST also leads to a decrease in model performance, especially on Mistral-V2, where the average decrease is 1.89%. These findings further confirm the key role of these components in improving model effectiveness.

[0069] Other experiments:

[0070] 1) Impact of the number of demonstrations

[0071] In order to explore the impact of different numbers of examples on the model, the present invention sets up situations where a total of 4, 8 and 12 examples are retrieved, that is, 1, 2 and 3 examples are selected from each retrieval library respectively. Figure 3 and Figure 4 The effects of different numbers of examples on the Mistral and Llama3 models are shown. When using the Mistral model, a configuration with 4 examples generally performs better, especially the 4-V2 configuration. However, when the number of examples increases to 8 and 12, model performance decreases, with the decrease being particularly pronounced for the 12-example configuration. On the Llama3 model, a configuration with 4 examples also demonstrates higher performance. Overall, for both models, a smaller number of examples leads to better results, likely because fewer examples reduce the complexity of the model's processing, allowing the model to more effectively focus on the most relevant contextual information.

[0072] 2) Example search strategy

[0073] In order to study the ability of MIXER in enhancing language alignment, we conduct four monolingual demonstration experiments: English-only demonstration (EO), translated target language-only demonstration (TO), source-target mixed code-only demonstration (STO), and target-source mixed code-only demonstration (TSO).

[0074] As shown in Table 5, the results show that although the proposed MIXER performs slightly worse than monolingual context learning (ICL) in some cases, it performs better in most cases, especially in the Mistral-V2 and Llama3-V1 models, where its performance is significantly better than other methods. This shows that MIXER not only ensures a full understanding of the target language, but also effectively preserves the source language information, thereby successfully achieving alignment between the source and target languages.

[0075] According to the experimental results provided by this product, the following analysis can be made:

[0076] Table 2 shows a performance comparison of different methods on multiple languages, specifically including the performance of different methods based on XLM-R and mBERT pre-trained models, and large language models (LLMs) based on Llama3 and Mistral. The evaluation metric is the F1 score. The MIXER method achieves the best overall average F1 score on both large language models (Llama3 and Mistral), with scores of 67.10 and 65.75, respectively, outperforming the baseline methods. Furthermore, MIXER achieves the highest F1 score for French and the second highest for Spanish.

[0077] In addition, if Figure 3 and Figure 4 As shown in the figure, ablation experiments found that when using 4 examples (1 example per retrieval library) for Mistral and Llama3, the model performed best. In particular, the Mistral model performed particularly well when configured as 4-V2 (the version using TF-IDF as the retrieval method). Therefore, when it is necessary to process cross-lingual sentiment analysis tasks quickly and efficiently and hope to reduce the increased complexity caused by excessive contextual information, a small number of highly relevant examples should be selected.

[0078] Table 1 SemEval-2016 dataset

[0079]

[0080] Table 2 Comparison of different cross-language methods

[0081]

[0082]

[0083] Note: In each language, bold indicates the best performance and underlined indicates the second best performance.

[0084] Table 3 Ablation experiment results

[0085]

[0086] Note: Comparison of CICD and LST ablation results under different retrieval benchmarks. Bold indicates the best performance, and underlined indicates the suboptimal performance.

[0087] Table 4 Results of different strategies

[0088]

[0089] Table 5 Results on different demonstration languages

[0090]

[0091]

[0092] Overall, we can infer that MIXER is particularly suitable for French and Spanish scenarios. In the absence of target languages, it can effectively utilize existing language data to complete predictions in the target language.

[0093] Parts of the present invention that are not described in detail may refer to the prior art or are well known to those skilled in the art, and this embodiment does not limit this and will not be described in detail here.

[0094] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are protected by the present invention.

Claims

1. A cross-language sentiment analysis framework at the aspect level, characterized by: The framework is an improved MIXER framework, including: a code mixing context presentation module and a language specific tag module; Improvements to the code mixed context demonstration module include: first, constructing a multilingual dataset based on the source and target language datasets to enrich semantic information; second, introducing a similarity scoring mechanism to optimize semantic alignment to enhance the model's ability to capture cross-language semantic differences; Improvements to the language-specific labeling module include: first, designing language category labels for the constructed multilingual dataset to mark the example language type to clarify the language identification; second, using language category labels as part of the model input to guide the model to pay attention to language differences and optimize the model's learning of emotional expressions in different languages.

2. The aspect-level cross-language sentiment analysis framework according to claim 1, characterized in that The framework also includes an efficient fine-tuning module for fine-tuning optimization parameters to support efficient adaptation of the overall framework on large-scale pre-trained models.

3. The aspect-level cross-language sentiment analysis framework according to claim 1, characterized in that The specific content of the first aspect of the improvement of the code mixing context demonstration module includes: based on the source language dataset and the target language dataset, through cross-replacement of aspect words, generating source-target datasets and target-source datasets, thereby obtaining four datasets of source language, target language, source-target and target-source, so as to enrich semantic information and realize cross-language knowledge fusion.

4. The aspect-level cross-language sentiment analysis framework according to claim 1, characterized in that The second improvement to the code mixed context demonstration module includes the introduction of a similarity scoring mechanism to perform example retrieval based on the semantic relevance between the query and the candidate sentences, thereby improving the model's ability to capture cross-language semantic differences.

5. The aspect-level cross-language sentiment analysis framework according to claim 1, characterized in that The similarity scoring mechanism is a cosine similarity scoring function.

6. The aspect-level cross-language sentiment analysis framework according to claim 2, characterized in that The efficient fine-tuning module uses LoRA technology for fine-tuning, updating a small number of parameters to improve the performance of specific tasks while maintaining model stability.

7. The aspect-level cross-language sentiment analysis framework according to claim 4, characterized in that The specific content of example retrieval based on the semantic relevance between the query and the candidate sentences includes: based on the calculated similarity score, selecting several most similar samples from the context retrieval dataset, and building a code hybrid demonstration for the current query.

8. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the content of the aspect-level cross-language sentiment analysis framework described in any one of claims 1 to 7 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the content of the aspect-level cross-language sentiment analysis framework described in any one of claims 1 to 7 is implemented.

10. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the content of the aspect-level cross-language sentiment analysis framework according to any one of claims 1 to 7.

Images