Domain self-adaption method and device based on instance cue word, equipment and storage medium
By generating and combining text prompt words, the problem of poor adaptability of natural language processing models in different fields is solved, the prediction accuracy and adaptability of the model in different fields is improved, and strict requirements for target domain data are avoided.
Patent Information
- Application Number
- CN202411072390.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-06
- Publication Date
- 2024-12-03
AI Technical Summary
The existing natural language processing model has poor adaptability among different fields, especially when the training data distribution does not match the input data distribution, the prediction accuracy decreases, and the existing domain adaptation methods are limited by the feature representation ability of the autoencoder and Reliance on label data.
By generating text prompt words related to the input instance, decomposing them into domain names and feature words, determining the target source domain prompt words based on mutual information and other methods, and combining them with the input instance as a new input to the task model to improve the model Adaptability and performance in different fields.
The model is highly adaptable and performance improvement in data in different fields, and does not require strict requirements on the target domain data, which improves the prediction accuracy of unknown domains.
Smart Images

Figure CN119067110A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of domain adaptation, and in particular to a domain adaptation method, device, equipment and storage medium based on instance prompt words. Background Art
[0002] In recent years, with the improvement of computing power and the abundance of data resources, large models have achieved significant performance improvements on many natural language processing (NLP) tasks, such as text classification, sentiment analysis, language translation, text generation, etc. In some benchmarks, large models have even outperformed humans. However, these models often rely on an important assumption that the training set and test set come from the same underlying distribution (i.e., independent and identically distributed). This assumption may not always hold true in reality, because text may come from many different sources, each with unique distribution properties. Therefore, when these models encounter inputs that do not match the distribution of the training data, they may not be able to understand and process these inputs correctly, resulting in inaccurate or even wrong predictions. This raises an important challenge, which is how to enable these models to maintain high prediction accuracy when faced with inputs that are different from the training distribution. In the field of NLP, there are many studies trying to solve this problem.
[0003] For example, some studies use domain adaptation methods (such as transfer learning, adaptive learning, or online learning techniques) to mitigate the out-of-distribution problem by adapting the model to the new input distribution. Existing domain adaptation methods mainly reconstruct or transform the features of the source domain and the target domain, mainly feature adaptation and instance adaptation. Feature adaptation extracts the features of the source domain and the target domain into a common feature space, so that the distance between the source domain and the target domain in this space is close enough to align the two, thereby improving the performance of the target domain. If some data in the source domain is similar to the target domain, these data can be weighted, and the model trained using these data will perform better in the target domain.
[0004] However, existing domain adaptation methods also have some limitations: First, they are limited by the feature representation capabilities of the autoencoder. Reconstruction-based domain adaptation methods need to use the encoder to learn the feature representations of the source and target domains, but the feature representation capabilities of the encoder may limit the effect of domain adaptation. Then, labeled target domain data is required. Domain adaptation methods based on sample generation need to synthesize labeled target domain samples, but the quality of synthesized samples may be affected by many factors, such as the difference in sample distribution between the source and target domains, the performance of the model itself, etc. The above problems make a domain adaptation system that can adapt to any data source a promising work. Summary of the invention
[0005] The main purpose of the present invention is to provide a domain adaptation method, device, equipment and storage medium based on instance prompt words, aiming to solve the technical problem that the existing natural language processing (NLP) model has poor adaptability between different fields.
[0006] To achieve the above object, the present invention provides a domain adaptation method based on instance prompt words, and the domain adaptation method based on instance prompt words comprises the following steps:
[0007] Selecting a task model, receiving an instance of the input task model, and generating a text prompt word for the instance according to a source domain of the task model;
[0008] Decomposing the text prompt word into a domain name and a feature word, and determining a target source domain prompt word of the instance according to the domain name and the feature word;
[0009] The target source domain prompt word and the instance are used as inputs of the task model to obtain a target output of the task model.
[0010] Optionally, the step of determining the target source domain prompt word of the instance according to the domain name and the feature word includes:
[0011] Based on the mutual information method, the domain name and the feature word are scored according to preset rules, and the target source domain prompt word of the instance is extracted from the domain name and the feature word according to the score.
[0012] Optionally, the step of extracting the source domain prompt word of the instance from the domain name and the feature word according to the score includes:
[0013] According to the scores, a preset number of feature words are selected as candidate source domain prompt words;
[0014] The candidate source domain prompt words are combined to obtain the target source domain prompt word of the instance.
[0015] Optionally, the step of decomposing the text prompt word into a domain name and a feature word comprises:
[0016] Preprocessing the text prompt words, wherein the preprocessing includes word segmentation and tokenization to remove stop words, and obtaining preprocessed text;
[0017] Domain names and characteristic words are extracted from the preprocessed text.
[0018] Optionally, the step of extracting domain names and feature words from the preprocessed text includes:
[0019] selecting the domain name from the preprocessed text using a predefined list of domain names;
[0020] The characteristic word associated with the domain name is selected from the preprocessed text.
[0021] Optionally, the step of generating text prompt words for the instance according to the source domain of the task model includes:
[0022] Analyze the text information of the instance and extract key features and background information of the instance;
[0023] Generate a preliminary text prompt word template according to the characteristics of the source domain and the key characteristics and background information of the instance;
[0024] The text prompt word template is optimized to obtain the text prompt word.
[0025] Optionally, the step of generating text prompt words for the instance according to the source domain of the task model includes:
[0026] Encoding the instance to generate a high-dimensional feature representation, and converting the high-dimensional feature representation into a feature representation of the source domain through a decoder;
[0027] A text prompt word of the instance is generated according to the feature representation of the source domain.
[0028] Furthermore, to achieve the above-mentioned object, the present invention also provides a domain adaptation device based on example prompt words, and the domain adaptation device based on example prompt words comprises:
[0029] A receiving module selects a task model, receives an instance of the input task model, and generates a text prompt word for the instance according to a source domain of the task model;
[0030] A generating module, decomposing the text prompt word into a domain name and a feature word, and determining a target source domain prompt word of the instance according to the domain name and the feature word;
[0031] The output module uses the target source domain prompt word and the instance as input of the task model to obtain the target output of the task model.
[0032] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a domain adaptation device based on instance prompt words, wherein the domain adaptation device based on instance prompt words comprises a memory, a processor, and a domain adaptation program based on instance prompt words stored in the memory and executable on the processor, wherein the domain adaptation program based on instance prompt words implements the steps of the domain adaptation method based on instance prompt words as described above when executed by the processor.
[0033] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a storage medium, on which a domain adaptation program based on instance prompt words is stored, and when the domain adaptation program based on instance prompt words is executed by a processor, the steps of the domain adaptation method based on instance prompt words as described above are implemented.
[0034] The present invention relates to a domain adaptation method based on instance prompt words, which selects a task model, receives an input instance, and generates text prompt words related to the instance according to the source domain of the task model. The text prompt words are decomposed into domain names and feature words, and the target source domain prompt words of the instance are determined according to the domain names and feature words. The target source domain prompt words and the instance are then associated as new inputs of the task model to obtain the target output of the task model. The present invention effectively improves the adaptability and performance of the model on data in different fields by generating unique text prompt words and combining an autoregressive prompt learning algorithm. The system can adapt to any target field without strict requirements on the data in the target domain. By generating unique prompt words, the prediction accuracy of the model in unknown domains can be effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] Figure 1 It is a structural diagram of a device hardware operating environment involved in an embodiment of a domain adaptation device based on example prompt words of the present invention;
[0036] Figure 2 It is a flowchart of a first embodiment of a domain adaptation method based on instance prompt words of the present invention;
[0037] Figure 3 It is a functional module diagram of a preferred embodiment of a domain adaptation device based on example prompt words of the present invention.
[0038] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0039] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.
[0040] It should be noted that in recent years, with the improvement of computing power and the abundance of data resources, large models have achieved significant performance improvements in many natural language processing (NLP) tasks, such as text classification, sentiment analysis, language translation, text generation, etc. In some benchmarks, the performance of large models even exceeds that of humans. However, these models often rely on an important assumption that the training set and the test set come from the same underlying distribution (that is, independent and identically distributed). This assumption may not always hold true in reality, because text may come from many different sources, each with unique distribution properties. Therefore, when these models encounter inputs that do not match the distribution of the training data, they may not be able to correctly understand and process these inputs, resulting in inaccurate or even wrong prediction results. This poses an important challenge, which is how to enable these models to maintain high prediction accuracy when faced with inputs that are different from the training distribution. In the field of NLP, there are many studies trying to solve this problem.
[0041] For example, some studies use domain adaptation methods (such as transfer learning, adaptive learning, or online learning techniques) to mitigate the out-of-distribution problem by adapting the model to the new input distribution. Existing domain adaptation methods mainly reconstruct or transform the features of the source domain and the target domain, mainly feature adaptation and instance adaptation. Feature adaptation extracts the features of the source domain and the target domain into a common feature space, so that the distance between the source domain and the target domain in this space is close enough to align the two, thereby improving the performance of the target domain. If some data in the source domain is similar to the target domain, these data can be weighted, and the model trained using these data will perform better in the target domain.
[0042] However, existing domain adaptation methods also have some limitations: First, they are limited by the feature representation capabilities of the autoencoder. Reconstruction-based domain adaptation methods need to use the encoder to learn the feature representations of the source and target domains, but the feature representation capabilities of the encoder may limit the effect of domain adaptation. Then, labeled target domain data is required. Domain adaptation methods based on sample generation need to synthesize labeled target domain samples, but the quality of synthesized samples may be affected by many factors, such as the difference in sample distribution between the source and target domains, the performance of the model itself, etc. The above problems make a domain adaptation system that can adapt to any data source a promising work.
[0043] Based on the above defects, the present invention provides a domain adaptation device based on example prompt words, referring to Figure 1 , Figure 1 It is a structural diagram of a device hardware operating environment involved in an embodiment of a domain adaptation device based on instance prompt words of the present invention.
[0044] like Figure 1As shown, the domain adaptation device based on the instance prompt word may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.
[0045] Those skilled in the art will understand that Figure 1 The hardware structure of the domain adaptation device based on instance prompt words shown in the figure does not constitute a limitation on the domain adaptation device based on instance prompt words, and may include more or less components than shown in the figure, or combine some components, or arrange the components differently.
[0046] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a domain adaptive program based on example prompt words. The operating system is a program for managing and controlling domain adaptive devices and software resources based on example prompt words, and supports the operation of the network communication module, the user interface module, the domain adaptive program based on example prompt words, and other programs or software; the network communication module is used to manage and control the network interface 1004; the user interface module is used to manage and control the user interface 1003.
[0047] exist Figure 1 In the hardware structure of the domain adaptation device based on the example prompt words shown, the network interface 1004 is mainly used to connect to the background server and perform data communication with the background server; the user interface 1003 is mainly used to connect to the client and perform data communication with the client; the processor 1001 can call the domain adaptation program based on the example prompt words stored in the memory 1005, and perform the following operations:
[0048] Selecting a task model, receiving an instance of the input task model, and generating a text prompt word for the instance according to a source domain of the task model;
[0049] Decomposing the text prompt word into a domain name and a feature word, and determining a target source domain prompt word of the instance according to the domain name and the feature word;
[0050] The target source domain prompt word and the instance are used as inputs of the task model to obtain a target output of the task model.
[0051] Furthermore, the step of determining the target source domain prompt word of the instance according to the domain name and the feature word includes:
[0052] Based on the mutual information method, the domain name and the feature word are scored according to preset rules, and the target source domain prompt word of the instance is extracted from the domain name and the feature word according to the score.
[0053] Furthermore, the step of extracting the source domain prompt word of the instance from the domain name and the feature word according to the score includes:
[0054] According to the scores, a preset number of feature words are selected as candidate source domain prompt words;
[0055] The candidate source domain prompt words are combined to obtain the target source domain prompt word of the instance.
[0056] Furthermore, the step of decomposing the text prompt word into a domain name and a feature word includes:
[0057] Preprocessing the text prompt words, wherein the preprocessing includes word segmentation and tokenization to remove stop words, and obtaining preprocessed text;
[0058] Domain names and characteristic words are extracted from the preprocessed text.
[0059] Furthermore, the step of extracting domain names and feature words from the preprocessed text includes:
[0060] selecting the domain name from the preprocessed text using a predefined list of domain names;
[0061] The characteristic word associated with the domain name is selected from the preprocessed text.
[0062] Furthermore, the step of generating text prompt words for the instance according to the source domain of the task model includes:
[0063] Analyze the text information of the instance and extract key features and background information of the instance;
[0064] Generate a preliminary text prompt word template according to the characteristics of the source domain and the key characteristics and background information of the instance;
[0065] The text prompt word template is optimized to obtain the text prompt word.
[0066] Furthermore, the step of generating text prompt words for the instance according to the source domain of the task model includes:
[0067] Encoding the instance to generate a high-dimensional feature representation, and converting the high-dimensional feature representation into a feature representation of the source domain through a decoder;
[0068] A text prompt word of the instance is generated according to the feature representation of the source domain.
[0069] The specific implementation of the domain adaptation device based on example prompt words of the present invention is basically the same as the embodiments of the domain adaptation method based on example prompt words described below, and will not be repeated here.
[0070] The present invention also provides a domain adaptation method based on instance prompt words based on the above domain adaptation device based on instance prompt words.
[0071] Reference Figure 2 , Figure 2 FIG. 4 is a flow chart of a first embodiment of a domain adaptation method based on instance prompt words of the present invention.
[0072] The embodiment of the present invention provides an embodiment of a domain adaptation method based on instance prompt words. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in an order different from that here.
[0073] In various embodiments of the domain adaptation method based on instance prompt words, the execution subject is a domain adaptation device based on instance prompt words.
[0074] The domain adaptation method based on instance prompt words is applied to a domain adaptation device based on instance prompt words, and the domain adaptation method based on instance prompt words includes:
[0075] Step S10, selecting a task model, receiving an instance of the input task model, and generating text prompt words for the instance according to a source domain of the task model;
[0076] In recent years, with the improvement of computing power and the abundance of data resources, large models have achieved significant performance improvements on many natural language processing (NLP) tasks, such as text classification, sentiment analysis, language translation, text generation, etc. In some benchmarks, large models have even outperformed humans. However, these models often rely on an important assumption that the training set and test set come from the same underlying distribution (i.e., independent and identically distributed). This assumption may not always hold true in reality, because text may come from many different sources, each with unique distribution properties. Therefore, when these models encounter inputs that do not match the distribution of the training data, they may not be able to understand and process these inputs correctly, resulting in inaccurate or even wrong predictions. This raises an important challenge, which is how to enable these models to maintain high prediction accuracy when faced with inputs that are different from the training distribution. In the field of NLP, there are many studies trying to solve this problem.
[0077] For example, some studies use domain adaptation methods (such as transfer learning, adaptive learning, or online learning techniques) to mitigate the out-of-distribution problem by adapting the model to the new input distribution. Existing domain adaptation methods mainly reconstruct or transform the features of the source domain and the target domain, mainly feature adaptation and instance adaptation. Feature adaptation extracts the features of the source domain and the target domain into a common feature space, so that the distance between the source domain and the target domain in this space is close enough to align the two, thereby improving the performance of the target domain. If some data in the source domain is similar to the target domain, these data can be weighted, and the model trained using these data will perform better in the target domain.
[0078] However, existing domain adaptation methods also have some limitations: First, they are limited by the feature representation capabilities of the autoencoder. Reconstruction-based domain adaptation methods need to use the encoder to learn the feature representations of the source and target domains, but the feature representation capabilities of the encoder may limit the effect of domain adaptation. Then, labeled target domain data is required. Domain adaptation methods based on sample generation need to synthesize labeled target domain samples, but the quality of synthesized samples may be affected by many factors, such as the difference in sample distribution between the source and target domains, the performance of the model itself, etc. The above problems make a domain adaptation system that can adapt to any data source a promising work.
[0079] The domain adaptation method based on instance prompt words in this embodiment is applicable to a device having a function of the domain adaptation method based on instance prompt words.
[0080] In this embodiment, this embodiment demonstrates the specific application of a domain adaptation method based on instance prompt words in tasks including but not limited to sentiment classification, text classification and language translation. This method enables the model to better adapt to the target domain by generating text prompt words related to each instance, thereby improving the accuracy and adaptability of various tasks:
[0081] Select a pre-trained model that is suitable for the required task. Taking the sentiment classification task as an example, select the T5 autoregressive language model and receive an input instance of the source domain of the task model, which can be a domain that the task model is not familiar with. For example, the task model is familiar with four source domains: restaurants, home furniture, electronic devices, and movies, and receives a comment from the airline service domain: "The food is cold and the seat is uncomfortable."
[0082] The method of receiving the instance may be to receive the text instance from the target domain through a user interface (such as a web form, a mobile application, an API, etc.). For example, a user enters a comment in an application: "The food is cold and the seat is uncomfortable." The specific method may be:
[0083] Web form: A user enters comments or text in a web form, and the system receives the input through an HTTP request. For example, a user submits a comment on an airline service review website: "The food was cold and the seat was uncomfortable."
[0084] Mobile application: A user enters input into a text box in a mobile application, and the application sends the input to the server through the API for processing. For example, a user submits a review in an airline app.
[0085] API interface: Receive text instances sent by other systems or applications through the API interface. For example, other applications call the sentiment analysis API and send text instances for sentiment classification.
[0086] According to the source domain of the task model (such as restaurant reviews), generate preliminary text prompt words for the input instance. Assume that the generated text prompt words are: "restaurant food seating".
[0087] Step S20, decomposing the text prompt word into a domain name and a feature word, and determining a target source domain prompt word of the instance according to the domain name and the feature word;
[0088] In this embodiment, the generated text prompt words are decomposed into domain names and feature words, for example, domain name: restaurant; feature words: food, seat.
[0089] According to the domain name and the characteristic words, the prompt words are further refined to generate the prompt words of the target source domain. The specific steps may include:
[0090] Feature scoring: Domain names and feature words are scored according to preset rules based on the mutual information method.
[0091] Feature screening: A preset number of feature words are screened out as candidate source domain prompt words based on the scores.
[0092] Prompt word generation: Combine candidate source domain prompt words to obtain the target source domain prompt word of the instance.
[0093] After calculation and analysis, the final prompt words were determined as: "The food in the restaurant is cold and the seats are uncomfortable."
[0094] Combine the target source domain prompt words with the input instance to form a new input instance: "The food in the restaurant is cold and the seats are uncomfortable. The food is cold and the seats are uncomfortable."
[0095] Among them, decomposing the prompt words is one of the key steps. Specific implementation methods may include:
[0096] Preprocessing text prompt words: preprocessing the text prompt words, including word segmentation, stop word removal and tokenization, to obtain preprocessed text.
[0097] Extract domain names and feature words: Use a predefined domain name list to select domain names from the preprocessed text. Select feature words associated with the domain name from the preprocessed text. Example: Predefined domain name list: ["restaurant", "hotel", "electronic products", "travel"], select domain name from the preprocessed text: "restaurant", extract feature words associated with the domain name: "food", "seat".
[0098] Feature selection algorithm: Use statistical and semantic-based methods to select features, such as mutual information, TF-IDF, etc. Example: Calculate the mutual information score between feature words and domain names:
[0099] Mutual information score: "food" -> 0.8, "seat" -> 0.6
[0100] Select the feature words with the highest scores as candidate feature words: "food", "seat"
[0101] Domain recognition: Identify domain names from candidate features using a predefined list of domain names or through a trained domain recognition model. Example: The domain recognition model determines that the domain name is: "restaurant"
[0102] Feature word selection: Use the feature selection algorithm to select the feature words with the highest correlation with the domain name from the remaining features. Example: Select feature words based on mutual information score: "food", "seat".
[0103] Step S30: taking the target source domain prompt word and the instance as input of the task model to obtain the target output of the task model.
[0104] In this embodiment, in the domain adaptation method based on instance cue words, combining the target source domain cue words with the input instance is one of the key steps. This step improves the adaptability and accuracy of the task model by generating a semantically consistent combined input.
[0105] Combine the target source domain prompt word with the input instance to form a new input instance. For example, combine the target source domain prompt word: "The restaurant food is cold and the seat is uncomfortable" with the input instance: "The food is cold and the seat is very uncomfortable" to form a new input instance: "The restaurant food is cold and the seat is uncomfortable. The food is cold and the seat is very uncomfortable".
[0106] Specific implementations may be:
[0107] Text concatenation method: put the target source domain prompt word before the input instance to ensure that the prompt word can provide prior information for the model. The combined input instance after concatenation is: "The food in the restaurant is cold and the seat is uncomfortable. The food is cold and the seat is very uncomfortable."
[0108] Or context-preserving method: the combined input instance retains the context information of the original input instance, while introducing the prompt words of the source domain, which helps the model understand the semantic background of the input instance.
[0109] For example, suppose the input instance is: "The food is cold and the seat is uncomfortable", and the generated target source domain prompt words are: "Restaurant food is cold and the seat is uncomfortable". Combine the input into a new input instance: "Restaurant food is cold and the seat is uncomfortable. The food is cold and the seat is uncomfortable".
[0110] The present invention relates to a domain adaptation method based on instance prompt words, which selects a task model, receives an input instance, and generates text prompt words related to the instance according to the source domain of the task model. The text prompt words are decomposed into domain names and feature words, and the target source domain prompt words of the instance are determined according to the domain names and feature words. The target source domain prompt words and the instance are then associated as new inputs of the task model to obtain the target output of the task model. The present invention effectively improves the adaptability and performance of the model on data in different fields by generating unique text prompt words and combining an autoregressive prompt learning algorithm. The system can adapt to any target field without strict requirements on the data in the target domain. By generating unique prompt words, the prediction accuracy of the model in unknown domains can be effectively improved.
[0111] Furthermore, a second embodiment of the domain adaptation method based on instance prompt words of the present invention is proposed. In the above step S20, the step of determining the target source domain prompt word of the instance according to the domain name and the feature word includes:
[0112] Step a1: Based on the mutual information method, the domain name and the feature word are scored according to preset rules, and the target source domain prompt word of the instance is extracted from the domain name and the feature word according to the score.
[0113] In this embodiment, the domain name and the feature words are scored, and then the target source domain prompt words of the instance are extracted from the domain name and the feature words according to the scores. The specific steps may include:
[0114] Calculate mutual information: Calculate the mutual information value between each feature word and the domain name. Mutual information is used to measure the mutual dependence between two variables. The higher the value, the stronger the correlation between the two variables.
[0115] Scoring: The mutual information value between each feature word and the domain name is used as the basis for scoring. For example, suppose the mutual information value of "food" is 0.8 and the mutual information value of "seat" is 0.6.
[0116] Filter feature words: Filter the feature words with the highest or top scores as candidate source domain prompt words based on the scoring results. In this example, both "food" and "seat" are selected because they have high mutual information values.
[0117] The method of calculating mutual information may include: collecting a large number of data samples from the source domain and the target domain, and preprocessing the data, including word segmentation, stop word removal, and tokenization. Calculating the frequency of occurrence of feature words in the source domain and the target domain, and calculating the frequency of occurrence of domain names in the source domain and the target domain. Calculating mutual information: For each feature word t and domain name d, calculate the mutual information value I(t; d), the formula is as follows:
[0118]
[0119] Among them, P(t,d) is the joint probability of feature word t and domain name d, and P(t) and P(d) are the marginal probabilities of feature words and domain names respectively.
[0120] Through mutual information calculation, each feature word gets a mutual information value as its score.
[0121] Generate target source domain prompt words based on the filtered feature words. For example: domain name: "restaurant", highest-scoring feature words: "food", "seat", target source domain prompt words: "restaurant food is cold and seats are uncomfortable".
[0122] In addition to the mutual information method, scoring can also be performed in the following ways:
[0123] TF-IDF method, calculate TF-IDF: calculate the term frequency (TF) and inverse document frequency (IDF) of each feature word in the source domain document. Example: The TF-IDF value of "food" is 0.7, and the TF-IDF value of "seat" is 0.5.
[0124] Point Mutual Information (PMI) method, calculate PMI: calculate the point mutual information between each feature word and the domain name. Example: the PMI value of "food" is 0.9, and the PMI value of "seat" is 0.7.
[0125] Information gain method, calculate information gain: calculate the information gain of each feature word for the classification result. Example: The information gain value of "food" is 0.75, and the information gain value of "seat" is 0.65.
[0126] Through the above steps, the mutual information-based method in this embodiment can effectively score domain names and feature words, and extract target source domain prompt words from them, thereby providing more accurate input for the task model and improving the adaptability and accuracy of the model in different fields.
[0127] Furthermore, a third embodiment of the domain adaptation method based on instance prompt words of the present invention is proposed. In the above step a1, the step of extracting the source domain prompt words of the instance from the domain name and the feature words according to the scores includes:
[0128] Step b1, selecting a preset number of feature words as candidate source domain prompt words according to the scores;
[0129] Step b2: combining the candidate source domain prompt words to obtain the target source domain prompt word of the instance.
[0130] In this embodiment, a preset number of feature words are set as candidate source domain prompt words. For example, the first two feature words with the highest scores are selected. According to the scoring results, the feature words with the highest scores are selected from all feature words.
[0131] For example, the comprehensive score of each feature word is calculated, and the comprehensive score can be obtained by weighted average or simple summation. Example: The comprehensive score of "food" = mutual information value (0.8) + TF-IDF value (0.7) + PMI value (0.9) + information gain value (0.75) = 3.15; the comprehensive score of "seat" = mutual information value (0.6) + TF-IDF value (0.5) + PMI value (0.7) + information gain value (0.65) = 2.45; the comprehensive score of "service" = mutual information value (0.5) + TF-IDF value (0.6) + PMI value (0.6) + information gain value (0.55) = 2.25.
[0132] Sorting and filtering: Sort the feature words in descending order according to their comprehensive scores.
[0133] Select the two feature words with the highest scores from the sorted list as candidate source domain prompt words. Example: The candidate feature words are "food" and "seat".
[0134] The selected candidate source domain prompt words are combined to generate the target source domain prompt words. For example, the combined target source domain prompt words are: "The restaurant food is cold, the seats are uncomfortable, and the service is poor."
[0135] The specific combination can be:
[0136] Syntactic combination: Combine candidate source domain prompt words according to the syntactic rules of natural language. For example, combine "food", "seat" and "service" into a coherent prompt word.
[0137] Semantic consistency: Ensure that the combined prompt words are semantically coherent and consistent. This can be done by checking the semantic relationship between the words, adjusting the word order, or adding appropriate conjunctions.
[0138] Optimize prompt words: Optimize prompt words according to the context to ensure that they can accurately reflect the semantics of the input instance. For example, according to the content of the input instance, combine "the food is cold", "the seat is uncomfortable", and "the service is poor" into a coherent prompt word.
[0139] For example, the candidate feature words are: "food", "seat", "service", and the combined target source domain prompt words are: "the restaurant food is cold, the seats are uncomfortable, and the service is poor."
[0140] Through the above method, this embodiment can effectively comprehensively score domain names and feature words based on multiple scoring methods, and extract target source domain prompt words from them, thereby providing more accurate input for the task model and improving the adaptability and accuracy of the model in different fields.
[0141] Furthermore, a fourth embodiment of the domain adaptation method based on instance prompt words of the present invention is proposed. In the above step S20, the step of decomposing the text prompt words into domain names and feature words includes:
[0142] Step c1, preprocessing the text prompt words, wherein the preprocessing includes word segmentation, stop word removal and tokenization to obtain preprocessed text;
[0143] Step c2, extracting domain names and feature words from the preprocessed text.
[0144] In this embodiment, this embodiment shows how to preprocess the text prompt words, including word segmentation, stop word removal and tokenization, to obtain the preprocessed text. The specific steps may include:
[0145] Word Segmentation: First, define the word segmentation rules. According to the language characteristics and task requirements, define the word segmentation rules. For Chinese, dictionary matching word segmentation algorithms such as the Jieba segmentation library can be used; for English, spaces and punctuation marks can be used for word segmentation. Second, apply the word segmentation tool to break down the text prompt into independent words or phrases using an appropriate word segmentation tool. For example: Input prompt: "Restaurant food seat service", Word segmentation results: Chinese: "Restaurant", "Food", "Seat", "Service"; English (assuming the prompt is "restaurant food seat service"): "restaurant", "food", "seat", "service".
[0146] Stop Word Removal: First, define the stop word list and prepare a list of common stop words. Stop words refer to words that are considered to make no substantial contribution to the document theme during the information retrieval process, such as "de", "shi", "zai" in Chinese, and "the", "is", "at" in English. Then, remove the stop words by matching the word segmentation results with the stop word list and removing the matched stop words. For example: Word segmentation results: Chinese: "Restaurant", "Food", "Seat", "Service"; English: "restaurant", "food", "seat", "service". Stop word removal results: There are no stop words in the word segmentation results, so they remain unchanged.
[0147] Tokenization: First, define the tokenization rules. According to the task requirements, define the tokenization rules, which can include part-of-speech tagging (POS tagging) and normalization. Second, apply the tokenization tool to perform tokenization on the words using an appropriate tokenization tool, converting each word into a standard format or performing part-of-speech tagging. For example: Part-of-speech tagging: This (PN) family (M) restaurant (NN) of (DE) food (NN) is (AD) very (AD) delicious (ADJ), (PU) service (NN) is also (AD) very (AD) considerate (ADJ).
[0148] Through the above steps, this embodiment can effectively preprocess the text prompt, including word segmentation, stop word removal, and tokenization, thereby laying a foundation for subsequent domain name and feature word extraction and improving the adaptability and accuracy of the task model across different domains.
[0149] Further, a fifth embodiment of the domain adaptation method based on instance prompts of the present invention is proposed. The above step c2 includes:
[0150] Step d1, selecting the domain name from the preprocessed text using a predefined domain name list;
[0151] Step d2, selecting the feature words associated with the domain name from the preprocessed text.
[0152] In this embodiment, specific steps of extracting domain names and feature words from preprocessed Chinese text are shown, including selecting domain names using a predefined domain name list and selecting feature words associated with the domain names from the preprocessed text. The preprocessing step has been completed in the previous step, including word segmentation, stop word removal, and tokenization. Assume that the preprocessed text prompt words are: Chinese: restaurant food seat service.
[0153] First define the predefined domain name list: create a list of all possible domain names. The list is based on known fields or topics. For example: Chinese domain name list: ["restaurant", "hotel", "airline service", "medical", "education"]. How to match domain names: match the words in the preprocessed text with the predefined domain name list to find the most relevant domain name. For example, preprocessed text: Chinese: "restaurant food seating service", matching result: Chinese domain name: "restaurant".
[0154] Secondly, feature words associated with the domain name are selected from the preprocessed text, and feature word association rules need to be defined, and association rules are defined based on the relationship between the domain name and the feature words. Associations can be established through methods such as semantic similarity and co-occurrence frequency.
[0155] Finally, feature words are extracted, and feature words associated with the domain name are selected from the preprocessed text.
[0156] For example: domain name: "restaurant", preprocessed text: "restaurant food seat service", association rules: "food", "seat", "service" can all be associated with "restaurant". Final extraction result: domain name: "restaurant", feature words: "food", "seat", "service".
[0157] Through the above steps, this embodiment can effectively extract domain names and feature words from the preprocessed Chinese text, thereby laying a foundation for subsequent prompt word generation and model adaptation, and improving the adaptability and accuracy of the task model in different fields.
[0158] Furthermore, a sixth embodiment of the domain adaptation method based on instance prompt words of the present invention is proposed. In the above step S10, the step of generating text prompt words for the instance according to the source domain of the task model includes:
[0159] Step e1, analyzing the text information of the instance to extract key features and background information of the instance;
[0160] Step e2, generating a preliminary text prompt word template according to the characteristics of the source domain and the key characteristics and background information of the instance;
[0161] Step e3, optimizing the text prompt word template to obtain the text prompt word.
[0162] In this embodiment, specific steps are demonstrated on how to generate text prompt words for an instance based on the source domain of the task model, including analyzing the text information of the instance, extracting key features and background information, generating a preliminary text prompt word template, and optimizing the text prompt word template to obtain the final text prompt word.
[0163] First, analyze the text information of the instance. The steps may include:
[0164] Text preprocessing: Preprocess the instance text, including word segmentation, stop word removal, and tokenization.
[0165] Extract key features: Identify and extract key features in the example text. These features may include nouns, adjectives, verbs, etc., which can represent the main content of the example.
[0166] Extract background information: Extract background information from instance text, such as time, place, and people, to help build context.
[0167] For example: input instance: "the food is cold and the seat is very uncomfortable"; word segmentation results: "food", "is", "cold", "and", "seat", "sitting", "very", "uncomfortable"; stop word removal results: "food", "cold", "seat", "uncomfortable"; key features: "food", "seat"; background information: "cold", "uncomfortable".
[0168] Then, a preliminary text prompt word template is generated, and the steps may include:
[0169] Generate templates based on source domain features: Generate preliminary text prompt word templates based on the source domain features of the task model, combined with the key features and background information of the instance.
[0170] Template construction: Use source domain feature words and instance feature words to combine to form a text prompt word template.
[0171] For example: source domain features: restaurant; instance features: "food", "seat"; background information: "cold", "uncomfortable"; preliminary template: "the food in the restaurant is cold and the seats are uncomfortable".
[0172] Finally, optimize the text prompt word template. The steps may include:
[0173] Optimization rules: Optimize the preliminary text prompt word template according to the task requirements and target domain characteristics. The optimization rules can include semantic consistency, context relevance, readability, etc.
[0174] Semantic optimization: Ensure that the words in the template are semantically coherent and consistent. For example, by adding conjunctions or adjusting word order, the template can be made more fluent like natural language.
[0175] Context optimization: Based on the background information of the instance, the template is context-optimized to make it more accurately reflect the context of the instance.
[0176] For example: preliminary template: "The food in the restaurant is cold and the seats are uncomfortable"; semantic optimization: "The food in the restaurant is cold, and the seats are uncomfortable"; contextual optimization: "The food in the restaurant is cold, and the seats are uncomfortable, and the overall experience is not good."
[0177] Through the above steps, this embodiment can generate text prompt words for instances according to the source domain of the task model, effectively improving the adaptability and accuracy of the model in different fields.
[0178] Furthermore, a seventh embodiment of the domain adaptation method based on instance prompt words of the present invention is proposed. In the above step S10, the step of generating text prompt words for the instance according to the source domain of the task model includes:
[0179] Step f1, encoding the instance to generate a high-dimensional feature representation, and converting the high-dimensional feature representation into a feature representation of the source domain through a decoder;
[0180] Step f2: generating text prompt words of the instance according to the feature representation of the source domain.
[0181] In this embodiment, it is demonstrated how to generate text prompt words for instances according to the source domain of the task model through encoding and decoding, including encoding the instances to generate high-dimensional feature representations, converting the high-dimensional feature representations into source domain feature representations through a decoder, and generating text prompt words for the instances according to the source domain feature representations.
[0182] First, encode the instance, including instance preprocessing: preprocess the instance text, including word segmentation, stop word removal, and tokenization. Then generate a high-dimensional feature representation, and use the encoder to convert the preprocessed instance text into a high-dimensional feature representation. Example: The high-dimensional feature representation generated by the encoder is a vector representation, such as 0.1, 0.3, 0.5, 0.7.
[0183] Second, decode the high-dimensional feature representation and convert it into the source domain feature representation: Use the decoder to convert the high-dimensional feature representation generated by the encoder into the source domain feature representation. For example, the decoder converts the high-dimensional feature representation into the source domain feature representation, such as "restaurant food seat service".
[0184] Finally, the text prompt words of the instance are generated based on the source domain feature representation, combined with the instance features: Based on the decoded source domain feature representation, the text prompt words are generated by combining the key features and background information of the instance. For example: the source domain feature representation is "restaurant food seat service", and the instance features are "food", "cold", "seat", and "uncomfortable".
[0185] Generate preliminary text prompt words: Generate preliminary text prompt words using source domain features and instance features. Example: The preliminary text prompt words are "The food in the restaurant is cold and the seats are uncomfortable."
[0186] Optimize text prompt words: Optimize the preliminary text prompt words based on semantic consistency, contextual relevance, etc. Example: The optimized text prompt words are "The food in the restaurant is cold and the seats are uncomfortable."
[0187] Through the above steps, this embodiment can generate text prompt words for instances according to the source domain of the task model, effectively improving the adaptability and accuracy of the model in different fields.
[0188] The present invention also provides a domain adaptation device based on example prompt words.
[0189] Reference Figure 3 , Figure 3 This is a functional module diagram of a first embodiment of a domain adaptation device based on example prompt words of the present invention. The domain adaptation device based on example prompt words comprises:
[0190] A receiving module selects a task model, receives an instance of the input task model, and generates a text prompt word for the instance according to a source domain of the task model;
[0191] A generating module, decomposing the text prompt word into a domain name and a feature word, and determining a target source domain prompt word of the instance according to the domain name and the feature word;
[0192] The output module uses the target source domain prompt word and the instance as input of the task model to obtain the target output of the task model.
[0193] In addition, an embodiment of the present invention further provides a storage medium.
[0194] The storage medium stores a domain adaptation program based on instance prompt words. When the domain adaptation program based on instance prompt words is executed by the processor, the steps of the domain adaptation method based on instance prompt words as described above are implemented.
[0195] The specific implementation of the storage medium of the present invention is basically the same as the above-mentioned embodiments of the domain adaptation method based on the example prompt word, and will not be repeated here.
[0196] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims. All equivalent structures or equivalent process changes made using the contents of the specification and drawings of the present invention, or directly or indirectly used in other related technical fields, are protected by the present invention.
[0197] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to the multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of boxes in the block diagram and / or the flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.
[0198] In addition, each functional module in each embodiment of the present application can be integrated together to form an independent part, or each module can exist separately, or two or more modules can be integrated to form an independent part. If the function is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk.
[0199] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0200] The above are only various implementations of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A domain adaptation method based on instance prompt words, characterized in that: The domain adaptation method based on instance prompt words comprises the following steps: Selecting a task model, receiving an instance of the input task model, and generating a text prompt word for the instance according to a source domain of the task model; Decomposing the text prompt word into a domain name and a feature word, and determining a target source domain prompt word of the instance according to the domain name and the feature word; The target source domain prompt word and the instance are used as inputs of the task model to obtain a target output of the task model.
2. The domain adaptation method based on instance prompt words according to claim 1, characterized in that: The step of determining the target source domain prompt word of the instance according to the domain name and the feature word comprises: Based on the mutual information method, the domain name and the feature word are scored according to preset rules, and the target source domain prompt word of the instance is extracted from the domain name and the feature word according to the score.
3. The domain adaptation method based on instance prompt words according to claim 2, characterized in that: The step of extracting the source domain prompt word of the instance from the domain name and the feature word according to the score comprises: According to the scores, a preset number of feature words are selected as candidate source domain prompt words; The candidate source domain prompt words are combined to obtain the target source domain prompt word of the instance.
4. The domain adaptation method based on instance prompt words according to claim 1, characterized in that: The step of decomposing the text prompt word into a domain name and a feature word comprises: Preprocessing the text prompt words, wherein the preprocessing includes word segmentation and tokenization to remove stop words, and obtaining preprocessed text; Domain names and characteristic words are extracted from the preprocessed text.
5. The domain adaptation method based on instance prompt words according to claim 4, characterized in that: The step of extracting domain names and feature words from the preprocessed text comprises: selecting the domain name from the preprocessed text using a predefined list of domain names; The characteristic word associated with the domain name is selected from the preprocessed text.
6. The domain adaptation method based on instance prompt words according to claim 1, characterized in that: The step of generating text prompt words for the instance according to the source domain of the task model comprises: Analyze the text information of the instance and extract key features and background information of the instance; Generate a preliminary text prompt word template according to the characteristics of the source domain and the key characteristics and background information of the instance; The text prompt word template is optimized to obtain the text prompt word.
7. The domain adaptation method based on instance prompt words according to claim 1, characterized in that: The step of generating text prompt words for the instance according to the source domain of the task model comprises: Encoding the instance to generate a high-dimensional feature representation, and converting the high-dimensional feature representation into a feature representation of the source domain through a decoder; A text prompt word of the instance is generated according to the feature representation of the source domain.
8. A domain adaptation device based on instance prompt words, characterized in that: The domain adaptation device based on the example prompt word includes: A receiving module selects a task model, receives an instance of the input task model, and generates a text prompt word for the instance according to a source domain of the task model; A generating module, decomposing the text prompt word into a domain name and a feature word, and determining a target source domain prompt word of the instance according to the domain name and the feature word; The output module uses the target source domain prompt word and the instance as input of the task model to obtain the target output of the task model.
9. A domain adaptation device based on instance prompt words, characterized in that: The instance-based domain adaptation device includes a memory, a processor, and an instance-based domain adaptation program stored in the memory and executable on the processor. When the instance-based domain adaptation program is executed by the processor, the steps of the instance-based domain adaptation method according to any one of claims 1 to 7 are implemented.
10. A storage medium, characterized in that: The storage medium stores a domain adaptation program based on instance prompt words, and when the domain adaptation program based on instance prompt words is executed by the processor, the steps of the domain adaptation method based on instance prompt words according to any one of claims 1 to 7 are implemented.
