Question answering method, device and equipment for business and storage medium

Abstract

The invention discloses a business question answering method and device, equipment and a storage medium, and relates to the technical field of artificial intelligence, and the method comprises the steps: obtaining source domain and target domain data sets corresponding to new and old businesses of an electric power business hall, and extracting subsets to form a differential training set; determining a semantic association distance based on the difference between the vocabulary feature vector and the word vector, and obtaining a clustering cluster semantic association difference analysis value in combination with the similarity between the inter-cluster vocabulary semantic association distance and the text topic vector after the word is clustered; the vocabulary topic distribution conditions of different data set clustering clusters and the analysis values are fused, and the overall semantic feature values of the vth topic of the source domain and the kth topic of the target domain are calculated; determining attention weight and establishing a matrix, and generating a migrated intelligent language model by combining the two types of data sets; and answering can be obtained by inputting the questioning service text. The method can improve the accuracy of the output reply of the intelligent language model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a business question and answer method and device, equipment and storage medium. BACKGROUND

[0002] The power business hall has a large amount of business. The question and answer robot based on the intelligent language model can not only effectively alleviate the pressure of the shortage of human resources, but also improve the service experience of customers to the power business hall, and improve the work quality and efficiency of the power marketing business. However, the power business hall has various types of business, and the question and answer logic corresponding to different businesses is obviously different. In addition, as new businesses continue to expand, the intelligent language model needs to be iteratively trained. Transfer learning is the current mainstream intelligent language model training scheme. Based on the trained base model, the similarity of the question and answer of new and old businesses is combined to reuse the existing model parameters to carry out adaptive training of new businesses, thereby greatly improving the model iteration efficiency.

[0003] However, in the actual scene of the power business hall, the word frequency distribution of the question and answer text of different businesses is different, and a single question and answer interaction may span multiple business fields, which makes the semantics of the same vocabulary in different contexts have great differences, and further leads to poor accuracy of the output answer of the intelligent language model after transfer training. SUMMARY

[0004] The present application provides a business question and answer method, device, equipment and storage medium, which can improve the accuracy of the output answer of the intelligent language model.

[0005] To achieve the above purpose, the present application adopts the following technical scheme: In a first aspect, the present application provides a business question and answer method, comprising: obtaining a source domain data set and a target domain data set; extracting subsets from the source domain data set and the target domain data set respectively to jointly form a differential training set; for each subset of the differential training set, determining the semantic association distance between different vocabularies based on the difference between the feature vectors of different vocabularies and the difference between the word vectors; clustering all vocabularies in each subset of the differential training set; based on the semantic association distance between the vocabularies belonging to different clusters and the similarity of the theme vectors of the question and answer texts, determining the semantic association difference analysis value between the clusters; based on the distribution of the vocabularies in the two clusters belonging to different data sets under different themes, combining the semantic association difference analysis value, obtaining the overall semantic feature value of the vth theme of the source domain data set and the kth theme of the target domain data set in each differential training set; determine an attention weight based on the overall word semantic feature value, and establish an attention weight matrix; generate a migrated intelligent language model based on the attention weight matrix, the source domain dataset, and the target domain dataset; input a question business text into the migrated intelligent language model to obtain a reply business text for the question business text.

[0006] Optionally, the semantic association distance between different words is determined by the following method: For each subset of the differentiated training set, calculate the Euclidean distance of the feature vectors corresponding to any two words in the subset to obtain a first difference value of the any two words in the subset; Calculate the Euclidean distance of the word vectors corresponding to the any two words in the subset to obtain a second difference value between the any two words in the subset; positively fuse the first difference value and the second difference value between the any two words in the subset to determine the semantic association distance between the any two words in the subset.

[0007] Optionally, the step of determining the semantic association difference analysis value between the clustering clusters is: analyze the distribution characteristics of the topic vectors of all question and answer texts corresponding to each word in each clustering cluster to determine the topic feature vector of each word; calculate the semantic association distance and the similarity degree of the topic feature vectors between any two words belonging to different data set clustering clusters, and positively fuse the negative correlation mapping result of the similarity degree and the semantic association distance to obtain the semantic association difference value between the any two words belonging to different data set clustering clusters; take the mean value of the semantic association difference values obtained by any two words belonging to different data set clustering clusters in each differentiated training set as the semantic association difference analysis value between the different data set clustering clusters.

[0008] Optionally, the topic feature vector of each word is determined by the following method: respectively obtain the mean value of the same position elements of the topic vectors of all question and answer texts in which each word in the clustering cluster is located, and construct a vector composed of all the mean values as the topic feature vector of each word in the clustering cluster.

[0009] Optionally, the overall semantic feature value of the vth topic of the source domain dataset and the kth topic of the target domain dataset in each differentiated training set is determined by the following method: For any one clustering cluster in the differentiated training set, take the mean value of the appearance frequency of all words in the clustering cluster in each topic as a first feature value; and construct a sequence composed of all the first feature values obtained by each clustering cluster as a feature sequence. The distance feature between the corresponding feature sequences of two clusters belonging to different datasets is obtained and positively fused with the semantic association difference analysis value to obtain the topic distribution difference between two clusters belonging to different datasets in the differential training set. Based on the differences in topic distribution and the differences between the first feature values ​​under different topics, the overall semantic feature value of the v-th topic in the source domain dataset and the k-th topic in the target domain dataset in each differential training set is determined.

[0010] Optionally, the overall semantic feature value is determined by the following formula:

[0011] in, This represents the source domain dataset in the differential training set. The first subject and target domain dataset The overall semantic feature value among the topics; Represents the first data point in the source domain dataset of the differential training set. The cluster and the target domain dataset of the first cluster. Differences in topic distribution among individual clusters; This indicates the first element in the source domain dataset of the differential training set. The cluster in the th order of ... The first feature value under each topic; This represents the first element in the target domain dataset of the differential training set. The cluster in the th order of ... The first feature value under each topic; This indicates the number of clusters.

[0012] Optionally, the process for determining attention weights is as follows: For all differential training sets, the source domain datasets, the first... The first subject and target domain dataset The overall semantic feature values ​​among the topics are normalized, and the mean of all normalized results is used as the difference weight. The difference between the natural number 1 and the difference weight is calculated. The sum of the differences between any two topics is calculated. The ratio between the difference and the sum is used as the attention weight between the nth topic in the source domain dataset and the nth topic in the target domain dataset.

[0013] Secondly, this application provides a business question-and-answer device, comprising: The acquisition module is used to acquire the source domain dataset and the target domain dataset; The processing module is configured to extract subsets from the source domain data set and the target domain data set respectively, and combine the subsets to form a differential training set; for each subset of the differential training set, determine semantic correlation distances between different words based on differences between feature vectors of the different words and differences between word vectors; cluster all words in each subset of the differential training set; determine semantic correlation difference analysis values between clusters based on semantic correlation distances between words belonging to different clusters and similarity degrees of topic vectors of the question and answer texts; obtain an overall semantic feature value of a vth topic of the source domain data set and a kth topic of the target domain data set in each differential training set based on distributions of words belonging to two clusters in different topics and the semantic correlation difference analysis values; determine attention weights based on the overall semantic feature values of the words, and establish an attention weight matrix; and generate a migrated intelligent language model based on the attention weight matrix, the source domain data set, and the target domain data set. The question and answer module is configured to input a question business text into the migrated intelligent language model, and obtain a reply business text for the question business text.

[0014] In a third aspect, the present application provides a computing device including a memory and a processor. The memory stores one or more computer programs including instructions, and when the instructions are executed by the processor, the computing device performs the method of any one of the first aspect.

[0015] In a fourth aspect, the present application provides a computer readable storage medium for storing a computer program for executing the method of any one of the first aspect.

[0016] From the above technical solutions, the present application has at least the following beneficial effects: In the present application, first, by distinguishing the source domain data set and the target domain data set, the question and answer text resources of the new and old businesses are accurately connected, providing a clear data basis for transfer learning, ensuring that model iteration can rely on existing business accumulation, while focusing on the characteristics of new businesses, greatly improving training efficiency.

[0017] Secondly, based on the double difference between the word feature vector and the word vector, the semantic correlation distance is calculated, combined with the cluster analysis and the topic distribution characteristics, the semantic changes of the same word in different business contexts are captured, effectively solving the problem of semantic ambiguity of words in cross-domain question and answer, and improving the understanding accuracy of the model for business texts.

[0018] Furthermore, by quantitatively calculating the semantic correlation difference analysis value and the overall semantic feature value, the semantic difference between the new and old business topics is accurately described, providing a scientific basis for the dynamic allocation of attention weights. The construction and application of the attention weight matrix enable the model to adaptively focus on the correlation features between the new and old business, weaken irrelevant differences, strengthen effective knowledge transfer, and further optimize the model adaptation effect.

[0019] Finally, the entire method forms a complete closed loop from data processing, semantic analysis to model generation. The trained intelligent language model not only quickly adapts to the new business scenarios of the power service hall, but also responds to cross-business domain question and answer demands, greatly improving the accuracy of the reply, continuously relieving the pressure on human resources, further optimizing the customer service experience, and ensuring the quality of the power marketing business during the business expansion process.

[0020] It should be understood that the description of technical features, technical solutions, beneficial effects or similar language in this application does not imply that all features and advantages can be achieved in any single embodiment. On the contrary, it can be understood that the description of a feature or beneficial effect means that the specific technical feature, technical solution or beneficial effect is included in at least one embodiment. Therefore, the description of technical features, technical solutions or beneficial effects in this specification does not necessarily refer to the same embodiment. Further, the technical features, technical solutions and beneficial effects described in this embodiment can be combined in any appropriate manner. Those skilled in the art will understand that the embodiments can be implemented without one or more specific technical features, technical solutions or beneficial effects of a particular embodiment. In other embodiments, additional technical features and beneficial effects can be identified in specific embodiments that do not embody all embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 A flowchart of a business question and answer method provided by an embodiment of the present application; Figure 2 A flowchart of obtaining a semantic correlation difference analysis value provided by an embodiment of the present application; Figure 3 A flowchart of obtaining an overall semantic feature value provided by an embodiment of the present application; Figure 4 A schematic diagram of a business question and answer device provided by an embodiment of the present application; Figure 5 A schematic diagram of a computing device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0022] The terms "first", "second" and "third" and the like in the specification and drawings of the present application are used to distinguish different objects, not to limit a specific order.

[0023] In the embodiments of the present application, the words such as "exemplary" or "for example" are used to mean serving as an example, instance, or illustration, and should not be necessarily construed as a preference or a benefit. Rather, use of the words such as "exemplary" or "for example" is intended to present concepts in a concrete manner.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description of the application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

[0025] For the sake of clear and concise description of each of the following embodiments, a brief introduction of the related art is first given: Transfer learning is an intelligent language model training method based on a trained base model, which reuses existing model parameters to carry out new business adaptation training by utilizing the similarity of old and new business QAs, aiming to accelerate model training speed and improve model cross-scenario adaptation capability.

[0026] The topic model refers to the LDA (Latent Dirichlet Allocation) model, which is used to mine the latent topic categories in the QA text, analyze the probability of each QA text belonging to each topic, and analyze the distribution of words in different topics.

[0027] The power business hall has rich and continuously expanding business types, and the QA robot needs to realize model iteration to adapt to new business through transfer learning, but the existing scheme has the following technical bottlenecks: the model after transfer training has poor accuracy in answering cross-business domain QAs, and it is difficult to meet the service needs after the expansion of multiple business scenarios and new business.

[0028] The main causes of this problem mainly include two aspects: on the one hand, there are natural differences in the word frequency distribution of different business QAs, resulting in inherent gaps in the semantic system of new and old businesses; on the other hand, a single QA interaction may span multiple business domains, making the semantics of the same word change greatly in different business contexts, and the existing transfer learning scheme lacks accurate capture and analysis of such dynamic changes in word semantics, relying only on simple similarity matching of new and old businesses to reuse model parameters, which cannot effectively eliminate semantic ambiguity, ultimately leading to model understanding deviation and insufficient answer accuracy for new business texts.

[0029] Therefore, the application provides a question and answer method of a service, which can be executed by a processing device, which can be a terminal or a server. The terminal includes, but is not limited to, a smart phone, a tablet computer, a notebook computer, a personal digital assistant or a smart wearable device, etc. The server can be a cloud server, for example, a central server in a central cloud computing cluster or an edge server in an edge cloud computing cluster. Of course, the server can also be a server in a local data center. The local data center refers to a data center directly controlled by a user.

[0030] In view of the characteristics of the power business hall multi-service type, cross-domain question and answer and continuous expansion of new services, the existing transfer learning scheme lacks capturing of dynamic changes of vocabulary semantics, resulting in insufficient accuracy of model reply. Based on the old and new service question and answer data, through data hierarchical processing, semantic multi-dimensional analysis, difference quantization modeling and attention mechanism optimization, a complete transfer learning closed loop is constructed. First, the source domain and target domain data set are distinguished to clearly train the basis, and then the vocabulary double vector difference calculation, clustering analysis and theme distribution feature mining are used to describe the semantic changes of the vocabulary in different business contexts and the theme differences between the old and new services. Finally, based on the quantitative difference results, the attention weight is dynamically allocated to guide the model to preferentially reuse the effective associated knowledge between the old and new services and weaken the irrelevant difference interference, so as to realize the accurate adaptation of the model to the new service scene and the cross-domain question and answer demand, and finally improve the interaction performance and reply accuracy of the question and answer robot.

[0031] In order to make the technical solutions of the application clearer and easier to understand, the following describes a question and answer method of a service provided by an embodiment of the application in conjunction with the accompanying drawings. As shown in Figure 1 The figure is a flow chart of a question and answer method of a service provided by an embodiment of the application. The method comprises: S101, the processing device acquires a source domain data set and a target domain data set.

[0032] The source domain data set is a data set constructed based on the question and answer text in the stage when the power business hall has not expanded new services, which contains complete question and answer interaction records of various existing services in this stage, records the question and answer logic, vocabulary usage habits and semantic association features in the old service scene, and is the basic data support for transfer learning.

[0033] The target domain data set is a data set constructed based on the question and answer text generated in the new service scene after the power business hall successfully expands new services, which reflects the question and answer demand, exclusive terminology, logic rules and semantic expression characteristics of the new service, and is the training data for the model to adapt to the new service.

[0034] Specifically, the processing device determines a source domain data set and a target domain data set based on the question and answer texts before and after the expansion of the service; and determines a topic vector of each question and answer text and a feature vector of each vocabulary by using a topic model on the question and answer texts and their vocabularies of each data set.

[0035] In the power business hall scene, there are obvious statistical differences in the distribution of vocabularies in question and answer texts of different service types. To realize model knowledge transfer in the cross-service scene, the application first constructs data sets in the stage of no expansion of new services and the stage of expansion of new services. The specific way is: Based on the question and answer texts in the stage of no expansion of new services in the business hall, a source domain data set is established; and based on the question and answer texts after the expansion of new services, a target domain data set is constructed; wherein each data set contains 500 question and answer texts corresponding to 500 question and answer interactions, and the vocabulary table capacity is 5000.

[0036] For all question and answer texts in each data set, an LDA (Latent Dirichlet Allocation) topic model is used for modeling analysis to obtain the word vector representation of each vocabulary in the vocabulary table, and at the same time, the latent topic categories covered by all texts and the probability of each question and answer text belonging to each topic are obtained, which are used to reflect the association degree of the corresponding text to a specific business type.

[0037] To further realize the structured association of topics and vocabularies, all topics of the double-domain data set are sorted and assigned a unique serial number, and then the probability of each question and answer text belonging to different topics is arranged in order of the topic serial number to obtain the topic vector of each question and answer text.

[0038] Further, it is considered that a single question and answer interaction may involve multiple types of services, and the cross of services increases the randomness of the word frequency distribution of the vocabulary in different texts, resulting in a decrease in the accuracy of the model in transferring the knowledge obtained from the training of the past services to the question and answer scene of new services.

[0039] Therefore, for each word vector in the vocabulary table, the frequency of its appearance in each topic is counted, and all the frequencies are arranged in order of the topic serial number to construct a feature vector of each vocabulary, which reflects the association strength of a specific topic (corresponding business type) and the vocabulary, and reflects the semantic relationship change of the same vocabulary in different service scenes. It should be noted that in the model training process, for the source domain data set, a BERT pre-training language model is used to complete the intelligent language model training for the business hall past services, and the optimization algorithm is an SGD (Stochastic Gradient Descent) optimization algorithm. The specific model training process is known to those skilled in the art and will not be described here. For convenience of description, the source domain data set is denoted as the source domain, and the target domain data set is denoted as the target domain.

[0040] The purpose of this step is to provide accurate basic data for subsequent transfer learning, by distinguishing between old and new business data, ensuring that the model can rely on existing business knowledge while focusing on new business characteristics for targeted training, laying a data foundation for subsequent semantic analysis, difference modeling and model optimization.

[0041] S102, the processing device extracts subsets from the source domain data set and the target domain data set respectively, which together constitute a differentiated training set; for each subset of the differentiated training set, the semantic association distance between different words is determined based on the difference between the feature vectors of different words and the difference between the word vectors.

[0042] The subset is a part of the question and answer text data set extracted from the source domain data set or the target domain data set, and each subset corresponds to a specific business type of question and answer interaction mode, which can focus on reflecting the semantic characteristics of a certain type of business and avoid the redundancy interference of full data.

[0043] The differentiated training set is a training data set formed by randomly combining a subset of the source domain data set and a subset of the target domain data set, and its role is to provide targeted training materials for the model to capture cross-business semantic differences and achieve knowledge transfer through direct comparison of old and new business data.

[0044] The feature vector is a quantitative representation of the frequency of a word appearing in different business topics, which is constructed by counting the frequency of each word appearing in each topic and arranging it in topic order, and can reflect the association strength of the word with different business topics.

[0045] The word vector is a vector representation obtained by mapping the word to a low-dimensional dense vector space, which can objectively reflect the semantic attributes of the word itself and the inherent semantic association between words, and is a basic tool for depicting word semantics in natural language processing.

[0046] The semantic association distance is an index that quantifies the semantic relevance between different words, which is calculated by combining the feature vector difference and the word vector difference. The greater the distance, the weaker the semantic association between the words, and vice versa.

[0047] To solve the problem of large overall semantic difference between different topics before and after the expansion of the power business hall, Bagging sampling method is used in this application to process each data set, and the specific data processing method is: Question and answer text data in the source domain and the target domain are extracted to form subsets, each of which corresponds to a different business type of question and answer interaction mode. The number of subsets in the source domain and the target domain is 10, and each subset contains 50 question and answer text data corresponding to 50 question and answer interactions.

[0048] Further, in order to compare the text differences between the new business not expanded and the new business expanded, the application constructs a differentiated training set suitable for intelligent language model transfer learning in the new business scenario by combining a randomly selected subset from the source domain subset and a randomly selected subset from the target domain subset. In this embodiment, the number of differentiated training sets is 100, and the implementer can determine it according to the actual situation.

[0049] Further, in the actual question and answer interaction in the power business hall, the semantic correlation under different business types has obvious differences, which makes it difficult to effectively transfer the knowledge obtained by the source domain model training to the target domain new business scenario, directly affecting the construction effect of the intelligent interaction model. Specifically, in the model training process for new business, since the semantic correlation differences of words between different businesses are not considered, the semantic changes of the same word in the new business scenario cannot be accurately identified, which causes information deviation in the knowledge transfer process and reduces the response accuracy of the model in the target domain new business question and answer interaction, making it difficult to meet the intelligent service demand in the new business scenario.

[0050] Based on the above analysis, the text differences in different subsets before and after the expansion of the power business hall are analyzed. Compared with the traditional analysis based on the correlation of the vector between the words, the application analyzes the semantic correlation features of the words in the subset by combining the frequency differences of the words in different business types before and after the expansion of the business in different themes.

[0051] For the source domain subset and the target domain subset in each differentiated training set, the processing device analyzes the feature vector difference (reflecting the difference in the frequency of the word in the new and old business themes) and the word vector difference (reflecting the inherent semantic difference of the word) of all the words in the subset, and determines the semantic correlation distance between any two words by positively fusing the two types of differences.

[0052] The semantic correlation distance between different words is determined by the following method: For each subset of the differentiated training set, the processing device calculates the Euclidean distance of the feature vectors corresponding to any two words in the subset, to obtain the first difference value of any two words in the subset; calculates the Euclidean distance of the word vectors corresponding to the any two words in the subset, to obtain the second difference value between the any two words in the subset; positively fuses the first difference value and the second difference value between the any two words in the subset to determine the semantic correlation distance between the any two words in the subset.

[0053] The Euclidean distance is a commonly used index to measure the straight-line distance between two vectors in a high-dimensional space. The calculation method is to square the sum of the difference values of the corresponding elements of the two vectors, and the larger the difference value, the larger the Euclidean distance, which can intuitively reflect the quantitative difference between the vectors.

[0054] The first difference value is a quantitative result obtained by calculating the Euclidean distance of the two lexical feature vectors, reflecting the difference in the frequency of the lexical in different business themes. The greater the distance, the more obvious the difference in the association of the lexical with each business theme.

[0055] The second difference value is a quantitative result obtained by calculating the Euclidean distance of the two lexical word vectors, reflecting the difference in the inherent semantic properties of the lexical. The greater the distance, the weaker the semantic association of the lexical.

[0056] Positive fusion refers to combining two or more related indicators according to certain logic (such as addition, multiplication, etc.), so that the combined result can comprehensively reflect the information of each indicator. Here, it is used to integrate the differences of the lexical in the two dimensions of business theme association and inherent semantics.

[0057] First, for each subset in the differentiated training set (including the source domain subset and the target domain subset), the processing device first focuses on the business theme association difference of the lexical, calculates the Euclidean distance of any two lexical feature vectors, and obtains the first difference value. This value can reflect the difference in the frequency of the lexical in different business themes. For example, if the same lexical appears frequently in the old business theme and infrequently in the new business theme, the first difference value of the lexical with other lexicals will increase significantly.

[0058] Next, the processing device focuses on the inherent semantic difference of the lexical, calculates the Euclidean distance of the above-mentioned any two lexical word vectors, and obtains the second difference value. This value can objectively reflect the semantic association of the lexical itself. For example, the Euclidean distance of the word vectors of electricity payment and fault repair is large, and the second difference value will increase accordingly, reflecting the difference in semantics.

[0059] Finally, the processing device integrates the first difference value and the second difference value into a semantic association distance through positive fusion, such as multiplication, weighted summation, etc. This integration process realizes the dual consideration of business scenario adaptability and lexical core semantics, so that the final semantic association distance can not only reflect the adaptation difference of the lexical in a specific business scenario, but also reflect the semantic association attribute of the lexical itself, providing a quantitative basis for subsequent capture of lexical semantic changes in cross-business context.

[0060] S103, the processing device clusters all lexicals in each subset of the differentiated training set; based on the semantic association distance between lexicals belonging to different clusters and the similarity of the theme vectors of the question and answer texts, a semantic association difference analysis value between the clusters is determined.

[0061] Clustering is an unsupervised learning method that groups lexicals with strong semantic association (close distance) into the same group to form a cluster, with the purpose of mining the inherent semantic grouping characteristics of lexicals and simplifying subsequent semantic difference analysis.

[0062] The clustering cluster is a grouping of words formed after the clustering operation, and the words in each cluster have strong semantic relevance, and the semantic difference between different clusters is relatively significant, which can intuitively reflect the core semantic set in a certain business scenario.

[0063] The theme vector is a quantitative representation of the theme attribute of the question and answer text, which is composed of the probability of each question and answer text belonging to each business theme in the theme order, and can reflect the association degree of the text with different business themes.

[0064] The similarity of the theme vector is an index for measuring the degree of fit of two question and answer texts in the theme distribution, which is calculated by cosine similarity and the like, and the higher the similarity, the more similar the business themes involved in the text, and vice versa. The theme difference is greater.

[0065] The semantic association difference analysis value is an index for quantifying the overall semantic difference between the clustering clusters of different data sets, which is obtained by integrating the semantic association distance of the words in the cluster and the similarity of the theme vector, and can comprehensively reflect the difference degree of the semantic set in two business scenarios.

[0066] First, the processing device, for each subset in the differentiated training set, according to the previously calculated semantic association distance between words, classifies the words with strong semantic association in each subset into a category, forming multiple clustering clusters. This step can group the scattered words according to semantic association, and convert the complex full-text semantic analysis into more efficient difference analysis between clustering clusters, reducing the complexity of subsequent calculations.

[0067] Then, the processing device extracts the theme vectors of the question and answer texts in which the words in the clustering clusters of different data sets (i.e. the clustering clusters of the source domain subset and the clustering clusters of the target domain subset) are located, calculates the similarity of these theme vectors, and then combines the previously obtained semantic association distance between the words in different clustering clusters. Through specific logic, the two types of information are integrated, and finally the semantic association difference analysis value between the clustering clusters is obtained.

[0068] This step not only considers the semantic association difference of the words themselves, but also takes into account the business theme difference of the texts in which the words are located, and realizes the comprehensive quantification of the difference between the semantic sets of new and old businesses.

[0069] Figure 2 A flowchart for obtaining a semantic association difference analysis value provided by an embodiment of the present application, the step of determining the semantic association difference analysis value between the clustering clusters is: S201, the processing device analyzes the distribution characteristics of the theme vectors of all question and answer texts corresponding to each word in each clustering cluster, and determines the theme feature vector of each word.

[0070] The distribution feature refers to the overall performance law of all question and answer text topic vectors corresponding to a certain vocabulary in a clustering cluster, is embodied as the concentration trend of each dimension element of the topic vector (i.e., the probability corresponding to different business topics), and can reflect the general correlation strength of the vocabulary in each business topic.

[0071] The topic feature vector is a summary vector generated based on all question and answer text topic vectors corresponding to a vocabulary, is a comprehensive quantitative representation of the correlation strength of the vocabulary in different business topics, and can reflect the topic tendency of the vocabulary in a specific business scenario.

[0072] The topic feature vector of each vocabulary is determined in the following manner: The processing device respectively acquires the mean value of the same position elements of the topic vectors of all question and answer texts in which each vocabulary in the clustering cluster is located, and constructs a vector composed of all mean values as the topic feature vector of each vocabulary in the clustering cluster.

[0073] The same position elements of the topic vector are probability values corresponding to the same business topic in the topic vector, each position corresponds to a fixed business topic one by one, for example, the 3rd element of the topic vector always represents the probability of the electricity payment theme, and the position consistency ensures the comparability of the topic features of different texts.

[0074] First, the processing device collects, for each vocabulary in each clustering cluster, the topic vectors corresponding to all question and answer texts in which the vocabulary appears, to ensure that all use scenarios of the vocabulary in the clustering cluster are covered. Then, the mean values of the same position elements of all topic vectors are calculated in the order of the positions of the topic vectors (i.e., in the fixed order of business topics), for example, the average value of the probability values of the vocabulary in the position for the fault repair theme is calculated, and the same is applied to all topic positions.

[0075] Finally, the mean values corresponding to all topic positions are arranged in the order of the positions of the original topic vectors in turn, and the vector formed is the topic feature vector of the vocabulary. This process eliminates the influence of accidental factors of a single question and answer text through statistical summarization, refines the stable correlation characteristics of the vocabulary in each business topic, and enables the topic feature vector to objectively and comprehensively reflect the business topic tendency of the vocabulary.

[0076] S202, the processing device calculates the semantic association distance between any two vocabularies belonging to different data set clustering clusters and the similarity degree of the topic feature vectors, and takes the result of the positive fusion of the negative correlation mapping result of the similarity degree and the semantic association distance as the semantic association difference value between any two vocabularies belonging to different data set clustering clusters.

[0077] The different data set clustering clusters are respectively clustering clusters from a source domain data set (old business) and a target domain data set (new business), which respectively bear core semantic sets in new and old business scenarios and are objects of cross-business semantic difference analysis.

[0078] The similarity degree of the topic feature vector is an index for measuring the matching degree of the two word topics, and the higher the similarity degree is, the more similar the association strength distribution of the two words in each business topic is, and vice versa, the greater the topic difference is.

[0079] The negative correlation mapping is to convert the similarity degree of the topic feature vector into a quantitative value consistent with the semantic difference trend, that is, the higher the similarity degree is, the smaller the mapping result is; the lower the similarity degree is, the larger the mapping result is, so as to ensure that the index can reflect the difference in the same direction with the semantic association distance.

[0080] The semantic association difference value is an index for comprehensively quantifying the semantic difference of two words in different data set clustering clusters, which integrates the semantic association distance of the words themselves and the topic difference, and can comprehensively reflect the semantic gap degree of the words in the cross-business scenario.

[0081] Firstly, the processing device focuses on any two words in the source domain clustering cluster and the target domain clustering cluster, respectively retrieves the semantic association distance (reflecting the difference in the inherent semantics of the words) calculated before and the similarity degree of the topic feature vectors (reflecting the difference in the business topic tendency of the words) newly calculated. Then, the similarity degree of the topic feature vector is negatively correlated, for example, when the similarity degree is 0.8 (high matching degree), the mapping is 0.2; when the similarity degree is 0.3 (low matching degree), the mapping is 0.7, so that the index can directly reflect the size of the topic difference. Finally, by means of positive fusion (such as addition, multiplication, etc.), the negatively correlated mapping result and the semantic association distance are integrated, and the obtained value is the semantic association difference value of the two cross-data set clustering cluster words. The calculation expression of the semantic association difference value is:

[0082] Among them, represents the semantic association difference value between the word vector belonging to the i-th clustering cluster of the source domain data set and the word vector belonging to the j-th clustering cluster of the target domain data set in the difference training set; represents the semantic association difference value between the word vector belonging to the i-th clustering cluster of the source domain data set and the word vector belonging to the j-th clustering cluster of the target domain data set in the difference training set; represents the i-th word vector in the i-th clustering cluster of the source domain data set in the difference training set; represents the i-th word vector in the i-th clustering cluster of the source domain data set in the difference training set; represents the i-th word vector in the i-th clustering cluster of the target domain data set; represents the i-th word vector in the i-th clustering cluster of the target domain data set; represents represents represents represents and a semantic association distance between a topic feature vector corresponding to the th word vector in the th cluster of the source domain data set in the th cluster of the target domain data set, a topic feature vector corresponding to the th word vector in the th cluster of the source domain data set in the th cluster of the target domain data set, a cosine similarity between an exponential function with a natural constant as a base.

[0083] The process realizes comprehensive quantification of semantic differences across business vocabularies, taking into account both semantic attribute differences of the vocabularies themselves and theme tendency differences in new and old business scenarios.

[0084] S203, the processing device takes the average of semantic association difference values obtained by any two vocabularies belonging to different data set clusters in each difference training set as a semantic association difference analysis value between different data set clusters.

[0085] First, for a single difference training set, the processing device first selects all cross-data set cluster vocabulary pairs, that is, each vocabulary in a cluster of the source domain subset is combined with each vocabulary in a cluster of the target domain subset to form all vocabulary pairs to be analyzed. Then, the previously calculated semantic association difference values of each cross-data set vocabulary pair are retrieved, which reflect the comprehensive differences of each pair of vocabularies in inherent semantics and theme tendency. Finally, the processing device performs arithmetic average calculation on all these semantic association difference values, and the average value obtained is the semantic association difference analysis value between the two different data set clusters in the difference training set.

[0086] The purpose of this step is to rise from individual vocabulary differences to overall cluster differences, eliminate the influence of accidental differences of individual vocabulary pairs through statistical summary, and depict the overall difference level of the core semantic set of new and old businesses, providing cluster difference quantification basis for subsequent calculation of theme distribution difference and overall semantic feature value.

[0087] S104, the processing device obtains an overall semantic feature value of the vth theme of the source domain data set and the kth theme of the target domain data set in each difference training set based on the distribution of vocabularies in different themes in the two clusters belonging to different data sets, and in combination with the semantic association difference analysis value.

[0088] The distribution of the words in different themes refers to the frequency distribution characteristics of all the words in the clustering cluster in each business theme, which is intuitively embodied by the average frequency of each word in different themes (i.e., the first characteristic value) and the characteristic sequence, and reflects the association strength distribution of the clustering cluster and each business theme.

[0089] The overall semantic characteristic value is an index for quantifying the overall semantic difference between the vth theme of the source domain and the kth theme of the target domain in the differentiated training set, integrates the theme distribution difference between the clustering clusters and the characteristic value difference within the theme, and can describe the semantic gap degree between the specific themes of the new and old businesses.

[0090] Figure 3 An overall semantic characteristic value acquisition flowchart is provided for the embodiments of the present application, and the overall semantic characteristic value of the vth theme of the source domain dataset and the kth theme of the target domain dataset in each differentiated training set is determined by the following method: S301, for any one clustering cluster in the differentiated training set, the processing device takes the average of the frequency of all the words in each theme in the clustering cluster as the first characteristic value; and takes the sequence formed by all the first characteristic values obtained by each clustering cluster as the characteristic sequence.

[0091] The first characteristic value is a statistical summary value of the frequency of all the words in a single clustering cluster under a certain specific business theme, which is obtained by arithmetic mean calculation and can reflect the association strength of the clustering cluster as a whole with the theme.

[0092] The characteristic sequence is a sequence formed by arranging the first characteristic values of a single clustering cluster under all business themes in a fixed order of the themes, which is a complete quantitative representation of the theme association strength distribution of the clustering cluster.

[0093] First, for any one clustering cluster (clustering cluster in the source domain subset or the target domain subset) in the differentiated training set, the processing device first determines all the preset business themes (such as electricity payment, fault repair, business handling, etc.) and the fixed order. Then, for each business theme, the frequency of all the words in the clustering cluster under the theme is counted, and the arithmetic mean of these frequencies is calculated. The mean value is the first characteristic value of the clustering cluster under the theme, for example, the frequencies of 10 words in the clustering cluster under the electricity payment theme are 0.6, 0.5, 0.7, …, and the mean value 0.6 is the first characteristic value of the clustering cluster under the theme.

[0094] Finally, the first characteristic values corresponding to all the business themes are arranged in a fixed order of the themes to form a sequence, which is the characteristic sequence of the clustering cluster. This process realizes the structured and quantitative presentation of the theme association characteristics of the clustering cluster, eliminates the random fluctuations of the frequency of a single word, and can completely reflect the association strength distribution of the clustering cluster and each business theme.

[0095] S302, the processing device obtains the distance feature between the corresponding feature sequences of the two cluster clusters belonging to different data sets, and performs forward fusion with the semantic association difference analysis value to obtain the theme distribution difference between the two cluster clusters belonging to different data sets in the difference training set.

[0096] The distance feature is an index for quantifying the similarity of the feature sequences of the two cluster clusters, reflecting the fit degree of the cluster clusters of different data sets in the distribution of the association strength of each business theme. The greater the distance, the more obvious the theme distribution difference of the two cluster clusters.

[0097] The theme distribution difference is an index for comprehensively quantifying the theme association feature difference of the cluster clusters of different data sets (source domain and target domain cluster clusters), integrating the distribution fit degree of the feature sequence and the overall semantic difference between the clusters, and can fully reflect the degree of estrangement of the two types of cluster clusters in the theme level.

[0098] First, the processing device retrieves the generated feature sequences of the two different data set cluster clusters (one from the source domain subset and one from the target domain subset) in the difference training set. The feature sequences have completely presented the association strength distribution of each in all business themes. Then, the distance feature between the two feature sequences is calculated, which directly reflects the similarity of the two types of cluster clusters in the theme association strength distribution. For example, the source domain cluster cluster has high feature value in the electricity payment theme and low feature value in the new business handling theme, while the target domain cluster cluster is the opposite, and the distance feature of the two will significantly increase. Finally, the processing device performs forward fusion on the distance feature and the semantic association difference analysis value between the two cluster clusters obtained before, and the result is the theme distribution difference between the two different data set cluster clusters.

[0099] S303, the processing device determines the overall semantic feature value of the vth theme of the source domain data set and the kth theme of the target domain data set in each difference training set based on the theme distribution difference and the difference between the first feature values under different themes.

[0100] The calculation expression of the overall semantic feature value is:

[0101] Among them, indicates the overall semantic feature value between the vth theme of the source domain data set and the kth theme of the target domain data set in the difference training set; indicates the theme distribution difference between the vth cluster cluster of the source domain data set and the kth cluster cluster of the target domain data set in the difference training set; ​​​​​a first feature value of the cluster under a first topic; a first feature value of the cluster under a first topic; a first feature value of the cluster under a first topic; a first feature value of the cluster under a first topic; a first feature value of the cluster under a first topic; a first feature value of the cluster under a first topic;

[0102] S105, the processing device determines an attention weight based on the overall vocabulary semantic feature value, establishes an attention weight matrix, and generates a migrated intelligent language model based on the attention weight matrix, the source domain data set, and the target domain data set.

[0103] The migrated intelligent language model is the final model that integrates old and new business knowledge. By adding an attention layer in the basic model and combining the source domain and target domain data sets for training, the model has the ability to accurately understand cross-business semantics and adapt to new business scenarios.

[0104] To further improve the question and answer interaction performance of the intelligent language model in the expansion of new business scenarios, the present application adds an attention mechanism in the migration learning training process, which aims to dynamically adjust the weight according to the overall vocabulary semantic correlation of the source domain subset and the target domain subset in different differential training sets. Specifically, the attention mechanism can automatically learn the alignment information of different types of businesses before and after the expansion of new businesses in the power business hall (i.e., the source domain and the target domain), improve the accuracy of the model in extracting cross-business vocabulary semantic information, and optimize knowledge migration.

[0105] Specifically, if the overall semantic feature value of the data set under the two topics before and after the expansion of new businesses is smaller, i.e., the overall semantic correlation is stronger, the overall vocabulary semantic correlation of the corresponding business in the migration learning is stronger, and its correlation is more conducive to the model adapting to the new business of the target domain, so a larger attention weight is configured; if the overall vocabulary semantic correlation of the source domain and the target domain question and answer text in the training set is weaker, the feature has less positive support for the model in the target domain migration learning; therefore, a smaller attention weight is set to improve the accuracy and adaptability of question and answer interaction.

[0106] The determination process of the attention weight is as follows: normalize the overall semantic feature value between the vth topic of the source domain data set and the kth topic of the target domain data set of all differential training sets, take the mean of all normalized results as the difference weight; calculate the difference between the natural number 1 and the difference weight; calculate the cumulative sum of the difference between any two topics; take the ratio between the difference and the cumulative sum as the attention weight between the vth topic of the source domain data set and the kth topic of the target domain data set. normalize the overall semantic feature value between the vth topic of the source domain data set and the kth topic of the target domain data set of all differential training sets, take the mean of all normalized results as the difference weight; calculate the difference between the natural number 1 and the difference weight; calculate the cumulative sum of the difference between any two topics; take the ratio between the difference and the cumulative sum as the attention weight between the vth topic of the source domain data set and the kth topic of the target domain data set.

[0107] ​First, the overall semantic feature values ​​between the v-th topic in the source domain and the k-th topic in the target domain in all differential training sets are used as input. The softmax function is then used to obtain the normalization result. The mean of all normalized results is used as the difference weight for the corresponding overall semantic feature values, denoted as . Attention weights are determined based on difference weights, and the expression for calculating attention weights is as follows:

[0108] in, Represents the first of the source domain datasets The first topic and target domain dataset The attention weights between topics reflect the role of the business before the expansion of the new business in the transfer learning of the intelligent language model in training the corresponding new business after the expansion. Represents the first of the source domain datasets The first subject and target domain dataset The weighted difference in the overall semantic feature values ​​among the topics; Represents the first of the source domain datasets The first topic and target domain dataset The weighted differences between the themes; Indicates the number of topics.

[0109] Furthermore, based on the above calculations, the attention weights between different topics in the source and target domains were obtained. These weights were then arranged according to the topic indices in the topic vectors to obtain the attention weight matrix.

[0110] Based on the source domain dataset, the BERT model was used to obtain the intelligent language model before expanding new businesses. Furthermore, an attention layer constructed by the attention weight matrix was added to the intelligent language model to obtain the semantic association between the source domain and the target domain. The BERT model and SGD optimization algorithm were used again. When training on different training sets, the attention weight matrix was changed and the model parameters were adjusted to complete transfer learning and obtain the intelligent language model after expanding new businesses.

[0111] S106. The processing device inputs the question business text into the migrated intelligent language model and obtains the response business text for the question business text.

[0112] When a user raises a business inquiry in a power business hall scenario, the processing equipment first receives the user's inquiry text, performs basic formatting and semantic preprocessing (such as word segmentation and removal of redundant information) on the text, and ensures that the text meets the input requirements of the model.

[0113] Subsequently, the processing device inputs the preprocessed question service text into the intelligent language model that has completed the migration training. The intelligent language model has mastered the question and answer logic, vocabulary semantic association and semantic adaptation rules of cross-domain services through previous training, and can quickly and accurately understand the main demands of the question text, including identifying the type of business involved (old business, new business or cross-domain business), key information (such as business name, user demand point, etc.).

[0114] Finally, the intelligent language model generates a targeted reply service text based on the built-in business knowledge and semantic understanding results, and feeds it back to the user according to the business rules of the power service hall and the user-friendly expression logic.

[0115] The purpose of this step is to convert the trained model into actual service capability, efficiently and accurately respond to various business inquiries of users, not only alleviate the pressure of human resources, but also ensure the stability of service quality after the expansion of new business, and ultimately achieve the goal of improving customer service experience and power marketing business efficiency.

[0116] Based on the above content description, the present application has the following beneficial effects: Firstly, by distinguishing the source domain and target domain data sets, the question and answer text resources of new and old businesses are accurately connected, providing a clear data basis for transfer learning, ensuring that model iteration can rely on existing business accumulation, while focusing on the characteristics of new business, greatly improving training efficiency.

[0117] Secondly, based on the dual difference calculation of semantic association distance of vocabulary feature vector and word vector, combined with clustering analysis and theme distribution characteristics, the semantic changes of the same vocabulary in different business contexts are captured, effectively solving the problem of fuzzy vocabulary semantics in cross-domain question and answer, and improving the understanding accuracy of the model for business text.

[0118] Furthermore, through the quantitative calculation of semantic association difference analysis value and overall semantic feature value, the accurate characterization of the semantic difference between new and old business themes is realized, providing a scientific basis for the dynamic allocation of attention weights; and the construction and application of the attention weight matrix enable the model to adaptively focus on the associated features between new and old businesses, weaken irrelevant differences, strengthen effective knowledge transfer, and further optimize the model adaptation effect.

[0119] Finally, the entire method forms a complete closed loop from data processing, semantic analysis to model generation. The trained intelligent language model not only quickly adapts to the new business scenarios of the power service hall, but also responds to cross-business domain question and answer demands, greatly improving the accuracy of the reply, continuously relieving the pressure of human resources, further optimizing the customer service experience, and ensuring the quality of power marketing business during the business expansion process.

[0120] The above is combined with Figures 1 to 3The question-and-answer method for the services provided in the embodiments of this application has been described in detail. The apparatus and equipment provided in the embodiments of this application will be described below with reference to the accompanying drawings.

[0121] like Figure 4 As shown in the figure, this is a schematic diagram of a question-and-answer device for a service provided in an embodiment of this application. The device includes: Module 401 is used to acquire the source domain dataset and the target domain dataset; Processing module 402 is used to extract subsets from the source domain dataset and the target domain dataset respectively to form a differentiated training set; for each subset of the differentiated training set, the semantic association distance between different words is determined based on the differences in feature vectors and word vectors of different words; all words in each subset of the differentiated training set are clustered; the semantic association difference analysis value between clusters is determined based on the semantic association distance between words belonging to different clusters and the similarity of topic vectors of the question-and-answer texts; based on the distribution of words in two clusters belonging to different datasets under different topics, combined with the semantic association difference analysis value, the overall semantic feature value of the v-th topic in the source domain dataset and the k-th topic in the target domain dataset in each differentiated training set is obtained; based on the overall word semantic feature value, attention weights are determined and an attention weight matrix is ​​established; based on the attention weight matrix, the source domain dataset, and the target domain dataset, a transferred intelligent language model is generated; The question-and-answer module 403 is used to input the question business text into the transferred intelligent language model and obtain the answer business text in response to the question business text.

[0122] Optionally, the processing module 402 is specifically used to calculate the Euclidean distance between the feature vectors corresponding to any two words in each subset of the differential training set, and obtain the first difference value of any two words in the subset; Calculate the Euclidean distance between the corresponding word vectors of any two words in the subset to obtain the second difference value between any two words in the subset; The semantic association distance between any two words in the subset is determined by positively fusing the first difference value and the second difference value.

[0123] Optionally, the processing module 402 is specifically used to analyze the distribution characteristics of the topic vectors of all question-and-answer texts corresponding to each word in each cluster, and to determine the topic feature vector of each word; The semantic correlation distance between any two words belonging to different data set clustering clusters and the similarity degree of the topic feature vector are calculated, and a negative correlation mapping result of the similarity degree is positively fused with the semantic correlation distance, to obtain a semantic correlation difference value between any two words belonging to different data set clustering clusters. The semantic correlation difference values obtained by any two words belonging to different data set clustering clusters in each differentiated training set are averaged to obtain a semantic correlation difference analysis value between different data set clustering clusters.

[0124] Optionally, the processing module 402 is specifically configured to obtain the mean value of the same position elements of the topic vectors of all question and answer texts in which each word in the clustering cluster is located, and construct a vector composed of all the mean values as the topic feature vector of each word in the clustering cluster.

[0125] Optionally, the processing module 402 is specifically configured to, for any one clustering cluster in the differentiated training set, take the mean value of the appearance frequency of each topic of all words in the clustering cluster as a first feature value, and construct a sequence composed of all the first feature values obtained by each clustering cluster as a feature sequence. The distance feature between the feature sequences corresponding to two clustering clusters belonging to different data sets is obtained, and is positively fused with the semantic correlation difference analysis value to obtain a topic distribution difference between two clustering clusters belonging to different data sets in the differentiated training set. Based on the topic distribution difference and the difference between the first feature values under different topics, an overall semantic feature value of the vth topic of the source domain data set and the kth topic of the target domain data set in each differentiated training set is determined. The overall semantic feature value is determined by the following formula:

[0126] Wherein, represents the overall semantic feature value between the vth topic of the source domain data set and the kth topic of the target domain data set in the differentiated training set. represents the topic distribution difference between the vth clustering cluster of the source domain data set and the kth clustering cluster of the target domain data set in the differentiated training set. represents the first feature value of the vth clustering cluster in the source domain data set under the kth topic in the differentiated training set. represents the first feature value of the vth clustering cluster in the target domain data set under the kth topic in the differentiated training set. represents the number of clustering clusters. ​​​​​​​​

[0127] Optionally, the processing module 402 is specifically configured to normalize overall semantic feature values between all differentiated training set source domain data set first topic and target domain data set first topic, and take the mean of all normalized results as a difference weight; calculate the difference between the natural number 1 and the difference weight; calculate the cumulative sum of the difference between all arbitrary two topics; and take the ratio between the difference and the cumulative sum as the attention weight between the source domain data set first 𝑣 topic and the target domain data set first 𝑘 topic.

[0128] The question and answer device for business according to the embodiments of the present application can correspond to performing the methods described in the embodiments of the present application, and the above-mentioned other operations and / or functions of each module / unit of the question and answer device for business are respectively for realizing the corresponding flow of each method in the embodiments shown, and for the sake of brevity, will not be repeated here. Figures 1 to 3

[0129] The embodiments of the present application also provide a computing device. As shown in the figure, it is a schematic diagram of a computing device provided by the embodiments of the present application, which includes a bus 701, a processor 702, a communication interface 703 and a memory 704. The processor 702, the memory 704 and the communication interface 703 communicate through the bus 701. Figure 5

[0130] The bus 701 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience, only one thick line is used to represent it in the figure, but it does not mean that there is only one bus or only one type of bus. Figure 5

[0131] The processor 702 can be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0132] The communication interface 703 is used for external communication.

[0133] ​​​The memory 704 can include volatile memory, such as random access memory (RAM), and non-volatile memory, such as read-only memory (ROM), floppy disks, hard disks, or solid state drives.

[0134] The executable code stored in the memory 704 is executed by the processor 702 to perform the aforementioned question-answering method for the business.

[0135] Specifically, in the case of the embodiment shown, and Figure 4 The modules or units of the question-answering apparatus for the business described in the embodiments are implemented by software, the software or program code required to perform the functions of the modules / units in the above embodiments can be stored in the memory 704 in part or in whole. The processor 702 executes the program code corresponding to each unit stored in the memory 704 to perform the aforementioned question-answering method for the business. Figure 4 Figure 4 The computer readable storage medium can be any available medium or data storage that can be used to store data and that can be accessed by a computing device. The computer readable storage medium can be a magnetic-based, optical-based, semiconductor-based, etc. The computer readable storage medium includes instructions that are executable by a processor of the computing device to execute all or part of the processes or functions described above.

[0136] The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computing device, all or part of the processes or functions described in the embodiments of the present application are generated.

[0137] The computer instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transmitted from one website, computer or data center to another website, computer or data center through wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.) mode.

[0138] The computer instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transmitted from one website, computer or data center to another website, computer or data center through wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.) mode.

[0139] ​The computer program product is executed by a computer, and the computer executes any of the aforementioned question-answering methods of the business. The computer program product can be a software installation package, and in a case where any of the aforementioned question-answering methods of the business is needed, the computer program product can be downloaded and executed on the computer.

[0140] The descriptions of the corresponding processes or structures of the various drawings are each focused on, and parts not described in detail in a certain process or structure can be referred to the related descriptions of other processes or structures.

[0141] The above merely provides a specific implementation of the present application, but the protection scope of the present application is not limited to this. Any change or replacement within the technical scope disclosed in the present application should be covered within the protection scope of the present application.

Claims

1. A question-and-answer method for business operations, characterized in that, The method includes: Obtain the source domain dataset and the target domain dataset; Subsets are extracted from the source domain dataset and the target domain dataset respectively to form a differential training set; for each subset of the differential training set, the semantic association distance between different words is determined based on the differences in feature vectors and word vectors of different words. Cluster all words in each subset of the differential training set; determine the semantic association difference analysis value between clusters based on the semantic association distance between words belonging to different clusters and the similarity of the topic vectors of the question and answer texts in which they belong; Based on the distribution of words in two clusters belonging to different datasets under different topics, and combined with the semantic association difference analysis value, the overall semantic feature value of the vth topic in the source domain dataset and the kth topic in the target domain dataset in each differential training set is obtained; Based on the overall lexical semantic feature values, attention weights are determined and an attention weight matrix is ​​established; based on the attention weight matrix, the source domain dataset, and the target domain dataset, a transferred intelligent language model is generated. Input the question business text into the transferred intelligent language model to obtain the response business text for the question business text.

2. The method according to claim 1, characterized in that, The semantic association distance between the different words is determined in the following way: For each subset of the differential training set, calculate the Euclidean distance between the feature vectors corresponding to any two words in the subset to obtain the first difference value of any two words in the subset; Calculate the Euclidean distance between the corresponding word vectors of any two words in the subset to obtain the second difference value between any two words in the subset; The semantic association distance between any two words in the subset is determined by positively fusing the first difference value and the second difference value.

3. The method according to claim 1, characterized in that, The steps for determining the semantic association difference analysis values ​​between clusters are as follows: Analyze the distribution characteristics of the topic vectors of all question-and-answer texts corresponding to each word in each cluster, and determine the topic feature vector of each word; Calculate the semantic association distance and the similarity of topic feature vectors between any two words belonging to different dataset clusters. The result of positively fusing the negative correlation mapping result of the similarity with the semantic association distance is used as the semantic association difference value between any two words belonging to different dataset clusters. The mean of the semantic association difference values ​​obtained by any two words belonging to different dataset clusters in each differential training set is used as the semantic association difference analysis value between different dataset clusters.

4. The method according to claim 3, characterized in that, The topic feature vector for each word is determined in the following way: The mean of the elements at the same position in the topic vectors of all question-and-answer texts containing each word in the cluster is obtained, and the vector formed by all the means is used as the topic feature vector of each word in the cluster.

5. The method according to claim 1, characterized in that, The overall semantic feature value of the v-th topic in the source domain dataset and the k-th topic in the target domain dataset in each differential training set is determined in the following way: For any cluster in the differential training set, the mean frequency of all words in the cluster in each topic is taken as the first feature value; the sequence of all first feature values ​​obtained from each cluster is taken as the feature sequence. The distance feature between the corresponding feature sequences of two clusters belonging to different datasets is obtained and positively fused with the semantic association difference analysis value to obtain the topic distribution difference between two clusters belonging to different datasets in the differential training set. Based on the differences in topic distribution and the differences between the first feature values ​​under different topics, the overall semantic feature value of the v-th topic in the source domain dataset and the k-th topic in the target domain dataset in each differential training set is determined.

6. The method according to claim 5, characterized in that, The overall semantic feature value is determined by the following formula: in, This represents the source domain dataset in the differential training set. The first subject and target domain dataset The overall semantic feature value among the topics; Represents the first data point in the source domain dataset of the differential training set. The cluster and the target domain dataset of the first cluster. Differences in topic distribution among individual clusters; This indicates the first element in the source domain dataset of the differential training set. The cluster in the th order of ... The first feature value under each topic; This represents the first element in the target domain dataset of the differential training set. The cluster in the th order of ... The first feature value under each topic; This indicates the number of clusters.

7. The method according to claim 1, characterized in that, The process of determining attention weights is as follows: For all differential training sets, the source domain datasets, the first... The first subject and target domain dataset The overall semantic feature values ​​among the topics are normalized, and the mean of all normalized results is used as the difference weight; the difference between the natural number 1 and the difference weight is calculated. Calculate the sum of the differences between any two topics; The ratio between the difference and the sum is used as the attention weight between the nth topic in the source domain dataset and the nth topic in the target domain dataset.

8. A question-and-answer device for a business, characterized in that, The device includes: The acquisition module is used to acquire the source domain dataset and the target domain dataset; The processing module is used to extract subsets from the source domain dataset and the target domain dataset respectively, forming a differentiated training set. For each subset of the differentiated training set, the semantic association distance between different words is determined based on the differences in feature vectors and word vectors. All words in each subset of the differentiated training set are clustered. The semantic association difference analysis value between clusters is determined based on the semantic association distance between words belonging to different clusters and the similarity of topic vectors in the question-and-answer texts. Based on the distribution of words in two clusters belonging to different datasets under different topics, combined with the semantic association difference analysis value, the overall semantic feature value of the v-th topic in the source domain dataset and the k-th topic in the target domain dataset in each differentiated training set is obtained. Based on the overall word semantic feature value, attention weights are determined, and an attention weight matrix is ​​established. Based on the attention weight matrix, the source domain dataset, and the target domain dataset, a transferred intelligent language model is generated. The question-and-answer module is used to input question business text into the transferred intelligent language model and obtain response business text for the question business text.

9. A computing device, characterized in that, Including memory and processor; The memory stores one or more computer programs, the one or more computer programs including instructions; when the instructions are executed by the processor, the computing device performs the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program for performing the method as described in any one of claims 1 to 7.

Images