A Multi-Dimensional Category Enhanced Word Vector Fusion Method and System Applicable to Digital Humans

Abstract

This invention proposes a multi-dimensional category-enhanced word vector fusion method and system suitable for digital humans, relating to the field of user interest model construction technology. The method involves constructing a question-and-answer database and collecting user input questions. Based on the standard questions corresponding to the user input questions and the user input questions themselves, question sample pairs are constructed. Each question sample pair is then labeled sequentially to obtain a set of positive and negative samples. A gradient descent algorithm is used to calculate the similarity between the positive and negative sample sets to obtain word vector scaling parameters and category weight parameters corresponding to the positive and negative sample sets. Based on the word vector scaling parameters, category weight parameters, and the set of positive and negative samples, a first extended vector corresponding to the user input question is formed. The cosine similarity between the first extended vector and the extended vectors corresponding to all standard questions is calculated. The standard question with the highest cosine similarity is selected, and the answer corresponding to the standard question is output to the user. This invention helps improve the accuracy of question-and-answer matching.

Description

Technical Field

[0001] This invention relates to the field of user interest model construction technology, and in particular to a multi-dimensional category-enhanced word vector fusion method and system applicable to digital humans. Background Technology

[0002] Currently, due to the rapid development of science and technology and the internet, online information resources are growing rapidly, and netizens have access to increasingly richer resources. Any questions they don't understand can be answered online. However, with the ever-increasing amount of information injected into the internet, a large amount of duplicate information has also emerged. Duplicate web pages are of little value to search engines; they not only excessively consume storage resources but also make it difficult for users to obtain the information they truly need, negatively impacting the user browsing experience. Therefore, in the face of massive amounts of data, efficiently and accurately checking sentence similarity is particularly important.

[0003] Chinese patent CN114997181A discloses an intelligent question-answering method and system based on user feedback correction. The method includes the following steps: Step 1: The user inputs a question, and the text is segmented and feature words are extracted; Step 2: The feature word extraction results and the text in the question-answer database are vectorized, and the cosine similarity of the text is calculated to generate a preliminary matching result; Step 3: The user-input text vector and the user feedback database vector are input into a deep learning network model for user feedback correction; Step 4: The results of the similarity matching and user feedback correction model are combined to output the final answer text to the user. However, the above scheme cannot effectively reflect the distribution of feature words and is easily affected by dataset skew, thus affecting the calculation of question similarity. Therefore, it is necessary to provide a multi-dimensional category-enhanced word vector fusion method and system suitable for digital humans to improve the accuracy of question-answering matching. Summary of the Invention

[0004] In view of this, this invention proposes a multi-dimensional category-enhanced word vector fusion method and system suitable for digital humans. By establishing a database containing standard questions and their corresponding answers and labeling positive and negative samples, the quality of model training data is improved, thereby enhancing the accuracy of question-answer matching.

[0005] This invention provides a multi-dimensional category-enhanced word vector fusion method suitable for digital humans, the method comprising:

[0006] Construct a question-and-answer database and collect user-input questions, wherein the question-and-answer database includes a set of standard questions and the answer corresponding to each standard question in the set of standard questions;

[0007] Based on the standard question corresponding to the user input question and the user input question, construct question sample pairs, and label each question sample pair in turn to obtain a set of positive and negative samples;

[0008] The similarity of the positive and negative sample sets is calculated based on the gradient descent algorithm to obtain the word vector scaling parameters and category weight parameters corresponding to the positive and negative sample sets. Based on the word vector scaling parameters, the category weight parameters and the positive and negative sample sets, a first extended vector corresponding to the user input question is formed. The similarity includes cosine similarity and dot product similarity.

[0009] Calculate the cosine similarity between the first extended vector and the extended vectors corresponding to all standard questions, select the standard question with the highest cosine similarity, and output the answer corresponding to the standard question to the user.

[0010] Based on the above technical solutions, preferably, after constructing the question-and-answer database and collecting user input questions, the method further includes:

[0011] Extract all technical terms from the standard problem set and classify them in multiple dimensions to obtain the one-hot encoded vector corresponding to the standard problem in the standard problem set under each dimension.

[0012] Based on the above technical solution, preferably, obtaining the word vector scaling parameters and category weight parameters corresponding to the positive and negative sample sets specifically includes:

[0013] Load the positive and negative sample set, initial word vector scaling parameters, and initial class weight parameters into the machine learning model, and use the BERT model to calculate the word vectors corresponding to the user input question and the standard question in the positive and negative sample set, respectively.

[0014] Calculate the one-hot encoded vectors corresponding to the user input question and the standard question in the positive and negative sample set, and obtain the extended vector corresponding to the sampled input question based on the one-hot encoded vectors and the sampled input question;

[0015] The similarity between the user input question and the standard question in each question sample pair in the positive and negative sample sets is calculated using cosine similarity, and the total loss function is calculated based on the transition word vector scaling parameter and the transition category weight parameter.

[0016] When the total loss function is greater than the loss threshold or the maximum number of iterations is reached, the machine learning model outputs the word vector scaling parameter and the category weight parameter.

[0017] More preferably, obtaining the extended vector corresponding to the sampling input problem specifically includes:

[0018] Select a sampled input problem from any pair of problem samples in the set of positive and negative samples, wherein the sampled input problem includes a first user input problem and a first standard problem corresponding to the first user input problem;

[0019] The sampling input question is calculated based on the BERT model, and the first word vector corresponding to the sampling input question is obtained;

[0020] Based on the transition word vector scaling parameter, the transition category weight parameter, the first word vector, and the one-hot encoding vector corresponding to the sampling input problem, obtain the extended vector corresponding to the sampling input problem.

[0021] More preferably, the expression for the total loss function is:

[0022] L total =L+L reg

[0023]

[0024] in, This represents the cosine similarity between the user input question u and the standard question w in the set of positive and negative samples. This represents the extended vector of the user input question u. Let y denote the extended vector of the standard problem w, · denote the dot product, ||| denote the Euclidean norm of the vector, L denotes the contrastive loss function, N denotes the total number of samples, and y i Indicates sample label, This represents the expanded vector of the i-th user input question u. Let w represent the extension vector of the i-th standard problem w. Let L represent the cosine similarity between the i-th user input question u and the i-th standard question w, γ represent the boundary parameter controlling the upper limit of negative sample similarity, max() represent the maximum value function, and L represent the maximum value function. reg Let λ represent the regularization term function, λ represent the regularization coefficient, and q represent the regularization term function. l Let L represent the weight parameters of the l-th category, m represent the dimension corresponding to the weight parameters of the l-th category, k represent the word vector scaling parameter, and L represent the weight parameters of the l-th category. total This represents the total loss function.

[0025] More preferably, the step of selecting a standard question with the highest cosine similarity and outputting the answer corresponding to the standard question to the user specifically includes:

[0026] Calculate the first cosine similarity between the first extended vector and the extended vector corresponding to the currently selected standard problem;

[0027] Determine whether the current maximum similarity value is less than the first cosine similarity. If the current maximum similarity value is less than the first cosine similarity, then assign the first cosine similarity value to the current maximum similarity value.

[0028] Determine whether all extended vectors of the standard question have been traversed. If all extended vectors have been traversed, output the answer to the standard question corresponding to the first cosine similarity.

[0029] More preferably, the expressions for the word vector scaling parameter and the category weight parameter are as follows:

[0030]

[0031] Where, k (t+1) This represents the word vector scaling parameter after the (t+1)th iteration. Let q represent the weight parameter of the i-th class after the (t+1)-th iteration. i Let L represent the weight parameter for the i-th category, k represent the word vector scaling parameter, and L represent the weight parameter for the i-th category. total Let represent the total loss function, t represent the number of steps in the current iteration, and η represent the learning rate.

[0032] A second aspect of this application provides a multi-dimensional category-enhanced word vector fusion system suitable for digital humans. The multi-dimensional category-enhanced word vector fusion system includes a data acquisition module, a sample expansion module, and an answer output module, wherein...

[0033] The data acquisition module is used to construct a question-and-answer database and collect user-input questions. The question-and-answer database includes a set of standard questions and the answer corresponding to each standard question in the set of standard questions.

[0034] The sample expansion module is used to construct question sample pairs based on the standard question corresponding to the user input question and the user input question, and to label each question sample pair in turn to obtain a set of positive and negative samples. The similarity of the positive and negative sample sets is calculated based on the gradient descent algorithm to obtain the word vector scaling parameters and category weight parameters corresponding to the positive and negative sample sets. Based on the word vector scaling parameters, the category weight parameters and the positive and negative sample sets, a first expansion vector corresponding to the user input question is formed, wherein the similarity includes cosine similarity and dot product similarity.

[0035] The answer output module is used to calculate the cosine similarity between the first extended vector and the extended vectors corresponding to all standard questions, select the standard question with the highest cosine similarity, and output the answer corresponding to the standard question to the user.

[0036] A third aspect of this application provides an electronic device including a processor, a memory, a user interface, and a network interface, wherein the memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory.

[0037] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement steps of a multi-dimensional category-enhanced word vector fusion method suitable for digital humans.

[0038] The multi-dimensional category-enhanced word vector fusion method and system for digital humans provided by this invention have the following advantages over existing technologies:

[0039] (1) By establishing a database containing standard questions and their corresponding answers, the integrity and accuracy of the data are ensured, the possibility of matching errors is reduced, and question sample pairs are constructed and positive and negative samples are labeled, which improves the quality of model training data and thus enhances the accuracy of question-and-answer matching. The word vectors are optimized by combining the gradient descent algorithm and multi-dimensional category weights are incorporated to enhance the expressive power of the word vectors, so that they not only contain semantic information but also category information, which improves the discriminativeness and recognition ability of the vectors. At the same time, by optimizing the scaling parameters and category weight parameters, the word vectors can better adapt to different categories of questions, which improves the flexibility and adaptability of the overall representation. The cosine similarity is used to quickly calculate the matching degree between the user question and the standard question. Combined with the optimized extended vector, an efficient matching process is achieved, which significantly improves the response speed of the question-and-answer process.

[0040] (2) By combining multi-dimensional category vectors with original word vectors, extended vectors are generated, which retain rich semantic information and introduce category information from professional fields, making the vector representation more comprehensive and detailed. Furthermore, the word vector scaling parameters are optimized through gradient descent algorithm to ensure that the expressive power of word vectors under different categories is enhanced, thereby improving the discriminability and recognition of vectors in specific contexts. Cosine similarity is used to calculate the similarity between the user input question and the standard question. Combined with the optimized extended vectors, subtle differences in semantics and categories are effectively captured, improving the accuracy of matching. Through the optimization of category weight parameters, the system can be compatible with and effectively handle multi-class problems, ensuring that a high level of matching performance is maintained in the context of multi-class problems. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 A flowchart illustrating the multi-dimensional category-enhanced word vector fusion method for digital humans provided by this invention;

[0043] Figure 2 A schematic diagram of the structure of the multi-dimensional category-enhanced word vector fusion system provided by the present invention;

[0044] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention.

[0045] Figure labeling: 1. Multi-dimensional category-enhanced word vector fusion system; 11. Data acquisition module; 12. Sample expansion module; 13. Answer output module; 2. Electronic device; 21. Processor; 22. Communication bus; 23. User interface; 24. Network interface; 25. Memory. Detailed Implementation

[0046] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0047] This invention discloses a multi-dimensional category-enhanced word vector fusion method applicable to digital humans, with reference to... Figure 1 The method includes steps S1 to S4.

[0048] Step S1: Construct a question-and-answer database and collect user input questions. The question-and-answer database includes a set of standard questions and the answer corresponding to each standard question in the set of standard questions.

[0049] In this step, all standard questions and their corresponding answers are prepared, and a mapping relationship between them is established. Technical terms are extracted from the set of standard questions, and these terms are categorized in multiple dimensions. The number of categorization dimensions is determined, as is the order of the elements in each dimension. Then, a one-hot vector is constructed for each dimension. Finally, positive and negative samples are obtained from the existing system, with more than m*100 samples in each class, and the number of positive and negative samples is equal.

[0050] In this embodiment, after constructing the question-and-answer database and collecting user input questions, the method further includes extracting all technical terms from the standard question set and classifying all technical terms in multiple dimensions to obtain the one-hot encoded vector corresponding to the standard questions in the standard question set under each dimension.

[0051] Furthermore, collect all standard questions and their corresponding answers from existing resources, databases, or documents. Establish an explicit correspondence between each standard question and its answer, ensuring correct matching. Remove duplicates and store the organized data in a database.

[0052] Using a manual approach, all technical terms are identified and extracted from a set of standard questions. The number of classification dimensions is determined, with each dimension representing a classification criterion, such as topic, variety, or application field. Under each dimension, the technical terms are classified, and the order of the elements in the classification set within each dimension is determined to establish a clear classification hierarchy. For example, under the variety dimension, a classification of rice varieties is established, with the technical terms in the following order: "Nanjing 9108", "Nanjing 5718", "Jingliangyou 534", and "Jingliangyou Huazhan". If a standard question contains "Jingliangyou 534", then the one-hot encoding vector for that question in that dimension is [0,0,1,0].

[0053] Step S2: Based on the standard question and the user input question corresponding to the user input question, construct question sample pairs, and label each question sample pair in turn to obtain a set of positive and negative samples.

[0054] In this step, user-submitted questions and standard questions based on the digital human's historical answers are collected, establishing a correspondence between them. Each pair of user questions is manually judged to determine whether it matches a standard question, and is labeled as a positive or negative sample. From the collected sample pairs, m*100 positive sample pairs and m*100 negative sample pairs are selected for each classification dimension (where m is the number of dimensions in the multi-dimensional classification), ensuring that the number of positive and negative samples is equal and that the samples comprehensively cover all elements in each category.

[0055] Step S3: Calculate the similarity of the positive and negative sample sets based on the gradient descent algorithm to obtain the word vector scaling parameters and category weight parameters corresponding to the positive and negative sample sets. Based on the word vector scaling parameters, category weight parameters, and the positive and negative sample sets, form the first extended vector corresponding to the user input question. The similarity includes cosine similarity and dot product similarity.

[0056] In this step, the parameters required for the machine learning model are set, including the negative sample similarity cap γ, the regularization coefficient λ, the learning rate η, the loss threshold, and the maximum number of iterations. Once set, these parameters will remain unchanged during the computation process unless the learning process restarts. All positive and negative sample data is loaded, mainly covering the following three parts: user input questions, selected standard questions, and evaluation labels (label 1 for positive samples and label 0 for negative samples). If an empirical formula exists, the parameter values ​​are initialized to the square root of the corresponding empirical formula; otherwise, the parameters k and q are initialized. i Initialize to 1. Initialize the iteration counter t to 1.

[0057] This step also includes steps S31 to S34.

[0058] Step S31: Load the positive and negative sample sets, initial word vector scaling parameters, and initial class weight parameters into the machine learning model, and use the BERT model to calculate the word vectors corresponding to the user input question and the standard question in the positive and negative sample sets, respectively.

[0059] In this step, both the standard questions and user-input questions are converted into 768-dimensional word vectors using a unified model (such as the BERT model). Let each standard question be w, then the word vector obtained from the standard question w using the BERT model is V. w .

[0060] Suppose there are m categories, each containing multiple subcategories. For example, the regional category includes "Beijing," "Shanghai," etc.; the variety category includes "Nanjing 9108," "Nanjing 5718," etc.; and the purpose category includes "commercial," "domestic," etc. Each category can be represented as a one-hot vector C. wi .

[0061] If a question belongs to multiple categories, it corresponds to multiple category vectors. A new extended vector is formed by combining the original word vectors with these category vectors.

[0062]

[0063] Where k represents the scaling parameter of the word vector, q i C represents the weight parameters of the i-th category, and the one-hot encoding vector of the standard problem w. wi When the standard problem w contains terms of a certain category, the one-hot vector element corresponding to that category is 1; otherwise, it is 0. m is the dimension of the category. The plus sign here indicates expansion along the vector dimension, i.e., adding a new dimension to construct the expanded vector.

[0064] There are two main methods for filtering standard questions based on similarity after the user inputs a question. One method is to use the dot product as a similarity metric and select the standard question with the highest similarity. The other method is to use cosine similarity for independent evaluation.

[0065] The dot product is a direct similarity measure with obvious linear properties, suitable for linear combinations. Let T be the measure of dot product similarity.

[0066] If the user inputs the question u, its expanded vector is: The extension vector of the standard problem w is The formula for calculating dot product similarity is as follows:

[0067]

[0068] A common method is to empirically determine the weights of word vectors and the weights of each category. The calculation formula is usually as follows:

[0069]

[0070] Where a represents semantic weight, b i Let each category have a weight, and satisfy the following conditions:

[0071]

[0072] Since all parameters are non-negative, it can be deduced that:

[0073]

[0074] Therefore, when using the dot product as a similarity metric to screen standard problems, the aforementioned relationship exists between it and the empirical formula. If the empirical formula is reliable, and the dot product is used for screening, k and q can be easily determined. i This allows for the rapid construction of new extended vectors.

[0075] In one example, since dot product similarity is highly sensitive to vector length and cannot directly measure directional similarity between vectors, and its results are not limited in range, cosine similarity can also be used. Cosine similarity was used in the selection process; however, calculating the parameters k and q... i The same method applies to calculating dot product similarity.

[0076] If the user inputs question u and the standard question w, then the cosine similarity is... The calculation formula is:

[0077]

[0078] in, Let represent the extended vector representations of the user input problem u and the standard problem w, respectively, where · denotes the dot product, and || || denotes the Euclidean norm of the vector.

[0079] The designed contrastive loss function L is as follows:

[0080]

[0081] Where N represents the total number of samples. i This represents the sample label. If the i-th sample is a positive sample pair, the value is 1; if it is a negative sample pair, the value is 0. This represents the expansion vector of the i-th user input question. Let represent the expansion vector of the i-th standard problem, and γ represent the boundary parameter that controls the upper limit of the similarity of negative samples. The boundary parameter is set to 0.1, and this value can be adjusted between 0.01 and 0.2 as needed, but it cannot be less than 0.01 or more than 0.2.

[0082] When y i When = 1 (positive sample pair), the goal is to make The loss term is as close to 1 as possible.

[0083] When y i When =0 (negative sample pairs), the goal is to make Therefore, the loss term is...

[0084] To prevent overfitting due to excessively large parameters, an L2 regularization term is introduced, with the following formula:

[0085]

[0086] Where λ is the regularization coefficient, typically set to 0.1, but can be adjusted as needed, and m is the parameter q. l The dimension, i.e., the number of categories.

[0087] The total loss function is the sum of the contrastive loss and the regularization term, i.e.:

[0088] L total =L+L reg

[0089] The optimization goal is to find the parameters k and q. i =[q1,q2,…,q m ], making the total loss function L total minimize.

[0090] To minimize the loss function, using gradient descent, we need to calculate the loss function with respect to parameters k and q. i The partial derivatives of .

[0091] Partial derivative with respect to parameter k:

[0092]

[0093] in It is the partial derivative of k in the comparison loss function, calculated using the chain rule for the positive sample portion (and similarly for the negative sample portion):

[0094]

[0095] For parameter q i Partial derivatives:

[0096]

[0097] Similarly, By comparing the chain rule of the loss function, we can calculate the following for the positive sample portion (and similarly for the negative sample portion):

[0098]

[0099] The partial derivatives of cosine similarity with respect to the parameters are as follows:

[0100]

[0101] Among them, C i This represents a vector with the same dimension as the extended vector, where the i-th class has 1s at all positions and 0s at the other positions.

[0102] Step S32: Calculate the one-hot encoding vectors corresponding to the user input problem and the standard problem in the positive and negative sample sets, and obtain the extended vector corresponding to the sampled input problem based on the one-hot encoding vectors and the sampled input problem.

[0103] In this step, the BERT model is first used to calculate the word vectors for user questions and standard questions in all samples. Then, based on the constructed multidimensional classification, corresponding technical terms are retrieved from user questions and standard questions. If they exist, the corresponding one-hot encoded variable is set to 1; otherwise, it is set to 0. This method obtains the one-hot encoded variables for each question. Next, parameters k and q are used... i Generate extended vectors for user questions and standard questions from all samples.

[0104] Select a sampled input problem from any pair of problem samples in the positive and negative sample sets. The sampled input problem includes a first user input problem and a first standard problem corresponding to the first user input problem. Calculate the sampled input problem according to the BERT model and obtain the first word vector corresponding to the sampled input problem. Obtain the extended vector corresponding to the sampled input problem based on the transition word vector scaling parameter, the transition category weight parameter, the first word vector, and the one-hot encoding vector corresponding to the sampled input problem.

[0105] Furthermore, the question sample pair includes both user questions and selected standard questions, both of which can be used as input to the expansion vector. Here, one question is randomly selected as an example. The sample pair includes both user questions and selected standard questions, both of which can be used as input to the expansion vector. Here, one question is randomly selected as an example. The sample pair includes both user questions and selected standard questions, both of which can be used as input to the expansion vector. First, all dimensions of the one-hot encoding vector for this question are initialized to 0. Then, using terminology from multidimensional classification, each term is searched in the question: if a term is found in the question, the value of the corresponding one-hot encoding vector is set to 1; if not found, the corresponding value remains 0. After searching for all terms, the one-hot vector for this question is obtained. Using parameters k and q... i Multiply them one by one with the corresponding word vectors and one-hot vector elements to generate an extended vector with a dimension greater than 768.

[0106] Step S33: Use cosine similarity to calculate the similarity between the user input question and the standard question in each question sample pair in the positive and negative sample sets, and calculate the total loss function based on the transition word vector scaling parameter and the transition category weight parameter.

[0107] Given the current parameters k and q i In this case, first calculate the regularization loss L. reg Then, the contrast loss L obtained in the previous step is compared with the regularization loss L. reg Adding them together, we get the total loss L. total The total loss is expressed with respect to the current parameters k and q. i The overall loss value of the machine learning model.

[0108] In this step, the expression for the total loss function is:

[0109]

[0110] L total =L+L reg

[0111] in, This represents the cosine similarity between the user input question u and the standard question w in the positive and negative sample sets. This represents the extended vector of the user input question u. Let y denote the extended vector of the standard problem w, · denote the dot product, ||| denote the Euclidean norm of the vector, L denotes the contrastive loss function, N denotes the total number of samples, and y i Indicates sample label, This represents the expanded vector of the i-th user input question u. Let w represent the extension vector of the i-th standard problem w. Let L represent the cosine similarity between the i-th user input question u and the i-th standard question w, γ represent the boundary parameter controlling the upper limit of negative sample similarity, max() represent the maximum value function, and L represent the maximum value function. reg Let λ represent the regularization term function, λ represent the regularization coefficient, and q represent the regularization term function. i Let m represent the weight parameter of the i-th category, m represent the dimension corresponding to the weight parameter of the i-th category, k represent the word vector scaling parameter, and L represent the weight parameter of the i-th category. total This represents the total loss function.

[0112] Step S34: When the total loss function is greater than the loss threshold or the maximum number of iterations is reached, the machine learning model outputs word vector scaling parameters and category weight parameters.

[0113] In this step, the expressions for the word vector scaling parameter and the category weight parameter are as follows:

[0114]

[0115] Where, k (t+1) This represents the word vector scaling parameter after the (t+1)th iteration. Let q represent the weight parameter of the i-th class after the (t+1)-th iteration. i Let L represent the weight parameter for the i-th category, k represent the word vector scaling parameter, and L represent the weight parameter for the i-th category. total Let represent the total loss function, t represent the number of steps in the current iteration, and η represent the learning rate.

[0116] By combining multi-dimensional category vectors with original word vectors to generate extended vectors, rich semantic information is preserved and category information from specialized domains is introduced, making the vector representation more comprehensive and detailed. Furthermore, the word vector scaling parameters are optimized using the gradient descent algorithm to ensure that the expressive power of word vectors under different categories is enhanced, improving the discriminativeness and recognition of vectors in specific contexts. Cosine similarity is used to calculate the similarity between the user input question and the standard question. Combined with the optimized extended vectors, subtle differences in semantics and categories are effectively captured, improving the accuracy of matching. Through the optimization of category weight parameters, the system can be compatible with and effectively handle multi-class problems, ensuring that a high level of matching performance is maintained even in multi-class contexts.

[0117] Step S4: Calculate the cosine similarity between the first extended vector and the extended vectors corresponding to all standard questions, select the standard question with the highest cosine similarity, and output the answer corresponding to the standard question to the user.

[0118] This step also includes steps S41 to S43.

[0119] Step S41: Calculate the first cosine similarity between the first extended vector and the extended vector corresponding to the currently selected standard problem.

[0120] Step S42: Determine whether the current maximum similarity value is less than the first cosine similarity. If the current maximum similarity value is less than the first cosine similarity, then assign the first cosine similarity value to the current maximum similarity value.

[0121] Step S43: Determine whether all the extended vectors of the standard questions have been traversed. If all the extended vectors have been traversed, output the answer of the standard question corresponding to the first cosine similarity.

[0122] In one example, the steps to obtain an answer based on user input are as follows:

[0123] Step 1: Define two variables, `tmp` and `final`, and initialize them to 0. `tmp` stores the currently found maximum similarity value. `final` stores the number of the standard question that has the maximum similarity to the user question's extended vector. After completing this, proceed to Step 2.

[0124] Step 2: The user enters the question they want to ask and submits it to the system backend. After completion, proceed to Step 3.

[0125] Step 3: Based on the user's input question, use the method described in Step 5 of Part 2 to generate the extended vector for the question. Proceed to Step 4 after completion.

[0126] Step 4: Select an expansion vector from the standard problem expansion vector set: if this is the first time performing this step, select the first one; otherwise, select the next vector after the previous one. Proceed to Step 5 after completing this step.

[0127] Step 5: Calculate the cosine similarity between the expanded vector of the user's question and the expanded vector of the currently selected standard question, and store the result in the variable Sim. After completion, proceed to step 6.

[0128] Step 6: If Sim≥tmp, it means that the current calculated similarity is greater than or equal to the previous maximum value, proceed to step 7; otherwise, proceed to step 9.

[0129] Step 7: Assign the value of Sim to tmp, update the maximum similarity, and ensure that tmp always stores the maximum similarity. After completion, proceed to step 8.

[0130] Step 8: Assign the current standard problem number to `final`, and update the standard problem number so that it is the number with the highest similarity. After completion, proceed to step 9.

[0131] Step 9: Determine whether all standard problem extension vectors have been traversed and processed. If not, return to step 4 and continue processing the next standard problem extension vector; otherwise, if all extension vectors have been traversed, proceed to step 10.

[0132] Step 10: Based on the standard question number stored in the variable `final`, find the corresponding standard answer and output that answer to the user.

[0133] Through the above steps, the system accurately finds the most matching standard question based on the user's input and provides the corresponding standard answer to the user. This process ensures that users receive the most relevant and accurate answers. By establishing a database containing standard questions and their corresponding answers, the system ensures data integrity and accuracy, reduces the possibility of matching errors, and improves the quality of model training data by constructing question sample pairs and labeling positive and negative samples, thereby enhancing the accuracy of question-and-answer matching. The system also optimizes word vectors using gradient descent and incorporates multi-dimensional category weights to enhance the expressive power of word vectors, making them include not only semantic information but also category information, improving the vector's discriminative and recognition capabilities. Furthermore, by optimizing scaling and category weight parameters, the word vectors can better adapt to different categories of questions, improving the overall flexibility and adaptability of the representation. Using cosine similarity to quickly calculate the matching degree between the user's question and the standard question, combined with the optimized extended vectors, the system achieves an efficient matching process, significantly improving the response speed of the question-and-answer process.

[0134] Based on the above method, this application discloses a multi-dimensional category-enhanced word vector fusion system suitable for digital humans, with reference to... Figure 2 The multi-dimensional category-enhanced word vector fusion system 1 includes a data acquisition module 11, a sample expansion module 12, and an answer output module 13, wherein...

[0135] Data acquisition module 11 is used to construct a question-and-answer database and collect user input questions. The question-and-answer database includes a set of standard questions and the answer corresponding to each standard question in the set of standard questions.

[0136] The sample expansion module 12 is used to construct question sample pairs based on the standard question corresponding to the user input question and the user input question, and to mark each question sample pair in turn to obtain a set of positive and negative samples. The similarity of the positive and negative sample sets is calculated based on the gradient descent algorithm to obtain the word vector scaling parameters and category weight parameters corresponding to the positive and negative sample sets. Based on the word vector scaling parameters, category weight parameters and the set of positive and negative samples, the first expansion vector corresponding to the user input question is formed.

[0137] The answer output module 13 is used to calculate the cosine similarity between the first extended vector and the extended vectors corresponding to all standard questions, select the standard question with the highest cosine similarity, and output the answer corresponding to the standard question to the user.

[0138] In one example, the data acquisition module 11 is used to extract all technical terms in the standard problem set and classify all technical terms in multiple dimensions to obtain the one-hot encoded vector corresponding to the standard problem in the standard problem set under each dimension.

[0139] In one example, the sample expansion module 12 loads positive and negative sample sets, initial word vector scaling parameters, and initial class weight parameters into the machine learning model. It then uses the BERT model to calculate the word vectors corresponding to the user input question and the standard question in the positive and negative sample sets, respectively. It calculates the one-hot encoded vectors corresponding to the user input question and the standard question in the positive and negative sample sets, and obtains the expanded vector corresponding to the sampled input question based on the one-hot encoded vectors and the sampled input question. It uses cosine similarity to calculate the similarity between the user input question and the standard question in each question sample pair in the positive and negative sample sets, and calculates the total loss function based on the transition word vector scaling parameters and the transition class weight parameters. When the total loss function is greater than the loss threshold or the maximum number of iterations is reached, the machine learning model outputs the word vector scaling parameters and the class weight parameters.

[0140] In one example, the sample expansion module 12 is used to select a sampled input question from any question sample pair in the positive and negative sample set, wherein the sampled input question includes a first user input question and a first standard question corresponding to the first user input question; calculate the sampled input question according to the BERT model to obtain the first word vector corresponding to the sampled input question; and obtain the expanded vector corresponding to the sampled input question according to the transition word vector scaling parameter, the transition category weight parameter, the first word vector, and the one-hot encoding vector corresponding to the sampled input question.

[0141] In one example, the expression for the total loss function is:

[0142]

[0143] L total =L+L reg

[0144] in, This represents the cosine similarity between the user input question u and the standard question w in the positive and negative sample sets. This represents the extended vector of the user input question u. Let y denote the extended vector of the standard problem w, · denote the dot product, ||| denote the Euclidean norm of the vector, L denotes the contrastive loss function, N denotes the total number of samples, and y i Indicates sample label, This represents the expanded vector of the i-th user input question u. Let w represent the extension vector of the i-th standard problem w. Let L represent the cosine similarity between the i-th user input question u and the i-th standard question w, γ represent the boundary parameter controlling the upper limit of negative sample similarity, max() represent the maximum value function, and L represent the maximum value function. reg Let λ represent the regularization term function, λ represent the regularization coefficient, and q represent the regularization term function. i Let m represent the weight parameter of the i-th category, m represent the dimension corresponding to the weight parameter of the i-th category, k represent the word vector scaling parameter, and L represent the weight parameter of the i-th category. total This represents the total loss function.

[0145] In one example, the answer output module 13 is used to calculate the first cosine similarity between the first extended vector and the extended vector corresponding to the currently selected standard question; determine whether the currently stored maximum similarity value is less than the first cosine similarity; if the currently stored maximum similarity value is less than the first cosine similarity, then assign the first cosine similarity to the currently stored maximum similarity value; determine whether all the extended vectors of the standard questions have been traversed; if all the extended vectors have been traversed, then output the answer to the standard question corresponding to the first cosine similarity.

[0146] In one example, the expressions for the word vector scaling parameter and the class weight parameter are as follows:

[0147]

[0148] Where, k (t+1) This represents the word vector scaling parameter after the (t+1)th iteration. Let q represent the weight parameter of the i-th class after the (t+1)-th iteration. i Let L represent the weight parameter for the i-th category, k represent the word vector scaling parameter, and L represent the weight parameter for the i-th category. total Let represent the total loss function, t represent the number of steps in the current iteration, and η represent the learning rate.

[0149] Please see Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 3 As shown, the electronic device 2 may include: at least one processor 21, at least one network interface 24, user interface 23, memory 25, and at least one communication bus 22.

[0150] The communication bus 22 is used to enable communication between these components.

[0151] The user interface 23 may include a display screen and a camera. Optionally, the user interface 23 may also include a standard wired interface and a wireless interface.

[0152] The network interface 24 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0153] The processor 21 may include one or more processing cores. The processor 21 connects to various parts of the server using various interfaces and lines, and performs various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 25, and by calling data stored in the memory 25. Optionally, the processor 21 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 21 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also be implemented as a separate chip without being integrated into the processor 21.

[0154] The memory 25 may include random access memory (RAM) or read-only memory. Optionally, the memory 25 may include non-transitory computer-readable storage medium. The memory 25 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 25 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 25 may also be at least one storage device located remotely from the aforementioned processor 21. Figure 3 As shown, the memory 25, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a multi-dimensional category-enhanced word vector fusion method suitable for digital humans.

[0155] exist Figure 3In the electronic device 2 shown, the user interface 23 is mainly used to provide an input interface for the user and obtain the user input data; while the processor 21 can be used to call a multi-dimensional category-enhanced word vector fusion method suitable for digital humans stored in the memory 25. When executed by one or more processors, the electronic device executes one or more methods as described in the above embodiments.

[0156] A computer-readable storage medium storing instructions that, when executed by one or more processors, cause a computer to perform one or more methods as described in the embodiments above.

[0157] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0158] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0159] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the shown or discussed mutual couplings or direct couplings or communication connections may be through some service interfaces; indirect couplings or communication connections between apparatuses or units may be electrical or other forms.

[0160] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0161] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0162] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0163] The above are merely exemplary embodiments of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Those skilled in the art will readily conceive of other embodiments of this disclosure upon considering the specification and the disclosure of practical truths. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure.

[0164] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A multi-dimensional category-enhanced word vector fusion method suitable for digital humans, characterized in that, The method includes: Construct a question-and-answer database and collect user-input questions, wherein the question-and-answer database includes a set of standard questions and the answer corresponding to each standard question in the set of standard questions; Extract all technical terms from the standard problem set and classify all technical terms in multiple dimensions to obtain the one-hot encoded vector corresponding to the standard problem in the standard problem set under each dimension; Based on the standard question corresponding to the user input question and the user input question, construct question sample pairs, and label each question sample pair in turn to obtain a set of positive and negative samples; The similarity of the positive and negative sample sets is calculated based on the gradient descent algorithm to obtain the word vector scaling parameters and category weight parameters corresponding to the positive and negative sample sets. Based on the word vector scaling parameters, the category weight parameters and the positive and negative sample sets, a first extended vector corresponding to the user input question is formed. The similarity includes cosine similarity and dot product similarity. The process of obtaining the word vector scaling parameters and category weight parameters corresponding to the positive and negative sample sets specifically includes: Load the positive and negative sample set, initial word vector scaling parameters, and initial class weight parameters into the machine learning model, and use the BERT model to calculate the word vectors corresponding to the user input question and the standard question in the positive and negative sample set, respectively. Calculate the one-hot encoded vectors corresponding to the user input question and the standard question in the positive and negative sample set, and obtain the extended vector corresponding to the sampled input question based on the one-hot encoded vectors and the sampled input question; The similarity between the user input question and the standard question in each question sample pair in the positive and negative sample sets is calculated using cosine similarity, and the total loss function is calculated based on the transition word vector scaling parameter and the transition category weight parameter. When the total loss function is greater than the loss threshold or the maximum number of iterations is reached, the machine learning model outputs the word vector scaling parameter and the category weight parameter. Calculate the cosine similarity between the first extended vector and the extended vectors corresponding to all standard questions, select the standard question with the highest cosine similarity, and output the answer corresponding to the standard question to the user. The first extended vector includes a multi-dimensional one-hot encoded vector and the original word vector.

2. The multi-dimensional category-enhanced word vector fusion method for digital humans as described in claim 1, characterized in that, The step of obtaining the extended vector corresponding to the sampling input problem specifically includes: Select a sampled input problem from any pair of problem samples in the set of positive and negative samples, wherein the sampled input problem includes a first user input problem and a first standard problem corresponding to the first user input problem; The sampling input question is calculated based on the BERT model, and the first word vector corresponding to the sampling input question is obtained; Based on the transition word vector scaling parameter, the transition category weight parameter, the first word vector, and the one-hot encoding vector corresponding to the sampling input problem, obtain the extended vector corresponding to the sampling input problem.

3. The multi-dimensional category-enhanced word vector fusion method for digital humans as described in claim 1, characterized in that, The expression for the total loss function is: L total < L+L reg in, This represents the cosine similarity between the user input question u and the standard question w in the set of positive and negative samples. This represents the extended vector of the user input question u. Let y denote the extended vector of the standard problem w, · denote the dot product, ||| denote the Euclidean norm of the vector, L denotes the contrastive loss function, N denotes the total number of samples, and y i Indicates sample label, This represents the expanded vector of the i-th user input question u. Let w represent the extension vector of the i-th standard problem w. Let L represent the cosine similarity between the i-th user input question u and the i-th standard question w, γ represent the boundary parameter controlling the upper limit of negative sample similarity, max() represent the maximum value function, and L represent the maximum value function. reg Let λ represent the regularization term function, λ represent the regularization coefficient, and q represent the regularization term function. l Let L represent the weight parameters of the l-th category, m represent the dimension corresponding to the weight parameters of the l-th category, k represent the word vector scaling parameter, and L represent the weight parameters of the l-th category. total This represents the total loss function.

4. The multi-dimensional category-enhanced word vector fusion method for digital humans as described in claim 1, characterized in that, The process of selecting a standard question with the highest cosine similarity and outputting the answer to the standard question to the user specifically includes: Calculate the first cosine similarity between the first extended vector and the extended vector corresponding to the currently selected standard problem; Determine whether the current maximum similarity value is less than the first cosine similarity. If the current maximum similarity value is less than the first cosine similarity, then assign the first cosine similarity value to the current maximum similarity value. Determine whether all extended vectors of the standard question have been traversed. If all extended vectors have been traversed, output the answer to the standard question corresponding to the first cosine similarity.

5. The multi-dimensional category-enhanced word vector fusion method for digital humans as described in claim 1, characterized in that, The expressions for the word vector scaling parameter and the category weight parameter are as follows: Where, k (t+1) This represents the word vector scaling parameter after the (t+1)th iteration. Let q represent the weight parameter of the i-th class after the (t+1)-th iteration. i Let L represent the weight parameter for the i-th category, k represent the word vector scaling parameter, and L represent the weight parameter for the i-th category. total Let represent the total loss function, t represent the number of steps in the current iteration, and η represent the learning rate.

6. A multi-dimensional category-enhanced word vector fusion system suitable for digital humans, characterized in that, The multi-dimensional category-enhanced word vector fusion system (1) includes a data acquisition module (11), a sample expansion module (12), and an answer output module (13), wherein, The data acquisition module (11) is used to construct a question-and-answer database and collect user input questions, wherein the question-and-answer database includes a set of standard questions and the answer corresponding to each standard question in the set of standard questions; Extract all technical terms from the standard problem set and classify all technical terms in multiple dimensions to obtain the one-hot encoded vector corresponding to the standard problem in the standard problem set under each dimension; The sample expansion module (12) is used to construct question sample pairs based on the standard question corresponding to the user input question and the user input question, to mark each question sample pair in turn to obtain a set of positive and negative samples, to calculate the similarity of the positive and negative sample sets based on the gradient descent algorithm to obtain the word vector scaling parameters and category weight parameters corresponding to the positive and negative sample sets, and to form a first expansion vector corresponding to the user input question based on the word vector scaling parameters, the category weight parameters and the positive and negative sample sets, wherein the similarity includes cosine similarity and dot product similarity; The process of obtaining the word vector scaling parameters and category weight parameters corresponding to the positive and negative sample sets specifically includes: Load the positive and negative sample set, initial word vector scaling parameters, and initial class weight parameters into the machine learning model, and use the BERT model to calculate the word vectors corresponding to the user input question and the standard question in the positive and negative sample set, respectively. Calculate the one-hot encoded vectors corresponding to the user input question and the standard question in the positive and negative sample set, and obtain the extended vector corresponding to the sampled input question based on the one-hot encoded vectors and the sampled input question; The similarity between the user input question and the standard question in each question sample pair in the positive and negative sample sets is calculated using cosine similarity, and the total loss function is calculated based on the transition word vector scaling parameter and the transition category weight parameter. When the total loss function is greater than the loss threshold or the maximum number of iterations is reached, the machine learning model outputs the word vector scaling parameter and the category weight parameter. The answer output module (13) is used to calculate the cosine similarity between the first extended vector and the extended vectors corresponding to all standard questions, select the standard question with the highest cosine similarity, and output the answer corresponding to the standard question to the user. The first extended vector includes a multi-dimensional one-hot encoded vector and the original word vector.

7. An electronic device, characterized in that, The device includes a processor (21), a memory (25), a user interface (23), and a network interface (24). The memory (25) is used to store instructions. The user interface (23) and the network interface (24) are used to communicate with other devices. The processor (21) is used to execute the instructions stored in the memory (25) to cause the electronic device (2) to perform the method as described in any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-5.

Images