Product Q&A Method Based on RoberTa-WWM

Abstract

This invention discloses a product question-answering method based on RoBERTa-WWM. First, a template-based question corpus generation method is designed to generate training corpus based on the data characteristics of a product knowledge graph. The corpus is then labeled with BIO sequences and semantic classification tags for training the question-answering model. Next, a question-based named entity recognition model based on RoBERTa-WWM-BiLSTM-CRF and a question-based semantic classification model based on RoBERTa-WWM-TextCNN are constructed to achieve semantic parsing of questions. Finally, based on the semantic parsing results, a Cypher query is constructed to retrieve and obtain the answer from the knowledge graph. This invention, by applying the RoBERTa-WWM algorithm to the design of product question-answering solutions, can obtain question-answering solutions that closely meet user needs, saving design costs and time.

Description

Technical Field

[0001] The present invention belongs to the technical field of commodity question answering, and specifically, it relates to a commodity question answering method based on RoBERTa-WWM using knowledge graph technology. ​​​​​​​​​​​​​​​

[0006] The present invention is implemented using the following technical solutions:

[0007] A product question-answering method based on RoBERTa-WWM is proposed, including:

[0008] Sequence labeling and semantic classification tagging are performed on the training corpus;

[0009] Construct a RoBERTa-WWM-BiLSTM-CRF model to identify named entities in product questions; and construct a RoBERTa-WWM-TextCNN model to perform semantic classification of product questions.

[0010] Construct a Cypher query statement to retrieve results from a graph database;

[0011] Sort the query results and match them with the product questions, then output the matching results.

[0012] In some embodiments of the present invention, the method further includes the step of generating training corpus, comprising:

[0013] The entity, attribute, and relationship information from the product knowledge graph are extracted as the basic data for generating question corpus;

[0014] Based on the question structure, question rule templates are constructed, and question templates are generated from different types of product entity data in the knowledge graph according to the question rule templates.

[0015] The training corpus of questions is obtained by filling in the entity, attribute, and relationship information in the product knowledge graph according to the question template.

[0016] In some embodiments of the present invention, semantic classification and labeling of the training corpus includes:

[0017] The entity type with attributes is expanded into entity classes and entity-attribute classes, with each entity-attribute class serving as a subcategory to further subdivide the question categories.

[0018] In some embodiments of the present invention, the constructed RoBERTa-WWM-BiLSTM-CRF model consists of a RoBERTa-WWM Chinese pre-trained language model, a BiLSTM layer, and a CRF layer; it identifies named entities for product questions, including:

[0019] The input text is encoded using RoBERTa-WWM, and the resulting sequence vector representation is passed to the BiLSTM layer. The BiLSTM layer performs semantic encoding on the sequence vector and outputs the score of each character corresponding to each category label. The CRF layer obtains the prediction sequence through the constraint information between adjacent labels, thereby obtaining the category of each character.

[0020] In some embodiments of the present invention, the constructed RoBERTa-WWM-TextCNN model consists of a word embedding layer, a convolutional layer, a max-pooling layer, and a fully connected layer; semantic classification of product questions specifically includes:

[0021] Based on RoBERTa-WWM, we perform context modeling and sentence-level semantic representation of questions. The vector corresponding to the “[CLS]” symbol output by the last layer of RoBERTa-WWM is used to represent the context of the question. Then, the question is classified after feature extraction by the fully connected TextCNN neural network.

[0022] In some embodiments of the present invention, sequence annotation of the training corpus specifically includes:

[0023] The BIO annotation method is used to convert tags with overlapping characters into two new tags.

[0024] In some embodiments of the present invention, the method further includes:

[0025] In generating question templates by generating question templates from different types of product entity data in the knowledge graph according to question rule templates, sentence subject replacement and synonym conversion are used to generalize and generate training corpus.

[0026] Compared with existing technologies, the advantages and positive effects of this invention are as follows: The product question answering method based on RoBERTa-WWM proposed in this invention first designs a template rule-based question corpus generation method to generate training corpus based on the data characteristics of the product knowledge graph, and performs BIO sequence annotation and semantic classification labeling on the corpus for question answering model training; then, it constructs a RoBERTa-WWM-BiLSTM base model.

[0027] The invention employs a CRF-based named entity recognition model and a RoBERTa-WWM-TextCNN-based semantic classification model to perform semantic parsing of questions. Finally, based on the semantic parsing results, a Cypher query is constructed to retrieve and obtain answers from the knowledge graph. By applying the RoBERTa-WWM algorithm to product question-and-answer solution design, this invention achieves a question-and-answer solution that closely meets user needs, saving design costs and time.

[0028] Other features and advantages of the present invention will become clearer after reading the detailed description of the embodiments of the present invention in conjunction with the accompanying drawings. Attached Figure Description

[0029] The accompanying drawings, as part of this invention, are provided to further illustrate the invention. The illustrative embodiments and descriptions are used to explain the invention but do not constitute an undue limitation thereof. Clearly, the drawings described below are merely some embodiments; those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0030] Figure 1 This is a schematic diagram of the steps of the product question-and-answer method based on Robeta-WWM proposed in this invention;

[0031] Figure 2 This is a schematic diagram of the question corpus generation method given in the embodiments of the present invention;

[0032] Figure 3 This is a schematic diagram of the question structure in an embodiment of the present invention;

[0033] Figure 4 This is a schematic diagram of the RoBERTa-WWM-BiLSTM-CRF model structure provided in this invention;

[0034] Figure 5 This is a diagram of the LSTM unit structure given in the embodiments of the present invention;

[0035] Figure 6 This is a CRF structure diagram given in an embodiment of the present invention;

[0036] Figure 7 This is a schematic diagram of the RoBERTa-WWM-TextCNN question semantic classification model presented in this invention.

[0037] It should be noted that these accompanying drawings and textual descriptions are not intended to limit the scope of the invention in any way, but rather to illustrate the concept of the invention to those skilled in the art by referring to specific embodiments. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments will be clearly and completely described below with reference to the accompanying drawings. The following embodiments are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0039] In the description of this invention, it should be noted that the terms "upper", "lower", "front", "rear", "left", "right", "vertical", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting this invention.

[0040] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or a connection indirectly through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0041] The present invention aims to propose a product question-and-answer method that improves the stability of question-and-answer results, meets as many question-and-answer needs as possible, reduces the repetitive nature of manual experience-based design, and saves costs and time.

[0042] Specifically, this invention proposes a product question-and-answer method based on RoberTa-WWM, such as... Figure 1 As shown, it includes:

[0043] S101. Generate training corpus and perform sequence labeling and semantic classification on the training corpus.

[0044] We design a template rule-based question corpus generation method to generate training corpus, and perform BIO sequence annotation and semantic classification labeling on the corpus for question answering model training.

[0045] S102. Construct a RoBERTa-WWM-BiLSTM-CRF model to identify named entities of product questions; and construct a RoBERTa-WWM-TextCNN model to perform semantic classification of product questions.

[0046] Entity extraction and classification are performed on user-input questions to understand the user's query intent.

[0047] In the area of ​​named entity recognition, we propose using BiLSTM-CRF as the baseline model and introducing the Chinese pre-trained model RoBERTa-WWM to construct the RoBERTa-WWM-BiLSTM-CRF model for named entity recognition of questions about home appliances.

[0048] In terms of question semantic classification, the question semantic classification model based on RoBERTa-WWM-TextCNN uses RoBERTa-WWM as the basis for question context modeling and sentence-level semantic representation. The vector corresponding to the “[CLS]” symbol output by the last layer of RoBERTa-WWM is used to represent the context of the question. Then, the question is classified after feature extraction by the fully connected layer TextCNN neural network.

[0049] S103. Construct a Cypher query statement to retrieve results from the graph database.

[0050] Based on the semantic parsing results of the questions, Cypher query statements are constructed, and corresponding results are retrieved in the Neo4j graph database. For each question, entity information and semantic classification can be obtained through model prediction. Finally, Cypher query statements are constructed based on this information to retrieve answers in the knowledge graph. If the entity information in the question input by the user is incomplete, the system will perform a fuzzy search and filter the results by combining other entity information present in the question.

[0051] S104. Sort the query results and match them with the product questions, then output the matching results.

[0052] The system sorts the results from the Cypher query and matches them with the question, returning the answer with the highest match to the user. If no entity or category information is obtained after parsing the question, the system automatically returns a friendly prompt to the user.

[0053] The product question-and-answer method proposed in this invention will be described in detail below.

[0054] 1. Training corpus generation.

[0055] In step S101 above, in some embodiments of the present invention, in order to obtain more question corpus data and improve the accuracy of the training model, a corpus dataset is constructed based on crawling data from e-commerce websites, using an automatic question corpus generation method. Based on the characteristics of product questions, especially those related to home appliances, a template-based question corpus generation method is designed, such as... Figure 2 As shown:

[0056] (1) Based on the content of the knowledge graph of home appliances, obtain the entity, attribute and relationship information as the basic data for generating question corpus.

[0057] For example, entity information includes the specific model of the appliance, company name, brand name, etc., attribute information includes the appliance's energy efficiency rating, size, weight, power, etc., and relationship information includes the relationship between the company and the brand, and between the appliance and the brand.

[0058] (2) Analyze the question structure and design question rule templates to generate questions from different types of home appliance entity data in the knowledge graph according to the rules.

[0059] Since questions are composed of known entities, relations, or attributes, expanding them into a natural language question allows each entity tag to have an associated relation or attribute tag, and the expanded question does not have semantic ambiguity.

[0060] In some embodiments of the present invention, to ensure that the questions cover more questioning scenarios, sentence subject substitution and synonym conversion are used to generalize and generate more training data, thus guaranteeing the diversity and accuracy of the data. For example, the question "What is the weight of the Haier refrigerator BCD-328WFPZH?" can be categorized by brand, type of appliance, model of appliance, and meaning of the question. Figure 4 Each part, such as Figure 3 As shown, the brand, category, and appliance model are derived from entity nodes in the knowledge graph. The question is intended to inquire about the weight of a product, and there are various ways to ask it, such as "How heavy is it?", "How many kilograms is it?", "How many kilograms is it?", etc. The question can also be enriched by adding some modal particles or transition words.

[0061] (3) Based on the question template, the entities, relationships and attributes in the home appliance knowledge graph can be filled in according to the rules to achieve automatic generation of question corpus. Some question examples are shown in Table 1 below.

[0062] Table 1

[0063]

[0064] 2. BIO sequence labeling of question corpus.

[0065] Named entity recognition typically involves two parts: entity boundary identification and entity category determination. In step S101 of this invention, sequence labeling of the training corpus can be divided into original labeling and joint labeling, depending on the labeling granularity.

[0066] In the invention task, the entity is composed of multiple words, which belongs to the joint annotation task. Compared with the original annotation, the joint annotation may have a dependency relationship between adjacent word tags. Therefore, BIO annotation is used to transform the joint annotation into the original annotation to solve this problem.

[0067] BIO tagging transforms tags in a joint tag that involve cross-word situations into two new tags. For example, the entity tag "BRAND" is transformed into "B-BRAND" and "I-BRAND". "B" represents the first word of the entity, "I" represents the word following the first word, and "O" indicates that the word does not belong to any predefined word segment type. Specific tagging examples are shown in Table 2 below.

[0068] Table 2

[0069]

[0070] In practical applications, a dictionary-based corpus BIO tagging tool can be developed. Based on the entity information obtained from the knowledge graph, a dictionary of the required tagged entities can be easily created. Each line of data consists of an entity name and a tag, separated by a space, such as "Haier BRAND". After reading the question corpus, each line of the sentence is split into individual Chinese characters or symbols, generating a tagging result set of the same length as the corpus, and by default, all of them are filled with "O" tags. Then, the tagged entities in the dictionary are traversed, and the entities in the corpus that match the dictionary are set as the corresponding tags, until all corpus tags are completed and written to a file.

[0071] 3. Construct a question classification method.

[0072] Before performing semantic classification of questions, it is necessary to define the category of the questions and add corresponding category labels for training the classification model. The quality of the question classification results directly affects the accuracy of the question-answering system. Currently, there is no unified standard for classifying Chinese questions, especially for domain-specific question-answering systems. Therefore, it is necessary to construct classification methods based on the characteristics of the data.

[0073] This invention uses a knowledge graph as its knowledge base, where the data consists of entities, attributes, and relationships. It considers expanding the entity type with attributes to "entity class" and "entity_attribute class". The category of the entity itself is used as the entity class, such as home appliance or company. The attribute categories possessed by the entity are divided into "entity_attribute classes", with each attribute serving as a subcategory, as shown in Table 3.

[0074] Table 3

[0075]

[0076] Subcategories are used to further subdivide the categories of questions, thereby improving the accuracy of question classification. Based on the classification results, the intent of the question can be clearly understood.

[0077] For example, the question "What are the parameters of Midea BCD-606WKPZM?" is categorized as "Home Appliances" without a specific attribute category. However, the question "What is the energy efficiency rating of Midea BCD-606WKPZM?" is directly categorized as "Home Appliances_Energy Efficiency Rating". Combining this with named entity recognition, we can understand that the semantics of this question are asking about the energy efficiency rating of the home appliance model "BCD-606WKPZM". Then, a Cypher query statement is constructed for this type of question: "MATCH(n)-[r:`Energy Efficiency Rating`]-(b)where n.name='BCD-606WKPZM'RETURN b". After querying the knowledge graph, the result is returned to the user, achieving automatic question answering.

[0078] 4. Named entity recognition.

[0079] The accuracy of Named Entity Recognition (NER) determines the performance of downstream tasks, enabling the identification of predefined entity objects from text.

[0080] Named entities are generally divided into generalized named entities, such as personal names, place names, and organization names, and domain-specific named entities, such as product models, brands, and companies in the home appliance product domain constructed in this paper. This invention proposes using BiLSTM-CRF as the baseline model and introducing the Chinese pre-trained model RoBERTa-WWM to construct the RoBERTa-WWM-BiLSTM-CRF model for named entity recognition of home appliance product questions. The model structure is as follows. Figure 4 As shown.

[0081] The RoBERTa-WWM-BiLSTM-CRF entity recognition model mainly consists of three parts: the RoBERTa-WWM Chinese pre-trained language model, the BiLSTM layer, and the CRF layer. RoBERTa-WWM encodes the input text and passes the resulting sequence vector representation to the BiLSTM layer. The BiLSTM layer further performs semantic encoding on the sequence vector, outputting the score for each character corresponding to each category label. The CRF layer uses constraint information between adjacent labels to obtain the final predicted sequence, thus determining the appropriate category for each character.

[0082] The algorithm module will be introduced below.

[0083] (1) Bidirectional Long Short-Term Memory Network (BiLSTM)

[0084] After processing the training data through the RoBERTa-WWM layer, accurate semantic vector representations are obtained. Inputting this result into the BiLSTM network helps improve the model's ability to capture contextual features of the dataset. BiLSTM addresses the vanishing and exploding gradient problems inherent in RNN networks, representing an improved version of the RNN model. Since unidirectional LSTMs can only capture historical information from the forward sequence during feature extraction, failing to fully utilize the text's contextual semantic information, BiLSTM incorporates a reverse LSTM layer. This layer learns features from both the forward and reverse directions, then concatenates the two sequences, allowing for the reasonable learning of contextual features at each time step and resulting in stronger memory capabilities.

[0085] BiLSTM is composed of LSTMs. A basic LSTM consists of three gate control units: a forget gate, an input gate, and an output gate. These gates control the proportion of information that is forgotten and carried over to the next time step. The basic structure of an LSTM unit is as follows: Figure 5 As shown.

[0086] from Figure 5 As can be seen, the most important structure within the LSTM is the transport band, which carries past information to the next time step to prevent gradient vanishing. Here, f... t i t and o t Let f represent the forget gate, input gate, and output gate at time t, respectively, allowing information to pass selectively. Forget gate f t The value of the vector is between 0 and 1, and the value of the vector represents the memory unit C. t-1 This indicates which information should be retained or discarded. 1 means retain, 0 means discard. Input gate i t This involves adding new information to the transmission band C and updating its state. Output gate o t Similar to the forget gate, it depends on the input word X at the current moment. t Compared to the hidden layer state h at the previous time step t-1 Finally, regarding the transmission band C t Perform a hyperbolic tangent operation on each element, and substitute it into the tanh function to obtain a new vector and o. t Performing the dot product operation yields the hidden layer state vector h. t h t Divide into two parts, one part is passed to the state vector h at the next time step. t One copy is used as the output of the LSTM. The specific calculation formulas are shown in (1-1) to (1-6).

[0087] f t =σ(W f ·[h t-1 ,x t ]+b f (1-1)

[0088] i t =σ(W i ·[h t-1 ,x t ]+b i (1-2)

[0089]

[0090]

[0091] o t =σ(W o ·[h t-1 ,x t ]+b o (1-5)

[0092] h t =o t⊙tanh(C t (1-6)

[0093] Where σ is the Sigmoid activation function. and C t These are the candidate vector and the final vector at time t, respectively. tanh is the hyperbolic tangent activation function, ⊙ represents the dot product operation, and b... i b f b c b o W represents the bias vector. i W f W c W o This represents the weight matrix of the hidden layer.

[0094] (2) Conditional Random Field (CRF)

[0095] BiLSTM outputs can contain meaningless characters, failing to consider dependencies between adjacent labels. For example, in named entity recognition, the label "I-BRAND" is positioned after "B-BRAND," making a label sequence like "O I-BRAND O" an invalid path, failing to meet task requirements. Conditional Random Fields (CRFs), as a discriminative model, add effective constraints between labels, significantly reducing the number of invalid predicted label sequences. CRFs predict the corresponding state sequence based on the input sequence, considering the current state features and dependencies between each label to find the optimal label sequence. The structure of a CRF is as follows... Figure 6 As shown.

[0096] In the CRF model, for the output sequence X = {x1, x2, ..., x} of the BiLSTM layer n Each of these sets of states has a corresponding candidate state sequence label Y. x The final label sequence Y = {y1, y2, ..., y3} is determined by calculating the score for each label sequence. n The calculation formula is shown in equation (1-7).

[0097]

[0098] Where P is the output score matrix of BiLSTM, with a size of n*k, where n is the number of words and k is the number of tags. i,j Let A represent the score of the j-th tag of the i-th word. Let A be a transition score matrix containing the sentence start and end tags, with a size of k+2. i,j The score representing the transition from label i to label j.

[0099] Finally, the scores of each label sequence are normalized and the probabilities are obtained. The label sequence with the highest probability is the final label sequence of the sentence. The normalization process is shown in formula (1-8).

[0100]

[0101] In step S102 of the present invention, the shortcomings of traditional question classification methods, such as rule-based and template matching regularization methods or machine learning methods, which suffer from poor generalization ability and high computational cost, are overcome. The present invention constructs a question semantic classification model based on RoBERTa-WWM-TextCNN: It uses RoBERTa-WWM as a foundation for context modeling and sentence-level semantic representation of questions. The vector corresponding to the "[CLS]" symbol output from the last layer of RoBERTa-WWM is used to represent the context of the question. Then, a fully connected TextCNN neural network is used to extract features from the questions before classification. The RoBERTa-WWM-TextCNN classification model consists of a word embedding layer, a convolutional layer, a max-pooling layer, and a fully connected layer. The model structure is as follows: Figure 7 As shown.

[0102] (1) Word embedding layer

[0103] The word embedding layer refers to the RoBERTa-WWM model converting the input text data into a fixed-length vector representation through character encoding, generating a word vector matrix. Where n is the length of the input sentence, and d is the dimension of the word vectors. Let Let represent the i-th word vector in the input. Then the input sentence X can be represented as X = [x1, x2, ..., x...]. n ].

[0104] (2) Convolutional layer

[0105] Convolutional layers are responsible for extracting the intrinsic features of the text. TextCNN uses one-dimensional convolutions, with convolution kernels... The width 'd' of the convolution kernel is the same as the dimension of the word vectors. 'h' is the height of the convolution kernel; this paper sets three convolution kernels with heights of 3, 4, and 5, with 1 input channel and 256 output channels. The convolution kernel 'w' is then compared with the i-th window in the word vector matrix E. Perform a convolution operation to obtain feature c. i The calculation formula is shown in (1-9).

[0106] c i =f(w·x) i:i+h-1 +b) (1-9)

[0107] Where f is the ReLU activation function and b represents the bias. The convolution kernel is convolved with the word vectors within all windows of the word vector matrix E to obtain multiple features, which are then concatenated to obtain an n-h+1 dimensional feature vector:

[0108] c = [c1, c2, ..., c n-h+1 (1-10)

[0109] (3) Pooling layer

[0110] Each obtained feature is subjected to max pooling in the pooling layer to reduce the data dimensionality and extract important feature information, as shown in formula (1-11). The results of pooling all convolutional kernels are then concatenated to obtain a new feature vector.

[0111] c m =max{c} (1-11)

[0112] (4) Fully connected layer

[0113] In the final fully connected layer, Dropout is added to prevent overfitting, and the Softmax function is used to output the probability results of each category. Finally, the category with the highest probability is output.

[0114] In step (3), the Neo4j graph database retrieval module constructs a Cypher query statement based on the result of the question semantic parsing and queries the corresponding result in the graph database. The specific steps are as follows:

[0115] After experiments with named entity recognition and question semantic classification, the generated model can be used for automatic question answering. The algorithm model identifies the entities and classification information in the user-input question, with the classification corresponding to relations in the knowledge graph. This converts the question into triples <entity, attribute, ?> that the Neo4j graph database can understand, where "?" represents the answer the user wants.

[0116] For the question "How much does the Xiaomi STYTJ02ZHM robot vacuum weigh?", the model predicts entity information such as "Xiaomi", "robot vacuum", and model "STYTJ02ZHM", as well as the semantic category "home appliances_product weight". Finally, based on this information, a Cypher query is constructed to retrieve the answer from the knowledge graph. An example of automatic question answering is shown in Table 4. If the user's input question lacks complete entity information, such as "How much does the Xiaomi STYTJ0 robot vacuum weigh?", the system performs a fuzzy search, filtering the results by combining other entity information in the question, such as "Xiaomi" and "robot vacuum", and then matching the remaining results with the question based on similarity, returning the answer with the highest matching degree to the user. If no entity or category information is obtained after parsing the question, the system automatically returns a user-friendly prompt.

[0117] Table 4

[0118]

[0119] At this point, users can execute the corresponding operations on home appliances based on the answers calculated and output by the system, thereby forming an effective decision.

[0120] The product question answering method based on RoBERTa-WWM proposed in this invention uses the RoBERTa-WWM Chinese pre-trained language model. First, based on the data characteristics of the product knowledge graph, a template rule-based question corpus generation method is designed to generate training corpus. The corpus is then labeled with BIO sequences and semantic classification tags for training the question answering model. Next, a question-based named entity recognition model based on RoBERTa-WWM-BiLSTM-CRF and a question-based semantic classification model based on RoBERTa-WWM-TextCNN are constructed to achieve semantic parsing of questions. Comparative experiments with other models verify the effectiveness of the proposed model. Finally, based on the semantic parsing results, a Cypher query is constructed to retrieve and obtain answers from the knowledge graph.

[0121] It should be noted that the RoBERTa-WWM pre-trained language model in this invention combines the advantages of Chinese Whole Word Masking (WWM) and the RoBERTa model, representing an improvement on the BERT model. While the network structure of RoBERTa-WWM is similar to BERT, consisting of multiple stacked Transformer layers, it makes several improvements in training data and methods: using larger-scale training data; removing the NSP (NextSentence Prediction) pre-training task; changing static masks to dynamic masks; and introducing Whole Word Masking (WWM). This invention introduces the RoBERTa-WWM Chinese pre-trained language model, which, trained on a large-scale Chinese dataset and combined with whole word masking, effectively learns the semantic information of Chinese words and phrases, making it highly suitable for processing Chinese NLP tasks. Based on this, a knowledge question answering method based on the RoBERTa-WWM model is proposed for product knowledge question answering tasks, capable of parsing the correct semantics of user questions, providing algorithmic support for subsequent home appliance product question answering systems.

[0122] It should be noted that the above description is not intended to limit the present invention, and the present invention is not limited to the examples given above. Any changes, modifications, additions or substitutions made by those skilled in the art within the scope of the present invention should also fall within the protection scope of the present invention.

Claims

1. A product question-and-answer method based on Robertia-WWM, characterized in that, include: Sequence labeling and semantic classification tagging are performed on the training corpus; Construct a RoBERTa-WWM-BiLSTM-CRF model to identify named entities in product questions; and construct a RoBERTa-WWM-TextCNN model to perform semantic classification of product questions. Construct a Cypher query statement to retrieve results from a graph database; Sort the query results and match them with the product questions, then output the matching results; The constructed RoBERTa-WWM-BiLSTM-CRF model consists of a RoBERTa-WWM Chinese pre-trained language model, a BiLSTM layer, and a CRF layer; Named entities that identify product questions include: RoBERTa-WWM is used to encode the input text, and the resulting sequence vector representation is passed to the BiLSTM layer. The BiLSTM layer performs semantic encoding on the sequence vector and outputs the score of each character corresponding to each category label. The CRF layer obtains the prediction sequence through the constraint information between adjacent labels, thereby obtaining the category of each character. The constructed RoBERTa-WWM-TextCNN model consists of word embedding layers, convolutional layers, max pooling layers, and fully connected layers; it performs semantic classification on product questions, specifically including: Based on RoBERTa-WWM, we perform context modeling and sentence-level semantic representation of questions. The vector corresponding to the "[CLS]" symbol output by the last layer of RoBERTa-WWM is used to represent the context of the question. Then, the question is classified after feature extraction by the fully connected TextCNN neural network.

2. The product question-and-answer method based on Robeta-WWM according to claim 1, characterized in that, The method further includes the step of generating training corpus, including: The entity, attribute, and relationship information from the product knowledge graph are extracted as the basic data for generating question corpus; Based on the question structure, question rule templates are constructed, and question templates are generated from different types of product entity data in the knowledge graph according to the question rule templates. The training corpus of questions is obtained by filling in the entity, attribute, and relationship information in the product knowledge graph according to the question template.

3. The product question-and-answer method based on Robeta-WWM according to claim 1, characterized in that, Semantic classification and labeling of the training corpus, including: The entity type with attributes is expanded into entity classes and entity-attribute classes, with each entity-attribute class serving as a subcategory to further subdivide the question categories.

4. The product question-and-answer method based on Roberta-WWM according to claim 1, characterized in that, Sequence labeling of the training corpus includes: The BIO annotation method is used to convert tags with overlapping characters into two new tags.

5. The product question-and-answer method based on Robeta-WWM according to claim 1, characterized in that, The method further includes: In generating question templates by generating question templates from different types of product entity data in the knowledge graph according to question rule templates, sentence subject replacement and synonym conversion are used to generalize and generate training corpus.

Images