A product question-answering method and device based on neural network model

Through the neural network model based on the Transformer model, the product corpus is constructed and pre-trained, the shortcomings of the Q&A system covering problems in the product field in the existing technology are solved, and the rapid and accurate response to product problems is achieved, and the efficiency of the product Q&A system is improved. and quality.

CN118585634BActive Publication Date: 2024-11-12WEAPON EQUIP RES INST OF CHINA NAT WEAPON EQUIP GRP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411069050.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-06
Publication Date
2024-11-12
Estimated Expiration
2044-08-06

AI Technical Summary

Technical Problem

The existing technology has limited coverage of question-and-answer systems in the product field, and it is difficult to quickly and accurately respond to problems in the product field.

Method used

A neural network model based on the Transformer model is adopted to crawl product corpus, build product corpus, use the TF-IDF model to generate word vectors, and pre-train the Transformer neural network model, and combine the Transformer encoder and decoder to generate answers.

Benefits of technology

It realizes quick and accurate responses to product field problems, improves the efficiency and quality of the product question-and-answer system, and enhances the understanding and response ability of product problems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118585634B_ABST
    Figure CN118585634B_ABST
Patent Text Reader

Abstract

The present invention discloses a product question-and-answer method and device based on a neural network model. The method comprises: forming a product corpus based on preprocessed product question-and-answer data; the TF‑IDF model takes the question and the real answer as input, generates a word vector corresponding to the input based on the input, and the word vector is a word vector of a question-and-answer pair consisting of the question and the real answer; generates an input vector based on the word vector, pre-trains a Transformer neural network model based on the input vector, and obtains a trained Transformer neural network model; obtains a question to be answered, the TF‑IDF model takes the question to be answered as input, generates a word vector corresponding to the input based on the input, and inputs the word vector into the trained Transformer neural network model to obtain an answer corresponding to the question to be answered. The method can make a relatively accurate and clear reply to product questions, thereby improving product efficiency and quality.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, and in particular, relates to a product question-answering method and device based on a neural network model. Background Art

[0002] Products are important products that meet user needs and solve problems. In order to satisfy users' convenient way of understanding products, quickly respond to user needs, improve user satisfaction, enhance user trust in products, and reduce labor costs, a product-oriented question-and-answer method is urgently needed, and there is currently no relatively mature Transformer model for the product field. Therefore, in order to solve the above problems, the present invention provides a product question-and-answer method and device based on the Transformer model, which expands the knowledge base by crawling product corpus, and establishes business question-and-answer pairs for product field content to cover a variety of product usage scenarios and issues that users are more concerned about. Summary of the invention

[0003] The present invention proposes a product question-and-answer method and device based on a neural network model, which is used to solve the limitations of the existing technology on the breadth and depth of question coverage of question-and-answer systems, and to achieve rapid and accurate technical responses to product field issues.

[0004] The present invention proposes a product question-answering method based on a neural network model, the method comprising:

[0005] Step S1: obtaining product question and answer data, preprocessing the product question and answer data, and forming a product corpus based on the preprocessed product question and answer data;

[0006] Step S2: Question-answer annotation is performed on several paragraphs in the product corpus, and the question-answer annotation includes questions and real answers; the TF-IDF model takes the questions and real answers as input, and generates a word vector corresponding to the input based on the input, and the word vector is a word vector of a question-answer pair consisting of the question and the real answer; an input vector is generated based on the word vector, and a Transformer neural network model is pre-trained based on the input vector to obtain a trained Transformer neural network model; wherein the Transformer neural network model includes a Transformer encoder and a Transformer decoder connected in sequence; the Transformer encoder The encoder encodes the input vector and obtains semantic information in the input vector, generates an intermediate vector corresponding to the context representation corresponding to the question, the context representation is multiple, the intermediate vector corresponds to the context representation one by one, and the intermediate vector is used as the first vector to be decoded; the first vector to be decoded is input into the Transformer decoder, the Transformer decoder fuses the first vector to be decoded with the input vector as the vector to be decoded, the Transformer decoder decodes the vector to be decoded to obtain multiple decoding result vectors, selects a decoding result vector from them, and outputs an answer corresponding to the question based on the selected decoding result vector;

[0007] Step S3: Obtain the question to be answered. The TF-IDF model takes the question to be answered as input, generates a word vector corresponding to the input based on the input, and inputs the word vector into the trained Transformer neural network model to obtain the answer corresponding to the question to be answered.

[0008] Preferably, the method comprises step S4: setting an evaluation index and using a cosine similarity model to evaluate the output result of the Transformer neural network model.

[0009] Preferably, the TF-IDF model generates a word vector corresponding to the input based on the input, including:

[0010] Step S11: extract keywords from the input, and perform the following operations for each keyword:

[0011] Calculate the keyword frequency TF, where TF is the number of times the keyword appears divided by the number of words in the input;

[0012] Calculate the inverse document frequency (IDF) of the keyword, where IDF is the total number of words in the input divided by the number of texts in the product corpus that include the keyword;

[0013] calculate , as the representation of the keyword, concatenating the representation of the keyword with the keyword to obtain the word vector corresponding to the keyword;

[0014] Step S12: taking the set of all word vectors as the word vector corresponding to the input.

[0015] Preferably, the Transformer encoder is composed of 6 encoder layers connected layer by layer, each encoder layer is composed of a multi-head self-attention mechanism module and a feedforward neural network module, the Transformer encoder is used to encode the input word vector, determine the semantic information in the input word vector, and generate an intermediate vector corresponding to the context representation corresponding to the word vector, the context representation is multiple, the intermediate vector corresponds to the context representation one by one, and the intermediate vector is used as the first vector to be decoded; the Transformer decoder includes multiple decoder layers connected layer by layer and with the same structure, each decoder layer includes two multi-head self-attention mechanism modules and a feedforward neural network module, the first vector to be decoded is input into the Transformer decoder, the first vector to be decoded is feature fused with the input vector as the vector to be decoded, each decoder layer of the Transformer decoder decodes the vector to be decoded layer by layer to obtain multiple decoding result vectors, selects a decoding result vector from them, and outputs an answer corresponding to the question based on the selected decoding result vector.

[0016] Preferably, the encoder layer adds attention to the input vector of the layer, and concatenates the input vector after adding attention with the head of the multi-head self-attention mechanism module of the encoder layer to generate a concatenated vector, wherein:

[0017]

[0018]

[0019] Self_Attention(q,k,v) is the self-attention mechanism in the self-attention mechanism module, q is the word vector corresponding to the question; k is all the word vectors formed from the product corpus, v is the actual value corresponding to all the word vectors formed from the product corpus, T is the transposition, d k is the word vector dimension, It is a multi-head self-attention mechanism module. is the i-th self-attention mechanism module in the multi-head self-attention mechanism module, with a dimension of d k , concat represents the connection operation, To splice the i-th self-attention mechanism module, is the set consisting of the i-th self-attention mechanism module, is a parameter matrix used to linearly transform the input vector and target vector of this layer. I is the number of self-attention mechanism modules in the multi-head self-attention mechanism module.

[0020] Preferably, the step of fusing the first vector to be decoded with the input vector to obtain the vector to be decoded includes:

[0021] The input vector and all intermediate vectors are weighted and summed according to the weights, and the calculation formula is:

[0022]

[0023] Where N is the total number of intermediate vectors, n is the number of the intermediate vector, is the nth intermediate vector output by the Transformer encoder, is the input vector, are all weights; each decoder layer of the Transformer decoder decodes the vector to be decoded layer by layer to obtain multiple decoding result vectors, that is, restore the scores of each intermediate vector; select a decoding result vector from them, and based on the selected decoding result vector, output an answer corresponding to the question, that is, the intermediate vector with the highest score is used as the answer, and the score is calculated as follows: N

[0024] in, is the score of the restored nth intermediate vector, The nth intermediate vector restored by the Transformer decoder.

[0025] The present invention proposes a product question-answering device based on a neural network model, the device comprising:

[0026] Preprocessing module: configured to obtain product question and answer data, preprocess the product question and answer data, and form a product corpus based on the preprocessed product question and answer data;

[0027] Training module: configured to perform question-answer annotation on several paragraphs in the product corpus, wherein the question-answer annotation includes questions and real answers; the TF-IDF model takes the questions and real answers as input, generates a word vector corresponding to the input based on the input, and the word vector is a word vector of a question-answer pair consisting of the question and the real answer; generates an input vector based on the word vector, and pre-trains the Transformer neural network model based on the input vector to obtain a trained Transformer neural network model; wherein the Transformer neural network model includes a Transformer encoder and a Transformer decoder connected in sequence; the Transformer The encoder encodes the input vector and obtains semantic information in the input vector, generates an intermediate vector corresponding to the context representation corresponding to the question, the context representation is multiple, the intermediate vector corresponds to the context representation one by one, and the intermediate vector serves as the first vector to be decoded; the first vector to be decoded is input into the Transformer decoder, the Transformer decoder performs feature fusion on the first vector to be decoded and the input vector as the vector to be decoded, the Transformer decoder decodes the vector to be decoded to obtain multiple decoding result vectors, selects a decoding result vector from them, and outputs an answer corresponding to the question based on the selected decoding result vector;

[0028] Answer acquisition module: configured to obtain questions to be answered. The TF-IDF model takes the questions to be answered as input, generates a word vector corresponding to the input based on the input, and inputs the word vector into the trained Transformer neural network model to obtain the answer corresponding to the question to be answered.

[0029] The present invention provides an electronic device, comprising:

[0030] at least one processor; and

[0031] a memory communicatively connected to the at least one processor; wherein,

[0032] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described above.

[0033] The present invention provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the method as described above.

[0034] The present invention first crawls product knowledge related corpus from Chinese and English online academic databases and product community websites, then performs data preprocessing on product knowledge, uses the processed word vector as input, puts it into the Transformer model for training, and finally puts the trained model into the server for application. The advantage of this is that the corpus is increased, so that the model can make more accurate and clear answers to product questions, improving product efficiency and quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A schematic diagram of the process of the product question-answering method based on the neural network model provided by the present invention;

[0036] Figure 2 A schematic diagram of the interface of the product question-answering method based on the neural network model provided by the present invention;

[0037] Figure 3 An interactive schematic diagram of the product question-answering method based on a neural network model provided by the present invention;

[0038] Figure 4 A schematic diagram of the architecture of a product question-and-answer device based on a neural network model provided by the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0040] Figure 1 A schematic diagram of a product question-answering method based on a neural network model is shown. It should be understood that in other embodiments, the order of some steps can be interchanged according to actual needs, or some steps can be omitted or deleted. Figure 1 As shown, the method includes:

[0041] Step S1: obtaining product question and answer data, preprocessing the product question and answer data, and forming a product corpus based on the preprocessed product question and answer data;

[0042] Step S2: Annotate several paragraphs in the product corpus with questions and answers, wherein the question and answer annotations include questions and true answers; the TF-IDF model takes the questions and true answers as input, and generates a word vector corresponding to the input based on the input, wherein the word vector is a word vector of a question-answer pair consisting of the question and the true answer; an input vector is generated based on the word vector, and a Transformer neural network model is pre-trained based on the input vector to obtain a trained Transformer neural network model; wherein the Transformer neural network model includes a Transformer encoder and a Transformer decoder connected in sequence; the Transformer encoder is a Transformer decoder. The input vector is encoded, and semantic information in the input vector is obtained to generate an intermediate vector corresponding to the context representation corresponding to the question, wherein the context representation is multiple, and the intermediate vector corresponds to the context representation one by one, and the intermediate vector is used as a first vector to be decoded; the first vector to be decoded is input into the Transformer decoder, and the Transformer decoder performs feature fusion on the first vector to be decoded and the input vector as the vector to be decoded, and the Transformer decoder decodes the vector to be decoded to obtain multiple decoding result vectors, selects a decoding result vector from them, and outputs an answer corresponding to the question based on the selected decoding result vector;

[0043] Step S3: Obtain the question to be answered. The TF-IDF model takes the question to be answered as input, generates a word vector corresponding to the input based on the input, and inputs the word vector into the trained Transformer neural network model to obtain the answer corresponding to the question to be answered.

[0044] Furthermore, the method includes step S4: setting an evaluation index and using a cosine similarity model to evaluate the result output by the Transformer neural network model.

[0045] Furthermore, the method includes deploying the trained Transformer neural network model on a server and providing a human-computer interaction interface to the outside.

[0046] The product question and answer data is preprocessed, including: converting the product knowledge into a text format, removing stop words, performing text segmentation, and performing information retrieval to form a product corpus.

[0047] Specifically, for text format conversion, the present invention unifies the format of crawled texts and converts texts in PDF and CAJ formats into Word formats. For HTML documents, regular expressions are used for parsing and processing. First, HTML tags are removed, for example, the pattern of <[^>]+> is used to match all tags and replace them with empty strings. Second, specific information is extracted using<a\s+href=[“’]([^”’]+)[“’]> The pattern matches all Tags and extract href attribute values; finally, verify and filter the extracted information to improve the reliability of the information.

[0048] Specifically, by removing stop words, the present invention removes adverbs, function words or sentiment words that have no actual meaning in natural language, which can not only reduce storage space and computing costs, but also improve the efficiency and accuracy of text processing and the response speed of the system.

[0049] Specifically, for text segmentation, the present invention adopts jieba segmentation and sets the product content to be segmented according to paragraphs, but is not limited to this method or technology or tool for Chinese segmentation of natural language. For example, methods or technologies based on conjunctions and special punctuation marks, word frequency grouping, HanLP segmenter, etc. are all within the protection scope of this invention.

[0050] The TF-IDF model integrates the TF-IDF (Term Frequency-Inverse Document Frequency) algorithm, which mines the features of keywords that appear frequently in the product corpus but less frequently in the entire document collection, and is used to evaluate the importance of a term to a specific document in the entire document collection.

[0051] The TF-IDF model generates a word vector corresponding to the input based on the input, including:

[0052] Step S11: extract keywords from the input, and perform the following operations for each keyword:

[0053] Calculate the keyword frequency TF, where TF is the number of times the keyword appears divided by the number of words in the input;

[0054] Calculate the inverse document frequency (IDF) of the keyword, where IDF is the total number of words in the input divided by the number of texts in the product corpus that include the keyword;

[0055] calculate , as the representation of the keyword, concatenating the representation of the keyword with the keyword to obtain the word vector corresponding to the keyword;

[0056] Step S12: taking the set of all word vectors as the word vector corresponding to the input.

[0057] In the present invention, when a word rarely appears in a text but is relatively common in the entire document collection, the TF-IDF will be very low, otherwise, the TF-IDF will be very high.

[0058] In the present invention, information retrieval is not limited to this method of processing documents, for example: BM25 model, LDA model, etc., and any method that improves retrieval accuracy is within the protection scope of this invention.

[0059] The Transformer encoder is composed of 6 encoder layers connected layer by layer, each encoder layer is composed of a multi-head self-attention mechanism module and a feedforward neural network module, the Transformer encoder is used to encode the input word vector, determine the semantic information in the input word vector, and generate an intermediate vector corresponding to the context representation corresponding to the word vector, the context representation is multiple, the intermediate vector corresponds to the context representation one by one, and the intermediate vector is used as the first vector to be decoded; the Transformer decoder includes multiple decoder layers connected layer by layer and with the same structure, each decoder layer includes two multi-head self-attention mechanism modules and a feedforward neural network module, the first vector to be decoded is input into the Transformer decoder, the first vector to be decoded is feature fused with the input vector as the vector to be decoded, each decoder layer of the Transformer decoder decodes the vector to be decoded layer by layer to obtain multiple decoding result vectors, selects a decoding result vector from them, and outputs an answer corresponding to the question based on the selected decoding result vector.

[0060] For the Transformer encoder, it obtains the context of the candidate answers of the input and captures the dependencies between different positions in the input sequence.

[0061] The encoder layer adds attention to the input vector of the layer, and concatenates the input vector after adding attention with the head of the multi-head self-attention mechanism module of the encoder layer to generate a concatenated vector, where:

[0062]

[0063]

[0064] Self_Attention(q,k,v) is the self-attention mechanism in the self-attention mechanism module, q is the word vector corresponding to the question; k is all the word vectors formed from the product corpus, v is the actual value corresponding to all the word vectors formed from the product corpus, T is the transposition, d k is the word vector dimension, It is a multi-head self-attention mechanism module. is the i-th self-attention mechanism module in the multi-head self-attention mechanism module, with a dimension of d k , concat represents the connection operation, To splice the i-th self-attention mechanism module, is the set consisting of the i-th self-attention mechanism module, is a parameter matrix used to linearly transform the input vector and target vector of this layer. I is the number of self-attention mechanism modules in the multi-head self-attention mechanism module.

[0065] In the present invention, the self-attention mechanism is a mechanism for processing sequence data, which can map an input sequence to an output sequence. Its advantage is that different correlations are considered at the same time, thereby improving the generalization ability of the model; the feedforward neural network projects the vector refined by the self-attention mechanism to a larger space, in which the RELU activation function can be used to extract useful information, and finally projects it to the original space of the word vector. The calculation formula is as follows:

[0066]

[0067] Encoder(x)=EncoderLayer(EncoderLayer(...(EncoderLayer(x))))

[0068] Among them, x represents the input word vector, and LayerNorm represents the layer normalization operation.

[0069] The step of fusing the first to-be-decoded vector with the input vector to obtain a to-be-decoded vector includes:

[0070] The input vector and all intermediate vectors are weighted and summed according to the weights, and the calculation formula is:

[0071]

[0072] Where N is the total number of intermediate vectors, n is the number of the intermediate vector, is the nth intermediate vector output by the Transformer encoder, is the input vector, All are weights;

[0073] Each decoder layer of the Transformer decoder decodes the vector to be decoded layer by layer to obtain multiple decoding result vectors, that is, restore the scores of each intermediate vector; select a decoding result vector from them, and based on the selected decoding result vector, output an answer corresponding to the question, that is, take the intermediate vector with the highest score as the answer, and the score is calculated as follows:

[0074] N

[0075] in, is the score of the restored nth intermediate vector, The nth intermediate vector restored by the Transformer decoder.

[0076] In this embodiment, the TF-IDF model inputs the question for encoding, calculates the TF-IDF value of each keyword in the question, and composes them into word vectors to obtain Representation in space.

[0077] In the present invention, the intermediate vector corresponds to the candidate answer. Passed as input to the Transformer neural network model, for example, to generate 10 candidate answers, the score is calculated as follows:

[0078] .

[0079] Select the candidate answer with the highest Weight_Sum score and output it as the final answer.

[0080] In this invention, the answer generation uses the Transformer decoder to generate the answer based on the semantic representation of the question and the context information. Through the self-attention mechanism and position encoding, the key information of the question and the semantics of the context are considered when generating the answer. The calculation formula is as follows:

[0081]

[0082] Decoder(x,Enc_out)=DecLayer(DecLayer(...(Dec_Layer(x,Enc_out))))

[0083] Where x represents the input vector and Enc_out represents the decoder output.

[0084] In the present invention, the Transformer decoder processes the answer corresponding to the question through a linear transformation layer and a normalization layer to obtain the final result. Among them, the linear transformation layer is used to map the intermediate representation to a vector space and output a vector. The vector can be used for regression or classification tasks, or as an intermediate output result of the model; the normalization layer is used to normalize the output vector of the linear transformation layer into a probability distribution. This probability distribution is used for classification tasks, and the probability of each category can be used to predict the probability that the sample belongs to that category. The Transformer neural network model uses the output of the encoder as the input of the decoder, and the calculation formula is as follows:

[0085] .

[0086] In the present invention, a bidirectional Transformer neural network model is used for pre-training. First, a keyword sequence set is constructed based on the preprocessed product corpus and converted into word embedding. During the pre-training, some paragraphs are annotated for question and answer using manual annotation. Finally, transfer learning is performed on all paragraphs in the library. After the pre-training is completed, it can be applied to the product question and answer step. For candidate answers, the Transformer neural network model can quickly use the self-attention mechanism to encode the words in the sequence, capture the full set of information and perform calculations, and finally return the highest matching and unique answer.

[0087] Optionally, the present invention can also use a Word2vec model, a Se2seq model, a GPT model, an XLNet model, or a T5 model for training to generate candidate answers.

[0088] The step S4: setting evaluation indicators, using the cosine similarity model to evaluate the output of the Transformer neural network model, wherein: using the semantic similarity loss function for evaluation and verification, the formula is as follows:

[0089]

[0090] in, represents the dot product of the output answer A and the true answer B, ||A|| and ||B|| represent the norm of the output answer A and the true answer B respectively (that is, the length of the answer).

[0091] The specific implementation method is to input the questions with known answers in the test set into the trained Transformer model to obtain the answer representation vector of the question. By experimenting and adjusting the question and answer data with known answers in the test set, observing the performance of the Transformer model under different thresholds, and selecting the most appropriate threshold. When the cosine similarity loss function exceeds the threshold, the question and answer are considered to match.

[0092] The present invention includes model deployment and application. Based on the simulation training results of the product question and answer model, it is determined whether the product model has passed the test. When the product question and answer model has passed the test, the product question and answer model is deployed on the server for application and accessed by calling the interface.

[0093] The following is a description of the device used to execute the present application. The specific implementation process and technical effects are described above and will not be repeated below.

[0094] Optionally, an embodiment of the present application provides a product question-answering device based on a neural network model, the device comprising:

[0095] Preprocessing module: configured to obtain product question and answer data, preprocess the product question and answer data, and form a product corpus based on the preprocessed product question and answer data;

[0096] Training module: configured to perform question-answer annotation on several paragraphs in the product corpus, wherein the question-answer annotation includes questions and real answers; the TF-IDF model takes the questions and real answers as input, generates a word vector corresponding to the input based on the input, and the word vector is a word vector of a question-answer pair consisting of the question and the real answer; generates an input vector based on the word vector, and pre-trains the Transformer neural network model based on the input vector to obtain a trained Transformer neural network model; wherein the Transformer neural network model includes a Transformer encoder and a Transformer decoder connected in sequence; the Transformer The encoder encodes the input vector and obtains semantic information in the input vector, generates an intermediate vector corresponding to the context representation corresponding to the question, the context representation is multiple, the intermediate vector corresponds to the context representation one by one, and the intermediate vector serves as the first vector to be decoded; the first vector to be decoded is input into the Transformer decoder, the Transformer decoder performs feature fusion on the first vector to be decoded and the input vector as the vector to be decoded, the Transformer decoder decodes the vector to be decoded to obtain multiple decoding result vectors, selects a decoding result vector from them, and outputs an answer corresponding to the question based on the selected decoding result vector;

[0097] Answer acquisition module: configured to obtain questions to be answered. The TF-IDF model takes the questions to be answered as input, generates a word vector corresponding to the input based on the input, and inputs the word vector into the trained Transformer neural network model to obtain the answer corresponding to the question to be answered.

[0098] The present invention provides a specific embodiment of a product question-answering device based on a neural network model, comprising:

[0099] The automatic question-answering module is used to answer questions through text and supports multiple question types, including factual questions, subjective questions, definitional questions, operational questions, and reasoning questions.

[0100] The semantic understanding module is used to process the input natural language and extract the key information and semantic structure of the question, including sub-modules such as text segmentation, stop word removal, and feature extraction.

[0101] The knowledge base module is structured and annotated to store and process natural language to facilitate question matching and answer generation, including knowledge related to test methods, test techniques and test tools.

[0102] The answer matching module is used to match the input question with the answers in the knowledge base and calculate the similarity. By comparing the scores of different answers, the most relevant answer is found.

[0103] The above-mentioned device is used to execute the method provided by the aforementioned embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0104] The above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASIC), or one or more digital singnal processors (DSP), or one or more field programmable gate arrays (FPGA). For another example, when a module is implemented in the form of a processing element scheduling program code, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0105] The above modules can be connected or communicated with each other via a wired connection or a wireless connection. The wired connection may include a metal cable, an optical cable, a hybrid cable, etc., or any combination thereof. The wireless connection may include a connection in the form of a LAN, a WAN, Bluetooth, a ZigBee, or NFC, or any combination thereof. Two or more modules may be combined into a single module, and any one module may be divided into two or more units. Those skilled in the art will clearly understand that for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the method embodiment, and will not be repeated in this application.

[0106] It should be noted that the above modules can be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASIC), or one or more microprocessors (DSP), or one or more field programmable gate arrays (FPGA). For example, when a module is implemented in the form of a processing element scheduling program code, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For example, these modules can be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0107] The electronic device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the electronic device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, near field communication (NFC) or other technologies. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covering the display screen, or a key, trackball or touchpad set on the housing of the electronic device, or an external keyboard, touchpad or mouse, etc.

[0108] The present application also provides a program product, such as a computer-readable storage medium, including a program, which is used to execute the above method embodiment when executed by a processor.

[0109] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0110] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0111] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0112] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (English: processor) to perform some steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (English: Read-Only Memory, referred to as: ROM), random access memory (English: Random Access Memory, referred to as: RAM), disk or optical disk and other media that can store program codes.

Claims

1. A product question-answering method based on a neural network model, characterized in that: The method comprises the following steps: Step S1: obtaining product question and answer data, preprocessing the product question and answer data, and forming a product corpus based on the preprocessed product question and answer data; Step S2: performing question-answer annotation on several paragraphs in the product corpus, wherein the question-answer annotation includes questions and real answers; The TF-IDF model takes the question and the true answer as input, generates a word vector corresponding to the input based on the input, and the word vector is a word vector of a question-answer pair consisting of the question and the true answer; generates an input vector based on the word vector, and pre-trains the Transformer neural network model based on the input vector to obtain a trained Transformer neural network model; wherein the Transformer neural network model includes a Transformer encoder and a Transformer decoder connected in sequence; the Transformer encoder encodes the input vector, obtains semantic information in the input vector, and generates an intermediate vector corresponding to the context representation corresponding to the question, the context representation is multiple, the intermediate vector corresponds to the context representation one by one, and the intermediate vector is used as the first vector to be decoded; the first vector to be decoded is input into the Transformer decoder, the Transformer decoder fuses the first vector to be decoded with the input vector for features, as the vector to be decoded, the Transformer decoder decodes the vector to be decoded, obtains multiple decoding result vectors, selects a decoding result vector from them, and outputs an answer corresponding to the question based on the selected decoding result vector; Step S3: obtaining a question to be answered, the TF-IDF model takes the question to be answered as input, generates a word vector corresponding to the input based on the input, inputs the word vector into the trained Transformer neural network model, and obtains an answer corresponding to the question to be answered; The TF-IDF model generates a word vector corresponding to the input based on the input, including: Step S11: extract keywords from the input, and for each keyword, perform the following operations: Calculate the keyword frequency TF, where TF is the number of times the keyword appears divided by the number of words in the input; Calculate the inverse document frequency (IDF) of the keyword, where IDF is the total number of words in the input divided by the number of texts in the product corpus that include the keyword; Calculate TF*IDF as the representation of the keyword, and concatenate the keyword representation with the keyword to obtain the word vector corresponding to the keyword; Step S12: taking the set of all word vectors as the word vector corresponding to the input; The Transformer encoder is composed of 6 encoder layers connected layer by layer, each encoder layer includes a multi-head self-attention mechanism module and a feedforward neural network module; the Transformer encoder is used to encode the input word vector, determine the semantic information in the input word vector, and generate an intermediate vector corresponding to the context representation corresponding to the word vector, the context representation is multiple, the intermediate vector corresponds to the context representation one by one, and the intermediate vector is used as the first vector to be decoded; the Transformer decoder includes multiple decoder layers connected layer by layer and with the same structure, each decoder layer includes two multi-head self-attention mechanism modules and a feedforward neural network module; the first vector to be decoded is input into the Transformer decoder, the first vector to be decoded is feature fused with the input vector as the vector to be decoded, each decoder layer of the Transformer decoder decodes the vector to be decoded layer by layer to obtain multiple decoding result vectors, selects a decoding result vector from them, and outputs an answer corresponding to the question based on the selected decoding result vector; The encoder layer adds attention to the input vector of the layer, and concatenates the input vector after adding attention with the head of the multi-head self-attention mechanism module of the encoder layer to generate a concatenated vector, where: MultiHead q,k,v =Concat{head_i}*ω o ,i=1,...,I Self_Attention(q,k,v) is the self-attention mechanism in the self-attention mechanism module, q is the word vector corresponding to the question; k is all the word vectors formed from the product corpus, v is the actual value corresponding to all the word vectors formed from the product corpus, T is the transposition, d k is the word vector dimension, MultiHead q,k,v is a multi-head self-attention mechanism module, head_i is the i-th self-attention mechanism module in the multi-head self-attention mechanism module, and the dimension is d k , concat represents the connection operation, Concat{head_i} is to concatenate {head_i}, {head_i} is the set composed of the i-th self-attention mechanism module, ω o is a parameter matrix used to linearly transform the input vector and target vector of this layer, and I is the number of self-attention mechanism modules in the multi-head self-attention mechanism module; The step of fusing the first to-be-decoded vector with the input vector to obtain the to-be-decoded vector includes: The input vector and all intermediate vectors are weighted and summed according to the weights, and the calculation formula is: Where N is the total number of intermediate vectors, n is the number of the intermediate vector, and Encoder n is the nth intermediate vector output by the Transformer encoder, is the input vector, ω n ,ω m All are weights; Each decoder layer of the Transformer decoder decodes the vector to be decoded layer by layer to obtain multiple decoding result vectors, that is, restore the scores of each intermediate vector; select a decoding result vector from them, and based on the selected decoding result vector, output an answer corresponding to the question, that is, take the intermediate vector with the highest score as the answer, and the score is calculated as follows: Among them, Score n is the score of the restored nth intermediate vector, Decoder n The nth intermediate vector restored by the Transformer decoder.

2. The method according to claim 1, characterized in that The method comprises step S4: setting an evaluation index and using a cosine similarity model to evaluate the result output by the Transformer neural network model.

3. A product question-answering device based on a neural network model, characterized in that: The device comprises: Preprocessing module: configured to obtain product question and answer data, preprocess the product question and answer data, and form a product corpus based on the preprocessed product question and answer data; Training module: configured to perform question-answer annotation on several paragraphs in the product corpus, wherein the question-answer annotation includes questions and real answers; the TF-IDF model takes the questions and real answers as input, generates a word vector corresponding to the input based on the input, and the word vector is a word vector of a question-answer pair consisting of the question and the real answer; generates an input vector based on the word vector, and pre-trains the Transformer neural network model based on the input vector to obtain a trained Transformer neural network model; wherein the Transformer neural network model includes a Transformer encoder and a Transformer decoder connected in sequence; the Transformer The encoder encodes the input vector and obtains semantic information in the input vector, generates an intermediate vector corresponding to the context representation corresponding to the question, the context representation is multiple, the intermediate vector corresponds to the context representation one by one, and the intermediate vector serves as the first vector to be decoded; the first vector to be decoded is input into the Transformer decoder, the Transformer decoder performs feature fusion on the first vector to be decoded and the input vector as the vector to be decoded, the Transformer decoder decodes the vector to be decoded to obtain multiple decoding result vectors, selects a decoding result vector from them, and outputs an answer corresponding to the question based on the selected decoding result vector; Answer acquisition module: configured to obtain questions to be answered. The TF-IDF model takes the questions to be answered as input, generates a word vector corresponding to the input based on the input, and inputs the word vector into the trained Transformer neural network model to obtain the answer corresponding to the question to be answered. The TF-IDF model generates a word vector corresponding to the input based on the input, including: Extract keywords from the input and for each keyword, perform the following operations: Calculate the keyword frequency TF, where TF is the number of times the keyword appears divided by the number of words in the input; Calculate the inverse document frequency (IDF) of the keyword, where IDF is the total number of words in the input divided by the number of texts in the product corpus that include the keyword; Calculate TF*IDF as the representation of the keyword, and concatenate the keyword representation with the keyword to obtain the word vector corresponding to the keyword; The set of all word vectors is used as the word vector corresponding to the input; The Transformer encoder is composed of 6 encoder layers connected layer by layer, each encoder layer includes a multi-head self-attention mechanism module and a feedforward neural network module; the Transformer encoder is used to encode the input word vector, determine the semantic information in the input word vector, and generate an intermediate vector corresponding to the context representation corresponding to the word vector, the context representation is multiple, the intermediate vector corresponds to the context representation one by one, and the intermediate vector is used as the first vector to be decoded; the Transformer decoder includes multiple decoder layers connected layer by layer and with the same structure, each decoder layer includes two multi-head self-attention mechanism modules and a feedforward neural network module; the first vector to be decoded is input into the Transformer decoder, the first vector to be decoded is feature fused with the input vector as the vector to be decoded, each decoder layer of the Transformer decoder decodes the vector to be decoded layer by layer to obtain multiple decoding result vectors, selects a decoding result vector from them, and outputs an answer corresponding to the question based on the selected decoding result vector; The encoder layer adds attention to the input vector of the layer, and concatenates the input vector after adding attention with the head of the multi-head self-attention mechanism module of the encoder layer to generate a concatenated vector, where: MultiHead q,k,v =Concat{head_i}*ω o ,i=1,...,I Self_Attention(q,k,v) is the self-attention mechanism in the self-attention mechanism module, q is the word vector corresponding to the question; k is all the word vectors formed from the product corpus, v is the actual value corresponding to all the word vectors formed from the product corpus, T is the transposition, d k is the word vector dimension, MultiHead q,k,v is a multi-head self-attention mechanism module, head_i is the i-th self-attention mechanism module in the multi-head self-attention mechanism module, and the dimension is d k , concat represents the connection operation, Concat{head_i} is to concatenate {head_i}, {head_i} is the set composed of the i-th self-attention mechanism module, ω o is a parameter matrix used to linearly transform the input vector and target vector of this layer, and I is the number of self-attention mechanism modules in the multi-head self-attention mechanism module; The step of fusing the first to-be-decoded vector with the input vector to obtain the to-be-decoded vector includes: The input vector and all intermediate vectors are weighted and summed according to the weights, and the calculation formula is: Where N is the total number of intermediate vectors, n is the number of the intermediate vector, and Encoder n is the nth intermediate vector output by the Transformer encoder, is the input vector, ω n ,ω m All are weights; Each decoder layer of the Transformer decoder decodes the vector to be decoded layer by layer to obtain multiple decoding result vectors, that is, restore the scores of each intermediate vector; select a decoding result vector from them, and based on the selected decoding result vector, output an answer corresponding to the question, that is, take the intermediate vector with the highest score as the answer, and the score is calculated as follows: Among them, Score n is the score of the restored nth intermediate vector, Decoder n The nth intermediate vector restored by the Transformer decoder.

4. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 2.

5. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 2.