Intelligent reply method and device, electronic equipment and storage medium
Through the intelligent reply method, feature extraction and weight operation are used, combined with semantics and character features, efficient, accurate and automatic Q&A in the fields of financial consulting and other fields is achieved, and the problem of low efficiency and accuracy of manual customer service is solved.
Patent Information
- Application Number
- CN202411075134.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-06
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-08-06
AI Technical Summary
In the prior art, when manual customer service handles a large number of user consultations in financial consulting and other fields, it has low efficiency and low accuracy, especially when facing untrained questions, the response error rate increases, resulting in failure to respond in time and low accuracy rate. The problem.
Using intelligent replies, by obtaining the question text set and the reply text set, feature extraction, word weight allocation, weight summing and mask splicing are performed, and the matching target reply text is found from the reply text set in combination with semantics and character features to realize automatic question and answer .
It improves the efficiency and accuracy of reply, reduces the error rate of manual customer service, and can respond quickly and accurately to user consultations under high load conditions.
Smart Images

Figure CN119066167A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence technology, and in particular to an intelligent reply method and device, an electronic device and a storage medium. Background Art
[0002] At present, in the question-and-answer field of the service industry, it is usually done manually to respond to users' inquiries or questions. For example, in the field of financial consulting, a large number of manual customer service staff are required to respond to each question or inquiry reported by users. However, this method is based on manual customer service staff to respond to users' questions. When there are too many users inquiring, users may not be responded to in a timely manner, resulting in low response efficiency. In addition, when manual customer service staff encounter questions that they have not been trained on, the error rate of manual customer service staff's responses increases, resulting in a low accuracy rate of responses. Therefore, how to improve the efficiency and accuracy of responses has become an urgent problem to be solved. Summary of the invention
[0003] The main purpose of the embodiments of the present application is to propose an intelligent reply method and device, an electronic device and a storage medium, aiming to improve the efficiency and accuracy of the reply.
[0004] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application proposes an intelligent reply method, the method comprising:
[0005] Obtain a question text set and an answer text set;
[0006] Extracting features from the question text set to obtain a question sentence feature set, and extracting features from the answer text set to obtain an answer sentence feature set;
[0007] Performing word weight assignment on the question sentence feature set and the answer sentence feature set to obtain weighted word weight features;
[0008] Performing weighted summation on the question sentence feature set according to the weighted word weight feature to obtain the question sentence weight feature, and performing weighted summation on the answer sentence feature set according to the weighted word weight feature to obtain the answer sentence weight feature;
[0009] Performing a mask splicing operation on the question sentence weight feature and the answer sentence weight feature to obtain a whole sentence weight feature;
[0010] According to the question sentence weight feature, the answer sentence weight feature and the whole sentence weight feature, a target answer text matching the question text set is searched from the answer text set, and the question text set is answered according to the target answer text.
[0011] In some embodiments, the step of assigning word weights to the question sentence feature set and the answer sentence feature set to obtain weighted word weight features includes:
[0012] Calculating word weights on the question sentence feature set and the answer sentence feature set using the trained word weight model to obtain word weight features;
[0013] Obtaining a question word vector set corresponding to the question sentence feature set, and obtaining a reply word vector set corresponding to the reply sentence feature set;
[0014] A weighted operation is performed on the question word vector set, the answer word vector set and the word weight feature to obtain the weighted word weight feature.
[0015] In some embodiments, performing weighted summation on the question sentence feature set according to the weighted word weight feature to obtain the question sentence weight feature, and performing weighted summation on the answer sentence feature set according to the weighted word weight feature to obtain the answer sentence weight feature, comprises:
[0016] Obtaining the question sentence length of the question sentence feature set and the number of question words contained in the question sentence length, and obtaining the reply sentence length of the reply sentence feature set and the number of reply words contained in the reply sentence length;
[0017] The question sentence weight feature is obtained by performing an average weighting operation on the question sentence feature set according to the weighted word weight feature, the question sentence length and the number of question words, and the reply sentence weight feature is obtained by performing an average weighting operation on the reply sentence feature set according to the weighted word weight feature, the reply sentence length and the number of reply words.
[0018] In some embodiments, performing a mask splicing operation on the question sentence weight feature and the answer sentence weight feature to obtain the whole sentence weight feature includes:
[0019] Obtain a question boundary identifier and a question filling identifier of the question sentence weight feature, and obtain a reply boundary identifier and a reply filling identifier of the reply sentence weight feature;
[0020] Performing a masking operation on the question boundary identifier and the question filling identifier to obtain a masked question sentence weight feature, and performing a masking operation on the answer boundary identifier and the answer filling identifier to obtain a masked answer sentence weight feature;
[0021] The masked question sentence weight feature and the masked answer sentence weight feature are concatenated to obtain the whole sentence weight feature.
[0022] In some embodiments, extracting features from the question text set to obtain a question sentence feature set, and extracting features from the answer text set to obtain a answer sentence feature set, includes:
[0023] Performing encoding operation on the question text set to obtain a question word vector set, and performing encoding operation on the answer text set to obtain an answer word vector set;
[0024] A pooling operation is performed on the question word vector set to obtain the question sentence feature set, and a pooling operation is performed on the answer word vector set to obtain the answer sentence feature set.
[0025] In some embodiments, encoding the question text set to obtain a question word vector set, and encoding the answer text set to obtain an answer word vector set, include:
[0026] The question text set is segmented by a preset identifier to obtain a question word set, and the answer text set is segmented by a preset identifier to obtain an answer word set;
[0027] A word embedding operation is performed on the question word set to obtain the question word vector, and a word embedding operation is performed on the answer word set to obtain the answer word vector.
[0028] In some embodiments, searching for a target answer text matching the question text set from the answer text set according to the question sentence weight feature, the answer sentence weight feature, and the whole sentence weight feature includes:
[0029] Calculate the semantic matching degree between the question sentence feature set and the answer sentence feature set according to the question sentence weight feature, the answer sentence weight feature and the whole sentence weight feature;
[0030] According to the semantic matching degree, a target reply text matching the question text set is searched from the reply text set.
[0031] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides an intelligent reply device, the device comprising:
[0032] A text acquisition module, used to acquire a question text set and an answer text set;
[0033] A feature extraction module, used for extracting features from the question text set to obtain a question sentence feature set, and extracting features from the answer text set to obtain an answer sentence feature set;
[0034] A weight allocation module, used for allocating word weights to the question sentence feature set and the answer sentence feature set to obtain weighted word weight features;
[0035] A sentence weighting module, used for performing weighted summation on the question sentence feature set according to the weighted word weight feature to obtain the question sentence weight feature, and performing weighted summation on the answer sentence feature set according to the weighted word weight feature to obtain the answer sentence weight feature;
[0036] A mask splicing module, used for performing a mask splicing operation on the weight feature of the question sentence and the weight feature of the answer sentence to obtain a weight feature of the whole sentence;
[0037] The intelligent reply module is used to find the target reply text matching the question text set from the reply text set according to the question sentence weight feature, the reply sentence weight feature and the whole sentence weight feature, and reply to the question text set according to the target reply text.
[0038] To achieve the above objectives, a third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in the first aspect is implemented.
[0039] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.
[0040] The intelligent reply method and device, electronic device and storage medium proposed in the present application obtain a question text set and a reply text set; perform feature extraction on the question text set to obtain a question sentence feature set, and perform feature extraction on the reply text set to obtain a reply sentence feature set; perform word weight assignment on the question sentence feature set and the reply sentence feature set to obtain a weighted word weight feature; perform weighted summation on the question sentence feature set according to the weighted word weight feature to obtain a question sentence weight feature, and perform weighted summation on the reply sentence feature set according to the weighted word weight feature to obtain a reply sentence weight feature; perform masking and splicing operations on the question sentence weight feature and the reply sentence weight feature to obtain a whole sentence weight feature; search for a target reply text that matches the question text set from the reply text set according to the question sentence weight feature, the reply sentence weight feature and the whole sentence weight feature, and reply to the question text set according to the target reply text. The present application achieves mutual matching of question and answer text and reply text by combining the word weights, question sentence weight features, reply sentence weight features and whole sentence weight features of the question text and reply text, and realizes automatic reply to the question and answer text by combining semantics and character features, thereby improving the efficiency and accuracy of the reply. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] Figure 1 is a flow chart of the intelligent reply method provided by an embodiment of the present application;
[0042] Figure 2 yes Figure 1 Flow chart of step S102 in FIG.
[0043] Figure 3 yes Figure 2 Flow chart of step S201 in FIG.
[0044] Figure 4 yes Figure 1 Flow chart of step S103 in FIG.
[0045] Figure 5 yes Figure 1 Flow chart of step S104 in FIG.
[0046] Figure 6 yes Figure 1 Flow chart of step S105 in FIG.
[0047] Figure 7 yes Figure 1 Flow chart of step S106 in FIG.
[0048] Figure 8 is a schematic diagram of the structure of the intelligent reply device provided in an embodiment of the present application;
[0049] Fig. 9 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION
[0050] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0051] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.
[0052] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.
[0053] First, some nouns involved in this application are analyzed:
[0054] Artificial intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing and expert systems. AI can simulate the information process of human consciousness and thinking. AI is also a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.
[0055] Based on this, the embodiments of the present application provide an intelligent reply method and device, an electronic device and a storage medium, aiming to improve the efficiency and accuracy of the reply.
[0056] The intelligent reply method and device, electronic device and storage medium provided in the embodiments of the present application are specifically illustrated through the following embodiments. First, the intelligent reply method in the embodiments of the present application is described.
[0057] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.
[0058] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.
[0059] The intelligent reply method provided in the embodiment of the present application relates to the field of artificial intelligence technology. The intelligent reply method provided in the embodiment of the present application can be applied to the terminal, can also be applied to the server side, and can also be software running in the terminal or the server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or it can be configured as a server cluster or distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the intelligent reply method, etc., but is not limited to the above forms.
[0060] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.
[0061] Figure 1 is an optional flowchart of the intelligent reply method provided in the embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S106.
[0062] Step S101, obtaining a question text set and an answer text set.
[0063] Step S102, extracting features from the question text set to obtain a question sentence feature set, and extracting features from the answer text set to obtain a answer sentence feature set.
[0064] Step S103, assigning word weights to the question sentence feature set and the answer sentence feature set to obtain weighted word weight features.
[0065] Step S104, weighted summing up the question sentence feature set according to the weighted word weight feature to obtain the question sentence weight feature, and weighted summing up the answer sentence feature set according to the weighted word weight feature to obtain the answer sentence weight feature.
[0066] Step S105, performing mask concatenation operation on the weight feature of the question sentence and the weight feature of the answer sentence to obtain the weight feature of the whole sentence.
[0067] Step S106, searching the target reply text that matches the question text set from the reply text set according to the question sentence weight feature, the reply sentence weight feature and the whole sentence weight feature, and replying to the question text set according to the target reply text.
[0068] Steps S101 to S106 shown in the embodiment of the present application are performed by obtaining a question text set and a reply text set; performing feature extraction on the question text set to obtain a question sentence feature set, and performing feature extraction on the reply text set to obtain a reply sentence feature set; performing word weight assignment on the question sentence feature set and the reply sentence feature set to obtain a weighted word weight feature; performing weighted summation on the question sentence feature set according to the weighted word weight feature to obtain a question sentence weight feature, and performing weighted summation on the reply sentence feature set according to the weighted word weight feature to obtain a reply sentence weight feature; performing masking and splicing operations on the question sentence weight feature and the reply sentence weight feature to obtain a whole sentence weight feature; searching for a target reply text that matches the question text set from the reply text set according to the question sentence weight feature, the reply sentence weight feature and the whole sentence weight feature, and replying to the question text set according to the target reply text. The present application achieves mutual matching of question and answer text and reply text by combining the word weights, question sentence weight features, reply sentence weight features and whole sentence weight features of the question text and reply text, and realizes automatic reply to the question and answer text by combining semantics and character features, thereby improving the efficiency and accuracy of the reply.
[0069] In step S101 of some embodiments, the question text set refers to a set of question texts raised by users, which can be specifically determined based on actual business and is not limited here.
[0070] For example, in the financial field, the question text set may be texts asking about the types of low-risk financial products or the types of insurance products.
[0071] Specifically, the answer text set refers to a text set that answers the question text, which can be determined based on actual business and is not limited here.
[0072] For example, in the financial field, if the question text is "What types of insurance are there?", the answer text can be "There are many types of insurance, including life insurance, health insurance, accident insurance, property insurance, liability insurance, credit insurance and guarantee insurance, etc., depending on the insured object and content."
[0073] Specifically, the question text set and the answer text set can be obtained from a preset question and answer database, and the question and answer database can be a backend database of the enterprise.
[0074] See also Figure 2 In some embodiments, step S102 may include but is not limited to steps S201 to S202:
[0075] Step S201, encode the question text set to obtain a question word vector set, and encode the answer text set to obtain an answer word vector set.
[0076] Step S202, performing a pooling operation on the question word vector set to obtain a question sentence feature set, and performing a pooling operation on the answer word vector set to obtain a answer sentence feature set.
[0077] In step S201 of some embodiments, the question word vector set refers to a set of word vector forms of the question text; the answer word vector set refers to a set of word vector forms of the answer text.
[0078] Specifically, the encoding layer in the trained reply model can be used to extract the question sentence feature set and the reply sentence feature set. The reply model is a neural network model composed of an encoding layer, a word weight layer, a sentence weight layer, a splicing layer and a reply output layer. Among them, the encoding layer is used to generate a context-related question sentence feature set in the question text set, and is also used to generate a context-related reply sentence feature set in the reply text set; the word weight layer is used to represent the weight distribution of each word in the question sentence feature set and the reply sentence feature set; the sentence weight layer is used to represent the weight distribution of each sentence composed of each word in the question sentence feature set and the reply sentence feature set, so as to improve the model learning ability to represent the semantic information between sentences; the splicing layer is used to represent the weight distribution of the overall features of the input question features and reply features; the reply output layer is used to output the reply text that matches the question text.
[0079] In one embodiment of the present invention, when training the reply model, first, the real reply text corresponding to the question text is obtained, and the sentence vector set of the question text and the reply text is extracted through the encoding layer, and then the word weight layer is used to assign word weights to the question sentence feature set and the reply sentence feature set, and then the question sentence feature set and the reply sentence feature set are weighted and summed through the sentence weight layer; secondly, the question sentence weight feature and the reply sentence weight feature are masked and spliced through the splicing layer, and then the reply text that matches the question text set is predicted through the reply output layer to obtain the predicted reply text; finally, the loss function is used to calculate the loss value of the real reply result and the predicted reply text, and the parameters of the reply model are adjusted according to the loss value until the loss value meets the preset threshold, and the trained reply model is obtained.
[0080] Specifically, the loss value can be calculated by the following formula:
[0081]
[0082] Among them, H represents the loss value, n represents the total number of features in the question word feature set and the answer sentence feature set; y i represents the real answer text corresponding to the weighted features of the question sentence, the answer sentence and the whole sentence after the i-th concatenation; p i Represents the predicted answer text corresponding to the weight feature of the i-th concatenated question sentence, the weight feature of the answer sentence, and the weight feature of the whole sentence.
[0083] See also Figure 3 In some embodiments, step S201 may include but is not limited to steps S301 to S302:
[0084] Step S301, segmenting the question text set by a preset identifier to obtain a question word set, and segmenting the answer text set by a preset identifier to obtain a answer word set.
[0085] Step S302, perform word embedding operation on the question word set to obtain question word vectors, and perform word embedding operation on the answer word set to obtain answer word vectors.
[0086] In step S301 of some embodiments, the preset identifiers include [CLS], [SEP] and [WORD] identifiers; wherein the [CLS] identifier is used to characterize the overall sequence of the question text set input; the [SEP] identifier is used to separate sentences in the input sequence; and the [WORD] identifier is used to identify the boundaries of each word in the input sequence.
[0087] For example, in the financial field, if the input question text set is what types of insurance are there, then the segmented question word set and the answer word set can be [CLS][WORD] insurance [WORD] types [WORD] types [WORD] there are [WORD] which [WORD] [SEP]; if the input answer text set is life insurance, health insurance, accident insurance, property insurance, liability insurance, credit insurance and guarantee insurance, etc., then the segmented answer word set can be [CLS][WORD] life [WORD] insurance, [WORD] health [WORD] insurance,
[0088] [WORD]Account[WORD] injury[WORD] insurance, [WORD] property[WORD] insurance, [WORD] liability[WORD] insurance[WORD] as well as [WORD] credit[WORD] insurance[WORD] and [WORD] guarantee[WORD] insurance[WORD] etc. [SEP].
[0089] In this embodiment, since the identifier is a symbol without obvious semantic information, the question text set and the answer text set are segmented by the identifier, which can better integrate the semantic information of each word in the text by combining the word and sentence dimensions, thereby better representing the semantics of the entire sentence.
[0090] In step S302 of some embodiments, the word embedding operation may be an Embedding operation, which can convert discrete vocabulary or symbols (such as words, characters or tokens) into continuous vector representations that capture the semantic and grammatical features of the input vocabulary.
[0091] Specifically, Embedding operations include Token Embeddings, Segment Embeddings, and Position Embeddings. Token Embeddings provide rich semantic features for each token, enabling the model to understand the meaning of each word and its changes in different contexts. Segment Embeddings helps the model distinguish different sentences or paragraphs, so that the model can understand whether each token belongs to the characteristics of the question text set or the answer text set when processing sentence pairs. Position Embeddings provides the position information of each token in the input sequence, enabling the model to better capture long-distance dependencies in the sequence.
[0092] In this embodiment, by performing word embedding operations on the question word set and the answer word set, question word vectors and answer word vectors with contextual semantic relationships can be generated, and the question word set and the answer word set can be represented as a low-dimensional dense vector space. The correlation between words in the question text and the answer text is taken into account, thereby more accurately representing the question text set and the answer text set.
[0093] In step S202 of some embodiments, the question sentence feature set refers to a set of sentence vector forms of the question text, and the answer sentence feature set refers to a set of sentence vector forms of the answer text.
[0094] Specifically, the pooling operation is a maximum pooling operation, which uses a certain number and size of convolution kernels to sequentially scan all areas of the question word vector set, and extracts feature values from the question word vector through each convolution kernel, and aggregates the embedding vectors of all tokens in the question word vector set sequence into a single vector to obtain the question sentence feature set.
[0095] Specifically, by using a certain number and size of convolution kernels to sequentially scan all areas of the reply word vector set, and extracting feature values from the reply word vector through each convolution kernel, the embedding vectors of all tokens in the reply word vector set sequence are aggregated into a single vector to obtain the reply sentence feature set.
[0096] In this embodiment, by performing pooling operations on the question word vector set and the answer word vector set, the word vector set sequence can be converted into a fixed-size sentence vector, that is, a sentence feature set, and the sentence feature set aggregates the information of the entire sequence and can accurately represent the semantic information between sentences. It achieves the removal of some unnecessary features while fully extracting important features from the question word vector set and the answer word vector set, thereby ensuring the semantic representation ability of the sentence feature set while reducing the computational complexity of the model, which facilitates the subsequent improvement of the efficiency of the model in providing intelligent replies.
[0097] See also Figure 4 In some embodiments, step S103 may include but is not limited to steps S401 to S403:
[0098] Step S401, using the trained word weight model to calculate word weights for the question sentence feature set and the answer sentence feature set to obtain word weight features.
[0099] Step S402, obtaining a question word vector set corresponding to the question sentence feature set, and obtaining a reply word vector set corresponding to the reply sentence feature set.
[0100] Step S403, performing a weighting operation on the question word vector set, the answer word vector set and the word weight feature to obtain a weighted word weight feature.
[0101] In step S401 of some embodiments, the word weight layer is a trained word weight model, which may be a Transformer model.
[0102] Specifically, the word weight feature can be calculated by the following formula:
[0103]
[0104] Among them, W n represents the weight feature of the nth word; n represents the total number of features in the question word feature set and the answer sentence feature set; t represents a question sentence feature or an answer sentence feature; Q n,t represents the t-th question sentence feature or answer sentence feature in the overall feature set n of the question word feature set and the answer sentence feature set; Q n Represents the overall feature set of the question word feature set and the answer sentence feature set.
[0105] In step S402 of some embodiments, the question word vector set and the answer word vector set are consistent with the question word vector set and the answer word vector set in step S201, which will not be repeated here.
[0106] In step S403 of some embodiments, specifically, by multiplying the question word vector set and the answer word vector set obtained after embedding with the word weight feature, a weighted word weight feature can be obtained.
[0107] Specifically, the weighted word weight feature can be expressed by the following formula:
[0108] E weight =E n *W n
[0109] Among them, E weight Represents the weighted word feature; E n =[e1,e2…e n ] represents the nth question word vector or answer word vector; W n Represents the weight feature of the nth word.
[0110] Among them, E weight Including E A and E B , E A represents the weight feature of the question word, E B Represents the answer word weight feature, and the total number of question word weight features and answer word weight features is also n.
[0111] In this embodiment, by performing weighted operations on the question word vector set, the answer word vector set and the word weight features, the weight distribution of each word in the question sentence feature set and the answer sentence feature set can be re-determined, thereby distinguishing that the same word has different weights in different contexts, and further paying more attention to important words in the context, thereby improving the semantic representation of the features and facilitating improving the accuracy of subsequent replies.
[0112] See also Figure 5 In some embodiments, step S104 may also include but is not limited to steps S501 to S502:
[0113] Step S501, obtain the question sentence length of the question sentence feature set, and the number of question words contained in the question sentence length, obtain the reply sentence length of the reply sentence feature set, and the number of reply words contained in the reply sentence length.
[0114] Step S502, perform an average weighting operation on the question sentence feature set according to the weighted word weight feature, the question sentence length and the number of question words to obtain the question sentence weight feature, and perform an average weighting operation on the reply sentence feature set according to the weighted word weight feature, the reply sentence length and the number of reply words to obtain the reply sentence weight feature.
[0115] In step S501 of some embodiments, specifically, in the word embedding operation, the length of the question sentence and the length of the answer sentence can be read through the [SEP] identifier. The sentence weight layer is used to represent the weight distribution of each sentence composed of each word in the question sentence feature set and the answer sentence feature set, thereby improving the ability of the model learning to represent the semantic information between sentences.
[0116] Specifically, the number of question words refers to the number of question words contained in the question sentence length; the number of answer words refers to the number of answer words contained in the answer sentence length.
[0117] In step S502 of some embodiments, the question sentence weight feature may be obtained by the following formula:
[0118]
[0119] Among them, E a_weightsum Represents the weight feature of the question sentence; d A represents the length of the question sentence, A represents the number of question words; E A Represents the question word weight feature in the weighted word weight feature.
[0120] Specifically, the weight feature of the reply sentence can be obtained by the following formula:
[0121]
[0122] Among them, E B_weightsum Represents the weight feature of the reply sentence; d B represents the length of the reply sentence, B represents the number of reply words; E B Represents the answer word weight feature in the weighted word weight feature.
[0123] Specifically, according to the [SEP] identifier, weight The weighted word feature E belonging to the question sentence feature A is separated A and the weighted word feature E belonging to the answer sentence feature B B .
[0124] In this embodiment, by performing an average weighting operation on the question sentence feature set according to the weighted word weight features, the question sentence length and the number of question words, and performing an average weighting operation on the reply sentence feature set according to the weighted word weight features, the reply sentence length and the number of reply words, the weight distribution of each sentence composed of each word in the question sentence feature set and the reply sentence feature set can be determined, which helps to understand each sentence more accurately, and by taking the average of the weighted results, the representation degree of different sentences can be balanced to avoid the situation where the representation of one sentence dominates the representation of the entire input sequence, which can further improve the accuracy of the semantic features of sentence representation.
[0125] See also Figure 6 In some embodiments, step S105 includes but is not limited to steps S601 to S603:
[0126] Step S601, obtaining a question boundary identifier and a question filling identifier of a question sentence weight feature, and obtaining a reply boundary identifier and a reply filling identifier of a reply sentence weight feature;
[0127] Step S602, performing a masking operation on the question boundary identifier and the question filling identifier to obtain a masked question sentence weight feature, and performing a masking operation on the answer boundary identifier and the answer filling identifier to obtain a masked answer sentence weight feature;
[0128] Step S603, concatenating the weight feature of the masked question sentence with the weight feature of the masked answer sentence to obtain the weight feature of the entire sentence.
[0129] In step S601 of some embodiments, the boundary identifier is a [WORD] identifier.
[0130] Specifically, in the masking process, since the length of the question sentence weight feature and the reply sentence weight feature needs to be consistent with the length of the overall input sequence in order to process the features together in the model, when the length of the question sentence weight feature or the reply sentence weight feature is shorter than the length of the overall sequence, padding is performed at the position with short length to ensure the consistency of the length sequence.
[0131] Specifically, the fill identifier can be represented by pad.
[0132] In step S602 of some embodiments, the mask operation can be implemented through the self-attention mechanism of the model.
[0133] Specifically, the first attention_mask (self-attention mask) determines a mask vector with the same length as the input sequence. Each element of the vector is 0 or 1. 1 corresponds to the actual input data of the token (such as insurance), and the position corresponding to 0 is the pad (filling mark), which has no specific meaning. The [WORD] identifier is also masked by the second attention_mask, so that the feature corresponding to the [WORD] identifier is 0, so that when calculating the self-attention of the token, the attention_mask will be used to weight the attention score corresponding to the token to ensure that the model does not pay attention to the masked position.
[0134] In this embodiment, by performing masking operations on question boundary identifiers and question filling identifiers, and performing masking operations on answer boundary identifiers and answer filling identifiers, it is ensured that the model can focus on valid input when processing sequence data, while ignoring unnecessary filling information and boundary identifier information, thereby improving the efficiency of the model's responses. By ignoring boundary identifiers, the model can be combined with traditional semantic similarity feature training, thereby improving the accuracy of the responses.
[0135] In step S603 of some embodiments, the whole sentence weight feature refers to a common feature sequence of a question sentence feature set and an answer sentence feature set input to the model.
[0136] Specifically, in the concatenation layer, the [CLS] identifier and the weight features of the masked question sentence and the weight features of the masked reply sentence after the identifier can be obtained as the weight features of the whole sentence.
[0137] See also Figure 7 In some embodiments, step S106 may include but is not limited to steps S701 to S702:
[0138] Step S701, calculating the semantic matching degree between the question sentence feature set and the answer sentence feature set based on the question sentence weight feature, the answer sentence weight feature and the whole sentence weight feature.
[0139] Step S702, searching the response text set for a target response text that matches the question text set based on the semantic matching degree.
[0140] In step S701 of some embodiments, specifically, the semantic similarity between the question sentence feature set and the answer sentence feature set may be calculated by using cosine similarity, Euclidean distance, or Jaccard similarity calculation method.
[0141] In step S702 of some embodiments, a target reply text matching the question text set is searched from the reply text set based on the semantic matching degree.
[0142] For example, the question text set is "I want to buy medical insurance for my whole family, please recommend the appropriate type of insurance"; and the response text set is "family comprehensive medical insurance, critical illness insurance, accident insurance and outpatient medical insurance, etc."; among them, the semantic matching degree of family comprehensive medical insurance is 0.98, the semantic matching degree of critical illness insurance is 0.78, the semantic matching degree of accident insurance is 0.6 and the semantic matching degree of outpatient medical insurance is 0.8, then family comprehensive medical insurance with a semantic matching degree of 0.98 is selected as the target response text.
[0143] Specifically, the question text set can be replied according to the target reply text, that is, the target reply text can be used by intelligent customer service to reply to customers.
[0144] For example, the customer's question text set is "I have 100,000 yuan of idle funds and want to make short-term investments. Please give me some investment advice." The intelligent customer service can reply to the customer with the text "Based on the amount of idle funds and the investment period, you can consider money market funds or short-term bond funds."
[0145] In this embodiment, by searching for a target reply text that matches the question text set from the reply text set based on the question sentence weight features, the reply sentence weight features and the whole sentence weight features, semantic representation can be performed in combination with word and sentence dimensions as well as traditional sentence feature dimensions, thereby improving the accuracy of the reply. By replying to the question text set based on the target reply text, it is possible to automatically make corresponding replies to the question and answer texts, thereby improving the efficiency of the reply.
[0146] The embodiment of the present application obtains a question text set and a reply text set; extracts features from the question text set to obtain a question sentence feature set, extracts features from the reply text set to obtain a reply sentence feature set; assigns word weights to the question sentence feature set and the reply sentence feature set to obtain a weighted word weight feature; performs weighted summation on the question sentence feature set according to the weighted word weight feature to obtain a question sentence weight feature, performs weighted summation on the reply sentence feature set according to the weighted word weight feature to obtain a reply sentence weight feature; performs masking and splicing operations on the question sentence weight feature and the reply sentence weight feature to obtain a whole sentence weight feature; finds a target reply text that matches the question text set from the reply text set according to the question sentence weight feature, the reply sentence weight feature and the whole sentence weight feature, and replies to the question text set according to the target reply text. The present application realizes the mutual matching of the question and answer text and the reply text by combining the word weights, question sentence weight features, reply sentence weight features and whole sentence weight features of the question text and the reply text, and realizes the automatic corresponding reply to the question and answer text in combination with semantics and character features, thereby improving the efficiency and accuracy of the reply.
[0147] See also Figure 8 The embodiment of the present application also provides an intelligent reply device, which can implement the above-mentioned intelligent reply method, and the device includes:
[0148] A text acquisition module, used to acquire a question text set and an answer text set;
[0149] A feature extraction module, used for extracting features from the question text set to obtain a question sentence feature set, and extracting features from the answer text set to obtain an answer sentence feature set;
[0150] A weight allocation module, used for allocating word weights to the question sentence feature set and the answer sentence feature set to obtain weighted word weight features;
[0151] A sentence weighting module, used for performing weighted summation on the question sentence feature set according to the weighted word weight feature to obtain the question sentence weight feature, and performing weighted summation on the answer sentence feature set according to the weighted word weight feature to obtain the answer sentence weight feature;
[0152] A mask splicing module, used for performing a mask splicing operation on the weight feature of the question sentence and the weight feature of the answer sentence to obtain a weight feature of the whole sentence;
[0153] The intelligent reply module is used to find the target reply text matching the question text set from the reply text set according to the question sentence weight feature, the reply sentence weight feature and the whole sentence weight feature, and reply to the question text set according to the target reply text.
[0154] The specific implementation of the intelligent reply device is basically the same as the specific implementation of the above-mentioned intelligent reply method, and will not be repeated here.
[0155] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned intelligent reply method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.
[0156] See also Fig. 9 , Fig. 9 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:
[0157] The processor 901 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;
[0158] The memory 902 may be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 may store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 902, and the processor 901 calls and executes the intelligent reply method of the embodiment of this application;
[0159] Input / output interface 903, used to implement information input and output;
[0160] Communication interface 904, used to realize communication interaction between the device and other devices, which can be realized by wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);
[0161] A bus 905 that transmits information between various components of the device (e.g., the processor 901, the memory 902, the input / output interface 903, and the communication interface 904);
[0162] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .
[0163] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the above-mentioned intelligent reply method is implemented.
[0164] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0165] The intelligent reply method, intelligent reply device, electronic device and storage medium provided in the embodiments of the present application obtain a question text set and a reply text set; perform feature extraction on the question text set to obtain a question sentence feature set, and perform feature extraction on the reply text set to obtain a reply sentence feature set; perform word weight assignment on the question sentence feature set and the reply sentence feature set to obtain a weighted word weight feature; perform weighted summation on the question sentence feature set according to the weighted word weight feature to obtain a question sentence weight feature, and perform weighted summation on the reply sentence feature set according to the weighted word weight feature to obtain a reply sentence weight feature; perform masking and splicing operations on the question sentence weight feature and the reply sentence weight feature to obtain a whole sentence weight feature; search for a target reply text that matches the question text set from the reply text set according to the question sentence weight feature, the reply sentence weight feature and the whole sentence weight feature, and reply to the question text set according to the target reply text. The present application achieves mutual matching of question and answer text and reply text by combining the word weights, question sentence weight features, reply sentence weight features and whole sentence weight features of the question text and reply text, and realizes automatic reply to the question and answer text by combining semantics and character features, thereby improving the efficiency and accuracy of the reply.
[0166] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.
[0167] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.
[0168] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0169] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.
[0170] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0171] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.
[0172] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. The mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0173] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0174] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0175] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.
[0176] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.
Claims
1. An intelligent reply method, characterized in that: The method comprises: Obtain a question text set and an answer text set; Extracting features from the question text set to obtain a question sentence feature set, and extracting features from the answer text set to obtain an answer sentence feature set; Performing word weight assignment on the question sentence feature set and the answer sentence feature set to obtain weighted word weight features; Performing weighted summation on the question sentence feature set according to the weighted word weight feature to obtain the question sentence weight feature, and performing weighted summation on the answer sentence feature set according to the weighted word weight feature to obtain the answer sentence weight feature; Performing a mask splicing operation on the question sentence weight feature and the answer sentence weight feature to obtain a whole sentence weight feature; According to the question sentence weight feature, the answer sentence weight feature and the whole sentence weight feature, a target answer text matching the question text set is searched from the answer text set, and the question text set is answered according to the target answer text.
2. The method according to claim 1, characterized in that The step of performing word weight assignment on the question sentence feature set and the answer sentence feature set to obtain weighted word weight features includes: Calculating word weights on the question sentence feature set and the answer sentence feature set using the trained word weight model to obtain word weight features; Obtaining a question word vector set corresponding to the question sentence feature set, and obtaining a reply word vector set corresponding to the reply sentence feature set; A weighted operation is performed on the question word vector set, the answer word vector set and the word weight feature to obtain the weighted word weight feature.
3. The method according to claim 1, characterized in that The step of performing weighted summation on the question sentence feature set according to the weighted word weight feature to obtain the question sentence weight feature, and performing weighted summation on the answer sentence feature set according to the weighted word weight feature to obtain the answer sentence weight feature comprises: Obtaining the question sentence length of the question sentence feature set and the number of question words contained in the question sentence length, and obtaining the reply sentence length of the reply sentence feature set and the number of reply words contained in the reply sentence length; The question sentence weight feature is obtained by performing an average weighting operation on the question sentence feature set according to the weighted word weight feature, the question sentence length and the number of question words, and the reply sentence weight feature is obtained by performing an average weighting operation on the reply sentence feature set according to the weighted word weight feature, the reply sentence length and the number of reply words.
4. The method according to claim 1, characterized in that: The masking and splicing operation is performed on the question sentence weight feature and the answer sentence weight feature to obtain the whole sentence weight feature, including: Obtain a question boundary identifier and a question filling identifier of the question sentence weight feature, and obtain a reply boundary identifier and a reply filling identifier of the reply sentence weight feature; Performing a masking operation on the question boundary identifier and the question filling identifier to obtain a masked question sentence weight feature, and performing a masking operation on the answer boundary identifier and the answer filling identifier to obtain a masked answer sentence weight feature; The masked question sentence weight feature and the masked answer sentence weight feature are concatenated to obtain the whole sentence weight feature.
5. The method according to any one of claims 1 to 4, characterized in that: The step of extracting features from the question text set to obtain a question sentence feature set and extracting features from the answer text set to obtain a answer sentence feature set comprises: Performing encoding operation on the question text set to obtain a question word vector set, and performing encoding operation on the answer text set to obtain an answer word vector set; A pooling operation is performed on the question word vector set to obtain the question sentence feature set, and a pooling operation is performed on the answer word vector set to obtain the answer sentence feature set.
6. The method according to claim 5, characterized in that The step of encoding the question text set to obtain a question word vector set, and encoding the answer text set to obtain an answer word vector set includes: The question text set is segmented by a preset identifier to obtain a question word set, and the answer text set is segmented by a preset identifier to obtain an answer word set; A word embedding operation is performed on the question word set to obtain the question word vector, and a word embedding operation is performed on the answer word set to obtain the answer word vector.
7. The method according to claim 6, characterized in that The step of searching the target answer text matching the question text set from the answer text set according to the question sentence weight feature, the answer sentence weight feature and the whole sentence weight feature comprises: Calculate the semantic matching degree between the question sentence feature set and the answer sentence feature set according to the question sentence weight feature, the answer sentence weight feature and the whole sentence weight feature; According to the semantic matching degree, a target reply text matching the question text set is searched from the reply text set.
8. An intelligent answering device, characterized in that: The device comprises: A text acquisition module, used to acquire a question text set and an answer text set; A feature extraction module, used for extracting features from the question text set to obtain a question sentence feature set, and extracting features from the answer text set to obtain an answer sentence feature set; A weight allocation module, used for allocating word weights to the question sentence feature set and the answer sentence feature set to obtain weighted word weight features; A sentence weighting module, used for performing weighted summation on the question sentence feature set according to the weighted word weight feature to obtain the question sentence weight feature, and performing weighted summation on the answer sentence feature set according to the weighted word weight feature to obtain the answer sentence weight feature; A mask splicing module, used for performing a mask splicing operation on the weight feature of the question sentence and the weight feature of the answer sentence to obtain a weight feature of the whole sentence; The intelligent reply module is used to find the target reply text matching the question text set from the reply text set according to the question sentence weight feature, the reply sentence weight feature and the whole sentence weight feature, and reply to the question text set according to the target reply text.
9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the intelligent reply method described in any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the intelligent reply method according to any one of claims 1 to 7 is implemented.
Citation Information
Patent Citations
Recommendation method based on Mask mechanism and hierarchical attention mechanism
CN111582443A
Question and answer matching method, question and answer matching device, electronic equipment and storage medium
CN115033674A
Reply method and device and electronic equipment
CN117808096A
