Question answering method, system, device and storage medium

Abstract

The present disclosure relates to the technical field of computer intelligence processing, and discloses a question reply method, system, device and storage medium. The question reply method comprises: searching a document related to a question when the question to be replied is received; sampling content in the document according to a sampling interval length, and calculating a matching score of the sampled content and the question, the matching score representing a probability that a reply to the question is the sampled content; and in the case that the matching score is higher than a score threshold, the sampled content is displayed so as to take the sampled content as the reply to the question. The question reply precision can be improved.

Description

Technical Field

[0001] This disclosure relates to the field of computer intelligent processing technology, specifically to a problem-solving method, system, device, and storage medium. Background Technology

[0002] Currently, in some intelligent question-answering scenarios, after a user submits a question, one or more documents can be retrieved based on the question, and then these documents are returned to the user as a response to the question. This method of question response has low precision; users cannot directly obtain the answer to the question (they still need to manually search for relevant content from the documents), resulting in a poor user experience.

[0003] Therefore, there is an urgent need for a more accurate problem-solving method. Summary of the Invention

[0004] In view of this, the present disclosure provides a problem response method, a problem response system, an electronic device, and a computer-readable storage medium, which can improve the accuracy of problem response.

[0005] This disclosure provides a method for answering questions, the method comprising:

[0006] When a question is received that needs to be answered, search for documents related to the question.

[0007] The content in the document is sampled according to the sampling interval length, and the matching score between the sampled content and the question is calculated. The matching score represents the probability that the response to the question is the sampled content.

[0008] If the matching score is higher than the score threshold, the sampled content is displayed as a response to the question.

[0009] This disclosure also provides a question-and-answer system, the system comprising:

[0010] The search module is used to search for documents related to a question when a question to be answered is received.

[0011] The sampling module is used to sample the content in the document according to the length of the sampling interval, and calculate the matching score between the sampled content and the question. The matching score represents the probability that the response to the question is the sampled content.

[0012] The response module is used to display the sampled content as a response to the question when the matching score is higher than the score threshold.

[0013] In another aspect, this disclosure provides a computer-readable storage medium for storing a computer program that, when executed by a processor, implements the method described above.

[0014] In another aspect, this disclosure provides an electronic device including a processor and a memory, the memory being used to store a computer program that, when executed by the processor, implements the method described above.

[0015] In some embodiments of this application, the content of a document related to a question is sampled according to the length of a sampling interval. If the matching score between the sampled content and the question is higher than a score threshold, the sampled content is used as the response to the question. Thus, when responding to a question, it is not necessary to return the document; instead, the content identified from the document that can be used as a response can be directly returned, improving the accuracy of the response. Attached Figure Description

[0016] The features and advantages of this disclosure will be more clearly understood by referring to the accompanying drawings, which are schematic and should not be construed as limiting the scope of this disclosure in any way.

[0017] Figure 1 A flowchart illustrating a problem-solving method provided in one embodiment of this application is shown;

[0018] Figure 2 A schematic diagram of a document provided in one embodiment of this application is shown;

[0019] Figure 3 A schematic diagram of the architecture of a response model provided in one embodiment of this application is shown;

[0020] Figure 4 A schematic diagram illustrating the training process of a response model provided in one embodiment of this application is shown;

[0021] Figure 5 A schematic diagram of the modules of a problem response system provided in one embodiment of this application is shown;

[0022] Figure 6 A schematic diagram of an electronic device provided in one embodiment of this application is shown. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0024] This application provides a highly accurate question-answering method that can improve user experience. This method can be applied to electronic devices, including but not limited to tablets, laptops, desktop computers, and servers. Please refer to [link to relevant documentation]. Figure 1 This is a flowchart illustrating a problem-solving method provided in one embodiment of this application. Figure 1 In this context, the method for answering questions includes the following steps:

[0025] Step S11: Upon receiving a question to be answered, search for documents related to the question.

[0026] The question to be answered can be a question entered by the user on a webpage or application interface. After receiving the user's input question, relevant documents can be searched using techniques well-known to those skilled in the art, such as inverted indexes and vector recall. Relevant documents include, but are not limited to, documents belonging to the same domain as the question or documents containing the same keywords as the question. For example, assuming the received question is "What cars are suitable for women to drive?", documents in the automotive field can be searched as relevant documents. Some or all of the content in the searched documents can serve as answers to the question. If multiple documents are found, step S12 can be executed separately for each document.

[0027] Step S12: Sample the content in the document according to the sampling interval length, and calculate the matching score between the sampled content and the question. The matching score represents the probability that the response to the question is the sampled content.

[0028] Specifically, the content in a document can be divided into multiple phrases. Each phrase can include one or more characters. The sampling interval length refers to the number of phrases sampled from the document, i.e., the number of phrases included in the sampled content. For example, if the content in the document is sampled according to a sampling interval length of 2, the sampled content will include 2 phrases. If the content in the document is sampled according to a sampling interval length of 4, the sampled content will include 4 phrases. The sampled content can be a continuous segment of content in the document. (See also...) Figure 2 This is a schematic diagram of a document provided for one embodiment of this application. Figure 2In this model, dashed lines represent the content of a document. The content within the thick outline can be used as sampled content. For the sampled content, a trained response model can be used to calculate a matching score between the sampled content and the question. A higher matching score indicates a higher probability that the sampled content can serve as a response to the question.

[0029] Understandably, within a document, the content that can serve as a response to a question is usually only a portion of the document's content. For example, a document might begin with background information on the automotive industry, cover relevant technologies in the middle, and conclude with recommendations for cars suitable for different groups of people. Taking the question "What cars are suitable for women?" as an example, sampling only the beginning of the document would miss suitable responses. Therefore, when sampling a document, it's possible to sample from different locations. The sampled content from each location can then be matched with the question to calculate a score, resulting in a matching score for each location. This allows for traversing the document's content across different locations, improving sampling accuracy.

[0030] Furthermore, the content in a document that can be used as a response to a question usually has a certain length (i.e., includes a certain number of phrases). If the sampling interval length is different from the length of the content used as a response, the correct content may not be sampled. For example, at the end of a document, 100 phrases might be used to describe cars suitable for different groups of people. Taking "What cars are suitable for women?" as an example, if the content is sampled at the end of the document according to a sampling interval length of 2, the correct content will not be collected. Therefore, when sampling a document, different sampling interval lengths can be used. Specifically, the sampling interval length can be an integer between 1 and n, where n is the total number of phrases in the document. Specifically, the content in the document can be sampled first according to a sampling interval length of 1, then according to a sampling interval length of 2, and so on. Each sampled content can be matched with the question to calculate a score, resulting in a matching score for the content with different sampling interval lengths. This prevents the problem of incorrect content being sampled due to the sampling interval length being different from the length of the content used as a response in the document.

[0031] In summary, in some embodiments of this application, the content in the document is sampled according to the sampling interval length, including:

[0032] The content of the document is sampled at multiple locations in the document, and at each location, the content of the document is sampled according to multiple different sampling interval lengths.

[0033] Specifically, at the k-th phrase in the document, samples can be taken sequentially from the k-th to the n-th phrase according to different sampling interval lengths. The sampling interval length includes 1, n-k+1, and integers between 1 and n-k+1, where k and n are integers, k ranging from 1 to n, and n is the total number of phrases in the document. The sampled content obtained at each location can be matched with the question to calculate a matching score. In short:

[0034] At the first phrase, the first phrase is sampled according to the sampling interval length 1; then the first and second phrases are sampled according to the sampling interval length 2. And so on.

[0035] At the second phrase, first sample the second phrase according to the sampling interval length 1; then sample the second and third phrases according to the sampling interval length 2. And so on.

[0036] ...

[0037] At the nth word group, the nth word group is obtained by sampling according to the sampling interval length of 1.

[0038] This allows for the traversal of the document's content, improving the accuracy of content sampling and thus enhancing the precision of question responses.

[0039] Step S13: If the matching score is higher than the score threshold, display the sampled content as a response to the question.

[0040] Specifically, based on the description of step S12, in some embodiments, the content in the document is sampled by traversal sampling. This sampling method yields a large number of samples, and some sampled content may be quite similar. For example, suppose at phrase A in the document, sampled content a1 obtained by sampling interval length 20 is "This car has a beautiful appearance," and sampled content a2 obtained by sampling interval length 21 is also "This car has a beautiful appearance." Sampled content a1 and sampled content a2 express the same semantic information, and their matching scores with the question may both be higher than the score threshold. In this case, if both sampled content a1 and sampled content a2 are displayed as responses to the question, it will result in content duplication and reduce the user experience.

[0041] Furthermore, the same (or similar) sampled content may be found from different documents. For example, both document 1 and document 2 may sample the phrase "This car has a beautiful appearance." If the matching score between these identical (or similar) sampled content and the question is higher than the score threshold, duplicate responses may be displayed, degrading the user experience.

[0042] Therefore, in some embodiments, displaying sampled content when the matching score is higher than a score threshold may include:

[0043] If multiple sampled contents have a semantic similarity exceeding the similarity threshold among those with a matching score higher than the score threshold, then one of those sampled contents will be displayed.

[0044] Specifically, semantic similarity refers to the similarity of the semantic information conveyed by different sampled content. For example, suppose sampled content A is "This car has a beautiful appearance," sampled content B is "This car has a beautiful appearance," and sampled content C is "This car is reasonably priced." Since sampled content A and sampled content B convey similar semantic information (i.e., they both convey the semantic information that the car has a beautiful appearance), and sampled content C conveys significantly different semantic information from sampled content A and B, when displaying sampled content, only sampled content A and sampled content C can be displayed, or only sampled content B or sampled content C can be displayed. In this way, duplicate responses can be filtered out, improving the user experience.

[0045] Specifically, sampled content with matching scores higher than a score threshold can be input into a language model, which will then determine the semantic similarity of these sampled contents.

[0046] Furthermore, to help users understand the credibility of the sampled content (i.e., the probability that it can be used as a response to a question), in some embodiments, displaying the sampled content when the matching score is higher than a score threshold may include:

[0047] The program displays the sampled content and its matching score with the question. Furthermore, if multiple samples have matching scores higher than the score threshold, the program sorts and displays the samples according to their matching scores.

[0048] In this way, users can judge the credibility of each response (i.e., the displayed sampled content) based on the matching score between the sampled content and the question, thereby improving the user experience.

[0049] In summary, in some embodiments of this application, the content of a document related to a question is sampled according to the sampling interval length. If the matching score between the sampled content and the question is higher than a score threshold, the sampled content is used as the response to the question. Thus, when responding to a question, it is not necessary to return the document; instead, the content identified from the document that can be used as a response can be directly returned, improving the accuracy of the response.

[0050] The following provides further details on the specific implementation of this application.

[0051] In some embodiments, the operation in step S12 above can be implemented using a response model. (See also...) Figure 3 This is a schematic diagram of the architecture of a response model provided in one embodiment of this application. Figure 3 In this model, the response model is a neural network model based on the BERT architecture, specifically including a word segmentation layer 31, a feature extraction layer 32, and a score calculation layer 33. The score calculation layer 33 can include a fully connected layer and a normalized exponential function layer (also known as a softmax layer). In step S12, the question and document can be input into the response model, which then performs the following operations to sample the content of the document and calculate a matching score:

[0052] 1) Divide the question into multiple first phrases and the content in the document into multiple second phrases. Specifically, this can be achieved through phrase division layer 31, dividing the question into multiple first phrases and the content in the document into multiple second phrases. Figure 3 In the word segmentation layer 31, rectangles without pattern filling can represent the first segmented word group, and rectangles with pattern filling can represent the second segmented word group.

[0053] 2) Extract the first feature vector from the first word group and the second feature vector from the second word group. Specifically, the first feature vector can be extracted from the first word group and the second feature vector can be extracted from the second word group through feature extraction layer 32. Figure 3 In the feature extraction layer 32, rectangles without pattern filling can represent the first feature vector, and rectangles with pattern filling can represent the second feature vector. There can be a one-to-one correspondence between the first word group and the first feature vector, and vice versa. Each feature vector is used to characterize the features of its corresponding word group.

[0054] 3) Sample the second feature vector according to the sampling interval length to sample the content of the document. Specifically, since the second feature vector and the second word group have a one-to-one correspondence, sampling the second feature vector is equivalent to sampling the content of the document in units of the second word group. For example, starting from the second second feature vector, sampling the second feature vector according to the sampling interval length of 2 can obtain the second and third feature vectors, which is equivalent to starting from the second second word group and sampling the content of the document according to the sampling interval length of 2 to obtain the second and third word groups.

[0055] 4) Calculate the matching score based on the first feature vector and the sampled second feature vector. In this embodiment, the first feature vector and the sampled second feature vector can be multiplied to obtain the third feature vector. Based on the third feature vector, a first probability that the response to the question is the sampled content and a second probability that the response to the question is not the sampled content can be determined, and the matching score can then be obtained from the first probability.

[0056] Specifically, the third feature vector can be input into the score calculation layer 33. Based on the third feature vector, the fully connected layer and the normalized exponential function layer in the score calculation layer 33 can classify the sampled content, i.e., determine the first probability that the sampled content belongs to a question response, and the second probability that the sampled content does not belong to a question. The matching score can be derived from the first probability. The matching score can be directly proportional to the first probability; that is, the higher the first probability, the higher the matching score. In some embodiments, the first probability can be directly used as the matching score. In other embodiments, the first probability can be multiplied by a specified coefficient, and the result can be used as the matching score. For example, the specified coefficient can be 100. Since the first probability reflects the probability that the sampled content is a question response, the matching score derived from the first probability has high accuracy.

[0057] In the above embodiments, by inputting the question and document into the response model, and having the response model calculate the matching score, the accuracy of the matching score can be improved.

[0058] This concludes the detailed explanation of step S12. The following section explains the training process of the response model.

[0059] See also Figure 4 This is a schematic diagram illustrating the training process of a response model provided in one embodiment of this application. Figure 4 In the training process, the response model may also include an error calculation layer 44. The error calculation layer 44 is used to calculate errors during the training of the response model. For a detailed explanation of the error calculation layer 44, please refer to the subsequent related instructions; it will not be elaborated upon here.

[0060] Specifically, the response model can be trained using the following methods:

[0061] 1) Obtain positive and negative sample documents related to the question. Positive sample documents include annotated first content, and negative sample documents include annotated second content. The first content is a response to the question, while the second content is not. Specifically, positive and negative sample documents can be obtained from the same document or from different documents. For example, in document A, annotating content that can be used as a response to the question yields a positive sample document, and annotating content that cannot be used as a response to the question yields a negative sample document. Similarly, in document A, annotating content that can be used as a response to the question yields a positive sample document, and in document B, annotating content that cannot be used as a response to the question yields a negative sample document.

[0062] 2) Input the question, positive sample documents and negative sample documents into the response model. The response model calculates the first matching score between the question and the first content, and the second matching score between the question and the second content. The response model is trained based on the first matching score and the second matching score.

[0063] The calculation of the first and second matching scores is largely similar to the description of matching score calculation in step S12, with the main difference being that the first matching score can be obtained based on the feature vectors of the question and the first content, while the second matching score can be obtained based on the feature vectors of the question and the second content. Simply put, during the training of the response model, it is not necessary to sample the feature vectors corresponding to the document content; instead, the matching score can be calculated directly using the labeled feature vectors corresponding to the first and second content. For example... Figure 4 In the process, assuming the first three feature vectors in the feature vectors corresponding to the first labeled content are the same as the first feature vectors in the positive sample document, the feature vector corresponding to the question can be multiplied by the first three feature vectors corresponding to the positive sample document. The resulting feature vector is then input into the score calculation layer 43 to obtain the first matching score. The calculation process for the second matching score is the same as that for the first matching score, and will not be elaborated here.

[0064] Error calculation layer 44 can be used to calculate the error between the first matching score and the second matching score. The error between the first and second matching scores can characterize the magnitude of the difference between them. If the difference between the first and second matching scores is large, then the error between them can be small; conversely, if the difference is small, then the error can be large. Training the response model involves adjusting its parameters based on the error between the first and second matching scores, ensuring that the error is less than a threshold (i.e., model convergence), meaning the first and second matching scores have a large difference. This training method yields a response model with high accuracy. After the response model is trained, the matching score calculated by the model can be used to determine whether the sampled content can serve as a response to the question.

[0065] In this embodiment, to allow a significant difference between the first matching score and the second matching score, the error calculation layer may include an error threshold. Based on the error threshold, the response model can be trained as follows:

[0066] Based on the error threshold and the first matching score, the first error of the first matching score is calculated. The smaller the difference between the first matching score and the error threshold, the smaller the first error.

[0067] Based on the error threshold and the second matching score, the second error of the second matching score is calculated. The smaller the difference between the second matching score and the error threshold, the larger the second error.

[0068] Based on the first and second errors, the parameters of the response model, including the error threshold, are adjusted.

[0069] In simple terms, the model is trained to ensure that the matching scores and error thresholds of sampled content that can be used as a response to a question are close to those of the question, while the matching scores and error thresholds of sampled content that cannot be used as a response to a question are significantly different. Once the response model is trained, the matching scores calculated by the response model can be used to determine whether the sampled content can be used as a response to the question.

[0070] In addition, compared to manually specifying a static error threshold, in this embodiment, the error threshold is an adjustable parameter. Through model training, the response model learns a better value, making the solution more reasonable and effective, and with a wider range of applications.

[0071] This concludes the explanation of the training process for the response model.

[0072] Please see Figure 5 This is a schematic diagram of a question response system provided in one embodiment of this application. Figure 5 In China, the question and answer system includes:

[0073] The search module is used to search for documents related to a question when a question is received that needs to be answered.

[0074] The sampling module is used to sample the content in the document according to the sampling interval length and calculate the matching score between the sampled content and the question. The matching score represents the probability that the response to the question is the sampled content.

[0075] The response module is used to display sampled content as a response to the question when the matching score is higher than the score threshold.

[0076] The beneficial effects of the question response system can be found in the relevant description of the question response method, which will not be repeated here.

[0077] Please see Figure 6 This is a schematic diagram of an electronic device provided according to an embodiment of this application. The electronic device includes a processor and a memory, the memory being used to store a computer program, which, when executed by the processor, implements the method described above.

[0078] The processor can be a central processing unit (CPU). It can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations thereof.

[0079] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the methods in the embodiments of this invention. The processor executes various functional applications and data processing by running the non-transitory software programs, instructions, and modules stored in the memory, thereby implementing the methods described in the above embodiments.

[0080] The memory may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the processor, etc. Furthermore, the memory may include high-speed random access memory and non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, which can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0081] One embodiment of this application also provides a computer-readable storage medium for storing a computer program that, when executed by a processor, implements the above-described method.

[0082] Although embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A question answering method, characterized by, The method comprises: Upon receiving a question to be replied, searching for a document related to the question; According to a sampling interval length, sampling content in the document and calculating a matching score of the sampled content and the question, the matching score representing a probability that a reply to the question is the sampled content; In a case where the matching score is higher than a score threshold, displaying the sampled content as a reply to the question; The sampling of content in the document and the calculation of the matching score of the sampled content and the question comprise: inputting the question and the document into a reply model, and performing the following operations by the reply model to sample content in the document and calculate the matching score: dividing the question into a plurality of first word groups and dividing content in the document into a plurality of second word groups; extracting a first feature vector from the first word groups and a second feature vector from the second word groups; sampling the second feature vector according to the sampling interval length to sample content in the document; multiplying the first feature vector and the sampled second feature vector to obtain a third feature vector; based on the third feature vector, determining a first probability that a reply to the question is the sampled content and a second probability that a reply to the question is not the sampled content; converting the first probability to obtain the matching score.

2. The method of claim 1, wherein, The sampling of content in the document according to the sampling interval length comprises: sampling content in the document at a plurality of positions of the document, wherein at each of the positions, content in the document is sampled according to a plurality of different sampling interval lengths.

3. The method of claim 2, wherein, The content in the document is divided into a plurality of word groups; and the sampling of content in the document at a plurality of positions of the document comprises: At the kth word group of the document, contents in the document are sampled in turn from the kth word group to the nth word group according to different sampling interval lengths, wherein the sampling interval length values include 1, n k+1 and integers between 1 and n k+1, k and n are integers, the value range of k is 1 to n, and n is the total number of word groups included in the document.

4. The method of claim 1, wherein, The displaying of the sampled content in the case where the matching score is higher than the score threshold comprises: in the sampled content with a matching score higher than the score threshold, if there are a plurality of sampled contents with a semantic similarity exceeding a similarity threshold, displaying one of the plurality of sampled contents.

5. The method of claim 1, wherein, The displaying of the sampled content in the case where the matching score is higher than the score threshold comprises: displaying the sampled content and the matching score of the sampled content and the question, and in a case where a plurality of sampled contents have matching scores higher than the score threshold, sorting and displaying the plurality of sampled contents according to the matching scores.

6. The method of claim 1, wherein, Before sampling content in the document, the reply model is trained based on the following method: obtaining positive sample documents and negative sample documents related to the question, wherein the positive sample documents include first content that has been labeled, the negative sample documents include second content that has been labeled, the first content is a reply to the question, and the second content is not a reply to the question; inputting the question, the positive sample document and the negative sample document into the reply model, calculating, by the reply model, a first matching score of the question and the first content and a second matching score of the question and the second content, and training the reply model based on the first matching score and the second matching score.

7. The method of claim 6, wherein, The reply model comprises an error threshold; and the training of the reply model based on the first matching score and the second matching score comprises: calculating a first error of the first matching score based on the error threshold and the first matching score, wherein the smaller the difference between the first matching score and the error threshold, the smaller the first error; calculating a second error of the second matching score based on the error threshold and the second matching score, wherein the smaller the difference between the second matching score and the error threshold, the greater the second error; adjusting parameters of the reply model including the error threshold based on the first error and the second error.

8. A question answering system, characterized by, The system comprises: a search module configured to search for a document related to a question to be replied when the question is received; a sampling module configured to sample content in the document according to a sampling interval length and calculate a matching score of the sampled content and the question, the matching score representing a probability that a reply to the question is the sampled content; a reply module configured to display the sampled content as a reply to the question when the matching score is higher than a score threshold. The sampling module is specifically configured to: input the question and the document into a reply model, and perform the following operations by the reply model to sample content in the document and calculate the matching score: divide the question into a plurality of first word groups and divide the content in the document into a plurality of second word groups; extract a first feature vector from the first word groups and a second feature vector from the second word groups; sample the second feature vector according to the sampling interval length to sample the content in the document; multiply the first feature vector and the sampled second feature vector to obtain a third feature vector; determine a first probability that a reply to the question is the sampled content and a second probability that a reply to the question is not the sampled content based on the third feature vector; convert the first probability to obtain the matching score.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium is configured to store a computer program, and the computer program is configured to be executed by a processor to implement the method according to any one of claims 1 to 7.

10. An electronic device, comprising: The electronic device comprises a processor and a memory, and the memory is configured to store a computer program, and the computer program is configured to be executed by the processor to implement the method according to any one of claims 1 to 7.

Images