Intelligent question-answering method and device based on voice and text collaboration
Through the intelligent question-and-answer method of voice and text collaboration, the problem of low conversion efficiency caused by independence among existing systems is solved, and efficient voice text conversion and knowledge base utilization is achieved. It is suitable for people with disabilities and improves user experience and system efficiency .
Patent Information
- Application Number
- CN202410201055.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-02-23
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-02-23
AI Technical Summary
The existing online question-and-answer system and voice call system are independent of each other, resulting in low efficiency in voice and text conversion, inability to use efficiently, unable to meet personalized needs, and unable to effectively serve people with disabilities.
Using intelligent question-answer methods based on voice and text collaboration, collect information through text input or voice online dictation/offline transliteration, match the sentence model and perform word segmentation processing, obtain the core semantic word set and the Q&A knowledge base to match, and provide many with high confidence Answers to improve conversion efficiency and knowledge base utilization.
It improves the conversion efficiency between speech and text, is suitable for people with disabilities, reduces labor costs, improves the utilization rate of user experience and knowledge base, and realizes the integration of speech and text Q&A systems.
Smart Images

Figure CN118069805B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of intelligent question answering technology, and in particular to an intelligent question answering method and device based on voice and text collaboration. Background Art
[0002] At present, most online question-and-answer robots, intelligent customer service robots, online customer service and voice-text service systems are independent of each other. If you need to meet the personalized needs of internal business, you need to rely on suppliers for transformation. If it involves connecting externally purchased systems, the difficulty of transformation and the procurement cost are very high. For suppliers, they are more willing to develop standard products rather than customized development, which leads to very limited internal use scenarios of external systems.
[0003] Most online Q&A robots only support text interaction, or the voice conversion efficiency is low. During the implementation process, it is impossible to effectively realize the voice-text mutual Q&A interaction, which leads to serious restrictions on scenarios and is difficult to promote to more scenarios. As for the traditional voice call system, whether it is a self-built call center or a hosted call center, it uses manual seat customer service to provide one-to-one manual voice service, which cannot provide intelligent analysis and intelligent automatic response, resulting in the need for a large number of manual customer service personnel and relatively low overall service efficiency.
[0004] In short, the voice-to-text conversion service and the question-and-answer service system are separated and independent from each other, resulting in low efficiency in the conversion between voice and text and between questions and answers, and they cannot be used together efficiently; the voice question-and-answer service system and the online question-and-answer service system are independent of each other, and the relevant knowledge base cannot be efficiently accumulated, resulting in resource duplication, waste of personnel and high costs; a simple text interactive or voice interactive online question-and-answer service system cannot provide services for some people with disabilities. Summary of the invention
[0005] The purpose of the present invention is to address the deficiencies of the above-mentioned prior art and to propose an intelligent question-answering method and device based on voice and text collaboration, which can improve the conversion efficiency of voice and text and the utilization rate of the knowledge base.
[0006] In a first aspect, the present invention provides an intelligent question-answering method based on voice and text collaboration, comprising:
[0007] Collect information through at least one of text input, online voice dictation or offline voice transcription, and match the information with models in a sentence model library, and use the model with the highest similarity as the most matching sentence model for the information;
[0008] Performing word segmentation processing on the most matching sentence model to obtain a core semantic word set, performing relevance matching on the core semantic word set and words in a question-answering knowledge base to obtain a plurality of answers whose confidence levels are not less than a minimum threshold;
[0009] The multiple answers are sorted according to confidence and output via text or voice.
[0010] The present invention combines voice and text to collect information and output answers, coordinates voice and text, and coordinates online voice dictation and offline voice transcription with a question-and-answer service system, thereby improving the conversion efficiency between voice and text and between questions and answers. In addition, by collecting information through voice, offline transcription and online dictation and outputting text or voice, the present invention can be suitable for some disabled people and has higher applicability. In addition, by outputting multiple answers to text, online voice dictation and offline voice transcription through only one system knowledge base, the utilization rate of the system knowledge base can be improved and the labor cost can be reduced. In addition, through the two-step matching process of sentence model matching and relevance matching, the accuracy of matching different sentence patterns to corresponding answers can be improved, thereby improving the user experience.
[0011] In combination with the first aspect, in some embodiments, collecting information through the online voice dictation includes:
[0012] According to the first long connection service newly established with the server, an online dictation user request is received, and the number of users requested by the current user is recorded according to a static global variable;
[0013] According to the number of users, with request time as the first-level priority score and user priority as the second-level priority score, a queue heap root is constructed, and the traversal order of the queue heap root is used as a queue order in descending order, and information is collected according to the queue order.
[0014] The present invention adopts the secondary index sorting of request time and user priority to regulate the order of users' real-time voice requests. Compared with the traditional first-in-first-out sorting queue, the present invention can reduce the time complexity of entering and leaving the queue, thereby improving the voice processing efficiency and improving the user experience.
[0015] In combination with the first aspect, in some embodiments, collecting information in sequence according to the queuing order includes:
[0016] According to the queuing order, the collected real-time first audio file stream is converted into online dictation text in an overlay manner, and the online dictation text is post-processed to obtain the text to be matched; the text to be matched is used to match with the model in the sentence model library; the post-processing includes: punctuation and number adjustment, and identification of sensitive words.
[0017] In combination with the first aspect, in some embodiments, collecting information through offline voice transcription includes:
[0018] According to the newly established second long connection service with the server, an offline user request is received, and the encoded second audio file stream is obtained. According to the cache byte group not greater than the storage threshold, the second audio file stream is voice-transcribed to collect information.
[0019] In combination with the first aspect, in some embodiments, the voice transcription of the second audio file stream and the information collection based on the cache byte group not greater than the storage threshold include:
[0020] The second audio file stream is segmented according to the cache byte group that is not greater than the storage threshold, and the multiple segmentation results are sequentially transcribed into speech to obtain multiple transcription results, the multiple transcription results are sequentially merged to obtain offline text, and the offline text is post-processed to obtain text to be matched.
[0021] In combination with the first aspect, in some embodiments, performing word segmentation processing on the best matching sentence model to obtain a core semantic word set includes:
[0022] The interrogative words and question feature words in the most matching sentence model are filtered, and the obtained filtering results are segmented to obtain sentence fragments, and the sentence fragments are segmented, and all auxiliary words in the segmentation results are removed to obtain a core semantic word set.
[0023] In combination with the first aspect, in some embodiments, matching the core semantic word set with words in the question-answer knowledge base for relevance to obtain multiple answers with confidence levels not less than a minimum threshold includes:
[0024] The core semantic word set is matched with the words in the question-answer knowledge base in turn by relevance, and the matching is retrieved through keyword relevance and word vector relevance to obtain multiple first answers, and the multiple first answers are sorted according to confidence to obtain multiple second answers whose confidence is not less than a minimum threshold.
[0025] The present invention matches the information with the models in the sentence model library, matches the core semantic word set with the words in the question and answer knowledge base for relevance, and filters the interrogative words and question feature words. It adopts a combination of speech recognition and sentence parsing to significantly improve the accuracy compared with traditional solutions. Even when the user's word order changes significantly, it can still match the correct question and provide an answer, thereby improving the user experience.
[0026] In conjunction with the first aspect, in some embodiments, before sequentially performing relevance matching on the core semantic word set and the words in the question-answering knowledge base, the method further includes:
[0027] Obtain a preset first answer channel category. If the first answer channel category is not preset, perform channel classification prediction on the core semantic word set to obtain a predicted second answer channel category; the second answer channel category is used to identify the domain category of the core semantic word set in the question and answer knowledge base.
[0028] In conjunction with the first aspect, in some embodiments, outputting by voice includes:
[0029] synthesize the multiple answers into a voice file, and receive the voice file according to the newly established third long connection service with the server;
[0030] If the voice file is real-time voice, the voice file is broadcasted in sequence according to the static global variable recording the number of users connected in real time and the queue order;
[0031] If the voice file is non-real-time voice, the voice file is stored in a buffer and broadcasted in sequence.
[0032] In a second aspect, the present invention provides an intelligent question-answering device based on voice and text collaboration, comprising: a collection and matching unit, a word segmentation and matching unit, and an output unit; wherein:
[0033] The collection and matching unit is used to collect information through at least any one of text input, online voice dictation or offline voice transcription, and match the information with the models in the sentence model library, and use the model with the highest similarity as the most matching sentence model for the information;
[0034] The word segmentation and matching unit is used to perform word segmentation processing on the best matching sentence model, obtain a core semantic word set, perform relevance matching on the core semantic word set with words in the question-answer knowledge base, and obtain multiple answers whose confidence levels are not lower than a minimum threshold;
[0035] The output unit is used to sort the multiple answers according to confidence and output them via text or voice. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 is a flowchart of an intelligent question-answering method based on voice and text collaboration provided in this embodiment;
[0037] Figure 2 is a structural diagram of an intelligent question-answering device based on voice and text collaboration provided in this embodiment;
[0038] Figure 3 Schematic diagram of the structure of an intelligent question-answering system provided in this embodiment. DETAILED DESCRIPTION
[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0040] Based on the above background, an intelligent question-answering method and device based on voice and text collaboration is provided, which can realize multiple voice and text input methods, and provide intelligent question-answering services with voice or text responses, so as to realize the collaborative integration of voice and text, and voice input also includes offline input and online input, thereby improving the overall coordination of the intelligent answering system, and different modes of input can expand the use scenarios of intelligent question-answering services, especially for some disabled people. Through intelligent voice dialogue, the workload of staff can be reduced to a certain extent, efficiency can be improved, costs can be saved, and new support points and room for industry development and progress can be provided for voice question-answering services and text question-answering services. In order to further illustrate the technical solution of the present invention, the following embodiments will be described in detail.
[0041] Example 1
[0042] See also Figure 1 , is a flow chart of an intelligent question-answering method based on voice and text collaboration provided in this embodiment, including steps S11 to S13, specifically:
[0043] Step S11, collect information through at least any one of text input, online voice dictation or offline voice transcription, and match the information with the models in the sentence model library, and use the model with the highest similarity as the most matching sentence model for the information.
[0044] In some embodiments, collecting information through text input includes: directly acquiring text input by the user through an input interface or an interface call.
[0045] In some embodiments, information is collected through the online voice dictation, including: receiving an online dictation user request based on a newly created first long connection service with a server, and recording the number of users currently requested by the user based on a static global variable; constructing a queue pile root based on the number of users, with the request time as the first-level priority score and the user priority as the second-level priority score, using the traversal order of the queue pile root as a descending queue order, and collecting information according to the queue order.
[0046] It is worth mentioning that the first long connection service is established for the scenario of real-time online voice dictation.
[0047] In some embodiments, a new long-connection Websocket service IatWebSocketServerHandler is created on the server side, which is specifically used to receive requests related to online dictation.
[0048] It is worth noting that online dictation is a type of speech recognition capability, which is used for instant transcription of audio. The intelligent voice engine of the smart answer system provides real-time speech recognition and instant transcription capabilities, with a delay of less than 300ms, and supports dialects, and can achieve a 98% recognition rate, thereby improving the user experience.
[0049] In some embodiments, if the request times are different, the user request with the earlier request time will be used as the parent node of the sorting heap root in the order of request time. When the request times are the same, the user request with the higher user priority will be used as the parent node of the sorting heap root in the order of user priority.
[0050] In some embodiments, if user priorities are different, user requests with higher user priorities are used as parent nodes of the sorting heap root in order of their priorities. When user priorities are the same, user requests with earlier request times are used as parent nodes of the sorting heap root in order of their request times.
[0051] In some embodiments, a static global variable (such as onlineCount) is designed to record the number of currently connected users, and to perform service queuing and flow limiting. The queuing operation is based on the user as the highest priority, and users with the same priority are constructed with the request time as the secondary priority. The traversal order of the large root heap is used as the queuing order. If a new user joins, it is quickly inserted, which can better improve the user experience.
[0052] It is worth noting that, according to the request time and user priority, a sorted heap root is established, and the traversal order of the obtained sorted heap root is the dequeue order. The time complexity of entering and exiting the sorted heap root is O(lgn), and the total time complexity of dequeuing and entering the queue is O(lgn). The time complexity of each entry of the traditional queue is O(1), but the time complexity of dequeuing is O(n), and the total time complexity of dequeuing and entering the queue is O(n). This embodiment uses secondary indicators (request time and user priority) to limit the flow of online users, and the total time complexity of entering and exiting the queue is lower, and the efficiency of entering and exiting the queue is higher, thereby improving the user experience.
[0053] In some embodiments, information is collected in sequence according to the queueing order, including: in sequence according to the queueing order, the collected real-time first audio file stream is converted into online dictation text in an overlay manner, and the online dictation text is post-processed to obtain text to be matched; the text to be matched is used to match with a model in a sentence model library; the post-processing includes: punctuation and number adjustment, and identification of sensitive words.
[0054] In some embodiments, the collected real-time first audio file stream is converted into online dictation text in an overwriting manner, including: after calling the Websocket interface, the Websocket interface continuously returns the question text converted from the real-time file stream, and the subsequent returned text will overwrite the previous text. For example, the text corresponding to the audio file is: How is the weather today, Websocket will continuously return the following "today", "today's weather", "how is the weather today".
[0055] In some embodiments, the online dictation text is post-processed, including: regularizing punctuation and numbers, replacing lists, and efficiently identifying pornographic and political content. The sorted text is provided to the next step.
[0056] In some embodiments, information is collected through the offline voice transcription, including: receiving an offline user request based on a newly created second long connection service with the server, and obtaining an encoded second audio file stream, and voice-transcribing the second audio file stream based on a cache byte group that is not greater than a storage threshold to collect information.
[0057] It is worth noting that the second long connection service is established for the scenario of offline voice transcription. Offline transcription refers to the technology of transcribing the recorded voice recording files into text. The intelligent answering system provides the ability to transcribe offline voice into text, and supports multiple audio encoding formats such as pcm, wav and mp3.
[0058] In some embodiments, a new long connection websocket service OstWebSocketServerHandler is created on the server side, which is specifically used to receive requests related to offline transcription.
[0059] In some embodiments, the second audio file stream is voice-transcribed and information is collected based on a cache byte group that is not greater than a storage threshold, including: the second audio file stream is segmented based on a cache byte group that is not greater than a storage threshold, and multiple segmentation results are sequentially voice-transcribed to obtain multiple transcription results, the multiple transcription results are sequentially merged to obtain offline text, and the offline text is post-processed to obtain text to be matched.
[0060] In some embodiments, the storage threshold is 16KB.
[0061] In some embodiments, the service receives an audio file stream converted into base64 encoding, and the service segments the audio stream using buffer byte groups no larger than 16KB, and then performs speech transcription in sequence, and finally merges all transcription results.
[0062] In some embodiments, the offline text is post-processed, including: regularizing punctuation marks and numbers, replacing lists, and efficiently identifying pornographic and political content. The sorted text is provided to the next step.
[0063] Step S12: perform word segmentation processing on the best matching sentence model to obtain a core semantic word set, perform relevance matching on the core semantic word set with words in the question and answer knowledge base, and obtain multiple answers whose confidence levels are not lower than a minimum threshold.
[0064] It is worth mentioning that through natural language processing (NLP) technology and analysis capabilities based on deep learning, we can analyze and understand user intentions, cooperate with the system knowledge base to provide text interaction functions such as question answering, knowledge recommendation and business process guidance, and at the same time, on this basis, intelligently mark the caller's questions with business tags, conduct business and data analysis, and based on the unified underlying architecture of the platform, analyze the concerns of each user, explore the deep-seated needs of users, and provide personalized and differentiated services to customers and customer groups.
[0065] In some embodiments, the best matching sentence model is segmented to obtain a core semantic word set, including: filtering question words and question feature words in the best matching sentence model, and segmenting the filtered results to obtain sentence fragments, segmenting the sentence fragments, and removing all auxiliary words in the segmentation results to obtain a core semantic word set.
[0066] In some embodiments, the core semantic word set is relevance matched with words in a question-and-answer knowledge base to obtain multiple answers whose confidence levels are not lower than a minimum threshold, including: relevance matching the core semantic word set with words in the question-and-answer knowledge base in turn, retrieving and matching through keyword relevance and word vector relevance to obtain multiple first answers, and sorting the multiple first answers according to confidence levels to obtain multiple second answers whose confidence levels are not lower than a minimum threshold.
[0067] It is worth noting that the first answer is a plurality of unsorted answers, and the second answer is a plurality of answers sorted by confidence, whose confidence values are not lower than the minimum threshold.
[0068] In some embodiments, before matching the core semantic word set with the words in the question and answer knowledge base in sequence for relevance, it also includes: obtaining a preset first answer channel category, and if the first answer channel category is not preset, performing channel classification prediction on the core semantic word set to obtain a predicted second answer channel category; the second answer channel category is used to identify the domain category of the core semantic word set in the question and answer knowledge base.
[0069] It is worth noting that the first answer channel category is a preset answer channel category, and the second answer channel category is obtained by channel classification prediction.
[0070] In some embodiments, when obtaining multiple answers whose confidence levels are not lower than a minimum threshold, first obtain a preset answer channel category. If no preset answer channel category is set, perform channel classification prediction on the core semantic stimulus, obtain the domain category of the core semantic word set in the question and answer knowledge base, and then perform relevance matching on the core semantic word set with the words in the question and answer knowledge base. Retrieve and match through keyword relevance and word vector relevance to obtain multiple answers. Sort the multiple answers according to their confidence levels, and obtain answers greater than or equal to the minimum threshold so that the obtained answers can be displayed.
[0071] In some embodiments, collecting information through at least any one of text input, online voice dictation or offline voice transcription for several consecutive times and outputting it through text or voice is regarded as a conversation round, that is, each conversation round has multiple conversations. When predicting the channel classification for each conversation, the classification probability of the current second answer channel category is used as a benchmark, and a preset movement probability is used to move from the current classification probability to a neighboring probability of the current classification probability (that is, to maintain the classification of the current domain category), and a preset jump probability is used to jump from the current classification probability to any probability of the current classification probability (that is, to change to a small category close to the current domain category), and the classification probability of the next channel category is predicted; wherein the sum of the movement probability and the jump probability is 1.
[0072] In some embodiments, a random walk search algorithm is used to perform channel classification prediction; in this case, the probability of a random walk jump is the jump probability.
[0073] In some embodiments, the ∈-greedy algorithm is used for channel classification prediction; at this time, the utilization probability of the ∈-greedy algorithm is the movement probability, and the exploration probability of the ∈-greedy algorithm is the jump probability, that is: the ∈ probability is the jump probability, the 1-∈ probability is the movement probability, ∈∈(0,1).
[0074] It is worth mentioning that, since the large domain of the conversation will not change much in a conversation round, the domain scope of the conversation mostly changes between small domains in the large domain. Therefore, when predicting the domain scope in a conversation round, the jump probability and movement probability are used to change between small domains, while keeping the probability of the overall large domain stable.
[0075] In some embodiments, the multiple first answers are sorted according to confidence, and if the number of second answers with confidence not less than a minimum threshold is 0, it is considered that the question is not understood or the answer is not given correctly, and a corresponding flag (such as isAnswer) is returned to inform the user; if the number of second answers with confidence not less than a minimum threshold is not 0, it is considered that the information input by the user has been understood and the answer is given correctly, and the one with the highest confidence is returned to the user as the main answer, and a list of answers for all the multiple second answers is returned for the user to choose from. The specific output method is described in step S13.
[0076] Step S13: sort the multiple answers according to their confidence levels and output them via text or voice.
[0077] It is worth mentioning that the intelligent voice system provides a variety of answer output methods for question intelligence, including text output and real-time intelligent voice output.
[0078] In some embodiments, output is performed via text, including: the intelligent voice system returns the answer with the highest confidence as the main answer to the user, and simultaneously returns a list of answers for all multiple second answers directly in the form of text, so that the user can receive it via text.
[0079] In some embodiments, output is performed through voice, including: synthesizing the multiple answers into a voice file, and receiving the voice file based on a newly established third long connection service with the server; if the voice file is real-time voice, broadcasting the voice file in sequence according to a static global variable that records the number of users connected in real time and the queue order; if the voice file is non-real-time voice, storing the voice file in a buffer and broadcasting it in sequence.
[0080] It is worth mentioning that the third long connection service is established for the scenario of voice output.
[0081] In some embodiments, the multiple answers are synthesized into a voice file, including: after obtaining a voice synthesis request, calling voice-related parameters for setting, and synthesizing the multiple answers into a base64-encoded audio file stream through the voice-related parameters to obtain a voice file, so that the user can play the received voice file in real time to complete the question-and-answer interaction, or save the voice file for delayed playback and playback; wherein the relevant parameters include: engine type (such as hosting in a small language), audio sampling rate, audio encoding format, speech speed, volume, pitch, background sound settings, English pronunciation method and pause style and other initialization parameters, to initialize the voice processing engine
[0082] In some embodiments, a new long connection websocket service TtsWebSocketServerHandler is created on the server side, which is specifically used to receive requests related to synthesized voice files.
[0083] In some embodiments, for requests that require real-time interaction, the number of currently connected users is recorded through the static global variable onlineCount to perform service queuing and flow limiting. For non-real-time interaction, there is no need to queue, and the intelligent answer system buffer can be used.
[0084] In some embodiments, when voice output is performed for real-time scenarios, a queue pile root is constructed again based on the number of users, with the output request time as the first-level priority score and the user priority as the second-level priority score, and the traversal order of the queue pile root is used as the descending queue order, and voice output is performed according to the queue order.
[0085] In some embodiments, if the output request times are different, the user request with the earlier output request time will be used as the parent node of the sorting heap root in the order of request time. When the output request times are the same, the user request with the higher user priority will be used as the parent node of the sorting heap root in the order of user priority.
[0086] In some embodiments, if user priorities are different, user requests with higher user priorities are used as parent nodes of the sorting heap root in order of their priorities. When user priorities are the same, user requests with earlier request times are used as parent nodes of the sorting heap root in order of their output times.
[0087] In some embodiments, when voice output is performed for a non-real-time scene, the voice file is stored in a buffer and broadcasted in sequence.
[0088] In some embodiments, when text input is used, text output can be directly used; when voice online dictation or voice offline transcription is used, output can be directly performed through voice.
[0089] In some embodiments, when information is collected through any of text input, online voice dictation, or offline voice transcription, it can be output through text or voice.
[0090] It is worth mentioning that the intelligent question-answering system has multi-round question-answering capabilities and can conduct multi-round dialogue scenarios of repetition, supplementation and follow-up. Users can ask questions to the intelligent answering system multiple times and repeat the above steps S11 to S13 multiple times to provide strong nonlinear generalization capabilities, which can identify core intentions based on complex contextual content and confirm the most appropriate answer plan.
[0091] The core idea of the intelligent voice question-answering system provided by the present invention is to first use a voice recognition algorithm to convert the question voice information into text information, then perform lexical, grammatical and semantic analysis on the question text information, and cooperate with the question-answering knowledge base to find the answer information that best matches the question and has the highest confidence. Its beneficial effects include:
[0092] (1) By integrating text input, voice input, text output and voice output, an integrated platform of text question-answering system and voice question-answering system is completed, which greatly improves the reuse efficiency and sedimentation capacity of the knowledge base, effectively increases the system service efficiency and avoids repeated waste of resources.
[0093] (2) By combining any two of text input, online audio dictation or offline voice transcription with text or voice, any combination of voice interaction and text interaction of the intelligent question-answering system is realized, thus expanding the application scenarios of the system.
[0094] (3) Match the information with the models in the sentence model library, and match the core semantic word set with the words in the question and answer knowledge base for relevance, and filter the interrogative words and question feature words. A combination of speech recognition and question parsing is used to significantly improve the accuracy compared to traditional solutions. Even when the user's word order changes significantly, the intelligent answering system can still match the correct question and provide an answer.
[0095] (4) The intelligent question-and-answer system is available 24 / 7, which can reduce personnel and resource costs. It can also regulate users' real-time voice requests through secondary indicator sorting, reduce the time complexity of entering and leaving the queue, thereby improving voice processing efficiency and user experience.
[0096] Example 2
[0097] See also Figure 2 , is a structural diagram of an intelligent question-answering device based on voice and text collaboration provided in this embodiment, including: a collection and matching unit 21, a word segmentation and matching unit 22 and an output unit 23.
[0098] Among them, the collection and matching unit 21 mainly collects information, performs sentence model matching on the collected information, and transmits the obtained best matching sentence model to the word segmentation and matching unit 22; after the word segmentation and matching unit 22 receives the best matching sentence model sent by the collection and matching unit 21, it performs word segmentation and relevance matching, obtains multiple answers, and transmits the multiple answers to the output unit 23; after the output unit 23 receives the multiple answers sent by the word segmentation and matching unit 22, it outputs text or voice.
[0099] The collection and matching unit 21 is used to collect information through at least any one of text input, online voice dictation or offline voice transcription, and match the information with the models in the sentence model library, and use the model with the highest similarity as the most matching sentence model for the information.
[0100] In some embodiments, information is collected through the online voice dictation, including: receiving an online dictation user request based on a newly created first long connection service with a server, and recording the number of users currently requested by the user based on a static global variable; constructing a queue pile root based on the number of users, with the request time as the first-level priority score and the user priority as the second-level priority score, using the traversal order of the queue pile root as a descending queue order, and collecting information according to the queue order.
[0101] In some embodiments, information is collected in sequence according to the queueing order, including: in sequence according to the queueing order, the collected real-time first audio file stream is converted into online dictation text in an overlay manner, and the online dictation text is post-processed to obtain text to be matched; the text to be matched is used to match with a model in a sentence model library; the post-processing includes: punctuation and number adjustment, and identification of sensitive words.
[0102] In some embodiments, information is collected through the offline voice transcription, including: receiving an offline user request based on a newly created second long connection service with the server, and obtaining an encoded second audio file stream, and voice-transcribing the second audio file stream based on a cache byte group that is not greater than a storage threshold to collect information.
[0103] In some embodiments, the second audio file stream is voice-transcribed and information is collected based on a cache byte group that is not greater than a storage threshold, including: the second audio file stream is segmented based on a cache byte group that is not greater than a storage threshold, and multiple segmentation results are sequentially voice-transcribed to obtain multiple transcription results, the multiple transcription results are sequentially merged to obtain offline text, and the offline text is post-processed to obtain text to be matched.
[0104] The word segmentation and matching unit 22 is used to perform word segmentation processing on the most matching sentence model, obtain a core semantic word set, perform relevance matching on the core semantic word set with words in the question-answer knowledge base, and obtain multiple answers whose confidence is not less than a minimum threshold.
[0105] In some embodiments, the best matching sentence model is segmented to obtain a core semantic word set, including: filtering question words and question feature words in the best matching sentence model, and segmenting the filtered results to obtain sentence fragments, segmenting the sentence fragments, and removing all auxiliary words in the segmentation results to obtain a core semantic word set.
[0106] In some embodiments, the core semantic word set is relevance matched with words in a question-and-answer knowledge base to obtain multiple answers whose confidence levels are not lower than a minimum threshold, including: relevance matching the core semantic word set with words in the question-and-answer knowledge base in turn, retrieving and matching through keyword relevance and word vector relevance to obtain multiple first answers, and sorting the multiple first answers according to confidence levels to obtain multiple second answers whose confidence levels are not lower than a minimum threshold.
[0107] In some embodiments, before matching the core semantic word set with the words in the question and answer knowledge base in sequence for relevance, it also includes: obtaining a preset first answer channel category, and if the first answer channel category is not preset, performing channel classification prediction on the core semantic word set to obtain a predicted second answer channel category; the second answer channel category is used to identify the domain category of the core semantic word set in the question and answer knowledge base.
[0108] The output unit 23 is used to sort the multiple answers according to their confidence levels and output them via text or voice.
[0109] In some embodiments, output is performed through voice, including: synthesizing the multiple answers into a voice file, and receiving the voice file based on a newly established third long connection service with the server; if the voice file is real-time voice, broadcasting the voice file in sequence according to a static global variable that records the number of users connected in real time and the queue order; if the voice file is non-real-time voice, storing the voice file in a buffer and broadcasting it in sequence.
[0110] This embodiment combines voice and text collection information through the collection and matching unit 21 and outputs answers through the output unit 23, coordinates voice and text, and coordinates online voice dictation and offline voice transcription with the question and answer service system, thereby improving the conversion efficiency between voice and text and question and answer, and collects information through voice, offline transcription and online dictation, and outputs text or voice, which can be suitable for some disabled people and has higher applicability. In addition, through the word segmentation and matching unit 22, a system knowledge base is used to output multiple answers for text, online voice dictation and offline voice transcription, which can improve the utilization rate of the system knowledge base and reduce labor costs. In addition, through the word segmentation and matching unit 22, a two-step matching process of sentence model matching and relevance matching is adopted, which can improve the accuracy of matching different sentence patterns to corresponding answers, thereby improving the user experience.
[0111] Example 3
[0112] See also Figure 3 , is a schematic diagram of the structure of an intelligent question-answering system provided by this embodiment. Based on Embodiment 1 and Embodiment 2, this embodiment provides a complete structure of an intelligent question-answering system. Figure 3 In the invention, the intelligent question and answer system includes: an intelligent question acquisition unit 31, an intelligent question and answer processing unit 32 and an intelligent question answering unit 33; wherein, the intelligent question acquisition unit 31 includes: an intelligent voice engine 301 and a text input unit 302; the intelligent voice engine 301 controls the intelligent voice online dictation unit 101 and the intelligent voice offline transcription unit 102 respectively; the intelligent question and answer processing unit 32 includes: an intelligent question and answer processing engine 311, a sentence model library unit 312, a grammatical semantic analysis module 313 and a system knowledge base 314; the intelligent question answering unit 33 includes: a text output unit 321 and the intelligent voice engine 301, and the intelligent voice engine 301 also controls the intelligent voice output unit 103.
[0113] It is worth noting that the intelligent voice output unit 103 can perform real-time voice output, or perform delayed output or repeated output according to user preferences.
[0114] The intelligent question and answer system can collect information input by the user through the text input unit 302, the intelligent voice online dictation unit 101 or the intelligent voice offline transcription unit 102. If the information is collected through the text input unit 302, the text input unit 302 directly transmits the information to the intelligent question and answer processing engine 311. If the information is collected through the intelligent voice online dictation unit 101 or the intelligent voice offline transcription unit 102, it is necessary to process it through different long connection services respectively, and then transmit the obtained text to be matched to the intelligent question and answer processing engine 311. After receiving the data, the intelligent question and answer processing engine 311 matches it with the sentence model library unit 312. The sentence model library unit 312 transmits the result to the grammatical semantic analysis module 313 for parsing, and transmits the parsing result to the system knowledge base for matching to obtain multiple answers, or the grammatical semantic analysis module 313 directly transmits the parsing result to the intelligent question and answer processing engine 311.
[0115] In some embodiments, the intelligent question acquisition unit 31 collects information through at least any one of text input, online voice dictation, or offline voice transcription.
[0116] In some embodiments, the statement model library unit 312 matches the information transmitted by the intelligent question and answer processing engine 311 with the models in the statement model library, takes the model with the highest similarity as the most matching statement model for the information, and transmits the most matching statement model to the statement model library unit 312.
[0117] In some embodiments, the grammatical semantic analysis module 313 performs word segmentation processing on the most matching sentence model transmitted by the sentence model library unit 312 , obtains a core semantic word set, and transmits the core semantic word set to the system knowledge base 314 .
[0118] In some embodiments, the system knowledge base 314 receives the core semantic word set transmitted by the grammatical semantic analysis module 313, performs relevance matching, obtains multiple answers with confidence levels not lower than a minimum threshold, and transmits the multiple answers to the intelligent question and answer processing engine 311.
[0119] In some embodiments, the intelligent question and answer processing engine 311 outputs multiple data received from the system knowledge base 314 according to different output methods; wherein the multiple answers are sorted according to confidence and output via text or voice.
[0120] The intelligent question and answer processing engine 311 outputs in different ways. If text output is performed, multiple answers are transmitted to the text output unit for direct output. If voice output is performed, multiple answers are transmitted to the intelligent voice output unit 103 for voice synthesis, and then the voice file is played.
[0121] The intelligent answering system provided in this embodiment integrates text input, voice input, text output and voice output to complete the integration platform of text question and answer system and voice question and answer system, greatly improves the reuse efficiency and precipitation capacity of knowledge base, effectively increases the system service efficiency, and avoids repeated waste of resources. By combining any two of text input, online audio dictation or offline voice transcription with text or voice, any combination of voice interaction mode and text interaction mode of intelligent question and answer system is realized, and the application scenario of the system is expanded. The information is matched with the model in the sentence model library, and the core semantic word set is matched with the words in the question and answer knowledge base for relevance, and the interrogative words and question feature words are filtered. The combination of speech recognition and question parsing is adopted, so that the accuracy rate is significantly improved compared with the traditional solution. Even when the user's word order question method changes significantly, the intelligent answering system can still match the correct question and provide an answer. The intelligent question and answer system is realized to receive people all year round, which can reduce personnel and resource costs, and regulate the user's real-time voice request through the secondary index sorting, reduce the time complexity of entering and leaving the team, thereby improving the voice processing efficiency and improving the user experience.
[0122] Those skilled in the art will appreciate that the embodiments of the present application may also provide computer program products. Therefore, the present application may adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the present application may adopt the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program codes.
[0123] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (apparatus), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A system that specifies the functions of a box or multiple boxes.
[0124] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction system, which is implemented in the process Figure 1A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0125] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0126] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. An intelligent question-answering method based on voice and text collaboration, It is characterized in that include: Information is collected through at least one of text input, online voice dictation or offline voice transcription, and the information is matched with the models in the sentence model library, and the model with the highest similarity is used as the most matching sentence model for the information; wherein, information is collected through the offline voice transcription, specifically: according to the newly established second long connection service with the server, an offline user request is received, and an encoded second audio file stream is obtained; according to the cache byte group not greater than the storage threshold, the second audio file stream is segmented, and the multiple segmentation results are sequentially voice-transcribed to obtain multiple transcription results, the multiple transcription results are sequentially merged to obtain offline text, and the offline text is post-processed to obtain text to be matched; Performing word segmentation processing on the most matching sentence model to obtain a core semantic word set, performing relevance matching on the core semantic word set and words in a question-answering knowledge base to obtain a plurality of answers whose confidence levels are not less than a minimum threshold; The multiple answers are sorted according to confidence and output via text or voice.
2. The intelligent question-answering method based on voice and text collaboration as claimed in claim 1, It is characterized in that Through the online voice dictation, information is collected, including: According to the first long connection service newly established with the server, an online dictation user request is received, and the number of users requested by the current user is recorded according to a static global variable; According to the number of users, with request time as the first-level priority score and user priority as the second-level priority score, a queue heap root is constructed, and the traversal order of the queue heap root is used as a queue order in descending order, and information is collected according to the queue order.
3. The intelligent question-answering method based on voice and text collaboration as claimed in claim 2, It is characterized in that The collecting information in sequence according to the queuing order includes: According to the queuing order, the collected real-time first audio file stream is converted into online dictation text in an overlay manner, and the online dictation text is post-processed to obtain the text to be matched; the text to be matched is used to match with the model in the sentence model library; the post-processing includes: punctuation and number adjustment, and identification of sensitive words.
4. The intelligent question-answering method based on voice and text collaboration as claimed in claim 1, It is characterized in that The performing word segmentation processing on the best matching sentence model to obtain a core semantic word set includes: The interrogative words and question feature words in the most matching sentence model are filtered, and the obtained filtering results are segmented to obtain sentence fragments, and the sentence fragments are segmented, and all auxiliary words in the segmentation results are removed to obtain a core semantic word set.
5. The intelligent question-answering method based on voice and text collaboration as claimed in claim 1, It is characterized in that The step of matching the core semantic word set with words in the question-answer knowledge base for relevance to obtain multiple answers with confidence levels not less than a minimum threshold includes: The core semantic word set is matched with the words in the question-answer knowledge base in turn by relevance, and the matching is retrieved through keyword relevance and word vector relevance to obtain multiple first answers, and the multiple first answers are sorted according to confidence to obtain multiple second answers whose confidence is not less than a minimum threshold.
6. The intelligent question-answering method based on voice and text collaboration as claimed in claim 5, It is characterized in that Before sequentially matching the core semantic word set with the words in the question-answering knowledge base for relevance, the method further includes: Obtain a preset first answer channel category. If the first answer channel category is not preset, perform channel classification prediction on the core semantic word set to obtain a predicted second answer channel category; the second answer channel category is used to identify the domain category of the core semantic word set in the question and answer knowledge base.
7. The intelligent question-answering method based on voice and text collaboration as claimed in claim 1, It is characterized in that Output via voice, including: synthesize the multiple answers into a voice file, and receive the voice file according to the newly established third long connection service with the server; If the voice file is real-time voice, the voice file is broadcasted in sequence according to the static global variable recording the number of users connected in real time and the queue order; If the voice file is non-real-time voice, the voice file is stored in a buffer and broadcasted in sequence.
8. An intelligent question-answering device based on voice and text collaboration, It is characterized in that include: Collection and matching unit, word segmentation and matching unit and output unit; wherein, The collection and matching unit is used to collect information through at least any one of text input, online voice dictation or offline voice transcription, and match the information with the models in the sentence model library, and use the model with the highest similarity as the most matching sentence model for the information; wherein, collecting information through the offline voice transcription is specifically: receiving an offline user request according to a newly created second long connection service with the server, and obtaining an encoded second audio file stream; segmenting the second audio file stream according to a cache byte group not greater than a storage threshold, and sequentially performing voice transcription on the obtained multiple segmentation results to obtain multiple transcription results, sequentially merging the multiple transcription results to obtain offline text, and post-processing the offline text to obtain text to be matched; The word segmentation and matching unit is used to perform word segmentation processing on the most matching sentence model, obtain a core semantic word set, perform relevance matching on the core semantic word set with words in the question-answer knowledge base, and obtain multiple answers whose confidence levels are not lower than a minimum threshold; The output unit is used to sort the multiple answers according to confidence and output them via text or voice.
Citation Information
Patent Citations
Voice answering method for combining intelligent answer with artificial answer
CN107315766A
Speech transcription method, device and equipment and readable storage medium
CN108845979A
Intelligent question and answer system and method based on knowledge base
CN109977204A
Voice recognition method and system
CN111009244A
Hypertension question-answering method and system based on deep learning
CN116737911A
