A method and system for text summarization using speech

Abstract

The application discloses a method and system for text excerpting by using voice, which can make a user quickly and accurately excerpt a corresponding book fragment by reading text content and with the aid of automatic completion technology. The technical scheme is as follows: converting information in a user voice signal into text; obtaining candidate sentences from preselected book contents according to the converted text; generating a candidate fragment according to the obtained candidate sentences; and generating a book fragment based on the candidate fragment.

Description

Technical Field

[0001] This invention relates to a text extraction technique, specifically to a method and system for text extraction using voice technology. Background Technology

[0002] Book excerpts, or book summaries for short, are the process of extracting excellent sentences and passages from books. They are a common method in Chinese language teaching and daily reading, helping readers to better understand the content of books, deepen their memory, and review what they have learned.

[0003] Traditional methods of book excerpting rely on manual copying, which is time-consuming, labor-intensive, and lacks the ability to permanently archive or widely share the excerpted content. With the development of speech recognition technology, Automatic Speech Recognition (ASR) has enabled the conversion of readers' readings into electronic text fragments for recording and storage. However, this technology still has several problems. First, humans make mistakes; readers inevitably add, omit, or mispronounce words, resulting in ASR results that don't accurately reflect the original text. Second, current ASR technology still cannot guarantee the accuracy of text recognition, requiring manual corrections to the ASR results, significantly reducing work efficiency.

[0004] Furthermore, whether using manual transcription or ASR recognition, readers are required to copy or read aloud the entire excerpt. When the excerpt is lengthy, this method becomes somewhat inefficient. Moreover, the above only addresses the transcription process itself; subsequent management, arrangement, and sharing of the excerpted content still require manual intervention. Summary of the Invention

[0005] The following provides a brief overview of one or more aspects to offer a basic understanding of them. This overview is not an exhaustive summary of all conceived aspects, nor is it intended to identify key or decisive elements of all aspects, nor to define the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form to prepare for the more detailed descriptions that follow.

[0006] The purpose of this invention is to solve the above-mentioned problems and provide a method and system for text extraction using voice. This allows users to quickly and accurately extract relevant book passages by reading the text aloud, supplemented by automatic completion technology. The extracted content can then be managed, arranged, and further shared through comments and distribution.

[0007] The technical solution of this invention is as follows: This invention discloses a method for text extraction using speech, the method comprising:

[0008] Convert information from the user's voice signal into text;

[0009] From a pre-selected list of books, candidate sentences are obtained based on the converted text;

[0010] Generate candidate fragments based on the obtained candidate sentences;

[0011] Book fragments are generated based on candidate fragments.

[0012] According to an embodiment of the method for text extraction using speech according to the present invention, after the step of converting information in the user's speech signal into text, the method further includes:

[0013] The converted text is then corrected.

[0014] According to one embodiment of the method for text extraction using speech according to the present invention, the pre-selected bibliography is determined based on the converted text.

[0015] According to one embodiment of the method for text extraction using speech according to the present invention, the process of obtaining candidate sentences from a pre-selected bibliography is to use an information retrieval algorithm to select candidate sentences from the pre-selected bibliography based on the converted text, wherein the objects of information retrieval include the frequency of text and / or corresponding pinyin, edit distance, Jaccard similarity, and cosine similarity.

[0016] According to an embodiment of the method for text extraction using speech according to the present invention, the step of generating candidate segments based on the obtained candidate sentences further includes:

[0017] Based on the candidate sentences relative to the converted text, the candidate sentences are initially screened. The initial screening criteria include the number, proportion, and dispersion of added characters, misspelled characters, and matching characters.

[0018] Based on the converted text, the initially selected candidate sentences are reordered, and missing sentences are automatically filled in to obtain candidate segments, so that the word order of the candidate segments is the same as that of the converted text.

[0019] According to an embodiment of the method for text extraction using speech according to the present invention, the process of reordering the initially selected candidate sentences to obtain candidate segments includes:

[0020] First, exhaustively enumerate all permutations and combinations of candidate sentences. Then, for each combination, arrange the candidate sentences from smallest to largest according to the average starting position of the matching characters of each candidate sentence relative to the converted text. Calculate the matching score based on the numbering order and interval of the arranged candidate sentences, as well as the features of the arranged candidate sentences relative to the converted text, including the number, proportion, and dispersion of added characters, misspellings, and matching characters. Select the combination with the highest matching score as the candidate segment; or

[0021] By using a neural network model to process generative question answering tasks, candidate segments are generated based on candidate sentences. This includes: inputting the number and content of each candidate sentence, as well as the converted text; the neural network model outputs a corresponding list of numbers; and the corresponding candidate sentences are arranged according to the output list of numbers to obtain candidate segments.

[0022] According to an embodiment of the method for text extraction using speech according to the present invention, the step of generating book fragments based on candidate fragments further includes:

[0023] Directly use the candidate excerpts as excerpts from the book; or

[0024] The candidate excerpts and the reordered list of candidate sentences are presented to the user through human-computer interaction, allowing the user to confirm that the excerpt they want to extract is from the book.

[0025] According to an embodiment of the method for text extraction using speech according to the present invention, the method further includes:

[0026] Manage, comment on, and import / export book excerpts.

[0027] This invention also discloses a system for text extraction using speech, the system comprising:

[0028] The speech-to-text module is used to convert information in the user's voice signal into text;

[0029] The candidate sentence acquisition module, connected to the speech-to-text module, is used to acquire candidate sentences from a pre-selected list of books based on the converted text.

[0030] The candidate fragment acquisition module, connected to the candidate sentence acquisition module, is used to generate candidate fragments based on the acquired candidate sentences;

[0031] The book fragment determination module is connected to the candidate fragment acquisition module and is used to generate text fragments based on the candidate fragments.

[0032] According to one embodiment of the system for text extraction using speech according to the present invention, the system further includes:

[0033] The text correction module is connected to the speech-to-text module and the candidate sentence acquisition module, and is used to correct the converted text.

[0034] According to one embodiment of the system for text extraction using speech according to the present invention, the system further includes:

[0035] The bibliography determination module, which precedes the candidate sentence acquisition module, is used to determine the source bibliography of candidate sentences based on the converted text.

[0036] According to one embodiment of the system for text extraction using speech according to the present invention, the candidate sentence acquisition module is further configured to use an information retrieval algorithm to acquire candidate sentences from a bibliography based on the converted text, wherein the objects of information retrieval include the frequency of text and / or corresponding pinyin, edit distance, Jaccard similarity, and cosine similarity.

[0037] According to an embodiment of the system for text extraction using speech according to the present invention, the candidate segment acquisition module is further configured as follows:

[0038] Based on the candidate sentences relative to the converted text, the candidate sentences are initially screened. The initial screening criteria include the number, proportion, and dispersion of added characters, misspelled characters, and matching characters.

[0039] Based on the converted text, the initially selected candidate sentences are reordered, and missing sentences are automatically filled in to obtain candidate segments, so that the word order of the candidate segments is the same as that of the converted text.

[0040] According to an embodiment of the system for text extraction using speech according to the present invention, the process of reordering the initially selected candidate sentences to obtain candidate segments in the candidate segment acquisition module further includes:

[0041] First, exhaustively enumerate all permutations and combinations of candidate sentences. Then, for each combination, arrange the candidate sentences from smallest to largest according to the average starting position of the matching characters of each candidate sentence relative to the converted text. Calculate the matching score based on the numbering order and interval of the arranged candidate sentences, as well as the features of the arranged candidate sentences relative to the converted text, including the number, proportion, and dispersion of added characters, misspellings, and matching characters. Select the combination with the highest matching score as the candidate segment; or

[0042] By using a neural network model to process generative question answering tasks, candidate segments are generated based on candidate sentences. This includes: inputting the number and content of each candidate sentence, as well as the converted text; the neural network model outputs a corresponding list of numbers; and the corresponding candidate sentences are arranged according to the output list of numbers to obtain candidate segments.

[0043] According to one embodiment of the system for text extraction using speech according to the present invention, the book segment determination module is further configured to present candidate segments and a reordered list of candidate sentences to the user through human-computer interaction, so that the user can confirm that the segment to be extracted is a book segment.

[0044] According to one embodiment of the system for text extraction using speech according to the present invention, the system further includes:

[0045] The book excerpt management module is used to manage book excerpts.

[0046] The book excerpt commentary module is used to provide book excerpts and commentary; and

[0047] The book excerpt import / export module is used to import and export book excerpts.

[0048] This invention also discloses a smart terminal for text extraction using voice, the smart terminal comprising:

[0049] Input devices;

[0050] Output devices;

[0051] processor;

[0052] The memory is configured to store a set of computer-executable instructions and computer-accessible data associated with the set of computer-executable instructions, which, when executed by the processor, cause the processor to perform the method described above.

[0053] The present invention also discloses a non-transitory computer-readable storage medium storing a series of computer-executable instructions, which, when executed by a computing device, cause the computing device to perform the method described above.

[0054] Compared with existing technologies, this invention has the following advantages: This invention converts the user's spoken audio signal into text content, corrects the transcribed text, and uses the corrected transcribed text to determine the user's reading material, further obtaining candidate sentences from the determined reading material. By filtering, sorting, and completing the candidate sentences, the excerpt to be extracted is obtained, enabling the storage, management, backup, and sharing of the excerpt content and corresponding comments. This invention allows users to quickly and accurately locate the entire content of a book excerpt simply by reading its beginning and end, and to store, manage, comment on, and share the excerpt accordingly, significantly improving the user experience. Attached Figure Description

[0055] The above-described features and advantages of the present invention will be better understood after reading the following detailed description of embodiments of the present disclosure in conjunction with the accompanying drawings. In the drawings, components are not necessarily drawn to scale, and components having similar related characteristics or features may have the same or similar reference numerals.

[0056] Figure 1 A flowchart of an embodiment of the method for text extraction using speech according to the present invention is shown.

[0057] Figure 2 A structural diagram of an embodiment of the system for text extraction using speech according to the present invention is shown.

[0058] Figure 3 A structural diagram of a smart terminal applying the method of the present invention is shown. Detailed Implementation

[0059] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. It should be noted that the aspects described below with reference to the accompanying drawings and specific embodiments are merely exemplary and should not be construed as limiting the scope of protection of the present invention in any way.

[0060] Figure 1 The flowchart of an embodiment of the method for text extraction using speech according to the present invention is shown. Please refer to... Figure 1 The implementation flow of each step of the method in this embodiment is as follows.

[0061] Step S101: Convert the information in the user's voice signal into text.

[0062] Specifically, the terminal acquires the user's voice input (such as content read aloud) and converts the voice information into text information through automatic speech recognition technology.

[0063] Step S102: Correct the transcribed text after speech recognition in step S101.

[0064] Specifically, through operations such as replacing homophones, near-homophones, synonyms, and / or near-synonyms, as well as inserting and / or deleting certain characters, a massive amount of book content is converted. The text processed by the aforementioned operations is used as the text before correction, and the original text before the aforementioned operations is used as the text after correction. A machine learning model is trained, and this machine learning model is used to correct the transcribed text obtained in step S101. The machine learning model here can be a neural network model with an encoder-decoder architecture, such as a recurrent neural network (RNN) or a Transformer, or a traditional machine learning model based on a statistical machine translation architecture.

[0065] Step S102 can improve the technical effect of the present invention, but it can also be omitted in another embodiment. That is, the text after speech recognition is used directly for subsequent processing without correcting speech recognition errors.

[0066] Step S103: Determine the reading material for the user based on the corrected transcribed text.

[0067] This process of determining the reading list is equivalent to a text classification task. It employs a machine learning model, taking the corrected transcribed text obtained in step S102 as input, and outputting the corresponding reading list. The list is then displayed to the reader in descending order of classification scores, and the reader ultimately confirms the reading list through human-computer interaction. The machine learning model here includes, but is not limited to, text classification models such as neural networks, decision trees, and support vector machines. The training data for the machine learning model can be obtained by randomly or intentionally selecting a number of sentences or passages from a massive amount of book content.

[0068] Similarly, step S103 can improve the technical effect of the present invention, but it can also be omitted in another embodiment, that is, the text after speech recognition is used directly for subsequent processing, without pre-screening a range of reading materials.

[0069] Step S104: Obtain candidate sentences from the determined reading list.

[0070] Specifically, information retrieval algorithms are used, including but not limited to the BM25 algorithm, neural network algorithms, etc. The features used by the algorithms include but not limited to the frequency of text and / or corresponding pinyin, edit distance, Jaccard similarity, cosine similarity, etc. Based on the corrected transcribed text obtained in step S102, several sentences are selected as candidate sentences from the corresponding sentences of the book determined in step S103, in descending order of information retrieval algorithm scores.

[0071] In one processing example of this step, the retrieval score for the corrected transcribed text Q and a certain bibliographic sentence d is: Where, q i Let W be the i-th morpheme in Q, n be the number of morphemes in Q, and W be the morpheme in Q. i The morpheme q i The weight.

[0072] W i There are several methods for calculating IDF, the most commonly used being IDF. Where N is the total number of sentences in the bibliography, n(q) i ) contains q i The number of sentences in the bibliography. R(q) i ,d) is the morpheme q iThe relevance score of sentence d in the bibliography is calculated using the following formula: in k1, k2, and b are adjustment factors, which are constants, and f i For q i The frequency of occurrence in d, qf i For q i In Q, the frequency of occurrence is given, dl represents the length of d, and avgdl represents the average length of sentences across all titles. It's important to note that the morphemes mentioned above can be Chinese characters, words, pinyin of Chinese characters, pinyin of words, or bigrams and trigrams. The final search score can be the search score for one type of morpheme or the sum of search scores for multiple morpheme types.

[0073] Step S105: Generate candidate fragments based on the candidate sentences obtained in step S104.

[0074] Specifically, step S105 includes the following steps:

[0075] (1) Based on the conditions of the number, proportion, and dispersion of added, misspelled, and matched characters in the candidate sentences relative to the corrected transcribed text obtained in step S102, a preliminary screening is performed on the candidate sentences to select those that meet the corresponding conditions. These conditions can be manual rules, such as the number, proportion, and dispersion of added, misspelled, and matched characters, each of which must be greater than or less than its corresponding threshold; or they can be trained using machine learning models (including but not limited to decision trees, logistic regression, etc.) to determine whether the conditions are met. Specifically, the dispersion can be characterized by various indicators, such as the number of deleted blocks (a single deleted character or consecutive deleted characters are considered a deleted block) or added blocks (a single added character or consecutive added characters are considered an added block), or the ratio of the number of deleted or added blocks to the length of the corrected transcribed text, etc.

[0076] (2) Based on the corrected transcribed text, the candidate sentences initially screened in step (1) are reordered and the sentences missed by the intermediate users are automatically completed to obtain candidate segments, so that the word order of the candidate segments is the same as that of the corrected transcribed text.

[0077] The autocomplete process is as follows: If there are skipped sentences in the reordered candidate sentences, the following steps are taken depending on whether the candidate sentences are ordered or out of order: Case 1) If the reordered candidate sentences are ordered (i.e., arranged according to their order of appearance in the book), all missing parts between the reordered candidate sentences are automatically completed; Case 2) If the reordered candidate sentences are out of order, the first and last sentences of the reordered candidate sentences are selected, the missing parts between them are completed, and the other candidate sentences are discarded. If the last sentence of a reordered candidate sentence appears before the first sentence of the reordered candidate sentence in the book, an error is displayed to the user, and subsequent steps are skipped.

[0078] It is important to note that the reordering process will further filter candidate sentences, and some candidate sentences that have passed the initial screening may be discarded during the reordering process.

[0079] Reordering algorithms include, but are not limited to, the following algorithms, where algorithm a and algorithm b are distinct examples:

[0080] a) Exhaustively enumerate all permutations and combinations of candidate sentences. Each candidate sentence has two options: select or not select. With n candidate sentences, there are 2n-1 possible combinations (excluding the case where no sentences are selected). For each combination, rank the candidate sentences in ascending order based on the average starting position of the matching characters relative to the corrected transcribed text. Calculate the matching score based on the order and interval of the ranked sentences, as well as the number, proportion, and dispersion of added, misspelled, and matching characters relative to the corrected transcribed text. Select the combination with the highest matching score as the candidate segment.

[0081] There are several ways to calculate the matching score here. For example, the linear calculation formula is score=k1·ins+k2·sub+k3·mat+k4·inr+k5·sur+k6·mar+k7·dsp, where ins represents the number of added characters, sub represents the number of deleted characters, mat represents the number of matched characters, inr represents the proportion of added characters, sur represents the proportion of misspelled characters, mar represents the proportion of matched characters, dsp represents the degree of dispersion, k1, k2, k4, k5, and k7 are penalty coefficients, which are usually negative constants, and k3 and k6 are reward coefficients, which are usually positive numbers.

[0082] (b) Treat the generation of candidate segments as a generative question-answering task, using neural network models with an encoder-decoder architecture, such as Recurrent Neural Networks (RNNs) or Transformers. Input the number and content of each candidate sentence, along with the corrected transcribed text. The model outputs a list of corresponding numbers. The candidate sentences are then arranged according to the output list of numbers to obtain candidate segments. Training data for the model can be obtained by randomly or intentionally selecting a number of sentences or passages from a large amount of book content.

[0083] Step S106: Obtain the final book fragment based on the candidate fragments.

[0084] Specifically, the candidate excerpts obtained in step S105, along with the reordered list of candidate sentences obtained in step S105, are presented to the reader through a human-computer interaction. The reader then confirms the excerpt they wish to extract, which becomes the final book excerpt. The final book excerpt can then be stored.

[0085] Step S106 can be omitted, for example, the candidate excerpt from step S105 can be used directly as the final book excerpt (i.e., without the reader's confirmation).

[0086] Figure 2 The structure of one embodiment of the system for text extraction using speech according to the present invention is shown. Please see below. Figure 2 The system in this embodiment includes: a speech-to-text module, a candidate sentence acquisition module, a candidate segment acquisition module, and a book segment determination module.

[0087] The data output from the speech-to-text module is sent to the candidate sentence acquisition module, the data output from the candidate sentence acquisition module is sent to the candidate segment acquisition module, and the data output from the candidate segment acquisition module is sent to the book segment determination module.

[0088] The bibliographies, sentences, and excerpts involved in the system are all provided by the data stored in the book content storage module.

[0089] The speech-to-text module is used to convert information in a user's voice signal into text. Specifically, the terminal acquires the user's input voice signal (e.g., content read aloud) and converts the voice information into text information using automatic speech recognition technology.

[0090] Preferably, a text correction module is also included between the speech-to-text module and the candidate sentence acquisition module to correct the converted text.

[0091] Specifically, through operations such as replacing homophones, near-homophones, synonyms, and / or near-synonyms, as well as inserting and / or deleting certain characters, a massive amount of book content is converted. The text processed by the aforementioned operations is used as the text before correction, and the original text before the aforementioned operations is used as the text after correction. A machine learning model is trained, and this machine learning model is used to correct the transcribed text obtained in step S101. The machine learning model here can be a neural network model with an encoder-decoder architecture, such as a recurrent neural network (RNN) or a Transformer, or a traditional machine learning model based on a statistical machine translation architecture.

[0092] Preferably, a bibliographic determination module is included between the text correction module and the candidate sentence acquisition module. This module determines the source bibliography of the candidate sentences based on the converted text. This bibliographic determination module functions like a text classification task, employing a machine learning model. It takes the corrected transcribed text as input and outputs the corresponding book titles, displaying them to the reader in descending order of classification scores. The reader then confirms the book title through human-computer interaction. The machine learning model here includes, but is not limited to, text classification models such as neural networks, decision trees, and support vector machines. The training data for the machine learning model can be obtained by randomly or intentionally selecting a number of sentences or passages from a vast amount of book content.

[0093] The candidate sentence acquisition module is used to obtain candidate sentences from a pre-selected list of books based on the converted text.

[0094] The candidate sentence acquisition module is further configured to use information retrieval algorithms, including but not limited to the BM25 algorithm, neural network algorithms, etc. The features used by the algorithm include but not limited to the frequency of the text and / or the corresponding pinyin, edit distance, Jaccard similarity, cosine similarity, etc. Based on the corrected transcribed text obtained in step S102, several sentences are selected as candidate sentences from the corresponding sentences of the determined book titles in descending order of information retrieval algorithm scores.

[0095] In one example, for the corrected transcribed text Q and a certain bibliographic sentence d, the retrieval score is: Where, q i Let W be the i-th morpheme in Q, n be the number of morphemes in Q, and W be the morpheme in Q. i The morpheme q i The weight.

[0096] W i There are several methods for calculating IDF, the most commonly used being IDF. Where N is the total number of sentences in the bibliography, n(q)i ) contains q i The number of sentences in the bibliography. R(q) i ,d) is the morpheme q i The relevance score of sentence d in the bibliography is calculated using the following formula: in k1, k2, and b are adjustment factors, which are constants, and f i For q i The frequency of occurrence in d, qf i For q i In Q, the frequency of occurrence is given, dl represents the length of d, and avgdl represents the average length of sentences across all titles. It's important to note that the morphemes mentioned above can be Chinese characters, words, pinyin of Chinese characters, pinyin of words, or bigrams and trigrams. The final search score can be the search score for one type of morpheme or the sum of search scores for multiple morpheme types.

[0097] The candidate fragment acquisition module is used to generate candidate fragments based on the acquired candidate sentences.

[0098] The candidate fragment acquisition module is further configured as follows:

[0099] (1) Based on the number, proportion, and dispersion of added, misspelled, and matched characters in the candidate sentences relative to the corrected transcribed text, a preliminary screening is performed to select candidate sentences that meet the corresponding conditions. These conditions can be manual rules, such as the number, proportion, and dispersion of added, misspelled, and matched characters, each of which must be greater than or less than its corresponding threshold; or they can be determined by training a machine learning model (including but not limited to decision trees, logistic regression, etc.). Specifically, the dispersion can be characterized by various indicators, such as the number of deleted blocks (a single deleted character or consecutive deleted characters are considered a deleted block) or added blocks (a single added character or consecutive added characters are considered an added block), or the ratio of the number of deleted or added blocks to the length of the corrected transcribed text, etc.

[0100] (2) Based on the corrected transcribed text, the candidate sentences initially screened in step (1) are reordered and the sentences missed by the intermediate users are automatically completed to obtain candidate segments, so that the word order of the candidate segments is the same as that of the corrected transcribed text.

[0101] The autocomplete process is as follows: If there are skipped sentences in the reordered candidate sentences, the following steps are taken depending on whether the candidate sentences are ordered or out of order: Case 1) If the reordered candidate sentences are ordered (i.e., arranged according to their order of appearance in the book), all missing parts between the reordered candidate sentences are automatically completed; Case 2) If the reordered candidate sentences are out of order, the first and last sentences of the reordered candidate sentences are selected, the missing parts between them are completed, and the other candidate sentences are discarded. If the last sentence of a reordered candidate sentence appears before the first sentence of the reordered candidate sentence in the book, an error is displayed to the user, and subsequent steps are skipped.

[0102] It is important to note that the reordering process will further filter candidate sentences, and some candidate sentences that have passed the initial screening may be discarded during the reordering process.

[0103] Reordering algorithms include, but are not limited to, the following algorithms, where algorithm a and algorithm b are distinct examples:

[0104] a) Exhaustively enumerate all permutations and combinations of candidate sentences. Each candidate sentence has two options: select or not select. With n candidate sentences, there are 2n-1 possible combinations (excluding the case where no sentences are selected). For each combination, rank the candidate sentences in ascending order based on the average starting position of the matching characters relative to the corrected transcribed text. Calculate the matching score based on the order and interval of the ranked sentences, as well as the number, proportion, and dispersion of added, misspelled, and matching characters relative to the corrected transcribed text. Select the combination with the highest matching score as the candidate segment.

[0105] There are several ways to calculate the matching score here. For example, the linear calculation formula is score=k1·ins+k2·sub+k3·mat+k4·inr+k5·sur+k6·mar+k7·dsp, where ins represents the number of added characters, sub represents the number of deleted characters, mat represents the number of matched characters, inr represents the proportion of added characters, sur represents the proportion of misspelled characters, mar represents the proportion of matched characters, dsp represents the degree of dispersion, k1, k2, k4, k5, and k7 are penalty coefficients, which are usually negative constants, and k3 and k6 are reward coefficients, which are usually positive numbers.

[0106] (b) Treat the generation of candidate segments as a generative question-answering task, using neural network models with an encoder-decoder architecture, such as Recurrent Neural Networks (RNNs) or Transformers. Input the number and content of each candidate sentence, along with the corrected transcribed text. The model outputs a list of corresponding numbers. The candidate sentences are then arranged according to the output list of numbers to obtain candidate segments. Training data for the model can be obtained by randomly or intentionally selecting a number of sentences or passages from a large amount of book content.

[0107] The book fragment identification module is used to generate text fragments based on candidate fragments. This module is further configured to present the candidate fragments and a reordered list of candidate sentences to the user through a human-computer interaction, allowing the user to confirm that the desired excerpt is from the book. Alternatively, in another embodiment, the user confirmation method can be omitted, and the candidate fragments obtained by the candidate fragment acquisition module can be directly used as the book fragments.

[0108] Preferably, the system also includes modules for further use of book excerpts: a book excerpt management module for managing book excerpts; a book excerpt commentary module for commenting on book excerpts; and a book excerpt import / export module for importing and exporting book excerpts.

[0109] like Figure 3 As shown, the present invention also discloses a smart terminal applying the above method. The smart terminal includes an input device, an output device, a processor, and a memory. The memory is configured to store a series of computer-executable instructions and computer-accessible data associated with the series of computer-executable instructions.

[0110] When this series of computer-executable instructions is executed by the processor, it causes the processor to perform the actions described above. Figure 1 The method described in the illustrated embodiments.

[0111] Furthermore, the present invention also discloses a non-transitory computer-readable storage medium storing a series of computer-executable instructions, which, when executed by a computing device, cause the computing device to perform the aforementioned actions. Figure 1 The method described in the illustrated embodiments.

[0112] Although the methods described above are illustrated and depicted as a series of actions for the sake of simplicity, it should be understood and appreciated that these methods are not limited by the order of the actions, as some actions may occur in a different order and / or concurrently with other actions from the illustrations and descriptions herein or not illustrated and described herein but which may be understood by those skilled in the art, according to one or more embodiments.

[0113] Those skilled in the art will further appreciate that the various illustrative logic blocks, modules, circuits, and algorithm steps described in conjunction with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, the various illustrative components, blocks, modules, circuits, and steps are described above in a generalized manner in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in different ways for each specific application, but such implementation decisions should not be construed as departing from the scope of the invention.

[0114] The various illustrative logic blocks, modules, and circuits described in conjunction with the embodiments disclosed herein can be implemented or performed using a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. The general-purpose processor may be a microprocessor, but in alternatives, it may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors cooperating with a DSP core, or any other such configuration.

[0115] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of both. The software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor such that the processor can read and write information to / from the storage medium. In an alternative, the storage medium may be integrated into the processor. The processor and storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In an alternative, the processor and storage medium may reside as discrete components in the user terminal.

[0116] In one or more exemplary embodiments, the described functionality may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software as a computer program product, the functionality may be stored or transmitted as one or more instructions or code on or through a computer-readable medium. A computer-readable medium includes both computer storage media and communication media, encompassing any medium that facilitates the transfer of a computer program from one location to another. A storage medium may be any available medium accessible to a computer. By way of example and not limitation, such a computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and is accessible to a computer. Any connection is also legitimately referred to as a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of a medium. As used in this article, disk and disc include compact discs (CDs), laser discs, optical discs, digital multi-purpose discs (DVDs), floppy disks, and Blu-ray discs. Disks typically reproduce data magnetically, while discs reproduce data optically using lasers. Combinations of these should also be included within the scope of computer-readable media.

[0117] The prior description of this disclosure is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to this disclosure will be apparent to those skilled in the art, and the general principles defined herein may be applied to other variations without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not intended to be limited to the examples and designs described herein, but should be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for text extraction using speech, characterized in that, The methods include: Convert information from the user's voice signal into text; From a pre-selected list of books, candidate sentences are obtained based on the converted text; Generate candidate fragments based on the obtained candidate sentences; Generate book excerpts based on candidate excerpts; The step of generating candidate fragments based on the obtained candidate sentences further includes: Based on the candidate sentences relative to the converted text, the candidate sentences are initially screened. The initial screening criteria include the number, proportion, and dispersion of added characters, misspelled characters, and matching characters. Based on the converted text, the initially selected candidate sentences are reordered, and missing sentences are automatically filled in to obtain candidate segments, so that the word order of the candidate segments is the same as that of the converted text. The process of reordering the initially selected candidate sentences to obtain candidate segments includes: First, exhaustively enumerate all permutations and combinations of candidate sentences. Then, for each combination, arrange the candidate sentences from smallest to largest according to the average starting position of the matching characters of each candidate sentence relative to the converted text. Calculate the matching score based on the numbering order and interval of the arranged candidate sentences, as well as the features of the arranged candidate sentences relative to the converted text, including the number, proportion, and dispersion of added characters, misspellings, and matching characters. Select the combination with the highest matching score as the candidate segment; or By using a neural network model to process generative question answering tasks, candidate segments are generated based on candidate sentences. This includes: inputting the number and content of each candidate sentence, as well as the converted text; the neural network model outputs a corresponding list of numbers; and the corresponding candidate sentences are arranged according to the output list of numbers to obtain candidate segments.

2. The method for text extraction using speech according to claim 1, characterized in that, Following the step of converting information from the user's voice signal into text, the following also applies: The converted text is then corrected.

3. The method for text extraction using speech according to claim 1, characterized in that, The pre-selected books are determined based on the converted text.

4. The method for text extraction using speech according to claim 1, characterized in that, The process of obtaining candidate sentences from the pre-selected books uses an information retrieval algorithm. Based on the transformed text, candidate sentences are selected from the pre-selected books. The information retrieval objects include the frequency of the text and / or the corresponding pinyin, edit distance, Jaccard similarity, and cosine similarity.

5. The method for text extraction using speech according to claim 1, characterized in that, The steps of generating book fragments based on candidate fragments further include: Directly use the candidate excerpts as excerpts from the book; or The candidate excerpts and the reordered list of candidate sentences are presented to the user through human-computer interaction, allowing the user to confirm that the excerpt they want to extract is from the book.

6. The method for text extraction using speech according to claim 1, characterized in that, The method also includes: Manage, comment on, and import / export book excerpts.

7. A system for text extraction using speech, characterized in that, The system includes: The speech-to-text module is used to convert information in the user's voice signal into text; The candidate sentence acquisition module, connected to the speech-to-text module, is used to acquire candidate sentences from a pre-selected list of books based on the converted text. The candidate fragment acquisition module, connected to the candidate sentence acquisition module, is used to generate candidate fragments based on the acquired candidate sentences; The book fragment determination module is connected to the candidate fragment acquisition module and is used to generate text fragments based on the candidate fragments. The candidate fragment acquisition module is further configured as follows: Based on the candidate sentences relative to the converted text, the candidate sentences are initially screened. The initial screening criteria include the number, proportion, and dispersion of added characters, misspelled characters, and matching characters. Based on the converted text, the initially selected candidate sentences are reordered, and missing sentences are automatically filled in to obtain candidate segments, so that the word order of the candidate segments is the same as that of the converted text. The process of reordering the initially selected candidate sentences to obtain candidate fragments in the candidate fragment acquisition module further includes: First, exhaustively enumerate all permutations and combinations of candidate sentences. Then, for each combination, arrange the candidate sentences from smallest to largest according to the average starting position of the matching characters of each candidate sentence relative to the converted text. Calculate the matching score based on the numbering order and interval of the arranged candidate sentences, as well as the features of the arranged candidate sentences relative to the converted text, including the number, proportion, and dispersion of added characters, misspellings, and matching characters. Select the combination with the highest matching score as the candidate segment; or By using a neural network model to process generative question answering tasks, candidate segments are generated based on candidate sentences. This includes: inputting the number and content of each candidate sentence, as well as the converted text; the neural network model outputs a corresponding list of numbers; and the corresponding candidate sentences are arranged according to the output list of numbers to obtain candidate segments.

8. The system for text extraction using speech according to claim 7, characterized in that, The system also includes: The text correction module is connected to the speech-to-text module and the candidate sentence acquisition module, and is used to correct the converted text.

9. The system for text extraction using speech according to claim 7, characterized in that, The system also includes: The bibliography determination module, which precedes the candidate sentence acquisition module, is used to determine the source bibliography of candidate sentences based on the converted text.

10. The system for text extraction using speech according to claim 7, characterized in that, The candidate sentence acquisition module is further configured to use an information retrieval algorithm to obtain candidate sentences from the bibliography based on the converted text. The information retrieval objects include the frequency of the text and / or the corresponding pinyin, edit distance, Jaccard similarity, and cosine similarity.

11. The system for text extraction using speech according to claim 7, characterized in that, The book excerpt identification module is further configured to present candidate excerpts and a reordered list of candidate sentences to the user through human-computer interaction, allowing the user to confirm that the excerpt to be extracted is a book excerpt.

12. The system for text extraction using speech according to claim 7, characterized in that, The system also includes: The book excerpt management module is used to manage book excerpts. The book excerpt commentary module is used to provide book excerpts and commentary; and The book excerpt import / export module is used to import and export book excerpts.

13. A smart terminal for text extraction using voice, characterized in that, Smart terminals include: Input devices; Output devices; processor; A memory configured to store a set of computer-executable instructions and computer-accessible data associated with the set of computer-executable instructions, which, when executed by the processor, cause the processor to perform the method as described in any one of claims 1 to 6.

14. A non-transitory computer-readable storage medium, characterized in that, A non-transitory computer-readable storage medium stores a series of computer-executable instructions that, when executed by a computing device, cause the computing device to perform the method as described in any one of claims 1 to 6.

Images