Speech recognition method, device and equipment based on poem hot words and storage medium

Abstract

The embodiment of the invention discloses a speech recognition method and device based on poem hot words, equipment and a storage medium. According to the technical scheme provided by the embodiment of the invention, the hot word graph is constructed according to the plurality of hot word segmented words obtained by performing word segmentation on the poem article, the plurality of voice fragments of the to-be-recognized voice are subjected to voice recognition based on the beam search mode to determine the candidate word paths of the voice fragments, and the candidate word paths are subjected to hot word matching according to the hot word graph. According to the method, a hot word graph is constructed through a plurality of hot word segmented words obtained by performing word segmentation on a poem article, and hot word enhancement processing is performed on candidate word paths of voice fragments determined by voice recognition, so that the recognition accuracy of the to-be-recognized voice on the hot words corresponding to the poem article is improved, and the recognition accuracy of the to-be-recognized voice on the corresponding hot words of the poem article is improved. And the speech recognition accuracy of the poems is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of speech recognition technology, and in particular to a speech recognition method, device, equipment and storage medium based on poetry and prose hot words. Background Art

[0002] With the development of speech recognition technology, speech recognition is increasingly widely used. When teachers are teaching, they often record the content of the class and perform speech recognition on the recorded content to determine the text corresponding to the content spoken by the teacher during the teaching process.

[0003] General speech recognition technology has a good effect on the recognition of commonly used words, but has a low recognition accuracy for specialized words in a certain field. For example, in the teaching of ancient Chinese literature and poetry, general speech recognition models have less speech data for ancient Chinese literature and poetry, and teachers have more diverse ways of expressing ancient Chinese literature and poetry in occasional teaching scenarios, so the recognition accuracy of poetry is low. Summary of the invention

[0004] The embodiments of the present application provide a speech recognition method, device, equipment and storage medium based on hot words in poetry to solve the technical problem of low recognition accuracy of poetry by speech recognition models in related technologies, and effectively improve the speech recognition accuracy of poetry.

[0005] In a first aspect, an embodiment of the present application provides a speech recognition method based on poetry hot words, comprising:

[0006] Obtain the speech and poetry articles to be recognized;

[0007] Segmenting the poem or article to obtain a plurality of hot word segmentations, and constructing a hot word graph according to the hot word segmentations;

[0008] Performing speech recognition on the multiple speech fragments of the speech to be recognized based on a beam search method to determine candidate word paths of the speech fragments;

[0009] Hot word matching is performed on the candidate word path according to the hot word graph, and a speech recognition result of the speech to be recognized is determined according to the hot word matching result.

[0010] In a second aspect, the embodiment of the present application provides a speech recognition device based on poetry hot words, including: an information acquisition module, a hot word map construction module, a path search module and a result determination module, wherein:

[0011] The information acquisition module is used to acquire the speech and poetry articles to be recognized;

[0012] The hot word graph construction module is used to segment the poem article to obtain multiple hot word segmentations, and construct a hot word graph according to the hot word segmentations;

[0013] The path search module is used to perform speech recognition on the multiple speech fragments of the speech to be recognized based on the beam search method, and determine the candidate word paths of the speech fragments;

[0014] The result determination module is used to perform hot word matching on the candidate word path according to the hot word graph, and determine the speech recognition result of the speech to be recognized according to the hot word matching result.

[0015] In a third aspect, an embodiment of the present application provides a speech recognition device based on poetry hot words, including: a memory and one or more processors;

[0016] The memory is used to store one or more programs;

[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the speech recognition method based on poetry hot words as described in the first aspect.

[0018] In a fourth aspect, an embodiment of the present application provides a storage medium storing computer executable instructions, which, when executed by a computer processor, are used to execute the speech recognition method based on poetry hot words as described in the first aspect.

[0019] The embodiment of the present application constructs a hot word graph according to multiple hot word segmentations obtained by segmenting poems and articles, and performs speech recognition on multiple speech segments of the speech to be recognized based on a beam search method to determine candidate word paths of the speech segments, performs hot word matching on the candidate word paths according to the hot word graph, and determines the speech recognition result of the speech to be recognized according to the hot word matching result, constructs a hot word graph by using multiple hot word segmentations obtained by segmenting poems and articles, performs hot word enhancement processing on the candidate word paths of the speech segments determined by speech recognition, thereby improving the recognition accuracy of the speech to be recognized for the hot words corresponding to the poems and articles, and improving the speech recognition accuracy of the poems. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a flow chart of a method for speech recognition based on poetry hot words provided in an embodiment of the present application;

[0021] Figure 2 This is a schematic diagram of a process for obtaining speech and poetry articles to be recognized provided by an embodiment of the present application;

[0022] Figure 3 is a flow chart of another method for speech recognition based on poetry hot words provided in an embodiment of the present application;

[0023] Figure 4 It is a schematic diagram of a hot word graph construction process provided by an embodiment of the present application;

[0024] Figure 5 It is a schematic diagram of a hot word graph provided in an embodiment of the present application;

[0025] Figure 6 This is a schematic diagram of path grouping display provided in an embodiment of the present application;

[0026] Figure 7 It is a structural schematic diagram of a speech recognition device based on poetry hot words provided in an embodiment of the present application;

[0027] Figure 8 It is a structural diagram of a speech recognition device based on poetry and prose hot words provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical scheme and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for the convenience of description, only the part related to the present application but not all the contents are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow chart describes each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of each operation can be rearranged. The above process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The above process can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc.

[0029] The speech recognition method based on poetry and prose hot words provided in this application can be applied to speech recognition of speech collected during the teaching process. It aims to construct a hot word graph through multiple hot word segmentations obtained by segmenting poetry and prose, and perform hot word enhancement processing on the candidate word path of speech segmentation determined by speech recognition, so as to improve the recognition accuracy of the recognized speech for the hot words corresponding to poetry and prose, and improve the speech recognition accuracy of poetry and prose.

[0030] In existing speech recognition solutions, speech recognition models are trained based on commonly used vocabulary. The speech recognition model has a good recognition effect on commonly used vocabulary, but has a low recognition accuracy for proprietary vocabulary in fields such as ancient Chinese literature and poetry. Hot word enhancement technology can improve the recognition accuracy of speech recognition in a certain field. When using hot word enhancement technology to assist speech recognition, in actual teaching scenarios, teachers have a variety of expression methods for explaining ancient Chinese literature and poetry. Pre-generated hot words are difficult to adapt to specific teaching content, and the speech recognition accuracy for poetry is low. Based on this, a speech recognition method based on poetry hot words in an embodiment of the present application is provided to solve the technical problem that the existing speech recognition solution has a low speech recognition accuracy for poetry.

[0031] Figure 1 A flow chart of a speech recognition method based on poetry hot words provided in an embodiment of the present application is given. The speech recognition method based on poetry hot words provided in an embodiment of the present application can be executed by a speech recognition device based on poetry hot words. The speech recognition device based on poetry hot words can be implemented in hardware and / or software and integrated in a speech recognition device based on poetry hot words.

[0032] The following description is made by taking the speech recognition device based on poetry hot words to perform the speech recognition method based on poetry hot words as an example. Figure 1 , the speech recognition method based on poetry hot words includes:

[0033] S110: Acquire the speech and poetry to be recognized.

[0034] Exemplarily, the speech to be recognized and the poetry article that need to be recognized are obtained, wherein the speech to be recognized and the poetry article correspond to the same course ID, the speech content in the speech to be recognized is related to the poetry article, and the speech content in the speech to be recognized includes part or all of the content in the poetry article. For example, the poetry article records the content of one or more ancient Chinese texts and poems, and the teacher will talk about part or all of the content of one or more ancient Chinese texts and poems during teaching. At this time, the speech content in the recorded speech to be recognized includes part or all of the content of one or more ancient Chinese texts and poems. Among them, the corresponding speech content and poetry article content are speech and text that express the same meaning.

[0035] Optionally, the speech to be recognized provided by the present solution may be obtained through speech recording, for example, by using a microphone to record the audio of a teacher teaching to obtain the speech to be recognized, or may be obtained through video recording, for example, by using a camera to record the teacher teaching to obtain a classroom recording video, and the audio data in the classroom recording video may be used as the speech to be recognized.

[0036] Optionally, the poems and articles provided by this solution can be uploaded by users (such as teachers), or can be obtained by identifying chapters of courseware information uploaded by users. Figure 2 As shown in the schematic diagram of a process for obtaining speech and poetry articles to be recognized, the speech recognition method based on poetry hot words provided in this scheme includes:

[0037] S111: Acquire the speech to be recognized and the courseware information corresponding to the speech to be recognized.

[0038] S112: Identify the chapters of the courseware information and determine the poems and articles corresponding to the courseware information.

[0039] For example, a complete class of ancient Chinese literature and poetry is recorded to obtain the corresponding recorded audio of the class, and the recorded audio is used as the voice to be recognized. The courseware information corresponding to the class is obtained, and the courseware information is the courseware used by the teacher when teaching, such as a PPT document, a Word document or other courseware document, and the courseware information is identified by chapter recognition technology to obtain the poetry and prose corresponding to the courseware information.

[0040] In one embodiment, when chapter recognition is performed on courseware information, the subject section and version information corresponding to the courseware information may be recognized, and the courseware information is input into a corresponding chapter recognizer according to the subject section and version information, and the chapter recognizer recognizes the poem or article corresponding to the courseware information. This solution improves the accuracy and efficiency of obtaining the poem or article by performing chapter recognition on the courseware information related to the speech to be recognized, and determines the poem or article corresponding to the courseware information, thereby improving the accuracy and efficiency of speech recognition.

[0041] S120: Segment the poem or article to obtain multiple hot word segmentations, and construct a hot word graph based on the hot word segmentations.

[0042] Exemplarily, after obtaining the poem or article, the poem or article is segmented, the text content in the poem or article is divided into a plurality of hot word segments, and a hot word graph is constructed based on the hot word analysis, wherein each hot word segment includes at least two hot word characters.

[0043] In one embodiment, the hot word graph provided by the present solution includes a plurality of hot word characters that have a phrase relationship (i.e., hot word characters that exist in the same hot word participle) connected by acceptable arcs starting from an initial node, and each acceptable arc corresponds to a set hot word score, and a fallback arc with a negative hot word score is added to the middle state of the acceptable arc. When the hot word match is successful, the hot word score corresponding to the acceptable arc will be added to the candidate word path. When the hot word match fails, the hot word score corresponding to the fallback arc will be removed from the candidate word path.

[0044] Optionally, the hot word graph is constructed based on the hot word segmentation, which can be based on the AC automaton (Aho-Corasickautomaton) composition method, or based on the finite state machine (FST, Finite-State Transducers) composition method. For example, the AC automaton composition method is used to construct the hot word graph based on the hot word segmentation, and the hot word graph is constructed quickly and accurately by the AC automaton composition method, which effectively improves the accuracy and efficiency of speech recognition.

[0045] S130: Performing speech recognition on multiple speech segments of the speech to be recognized based on a beam search method, and determining candidate word paths of the speech segments.

[0046] Exemplarily, multiple speech segments of the speech to be recognized are obtained, and speech recognition is performed on each speech segment based on a beam search method to determine the candidate word path corresponding to each speech segment. Among them, the beam search searches for candidate words at different moments of the speech segment in turn (the speech segment includes multiple speech segment frames, and one speech segment frame of the speech segment corresponds to one moment) based on a set first quantity (i.e., beam value), determines a first number of candidate words and the prediction probabilities corresponding to these candidate words, and at each moment, the paths corresponding to the first number of candidate words before the prediction probability are used as candidate word paths. For example, if the first quantity is set to 10, the beam search searches for candidate words at different moments of the speech segment, and when determining 10 candidate words and the corresponding prediction probabilities, at each moment, the paths corresponding to the candidate words before the prediction probability are used as candidate word paths at the corresponding moment, that is, the beam search at each moment will determine the 10 paths with the highest prediction probabilities of the candidate words as new candidate word paths.

[0047] S140: performing hot word matching on the candidate word path according to the hot word graph, and determining the speech recognition result of the speech to be recognized according to the hot word matching result.

[0048] Exemplarily, the hot word graph determined above is used to perform hot word matching on the candidate word path corresponding to the speech segment, and the segment speech recognition result corresponding to the speech segment is determined according to the hot word matching result. Optionally, after obtaining the segment speech recognition result corresponding to each speech segment, the segment speech recognition results corresponding to each speech segment can be spliced ​​into the speech recognition result of the speech to be recognized.

[0049] In one embodiment, when multiple candidate word paths are outputted by performing speech recognition on a speech segment at a time based on beam search, these candidate word paths are matched with hot words using a hot word graph to determine the hot word matching results corresponding to these candidate word paths at this time, and at the same time, these candidate word paths are scored according to the hot word matching results. When the speech recognition and hot word matching of the last moment of the speech segment are completed, the candidate word path with the highest path score is selected from the multiple candidate word paths finally determined as the segment speech recognition result corresponding to the speech segment.

[0050] In the above, a hot word graph is constructed according to multiple hot word segmentations obtained by segmenting poems and articles, and speech recognition is performed on multiple speech segments of the speech to be recognized based on a beam search method to determine the candidate word paths of the speech segments, hot word matching is performed on the candidate word paths according to the hot word graph, and the speech recognition result of the speech to be recognized is determined according to the hot word matching result, a hot word graph is constructed by segmenting poems and articles with multiple hot word segmentations, and hot word enhancement processing is performed on the candidate word paths of the speech segments determined by speech recognition, thereby improving the recognition accuracy of the speech to be recognized for the hot words corresponding to the poems and articles, and improving the speech recognition accuracy of the poems.

[0051] Based on the above embodiments, Figure 3 A flowchart of another method for speech recognition based on poetry hot words provided in an embodiment of the present application is given. The method for speech recognition based on poetry hot words is a specific embodiment of the above-mentioned method for speech recognition based on poetry hot words. Figure 3 , the speech recognition method based on poetry hot words includes:

[0052] S210: Acquire the speech and poetry to be recognized.

[0053] S220: Segmenting the poem or article according to the punctuation marks in the poem or article to obtain a plurality of first segmented words, removing the punctuation marks in the poem or article, and segmenting the poem or article after removing the punctuation marks according to a set number of words in the phrase to obtain a plurality of second segmented words.

[0054] S230: Determine the hot word segmentation of the poem or article according to the first segmentation and the second segmentation.

[0055] Exemplarily, after obtaining the poem or article corresponding to the speech to be recognized, the punctuation marks (such as ",",".",";", etc.) in the poem or article are segmented to obtain the first segmented words. Further, the punctuation marks in the poem or article are removed, and the poem or article without the punctuation marks is segmented according to the set number of words in the phrase to obtain multiple second segmented words. The set number of words in the phrase is set to one or more. When it is set to multiple set number of words in the phrase, the poem or article without the punctuation marks is segmented according to each set number of words in the phrase to obtain multiple second segmented words. The set number of words in the phrase is greater than 1. Optionally, the set number of words in the phrase can be set to 2 or 3.

[0056] After determining the first participle and the second participle, the hot word participle of the poem or article can be determined based on the first participle and the second participle. For example, the set of the first participle and the second participle is used as the hot word participle of the poem or article. This solution uses a multi-hot word segmentation scheme to perform word segmentation, which can effectively solve the problem that it is difficult to hit the hot word when teachers explain ancient texts and poems in various expressions during actual teaching, improve the probability of hitting the hot word, and improve the accuracy of speech recognition in the ancient text and poem scene.

[0057] Optionally, before determining the hot word segmentation of the poem or article based on the first segmentation and the second segmentation, the first segmentation and the second segmentation may be subjected to deduplication processing, and then the hot word segmentation of the poem or article is determined based on the deduplicated first segmentation and the second segmentation. This solution reduces overlapping hot word segmentations by deduplicating the first segmentation and the second segmentation, thereby improving the accuracy and efficiency of generating a hot word graph.

[0058] For example, the number of set phrase words is set to 2 and 3, and for the poem article "The more respectful, the more courtesy", the punctuation mark ", " is used as a separator to obtain the first participle corresponding to "the more respectful" and "the more courtesy". The poem article without punctuation marks is "The more respectful, the more courtesy", and the poem article without punctuation marks is segmented according to the set number of phrase words 2 and 3, respectively, to obtain the second participle corresponding to "the more respectful", "the more respectful", "the more courtesy", "the more courtesy", "the more respectful", "the more respectful" and "the more courtesy", and after removing the duplication of the first participle and the second participle, the hot word participles of "the more respectful", "the more courtesy", "the more respectful", "the more respectful" and "the more courtesy" are obtained. In one embodiment, the poem article can also be segmented based on the Jieba word segmentation method.

[0059] S240: Using AC automatic structure diagram method, construct a hot word graph according to the hot word segmentation.

[0060] Exemplarily, based on the hot word segmentation determined above, an AC automatic structure diagram is used to construct a hot word graph. Figure 4As shown in the schematic diagram of the construction process of the hot word graph provided, when the speech recognition method based on poem hot words provided by this solution uses the AC automaton to construct the graph and constructs the hot word graph according to the hot word segmentation, it includes:

[0061] S241: Split the hot word segmentation into multiple hot word characters.

[0062] S242: Generate acceptable arcs according to each hot word character and the set hot word score.

[0063] S243: Add a backtracking arc in the intermediate state of the acceptable arc based on the hot word score to obtain the hot word graph of the hot word segmentation.

[0064] Exemplarily, using the AC automaton to construct the graph, split the determined hot word segmentation into multiple hot word characters. Starting from the initial node, take each hot word character and the set hot word score as the output and connect the initial node and the hot word character, and two consecutive hot word characters that exist in the same hot word segmentation in the order of the characters. Generate acceptable arcs between the initial node and the hot word character, and between the hot word characters.

[0065] Furthermore, add a backtracking arc with a negative hot word score to the intermediate state of each acceptable arc to obtain the hot word graph of the hot word segmentation. The value corresponding to the negative hot word score is the negative value of the product of the number of acceptable arcs skipped and the hot word score. Among them, the backtracking arc is used to remove the increased score when the hot word matching fails. By using the AC automaton to construct the graph, this solution generates acceptable arcs according to each hot word character and the set hot word score, and adds a backtracking arc with a negative hot word score to the intermediate state of the acceptable arc, accurately and quickly obtaining the hot word graph of the hot word segmentation, effectively improving the generation accuracy and efficiency of the hot word graph. And by using the AC automaton to construct the graph and the segmentation scheme of multiple hot words, it can effectively solve the situation that it is not easy to hit the hot words under various expressions when the teacher explains ancient Chinese poems and ci-poems during actual teaching, improve the hot word hit probability, and improve the speech recognition accuracy in the scenario of ancient Chinese poems and ci-poems.

[0066] As Figure 5 As shown in the schematic diagram of a hot word graph provided, assuming that the hot word score is set to 8, use the AC automaton to construct the hot word graph for the hot word segmentations of "the expression becomes more respectful", "the etiquette becomes more sincere", "the expression becomes", "becomes more respectful", "the etiquette becomes", and "becomes more sincere", and obtain as Figure 5The hot word graph shown. Among them, node A is the initial node of the hot word graph, and the node corresponding to the hot word text "to" is the termination node of the hot word graph. Among them, starting from the termination node, the arcs that are connected in sequence and point to the next hot word text are acceptable arcs, and the corresponding hot word score is 8. The hot word score corresponding to the termination node pointing to the initial node is 0. The arcs corresponding to the hot word text that is wrongly connected to other hot word texts or the initial node (that is, the text order of the connected nodes does not conform to the text order in the poem or article) are backtracking arcs, and the corresponding negative hot word score is the negative value of the product of the number of acceptable arcs skipped and 8. For example, the negative hot word score corresponding to the backtracking arc of "color" pointing to the initial node is -8, and the negative hot word score corresponding to the backtracking arc of "ritual" pointing to "color" is -24.

[0067] S250: Perform voice endpoint detection on the speech to be recognized to obtain multiple speech segments of the speech to be recognized.

[0068] S260: Based on the beam search method, perform speech decoding on the speech segments to determine the first number of candidate words of the speech segments at multiple moments, and add the hot word texts that are homophonic with the second number of candidate words before the prediction probability in the hot word graph at the peak moment of the beam search as candidate words.

[0069] S270: Determine the candidate word paths of each speech segment at each moment based on the candidate words.

[0070] Exemplarily, perform voice endpoint detection (VAD, Voice Activity Detection) on the speech to be recognized to obtain multiple speech segments of the speech to be recognized. Among them, the voice endpoint detection can accurately locate multiple nodes of starting to speak and ending to speak from the speech to be recognized with noise, and divide the speech to be recognized into multiple speech segments according to the start node and the end node. By performing voice endpoint detection on the speech to be recognized to obtain multiple speech segments, the processing of speech data without speech content in the speech to be recognized is reduced, and while ensuring the speech recognition accuracy, the speech recognition efficiency is improved.

[0071] Furthermore, a beam search method based on Connectionist Temporal Classification (CTC) is used to perform speech decoding on these speech segments respectively, and a first number of candidate words at multiple moments are determined for the speech segments. For example, a speech recognition module is used to perform speech recognition processing on the speech segments to obtain speech recognition outputs (tokens) at multiple moments, and the beam search method is used to decode the speech recognition outputs, and a first number of candidate words are obtained at each moment. Among them, the search results corresponding to the beam search include the search results at the peak moment (CTC peak moment) and the search results at the non-peak moment (CTC non-peak moment). In one embodiment, it is possible to determine whether the current moment is a peak moment according to the content of the candidate word with the highest score (i.e., the topk 1 candidate word). For example, it is determined whether the topk 1 candidate word is a candidate word without content. When the topk 1 candidate word is not a candidate word without content (i.e., the topk 1 candidate word is a candidate word with content), it is determined that the current moment is a peak moment. When the topk 1 candidate word is not blank, it is determined that the current moment is a peak moment. Among them, the candidate word being blank represents that the current speech segment frame is blank (i.e., there is no speech), and the decoding output is a token without content.

[0072] For determining a first number of candidate words at the peak moment of the beam search, determine the top second number of candidate words in terms of prediction probability among the first number of candidate words, and add the hot word texts homophonic with the top second number of candidate words in the hot word map as candidate words, so as to achieve the expansion of candidate words at the peak moment, making the expanded candidate words greater than or equal to the first number. For example, among the top second number of candidate words in terms of prediction probability at the peak moment, one of the candidate words is "欲" (yu), which is homophonic with "愈" (yu) in the hot word map, both being "yu", then "愈" is added to the candidate words at that moment. Among them, the second number is less than the first number. For example, the first number can be 10, and the second number can be 3, that is, at the peak moment of the beam search, the hot word texts homophonic with the top 3 candidate words in terms of prediction probability in the hot word map are added as candidate words, making the candidate words greater than or equal to 10. At the non-peak moment, directly search for 10 candidate words with the highest prediction probabilities.

[0073] Furthermore, the candidate word paths of each voice segment at each moment are determined based on the candidate words determined above. Among them, for the candidate word paths corresponding to non-peak moments, the corresponding number of candidate word paths is the first number, and for the candidate word paths corresponding to peak moments, the corresponding number of candidate word paths is the sum of the first number and the number of candidate words added from the hot word graph. This scheme expands the search results of the beam search by adding candidate words from the hot word graph at the peak moment of the beam search, reduces the situation in which the traditional beam search searches for candidate words according to a fixed search quantity (i.e., beam value), the candidate words are more restricted, and it is not easy to hit the hot words. Adding candidate words from the hot word graph at the peak moment of the beam search can make the clustering more divergent, thereby improving the accuracy and decoding speed of speech recognition in the field of ancient Chinese and ancient poetry in the teaching scenario of ancient Chinese and ancient poetry.

[0074] S280: Hot word matching is performed on the candidate word paths of the voice segments at each time in sequence according to the hot word graph.

[0075] S290: Determine the path grouping of the voice fragments at each moment according to the hot word matching result, and determine the speech recognition result of the to-be-recognized speech according to the path grouping of each voice fragment at the last moment.

[0076] Exemplarily, for each voice segment, the candidate word paths of the voice segment at each moment are matched with hot words in turn according to the hot word graph. For example, when the candidate word paths of the voice segment at each moment are obtained, the hot word graph is used to match the candidate word paths at the corresponding moment, and the hot word matching result corresponding to the moment is obtained, and the path scores corresponding to each candidate word path after the hot word matching are determined.

[0077] The path grouping of the voice fragments at each moment is determined according to the hot word matching results, wherein the path grouping may include the path grouping obtained based on the beam search, and may also include the path grouping determined based on the leading score after the hot word matching.

[0078] In a possible embodiment, the speech recognition method based on poetry hot words provided by the present scheme, when determining the path grouping of speech fragments at each moment according to the hot word matching results, includes: determining a first path grouping based on a first number of candidate words obtained by beam search; determining a second path grouping for candidate words whose hot word matching results are already hitting the hot words; and determining a third path grouping for candidate words whose hot word matching results are starting to hit the hot words.

[0079] The path grouping provided by this solution includes a first path grouping determined based on a beam search and a second path grouping and a third path grouping determined based on the hot word matching results. Exemplarily, after hot word matching is performed on the candidate word paths of the voice segment at each moment in turn according to the hot word graph, the candidate word paths corresponding to the first number of candidate words obtained based on the beam search are saved as the first path grouping (including the first number of candidate word paths). Determine the candidate words that have hit the hot words, and save the candidate word paths corresponding to the candidate words that have hit the hot words as the second path grouping. Determine the candidate words that start to hit the hot words, and save the candidate word paths corresponding to the candidate words that start to hit the hot words as the third path grouping. Optionally, the first number of candidate word paths before the path score in the candidate word path corresponding to the candidate words that start to hit the hot words can be saved as the third path grouping (including the first number of candidate word paths).

[0080] like Figure 6 As shown in a schematic diagram of a path grouping display, assuming that after a hot word match is performed on the candidate word path of a voice segment at a certain moment, the candidate word paths corresponding to the 10 candidate words obtained based on the beam search are saved as the first path grouping S1, one of which is "we want lust", and its path score increased by 8 points after the hot word match. The candidate word paths corresponding to the candidate words that have hit the hot words are saved as the second path grouping S2, and the candidate word path corresponding to the second path grouping is "we want color healing", and its path score increased by 16 points after the hot word match. The candidate word paths with the top 10 path scores in the candidate word paths corresponding to the candidate words that start to hit the hot words are saved as the third path grouping S3, one of which is "we want four healing", and its path score increased by 8 points after the hot word match.

[0081] This solution expands the number of candidate word paths by determining the second path grouping and the third path grouping based on the hot word matching results, increases the diversity of candidate word paths, and improves the decoding speed and accuracy of speech recognition in the ancient Chinese literature and poetry teaching scenario.

[0082] Furthermore, the path grouping of each speech segment at the last moment (i.e., the last determined path grouping of the speech segment) is determined, and the candidate word path with the highest path score in the path grouping is determined as the speech recognition result of the speech to be recognized. This solution determines the path grouping of the speech segment according to the hot word matching results, expands the number of candidate word paths, improves the decoding speed by dynamically changing the search range, increases the diversity of candidate word paths, and improves the decoding speed and accuracy of speech recognition in the teaching scenario of ancient Chinese literature and poetry.

[0083] In the above, a hot word graph is constructed based on multiple hot word segmentations obtained by segmenting poems and articles, and multiple voice segments of the speech to be recognized are subjected to speech recognition based on a beam search method to determine the candidate word path of the speech segment, and the candidate word path is matched with hot words according to the hot word graph, and the speech recognition result of the speech to be recognized is determined according to the hot word matching result, and a hot word graph is constructed by multiple hot word segmentations obtained by segmenting poems and articles, and the candidate word path of the speech segment determined by speech recognition is subjected to hot word enhancement processing, so as to improve the recognition accuracy of the corresponding hot words of the poetry and articles for the speech to be recognized, and improve the speech recognition accuracy of the poetry and articles. At the same time, the use of the AC automatic structure diagram method and the multi-hot word segmentation scheme can effectively solve the situation that it is not easy to hit the hot words in the various expressions when the teacher explains the ancient texts and poems in actual teaching, improve the probability of hitting the hot words, and improve the accuracy of speech recognition in the ancient texts and poems. And by adding candidate words from the hot word graph at the peak moment of the beam search, expanding the search results of the beam search, and determining the number of expanded candidate word paths based on the hot word matching results, the diversity of the candidate word paths is increased, and the accuracy and speed of speech recognition in the field of ancient Chinese literature and poetry in the teaching scenario of ancient Chinese literature and poetry are effectively improved.

[0084] Figure 7 A schematic diagram of the structure of a speech recognition device based on poetry hot words provided in an embodiment of the present application is given. Figure 7 The speech recognition device based on poetry hot words includes an information acquisition module 71, a hot word map construction module 72, a path search module 73 and a result determination module 74.

[0085] Among them, the information acquisition module 71 is used to obtain the speech and poetry articles to be recognized; the hot word map construction module 72 is used to segment the poetry articles to obtain multiple hot word segmentations, and construct a hot word map based on the hot word segmentations; the path search module 73 is used to perform speech recognition on multiple speech segments of the speech to be recognized based on the beam search method, and determine the candidate word path of the speech segment; the result determination module 74 is used to perform hot word matching on the candidate word path according to the hot word map, and determine the speech recognition result of the speech to be recognized according to the hot word matching result.

[0086] In the above, a hot word graph is constructed according to multiple hot word segmentations obtained by segmenting poems and articles, and speech recognition is performed on multiple speech segments of the speech to be recognized based on a beam search method to determine the candidate word paths of the speech segments, hot word matching is performed on the candidate word paths according to the hot word graph, and the speech recognition result of the speech to be recognized is determined according to the hot word matching result, a hot word graph is constructed by segmenting poems and articles with multiple hot word segmentations, and hot word enhancement processing is performed on the candidate word paths of the speech segments determined by speech recognition, thereby improving the recognition accuracy of the speech to be recognized for the hot words corresponding to the poems and articles, and improving the speech recognition accuracy of the poems.

[0087] In a possible embodiment, when the information acquisition module 71 acquires the speech and poetry to be recognized, it includes:

[0088] Obtain the speech to be recognized and the courseware information corresponding to the speech to be recognized;

[0089] Identify the chapters of the courseware information and determine the poems and articles that correspond to the courseware information.

[0090] In a possible embodiment, when the hot word graph construction module 72 performs word segmentation on a poem or article to obtain a plurality of hot word segmentations, the hot word graph construction module 72 includes:

[0091] Segment the poem or article according to the punctuation marks in the poem or article to obtain multiple first segmentations;

[0092] Removing punctuation marks from the poem or article, and segmenting the poem or article after the punctuation marks are removed according to the set number of words in the phrase to obtain a plurality of second segmented words;

[0093] Determine the hot word participles of poetry or prose based on the first participle and the second participle.

[0094] In a possible embodiment, the hot word graph construction module 72 performs a deduplication process on the first segmentation and the second segmentation before determining the hot word segmentations of the poem or article based on the first segmentation and the second segmentation.

[0095] In a possible embodiment, when the hot word graph construction module 72 constructs the hot word graph according to the hot word segmentation, it includes:

[0096] The AC automatic structure diagram method is adopted to construct the hot word graph according to the hot word segmentation.

[0097] In a possible embodiment, when the hot word graph construction module 72 constructs the hot word graph according to the hot word segmentation by using the AC automatic graph construction method, it includes:

[0098] Split the hot word into multiple hot word characters;

[0099] Generate an acceptable arc based on each hot word text and the set hot word score;

[0100] Add fallback arcs in the middle state of acceptable arcs based on the hot word scores to obtain the hot word graph of hot word segmentation.

[0101] In a possible embodiment, the path search module 73 performs speech recognition on multiple speech segments to be recognized based on a beam search method to determine candidate word paths of the speech segments, including:

[0102] Performing voice endpoint detection on the voice to be recognized to obtain multiple voice fragments of the voice to be recognized;

[0103] Decoding the speech fragments based on the beam search method, determining a first number of candidate words for the speech fragments at multiple times, and adding the hot word characters in the hot word graph that are homophonic with the second number of candidate words before the predicted probability as candidate words at the peak time of the beam search;

[0104] Based on the candidate words, the candidate word path of each voice segment at each time is determined.

[0105] In a possible embodiment, when the result determination module 74 performs hot word matching on the candidate word path according to the hot word graph and determines the speech recognition result of the speech to be recognized according to the hot word matching result, it includes:

[0106] According to the hot word graph, hot word matching is performed on the candidate word paths of the voice segments at each moment in turn;

[0107] Determine the path grouping of voice segments at each time point based on the hot word matching results;

[0108] The speech recognition result of the speech to be recognized is determined according to the path grouping of each speech fragment at the last moment.

[0109] In a possible embodiment, when the result determination module 74 determines the path grouping of the voice fragments at each time according to the hot word matching result, it includes:

[0110] Determine a first path grouping based on a first number of candidate words obtained by beam searching;

[0111] Determine a second path grouping for candidate words whose hot word matching results are hit hot words;

[0112] The hot word matching result is used to determine the third path grouping of candidate words that start to hit the hot word.

[0113] It is worth noting that in the above-mentioned embodiment of the speech recognition device based on poetry and hot words, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present application.

[0114] The embodiment of the present application also provides a speech recognition device based on poetry hot words, and the speech recognition device based on poetry hot words can be integrated with the speech recognition device based on poetry hot words provided in the embodiment of the present application. Figure 8 Schematic diagram of a speech recognition device based on poetry hot words provided by an embodiment of the present application. Figure 8The speech recognition device based on poetry hot words includes: an input device 83, an output device 84, a memory 82 and one or more processors 81; the memory 82 is used to store one or more programs; when one or more programs are executed by one or more processors 81, the one or more processors 81 implement the speech recognition method based on poetry hot words as provided in the above embodiment. The input device 83, the output device 84, the memory 82 and the processor 81 can be connected via a bus or other means. Figure 8 The example of connecting through bus is taken in the following.

[0115] The memory 82, as a computing device readable storage medium, can be used to store software programs, computer executable programs and modules, such as the program instructions / modules corresponding to the speech recognition method based on poetry hot words provided in any embodiment of the present application (for example, the information acquisition module 71, the hot word map construction module 72, the path search module 73 and the result determination module 74 in the speech recognition device based on poetry hot words). The memory 82 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application required for at least one function; the storage data area may store data created according to the use of the device, etc. In addition, the memory 82 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 82 may further include a memory remotely arranged relative to the processor 81, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0116] The input device 83 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the device. The output device 84 may include a display device such as a display screen.

[0117] The processor 81 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 82, that is, realizes the above-mentioned speech recognition method based on poetry hot words.

[0118] The above-mentioned speech recognition device, equipment and computer based on poetry hot words can be used to execute the speech recognition method based on poetry hot words provided in any of the above-mentioned embodiments, and have corresponding functions and beneficial effects.

[0119] The embodiment of the present application also provides a storage medium storing computer executable instructions. When the computer executable instructions are executed by a computer processor, they are used to execute the speech recognition method based on poetry hot words as provided in the above embodiment. The speech recognition method based on poetry hot words includes: obtaining a speech to be recognized and a poetry article; segmenting the poetry article to obtain multiple hot word segmentations, and constructing a hot word graph based on the hot word segmentations; performing speech recognition on multiple speech segments of the speech to be recognized based on a beam search method, and determining candidate word paths of the speech segments; performing hot word matching on the candidate word paths according to the hot word graph, and determining the speech recognition result of the speech to be recognized according to the hot word matching result.

[0120] Storage medium - any of various types of memory devices or storage devices. The term "storage medium" is intended to include: installation media, such as CD-ROM, floppy disk or tape device; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory, such as flash memory, magnetic media (such as hard disk or optical storage); registers or other similar types of memory elements, etc. Storage media may also include other types of memory or combinations thereof. In addition, the storage medium may be located in the first computer system in which the program is executed, or may be located in a different second computer system, which is connected to the first computer system via a network (such as the Internet). The second computer system can provide program instructions to the first computer for execution. The term "storage medium" may include two or more storage media that may reside in different locations (for example, in different computer systems connected by a network). The storage medium may store program instructions (for example, embodied as a computer program) that can be executed by one or more processors.

[0121] Of course, the storage medium for storing computer executable instructions provided in the embodiment of the present application, whose computer executable instructions are not limited to the speech recognition method based on poetry hot words provided above, can also execute related operations in the speech recognition method based on poetry hot words provided in any embodiment of the present application.

[0122] The speech recognition device, equipment and storage medium based on poetry hot words provided in the above embodiments can execute the speech recognition method based on poetry hot words provided in any embodiment of the present application. For technical details not described in detail in the above embodiments, please refer to the speech recognition method based on poetry hot words provided in any embodiment of the present application.

[0123] The above are only preferred embodiments of the present application and the technical principles used. The present application is not limited to the specific embodiments provided herein, and various obvious changes, readjustments and substitutions that can be made by those skilled in the art will not deviate from the scope of protection of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the claims.

Claims

1. A speech recognition method based on poetry hot words, characterized in that: include: Obtain the speech and poetry articles to be recognized; Segmenting the poem or article to obtain a plurality of hot word segmentations, and constructing a hot word graph according to the hot word segmentations; Performing speech recognition on the multiple speech fragments of the speech to be recognized based on a beam search method to determine candidate word paths of the speech fragments; Hot word matching is performed on the candidate word path according to the hot word graph, and a speech recognition result of the speech to be recognized is determined according to the hot word matching result.

2. The method for speech recognition based on poetry hot words according to claim 1 is characterized in that: The step of obtaining the speech and poetry to be recognized includes: Acquire the speech to be recognized and the courseware information corresponding to the speech to be recognized; The courseware information is identified by chapter identification to determine the poem or article corresponding to the courseware information.

3. The method for speech recognition based on poetry hot words according to claim 1 is characterized in that: The segmentation of the poem article to obtain multiple hot word segmentations includes: Segmenting the poem article according to punctuation marks in the poem article to obtain a plurality of first segmentations; Removing punctuation marks from the poem or article, and segmenting the poem or article from which the punctuation marks have been removed according to the set number of words in the phrase to obtain a plurality of second segmented words; The hot word segmentations of the poem article are determined according to the first segmentations and the second segmentations.

4. The method for speech recognition based on poetry hot words according to claim 3 is characterized in that: Before determining the hot word segmentation of the poem article according to the first segmentation and the second segmentation, the method further includes: Deduplication processing is performed on the first participle and the second participle.

5. The method for speech recognition based on poetry hot words according to claim 1 is characterized in that: The step of constructing a hot word graph according to the hot word segmentation comprises: An AC automatic structure diagram method is adopted to construct a hot word graph according to the hot word segmentation.

6. The method for speech recognition based on poetry hot words according to claim 5 is characterized in that: Adopting the AC automatic structure diagram method, constructing a hot word graph according to the hot word segmentation, including: Splitting the hot word segmentation into multiple hot word characters; Generate an acceptable arc according to each of the hot word characters and the set hot word score; A fallback arc is added to the intermediate state of the acceptable arc based on the hot word score to obtain a hot word graph of the hot word segmentation.

7. The method for speech recognition based on poetry hot words according to claim 1 is characterized in that: The step of performing speech recognition on the plurality of speech segments of the speech to be recognized based on a beam search method to determine candidate word paths of the speech segments includes: Performing voice endpoint detection on the speech to be recognized to obtain multiple voice fragments of the speech to be recognized; Decoding the speech fragments based on a beam search method, determining a first number of candidate words for the speech fragments at multiple times, and adding hot word characters in the hot word graph that are homophonic with a second number of candidate words before the predicted probability as candidate words at a peak time of the beam search; Based on the candidate words, the candidate word paths of the voice fragments at each time are determined.

8. The method for speech recognition based on poetry hot words according to claim 7 is characterized in that: The step of performing hot word matching on the candidate word path according to the hot word graph, and determining the speech recognition result of the speech to be recognized according to the hot word matching result, includes: According to the hot word graph, hot word matching is performed on the candidate word paths of the voice segments at each time in sequence; Determine the path grouping of the voice fragments at each time according to the hot word matching results; The speech recognition result of the speech to be recognized is determined according to the path grouping of each of the speech fragments at the last moment.

9. The method for speech recognition based on poetry hot words according to claim 8 is characterized in that: Determining the path grouping of the voice fragments at each time according to the hot word matching results includes: Determine a first path grouping based on a first number of candidate words obtained by beam searching; Determine a second path grouping for candidate words whose hot word matching results are hit hot words; The hot word matching result is used to determine the third path grouping of candidate words that start to hit the hot word.

10. A speech recognition device based on poetry and prose hot words, characterized in that: It includes information acquisition module, hot word map construction module, path search module and result determination module, among which: The information acquisition module is used to acquire the speech and poetry articles to be recognized; The hot word graph construction module is used to segment the poem article to obtain multiple hot word segmentations, and construct a hot word graph according to the hot word segmentations; The path search module is used to perform speech recognition on the multiple speech fragments of the speech to be recognized based on the beam search method, and determine the candidate word paths of the speech fragments; The result determination module is used to perform hot word matching on the candidate word path according to the hot word graph, and determine the speech recognition result of the speech to be recognized according to the hot word matching result.

11. A speech recognition device based on poetry and prose hot words, characterized in that: include: memory and one or more processors; The memory is used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the speech recognition method based on poetry hot words as described in any one of claims 1-9.

12. A storage medium storing computer executable instructions, characterized in that: The computer executable instructions, when executed by a computer processor, are used to execute the speech recognition method based on poetry hot words as described in any one of claims 1-9.

Images