File processing method, device, and electronic equipment

Abstract

The embodiments of the present application disclose a file processing method, device, and electronic device. The file processing method includes: collecting a first audio, performing speech recognition on the first audio, and obtaining a first recognition result; wherein the first audio is associated with a first text; collecting a first image, performing text recognition on the first image, and obtaining a second recognition result; wherein the first image includes a presentation interface of the first text based on the launch of a first application; matching the first recognition result with the second recognition result to obtain a matching result; and identifying matching content in the first text based on the matching result. The embodiments of the present application facilitate users to determine the correspondence between the current speech and the corresponding text, thereby improving the user experience.

Description

Technical Field

[0001] The embodiments of the present application relate to file processing technology, and in particular to a file processing method and device, and electronic equipment. Background Art

[0002] During a speaker's PowerPoint presentation, real-time speech recognition using a speech recognition system can help users better organize content. However, speech recognition systems are generally designed for generalized scenarios, with high accuracy for transcribing common words. However, their accuracy decreases for specialized vocabulary or terminology that only appears in specific scenarios. Customizing a speech recognition system optimized for specific scenarios is costly, and manual switching between different speech recognition systems is required for PowerPoint presentations on different topics. Summary of the Invention

[0003] To solve existing technical problems, the embodiments of the present application provide a file processing method and device, and an electronic device.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] A file processing method, comprising:

[0006] Collecting a first audio, performing speech recognition on the first audio, and obtaining a first recognition result; wherein the first audio is associated with a first text; and

[0007] Capturing a first image, performing text recognition on the first image, and obtaining a second recognition result; wherein the first image includes a presentation interface of the first text based on a first application startup;

[0008] Matching the first recognition result with the second recognition result to obtain a matching result;

[0009] Matching content in the first text is identified based on the matching result.

[0010] As an implementation manner, performing text recognition on the first image to obtain a second recognition result includes:

[0011] Acquire a presentation interface of the first text in the first image, and identify a text block in the presentation interface;

[0012] Recognize the characters in the text block as the second recognition result.

[0013] As an implementation manner, matching the first recognition result with the second recognition result to obtain a matching result includes:

[0014] Obtaining first text content in the first recognition result, and performing text and / or semantic text matching on the first text content with the recognized text of the text block;

[0015] The text block recognition text with the highest text and / or semantic matching degree with the first text content is found as the matching result.

[0016] As an implementation manner, the identifying content in the second recognition result based on the matching result includes:

[0017] The display attributes of the area where the text block recognized characters having the highest text and / or semantic matching degree with the first text content are located are adjusted to distinguish them from other text block recognized characters.

[0018] As an implementation mode, the method further includes:

[0019] Identify keywords in the first text content, obtain domain-related words of the keywords based on a local knowledge graph library, and use the domain-related words as a hot word list;

[0020] The hot word model network is trained based on the hot word list, and the first text content is corrected based on the hot word model network.

[0021] As an implementation mode, the method further includes:

[0022] Identifying keywords in the first text content, expanding the knowledge graph based on the keywords, obtaining domain-related words of the keywords, and obtaining expanded data;

[0023] Vocabulary training is performed in a set optimization model based on the expanded data to correct the first text content.

[0024] As an implementation mode, the method further includes:

[0025] Based on the corrected first text content and the domain-related words of the keyword, text and / or semantic text matching is performed with the recognized text of the text block, and the recognized text of the text block with the highest text and / or semantic matching degree with the first text content is determined again;

[0026] If the re-determined text block recognition characters are different from the previously identified text block recognition characters, the identification of the previously identified text block is canceled.

[0027] As an implementation manner, collecting the first audio, performing speech recognition on the first audio, and obtaining a first recognition result includes:

[0028] Determine keywords in the first text, obtain domain-related words of the keywords based on a knowledge graph library, and use the domain-related words as a hot word list;

[0029] The hot word model network is trained based on the hot word table, and speech recognition is performed on the collected first audio based on the hot word model network to obtain a first recognition result.

[0030] As an implementation manner, collecting the first audio, performing speech recognition on the first audio, and obtaining a first recognition result includes:

[0031] Determine keywords in the second recognition result, obtain domain-related words of the keywords based on the knowledge graph library, and use the domain-related words as a hot word list;

[0032] The hot word model network is trained based on the hot word table, and speech recognition is performed on the collected first audio based on the hot word model network to obtain a first recognition result.

[0033] According to a second aspect of an embodiment of the present application, there is provided a file processing device, comprising:

[0034] A first recognition unit is configured to collect a first audio, perform speech recognition on the first audio, and obtain a first recognition result; wherein the first audio is associated with a first text; and

[0035] A second recognition unit is configured to capture a first image, perform text recognition on the first image, and obtain a second recognition result; wherein the first image contains a presentation interface of the first text based on the first application startup;

[0036] a matching unit, configured to match the first recognition result with the second recognition result to obtain a matching result;

[0037] An identification unit is used to identify matching content in the first text based on the matching result.

[0038] As an implementation manner, the second identification unit is further configured to:

[0039] Acquire a presentation interface of the first text in the first image, and identify a text block in the presentation interface;

[0040] Recognize the characters in the text block as the second recognition result.

[0041] As an implementation manner, the matching unit is further configured to:

[0042] Obtaining first text content in the first recognition result, and performing text and / or semantic text matching on the first text content with the recognized text of the text block;

[0043] The text block recognition text with the highest text and / or semantic matching degree with the first text content is found as the matching result.

[0044] As an implementation manner, the identification unit is used to:

[0045] The display attributes of the area where the text block recognized characters having the highest text and / or semantic matching degree with the first text content are located are adjusted to distinguish them from other text block recognized characters.

[0046] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising at least one processor and a memory for storing a computer program that can run on the processor, wherein when the computer program is executed by the processor, the steps of the file processing method can be executed.

[0047] The file processing method, device, and electronic device of the embodiment of the present application can perform voice recognition on the voice speaker, and based on the voice recognition result, determine whether the speech content of the voice speaker matches the evolving content displayed by the electronic device, and can mark the current evolving content, so that the speaker and the audience can determine the current speech position based on the mark. In this way, from the speaker's perspective, the current speech points and positions can be automatically marked during the speech process. From the audience's perspective, even if the speaker does not highlight the key points in the speech display interface, the speech content can be quickly tracked by automatically marking the speech content. The embodiment of the present application greatly improves the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.

[0049] Figure 1 A flowchart of a file processing method according to an embodiment of the present application;

[0050] Figure 2 A flowchart of a file processing method according to an embodiment of the present application;

[0051] Figure 3 A flowchart of a file processing method according to an embodiment of the present application;

[0052] Figure 4 A schematic diagram of the structure of a file processing device according to an embodiment of the present application;

[0053] Figure 5 This is a schematic diagram of the composition structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application. Unless there is a conflict, the embodiments in the present application and the features in the embodiments can be arbitrarily combined with each other. The steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions. In addition, although a logical order is shown in the flowchart, in some cases, the steps shown or described can be performed in an order different from that here.

[0055] The present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0056] The present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0057] Figure 1 This is a flowchart of the file processing method of the embodiment of the present application, such as Figure 1 As shown, the file processing method of the embodiment of the present application includes the following processing steps:

[0058] Step 101: collect first audio, perform speech recognition on the first audio, and obtain a first recognition result.

[0059] In an embodiment of the present application, the first audio is associated with the first text. The first audio refers to the speaker presenting his or her speech to the audience via a projector or display screen, that is, the corresponding voice speech is combined with the presented content. Typical application scenarios include speeches, technical exchange meetings, video conferences, etc. That is, in the application scenarios of the embodiment of the present application, there is corresponding text display content for the first audio.

[0060] The first audio here mainly refers to the speech of the speaker, such as the speech of the speaker in a speech or communication meeting, etc. After the first audio is collected, speech recognition is performed on the collected first audio to identify it as corresponding text content.

[0061] Step 102: Capture a first image, perform text recognition on the first image, and obtain a second recognition result.

[0062] In an embodiment of the present application, the first image includes a presentation interface of the first text launched based on the first application. The first image here can be an image directly captured from the projection screen of the speaker's speech content, such as by photographing the projection or display screen with a camera to obtain the first image, or it can be obtained by calling the file of the text content based on the speaker's speech to obtain the currently presented interface as the first image. For example, when the application scenario is a scenario in which the speaker gives a speech based on files such as PPT, the first image of the speaker's current display interface can be obtained through the electronic device used by the speaker to deliver the PPT; the first application at this time can be an application for PPT files such as Office. The file presented by the speaker can also be PDF, Word, etc. The embodiment of the present application supports multiple ways to obtain the first image.

[0063] In the embodiment of the present application, step 101 and step 102 can be performed in parallel, and there is no particular order between them. Alternatively, the first image may be acquired first, and then text recognition may be performed on the currently acquired first audio.

[0064] Step 103: Match the first recognition result with the second recognition result to obtain a matching result.

[0065] Here, matching the first recognition result with the second recognition result can be performed by segmenting the text information in the first recognition result to obtain the keywords or key sentences in the first recognition result. The keywords or key sentences here refer to words or sentences with certain meanings. For example, the keywords or key sentences in the first recognition result can be obtained by matching words through a corpus and performing semantic analysis on the sentences based on the matched words. The obtained keywords or key sentences are matched with the second recognition result to determine the matching keywords or key sentences and obtain a matching result. The keywords or key sentences here can be words in the first text whose word frequency exceeds a set threshold, or some words with the highest word frequency. The key sentence can be a sentence containing the keyword.

[0066] In an embodiment of the present application, as an implementation method, the keywords or key sentences in the second recognition result can also be identified and matched with the text content in the first recognition result, as long as the matching keywords or key sentences between the first recognition result and the second recognition result can be determined.

[0067] Step 104: Identify matching content in the first text based on the matching result.

[0068] In an embodiment of the present application, after the first recognition result is determined, the matching keywords or key sentences of the first recognition result and the second recognition result are determined, so as to determine the content in the second recognition result that matches the keywords or key sentences in the first recognition result, and determine the position of the matching content in the second recognition result in the first text, and mark the matching content in the first text based on the determined position to highlight the part, so as to indicate to the speaker or audience that the part is related to the current speech content, so that the speaker and the audience can determine the content at the position of the display interface corresponding to the speaker's current speech based on the content identification, so that the audience can follow the speaker's speech content closely.

[0069] Figure 2 This is a flowchart of the file processing method of the embodiment of the present application, such as Figure 2 As shown, the file processing method of the embodiment of the present application includes the following processing steps:

[0070] Step 201: collect first audio, perform speech recognition on the first audio, and obtain a first recognition result.

[0071] In an embodiment of the present application, the first audio is associated with the first text. The first audio refers to the speaker presenting his or her speech to the audience via a projector or display screen, that is, the corresponding voice speech is combined with the presented content. Typical application scenarios include speeches, technical exchange meetings, video conferences, etc. That is, in the application scenarios of the embodiment of the present application, there is corresponding text display content for the first audio.

[0072] The first audio here mainly refers to the speech of the speaker, such as the speech of the speaker in a speech or communication meeting, etc. After the first audio is collected, speech recognition is performed on the collected first audio to identify it as corresponding text content.

[0073] Step 202 , capture a first image, obtain a presentation interface of the first text in the first image, identify a text block in the presentation interface; and identify text in the text block as the second recognition result.

[0074] In an embodiment of the present application, the first image includes a presentation interface of the first text launched based on the first application. The first image here can be an image directly captured from the projection screen of the speaker's speech content, such as by photographing the projection or display screen with a camera to obtain the first image, or it can be obtained by calling the file of the text content based on the speaker's speech to obtain the currently presented interface as the first image. For example, when the application scenario is a scenario in which the speaker gives a speech based on files such as PPT, the first image of the speaker's current display interface can be obtained through the electronic device used by the speaker to deliver the PPT; the first application at this time can be an application for PPT files such as Office. The file presented by the speaker can also be PDF, Word, etc. The embodiment of the present application supports multiple ways to obtain the first image.

[0075] Here, the presentation interface corresponding to the current display interface of the first text can be obtained through the first application of the first text. That is, in an embodiment of the present application, it is necessary to associate the current speech content of the speaker with the display content in the first text so as to identify the associated content in the first text, so as to facilitate the speaker and the audience to know the correspondence between the current speech content and files such as PPT, so that the audience can follow the speaker's voice to determine the relevant key points or specific content of the current speech.

[0076] In the embodiment of the present application, step 101 and step 102 can be performed in parallel, and there is no particular order between them. Alternatively, the first image may be acquired first, and then text recognition may be performed on the currently acquired first audio.

[0077] Step 203: Match the first recognition result with the second recognition result to obtain a matching result.

[0078] Here, matching the first recognition result with the second recognition result can be performed by segmenting the text information in the first recognition result to obtain the keywords or key sentences in the first recognition result. The keywords or key sentences here refer to words or sentences with certain meanings. For example, the keywords or key sentences in the first recognition result can be obtained by matching words through a corpus and performing semantic analysis on the sentences based on the matched words. The obtained keywords or key sentences are matched with the second recognition result to determine the matching keywords or key sentences and obtain a matching result. The keywords or key sentences here can be words in the first text whose word frequency exceeds a set threshold, or some words with the highest word frequency. The key sentence can be a sentence containing the keyword.

[0079] In an embodiment of the present application, as an implementation method, the keywords or key sentences in the second recognition result can also be identified and matched with the text content in the first recognition result, as long as the matching keywords or key sentences between the first recognition result and the second recognition result can be determined.

[0080] Specifically, the first text content in the first recognition result is obtained, and the first text content is matched with the recognized text of the text block for text and / or semantic text matching respectively; the recognized text of the text block with the highest text and / or semantic matching degree with the first text content is found as the matching result. In the embodiment of the present application, not only the text content is matched, but also the semantic matching is included. For example, when the keyword "communication" is involved in the speaker's speech, it can be associated with "channel", "signal-to-noise ratio", "Shannon principle", etc. in the second recognition result to accurately determine that the speaker's speech content corresponds to the text content of the first text.

[0081] Step 204: Identify matching content in the first text based on the matching result.

[0082] In an embodiment of the present application, after the first recognition result is determined, the matching keywords or key sentences of the first recognition result and the second recognition result are determined, so as to determine the content in the second recognition result that matches the keywords or key sentences in the first recognition result, and determine the position of the matching content in the second recognition result in the first text, and mark the matching content in the first text based on the determined position to highlight the part, so as to indicate to the speaker or audience that the part is related to the current speech content, so that the speaker and the audience can determine the content at the position of the display interface corresponding to the speaker's current speech based on the content identification, so that the audience can follow the speaker's speech content closely.

[0083] In an embodiment of the present application, the display attributes of the area containing the text block recognized text that has the highest text and / or semantic match with the first text content can be adjusted to distinguish it from other text block recognized text. The display attributes include display parameters such as display brightness, display grayscale, and display color. By identifying the matching content as different from other text content, the relevant content of the first text corresponding to the speaker's current speech is displayed.

[0084] Figure 3 This is a flowchart of the file processing method of the embodiment of the present application, such as Figure 3 As shown, the file processing method of the embodiment of the present application includes the following processing steps:

[0085] Step 301: collect first audio, perform speech recognition on the first audio, and obtain a first recognition result.

[0086] In an embodiment of the present application, the first audio is associated with the first text. The first audio refers to the speaker presenting his or her speech to the audience via a projector or display screen, that is, the corresponding voice speech is combined with the presented content. Typical application scenarios include speeches, technical exchange meetings, video conferences, etc. That is, in the application scenarios of the embodiment of the present application, there is corresponding text display content for the first audio.

[0087] The first audio here mainly refers to the speech of the speaker, such as the speech of the speaker in a speech or communication meeting, etc. After the first audio is collected, speech recognition is performed on the collected first audio to identify it as corresponding text content.

[0088] Here, when the first audio is recognized, the keywords in the first text are determined, and the domain-related words of the keywords are obtained based on the knowledge graph library, and the domain-related words are used as a hot word list; the hot word model network is trained based on the hot word list, and the collected first audio is subjected to speech recognition based on the hot word model network to obtain a first recognition result. Here, the keywords in the first text are identified, and a related hot word list is constructed, thereby improving the accuracy of the keywords in the hot word list. The embodiment of the present application takes into account that the content in the first text is the speech content based on which the speaker is speaking, and its text content has a high accuracy. As long as the text blocks in the first text are accurately recognized, the corresponding text content is relatively accurate. Therefore, constructing a hot word list based on the content in the first text and constraining the recognition result of the first audio can improve its speech recognition accuracy.

[0089] As an implementation method, keywords in the second recognition result are determined, domain-related words for the keywords are obtained based on a knowledge graph library, and the domain-related words are used as a hot word list; a hot word model network is trained based on the hot word list, and speech recognition is performed on the collected first audio based on the hot word model network to obtain a first recognition result. The implementation principle of this implementation method is the same as the aforementioned implementation principle, namely, the audio recognition result is constrained by the recognition result of the first image, thereby improving the accuracy of speech recognition.

[0090] Step 302 , capture a first image, obtain a presentation interface of the first text in the first image, identify a text block in the presentation interface; and identify text in the text block as the second recognition result.

[0091] In an embodiment of the present application, the first image includes a presentation interface of the first text launched based on the first application. The first image here can be an image directly captured from the projection screen of the speaker's speech content, such as by photographing the projection or display screen with a camera to obtain the first image, or it can be obtained by calling the file of the text content based on the speaker's speech to obtain the currently presented interface as the first image. For example, when the application scenario is a scenario in which the speaker gives a speech based on files such as PPT, the first image of the speaker's current display interface can be obtained through the electronic device used by the speaker to deliver the PPT; the first application at this time can be an application for PPT files such as Office. The file presented by the speaker can also be PDF, Word, etc. The embodiment of the present application supports multiple ways to obtain the first image.

[0092] Here, the presentation interface corresponding to the current display interface of the first text can be obtained through the first application of the first text. That is, in an embodiment of the present application, it is necessary to associate the current speech content of the speaker with the display content in the first text so as to identify the associated content in the first text, so as to facilitate the speaker and the audience to know the correspondence between the current speech content and files such as PPT, so that the audience can follow the speaker's voice to determine the relevant key points or specific content of the current speech.

[0093] In the embodiment of the present application, step 101 and step 102 can be performed in parallel, and there is no particular order between them. Alternatively, the first image may be acquired first, and then text recognition may be performed on the currently acquired first audio.

[0094] Step 303: Match the first recognition result with the second recognition result to obtain a matching result.

[0095] Here, matching the first recognition result with the second recognition result can be performed by segmenting the text information in the first recognition result to obtain the keywords or key sentences in the first recognition result. The keywords or key sentences here refer to words or sentences with certain meanings. For example, the keywords or key sentences in the first recognition result can be obtained by matching words through a corpus and performing semantic analysis on the sentences based on the matched words. The obtained keywords or key sentences are matched with the second recognition result to determine the matching keywords or key sentences and obtain a matching result. The keywords or key sentences here can be words in the first text whose word frequency exceeds a set threshold, or some words with the highest word frequency. The key sentence can be a sentence containing the keyword.

[0096] In an embodiment of the present application, as an implementation method, the keywords or key sentences in the second recognition result can also be identified and matched with the text content in the first recognition result, as long as the matching keywords or key sentences between the first recognition result and the second recognition result can be determined.

[0097] Specifically, the first text content in the first recognition result is obtained, and the first text content is matched with the recognized text of the text block for text and / or semantic text matching respectively; the recognized text of the text block with the highest text and / or semantic matching degree with the first text content is found as the matching result. In the embodiment of the present application, not only the text content is matched, but also the semantic matching is included. For example, when the keyword "communication" is involved in the speaker's speech, it can be associated with "channel", "signal-to-noise ratio", "Shannon principle", etc. in the second recognition result to accurately determine that the speaker's speech content corresponds to the text content of the first text.

[0098] In an embodiment of the present application, after performing speech recognition on the first audio to obtain the first recognition result, it is also necessary to perform recognition correction on the first recognition result to improve the recognition accuracy of the recognition result. Specifically, as an implementation method, the keywords in the first text content are identified, the field-related words of the keywords are obtained based on the local knowledge graph library, and the field-related words are used as a hot word list; based on the hot word list, the hot word model network is trained, and the first text content is corrected based on the hot word model network. The hot word model network here contains a word library composed of local prior recognition words, or contains words in the first text, and of course can also contain words in the network dictionary, so as to perform recognition correction on the text content in the current recognition result to further improve the accuracy of the recognition result.

[0099] As an implementation method, keywords in the first text content are identified, the knowledge graph is expanded based on the keywords, and field-related words of the keywords are obtained to obtain expanded data; vocabulary training is performed in a set optimization model based on the expanded data, and the first text content is corrected.

[0100] Here, the knowledge graph is expanded based on the keyword, that is, based on the keyword, a network engine or corpus is searched for words related to the associated word. The optimization model includes the LM (Levenberg-Marquardt) algorithm model.

[0101] The recognition performance of the speech recognition system depends on the corpus coverage of the language model and the acoustic model, especially the language model. If the corpus coverage is incomplete, there will be recognition errors such as different characters with the same sound. The usual solution is to collect relevant field corpus, or manually think of some field keywords as seeds, then crawl the corpus, and then train the LM. However, the field corpus is generally scarce. For some corpus with strong confidentiality (such as banks, etc.), it is generally difficult to obtain relevant corpus. In view of this, in the embodiment of the present application, it is preferred to extract several keywords based on a small number of background recognition texts such as the first text in the current recognition field through a keyword extraction method. Then, using these keywords as roots, one or two levels of associated words are retrieved in the knowledge graph, and then all these keywords and their associated words are combined into a hot word list (hotword list). According to the hotword list, a hot word decoding network is constructed, and the online recognition system is enhanced in real time. At the same time, hotwords are used as seeds to crawl relevant corpus and train the LM model. In the embodiment of the present application, the online recognition system only has certain corpus, such as related words in related dictionaries or limited texts, which inevitably have incompleteness in specific fields. Therefore, it is necessary to analyze errors based on the recognized text, extract domain keywords from the text based on the keyword extraction method, input these keywords into the knowledge graph, find the domain-related words of these words, and output a hot word list, which may generally be more than 10 to 100; perform LM training based on the hot word list to correct the recognition results.

[0102] Step 304 : Adjust the display attributes of the area where the text block recognized characters having the highest text and / or semantic matching degree with the first text content are located, so as to distinguish them from other text block recognized characters.

[0103] In the embodiment of the present application, after the first text content in the first recognition result is corrected, it is necessary to match it again with the second recognition result based on the corrected content. That is, based on the corrected first text content and the domain-related words of the keyword, text and / or semantic text matching is performed with the recognized text of the text block, and the recognized text of the text block with the highest text and / or semantic matching degree with the first text content is determined again; if the re-determined recognized text block is different from the previously identified text block, the identification of the previously identified text block is canceled.

[0104] In an embodiment of the present application, the display attributes of the area containing the text block recognized text that has the highest text and / or semantic match with the first text content can be adjusted to distinguish it from other text block recognized text. The display attributes include display parameters such as display brightness, display grayscale, and display color. By identifying the matching content as different from other text content, the relevant content of the first text corresponding to the speaker's current speech is displayed.

[0105] Figure 4 This is a schematic diagram of the structure of the file processing device according to an embodiment of the present application. Figure 4 As shown, the file processing device of the embodiment of the present application includes:

[0106] A first recognition unit 40 is configured to collect a first audio, perform speech recognition on the first audio, and obtain a first recognition result; wherein the first audio is associated with a first text;

[0107] The second recognition unit 41 is configured to capture a first image, perform text recognition on the first image, and obtain a second recognition result; wherein the first image contains a presentation interface of the first text based on the first application startup;

[0108] a matching unit 42, configured to match the first recognition result with the second recognition result to obtain a matching result;

[0109] The identification unit 43 is configured to identify matching content in the first text based on the matching result.

[0110] In the embodiment of the present application, as an implementation mode, the second identification unit 41 is further configured to:

[0111] Acquire a presentation interface of the first text in the first image, and identify a text block in the presentation interface;

[0112] Recognize the characters in the text block as the second recognition result.

[0113] In the embodiment of the present application, as an implementation method, the matching unit 42 is further configured to:

[0114] Obtaining first text content in the first recognition result, and performing text and / or semantic text matching on the first text content with the recognized text of the text block;

[0115] The text block recognition text with the highest text and / or semantic matching degree with the first text content is found as the matching result.

[0116] In the embodiment of the present application, as an implementation method, the identification unit 43 is further configured to:

[0117] The display attributes of the area where the text block recognized characters having the highest text and / or semantic matching degree with the first text content are located are adjusted to distinguish them from other text block recognized characters.

[0118] In an embodiment of the present application, keywords in the first text content are identified, domain-related words of the keywords are obtained based on a local knowledge graph library, and the domain-related words are used as a hot word list; a hot word model network is trained based on the hot word list, and the first text content is modified based on the hot word model network. Alternatively, keywords in the first text content are identified, a knowledge graph is expanded based on the keywords, domain-related words of the keywords are obtained, and expanded data is obtained; vocabulary training is performed in a set optimization model based on the expanded data, and the first text content is modified.

[0119] In this way, the identification unit 43 is also used to: perform text and / or semantic text matching with the recognized text of the text block based on the corrected first text content and the field-related words of the keyword, and re-determine the text block recognized text with the highest text and / or semantic matching degree with the first text content; if the re-determined text block recognized text is different from the previously identified text block recognized text, cancel the identification of the previously identified text block.

[0120] In an embodiment of the present application, speech recognition is performed on the first audio to obtain a first recognition result, including: determining keywords in the first text, obtaining domain-related words of the keywords based on the knowledge graph library, and using the domain-related words as a hot word list; training a hot word model network based on the hot word list, and performing speech recognition on the collected first audio based on the hot word model network to obtain a first recognition result. Alternatively, as an implementation method, keywords in the second recognition result are determined, domain-related words of the keywords are obtained based on the knowledge graph library, and using the domain-related words as a hot word list; training a hot word model network based on the hot word list, and performing speech recognition on the collected first audio based on the hot word model network to obtain a first recognition result.

[0121] In the file processing device of the embodiment of the present application, the first recognition unit 40, the second recognition unit 41, the matching unit 42 and the identification unit 43 can all be implemented by the central processing unit (CPU), digital signal processor (DSP), microcontroller unit (MCU) or programmable gate array (FPGA) in the terminal in actual applications.

[0122] It should be noted that the file processing device of the embodiment of the present application solves the problem based on a principle similar to that of the aforementioned file processing method. Therefore, the implementation process and implementation principle of the file processing device can refer to the description of the implementation process and implementation principle of the aforementioned method, and the repeated parts will not be repeated.

[0123] The present application also discloses a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program is used to perform at least Figures 1 to 3 The computer-readable storage medium may be a memory.

[0124] An embodiment of the present application also provides an electronic device. Figure 5 This is a schematic diagram of the hardware structure of the electronic device according to the embodiment of the present application. Figure 5 As shown, the electronic device includes: a communication component 63 for data transmission, at least one processor 61 and a memory 62 for storing computer programs that can be run on the processor 61. The various components in the electronic device are coupled together through a bus system 64. It can be understood that the bus system 64 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 64 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Figure 5 Various buses are labeled as bus system 64.

[0125] Wherein, when the processor 61 executes the computer program, it at least performs Figures 1 to 5 Any of the steps of the method shown.

[0126] It is understood that the memory 62 can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a magnetic disk or a magnetic tape. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus random access memory (DRRAM).The memory 62 described in the embodiments of the present application is intended to include, but is not limited to, these and any other suitable types of memory.

[0127] The methods disclosed in the above embodiments of the present application can be applied to or implemented by processor 61. Processor 61 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in processor 61 or instructions in software form. The above processor 61 may be a general-purpose processor, a DSP, or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc. Processor 61 can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of the present application can be directly implemented as being executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium located in memory 62. Processor 61 reads the information in memory 62 and, in conjunction with its hardware, completes the steps of the above method.

[0128] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0129] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0130] In addition, all functional units in the embodiments of the present application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the above-mentioned integrated units can be implemented in the form of hardware or in the form of hardware plus software functional units.

[0131] Those skilled in the art will appreciate that all or part of the steps of the above-mentioned method embodiments may be implemented by hardware associated with program instructions, and the aforementioned program may be stored in a computer-readable storage medium. When the program is executed, the program executes the steps of the above-mentioned method embodiments. The aforementioned storage medium includes various media that can store program codes, such as mobile storage devices, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0132] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0133] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0134] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0135] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0136] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A file processing method, comprising: Collecting a first audio, performing speech recognition on the first audio, and obtaining a first recognition result; wherein the first audio is associated with a first text; and Capturing a first image, performing text recognition on the first image, and obtaining a second recognition result; wherein the first image includes a presentation interface of the first text based on a first application startup; Matching the first recognition result with the second recognition result to obtain a matching result; and identifying matching content in the first text based on the matching result; The first text content in the first recognition result is corrected, and the second recognition result is matched again based on the corrected first text content. If the matching content in the re-determined first text is different from the matching content in the previously identified first text, the matching content in the previously identified first text is unidentified.

2. The method according to claim 1, wherein performing text recognition on the first image to obtain a second recognition result comprises: Acquire a presentation interface of the first text in the first image, and identify a text block in the presentation interface; Recognize the characters in the text block as the second recognition result.

3. The method according to claim 2, wherein matching the first recognition result with the second recognition result to obtain a matching result comprises: Obtaining first text content in the first recognition result, and performing text and / or semantic text matching on the first text content with the recognized text of the text block; The text block recognition text with the highest text and / or semantic matching degree with the first text content is found as the matching result.

4. The method according to claim 3, wherein the step of identifying the identification content in the second recognition result based on the matching result comprises: The display attributes of the area where the text block recognized characters having the highest text and / or semantic matching degree with the first text content are located are adjusted to distinguish them from other text block recognized characters.

5. The method according to claim 3, further comprising: Identify keywords in the first text content, obtain domain-related words of the keywords based on a local knowledge graph library, and use the domain-related words as a hot word list; The hot word model network is trained based on the hot word list, and the first text content is corrected based on the hot word model network.

6. The method according to claim 3, further comprising: Identify keywords in the first text content, expand the knowledge graph itself based on the keywords, obtain domain-related words of the keywords, and obtain expanded data; Vocabulary training is performed in a set optimization model based on the expanded data to correct the first text content.

7. The method according to claim 5 or 6, further comprising: Based on the corrected first text content and the domain-related words of the keyword, text and / or semantic text matching is performed with the recognized text of the text block, and the recognized text of the text block with the highest text and / or semantic matching degree with the first text content is determined again; If the re-determined text block recognition characters are different from the previously identified text block recognition characters, the identification of the previously identified text block is canceled.

8. The method according to claim 1, wherein collecting the first audio, performing speech recognition on the first audio, and obtaining a first recognition result comprises: Determine keywords in the first text, obtain domain-related words of the keywords based on a knowledge graph library, and use the domain-related words as a hot word list; The hot word model network is trained based on the hot word table, and speech recognition is performed on the collected first audio based on the hot word model network to obtain a first recognition result.

9. The method according to claim 1, wherein collecting the first audio, performing speech recognition on the first audio, and obtaining a first recognition result comprises: Determine keywords in the second recognition result, obtain domain-related words of the keywords based on the knowledge graph library, and use the domain-related words as a hot word list; The hot word model network is trained based on the hot word table, and speech recognition is performed on the collected first audio based on the hot word model network to obtain a first recognition result.

10. A file processing device comprising: a first recognition unit, configured to collect a first audio, perform speech recognition on the first audio, and obtain a first recognition result; wherein the first audio is associated with a first text; A second recognition unit is configured to capture a first image, perform text recognition on the first image, and obtain a second recognition result; wherein the first image contains a presentation interface of the first text based on the first application startup; a matching unit, configured to match the first recognition result with the second recognition result to obtain a matching result; an identification unit, configured to identify matching content in the first text based on the matching result; The matching unit is also used to correct the first text content in the first recognition result, and match it with the second recognition result again based on the corrected first text content. If the matching content in the re-determined first text is different from the matching content in the previously identified first text, the matching content in the previously identified first text is canceled.

11. The apparatus according to claim 10, wherein the second identification unit is further configured to: Acquire a presentation interface of the first text in the first image, and identify a text block in the presentation interface; Recognize the characters in the text block as the second recognition result.

12. The device according to claim 11, wherein the matching unit is further configured to: Obtaining first text content in the first recognition result, and performing text and / or semantic text matching on the first text content with the recognized text of the text block; The text block recognition text with the highest text and / or semantic matching degree with the first text content is found as the matching result.

13. The apparatus according to claim 12, wherein the identification unit is further configured to: The display attributes of the area where the text block recognized characters having the highest text and / or semantic matching degree with the first text content are located are adjusted to distinguish them from other text block recognized characters.

14. An electronic device comprising at least one processor and a memory for storing a computer program capable of running on the processor, wherein when the computer program is executed by the processor, the steps of the file processing method according to any one of claims 1 to 9 can be performed.

Images