Prompt text display method of prompter, head-mounted display equipment and medium

Abstract

The embodiment of the invention discloses a prompter prompt text display method, head-mounted display equipment and a medium. A specific embodiment of the method comprises the following steps: converting received speech recognition text information to obtain recognition pinyin data; performing conversion processing on the preset prompting text information to obtain prompting pinyin data and a prompting position mapping table; the following display steps are executed: determining starting position information and ending position information corresponding to the recognized pinyin data; determining a text corresponding to the voice recognition text information in the preset teleprompter text information as a teleprompter prompt text; and performing highlight display processing on the prompt text of the prompter. According to the embodiment, the problem of matching failure caused by Chinese homophone and near-tone character recognition errors can be effectively solved, then the situation that the follow-up progress is lost or skipping errors occur is reduced, the user experience is improved, network transmission overhead and processing delay can be reduced, and then the real-time performance of follow-up response is improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the technical field of computer technology, and particularly to teleprompter prompt text display method, head-mounted display device and medium. BACKGROUND

[0002] Automatic speech recognition (ASR) and text matching technology is the core technology in real-time teleprompter, subtitle generation and voice interaction system. The purpose is to accurately align and match the user's real-time speech content with the pre-set text content, so as to realize key functions such as text highlighting, progress tracking, intelligent prompting, etc. At present, when displaying the prompt text in the teleprompter, the commonly used method is: a hybrid strategy based on text matching, that is, by using direct matching, incremental matching and other methods, the ASR recognized text result is matched with the teleprompter original text word by word or word by word, and then the matched prompt text is displayed.

[0003] However, in practice, it is found that when the above method is used to display the prompt text in the teleprompter, the following technical problems often exist: The hybrid strategy based on text matching highly depends on the absolute accuracy of the automatic speech recognition result at the text level, and cannot effectively deal with the matching failure problem caused by the recognition error of Chinese homophonic characters and near-homophonic characters, thereby causing the loss or jump error of the reading progress, interrupting the user experience, and thus reducing the user experience; at the same time, in order to accurately recognize the text, the hybrid strategy needs to upload the speech data to the cloud, rely on large online language models for recognition and error correction, and thus generate a large network transmission overhead and processing delay, reducing the real-time response of the reading.

[0004] The above information disclosed in this BACKGROUND section is only for the purpose of enhancing the understanding of the background of the present inventive concepts, and therefore, it can contain information that is not prior art known to those of ordinary skill in the art. SUMMARY

[0005] The summary of the present disclosure is used to introduce the concepts in a brief manner, which will be described in detail in the following detailed description. The summary of the present disclosure is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to be used to limit the scope of the claimed technical solutions.

[0006] Some embodiments of the present disclosure propose a teleprompter prompt text display method, a head-mounted display device and a computer readable medium to solve one or more of the technical problems mentioned in the above BACKGROUND section.

[0007] In a first aspect, some embodiments of the present disclosure provide a teleprompter prompt text display method, which comprises: performing conversion processing on received speech recognition text information to obtain recognition pinyin data; performing conversion processing on preset teleprompter text information to obtain teleprompter pinyin data and a teleprompter position mapping table; selecting a target continuous word sequence from the recognition pinyin data, and performing the following display steps according to the target continuous word sequence: in response to determining that there is a continuous sequence in the teleprompter pinyin data that matches the target continuous word sequence, determining starting position information and ending position information corresponding to the recognition pinyin data according to the teleprompter position mapping table, the recognition pinyin data, and the teleprompter pinyin data; determining, based on the starting position information and the ending position information, text in the preset teleprompter text information that corresponds to the speech recognition text information as teleprompter prompt text; and performing highlight display processing on the teleprompter prompt text.

[0008] Optionally, the method further comprises: in response to determining that there is no continuous sequence in the teleprompter pinyin data that matches the target continuous word sequence, modifying the target continuous word sequence to obtain a modified target continuous word sequence; and in response to determining that the modified target continuous word sequence satisfies a preset quantity condition, taking the modified target continuous word sequence as the target continuous word sequence and performing the matching step again.

[0009] Optionally, the method further comprises: for a character in the speech recognition text information that satisfies a preset order condition, performing the following retrieval step: in response to determining that the character is Chinese, determining each character in the speech recognition text information that satisfies a preset selection condition as a character sequence, and performing the following conversion step according to the character sequence: determining whether there is a character that is a multi-syllable word in the character sequence; in response to determining that there is a character that is a multi-syllable word in the character sequence and the character sequence exists in a multi-syllable word dictionary, converting the character sequence into a capitalized pinyin form according to the multi-syllable word dictionary to obtain a pinyin conversion result corresponding to the character sequence; deleting the character sequence from the speech recognition text information to obtain updated speech recognition text information; and in response to determining that the updated speech recognition text information is empty, determining recognition pinyin data corresponding to the speech recognition text information according to the obtained pinyin conversion results.

[0010] Optionally, the method further comprises: in response to determining that there is no character that is a multi-syllable word in the character sequence, converting the character into a capitalized pinyin form to obtain a pinyin conversion result corresponding to the character; and deleting the character from the speech recognition text information to obtain updated speech recognition text information.

[0011] Optionally, the method further comprises: in response to determining that the character is English, determining each character in the speech recognition text information that meets a preset English conversion condition as a target character sequence; performing conversion processing on the target character sequence to obtain a pinyin conversion result corresponding to the target character sequence; and deleting the target character sequence from the speech recognition text information to obtain updated speech recognition text information.

[0012] Optionally, the method further comprises: determining position information of each character in the word suggestion pinyin data according to the word suggestion pinyin data to obtain each position information; and determining a word suggestion position mapping table corresponding to the word suggestion pinyin data according to the obtained each position information and a position corresponding to each character included in the preset word suggestion text information.

[0013] Optionally, the method further comprises: determining end position information corresponding to the recognition pinyin data according to the word suggestion position mapping table and the word suggestion pinyin data; and determining start position information corresponding to the recognition pinyin data according to the recognition pinyin data, a recognition position mapping table corresponding to the recognition pinyin data, the end position information, and the word suggestion pinyin data.

[0014] In a second aspect, some embodiments of the present disclosure provide a head-mounted display device, comprising: one or more processors; a storage device configured to store one or more programs; an optical engine and optical elements configured to image in front of a user's eyes; and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any of the implementations of the first aspect.

[0015] In a third aspect, some embodiments of the present disclosure provide a computer readable medium having stored thereon a computer program, wherein the program, when executed by a processor, implements the method described in any of the implementations of the first aspect.

[0016] The above various embodiments of the present disclosure have the following beneficial effects: through the teleprompter prompt text display method of some embodiments of the present disclosure, the problem of matching failure caused by Chinese homophonic character and near-homophonic character recognition error can be effectively addressed, thereby reducing the occurrence of reading progress loss or jumping error, improving the user experience, and reducing network transmission overhead and processing delay, thereby improving the real-time performance of the reading response. Specifically, the reason for failing to effectively address the problem of matching failure caused by Chinese homophonic character and near-homophonic character recognition error, thereby leading to reading progress loss or jumping error, causing the user experience to be interrupted, and thereby reducing the user experience, and generating a large network transmission overhead and processing delay, reducing the real-time performance of the reading response, is that the mixed strategy based on text matching highly depends on the absolute accuracy of the automatic speech recognition result at the text level, and cannot effectively address the problem of matching failure caused by Chinese homophonic character and near-homophonic character recognition error, thereby leading to reading progress loss or jumping error, causing the user experience to be interrupted, and thereby reducing the user experience; at the same time, in order to accurately recognize the text, the mixed strategy needs to upload the speech data to the cloud and rely on a large online language model for recognition and error correction, thereby generating a large network transmission overhead and processing delay, and reducing the real-time performance of the reading response. Based on this, the teleprompter prompt text display method of some embodiments of the present disclosure first processes the received speech recognition text information to obtain recognition pinyin data. Thus, the above speech recognition text information can be converted into recognition pinyin data. Then, the preset teleprompter text information is processed to obtain teleprompter pinyin data and a teleprompter position mapping table. Thus, the preset teleprompter text information can be converted into teleprompter pinyin data, and the corresponding relationship between the characters and words included in the teleprompter pinyin data can be obtained. Subsequently, a target continuous word sequence is selected from the recognition pinyin data, and according to the target continuous word sequence, the following display steps are performed: first, in response to determining that there is a continuous sequence in the teleprompter pinyin data that matches the target continuous word sequence, the starting position information and the ending position information corresponding to the recognition pinyin data are determined according to the teleprompter position mapping table, the recognition pinyin data, and the teleprompter pinyin data. Thus, the starting position information and the ending position information corresponding to the recognition pinyin data in the teleprompter pinyin data can be obtained. Then, based on the starting position information and the ending position information, the text in the teleprompter text information corresponding to the speech recognition text information is determined as the teleprompter prompt text. Thus, the teleprompter prompt text can be obtained. Finally, the teleprompter prompt text is highlighted and displayed. Thus, the text passage corresponding to the current text used by the user in the teleprompter prompt text can be highlighted and displayed.Also because it is not simply dependent on the cloud large language model for text correction, or only on the local for accurate text matching, but by converting the speech recognition text information and the preset word prompt text information into pinyin data uniformly, the matching dimension is changed from character shape to character sound, so that the matching failure problem caused by the difference in character shape of Chinese homophonic characters and near-sound characters can be effectively solved. Moreover, the constructed pinyin sequence matching mechanism does not depend on network connection and can complete matching locally, thereby reducing the cloud data transmission overhead and waiting delay generated for obtaining high-precision text recognition results, and improving the real-time performance of the follow-up response in the offline scene. Thus, the matching failure problem caused by the recognition error of Chinese homophonic characters and near-sound characters can be effectively solved, thereby reducing the follow-up progress loss or jump error, improving the user experience, and reducing the network transmission overhead and processing delay, thereby improving the real-time performance of the follow-up response. BRIEF DESCRIPTION OF DRAWINGS

[0017] The above and other features, advantages, and aspects of embodiments of the present disclosure will become more apparent by describing in detail some embodiments thereof with reference to the attached drawings. The same or similar components have the same or similar reference numbers throughout the drawings. It should be understood that the drawings are schematic and elements and components are not necessarily drawn to scale.

[0018] Figure 1 is an exemplary system architecture diagram of a word prompter prompt text display method according to some embodiments of the present disclosure; Figure 2 is a flowchart of a word prompter prompt text display method according to some embodiments of the present disclosure; Figure 3 is a flowchart of another word prompter prompt text display method according to some embodiments of the present disclosure; Figure 4 is a hardware structure schematic diagram of a head-mounted display device suitable for implementing some embodiments of the present disclosure; Figure 5 is a full flowchart of a word prompter prompt text display method according to some embodiments of the present disclosure. DETAILED DESCRIPTION

[0019] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms, and should not be interpreted as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes, and are not intended to limit the scope of protection of the present disclosure.

[0020] It should be noted that only the parts related to the present application are shown in the drawings for the purpose of description. The embodiments in the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0021] It should be noted that the concepts of "first", "second", etc. mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0022] It should be noted that the modification of "one" or "multiple" mentioned in the present disclosure is illustrative rather than limiting, and those skilled in the art should understand that unless the context clearly indicates otherwise, it should be understood as "one or more".

[0023] The names of the messages or information exchanged between the plurality of devices in the embodiments of the present disclosure are only for illustrative purposes, and are not used to limit the scope of the messages or information.

[0024] Figure 1 An exemplary system architecture 100 of an embodiment of a teleprompter text display method applied to a head-mounted display device that can be applied to the present disclosure is shown.

[0025] As shown in Figure 1 , the exemplary system architecture 100 can include a head-mounted display device 11 and a terminal device 12.

[0026] The head-mounted display device 11 can include one or two optical machines and optical elements 111. The optical machines and optical elements are used to display a virtual interface. In addition, the head-mounted display device 11 further includes a frame 112. In some embodiments, the sensors, processing units, memories and batteries of the head-mounted display device 11 can be placed inside the frame 112. In some optional implementations of some embodiments, one or more components of the sensors, processing units, memories and batteries can also be integrated into another independent accessory (not shown) and connected to the frame 112 through a data line 13. In some optional implementations of some embodiments, the head-mounted display device 11 can only have display function and part of the sensors, while the terminal device 12 provides data processing, data storage, power supply capability and other capabilities.

[0027] The touch component 121 included in the terminal device 12 can be a touch screen or a touchpad. The touch screen can be used to display content and control window dragging, and the touchpad can be used to control window dragging. In some embodiments, the head-mounted display device 11 and the terminal device 12 can be communicatively connected. The communication connection can be a wired connection or a Bluetooth connection.

[0028] It should be understood that Figure 1The number of head-mounted display devices and terminal devices in the system is merely illustrative. Any suitable number of head-mounted display devices and terminal devices can be provided according to implementation needs.

[0029] Further referring to Figure 2 Fig. 200 shows a flow 200 of some embodiments of a teleprompter prompt text display method according to the present disclosure. The teleprompter prompt text display method comprises the following steps: Step 201, transforming the received speech recognition text information to obtain recognition pinyin data.

[0030] In some embodiments, the execution subject (e.g., a head-mounted display device) of the teleprompter prompt text display method can transform the received speech recognition text information to obtain recognition pinyin data. The speech recognition text information can represent the text obtained by transforming and filtering the received speech. The recognition pinyin data can represent the pinyin data corresponding to the speech recognition text information. The characters in the speech recognition text information are arranged in order.

[0031] Optionally, before step 201, first, in response to receiving speech data of a target user, the execution subject can transform the speech data into text data by automatic speech recognition technology. The target user can represent a user wearing the head-mounted display device. The speech data can represent the audio of the target user's current speech. Then, the text data can be filtered by keyword accurate matching method and regular expression to obtain speech recognition text information to filter the emotional words and punctuation in the text data. The head-mounted display device can be an AR glasses.

[0032] In some optional implementations of some embodiments, the execution subject can transform the received speech recognition text information to obtain recognition pinyin data by the following steps: First, for the characters in the speech recognition text information that meet the preset order condition, the following retrieval steps are performed: First sub-step, in response to determining that the character is Chinese, determining each character in the speech recognition text information that meets the preset selection condition as a character sequence, and performing the following conversion steps according to the character sequence: The first execution step is to determine whether there are characters that are polyphonic in the above character sequence. Among them, the above preset selection condition can be that the character meets the above preset order condition and the first preset number of characters after the character that meets the above preset order condition, and the first character among the first preset number of characters is adjacent to the character that meets the above preset order condition. The above first preset number can be 3. The above first preset number of characters can be each character arranged in sequence. Among them, the above preset order condition can be that the order of the character is the first in the above speech recognition text information. In practice, the above execution entity can determine whether there are characters that are polyphonic in the above character sequence by looking up a polyphonic character dictionary.

[0033] The second execution step is to, in response to determining that there are characters that are polyphonic in the above character sequence and the above character sequence exists in the polyphonic character dictionary, convert the above character sequence into the capitalized pinyin form according to the above polyphonic character dictionary, and obtain the pinyin conversion result corresponding to the above character sequence. Among them, the pinyin conversion result of the above character sequence can represent the capitalized pinyin form corresponding to each character in the above character sequence. In practice, the above execution entity can convert the above character sequence into the capitalized pinyin form by looking up a polyphonic character dictionary, and obtain the pinyin conversion result corresponding to the above character sequence. For example, if the character sequence is "méng zhī chī chī", the corresponding pinyin conversion result can be "MENG ZHI CHI CHI".

[0034] The third execution step is to delete the above character sequence from the above speech recognition text information to obtain the updated speech recognition text information.

[0035] The fourth execution step is to, in response to determining that there are characters that are polyphonic in the above character sequence and the above character sequence does not exist in the above polyphonic character dictionary, delete the last character in the above character sequence to obtain the updated character sequence.

[0036] The fifth execution step is to, in response to determining that the above updated character sequence meets the preset change condition, use the updated character sequence as the character sequence and execute the above conversion step again. Among them, the above preset change condition can be that the number of characters included in the above updated character sequence is greater than or equal to the second preset value. The above second preset value can be 2.

[0037] The second step involves determining the corresponding Pinyin data for the speech recognition text information based on the obtained Pinyin conversion results, in response to the confirmation that the updated speech recognition text information is empty. In practice, the executing entity can concatenate the obtained Pinyin conversion results using spaces, or separate the Pinyin conversion results with spaces. For example, if the Pinyin conversion results are “MENG ZHI CHI CHI”, “BAO”, “BU”, “MAO”, and “SI”, then the recognized Pinyin data can be “MENG ZHI CHI CHI BAO BU MAO SI”.

[0038] Optionally, the above conversion steps further include: first, in response to determining that the updated character sequence does not meet the preset change conditions, converting the updated character sequence into uppercase Pinyin form to obtain the Pinyin conversion result corresponding to the updated character sequence. The Pinyin conversion result of the updated character sequence can represent the uppercase Pinyin form corresponding to the updated character sequence. In practice, the execution entity can use the pypinyin library included in the Python language to convert the updated character sequence into uppercase Pinyin form to obtain the Pinyin conversion result corresponding to the updated character sequence.

[0039] Then, the updated character sequence is deleted from the speech recognition text information to obtain the updated speech recognition text information.

[0040] Optionally, after step 201, firstly, in response to determining that there are no polyphonic characters in the above character sequence, the executing entity can convert the above characters into uppercase pinyin form to obtain the pinyin conversion result corresponding to the above characters. The pinyin conversion result of the above characters can represent the uppercase pinyin form corresponding to the above characters. In practice, the executing entity can use the pypinyin library included in the Python language to convert the above characters into uppercase pinyin form to obtain the pinyin conversion result corresponding to the above characters.

[0041] Then, the aforementioned characters are deleted from the speech recognition text information to obtain the updated speech recognition text information.

[0042] Optionally, after step 201, firstly, in response to determining that the characters are in English, the executing entity can determine each character in the speech recognition text information that satisfies a preset English conversion condition as the target character sequence. The preset English conversion condition can be that the characters satisfy the preset order condition, and that the characters following the characters satisfying the preset order condition are all in English.

[0043] Then, the target character sequence is converted to obtain the corresponding Pinyin conversion result. The Pinyin conversion result represents the lowercase Pinyin form of each character in the target character sequence. In practice, the execution entity can convert the target character sequence to lowercase Pinyin using string methods. For example, if the speech recognition text information is "Test sentence" and the character is "T", then the target character sequence is "Test", and the Pinyin conversion result is "test".

[0044] Finally, the target character sequence is deleted from the speech recognition text information to obtain the updated speech recognition text information.

[0045] Optionally, after step 201, in response to determining that the above character is neither English nor Chinese, the above character is determined as the pinyin conversion result corresponding to the above character, and the above character is deleted from the above speech recognition text information to obtain updated speech recognition text information.

[0046] Optionally, after step 201, in response to determining that the updated speech recognition text information is not empty, the above-mentioned execution entity may use the updated speech recognition text information as speech recognition text information, and for characters in the speech recognition text information that meet the preset order conditions, perform the above-mentioned retrieval steps again.

[0047] Step 202: Convert the preset prompting text information to obtain prompting pinyin data and prompting position mapping table.

[0048] In some embodiments, the aforementioned execution entity can convert the preset prompting text information to obtain prompting pinyin data and a prompting position mapping table. The preset prompting text information can represent the text obtained after filtering the text displayed in the aforementioned head-mounted display device. The prompting pinyin data can represent the pinyin data corresponding to the preset prompting text information. The prompting position mapping table can represent the mapping relationship between the position (index) of each character in the prompting pinyin data and the position (index) of each character in the preset prompting text information. It should be noted that the method of converting the preset prompting text information to obtain prompting pinyin data is the same as the method of converting the received speech recognition text information to obtain recognition pinyin data, and will not be repeated here; please refer to step 201.

[0049] In some optional implementations of certain embodiments, the aforementioned execution entity can obtain the prompting position mapping table through the following steps: The first step is to determine the position information of each character in the aforementioned Pinyin prompting data, thus obtaining the position information. The position information can represent the position (index) corresponding to the character included in the Pinyin prompting data. In practice, for each character in the Pinyin prompting data, the executing entity can determine the index of the corresponding character in the Pinyin prompting character array as the position information of that character, thus obtaining the position information. The Pinyin prompting character array can represent a one-dimensional array with each character in the Pinyin prompting data as its element value. For example, if the Pinyin prompting data is "wen zi zhuan huan", then the corresponding position information can be "0,1,2,3,4,5,…,16".

[0050] The second step involves determining the prompting position mapping table corresponding to the aforementioned prompting pinyin data based on the obtained position information and the position corresponding to each character in the aforementioned preset prompting text information. In practice, firstly, for each character in the aforementioned preset prompting text information, the executing entity can determine the index of the word corresponding to the character in the prompting pinyin array as the position corresponding to the character. Then, the position information corresponding to the position of each character in the aforementioned preset prompting text information is determined from the aforementioned position information. Finally, the position of each character in the aforementioned preset prompting text information and its corresponding position information are combined to form the prompting position mapping table. For example, if the prompting pinyin data is “wen zizhuan huan”, the corresponding position information can be “0,1,2,3,4,5,…,16”, then the prompting position mapping table can be “[0]->[0],[1]->[0],[2]->[0],[3]->[0],[4]->[1],[5]->[1],[6]->[1],…,

[16] ->[3]” (wen->0,zi->1,zhuan->2,huan->3).

[0051] Step 203: Select the target continuous word sequence from the recognized pinyin data, and perform the following display steps based on the target continuous word sequence: Step 2031: In response to determining that there is a continuous sequence in the prompting pinyin data that matches the target continuous word sequence, the start position information and end position information of the corresponding recognized pinyin data are determined according to the prompting position mapping table, the recognized pinyin data and the prompting pinyin data.

[0052] In some embodiments, in response to determining that a continuous sequence matching the target continuous word sequence exists in the aforementioned prompting pinyin data, the executing entity can determine the start and end position information corresponding to the aforementioned recognition pinyin data based on the aforementioned prompting position mapping table, the aforementioned recognition pinyin data, and the aforementioned prompting pinyin data. The target continuous word sequence can represent a target number of words selected sequentially forward from the last word of the aforementioned prompting pinyin data. Each word in the target number of words can represent a continuous character sequence separated by spaces. The target number can be 8. The start position information can represent the position (index) of the first word included in the target word sequence. The target word sequence can represent a continuous word sequence in the aforementioned preset prompting text information that matches the speech recognition text information corresponding to the aforementioned recognition pinyin data. The end position information can represent the position (index) of the last word included in the target word sequence. It should be noted that the continuous sequence matching the aforementioned target continuous word sequence can be the same as the aforementioned target continuous word sequence.

[0053] In some optional implementations of certain embodiments, in response to determining that there exists a continuous sequence in the aforementioned prompting pinyin data that matches the aforementioned target continuous word sequence, the aforementioned execution entity can determine the start position information and end position information corresponding to the aforementioned prompting pinyin data through the following steps based on the aforementioned prompting position mapping table, the aforementioned recognized pinyin data, and the aforementioned prompting pinyin data: The first step is to determine the end position information of the corresponding pinyin recognition data based on the above-mentioned prompting position mapping table and the above-mentioned prompting pinyin data. In practice, firstly, the above-mentioned execution entity can determine the index of each character in the above-mentioned continuous sequence included in the above-mentioned prompting pinyin character array as the position information of the above-mentioned character, and obtain the position information of each character in the above-mentioned continuous sequence. Then, through the above-mentioned prompting position mapping table, the position corresponding to the position information of the last character in the above-mentioned continuous sequence in the above-mentioned preset prompting text information is determined as the end position information of the above-mentioned pinyin recognition data. For example, if the continuous sequence is “pin yin shu ju”, the position information corresponding to “u” is 64, and by looking up the prompting position mapping table “…,

[63] ->

[14] ,

[64] ->

[14] …”, the end position information of the corresponding pinyin recognition data is obtained as 14.

[0054] The second step involves determining the starting position information corresponding to the aforementioned pinyin recognition data, based on the pinyin recognition data, the corresponding recognition position mapping table, the ending position information, and the prompting pinyin data. The recognition position mapping table represents the mapping relationship between the positions of characters in the pinyin recognition data and the positions of characters in the speech-recognized text information. In practice, firstly, the executing entity can use the recognition position mapping table corresponding to the pinyin recognition data to determine the position of the character in the speech-recognized text information corresponding to the position information of the last character in the continuous sequence, thus obtaining the target position value. Then, the difference between the ending position information and the target position value is determined as the starting position information. It should be noted that the method for determining the recognition position mapping table based on the pinyin recognition data is the same as the method for determining the prompting position mapping table based on the prompting pinyin data, and will not be repeated here; please refer to step 202.

[0055] Step 2032: Based on the start position information and end position information, determine the text corresponding to the speech recognition text information in the preset prompting text information as the prompting text.

[0056] In some embodiments, the execution entity can determine the text corresponding to the speech recognition text in the preset prompting text information as the teleprompter prompt text based on the start position information and end position information. The teleprompter prompt text can represent the text in the preset prompting text information that matches the speech recognition text information. In practice, firstly, the execution entity can use the start position information and end position information as indexes to find each word corresponding to the prompting pinyin data. Then, through the prompting pinyin mapping table and the obtained words, the teleprompter prompt text corresponding to the prompting pinyin data in the preset prompting text information is determined. The prompting pinyin mapping table can represent the mapping relationship between characters in the preset prompting text information and words in the prompting pinyin data. The recognition pinyin mapping table can represent the mapping relationship between characters in the speech recognition text information and words in the recognition pinyin data.

[0057] Step 2033: Highlight the prompt text in the teleprompter.

[0058] In some embodiments, the execution entity may highlight the teleprompter prompt text. In practice, the execution entity may highlight the teleprompter prompt text based on strong light (≥10000 lux) and yellow.

[0059] In addressing the technical problems mentioned above, and considering the specific application scenario—where stutterers rely on teleprompters to maintain fluency when reading speeches aloud in important situations—the following technical problem arises: users' abnormal speech rhythm (syllable repetition, involuntary pauses) due to physiological or psychological factors can cause recognition breaks or page errors in traditional speech-driven teleprompter systems based on strict timing matching. This leads to a mismatch between displayed prompts and the user's speech content, interrupting important presentations and reducing the integrity and timeliness of information delivery. Users then need to manually modify the teleprompter content, further reducing the system's usability and reliability and increasing processing time. To address the specific requirements of this application scenario—adapting to non-uniform speech rhythms and syllable and word repetition—we have decided to adopt the following solution: Optionally, after step 203, firstly, in response to determining that the modified target continuous word sequence does not meet the preset quantity condition, the following steps are performed based on the identified pinyin data and the prompted pinyin data: First, based on the aforementioned recognized pinyin data, a set of prompting sub-windows corresponding to the aforementioned prompting pinyin data is determined. The prompting sub-windows in this set can represent the continuous word sequence included in the aforementioned prompting pinyin data. In practice, firstly, the executing entity can determine the target length by summing the maximum index of the recognized pinyin array with 1. Then, the length of the prompting sub-window is determined by summing the target length with a first preset value. Here, the specific value of the first preset value is not limited. The first preset value can be +3 or -3. Next, starting from the first word included in the aforementioned prompting pinyin data, a prompting sub-window is determined with the aforementioned prompting sub-window length as its length. Then, in response to determining that the last word included in the aforementioned prompting sub-window is different from the last word included in the aforementioned prompting pinyin data, the aforementioned prompting sub-window is taken as the target sub-window, and the following sliding steps are performed: First, the target sub-window is slid one word to the right to obtain the prompting sub-window. Then, in response to the determination that the last word of the above-mentioned prompting sub-window is different from the last word of the above-mentioned prompting pinyin data, the above-mentioned prompting sub-window is used as the target sub-window, and the above sliding steps are performed again. Finally, the obtained prompting sub-windows are determined as the prompting sub-window set.

[0060] Secondly, a set of two-dimensional matching matrices is constructed corresponding to the aforementioned recognized pinyin data and the aforementioned prompting sub-window set. The aforementioned preset quantity condition indicates that the number of words included in the modified target continuous word sequence is greater than or equal to a third preset value. The third preset value can be 4. The row indices of the two-dimensional matching matrices in the aforementioned two-dimensional matching matrix set correspond one-to-one with the words included in the aforementioned recognized pinyin data. The column indices of the aforementioned two-dimensional matching matrices correspond one-to-one with each word included in the aforementioned prompting sub-window. The two-dimensional matching matrices in the aforementioned two-dimensional matching matrix set can be represented as matrices with the number of rows as the number of words included in the aforementioned recognized pinyin data and the number of columns as the number of words included in the aforementioned prompting sub-window. In practice, for each two-dimensional matching matrix in the aforementioned two-dimensional matching matrix set, the executing entity can construct a two-dimensional matching matrix corresponding to the aforementioned recognized pinyin data and the aforementioned prompting sub-window using a standard library in a programming language. The aforementioned standard library can be the NumPy library included in Python. It should be noted that the prompting sub-windows in the aforementioned prompting sub-window set and the two-dimensional matching matrices in the aforementioned two-dimensional matching matrix set can be in one-to-one correspondence.

[0061] Next, for each two-dimensional matching matrix in the above set of two-dimensional matching matrices, based on the above-recognized pinyin data and the corresponding word prompting sub-window of the above two-dimensional matching matrix, the following steps are performed: The first step is to determine the element value of each cell in the aforementioned two-dimensional matching matrix based on the words in the recognized pinyin data and the words in the corresponding prompting sub-window. In practice, for each cell in the aforementioned two-dimensional matching matrix, the executing entity can first use the row index of the cell as the target row index and the column index as the target column index. Then, by recognizing the pinyin array, the word in the recognized pinyin array with the target row index is determined as the target recognized word, and by using the prompting sub-window array, the word in the prompting sub-window array with the target column index is determined as the target prompting word. The recognized pinyin array can represent a one-dimensional array corresponding to the recognized pinyin data. Each word in the recognized pinyin array corresponds to an index. The prompting sub-window array can represent a one-dimensional array corresponding to the prompting sub-window. Each word in the prompting sub-window array corresponds to an index. Finally, in response to determining that the target recognized word and the target prompting word are the same, the element value in the cell is set to 0. In response to the determination that the target recognition word and the target prompt word are different, the element value in the cell is set to 1.

[0062] The second step is to determine the target matching matrix based on the obtained element values ​​and the aforementioned two-dimensional matching matrix. The target matching matrix represents the matrix obtained by filling each of the aforementioned element values ​​into its corresponding cell. In practice, the executing entity can fill each of the aforementioned element values ​​into its corresponding cell to obtain the target matching matrix.

[0063] Secondly, based on the obtained target matching matrices, a set of cumulative cost matrices is determined. In practice, firstly, for the target matching matrices in the above target matching matrices, the execution entity can perform the following steps: First, construct a matrix with the same dimension as the target matching matrix as the initial cumulative cost matrix. Next, determine each cell included in the initial cumulative cost matrix as a cumulative cell. Then, for each cumulative cell included in the initial cumulative cost matrix, perform the following steps: First, in response to determining that there are cells above, to the left, and to the upper left of the above cumulative cell, determine the smallest element value among the three adjacent cumulative cells above, to the left, and to the upper left of the above cumulative cell as the target element value. In response to determining that there is a cell above the above cumulative cell, but no cells to the left or to the upper left, determine the element value above the above cumulative cell as the target element value. In response to determining that there is a cell to the left of the above cumulative cell, but no cells above or to the upper left, determine the element value to the left of the above cumulative cell as the target element value. In response to determining that there are no cells to the left, above, or to the upper left of the above cumulative cell, determine 0 as the target element value. Then, the sum of the element values ​​corresponding to the cells with the same row and column indices as the cumulative cells in the target matching matrix and the target element values ​​is determined as the cumulative element values. Next, these cumulative element values ​​are filled into the cumulative cells. Then, the initial cumulative cost matrix after filling is determined as the cumulative cost matrix. The cumulative cost matrices in the aforementioned cumulative cost matrix set can represent the initial cumulative cost matrix after filling. Finally, the resulting cumulative cost matrices are determined as the cumulative cost matrix set. It should be noted that there can be a one-to-one correspondence between the target matching matrices in the aforementioned target matching matrices and the cumulative cost matrices in the aforementioned cumulative cost matrix set.

[0064] Next, based on the aforementioned set of cumulative cost matrices, a target prompting sub-window is determined. This target prompting sub-window is characterized by the prompting sub-window whose terminating cell in the cumulative cost matrix included in the aforementioned set of cumulative cost matrices has the smallest element value. The terminating cell is characterized by the cell in the aforementioned cumulative cost matrix where both the row index and column index are the largest. In practice, the execution entity can determine the cumulative cost matrix with the smallest terminating cell value by comparing the terminating cells of each cumulative cost matrix set included in the aforementioned set of cumulative cost matrices. Then, the prompting sub-window corresponding to the target cumulative cost matrix is ​​determined as the target prompting sub-window.

[0065] Next, based on the aforementioned target word-prompting sub-window, the alignment quantization result and average quantization result between the continuous word sequence included in the target word-prompting sub-window and the aforementioned recognized pinyin data are determined. The alignment quantization result can represent the element value corresponding to the terminating cell of the target cumulative cost matrix. The average quantization result can represent the ratio of the alignment quantization result to the number of words included in the aforementioned recognized pinyin data. In practice, firstly, the executing entity can determine the element value corresponding to the terminating cell of the target cumulative cost matrix as the alignment quantization result. Then, the number of words included in the aforementioned recognized pinyin data is determined as the first target number. Finally, the ratio of the alignment quantization result to the first target number is determined as the average quantization result.

[0066] Finally, in response to determining that the alignment quantization result satisfies the first preset threshold condition and the average quantization result satisfies the second preset threshold condition, the teleprompter prompt text corresponding to the target teleprompter sub-window is determined, and the teleprompter prompt text is highlighted. The first preset threshold condition can be that the alignment quantization result is less than or equal to a seventh preset value. The second preset threshold condition can be that the average quantization result is less than or equal to an eighth preset value. Here, the specific values ​​of the seventh and eighth preset values ​​are not limited. The seventh preset value can be 3. The eighth preset value can be 0.5. In practice, firstly, the executing entity can determine the text corresponding to the teleprompter pinyin data in the preset teleprompter text information as the teleprompter prompt text using the teleprompter pinyin mapping table and the words included in the target teleprompter sub-window. Then, the teleprompter prompt text is highlighted based on strong light (≥10000 lux) and yellow.

[0067] The above-described technical solution, as an inventive point of this disclosure, addresses technical problem two: "The speech-driven prompting system based on strict timing matching experiences recognition breaks or page errors, reducing the integrity and timeliness of information transmission, as well as the practicality and reliability of the auxiliary system, and increasing the time consumption." The reasons for these issues are as follows: Abnormal speech rhythm (syllable repetition, involuntary pauses) caused by physiological or psychological factors can lead to recognition breaks or page errors in traditional speech-driven prompting systems based on strict timing matching. This results in speaker distraction, disrupted speech rhythm, and interruptions of important speeches, reducing the effectiveness of information transmission. Users are required to manually modify the prompter content, further reducing the practicality and reliability of the auxiliary system and increasing the time consumption. Solving these factors can reduce the occurrence of recognition breaks or page errors in the prompting system, improve the effectiveness of information transmission, enhance the practicality and reliability of the auxiliary system, and reduce the time consumption. To achieve this effect, the teleprompter text display method disclosed herein treats the recognized pinyin data and the prompted pinyin data as two scalable temporal paths respectively. By constructing a target matching matrix, the shortest alignment path is determined, automatically compensating for temporal distortions such as syllable repetition and involuntary pauses during user pronunciation. This reduces instances of recognition breaks or incorrect page scrolling in the teleprompter system, improves information delivery effectiveness, enhances the practicality and reliability of the auxiliary system, and reduces processing time.

[0068] In addressing the technical problems mentioned above, and considering the application scenario—where maintenance engineers in high-risk industrial facilities such as chemical plants and nuclear power plants need to perform critical operations based on complex and lengthy emergency handling manuals displayed on head-mounted displays during sudden malfunctions—the following technical problem often arises: Significant environmental noise or the isolation provided by protective equipment can drastically reduce the recognition rate of traditional voice interaction, leading to difficulties in obtaining critical operation instructions, delays in step execution, or sequence errors. This exposes the repair process to a higher risk of misoperation, potentially escalating the fault and causing secondary safety accidents, thereby reducing the efficiency and reliability of emergency response and threatening personnel and facility safety. To address the following requirements for this application scenario: adaptability to extremely low signal-to-noise ratio voice input, adaptability to complex operation steps, and adaptability to non-standard technical terminology, we have decided to adopt the following solution: Optionally, after step 203, firstly, an initial dynamic programming matrix is ​​constructed, along with an array of recognized pinyin words corresponding to the aforementioned recognized pinyin data and an array of prompting pinyin words corresponding to the aforementioned prompting pinyin data. The initial dynamic programming matrix can represent a two-dimensional array. In practice, the executing entity can first construct a two-dimensional array with rows representing a fourth preset value and columns representing a fifth preset value. The fourth preset value can be the sum of the number of words included in the recognized pinyin data and 1. The fifth preset value can be the sum of the number of words included in the prompting pinyin data and 1. The rows of the two-dimensional array correspond to the recognized pinyin data. The columns of the two-dimensional array correspond to the prompting pinyin data. For example, if the number of words included in the recognized pinyin data is m, and the number of words included in the prompting pinyin data is n, then the dimension of the two-dimensional array is (m+1). (n+1). Then, set each element of the first row and first column of the above two-dimensional array to 0 to obtain the initial dynamic programming matrix. Finally, construct the corresponding recognition pinyin array and the corresponding prompting pinyin array using the split() method of Python strings. For example, if the recognition pinyin data is [TIANTING HAO WO MEN CHU QU BA], the corresponding recognition pinyin array is {A[0]->TIAN, A[1]->TING, ..., A[7]->BA}.

[0069] Then, based on the preset function, preset scoring constraint information, the aforementioned recognized pinyin array, and the aforementioned prompting pinyin array, the initial dynamic programming matrix is ​​filled to obtain the target dynamic programming matrix. The preset function can represent a function used to determine the element value of each cell in the initial dynamic programming matrix. The preset scoring constraint information can represent the score for matching and the score for non-matching in the preset function. The preset scoring constraint information can be "match: +2, non-match: -1", and the target dynamic programming matrix can represent the matrix obtained after filling the initial dynamic programming matrix. In practice, firstly, for each cell in the initial dynamic programming matrix that is not in the first row and not in the first column, the executing entity can input the matching result of the recognized pinyin data and the prompting pinyin data into the preset function to obtain the element value corresponding to the cell. Then, the element value can be filled into the cell to obtain the updated initial dynamic programming matrix as the target dynamic programming matrix. As an example, the preset function can be: .

[0070] in, This can characterize the first element in the above initial dynamic programming matrix. line, number The element values ​​of the column. This can characterize the first element in the above initial dynamic programming matrix. line, number The element values ​​of the column. This can be characterized by the index in the above-mentioned pinyin recognition array being The element value is the same as the index in the above-mentioned word-prompting pinyin array. A score of +2 is given for matching element values ​​or a score of -1 is given for non-matching them. It can characterize the first line, number The element values ​​of the column. It can represent a negative score. The negative score mentioned above can be -2. It can characterize the first line, number The element values ​​of the column.

[0071] Next, based on the aforementioned target dynamic programming matrix, the cell data within the matrix that satisfies a preset condition is determined. This preset condition can be that the element value corresponding to the cell in the target dynamic programming matrix is ​​the largest. The cell data can represent the cell with the largest element value in the target dynamic programming matrix. In practice, the executing entity can determine the cell with the largest corresponding element value in the target dynamic programming matrix as the cell data.

[0072] Then, based on the cell data and the target dynamic programming matrix, the optimal alignment path corresponding to the above-mentioned pinyin recognition data is determined. The optimal alignment path can represent a continuous path from the cell with a score of 0 to the cell data. In practice, the executing entity can use a path backtracking method based on state transition rules to trace back from the cell data to the target dynamic programming matrix. In response to tracing back to a cell with an element value of 0, each cell in the backtracking process is determined as the optimal alignment path corresponding to the above-mentioned pinyin recognition data. The backtracking can be tracing back from the cell data to the upper left, upper right, or lower left of the target dynamic programming matrix. For example, the optimal alignment path could be H[3][6]->H[4][7]->H[5][8]->H[6][9]->H[7]

[10] ->H[8]

[11] .

[0073] Secondly, based on the aforementioned optimal alignment path, the matching quantization result between the identified pinyin data and the continuous word sequence included in the corresponding titled pinyin data is determined. This matching quantization result characterizes the score of the matching degree between the continuous word sequence included in the titled pinyin data corresponding to the optimal alignment path and the identified pinyin data. In practice, firstly, the executing entity can determine the sum of the element values ​​of each cell included in the optimal alignment path as the target matching score. Then, the number of each cell included in the optimal alignment path is determined as the second target number. Finally, the ratio of the target matching score to the second target number is determined as the matching quantization result.

[0074] Subsequently, in response to determining that the above matching quantization result satisfies the third preset threshold condition, the start position information and end position information corresponding to the above-mentioned recognized pinyin data are determined according to the above-mentioned optimal alignment path. Among them, the above-mentioned third preset threshold condition can be characterized by the above-mentioned matching quantization result being greater than or equal to the sixth preset value. Here, the specific value of the above-mentioned sixth preset value is not limited. For example, the sixth preset value can be 2.5. In practice, firstly, the above-mentioned execution entity can determine the difference between the column index of the first cell included in the above-mentioned optimal alignment path and 1 as the start position information, and determine the difference between the column index of the last cell included in the above-mentioned optimal alignment path and 1 as the end position information. For example, if the word-prompting pinyin data is [JIN TIAN TIAN QI ZHEN HAO WO MEN CHU QU WAN BA], the optimal alignment path can be H[3][6]->H[4][7]->H[5][8]->H[6][9]->H[7]

[10] ->H[8]

[11] , then the start position information of the word-prompting pinyin data corresponding to the above-mentioned recognized pinyin data can be 5, and the end position information can be 10.

[0075] Finally, based on the aforementioned start and end position information, the teleprompter prompt text is determined and highlighted. It should be noted that the step of determining the teleprompter prompt text based on the start and end position information can be found in step 2032.

[0076] The above-described technical solution, as an inventive point of this disclosure, addresses technical problem three: "The low accuracy of voice interaction recognition, even leading to escalation of faults and secondary safety accidents, thereby reducing the efficiency and reliability of emergency response and threatening personnel and facility safety." The reasons for this low accuracy, escalation of faults, and secondary safety accidents, thus reducing the efficiency and reliability of emergency response and threatening personnel and facility safety, are as follows: Significant environmental noise or the isolation caused by protective equipment can drastically reduce the accuracy of traditional voice interaction recognition, leading to difficulties in obtaining critical operation instructions, delays in step execution, or sequence errors. This exposes the repair process to a higher risk of misoperation, potentially escalating faults and causing secondary safety accidents, thereby reducing the efficiency and reliability of emergency response and threatening personnel and facility safety. Solving these factors can improve the accuracy of voice interaction recognition, reduce the occurrence of escalation of faults and secondary safety accidents, thereby improving the efficiency and reliability of emergency response and protecting personnel and facility safety. To achieve this effect, the teleprompter prompt text display method disclosed herein first transforms word omissions, word insertions, or word substitutions in speech recognition into a quantifiable sequence of editing operations, obtaining a target dynamic programming matrix containing all alignment paths. Then, based on the aforementioned target dynamic programming matrix, the optimal alignment path is selected. Subsequently, in response to the quantization result of determining the optimal alignment path satisfying a third preset threshold condition, the teleprompter prompt text is determined based on the optimal alignment path and highlighted. This improves the accuracy of speech interaction recognition, reduces the likelihood of escalating faults and causing secondary safety incidents, thereby enhancing the efficiency and reliability of emergency response and protecting personnel and facility safety.

[0077] The above embodiments of this disclosure have the following beneficial effects: the teleprompter prompt text display method of some embodiments of this disclosure can effectively deal with the matching failure problem caused by the recognition error of Chinese homophones and near-homophones, thereby reducing the occurrence of reading progress loss or jump error, improving the user experience, and reducing network transmission overhead and processing delay, thereby improving the real-time performance of reading response. Specifically, the inability to effectively handle matching failures caused by errors in recognizing Chinese homophones and near-homophones leads to lost reading progress or incorrect jumps, interrupting the user experience and thus reducing user satisfaction. Furthermore, the significant network transmission overhead and processing latency reduce the real-time performance of the reading response. This is because the hybrid strategy based on text matching heavily relies on the absolute accuracy of the automatic speech recognition results at the text level, making it unable to effectively handle matching failures caused by errors in recognizing Chinese homophones and near-homophones, leading to lost reading progress or incorrect jumps, interrupting the user experience, and thus reducing user satisfaction. Simultaneously, to accurately recognize text, this hybrid strategy requires uploading speech data to the cloud, relying on large online language models for recognition and error correction, resulting in significant network transmission overhead and processing latency, reducing the real-time performance of the reading response. Based on this, some embodiments of the teleprompter prompt text display method of this disclosure first convert the received speech-recognized text information to obtain recognized pinyin data. Thus, the aforementioned speech-recognized text information can be converted into recognized pinyin data. Then, the preset prompting text information is converted to obtain prompting pinyin data and a prompting position mapping table. This allows the preset prompting text information to be converted into prompting pinyin data, and the correspondence between characters and words included in the prompting pinyin data can be obtained. Next, a target continuous word sequence is selected from the aforementioned recognized pinyin data. Based on the target continuous word sequence, the following display steps are performed: First, in response to determining that a continuous sequence matching the target continuous word sequence exists in the aforementioned prompting pinyin data, the start and end position information corresponding to the aforementioned recognized pinyin data is determined based on the prompting position mapping table, the aforementioned recognized pinyin data, and the aforementioned prompting pinyin data. This allows the start and end position information corresponding to the aforementioned recognized pinyin data in the aforementioned prompting pinyin data to be obtained. Then, based on the aforementioned start and end position information, the text corresponding to the aforementioned speech recognition text information in the aforementioned prompting text information is determined as the teleprompter prompt text. This allows the teleprompter prompt text to be obtained. Finally, the aforementioned teleprompter prompt text is highlighted. Therefore, the corresponding text paragraph currently being used by the user can be highlighted in the teleprompter prompt text.Because it doesn't simply rely on large-scale language models in the cloud for text correction or perform precise text matching only on the local machine, but instead converts both the speech recognition text information and the pre-set prompting text information into pinyin data, the matching dimension shifts from character shape to character sound. This effectively addresses matching failures caused by differences in the shapes of homophones and near-homophones in Chinese. Furthermore, the constructed pinyin sequence matching mechanism does not rely on a network connection and can complete matching locally. This reduces the cloud data transmission overhead and waiting latency in obtaining high-precision text recognition results, improving the real-time performance of offline reading responses. Therefore, it effectively addresses matching failures caused by errors in recognizing homophones and near-homophones in Chinese, reducing the occurrence of lost reading progress or incorrect jumps, improving the user experience, and reducing network transmission overhead and processing latency, thus enhancing the real-time performance of reading responses.

[0078] Further reference Figure 3 This illustrates a flow 300 of another embodiment of the teleprompter prompt text display method. The flow 300 of this teleprompter prompt text display method includes the following steps: Step 301: The received speech recognition text information is converted and processed to obtain the recognized pinyin data.

[0079] Step 302: Convert the preset prompting text information to obtain prompting pinyin data and prompting position mapping table.

[0080] Step 303: Select the target continuous word sequence from the recognized pinyin data, and perform the following display steps based on the target continuous word sequence: Step 3031: In response to determining that there is a continuous sequence in the prompting pinyin data that matches the target continuous word sequence, the start position information and end position information of the corresponding recognized pinyin data are determined according to the prompting position mapping table, the recognized pinyin data and the prompting pinyin data.

[0081] Step 3032: Based on the start position information and end position information, determine the text corresponding to the speech recognition text information in the preset prompting text information as the prompting text.

[0082] Step 3033: Highlight the prompt text from the teleprompter.

[0083] In some embodiments, the specific implementation of steps 301-3033 and the resulting technical effects can be found in [reference]. Figure 2 Steps 201-2033 in the corresponding embodiments will not be repeated here.

[0084] Step 3034: In response to determining that there is no continuous sequence in the prompting pinyin data that matches the target continuous word sequence, the target continuous word sequence is modified to obtain the modified target continuous word sequence.

[0085] In some embodiments, in response to determining that there is no continuous sequence in the aforementioned prompting pinyin data that matches the aforementioned target continuous word sequence, the executing entity may modify the aforementioned target continuous word sequence to obtain a modified target continuous word sequence. In practice, in response to determining that there is no continuous sequence in the aforementioned prompting pinyin data that matches the aforementioned target continuous word sequence, the executing entity may delete the first word included in the aforementioned target continuous word sequence to obtain a modified target continuous word sequence.

[0086] Step 3035: In response to determining that the modified target continuous word sequence meets the preset quantity condition, the modified target continuous word sequence is used as the target continuous word sequence, and the display step is executed again.

[0087] In some embodiments, in response to determining that the modified target continuous word sequence meets a preset quantity condition, the modified target continuous word sequence is used as the target continuous word sequence, and the above display step is performed again.

[0088] from Figure 3 It can be seen from this that, with Figure 2 Compared to the description of some corresponding embodiments, Figure 3 The flowchart 300 of the teleprompter prompt text display method in some corresponding embodiments reflects that by gradually reducing the number of words included in the target continuous word sequence and matching it with the above-mentioned teleprompter pinyin data, the computational complexity is effectively reduced, enabling the system to quickly lock the teleprompter prompt text within limited hardware computing power and time, thereby improving matching efficiency and the system's real-time response capability, and thus enhancing the user experience.

[0089] like Figure 4As shown, the head-mounted display device 400 includes a processing unit (CPU) 401, a memory (ROM) 402, an input unit 403, and an output unit 404, wherein the processing unit 401, memory 402, input unit 403, and output unit 404 are interconnected via a bus 405. Here, the methods according to some embodiments of this disclosure can be implemented as a computer program and stored in the memory 402. The processing unit 401 in the head-mounted display device 400 implements the virtual keyboard display function defined in some embodiments of this disclosure by calling the aforementioned computer program stored in the memory 402. In some implementations, the input unit 403 may include devices such as a camera, microphone, gyroscope, accelerometer, and magnetometer, and the output unit 404 may be devices that can display content, such as optomechanical and optical elements. The aforementioned optomechanical and optical elements may be microdisplays. Thus, when the processing unit 401 calls the aforementioned computer program to execute the split-screen display function, it can control the input unit 403 to acquire user gestures, voice, and other operation commands, and control the output unit 404 to display the content.

[0090] It should be noted that, in some embodiments of this disclosure, the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0091] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0092] The aforementioned computer-readable medium may be included in the aforementioned head-mounted display device; or it may exist independently and not assembled into the head-mounted display device. The aforementioned computer-readable medium carries one or more programs, which, when executed by the head-mounted display device, cause the head-mounted display device to: convert received speech-recognition text information to obtain recognized pinyin data; convert preset prompting text information to obtain prompting pinyin data and a prompting position mapping table; select a target continuous word sequence from the aforementioned recognized pinyin data, and perform the following display steps based on the target continuous word sequence: in response to determining that there exists a continuous sequence in the aforementioned prompting pinyin data that matches the target continuous word sequence, determine the start position information and end position information corresponding to the aforementioned recognized pinyin data based on the aforementioned prompting position mapping table, the aforementioned recognized pinyin data, and the aforementioned prompting pinyin data; based on the aforementioned start position information and the aforementioned end position information, determine the text in the aforementioned preset prompting text information corresponding to the aforementioned speech-recognition text information as the teleprompter prompt text; and highlight the aforementioned teleprompter prompt text.

[0093] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0094] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0095] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0096] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A teleprompter prompt text display method applied to a head-mounted display device, comprising: transforming received speech recognition text information to obtain recognition pinyin data; transforming preset teleprompter text information to obtain teleprompter pinyin data and a teleprompter position mapping table; selecting a target continuous word sequence from the recognition pinyin data, and performing the following display steps according to the target continuous word sequence: in response to determining that there is a continuous sequence in the teleprompter pinyin data that matches the target continuous word sequence, determining starting position information and ending position information corresponding to the recognition pinyin data according to the teleprompter position mapping table, the recognition pinyin data, and the teleprompter pinyin data; based on the starting position information and the ending position information, determining text in the preset teleprompter text information corresponding to the speech recognition text information as teleprompter prompt text; highlighting the teleprompter prompt text.

2. The method of claim 1, wherein, The method further comprises: in response to determining that there is no continuous sequence in the teleprompter pinyin data that matches the target continuous word sequence, modifying the target continuous word sequence to obtain a modified target continuous word sequence; in response to determining that the modified target continuous word sequence satisfies a preset quantity condition, taking the modified target continuous word sequence as the target continuous word sequence and performing the display steps again.

3. The method of claim 1, wherein, The transformation of the received speech recognition text information to obtain recognition pinyin data comprises: for characters in the speech recognition text information that satisfy a preset order condition, performing the following retrieval steps: in response to determining that the characters are Chinese, taking each character in the speech recognition text information that satisfies a preset selection condition as a character sequence, and performing the following conversion steps according to the character sequence: determining whether there is a character that is a multi-syllable word in the character sequence; in response to determining that there is a character that is a multi-syllable word in the character sequence and the character sequence exists in a multi-syllable word dictionary, converting the character sequence to a capitalized pinyin form according to the multi-syllable word dictionary to obtain a pinyin conversion result corresponding to the character sequence; deleting the character sequence from the speech recognition text information to obtain updated speech recognition text information; in response to determining that the updated speech recognition text information is empty, determining recognition pinyin data corresponding to the speech recognition text information according to the obtained pinyin conversion results.

4. The method of claim 3, wherein, The conversion steps further comprise: in response to determining that there is no character that is a multi-syllable word in the character sequence, converting the character to a capitalized pinyin form to obtain a pinyin conversion result corresponding to the character; deleting the character from the speech recognition text information to obtain updated speech recognition text information.

5. The method of claim 3, wherein, The retrieval steps further comprise: in response to determining that the characters are English, determining each character in the speech recognition text information that satisfies a preset English conversion condition as a target character sequence; performing conversion processing on the target character sequence to obtain a pinyin conversion result corresponding to the target character sequence; delete the target character sequence from the speech recognition text information to obtain updated speech recognition text information.

6. The method of claim 1, wherein, The worder position mapping table is obtained by the following steps: According to the worder pinyin data, determine the position information of each character in the worder pinyin data to obtain each position information; According to the obtained each position information and the position corresponding to each character included in the preset worder text information, determine the worder position mapping table corresponding to the worder pinyin data.

7. The method of claim 1, wherein, According to the worder position mapping table, the recognition pinyin data and the worder pinyin data, determine the starting position information and the ending position information corresponding to the recognition pinyin data, including: According to the worder position mapping table and the worder pinyin data, determine the ending position information corresponding to the recognition pinyin data; According to the recognition pinyin data, the recognition position mapping table corresponding to the recognition pinyin data, the ending position information and the worder pinyin data, determine the starting position information corresponding to the recognition pinyin data.

8. The method of claim 1, wherein, The method further comprises: According to the recognition pinyin data, determine a worder sub-window set corresponding to the worder pinyin data; Construct a two-dimensional matching matrix set corresponding to the recognition pinyin data and the worder sub-window set; For each two-dimensional matching matrix in the two-dimensional matching matrix set, according to the recognition pinyin data and the worder sub-window corresponding to the two-dimensional matching matrix, the following steps are performed: For each cell included in the two-dimensional matching matrix, according to the word included in the recognition pinyin data corresponding to the cell and the word included in the worder sub-window corresponding to the cell, determine the element value corresponding to the cell; According to the obtained each element value and the two-dimensional matching matrix, determine a target matching matrix; According to the obtained each target matching matrix, determine a cumulative cost matrix set; According to the cumulative cost matrix set, determine a target worder sub-window; Based on the target worder sub-window, determine the alignment quantization result and the average quantization result between the continuous word sequence included in the target worder sub-window and the recognition pinyin data; In response to determining that the alignment quantization result satisfies a first preset threshold condition and the average quantization result satisfies a second preset threshold condition, determine a worder prompt text corresponding to the target worder sub-window, and highlight display the worder prompt text.

9. A head-mounted display device comprising: one or more processors; storage for storing one or more programs; optics and optical elements for imaging in front of a user's eye; when the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of claims 1-8.

10. A computer readable medium having stored thereon a computer program, wherein, The computer program is executed by the processor to implement the method of any one of claims 1-8. The computer program is executed by the processor to implement the method of any one of claims 1-8.

Images