Teleprompter display method, device, equipment, computer readable medium and program product

Abstract

Embodiments of the present disclosure disclose a teleprompter display method, device, equipment, computer readable medium and program product. A specific embodiment of the method comprises: generating a speech recognition result based on collected user speech; matching the speech recognition result based on target prompt source information to obtain matching position information; generating scrolling information according to the matching position information; and adjusting teleprompter display content corresponding to the target prompt source information according to the scrolling information. The embodiment improves the teleprompter experience and teleprompter synchronization effect of the user when speaking.

Description

Technical Field

[0001] Embodiments of this disclosure relate to the field of computer technology, and more specifically to teleprompter display methods, apparatus, devices, computer-readable media, and program products. Background Technology

[0002] A prompting system is an auxiliary tool used to help users speak according to a pre-prepared script, reducing situations such as forgetting or skipping words. Currently, the common method for prompting is to manually or periodically scroll through the script during the speech.

[0003] However, when using the above methods, the following technical problems often exist: the manual method requires distraction and has a poor user experience; the timed scrolling method cannot scroll the text according to the actual progress of the speech and has poor synchronization effect.

[0004] The information disclosed in this background section is only intended to enhance the understanding of the background of the inventive concept, and therefore may contain information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The summary portion of this disclosure is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description portion. This summary portion is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0006] Some embodiments of this disclosure provide methods, apparatuses, devices, computer-readable media, and computer program products for prompting text to address one or more of the technical problems mentioned in the background section above.

[0007] In a first aspect, some embodiments of this disclosure provide a prompting display method, which includes: generating a speech recognition result based on collected user speech; matching the speech recognition result with target prompt source information to obtain matching position information; generating scrolling information based on the matching position information; and adjusting the prompting display content corresponding to the target prompt source information based on the scrolling information.

[0008] Optionally, the above-mentioned generation of speech recognition results based on the collected user speech includes: performing speech recognition on the collected user speech to obtain speech recognition information; and dynamically adjusting the recognized speech recognition information to obtain adjusted text information as the speech recognition result.

[0009] Optionally, the above-mentioned process of performing speech recognition on the collected user speech to obtain speech recognition information includes: generating noise level information of the user speech; adjusting the noise reduction window information in response to determining that the noise level information meets preset noise conditions to obtain updated noise reduction window information; performing noise reduction processing on the user speech according to the updated noise reduction window information to obtain denoised user speech; and performing speech recognition on the obtained denoised user speech to obtain speech recognition information.

[0010] Optionally, the above-mentioned dynamic adjustment of the recognized speech recognition information to obtain adjusted text information as the speech recognition result includes: performing dynamic correction processing on the recognized speech recognition information to obtain corrected speech recognition information; adding the obtained corrected speech recognition information to the history queue; correcting the corrected speech recognition information according to the history queue to obtain corrected speech recognition information; and generating adjusted text information according to the corrected speech recognition information.

[0011] Optionally, generating adjusted text information based on the corrected speech recognition information includes: determining the number of characters included in the corrected speech recognition information; in response to determining that the number of characters meets a preset character condition, performing sentence segmentation processing on the corrected speech recognition information to obtain segmented speech recognition information; and performing punctuation optimization processing on the segmented speech recognition information to obtain optimized speech recognition information as adjusted text information.

[0012] Optionally, the above-mentioned matching of the speech recognition result based on the target prompt source information to obtain matching position information includes: in response to determining that the speech recognition result meets the preset character change conditions, determining the speech recognition result as text information to be matched; performing text truncation processing on the text information to be matched to obtain each truncated text; for each obtained truncated text, performing the following steps: determining the current visible range text information of the target prompt source information; matching the truncated text based on the current visible range text information or the target prompt source information to obtain the matching position corresponding to the truncated text; generating matching position information according to each obtained matching position.

[0013] Optionally, determining the current visible range text information of the target prompt source information includes: determining whether there is a matching result in the cache corresponding to the truncated text; and determining the current visible range text information of the target prompt source information in response to determining that there is no matching result in the cache corresponding to the truncated text.

[0014] Optionally, the above-mentioned matching of the truncated text based on the currently visible text information or the target prompt source information to obtain the matching position corresponding to the truncated text includes: matching the truncated text based on the currently visible text information to obtain a matching result; determining the matching result as the matching position in response to determining that the matching result indicates a successful match; and matching the truncated text based on the target prompt source information in response to determining that the matching result indicates a failed match to obtain a matching result as the matching position.

[0015] Optionally, the above matching of the truncated text includes: performing an exact match on the truncated text to obtain a first matching result; in response to determining that the first matching result indicates a matching failure, determining the longest common substring corresponding to the truncated text in the currently visible text information or the target prompt source information; matching the longest common substring to obtain a second matching result; in response to determining that the second matching result indicates a matching failure, determining the tail string corresponding to the truncated text; matching the tail string to obtain a third matching result; in response to determining that the third matching result indicates a matching failure, extracting each phrase from the truncated text; matching each phrase to obtain a fourth matching result; in response to determining that the fourth matching result indicates a matching failure, performing a fuzzy match on the truncated text to obtain a fifth matching result; and in response to determining that the fifth matching result indicates a successful match, determining the fifth matching result as the matching position corresponding to the truncated text.

[0016] Optionally, the above-mentioned matching of the truncated text includes: in response to determining that the first matching result indicates a successful match, determining the first matching result as the matching position corresponding to the truncated text; in response to determining that the second matching result indicates a successful match, determining the second matching result as the matching position corresponding to the truncated text; in response to determining that the third matching result indicates a successful match, determining the third matching result as the matching position corresponding to the truncated text; and in response to determining that the fourth matching result indicates a successful match, determining the fourth matching result as the matching position corresponding to the truncated text.

[0017] Optionally, generating scrolling information based on the matching position information includes: determining the distance between the matching position information and the current display position; generating a scrolling step based on the distance, the time interval between the last adjustment of the prompt display content and the scrolling speed; and generating a scrolling position as scrolling information based on the current display position and the scrolling step.

[0018] Optionally, the method further includes: determining the user's speech rate corresponding to the user's voice; updating the scrolling speed according to the scrolling speed and a preset maximum scrolling speed in response to determining that the user's speech rate and the scrolling speed meet a first preset speed condition; updating the scrolling speed according to the scrolling speed and a preset minimum scrolling speed in response to determining that the user's speech rate and the scrolling speed meet a second preset speed condition; and updating the scrolling speed to a preset speed in response to determining that the pause duration of the user's voice meets a preset duration condition.

[0019] Optionally, the above method further includes: generating the next scroll position as predicted scroll information based on the average historical scroll speed, the average historical scroll acceleration, and the current display position; and determining the visible range text information corresponding to the predicted scroll information in the target prompt source information based on the predicted scroll information, wherein the visible range text information is used to prioritize matching the text corresponding to the next user voice.

[0020] Secondly, some embodiments of this disclosure provide a prompting display device, the device comprising: a first generation unit configured to generate a speech recognition result based on collected user speech; a matching unit configured to match the speech recognition result based on target prompt source information to obtain matching position information; a second generation unit configured to generate scrolling information based on the matching position information; and an adjustment unit configured to adjust the prompting display content corresponding to the target prompt source information based on the scrolling information.

[0021] Optionally, the first generation unit is further configured to: perform speech recognition on the collected user speech to obtain speech recognition information; and dynamically adjust the recognized speech recognition information to obtain adjusted text information as the speech recognition result.

[0022] Optionally, the first generation unit is further configured to: generate noise level information of the user's speech; adjust the noise reduction window information in response to determining that the noise level information meets preset noise conditions to obtain updated noise reduction window information; perform noise reduction processing on the user's speech according to the updated noise reduction window information to obtain noise-reduced user speech; and perform speech recognition on the obtained noise-reduced user speech to obtain speech recognition information.

[0023] Optionally, the first generation unit is further configured to: perform dynamic correction processing on the recognized speech recognition information to obtain corrected speech recognition information; add the obtained corrected speech recognition information to a history queue; correct the corrected speech recognition information according to the history queue to obtain corrected speech recognition information; and generate adjusted text information according to the corrected speech recognition information.

[0024] Optionally, the first generation unit is further configured to: determine the number of characters included in the corrected speech recognition information; in response to determining that the number of characters meets a preset character condition, perform sentence segmentation processing on the corrected speech recognition information to obtain segmented speech recognition information; and perform punctuation optimization processing on the segmented speech recognition information to obtain optimized speech recognition information as adjusted text information.

[0025] Optionally, the matching unit is further configured to: in response to determining that the speech recognition result satisfies a preset character change condition, determine the speech recognition result as text information to be matched; perform text truncation processing on the text information to be matched to obtain each truncated text; for each obtained truncated text, perform the following steps: determine the current visible range text information of the target prompt source information; match the truncated text based on the current visible range text information or the target prompt source information to obtain the matching position corresponding to the truncated text; generate matching position information according to each obtained matching position.

[0026] Optionally, the matching unit is further configured to: determine whether there is a matching result in the cache corresponding to the truncated text; and in response to determining that there is no matching result in the cache corresponding to the truncated text, determine the current visible range text information of the target prompt source information.

[0027] Optionally, the matching unit is further configured to: match the truncated text based on the currently visible text information to obtain a matching result; in response to determining that the matching result indicates a successful match, determine the matching result as the matching position; in response to determining that the matching result indicates a failed match, match the truncated text based on the target prompt source information to obtain a matching result as the matching position.

[0028] Optionally, the matching unit is further configured to: perform precise matching on the truncated text to obtain a first matching result; in response to determining that the first matching result indicates a matching failure, determine the longest common substring corresponding to the truncated text in the currently visible text information or the target prompt source information; match the longest common substring to obtain a second matching result; in response to determining that the second matching result indicates a matching failure, determine the tail string corresponding to the truncated text; match the tail string to obtain a third matching result; in response to determining that the third matching result indicates a matching failure, extract each phrase from the truncated text; match each phrase to obtain a fourth matching result; in response to determining that the fourth matching result indicates a matching failure, perform fuzzy matching on the truncated text to obtain a fifth matching result; in response to determining that the fifth matching result indicates a successful matching, determine the fifth matching result as the matching position corresponding to the truncated text.

[0029] Optionally, the matching unit is further configured to: in response to determining that the first matching result indicates a successful match, determine the first matching result as the matching position corresponding to the truncated text; in response to determining that the second matching result indicates a successful match, determine the second matching result as the matching position corresponding to the truncated text; in response to determining that the third matching result indicates a successful match, determine the third matching result as the matching position corresponding to the truncated text; and in response to determining that the fourth matching result indicates a successful match, determine the fourth matching result as the matching position corresponding to the truncated text.

[0030] Optionally, the second generation unit is further configured to: determine the distance between the aforementioned matching position information and the current display position; generate a scroll step size based on the aforementioned distance, the time interval between the last adjustment of the prompt display content and the scroll speed; and generate a scroll position as scroll information based on the current display position and the aforementioned scroll step size.

[0031] Optionally, the prompting display device further includes: a determining unit, a first updating unit, a second updating unit, and a third updating unit. The determining unit is configured to determine the user's speech rate corresponding to the user's voice; the first updating unit is configured to update the scrolling speed according to the scrolling speed and a preset maximum scrolling speed in response to determining that the user's speech rate and the scrolling speed meet a first preset speed condition; the second updating unit is configured to update the scrolling speed according to the scrolling speed and a preset minimum scrolling speed in response to determining that the user's speech rate and the scrolling speed meet a second preset speed condition; and the third updating unit is configured to update the scrolling speed to a preset speed in response to determining that the pause duration of the user's voice meets a preset duration condition.

[0032] Optionally, the prompting display device further includes a position generation unit and an information determination unit. The position generation unit is configured to generate a predicted scrolling position as predicted scrolling information based on the average historical scrolling speed, the average historical scrolling acceleration, and the current display position. The information determination unit is configured to determine, based on the predicted scrolling information, the visible range text information corresponding to the predicted scrolling information in the target prompt source information, wherein the visible range text information is used for priority matching of the text corresponding to the next user's speech.

[0033] Thirdly, some embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation of the first aspect above.

[0034] Fourthly, some embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method described in any of the implementations of the first aspect above.

[0035] Fifthly, some embodiments of this disclosure provide a computer program product, including a computer program that, when executed by a processor, implements the method described in any of the implementations of the first aspect above.

[0036] The above-described embodiments of this disclosure have the following beneficial effects: the prompting display method of some embodiments of this disclosure improves the prompting experience and synchronization effect for users speaking. Specifically, the reasons for poor user experience and synchronization effect are: manual operation requires distraction, resulting in a poor user experience; and timed scrolling cannot scroll the text according to the actual progress of speaking, resulting in poor synchronization effect. Based on this, the prompting display method of some embodiments of this disclosure first generates a speech recognition result based on the collected user speech. This allows for the recognition of the user's speech during the speaking process. Next, based on the target prompt source information, the speech recognition result is matched to obtain matching position information. This allows for the location of the matching position of the adjusted text information in the target prompt source information; the matching position can be understood as the position corresponding to the current speech. Then, scrolling information is generated based on the matching position information. This allows for the determination of the scrolling-related parameters required during the display process based on the matching position. Finally, the prompting display content corresponding to the target prompt source information is adjusted based on the scrolling information. Therefore, the prompting content can follow the user's speaking progress during the speaking process without the need for manual operation, thereby improving the user's prompting experience and the prompting synchronization effect. Attached Figure Description

[0037] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0038] Figure 1 This is a flowchart of some embodiments of the teleprompter display method according to the present disclosure;

[0039] Figure 2 This is a schematic diagram illustrating an application scenario of a prompting display method according to some embodiments of the present disclosure;

[0040] Figure 3 These are schematic diagrams illustrating the structure of some embodiments of the teleprompter display device according to the present disclosure;

[0041] Figure 4 This is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. Detailed Implementation

[0042] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0043] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0044] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0045] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0046] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0047] Before performing any of the operations involving the collection, storage, or use of user personal information (such as user voice, voice recognition information, and target prompt source information) disclosed in this disclosure, the relevant organizations or individuals shall fulfill their obligations, including conducting personal information security impact assessments, informing personal information subjects, and obtaining prior authorization and consent from personal information subjects.

[0048] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0049] Figure 1 A flow 100 of some embodiments of a teleprompter display method according to the present disclosure is shown. The teleprompter display method includes the following steps:

[0050] Step 101: Generate speech recognition results based on the collected user speech.

[0051] In some embodiments, the entity executing the prompting method (e.g., a head-mounted display device or a terminal device) can generate speech recognition results based on the collected user speech. The head-mounted display device can be a display device used to image in front of the user's eyes. For example, the head-mounted display device can be, but is not limited to, AR glasses, MR glasses, or VR glasses. Preferably, the AR glasses can be binocular diffractive waveguide AR glasses. The head-mounted display device can be equipped with an audio acquisition device. The audio acquisition device can include a microphone. The terminal device can be, but is not limited to, a mobile phone, a tablet computer, a computer, or a display device for prompting. The terminal device can be equipped with an audio acquisition device. For example, a user can wear a head-mounted display device during a speech, and the user's speech can be collected by the audio acquisition device on the head-mounted display device. Users can also use a mobile phone or tablet computer during live streaming, and the user's speech can be collected by the audio acquisition device on the mobile phone or tablet computer. The audio acquisition device can be pre-set with acquisition parameter information. The acquisition parameter information can include, but is not limited to, at least one of the following: sampling rate, audio channels, audio format, and buffer size. For example, the acquisition parameters could be: sampling rate: 16000Hz; audio channel: mono; audio format: PCM 16-bit; buffer size: 160ms. In practice, the aforementioned execution entity can perform speech recognition on the acquired user speech and obtain the speech recognition information as the speech recognition result.

[0052] In some optional implementations of certain embodiments, the aforementioned execution entity can perform denoising processing on the acquired user speech using a wavelet transform denoising algorithm to obtain denoised user speech. Then, according to a speech recognition algorithm, the denoised user speech is recognized to obtain the corresponding text of the denoised user speech as the semantic recognition result. Here, the aforementioned speech recognition algorithm is not specifically limited and can be adjusted according to actual needs. For example, the speech recognition algorithm can be an ASR algorithm.

[0053] In some optional implementations of certain embodiments, the aforementioned execution entity may generate speech recognition results based on the collected user speech through the following steps:

[0054] The first step is to perform speech recognition on the collected user speech to obtain speech recognition information. In practice, the aforementioned entity can send the collected user speech to a speech recognition server and receive the recognition result from the server as the speech recognition information. For example, the entity can use WebSocket real-time communication technology, HTTP polling, or gRPC communication technology to send the collected user speech to the speech recognition server. In practice, the entity can also use a pre-deployed local offline speech recognition engine for speech recognition. The speech recognition information can be the text of the recognized user speech.

[0055] The second step involves dynamically adjusting the recognized speech information to obtain the adjusted text information as the speech recognition result. In practice, the aforementioned execution entity can perform sentence segmentation on the speech recognition information to dynamically adjust it and obtain the adjusted text information. For example, BERT or GPT models can be used for sentence segmentation to divide long texts into shorter texts. Therefore, the prompting content can be dynamically adjusted based on the user's speech, thereby dynamically adjusting the speech recognition result during the recognition process to improve the accuracy of the speech recognition result and facilitate subsequent text matching.

[0056] In some optional implementations of certain embodiments, the aforementioned execution entity may perform speech recognition on the collected user speech through the following steps to obtain speech recognition information:

[0057] The first step is to generate noise level information for the user's speech. In practice, the execution entity can generate the root mean square (RMS) of the user's speech as noise level information.

[0058] The second step involves adjusting the noise reduction window information in response to the determination that the noise level information meets a preset noise condition, thereby obtaining updated noise reduction window information. The preset noise condition can be that the noise level information is greater than a preset threshold. The noise reduction window information can include the window size of the moving average filter. For example, the default window size can be 5. In practice, the default window size can be increased by a preset value to obtain updated noise reduction window information. For example, the preset value can be 3.

[0059] The third step is to perform noise reduction processing on the user's speech based on the updated noise reduction window information to obtain the noise-reduced user speech. In practice, the user's speech can be processed by performing moving average filtering on the updated noise reduction window information to obtain the noise-reduced user speech.

[0060] The fourth step is to perform speech recognition on the noise-reduced user speech to obtain speech recognition information. The method for speech recognition can be referenced from step 101, and will not be repeated here. This effectively filters out surrounding human voices and mechanical noise, improving the speech recognition effect.

[0061] In some optional implementations of certain embodiments, the aforementioned execution entity may dynamically adjust the recognized speech recognition information through the following steps to obtain adjusted text information as the speech recognition result:

[0062] The first step is to dynamically correct the recognized speech recognition information to obtain corrected speech recognition information. In practice, firstly, the aforementioned executing entity can perform speech recognition on the user's speech again to obtain comparison speech recognition information. The comparison speech recognition information can be represented as text obtained by performing speech recognition on the user's speech again. Then, the similarity between the aforementioned speech recognition information and the comparison speech recognition information can be generated. Next, in response to determining that the similarity is less than a preset similarity, the aforementioned speech recognition information can be corrected to obtain corrected speech recognition information. For example, the text portion in the aforementioned speech recognition information with a confidence level greater than a preset confidence level can be identified first. Then, the text portion can be merged with the text portion in the comparison speech recognition information that differs from the aforementioned text portion to obtain corrected speech recognition information. For example, when a user says "The weather is nice today," it may be recognized as "The weather is nice today" the first time and "The weather is really nice today" the second time. By calculating the similarity between the two recognition results (approximately 0.7), partial correction is required. Because the first recognition was highly reliable, the system retained "Today's weather" and adopted "It's really good" from the second recognition, resulting in the corrected speech recognition information "Today's weather is really good".

[0063] The second step is to add the corrected speech recognition information to the history queue. This history queue can be a fixed-size queue, for example, a queue with a fixed size of 5. When the queue exceeds 5 records, the earliest record added to it can be automatically deleted.

[0064] The third step involves correcting the revised speech recognition information based on the aforementioned historical record queue, resulting in corrected speech recognition information. In practice, the executing entity can first deduplicate each piece of revised speech recognition information in the historical record queue. Then, based on the previous revised speech recognition information in the historical record queue, the newly added revised speech recognition information can be corrected to obtain corrected speech recognition information. For example, correction can be performed using a language model. The language model could be GPT, BERT, etc. The language model can detect low-probability words or phrases in the newly added revised speech recognition information and replace them with higher-probability candidate words. For example, the historical record queue might contain phrases like "Is everyone ready?" When a user says "Let's go eat," the language model can analyze words indicating a group, such as "everyone" and "we," in the historical record queue to infer that the current context may involve a group activity. Therefore, "Let's go eat" can be optimized to "Let's go eat together," making the expression more natural and complete. This improves the accuracy of subsequent recognition.

[0065] The fourth step is to generate adjusted text information based on the corrected speech recognition information. In practice, the executing entity can determine the corrected speech recognition information as the adjusted text information and use the adjusted text information as the speech recognition result.

[0066] In some optional implementations of certain embodiments, the aforementioned execution entity can generate adjusted text information based on the corrected speech recognition information through the following steps:

[0067] The first step is to determine the number of characters included in the above-mentioned corrected speech recognition information.

[0068] The second step involves processing the corrected speech recognition information into segments based on the determined number of characters, obtaining segmented speech recognition information. The preset character condition can be a number of characters greater than a preset character limit. For example, the preset character limit could be 100. In practice, BERT or GPT models can be used for segmentation to divide long texts into shorter texts.

[0069] The third step involves optimizing the punctuation marks on the aforementioned speech recognition information to obtain optimized speech recognition information as the adjusted text information. In practice, BERT or GPT models can be used for punctuation mark optimization to optimize the position and type of punctuation marks. This allows for more accurate recognition of pauses and sentence breaks in natural speech.

[0070] Optionally, firstly, the aforementioned executing entity can clean the obtained adjusted text information to obtain cleaned adjusted text information. In practice, the aforementioned executing entity can delete filler words that meet preset deletion conditions included in the adjusted text information to obtain cleaned adjusted text information. The aforementioned preset deletion conditions can be that the filler word is the same as the filler word in a preset filler word table. The filler words in the aforementioned preset filler word table can represent words that are preset to be deleted from the adjusted text information. For example, the filler words included in the preset filler word table can be: "uh, um, ah, actually, to put it bluntly, that is to say".

[0071] Then, the core vocabulary information in each sentence of the adjusted text information after cleaning can be determined. This core vocabulary information can represent the topic words extracted from the text. These topic words can represent words that summarize the core content of the text. In practice, the executing agent can extract the core vocabulary information in each sentence of the adjusted text information using a BERT model. The BERT model can be represented as a Transformer-based bidirectional encoder.

[0072] Subsequently, based on preset font background information and preset font size information, the core vocabulary information in each sentence of text is modified to alter the adjusted text information, resulting in the modified adjusted text information. The preset font background information can represent the font background. For example, the preset font background information can represent yellow. The preset font size information can represent the font size. For example, the preset font size information can represent a font size of 3. In practice, the executing entity can determine the font size of the core vocabulary information in each sentence of text as the preset font size information, and set the font background of the core vocabulary information to the preset font background information.

[0073] Step 102: Based on the target prompt source information, match the speech recognition results to obtain the matching location information.

[0074] In some embodiments, the execution entity can match the speech recognition result based on the target prompt source information to obtain matching position information. The target prompt source information can be the user's current speech file. The speech file can be pre-uploaded by the user. For example, the speech file can be a speech script or a product introduction script used during a live broadcast. In practice, the execution entity can use a fuzzy matching algorithm to identify the line position of the speech recognition result in the target prompt source information as the matching position information.

[0075] In some optional implementations of certain embodiments, the aforementioned execution entity may obtain matching location information by matching the speech recognition results based on the target prompt source information through the following steps:

[0076] The first step involves determining that the adjusted text information meets a preset character change condition, and then identifying this adjusted text information as the text to be matched. This preset character change condition can be that the number of characters in the adjusted text information is greater than a preset character change value. For example, the preset character change value could be 4. It should be noted that an adjusted text message can contain multiple identical keywords, such as "artificial intelligence." If matching begins within the preset character change value, matching will fail, and the text content that the user is actually reading will not be found, resulting in repeated scrolling and jumping. Matching will be reduced if the preset character change value is exceeded before attempting to match again.

[0077] The second step is to truncate the text information to be matched, obtaining individual truncated texts. In practice, this is done by using punctuation marks as boundaries to truncate the text.

[0078] Third, for each truncated text obtained, perform the following steps:

[0079] The first sub-step is to determine the currently visible text information of the aforementioned target prompt source information. The currently visible text information can be the text content currently displayed on the screen.

[0080] The second sub-step involves matching the truncated text based on the currently visible text information or the target prompt source information to obtain the matching position corresponding to the truncated text.

[0081] The fourth step is to generate matching position information based on the obtained matching positions. In practice, the executing entity can determine the earliest matching position among the various matching positions as the matching position information.

[0082] In some optional implementations of certain embodiments, the aforementioned execution entity may perform text truncation processing on the text information to be matched through the following steps to obtain each truncated text:

[0083] The first step is to perform word segmentation on the text information to be matched, resulting in segmented text information. In practice, the execution entity can use a word segmentation tool to perform word segmentation on the text information to be matched. Here, the specific type of word segmentation tool is not limited; for example, the word segmentation tool could be jieba.

[0084] The second step involves truncating the segmented text to be matched, resulting in individual truncated texts. In practice, the executing entity can define a preset number of words in the segmented text as a truncated text, thus obtaining individual truncated texts. Here, the specific value of the preset number is not limited; for example, the preset number could be 8.

[0085] In some optional implementations of certain embodiments, the aforementioned execution entity may perform text truncation processing on the text information to be matched after the above word segmentation processing through the following steps to obtain each truncated text:

[0086] The first step is to perform semantic recognition processing on the segmented text information to be matched, obtaining the semantic information corresponding to the segmented text information. This semantic information characterizes the semantics of the segmented text information. In practice, the execution entity can use NLP (Natural Language Processing) algorithms to perform semantic recognition processing on the segmented text information to obtain the corresponding semantic information.

[0087] The second step involves segmenting the segmented text based on the semantic information described above, resulting in truncated text segments. In practice, the execution entity can use NLP (Natural Language Processing) algorithms to segment the segmented text based on the semantic information to obtain truncated text segments.

[0088] In some optional implementations of certain embodiments, the aforementioned execution entity may perform text truncation processing on the text information to be matched through the following steps to obtain each truncated text:

[0089] The first step is to perform the following steps for each sentence of text included in the above text information to be matched:

[0090] The first sub-step involves encoding the text to obtain the corresponding sentiment semantic feature information. This sentiment semantic feature information can be a feature vector representing the emotional attribute of the text. For example, it could be a feature vector representing the emotion of the text as "excitement." In practice, the executing entity can use the BERT-CN model to encode the text and obtain the corresponding sentiment semantic feature information. It should be noted that each sentence in the text can represent a sentence divided by punctuation marks in the text to be matched; the specific type of punctuation mark is not limited, for example, it can be a period.

[0091] The second sub-step involves performing a dimensionality transformation on the obtained sentiment semantic feature information to obtain altered sentiment semantic feature information. This altered sentiment semantic feature information can represent the sentiment semantic feature information after the dimensionality transformation. In practice, the executing entity can use a one-dimensional convolutional layer to perform the dimensionality transformation on the obtained sentiment semantic feature information to obtain the altered sentiment semantic feature information. Here, the specific dimensionality of the transformation is not limited and can be adjusted according to actual needs; for example, the sentiment semantic feature information can be transformed from 768 dimensions to 256 dimensions.

[0092] The third sub-step involves semantic condensation of the aforementioned altered emotional semantic features to obtain deep semantic emotional features. These deep semantic emotional features can be feature vectors representing the emotional attributes in the text that go beyond literal expression and are implied in semantic logic and contextual relationships. For example, deep semantic emotional features could represent the deep emotion of the text as intense. In practice, the executing entity can use a self-attention pooling algorithm to perform semantic condensation on the altered emotional semantic features to obtain deep semantic emotional features.

[0093] The fourth sub-step involves performing sentiment representation decomposition on the aforementioned deep semantic sentiment feature information to obtain text-specific feature information and text-common feature information. Specifically, the text-specific feature information represents the feature vector of fine-grained textual characteristic representation. The text-common feature information represents the feature vector of fine-grained textual commonality representation. The fine-grained textual characteristic representation represents the unique, micro-level feature attributes of the aforementioned text. The fine-grained text-common feature representation indicates that during text processing, shared features between texts are extracted and aligned at the fine-grained semantic unit level, while retaining subtle semantic details within the text, achieving accurate fusion and representation between texts. In practice, the executing entity can perform sentiment representation decomposition on the aforementioned deep semantic sentiment feature information through an independent Transformer encoding sub-layer to obtain text-specific feature information. Then, a shared Transformer encoding sub-layer can be used to perform sentiment representation decomposition on the aforementioned deep semantic sentiment feature information to obtain text-common feature information. The aforementioned Transformer coding sublayer can be a coding sublayer containing two layers of multi-head attention (number of heads = 8).

[0094] The fifth sub-step involves correcting the aforementioned text-specific and text-common features to obtain fine-grained modal information. This fine-grained modal information characterizes the corrected text-specific and text-common features. In practice, the executing entity can maximize the feature differences between the text-specific and text-common features using a difference loss function, and minimize the deviation between the corrected text-specific and text-common features and the deep semantic sentiment features using a reconstruction loss function, thereby correcting the text-specific and text-common features to obtain fine-grained modal information. The weighting coefficient between the difference loss function and the reconstruction loss function can be 3:7.

[0095] The second step is to determine the obtained fine-grained modal information as a fine-grained modal information sequence.

[0096] The third step is to determine the similarity between each pair of adjacent fine-grained modal information in the above fine-grained modal information sequence, thus obtaining each similarity information. The similarity information in each of these similarity information sets can characterize the cosine similarity between each pair of adjacent fine-grained modal information.

[0097] The fourth step involves truncating the text to be matched based on the aforementioned similarity information, resulting in truncated texts. Each truncated text represents the text obtained after truncating the text to be matched. In practice, the executing entity can use the positions of texts with similarity scores less than the preset similarity threshold as segmentation points to truncate the text to be matched, obtaining truncated texts. The specific value of the preset similarity threshold is not limited and can be adjusted according to actual needs; for example, the preset similarity threshold could be 70%.

[0098] In some optional implementations of certain embodiments, the execution entity may determine the current visible range text information of the target prompt source information through the following steps:

[0099] The first step is to determine if a matching result exists in the cache corresponding to the truncated text. Therefore, cached historical matching results can be directly used for subsequent matching, avoiding redundant calculations.

[0100] The second step is to determine the current visible range of the target prompt source information in response to the determination that there is no matching result corresponding to the above truncated text in the cache.

[0101] In some optional implementations of certain embodiments, the execution entity may perform the following steps to match the truncated text based on the currently visible text information or the target prompt source information, thereby obtaining the matching position corresponding to the truncated text:

[0102] The first step is to match the truncated text based on the currently visible text information to obtain the matching results. Therefore, matching can be prioritized within the currently visible text range.

[0103] The second step is to determine the matching result as the matching position in response to the determination that the above matching result indicates a successful match.

[0104] Third, in response to the determination that the above matching result indicates a matching failure, the truncated text is matched based on the target prompt source information, and the matching result is used as the matching position. Therefore, if there is no matching text for the truncated text within the currently visible text range, matching can be performed within a larger range, thereby optimizing the matching strategy and shortening the matching time.

[0105] In some optional implementations of certain embodiments, the aforementioned execution entity may match the truncated text through the following steps:

[0106] The first step is to perform an exact match on the truncated text to obtain the first matching result. In practice, the Knuth-Morris-Pratt algorithm can be used to perform an exact match on the truncated text within the currently visible text information or the target prompt source information, and the number of matched lines can be used as the first matching result. If an exact match fails, an empty value or a preset failure value can be used as the first matching result.

[0107] The second step, in response to the determination that the first matching result indicates a matching failure, is to determine the longest common substring in the currently visible text information or the target prompt source information that corresponds to the truncated text. The longest common substring can be the longest common string between the original text and the truncated text (the order of the strings is from front to back). In practice, a dynamic programming algorithm can be used to find the longest common substring.

[0108] The third step is to match the longest common substring to obtain the second matching result. In practice, in response to determining that the longest common substring is not empty, the line number of the longest common substring in the target prompt source information can be determined as the second matching result. In response to determining that the longest common substring is empty, an empty value or a preset failure value can be determined as the second matching result.

[0109] Fourth, in response to the determination that the second matching result indicates a matching failure, determine the trailing string corresponding to the truncated text. The trailing string can be the last preset number of characters of the truncated text.

[0110] The fifth step is to match the aforementioned trailing string to obtain the third matching result. In practice, the trailing string can be matched within the currently visible text information or the target prompt source information. When the trailing string is matched, the line number of the matched string in the target prompt source information can be determined as the third matching result. When the trailing string is not matched, an empty value or a preset failure value can be determined as the third matching result.

[0111] Step six: In response to the determination that the third matching result indicates a matching failure, extract each phrase from the truncated text. In practice, the executing entity can extract phrases with preset parts of speech combinations from the truncated text. For example, preset parts of speech combinations may include, but are not limited to: adjective + noun, verb + noun.

[0112] Step 7: Match each of the above phrases to obtain the fourth matching result. In practice, each phrase can be matched within the currently visible text information or the target prompt source information. When a phrase is matched, the minimum number of lines in the target prompt source information for each matched phrase can be determined as the fourth matching result. When no phrase is matched, an empty value or a preset failure value can be determined as the fourth matching result.

[0113] Step 8: In response to the determination that the fourth matching result indicates a matching failure, a fuzzy match is performed on the truncated text to obtain the fifth matching result. The fifth matching result can be the number of lines matched by the fuzzy match.

[0114] In the ninth step, in response to the determination that the fifth matching result indicates a successful match, the fifth matching result is set as the matching position corresponding to the truncated text. Therefore, a multi-level matching strategy can be employed to ensure that the optimal matching position is found in all situations.

[0115] In some optional implementations of certain embodiments, the aforementioned execution entity may match the truncated text through the following steps:

[0116] The first step is to determine the first matching result as the matching position corresponding to the truncated text in response to the determination that the first matching result indicates a successful match.

[0117] The second step is to determine the matching position corresponding to the truncated text in response to the determination that the second matching result indicates a successful match.

[0118] Third, in response to the determination that the third matching result indicates a successful match, the third matching result is determined as the matching position corresponding to the truncated text.

[0119] Fourth step: In response to the determination that the above fourth matching result indicates a successful match, the above fourth matching result is determined as the matching position corresponding to the above truncated text.

[0120] Step 103: Generate scrolling information based on the matching position information.

[0121] In some embodiments, the execution entity can generate scrolling information based on the matching position information. This scrolling information can be parameter information regarding the scrolling distance from the current prompting position to the matching position. The current prompting position can be represented by a row number. In practice, the execution entity can determine the interval distance as the difference between the matching position and the current prompting position. Then, the sum of the interval distance and a preset adjustment row number can be used to determine the scrolling information. The preset adjustment row number can serve as a buffer row number for scrolling, ensuring that the scrolled prompting content doesn't abruptly appear on the first line of the screen, but rather in the upper half. For example, the preset adjustment row number can be 1.

[0122] In some optional implementations of certain embodiments, the aforementioned execution entity can generate scrolling information based on the aforementioned matching position information through the following steps:

[0123] The first step is to determine the distance between the matched position information and the current display position. The current display position can be the number of rows currently displayed on the screen. In practice, the executing entity can determine the distance between the matched position information and the current display position as the pixel distance between the matched position information and the current display position.

[0124] The second step involves generating a scrolling step size based on the aforementioned distance, the time interval since the last adjustment of the prompt display content, and the scrolling speed. The scrolling speed can be a base scrolling speed coefficient used to adjust the overall scrolling speed. The executing entity can first determine the product of the aforementioned time interval and the aforementioned scrolling speed. Then, it can take the minimum value between the aforementioned product and a preset ratio. Finally, the product of the aforementioned distance and the minimum value is determined as the scrolling step size.

[0125] The third step is to generate the scroll position as scroll information based on the current display position and the scroll step size mentioned above. In practice, the scroll information can be determined by the sum of the pixel x-coordinate corresponding to the current display position and the scroll step size. This scroll information can then be used as the pixel x-coordinate of the position to be displayed after scrolling. This allows for smooth scrolling and avoids abrupt jumps.

[0126] Optionally, the aforementioned implementing entity may also perform the following steps:

[0127] The first step is to determine the user's speech rate corresponding to the above user voice. Here, the user's speech rate can be expressed as the average number of words spoken per minute for the user's voice.

[0128] The second step involves updating the scrolling speed based on the user's speaking speed and the scrolling speed, in response to the determination that they meet a first preset speed condition. The first preset speed condition can be that the user's speaking speed is greater than the product of the scrolling speed and a first preset coefficient. For example, the first preset coefficient can be 1.5. The preset maximum scrolling speed can be 2.0. In practice, the executing entity can determine the first adjusted scrolling speed as the product of the scrolling speed and a second preset coefficient. The second preset coefficient can be 1.2. Then, the minimum value between the first adjusted scrolling speed and the preset maximum scrolling speed can be determined as the updated scrolling speed to update the scrolling speed.

[0129] The third step involves updating the scrolling speed based on the user's speaking speed and the scrolling speed, in response to the determination that they meet a second preset speed condition. The first preset speed condition can be that the user's speaking speed is less than the product of the scrolling speed and a third preset coefficient. For example, the third preset coefficient can be 0.5. The preset minimum scrolling speed can be 0.5. In practice, the executing entity can determine the second adjusted scrolling speed as the product of the scrolling speed and a fourth preset coefficient. The fourth preset coefficient can be 0.8. Then, the maximum value between the second adjusted scrolling speed and the preset minimum scrolling speed can be determined as the updated scrolling speed to update the scrolling speed.

[0130] The fourth step involves updating the scrolling speed to the preset speed in response to the determination that the pause duration of the user's speech meets a preset duration condition. The preset duration condition can be a pause duration longer than a preset duration. For example, the preset duration could be 1.5 seconds. The preset speed could be 0.5 or 0. This allows for dynamic adjustment of the scrolling speed to adapt to changes in the speaker's speaking rate.

[0131] Step 104: Adjust the prompt content corresponding to the target prompt source information based on the scrolling information.

[0132] In some embodiments, the executing entity can adjust the prompting content corresponding to the target prompt source information based on the scrolling information. The prompting content can be the text in the target prompt source information that needs to indicate the user's speaking progress. In practice, the executing entity can scroll the text of the target prompt source information displayed on the screen according to the scrolling information, and make the prompting content corresponding to the matching position information prominently displayed on the screen.

[0133] As an example, the adjusted prompt display content can be as follows: Figure 2As shown. The matching location information can be the second line of the target prompt source information. If the prompt content of the second line is "We need to pay attention to the development trend of artificial intelligence", then the text of the second line can be enlarged on the screen for highlighting.

[0134] Optionally, the aforementioned execution entity can generate the next scroll position as predicted scroll information based on the average historical scroll speed, average historical scroll acceleration, and the current display position. In practice, the product of the average historical scroll speed and the preset prediction duration can be first determined as the first product. Then, the product of the average historical scroll acceleration and the square of the preset coefficient and the preset prediction duration can be determined as the second product. Finally, the sum of the current display position, the first product, and the second product can be determined as the next scroll position, which is the predicted scroll information. Then, based on the predicted scroll information, the visible range text information corresponding to the predicted scroll information in the target prompt source information can be determined. The visible range text information corresponding to the predicted scroll information can be the text content in the target prompt source information displayed within the screen range after scrolling the screen display content according to the predicted scroll information. The visible range text information can be used to prioritize matching the text corresponding to the next user voice.

[0135] Optionally, the aforementioned execution entity can respond to the detected gesture operation of the user corresponding to the aforementioned user's voice by capturing images of the gesture operation to obtain various gesture images. The gesture operation can be the user displaying a gesture in front of the execution entity's camera. In practice, images of the user's hand can be continuously captured by the camera of the execution entity to obtain various gesture images. Then, the control type can be identified based on the various gesture images. In practice, the various gesture images can be matched with gesture images in a preset set of standard gesture images to identify the matching standard gesture image set, and the control type corresponding to the matching standard gesture image set can be determined as the identified control type. Each standard gesture image set can be images of various standard gestures corresponding to the same control type. Finally, the prompting display content corresponding to the aforementioned target prompt source information can be controlled according to the aforementioned control type. The control logic corresponding to each control type can be pre-set. For example, the control type corresponding to the five-finger gesture can be to pause the update and adjustment of the prompting display content. The control type corresponding to the thumb and forefinger gesture can be to zoom in on the prompting display content. Control types can include, but are not limited to: pause, fast forward, rewind, zoom out, zoom in, brighten, and dim. This allows users to control the display of prompts through gesture interaction.

[0136] Optionally, the aforementioned implementing entity may also perform the following steps:

[0137] The first step is to extract contextual text information from the target prompt source information based on the matching location information mentioned above. In practice, the executing entity can use the matching location information as the center and read the text content from the first preset number of lines before to the second preset number of lines after in the target prompt source information as the extracted contextual text information.

[0138] The second step involves extracting a summary and keywords from the aforementioned contextual text information to form historical context information. In practice, the executing entity can use summary extraction algorithms and keyword extraction algorithms to extract the summary and keywords from the contextual text information, respectively. Then, the extracted summary and keywords can be combined to form historical context information.

[0139] The third step involves inputting the matching location information, the text embedding vector sequence corresponding to the target cue source information, and the historical context information into the input layer of a pre-trained content prediction model. The content prediction model can be a neural network model that takes the matching location information, the text embedding vector sequence corresponding to the target cue source information, and the historical context information as input data, and outputs the predicted content of the next speech. The text embedding vector sequence corresponding to the target cue source information can be a sequence composed of the embedding vectors of each word. The content prediction model can include an input layer, a graph construction layer, a graph attention layer, a dynamic memory enhancement layer, and a content prediction layer. The input layer provides all the input information required by the model, including the matching location information representing the current speech position, the text embedding vector sequence representing the original text content, and the historical context information representing the historical context.

[0140] The fourth step involves inputting the text embedding vector sequence into the graph construction layer to obtain graph-structured data. The graph construction layer can be used to build a fully connected graph based on target cue source information, where each node represents a word, and edges represent the relationships between words, facilitating the subsequent use of graph attention mechanisms to capture global dependencies. The node feature dimension can be equal to the word embedding dimension (e.g., 768 dimensions), and the edge weights can be initialized to 1, indicating that all words initially have the same association strength. Specifically, the graph construction layer can first construct a fully connected graph. Each node is connected to all other nodes, and the initial edge weights are set to 1. Then, the text embedding vectors can be assigned to the node features. Afterward, an adjacency matrix can be used to represent the edge relationships.

[0141] Fifth, the graph structure data and matching position information described above are input into the graph attention layer to obtain the updated node features. The graph attention layer can use a Graph Attention Network (GAT) to process the graph structure data and calculate the importance weight of each node. Utilizing the graph attention mechanism, the weights between nodes are dynamically adjusted based on the matching position information representing the current speaking position, highlighting contextual information related to the current speaking position. The graph attention layer can have 8 attention heads, each head can have 128 hidden units, and the activation function chosen is LeakyReLU.

[0142] The sixth step involves inputting the updated node features and historical context information into the dynamic memory enhancement layer to obtain the updated memory state. The dynamic memory enhancement layer can incorporate a dynamic memory module to store and update historical context information, and dynamically update the memory state based on the current speaking position to predict future content. The dynamic memory enhancement layer can have 64 memory slots, each with a dimension of 128. The update rule can employ a gating mechanism (e.g., GRU), and finally, an output gate can be used to control the reading of the memory state to obtain the updated memory state.

[0143] Step 7: Input the updated memory state into the content prediction layer to obtain the predicted content. The content prediction layer can use fully connected layers to predict the next possible content. When training the content prediction model, cross-entropy loss can be used as the loss function, and the Adam optimizer can be used for model optimization. Specifically, the content prediction layer can flatten the updated memory state into a one-dimensional vector. Then, two fully connected layers can be used for mapping to obtain the predicted content.

[0144] Step 8: Based on the above predictions, construct the corresponding query text. In practice, keywords can be extracted from the predictions as prediction keywords first. Then, a preset text template can be used to construct the query text corresponding to the prediction keywords. For example, the preset text template could be "Please provide the latest research on XXX". Here, "XXX" can be the string position of the keyword to be entered. For example, the prediction keyword corresponding to the prediction content could be "climate change policy". The constructed query text could be "Please provide the latest research on climate change policy".

[0145] Step nine: Obtain the query content sequence corresponding to the above query text. In practice, an API can be used to call an external search engine or internal knowledge base to obtain the query content sequence corresponding to the above query text. The query content sequence can be the result of ranking search results based on relevance scores.

[0146] Step 10: For each query in the above query sequence, generate a summary of the corresponding query. In practice, a text summarization algorithm (such as BERTSUM) can be used to generate the summary of the query.

[0147] The eleventh step involves retrieving information on each entity and relationship related to the predicted content from a pre-constructed knowledge graph. This knowledge graph can be an encyclopedic knowledge graph. In practice, information on each entity and relationship related to the predicted keywords can be retrieved from the knowledge graph. Entity information can include entity names and corresponding attributes. For example, for "climate change policy," relevant entities such as "Paris Agreement," "carbon emissions," and "renewable energy" can be found.

[0148] Step 12: Based on the aforementioned entity and relation information, construct a local knowledge graph. In practice, knowledge graph construction methods from graph databases can be used to build a local knowledge graph for the corresponding predicted keywords from the aforementioned entity and relation information.

[0149] The thirteenth step is to extract the entity groups from the aforementioned local knowledge graph. In practice, entity groups can be extracted using the relationship chains within the local knowledge graph, with each entity group comprising individual entities. For example, the extracted entity groups could include: climate change policy - Paris Agreement - international cooperation; carbon tax - carbon emissions - energy transition.

[0150] Step fourteen: Based on the aforementioned entity groups, generate associated topics. In practice, for each entity group, at least one topic matching at least two entities in the group can be determined from a pre-defined topic set, resulting in associated topics. Here, matching can include precise text matching and fuzzy text matching. For example, the generated associated topics could include: the role of international cooperation in addressing climate change; the impact of carbon tax policies on economic development; and the latest advancements in renewable energy technologies. The pre-defined topic set can be stored in a search engine.

[0151] Step 15: Based on the above predictions, sort the related topics to obtain a sequence of related topics. In practice, the related topics can be sorted in descending order according to their similarity to the predicted keywords to obtain the sequence of related topics.

[0152] Step sixteen: For each related topic in the aforementioned related topic sequence, generate corresponding suggestion content as enhanced suggestion content. In practice, the executing entity can use an API to call an external search engine or internal knowledge base to obtain search content corresponding to the aforementioned related topics. Then, a summary can be extracted from the search content as the corresponding suggestion content, i.e., enhanced suggestion content. Thus, enhanced suggestion content can serve as a further extension of the directly queried content, providing users with richer suggestion content.

[0153] Step seventeen: Display the generated summary information and enhanced prompts. In practice, the above-mentioned summary information and enhanced prompts can be displayed next to the current prompt content.

[0154] Steps one through seventeen above constitute an inventive aspect of this disclosure, solving the technical problem that "traditional teleprompter displays can only passively display the content of the speech manuscript. When it is necessary to explain content outside the manuscript, they cannot proactively provide relevant background information or prompts to the speaker, causing the speaker to need to prepare relevant content on their own or rely entirely on improvisation, resulting in a poor user experience." Factors leading to a poor user experience often include: traditional teleprompter displays can only passively display the content of the speech manuscript, and when it is necessary to explain content outside the manuscript, they cannot proactively provide relevant background information or prompts to the speaker, causing the speaker to need to prepare relevant content on their own or rely entirely on improvisation. Solving these factors can improve the user experience. To achieve this effect, this disclosure introduces advanced natural language processing technology, which can capture global dependencies and semantic information in the original speech text. Combined with dynamic context modeling, it dynamically adjusts the context weights according to the current speaking position, enabling more accurate prediction of the following content. Then, based on the predicted next content, it can automatically retrieve relevant information from external resources (such as knowledge graphs and search engines) and generate concise and enhanced prompts. This dynamic loading function significantly improves the user's preparation efficiency, reduces cognitive burden, meets the user's personalized prompting needs, and enhances the user experience.

[0155] The above-described embodiments of this disclosure have the following beneficial effects: the prompting display method of some embodiments of this disclosure improves the prompting experience and synchronization effect for users speaking. Specifically, the reasons for poor user experience and synchronization effect are: manual operation requires distraction, resulting in a poor user experience; and timed scrolling cannot scroll the text according to the actual progress of speaking, resulting in poor synchronization effect. Based on this, the prompting display method of some embodiments of this disclosure first generates a speech recognition result based on the collected user speech. This allows for the recognition of the user's speech during the speaking process. Next, based on the target prompt source information, the speech recognition result is matched to obtain matching position information. This allows for the location of the matching position of the adjusted text information in the target prompt source information; the matching position can be understood as the position corresponding to the current speech. Then, scrolling information is generated based on the matching position information. This allows for the determination of the scrolling-related parameters required during the display process based on the matching position. Finally, the prompting display content corresponding to the target prompt source information is adjusted based on the scrolling information. Therefore, the prompting content can follow the user's speaking progress during the speaking process without the need for manual operation, thereby improving the user's prompting experience and the prompting synchronization effect.

[0156] Further reference Figure 3 As an implementation of the methods shown in the above figures, this disclosure provides some embodiments of a prompting display device, which are similar to... Figure 1 Corresponding to the method embodiments shown, the device can be specifically applied to various electronic devices.

[0157] like Figure 3 As shown, the prompting display device 300 in some embodiments includes: a first generation unit 301, a matching unit 302, a second generation unit 303, and an adjustment unit 304. The first generation unit 301 is configured to generate a speech recognition result based on collected user speech; the matching unit 302 is configured to match the speech recognition result based on target prompt source information to obtain matching position information; the second generation unit 303 is configured to generate scrolling information based on the matching position information; and the adjustment unit 304 is configured to adjust the prompting display content corresponding to the target prompt source information based on the scrolling information.

[0158] Optionally, the first generation unit 301 is further configured to: perform speech recognition on the collected user speech to obtain speech recognition information; and dynamically adjust the recognized speech recognition information to obtain adjusted text information as the speech recognition result.

[0159] Optionally, the first generation unit 301 is further configured to: generate noise level information of the user speech; adjust the noise reduction window information in response to determining that the noise level information meets the preset noise conditions to obtain updated noise reduction window information; perform noise reduction processing on the user speech according to the updated noise reduction window information to obtain noise-reduced user speech; and perform speech recognition on the obtained noise-reduced user speech to obtain speech recognition information.

[0160] Optionally, the first generation unit 301 is further configured to: perform dynamic correction processing on the recognized speech recognition information to obtain corrected speech recognition information; add the obtained corrected speech recognition information to the history queue; correct the corrected speech recognition information according to the history queue to obtain corrected speech recognition information; and generate adjusted text information according to the corrected speech recognition information.

[0161] Optionally, the first generation unit 301 is further configured to: determine the number of characters included in the corrected speech recognition information; in response to determining that the number of characters meets a preset character condition, perform sentence segmentation processing on the corrected speech recognition information to obtain segmented speech recognition information; and perform punctuation optimization processing on the segmented speech recognition information to obtain optimized speech recognition information as adjusted text information.

[0162] Optionally, the matching unit 302 is further configured to: in response to determining that the speech recognition result meets the preset character change conditions, determine the speech recognition result as text information to be matched; perform text truncation processing on the text information to be matched to obtain each truncated text; for each obtained truncated text, perform the following steps: determine the current visible range text information of the target prompt source information; match the truncated text based on the current visible range text information or the target prompt source information to obtain the matching position corresponding to the truncated text; generate matching position information according to each obtained matching position.

[0163] Optionally, the matching unit 302 is further configured to: determine whether there is a matching result in the cache corresponding to the truncated text; and in response to determining that there is no matching result in the cache corresponding to the truncated text, determine the current visible range text information of the target prompt source information.

[0164] Optionally, the matching unit 302 is further configured to: match the truncated text based on the currently visible text information to obtain a matching result; in response to determining that the matching result indicates a successful match, determine the matching result as a matching position; in response to determining that the matching result indicates a failed match, match the truncated text based on the target prompt source information to obtain a matching result as a matching position.

[0165] Optionally, the matching unit 302 is further configured to: perform precise matching on the truncated text to obtain a first matching result; in response to determining that the first matching result indicates a matching failure, determine the longest common substring corresponding to the truncated text in the currently visible text information or the target prompt source information; match the longest common substring to obtain a second matching result; in response to determining that the second matching result indicates a matching failure, determine the tail string corresponding to the truncated text; match the tail string to obtain a third matching result; in response to determining that the third matching result indicates a matching failure, extract each word group from the truncated text; match each word group to obtain a fourth matching result; in response to determining that the fourth matching result indicates a matching failure, perform fuzzy matching on the truncated text to obtain a fifth matching result; in response to determining that the fifth matching result indicates a successful matching, determine the fifth matching result as the matching position corresponding to the truncated text.

[0166] Optionally, the matching unit 302 is further configured to: in response to determining that the first matching result indicates a successful match, determine the first matching result as the matching position corresponding to the truncated text; in response to determining that the second matching result indicates a successful match, determine the second matching result as the matching position corresponding to the truncated text; in response to determining that the third matching result indicates a successful match, determine the third matching result as the matching position corresponding to the truncated text; and in response to determining that the fourth matching result indicates a successful match, determine the fourth matching result as the matching position corresponding to the truncated text.

[0167] Optionally, the second generation unit 303 is further configured to: determine the distance between the matching position information and the current display position; generate a scrolling step size based on the distance, the time interval between the last adjustment of the prompting display content and the scrolling speed; and generate a scrolling position as scrolling information based on the current display position and the scrolling step size. Optionally, the prompting display device 300 may further include: a determining unit, a first updating unit, a second updating unit and a third updating unit (not shown in the figure). The determining unit is configured to determine the user's speech rate corresponding to the user's speech; the first updating unit is configured to update the scrolling speed based on the scrolling speed and a preset maximum scrolling speed in response to determining that the user's speech rate and the scrolling speed meet a first preset speed condition; the second updating unit is configured to update the scrolling speed based on the scrolling speed and a preset minimum scrolling speed in response to determining that the user's speech rate and the scrolling speed meet a second preset speed condition; and the third updating unit is configured to update the scrolling speed to a preset speed in response to determining that the pause duration of the user's speech meets a preset duration condition.

[0168] Optionally, the prompting display device 300 may further include a position generation unit and an information determination unit (not shown in the figure). The position generation unit is configured to generate a predicted scrolling position as predicted scrolling information based on the average historical scrolling speed, the average historical scrolling acceleration, and the current display position. The information determination unit is configured to determine, based on the predicted scrolling information, the visible range text information corresponding to the predicted scrolling information in the target prompt source information, wherein the visible range text information is used for priority matching of the text corresponding to the next user's speech.

[0169] It is understandable that the units described in the prompting display device 300 are related to the reference. Figure 1 The steps in the described method correspond to each other. Therefore, the operations, features, and beneficial effects described above for the method also apply to the teleprompter display device 300 and the units contained therein, and will not be repeated here.

[0170] The following is for reference. Figure 4 It shows a schematic diagram of the structure of an electronic device 400 (e.g., a head-mounted display device or a terminal device) suitable for implementing some embodiments of the present disclosure. Figure 4 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0171] like Figure 4 As shown, the electronic device 400 may include a processing unit 401 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage device 408 into a random access memory (RAM) 403. The RAM 403 also stores various programs and data required for the operation of the electronic device 400. The processing unit 401, ROM 402, and RAM 403 are interconnected via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0172] Typically, the following devices can be connected to I / O interface 405: input devices 406 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 407 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; and communication devices 409. Communication device 409 allows electronic device 400 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 4 An electronic device 400 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 4Each box shown can represent a device or multiple devices as needed.

[0173] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 409, or installed from storage device 408, or installed from ROM 402. When the computer program is executed by processing device 401, it performs the functions defined above in the methods of some embodiments of this disclosure.

[0174] It should be noted that, in some embodiments of this disclosure, the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0175] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0176] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: generate a speech recognition result based on the collected user speech; match the speech recognition result with target prompt source information to obtain matching position information; generate scrolling information based on the matching position information; and adjust the prompting content corresponding to the target prompt source information based on the scrolling information.

[0177] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0178] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0179] The units described in some embodiments of this disclosure can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor may be described as including a first generation unit, a matching unit, a second generation unit, and an adjustment unit. The names of these units do not necessarily limit the specific unit; for example, the first generation unit may also be described as "a unit that generates speech recognition results based on collected user speech."

[0180] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0181] Some embodiments of this disclosure also provide a computer program product, including a computer program that, when executed by a processor, implements any of the above-described prompting display methods.

[0182] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A teleprompter display method, comprising: Based on the collected user speech, generate speech recognition results; Based on the target prompt source information, the speech recognition result is matched to obtain the matching location information; Based on the matching position information, scrolling information is generated; Based on the scrolling information, the prompting content corresponding to the target prompt source information is adjusted; The step of matching the speech recognition result based on the target prompt source information to obtain the matching location information includes: In response to determining that the speech recognition result meets the preset character change conditions, the speech recognition result is determined as the text information to be matched; The text information to be matched is truncated to obtain each truncated text. For each truncated text obtained, perform the following steps: Determine the current visible range of the target prompt source information; Based on the currently visible text information or the target prompt source information, the truncated text is matched to obtain the matching position corresponding to the truncated text; Based on the obtained matching positions, generate matching position information.

2. The method according to claim 1, wherein, The process of generating speech recognition results based on collected user speech includes: The collected user voice is subjected to speech recognition to obtain speech recognition information; The recognized speech information is dynamically adjusted to obtain the adjusted text information as the speech recognition result.

3. The method according to claim 2, wherein, The process of performing speech recognition on the collected user speech to obtain speech recognition information includes: Generate noise level information for the user's speech; In response to determining that the noise level information meets the preset noise conditions, the noise reduction window information is adjusted to obtain updated noise reduction window information; Based on the updated noise reduction window information, the user's voice is subjected to noise reduction processing to obtain noise-reduced user voice; The noise-reduced user speech is subjected to speech recognition to obtain speech recognition information.

4. The method according to claim 2, wherein, The step of dynamically adjusting the recognized speech recognition information to obtain adjusted text information as the speech recognition result includes: The recognized speech recognition information is dynamically corrected to obtain corrected speech recognition information; Add the obtained corrected speech recognition information to the history queue; Based on the historical record queue, the corrected speech recognition information is corrected to obtain corrected speech recognition information; Based on the corrected speech recognition information, the adjusted text information is generated.

5. The method according to claim 4, wherein, The step of generating adjusted text information based on the corrected speech recognition information includes: Determine the number of characters included in the corrected speech recognition information; In response to determining that the number of characters meets a preset character condition, the corrected speech recognition information is processed by segmentation to obtain segmented speech recognition information; The punctuation marks of the sentence segmentation speech recognition information are optimized to obtain optimized speech recognition information as adjusted text information.

6. The method according to claim 5, wherein, The text information determining the current visible range of the target prompt source information includes: Determine whether a matching result corresponding to the truncated text exists in the cache; In response to determining that there is no matching result corresponding to the truncated text in the cache, the current visible range text information of the target prompt source information is determined.

7. The method according to claim 1, wherein, The step of matching the truncated text based on the currently visible text information or the target prompt source information to obtain the matching position corresponding to the truncated text includes: Based on the currently visible text information, the truncated text is matched to obtain the matching result; In response to determining that the matching result indicates a successful match, the matching result is determined as the matching position; In response to determining that the matching result indicates a matching failure, the truncated text is matched based on the target prompt source information to obtain the matching result as the matching position.

8. The method according to claim 7, wherein, The matching of the truncated text includes: The truncated text is precisely matched to obtain the first matching result; In response to determining that the first matching result indicates a matching failure, the longest common substring corresponding to the truncated text is determined in the currently visible text information or the target prompt source information; The longest common substring is matched to obtain a second matching result; In response to determining that the second matching result indicates a matching failure, the tail string corresponding to the truncated text is determined; The tail string is matched to obtain a third matching result; In response to determining that the third matching result indicates a matching failure, each phrase is extracted from the truncated text; The four matching results are obtained by matching each of the aforementioned phrases. In response to determining that the fourth matching result indicates a matching failure, a fuzzy match is performed on the truncated text to obtain a fifth matching result; In response to determining that the fifth matching result indicates a successful match, the fifth matching result is determined as the matching position corresponding to the truncated text.

9. The method according to claim 8, wherein, The matching of the truncated text includes: In response to determining that the first matching result indicates a successful match, the first matching result is determined as the matching position corresponding to the truncated text; In response to determining that the second matching result indicates a successful match, the second matching result is determined as the matching position corresponding to the truncated text; In response to determining that the third matching result indicates a successful match, the third matching result is determined as the matching position corresponding to the truncated text; In response to determining that the fourth matching result indicates a successful match, the fourth matching result is determined as the matching position corresponding to the truncated text.

10. The method according to claim 1, wherein, The step of generating scrolling information based on the matching position information includes: Determine the distance between the matched location information and the current display location; The scroll step is generated based on the distance, the time interval between the last adjustment of the prompt display content, and the scroll speed; Based on the current display position and the scroll step size, a scroll position is generated as scroll information.

11. The method according to claim 10, wherein, The method further includes: Determine the user's speech rate corresponding to the user's voice; In response to determining that the user's speaking speed and the scrolling speed meet a first preset speed condition, the scrolling speed is updated according to the scrolling speed and the preset maximum scrolling speed; In response to determining that the user's speaking speed and the scrolling speed meet a second preset speed condition, the scrolling speed is updated according to the scrolling speed and a preset minimum scrolling speed; In response to determining that the pause duration of the user's voice meets a preset duration condition, the scrolling speed is updated to a preset speed.

12. The method according to claim 1, wherein, The method further includes: Based on the average historical scroll speed, average historical scroll acceleration, and current display position, the next scroll position is generated as predicted scroll information; Based on the predicted scrolling information, the visible range text information corresponding to the predicted scrolling information in the target prompt source information is determined, wherein the visible range text information is used to prioritize the matching of the text corresponding to the next user's voice.

13. A teleprompter display device, comprising: The first generation unit is configured to generate speech recognition results based on the collected user speech. The matching unit is configured to match the speech recognition result based on the target prompt source information to obtain the matching location information; The second generation unit is configured to generate scrolling information based on the matching position information; The adjustment unit is configured to adjust the prompt display content corresponding to the target prompt source information based on the scrolling information; The matching unit is configured as follows: In response to determining that the speech recognition result meets the preset character change conditions, the speech recognition result is determined as the text information to be matched; The text information to be matched is truncated to obtain each truncated text. For each truncated text obtained, perform the following steps: Determine the current visible range of the target prompt source information; Based on the currently visible text information or the target prompt source information, the truncated text is matched to obtain the matching position corresponding to the truncated text; Based on the obtained matching positions, generate matching position information.

14. An electronic device comprising: One or more processors; Storage device, on which one or more programs are stored, When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-12.

15. A computer-readable medium having a computer program stored thereon, wherein, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-12.

16. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-12.

Images