A method for interrupting and resuming narration in a large-screen intelligent digital human guide
By generating multi-dimensional tour context snapshots and realizing the synchronous restoration of voice, animation and multimedia, the problem of synchronous restoration of multimodal data streams that are interrupted in the large-screen AI digital human tour system is solved, improving the continuity of interaction and system stability, and ensuring the high availability and natural transition effect of the tour service.
Patent Information
- Application Number
- CN202611316617.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-28
- Publication Date
- 2026-11-17
AI Technical Summary
Existing technologies in large-screen AI digital human guide systems suffer from several drawbacks when interrupting and resuming narration. These include: a single and incomplete dimension of breakpoint information recording; a lack of synchronization mechanism for multi-channel media recovery; a lack of natural transition and interactive feedback during the continuation process; and a lack of rollback and fault tolerance mechanisms under abnormal conditions. Consequently, these issues result in low accuracy of multimodal data stream synchronization recovery, poor interactive continuity, and insufficient system stability.
By monitoring interruption signals, multi-dimensional information including a snapshot of the navigation context is generated, including the current node identifier, the duration of playback within a sentence, the global offset of the digital human-driven script, and the status of associated media materials. This enables the synchronous recovery of voice, animation, and multimedia, and a degraded recovery mode is adopted in abnormal situations to ensure the system's fault tolerance and robustness.
It achieves high-precision synchronous recovery of multimodal data streams in interrupted and resumed scenarios, ensuring interactive continuity and natural transition effects, improving the availability and stability of the system under abnormal operating conditions, and providing complete breakpoint information recording and rigorous collaborative synchronization strategies.
Smart Images

Figure CN123123671A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of artificial intelligence and multimedia interaction technology, and in particular to a method for interrupting and resuming narration in a large-screen intelligent digital human guide. Background Technology
[0002] In public exhibition venues such as museums, science and technology museums, corporate exhibition halls, and government service halls, AI digital human guided tour systems based on large screens are becoming increasingly popular. These systems use virtual avatars with synchronized lip movements, facial expressions, and body movements, integrating voice narration with multimedia content such as text, images, and videos to provide visitors with immersive guided tours and explanations, significantly improving information delivery efficiency and the overall exhibition experience.
[0003] Ideally, the system should be able to smoothly deliver a linear explanation of the preset content. However, in real-world interactive scenarios, visitors often have immediate questions midway through the explanation (such as inquiring about the specific era of the exhibits, the definition of terms, or background details), necessitating an interruption of the current explanation and a switch to a question-and-answer interaction mode. After the question-and-answer session, the system must be able to naturally resume the explanation from the precise point of interruption, accurately connecting the playback progress before the interruption, without repeating already explained content or omitting subsequent information, thus ensuring the coherence and completeness of the guided tour narrative.
[0004] This "interruption-resumption" requirement faces unique technical complexities in large-screen AI digital human guided tour scenarios. Specifically, the guided tour content is presented as a synchronous playback process of multimodal data streams, involving: decoding and outputting the audio stream, real-time parsing and execution of the digital human-driven script (used to control animation parameters such as lip movements, facial expressions, and gestures), and the coordinated scheduling of multiple media resources such as exhibit images, videos, and dynamic subtitles. Deviations in the recovery of any data stream can lead to significant sensory misalignments, such as "the digital human's movements being out of sync with the audio progress" or "the on-screen content being inconsistent with the narration's theme," thereby severely damaging the user's immersion and the system's reliability.
[0005] To address the aforementioned issues, existing technologies have proposed several processing strategies, but none have provided a complete solution suitable for AI digital human navigation scenarios, as detailed below: Firstly, there is a crude management approach based on "full reset" or "direct redirection".
[0006] Some systems employ a simplistic either-or strategy: either completely prohibiting interruptions and requiring users to wait until the explanation is finished before asking questions, which sacrifices the real-time nature and naturalness of the interaction; or using "replay after interruption," which replays the explanation of the current exhibit or section after the Q&A session, preserving the integrity of the content but causing repetitive playback, which can easily annoy users; or using "jump after interruption," which directly skips the unfinished explanation, resulting in missing information and affecting the actual effectiveness of the guided tour.
[0007] Secondly, state recording and recovery based on a single audio timestamp.
[0008] A few systems attempt to resume playback by recording audio playback timestamps. The approach involves recording the audio playback progress when an interruption occurs, and then simply positioning the audio player to that timestamp upon resumption. However, this solution fails to simultaneously record the progress of the digital human-driven script and the specific state of the multimedia player. This results in the digital human animation and audio being out of sync after resumption, and related multimedia content (such as videos and images) failing to connect precisely, causing a misalignment between sound, image, and human, and leading to a chaotic display logic.
[0009] In summary, existing technologies generally suffer from the following shortcomings when addressing the need for interrupted and resumed narration in large-screen AI digital human guided tours: 1. Breakpoint information recording is limited in scope and incomplete: Most solutions only record coarse-grained paragraph identifiers or audio timestamps, failing to capture and save key, multi-dimensional breakpoint information such as "playback duration within a sentence", "global offset of the digital human-driven script", and "real-time playback status of each associated media material (such as video frame position, image switching markers, etc.)", resulting in a weak foundation for recovery.
[0010] 2. Lack of synchronization mechanism for multi-channel media recovery: Due to incomplete breakpoint information, the voice engine, animation rendering pipeline and media player cannot be reset in a coordinated manner during recovery, resulting in a misalignment of "sound, picture and person". This is the core technical problem affecting the continuation experience.
[0011] 3. Lack of natural transition and interactive feedback in the continuation process: After the Q&A session ends, the system often abruptly cuts off from the end point, lacking a smooth transition and connecting language from the "Q&A / listening" state to the "explanation" state. This makes the interaction process mechanical and stiff, disrupting the smoothness of the guided tour narrative.
[0012] 4. Lack of rollback and fault tolerance mechanisms under abnormal operating conditions: The existing solution does not specify a reliable rollback strategy when abnormalities occur in the question-and-answer process (such as speech recognition failure, semantic understanding timeout, or user-initiated cancellation), resulting in insufficient availability and stability of the system under non-ideal operating conditions.
[0013] Therefore, how to provide a method for interrupting and resuming large-screen intelligent digital human guides, and improve the accuracy of multimodal data stream synchronization recovery, interactive continuity and natural transition effect in interrupted and resumed scenarios, as well as the fault tolerance and robustness of the system under abnormal working conditions, has become an urgent technical problem to be solved. Summary of the Invention
[0014] The technical problem to be solved by the present invention is to provide a method for interrupting and resuming the presentation of a large-screen intelligent digital human guide, thereby improving the accuracy of multimodal data stream synchronization recovery, the continuity of interaction and the natural transition effect of the presentation in the interruption and resumption scenario, as well as the fault tolerance and robustness of the system under abnormal working conditions.
[0015] This invention provides a method for interrupting and resuming narration in a large-screen intelligent digital human guided tour, including: Step S1: Monitor for interruption signals while the guided tour is playing; Step S2: In response to the detection of a valid interruption signal, pause the voice playback, digital human animation playback, and multimedia playback; Step S3: Generate and store a navigation context snapshot. The navigation context snapshot shall include at least the current navigation node identifier, the current explanatory sentence index, the playback duration within the current sentence, the global playback offset of the digital human-driven script, and the playback status of the associated navigation display materials. Step S4: Enter the question-and-answer state, obtain the question information and generate the corresponding response information, and play the response information; Step S5: After the Q&A session ends, read the navigation context snapshot, generate a continuation transition statement, and play it. Step S6: Based on the breakpoint information in the navigation context snapshot, synchronously resume the voice playback, the digital human animation playback, and the multimedia playback to continue the navigation playback from the breakpoint.
[0016] Furthermore, in step S1, the step of monitoring the interruption signal includes at least one of the following conditions: Condition A: The text obtained through automatic speech recognition contains interrogative semantics, and the speech energy is higher than the preset multiple of the environmental noise baseline; The judgment of the question semantics is based on a preset question lexicon and syntactic structure analysis. The preset multiple is 1.5 to 3.0 times the environmental noise baseline, and the sampling window of the speech energy is 200ms to 500ms to avoid false triggering by instantaneous noise. Condition B: A preset control in the interactive area is detected to be triggered; The preset control includes physical buttons, touch screen buttons, or air gesture recognition areas, and when the control is triggered, it is accompanied by tactile or visual feedback signals to confirm the interruption intention. Condition C: A preset wake word was detected; The wake word is at least one custom keyword, which is identified by a locally deployed keyword wake-up model. After the model identifies the wake word, it outputs a confidence score. Only when the confidence score is higher than a preset threshold is it determined to be a valid interruption signal. Condition D: A visitor is detected to be continuously staring at the digital human area for more than a threshold duration; The gaze detection is based on the fusion of the face orientation and gaze direction captured by the camera. The threshold duration is 1.5 seconds to 3.0 seconds. When the visitor's gaze direction deviates from the digital human area by more than a preset angle, the gaze timer is reset to prevent accidental triggering.
[0017] Furthermore, in step S3, the navigation context snapshot also includes a snapshot identifier and a snapshot generation timestamp.
[0018] Furthermore, in the navigation context snapshot, the playback status of the associated navigation display material includes at least one of the following: whether the image has been displayed, the video playback position, and whether the video is in a loop playback state; The video playback position record is a triplet data structure containing a video stream timestamp, a video frame index, and an audio track level value, which is used to achieve precise video frame-level positioning during recovery. Furthermore, the navigation context snapshot also includes a playback speed indicator and a subtitle track indicator corresponding to the associated navigation display material. The playback speed indicator is used to maintain the original playback rate of the material during restoration, and the subtitle track indicator is used to reactivate the corresponding dynamic subtitle track during restoration. In step S6, when restoring the multimedia playback based on the navigation context snapshot, the method further includes: performing frame-level positioning of the video player based on the triplet data structure, restoring the video playback rate based on the playback speed identifier, restoring the subtitle display based on the subtitle track identifier, and controlling the display or hiding of the image display layer based on whether the image has been displayed.
[0019] Furthermore, in step S4, if the question-and-answer session times out, voice recognition fails, or the user actively cancels the question-and-answer session, the playback of the response information is skipped, and the process proceeds directly to step S5.
[0020] Furthermore, in step S5, the transition statement is selected from a preset template library, and the digital human is driven to play a transition animation to complete the switch from a question-and-answer posture to a lecturing posture.
[0021] Furthermore, step S6 specifically includes: Step S61: In response to the end of the question and answer signal, load the navigation context snapshot, parse and verify the integrity and validity of the global playback offset of the digital human driven script, the current explanatory sentence index, the playback duration within the sentence, and the playback status of the associated navigation display materials; Step S62: Based on the global playback offset of the digital human driving script, send a preloading instruction to the animation rendering pipeline. The preloading instruction instructs the animation rendering pipeline to pre-calculate and cache the driving frame data after the global playback offset of the digital human driving script within a preset number of frame periods, so as to eliminate the rendering delay at the beginning of the playback recovery stage. Step S63: Based on the current explanatory sentence index and the playback duration within the sentence, locate the corresponding sampling point of the speech stream within the sentence using the speech decoder, and fill the audio output buffer with the speech data after the location point using a silent or low-volume pre-decoding method. Step S64: Based on the playback status of the associated navigation display materials, restore the playback progress of the image display layer, video player and dynamic subtitle track respectively. For video playback, restore the video playback speed, audio track level and subtitle track selection status. Step S65: After the voice decoder, the animation rendering pipeline, and the multimedia player have all completed positioning and preloading, a synchronization trigger signal is generated based on the same global time base. The synchronization trigger signal is used to start simultaneously: Starting from the global playback offset of the digital human driving script, the animation frame sequence of lip movements, facial expressions, and gestures is restored for playback; Starting from the speech sampling points corresponding to the duration already played within the sentence, restore the audio output; Starting from the progress position corresponding to the playback status of the associated guide display materials, the multimedia display is restored; Step S66: Within the first preset monitoring window after the synchronous recovery is started, continuously monitor the time offset between the current playback frame of the digital human driving script and the current output sampling point of the voice decoder, and determine whether the time offset is within the preset tolerance range. Step S67: If the time offset exceeds the preset tolerance range, a dynamic correction operation is triggered: calculate the actual offset between the digital human driving script and the voice decoder, and adjust the playback progress of the digital human driving script by adjusting the playback rate or frame jump compensation based on the current playback position of the voice decoder, so that the two converge back to the preset tolerance range within the second preset monitoring window. Step S68: If the digital human driving script fails to recover or the driving frame data verification fails, switch to the downgrade recovery mode: pause the digital human animation playback, maintain the synchronous recovery of the voice playback and the multimedia playback, and overlay a preset static placeholder image or a preset waiting prompt message in the digital human display area. At the same time, continuously attempt to reload the digital human driving script in the background until the loading is successful or the navigation segment ends.
[0022] Furthermore, the method also includes: Step S7: After the recovery is complete, clear or archive the navigation context snapshot and continue the current navigation until the end of the navigation segment.
[0023] Furthermore, in step S2, the operations of pausing voice playback, digital human animation playback, and multimedia playback are completed within the same control cycle as the operation of generating the navigation context snapshot, in order to prevent breakpoint drift.
[0024] Furthermore, in step S4, during the question-and-answer state, the navigation context snapshot is in a read-only state and cannot be rewritten by the navigation progress.
[0025] The advantages of this invention are: 1. Regarding synchronization recovery accuracy, its core lies in capturing a multi-dimensional contextual snapshot during interruption, including the playback duration within a sentence, the global offset of the digital human-driven script, and the video frame-level triplet state. During recovery, it coordinates the synchronous launch of voice, animation, and multimedia using a unified global time base. Subsequently, it eliminates minute offsets in real time through closed-loop monitoring and dynamic correction mechanisms (such as rate adjustment or frame jump compensation), thereby ensuring precise alignment of "sound, image, and person" at the frame level. Regarding interactive coherence and natural transition, this method does not abruptly cut off after the question-and-answer session ends. Instead, it calls preset transition statements and drives the digital human to transition from a question-and-answer posture to a lecturing posture. The method features animated transitions and skip logic for situations such as question-and-answer timeouts or recognition failures, ensuring the main flow is not interrupted by abnormal pauses and guaranteeing smooth narrative transitions. Regarding robustness in handling abnormal conditions, the method includes a robust degradation and recovery mode. When the digital human-driven script fails to load or fails verification, the system prioritizes ensuring the core playback functions of voice and multimedia visuals remain unaffected, displays a static placeholder prompt in the digital human area, and continuously attempts to reload the script in the background. This maintains the availability and stability of the guided tour service even under less than ideal conditions, ultimately ensuring the entire interrupted and resumed narration process is highly accurate, natural, and reliable.
[0026] 2. By constructing multi-dimensional breakpoint snapshots that include guide node identifiers, current narration sentence indexes, playback duration within a sentence, global playback offset of the digital human-driven script, and playback status of associated guide display materials (specifically including image display identifiers, triplet data structures for video playback positions, playback speed identifiers, and subtitle track identifiers), the breakpoint information is accurate down to the sentence level and animation-driven frame level. It also covers fine-grained states such as video frames, audio track levels, and subtitle tracks, providing a complete and accurate data foundation for subsequent continuation of the narration. This ensures that the entire playback progress before the interruption can be accurately restored during recovery, avoiding speculation or near-synchronization caused by missing information, and fundamentally guaranteeing the accuracy of multimodal content connection.
[0027] 3. A rigorous collaborative synchronization and dynamic correction strategy was adopted during the recovery phase: First, preloading instructions were sent to the animation rendering pipeline to eliminate rendering latency; the speech decoder was positioned to the corresponding sampling point within the sentence and the audio buffer was pre-filled; frame-level positioning of the video player was performed based on the triplet data structure, and playback speed and subtitle track were restored; then, speech, animation, and multimedia were simultaneously triggered based on a unified global time reference; after recovery started, the time offset between the digital human driving script and the speech decoder was continuously monitored, and when the tolerance was exceeded, the animation playback execution rate was adjusted or frame jump compensation was performed based on the speech playback position, so that the two could quickly reconverge. This mechanism not only ensured accurate synchronization at the moment of recovery, but also had the ability to automatically correct deviations during operation, and could maintain the consistency and coordination of multimodal data streams over a long period of time, ensuring that the digital human's lip movements, expressions, gestures, speech output, and screen content were always closely coordinated.
[0028] 4. A robust built-in exception handling and degradation recovery mechanism is in place: When a timeout occurs during the Q&A session, voice recognition fails, or the user actively cancels, the system skips the response playback and directly proceeds to the next step, avoiding waiting or status confusion due to Q&A anomalies. Simultaneously, when the digital human driving script fails to recover or the driving frame data verification fails, the system automatically switches to degradation mode, pausing digital human animation playback while maintaining synchronized recovery of voice and multimedia playback. A preset static placeholder image or waiting prompt message is displayed in the digital human area, and the background continuously attempts to reload the driving script until successful. This dual fault-tolerant design ensures uninterrupted navigation services even when faced with real-world engineering uncertainties such as unavailable voice services and abnormal resource loading, significantly improving the system's availability and stability in complex operating environments.
[0029] 5. Before resuming playback after the Q&A session, a natural transition is introduced: a continuation transition statement that semantically connects with the current tour context is selected from a preset template library, and the digital human plays a transition animation switching from a Q&A posture to a narration posture. This design provides clear interactive feedback to the user on both the cognitive and visual levels, informing them that the Q&A session has ended and the narration is about to continue. This effectively avoids the mechanical discontinuity caused by abruptly cutting off the breakpoint, making the entire interruption and continuation process smoother and more natural, enhancing the smoothness of the tour narrative and the immersive user experience.
[0030] 6. At the system architecture level, the atomicity and consistency of breakpoint recording are ensured: Pausing audio playback, digital human animation playback, and multimedia playback are completed within the same control cycle as generating the tour context snapshot, eliminating any time gap between the pause command and the snapshot recording, thus completely preventing breakpoint drift. Simultaneously, during the question-and-answer session, the tour context snapshot is set to read-only, prohibiting the tour progress from rewriting it, preventing the background progress from contaminating the saved breakpoint state. These two mechanisms together guarantee the purity and reliability of breakpoint snapshots, providing a solid and tamper-proof state benchmark for subsequent accurate recovery. Attached Figure Description
[0031] The present invention will be further described below with reference to the accompanying drawings and embodiments.
[0032] Figure 1 This is a flowchart of a method for interrupting and resuming narration in a large-screen intelligent digital human guide according to the present invention. Detailed Implementation
[0033] The overall approach of the technical solution in this application is as follows: By constructing a full-link processing framework of "real-time snapshot capture—multi-channel collaborative preloading—global synchronous triggering—dynamic closed-loop correction—anomaly degradation and fault tolerance," the system systematically solves the problems of multimodal data flow misalignment and interaction fragmentation faced by large-screen AI digital human guides in interrupted and resumed scenarios. Specifically, within the same control cycle where the interruption occurs, voice, animation, and multimedia playback are simultaneously paused, and a multi-dimensional context snapshot covering the playback progress within a sentence, the global offset of the driving script, and the triplet state of the video frame is generated, thereby providing an accurate and tamper-proof state benchmark for subsequent recovery. After the question and answer session ends, instead of a direct hard cut, transitional statements are played first, and the digital human is driven to complete a posture switch to create a natural interactive transition. Next, based on the snapshot information, the animation rendering pipeline, voice decoder, and multimedia player are preloaded and frame-level positioned respectively. Then, the restoration of the three playbacks is triggered simultaneously with a unified global time base to ensure that the "sound, picture, and person" are aligned at the frame level at the moment of startup. In the first monitoring window after restoration, the time offset between animation and voice is continuously monitored. Once the tolerance is exceeded, the rate adjustment or frame jump compensation is performed based on the voice to make the two converge to a synchronized state quickly. At the same time, for non-ideal working conditions such as question and answer timeout, recognition failure, or abnormal loading of driver scripts, skip logic and degraded recovery mode are preset to prioritize the continuity of voice and picture playback and continuously attempt to restore animation in the background, so as to maintain the high availability and stability of the tour guide service even in complex engineering environments.
[0034] In one optional implementation, this method can be deployed on an industrial control host running a Windows or Linux operating system. This host is equipped with at least one independent graphics processing unit (GPU) for real-time rendering of the digital human animation, and at least one audio codec for decoding and outputting the audio stream. The host connects to a large-screen display device via HDMI or DisplayPort and to a microphone array via USB or I²S. Each functional module adopts a modular design; the audio decoder, animation rendering pipeline, and multimedia player run in independent threads or processes, synchronizing state and exchanging data through shared memory or inter-process communication (IPC). Upon system startup, the digital human driving script, audio resource package, and multimedia material list are pre-loaded into memory, and a double-buffering mechanism is used to maintain the playback state, ensuring immediate response in case of interruption. It should be understood that the above hardware and software configurations are merely illustrative. Those skilled in the art can choose equivalent alternatives based on actual product requirements, such as using an embedded Linux system with an ARM architecture processor and Mali series graphics acceleration units, without affecting the implementation of the core method of this invention.
[0035] Please refer to Figure 1As shown, a preferred embodiment of the present invention, a method for interrupting and resuming narration in a large-screen intelligent digital human guide, includes: Step S1: Monitor for interruption signals while the guided tour is playing; In this step, the system captures the user's interruption intent in real time by monitoring multiple interaction channels in parallel. Specifically, it includes the following four trigger conditions, and the system can enable one or more combinations of these conditions according to the actual product form and scenario requirements: For condition A (based on speech semantics and energy detection), the system collects ambient audio streams via a microphone array. On one hand, it runs a locally deployed Automatic Speech Recognition (ASR) engine to transcribe the user's speech into text in real time. Based on a pre-defined interrogative vocabulary (e.g., including interrogative pronouns and modal particles such as "what," "when," "why," and "how") combined with a syntactic structure analysis model, it determines whether the current text contains interrogative intent. On the other hand, the system simultaneously calculates the average energy value of the speech signal within a sliding window of 200 to 500 milliseconds and compares it with a pre-learned ambient noise baseline. Only when the semantics are determined to be an interrogative sentence and the speech energy exceeds the noise baseline by 1.5 to 3.0 times is the signal marked as a potential interruption request. This dual-condition joint determination mechanism effectively filters out sudden noise or irrelevant human voices in the exhibition hall environment, reducing the false trigger rate.
[0036] For condition B (triggered by a preset control), the system continuously monitors various input devices in the interaction area, including but not limited to physical buttons, virtual buttons on the touchscreen, or air gesture areas based on depth camera recognition. Once a control is detected to be triggered, the system immediately generates tactile feedback (such as vibration) or visual feedback (such as button highlighting or flashing) to clearly confirm to the user that their interruption command has been received by the system, thereby avoiding repeated triggering.
[0037] For condition C (based on wake word detection), the system loads a locally trained keyword wake-up model. This model can decode the user's speech in real time and output a confidence score for a preset wake word (e.g., "Hello Xiaozhi" or "I want to ask a question"). The system only confirms that a valid interruption signal has been detected when the confidence score is higher than a preset threshold (e.g., 0.8). Local deployment ensures fast response times and avoids the impact of network latency on the real-time performance of the interaction.
[0038] For condition D (based on gaze duration detection), the system captures real-time facial images of visitors using a high-definition camera and determines whether the visitor is currently gazing at the digital human display area based on a face detection algorithm and a gaze estimation model. When the determination result is a gaze state, the system starts a gaze timer to accumulate the time. Once the continuous gaze duration reaches a preset threshold between 1.5 seconds and 3.0 seconds (e.g., set to 2 seconds), it is considered a valid interruption signal. It should be noted that if, during the timing process, the visitor's gaze is detected to deviate from the digital human area by more than a preset angle (e.g., a horizontal or vertical deviation of more than 15 degrees), the gaze timer is immediately reset to prevent false triggering due to brief gaze shifts.
[0039] Step S2: In response to the detection of a valid interruption signal, pause the voice playback, digital human animation playback, and multimedia playback; Step S3: Generate and store a navigation context snapshot. The navigation context snapshot includes at least the current navigation node identifier (node_id), the current narration sentence index (sentence_index), the playback duration within the current sentence (sentence_elapsed_ms), the global playback offset of the digital human-driven script (script_offset_ms), and the playback status of the associated navigation display material (media_progress). In this step, to ensure the atomicity and consistency of the captured breakpoint states, the system executes playback pause and snapshot capture operations in parallel within the same control cycle. Specifically, within the same interrupt service routine or high-priority task processing function, the system simultaneously issues pause commands to the voice decoder, animation rendering engine, and multimedia playback controller, and immediately initiates the context snapshot acquisition process.
[0040] The generated navigation context snapshot is a structured data object that contains at least the following key fields: (1) snapshot_id: A unique identifier for a snapshot, used to distinguish snapshots generated at different points in time; (2) node_id: The identifier of the current guide node, used to indicate the exhibition area or topic of explanation where the user is located; (3) sentence_index: The global index of the currently playing explanatory sentence; (4) sentence_elapsed_ms: The number of milliseconds that have elapsed in the current sentence, accurate to the millisecond level; (5) script_offset_ms: Global playback offset of the digital human driving script. This offset is used to index the keyframe data of the digital human's lip movements, facial expressions and body movements. (6) media_progress: The playback status of the associated navigation display materials. This field is a nested object used to record the real-time progress of the image, video and subtitle tracks respectively.
[0041] To support frame-level precision recovery after the Q&A session, the `media_progress` field for video footage contains a triplet data structure, specifically consisting of a video stream timestamp, a video frame index, and an audio track level value. The video stream timestamp is used to synchronize video progress, the video frame index is used for rapid keyframe location, and the audio track level value is used to assist in controlling the smoothness of audio fade-in effects during recovery. Additionally, the snapshot includes a playback speed indicator and a subtitle track indicator; the former records the video playback rate (e.g., 1.0x or 1.5x), and the latter marks the currently active subtitle stream (e.g., Chinese or English subtitles). For image materials, a boolean indicator records whether the material has been displayed.
[0042] In addition, the snapshot can also include a created_at timestamp to record the specific time the snapshot was generated, which facilitates subsequent log tracking and snapshot lifecycle management.
[0043] Example of a navigation context snapshot: {"snapshot_id":"snap_001","node_id":"zone_ancient","sentence_index":2,"sentence_elapsed_ms":1600,"script_offset_ms":4800,"media_progress":{"img_bronze":"shown","vid_foundry":"not_started"}} Note: script_offset_ms is used to restore the timeline of AI digital lip movements, expressions, and gestures; media_progress is used to restore the display progress of exhibit images / videos. Both must be saved together to avoid misalignment of "sound, image, and person" when continuing the presentation.
[0044] Step S4: Enter the question-and-answer state, obtain the question information and generate the corresponding response information, and play the response information; In this step, after entering the question-and-answer mode, the system first pauses the playback of the background music to reduce environmental interference and activates the microphone array's beamforming algorithm to focus the sound pickup direction on the area where visitors are standing in front of the large screen. The system also starts a question-and-answer timeout timer (the default setting is 15 seconds, which can be configured according to the actual flow of people in the exhibition hall).
[0045] To obtain the question information, the system sends the recognized text returned by the ASR engine to the Natural Language Understanding (NLU) module in real time. The NLU module first performs intent classification to determine whether the user's question belongs to a preset high-frequency question (such as "What era is this artifact from?" or "Who is the author?") or an open-ended exploratory question. For questions that match the preset FAQ library, the system directly retrieves the corresponding answer text from the local knowledge base; for open-ended questions that do not match the FAQ library, the system constructs a prompt containing contextual information about the current exhibit and calls the locally deployed or cloud-based Large Language Model (LLM) interface to generate the response content. To ensure timely response, the LLM call adopts an asynchronous non-blocking approach and sets a separate generation timeout threshold (e.g., 3 seconds). If the timeout occurs, it automatically degrades to a general response (e.g., "I don't have more information about this question at the moment, let's continue our visit").
[0046] When playing the response information, the system uses a text-to-speech (TTS) engine to synthesize the response text into speech data and drives the digital human to play the corresponding lip movements and body animations. During the response playback, the system continuously monitors the user's voice input to support follow-up questions; if new user voice input is detected within a preset time window (e.g., 2 seconds) after the response playback ends, the system automatically continues the question-and-answer state without triggering the continuation process.
[0047] During the Q&A session, the tour context snapshot saved in step S3 is marked as read-only (is_readonly=true) at the database level. Any write operation from the tour progress update thread is intercepted, and the system records a warning log so that maintenance personnel can check for any concurrent logic conflicts.
[0048] Question-and-answer processing can use a local FAQ (Frequently Asked Questions) database for matching, or call a Large Language Model (LLM) to generate responses.
[0049] Step S5: After the Q&A session ends, read the navigation context snapshot, generate a continuation transition statement (which can be selected from the template library, such as "Okay, I'll continue to introduce it to you.") and play it; Step S6: Based on the breakpoint information in the navigation context snapshot, synchronously resume the voice playback, the digital human animation playback, and the multimedia playback to continue the navigation playback from the breakpoint.
[0050] In step S1, the step of monitoring the interruption signal includes at least one of the following conditions: Condition A: The text obtained through Automatic Speech Recognition (ASR) contains interrogative semantics and the speech energy is higher than the preset multiple of the ambient noise baseline; The judgment of the question semantics is based on a preset question lexicon and syntactic structure analysis. The preset multiple is 1.5 to 3.0 times the environmental noise baseline, and the sampling window of the speech energy is 200ms to 500ms to avoid false triggering by instantaneous noise. Condition B: A preset control in the interactive area is detected to be triggered; The preset control includes physical buttons, touch screen buttons, or air gesture recognition areas, and when the control is triggered, it is accompanied by tactile or visual feedback signals to confirm the interruption intention. Condition C: A preset wake word was detected; The wake word is at least one custom keyword, which is identified by a locally deployed keyword wake-up model. After the model identifies the wake word, it outputs a confidence score. Only when the confidence score is higher than a preset threshold is it determined to be a valid interruption signal. Condition D: A visitor is detected to be continuously staring at the digital human area for more than a threshold duration; The gaze detection is based on the fusion of the face orientation and gaze direction captured by the camera. The threshold duration is 1.5 seconds to 3.0 seconds. When the visitor's gaze direction deviates from the digital human area by more than a preset angle, the gaze timer is reset to prevent accidental triggering.
[0051] In step S3, the navigation context snapshot also includes a snapshot identifier and a snapshot generation timestamp (created_at).
[0052] In the navigation context snapshot, the playback status of the associated navigation display material includes at least one of the following: whether the image has been displayed, the video playback position, and whether the video is in a loop playback state; The video playback position record is a triplet data structure containing a video stream timestamp, a video frame index, and an audio track level value, which is used to achieve precise video frame-level positioning during recovery. Furthermore, the navigation context snapshot also includes a playback speed indicator and a subtitle track indicator corresponding to the associated navigation display material. The playback speed indicator is used to maintain the original playback rate of the material during restoration, and the subtitle track indicator is used to reactivate the corresponding dynamic subtitle track during restoration. In step S6, when restoring the multimedia playback based on the navigation context snapshot, the method further includes: performing frame-level positioning of the video player based on the triplet data structure, restoring the video playback rate based on the playback speed identifier, restoring the subtitle display based on the subtitle track identifier, and controlling the display or hiding of the image display layer based on whether the image has been displayed.
[0053] In step S4, if the question-and-answer session times out, voice recognition fails, or the user actively cancels the question-and-answer session, the playback of the response information is skipped, and the process proceeds directly to step S5.
[0054] In step S5, the transition statement is selected from a preset template library and the digital human is driven to play a transition animation to complete the switch from a question-and-answer posture to a lecturing posture.
[0055] Step S6 specifically includes: Step S61: In response to the end of the question and answer signal, load the navigation context snapshot, parse and verify the integrity and validity of the global playback offset of the digital human driven script, the current explanatory sentence index, the playback duration within the sentence, and the playback status of the associated navigation display materials; After the Q&A session ends, the system first checks whether it is in a downgraded state or a user-initiated cancellation state. If it ends normally, the snapshot loading process is triggered. During loading, the system first reads the snapshot object from the memory cache. If the cache is not hit, it loads it from the persistent storage. The specific logic for verifying integrity and validity includes: (1) checking whether each necessary field is empty or has a default value; (2) verifying whether script_offset_ms is within the total duration of the driving script; (3) verifying whether sentence_index corresponds to a valid sentence resource; (4) verifying whether each video ID in media_progress is still in the material list. If any verification fails, the system discards the snapshot, records the error log, and triggers a safety rollback strategy—replaying from the starting position of the current navigation node.
[0056] Step S62: Based on the global playback offset of the digital human driving script, send a preloading instruction to the animation rendering pipeline. The preloading instruction instructs the animation rendering pipeline to pre-calculate and cache the driving frame data after the global playback offset of the digital human driving script within a preset number of frame periods, so as to eliminate the rendering delay at the beginning of the playback recovery stage. In its implementation, the preloading instruction is achieved by inserting a high-priority command into the command queue of the animation rendering pipeline. The command parameters include the starting offset `script_offset_ms` and the number of preloaded frames (default 30 frames, corresponding to 1 second @ 30fps). Upon receiving this command, the animation rendering pipeline starts a background worker thread that, starting from the corresponding offset position in the driver script file, sequentially calculates the bone transformation matrix, blendshapeweights, and gesture parameters for each frame, storing the results in a ring buffer. When the buffer is full or the number of preloaded frames reaches the preset limit, the worker thread automatically suspends, waiting for a synchronization trigger signal from the main thread. This preloading mechanism ensures that the animation rendering pipeline does not need to read and calculate the driver frames from disk again upon resuming playback, thus keeping the first frame rendering latency within 16.7 milliseconds (one frame).
[0057] Step S63: Based on the current explanatory sentence index and the playback duration within the sentence, locate the corresponding sampling point of the speech stream within the sentence using the speech decoder, and fill the audio output buffer with the speech data after the location point using a silent or low-volume pre-decoding method. The specific implementation of locating the corresponding sampling point by the speech decoder is as follows: The system obtains the starting byte offset of the audio data corresponding to the current narration sentence in the PCM (Pulse Code Modulation) stream from the speech resource manager based on the current narration sentence index. Then, combined with the playback duration within the sentence and the audio encoding format (e.g., sampling rate 16000Hz, 16-bit quantization, mono), the precise byte offset position is calculated: byte_offset = sentence_start_byte + (elapsed_ms * sample_rate * bit_depth * channels / 8000). The decoder calls the seek() interface to jump to this position and fills the audio output buffer with the data of the subsequent 512 sampling points using a silent pre-decoding method (i.e., output audio data multiplied by an attenuation factor of 0.01). This ensures that the audio output can start immediately when recovery is triggered, without distortion or blanking due to an empty buffer.
[0058] Step S64: Based on the playback status of the associated navigation display materials, restore the playback progress of the image display layer, video player and dynamic subtitle track respectively. For video playback, restore the video playback speed, audio track level and subtitle track selection status. When resuming video playback, the system first parses the triplet data structure in the snapshot. For the video stream timestamp, the system calls the video player's SeekToTimestamp() method for location; for the frame index, as an auxiliary verification method, after location is completed, the system reads the current actual frame number and compares it with the frame index recorded in the snapshot. If the difference exceeds a preset threshold (e.g., ±2 frames), a keyframe (I-frame) decoding refresh is forcibly executed; for the audio track level value, in the initial stage after restoration (within 200 milliseconds), the system performs an audio fade-in with this level value as the target to avoid sudden volume changes during restoration. The restoration of the subtitle track is achieved by sending a track switching command to the subtitle renderer. This command contains the subtitle track identifier recorded in the snapshot. The subtitle renderer reloads the corresponding subtitle file and jumps to the matching timestamp position based on this.
[0059] Step S65: After the voice decoder, the animation rendering pipeline, and the multimedia player have all completed positioning and preloading, a synchronization trigger signal is generated based on the same global time base. The synchronization trigger signal is used to start simultaneously: Starting from the global playback offset of the digital human driving script, the animation frame sequence of lip movements, facial expressions, and gestures is restored for playback; Starting from the speech sampling points corresponding to the duration already played within the sentence, restore the audio output; Starting from the progress position corresponding to the playback status of the associated guide display materials, the multimedia display is restored; The "same global time base" is implemented as a high-precision performance counter provided by the operating system (such as QueryPerformanceCounter in Windows or clock_gettime(CLOCK_MONOTONIC) in Linux). After confirming that the three modules (speech decoder, animation rendering pipeline, and multimedia player) are all ready, the system reads the current performance counter value, adds a preset safety lead time (e.g., 50 milliseconds) as the target startup time, and then writes this target time into the event waiting mechanism of each module. Each module waits in its own thread for the target time to arrive before simultaneously executing the startup operation. This collaborative triggering method based on a global time base, compared to sequential serial startup, can control the startup time deviation between the three modules within ±1 millisecond.
[0060] Step S66: Within the first preset monitoring window after the synchronous recovery is started, continuously monitor the time offset between the current playback frame of the digital human driving script and the current output sampling point of the voice decoder, and determine whether the time offset is within the preset tolerance range. Step S67: If the time offset exceeds the preset tolerance range, a dynamic correction operation is triggered: calculate the actual offset between the digital human driving script and the voice decoder, and adjust the playback progress of the digital human driving script by adjusting the playback rate or frame jump compensation based on the current playback position of the voice decoder, so that the two converge back to the preset tolerance range within the second preset monitoring window. The "first preset monitoring window" is set to the first 5 seconds after recovery, with a monitoring frequency of once every 100 milliseconds. The time offset is calculated as: delta = script_current_time - audio_current_time, in milliseconds. A positive value indicates that the animation is ahead of the audio, and a negative value indicates that the animation is behind the audio. The preset tolerance range is set to [-80ms, +80ms], which is based on the human eye's perception threshold for audio-visual synchronization (generally considered imperceptible within ±100ms) and retains a safety margin.
[0061] If delta is detected to be outside the tolerance range, the specific logic of the dynamic correction operation is as follows: When delta > 80ms (animation ahead), the system temporarily adjusts the animation playback rate to 0.9x, continuing until delta falls back to the tolerance range, and then restores it to 1.0x. If delta > 300ms (severely ahead), frame jump compensation is directly executed, jumping the animation playback position to audio_current_time + 50ms, and performing smooth interpolation transition within 0.5 seconds before and after the jump point (i.e., mixing the skeletal poses before and after the jump through linear interpolation to avoid abrupt changes in motion).
[0062] When delta < -80ms (animation lag), the system temporarily adjusts the animation playback rate to 1.1x and continues to run until delta recovers to the tolerance range, then restores it to 1.0x. If delta < -300ms (severe lag), the lagging drive frame is discarded directly, the animation playback position is quickly moved to audio_current_time - 50ms, and playback continues from that position.
[0063] The "second preset monitoring window" is set to be within 3 seconds after the dynamic correction is triggered. Within this window, the system monitors the convergence of the offset at a higher frequency of every 50 milliseconds. If delta still fails to converge to the tolerance range at the end of 3 seconds, the system will force a hard jump to reposition the animation playback position based on the voice playback position.
[0064] Step S68: If the digital human driving script fails to recover or the driving frame data verification fails, switch to the downgrade recovery mode: pause the digital human animation playback, maintain the synchronous recovery of the voice playback and the multimedia playback, and overlay a preset static placeholder image or a preset waiting prompt message in the digital human display area. At the same time, continuously attempt to reload the digital human driving script in the background until the loading is successful or the navigation segment ends.
[0065] In practical engineering applications, abnormal situations such as storage media failure, data transmission verification errors, or corrupted driver script files may occur, causing the system to be unable to load the digital human driver script or its keyframe data normally. Therefore, this method includes a comprehensive degradation and recovery strategy.
[0066] In this mode, the system will pause the digital human animation playback function, but the synchronous recovery process of voice playback and multimedia content display will continue according to the existing logic to ensure that the main information being explained is not affected. Simultaneously, the system will overlay a preset static placeholder image (such as a high-definition static image of the digital human) in the digital human display area or scroll at the top of the screen to display user-friendly waiting prompts such as "Digital human loading, please wait..." to explain the current status to the user and avoid confusion caused by a black screen or blank display.
[0067] During the degraded state, the system continuously attempts to reload the digital human-driven script in the background with a low priority and repeatedly verifies the loaded data. If the loading is successful, the system will attempt to seamlessly resume animation playback; if it fails until the end of the current tour segment, the system will close the degrade prompt at the end of the segment and proceed to the next normal tour flow to ensure that the overall tour service can continue to operate without complete paralysis.
[0068] The method further includes: Step S7: After the recovery is complete, clear or archive the navigation context snapshot and continue the current navigation until the end of the navigation segment.
[0069] Regarding snapshot lifecycle management, to ensure that memory and storage resources are not consumed indefinitely, the system presets the effective lifecycle of a snapshot when it is generated. When the navigation node corresponding to the snapshot finishes playing or the user actively exits the current navigation, the system automatically marks the snapshot as invalid and deletes it.
[0070] Specifically, during the question-and-answer phase, to prevent accidental modifications to the snapshot content by the current navigation background program or due to interference from concurrent tasks, the system sets the generated snapshot object to read-only mode. Any attempt to write new data to this snapshot (such as updating the playback progress) will be rejected by the operating system or runtime environment and logged as an exception. This mechanism ensures that the breakpoint information saved in the snapshot remains pure and uncontaminated by any external factors during the question-and-answer phase, thus providing an unbreakable data foundation for subsequent accurate recovery. After the recovery is complete and the playback resumes successfully, the snapshot will be archived to the historical log database for subsequent data analysis or exception backtracking.
[0071] In step S2, the operations of pausing voice playback, digital human animation playback, and multimedia playback are completed within the same control cycle as the operation of generating the navigation context snapshot, in order to prevent breakpoint drift.
[0072] In step S4, during the question-and-answer state, the navigation context snapshot is in a read-only state and cannot be rewritten by the navigation progress.
[0073] To more clearly illustrate the implementation effects of this invention, a complete interactive scenario example is given below: A provincial museum's "Bronze Ware Exhibition Hall" features a large-screen intelligent digital human guide system, named "Xiao Bo." When visitor Mr. Wang approaches the screen, the system automatically activates via a human body sensor, and "Xiao Bo" begins its explanation: "Welcome to the Bronze Ware Exhibition Hall. This exhibit displays bronze ritual vessels from the Western Zhou Dynasty, the most representative of which is this square ding with animal mask design..." When the explanation reached about 1.2 seconds into the third sentence (just after the phrase "beast-face pattern square ding"), Mr. Wang raised a question and asked, "What is this ding used for?" At this moment, the microphone array picked up Mr. Wang's voice, the ASR engine transcribed the speech into text, the semantic analysis module recognized "what it was used for" as a question, and the speech energy detection module determined that the current speech energy exceeded 2.0 times the environmental noise baseline. The system confirmed that a valid interruption signal had been captured.
[0074] The system immediately completes the following operations within the same control cycle (approximately 30 milliseconds): (1) Pauses the voice playback, digital human animation, and the Fangding image that is being displayed by "Xiao Bo"; (2) Generates a navigation context snapshot, recording: sentence_index=3, sentence_elapsed_ms=1200, script_offset_ms=5200, and media_progress records that the Fangding image has been displayed but the video has not yet played. Subsequently, the system enters a question-and-answer state, the background music automatically fades out, and "Xiao Bo" switches to a listening posture (leaning slightly forward with its eyes facing the questioner).
[0075] The system sends the question "What is this ding used for?" to the NLU module, which matches the FAQ entry for "Use of the square ding with animal face pattern," and returns the answer: "The square ding with animal face pattern was a ritual vessel used in sacrificial ceremonies during the Western Zhou Dynasty. It was usually used to hold meat offerings and was a symbol of power and status." The TTS engine synthesizes the answer into speech, and "Xiao Bo" plays the response in a gentle tone, accompanied by appropriate lip movements and gestures.
[0076] After the Q&A session, the system loads the previously saved snapshot, first playing the transitional phrase "Okay, let's continue looking at the details of this square ding," and then prompting "Xiao Bo" to switch from a listening posture back to a lecturing posture (arms naturally lowered, facing the large screen). Subsequently, the system synchronously restores the presentation based on the snapshot information: the animation rendering pipeline preloads the driver frame starting at 5200ms, the speech decoder locates the 1200ms position of the third sentence and pre-fills the buffer, and the multimedia player ensures the image of the square ding remains displayed. Once all three are ready, the playback resumes simultaneously based on the same global timeline, continuing the explanation from "About three thousand years ago…" (the unfinished part of the third sentence), achieving a seamless transition.
[0077] Throughout the process, Mr. Wang felt that "Xiao Bo" answered the question first after being interrupted, and then naturally said "Let's continue," and then continued from where he had been interrupted. Everything was smooth and natural, without any stuttering, frame skipping, or repetition, which was completely in line with the real conversational habits between people.
[0078] This example is merely a typical application scenario of the present invention. Those skilled in the art should understand that the present invention is also applicable to various public display scenarios such as exhibit guidance in science and technology museums, product introductions in corporate exhibition halls, and business guidance in government service halls. Its core methods and scope of protection are not limited by changes in the specific type of exhibits or exhibition hall environment.
[0079] While specific embodiments of the present invention have been described above, those skilled in the art should understand that the specific embodiments described are merely illustrative and not intended to limit the scope of the invention. Equivalent modifications and variations made by those skilled in the art in accordance with the spirit of the invention should be covered within the scope of protection of the claims of the present invention.
Claims
1. A method for interrupting and resuming narration in a large-screen intelligent digital human guide, characterized in that: include: Step S1: Monitor for interruption signals while the guided tour is playing; Step S2: In response to the detection of a valid interruption signal, pause the voice playback, digital human animation playback, and multimedia playback; Step S3: Generate and store a navigation context snapshot. The navigation context snapshot shall include at least the current navigation node identifier, the current explanatory sentence index, the playback duration within the current sentence, the global playback offset of the digital human-driven script, and the playback status of the associated navigation display materials. Step S4: Enter the question-and-answer state, obtain the question information and generate the corresponding response information, and play the response information; Step S5: After the Q&A session ends, read the navigation context snapshot, generate a continuation transition statement, and play it. Step S6: Based on the breakpoint information in the navigation context snapshot, synchronously resume the voice playback, the digital human animation playback, and the multimedia playback to continue the navigation playback from the breakpoint.
2. The method for interrupting and resuming narration in a large-screen intelligent digital human guide as described in claim 1, characterized in that, In step S1, the step of monitoring the interruption signal includes at least one of the following conditions: Condition A: The text obtained through automatic speech recognition contains interrogative semantics, and the speech energy is higher than the preset multiple of the environmental noise baseline; The judgment of the question semantics is based on a preset question lexicon and syntactic structure analysis. The preset multiple is 1.5 to 3.0 times the environmental noise baseline, and the sampling window of the speech energy is 200ms to 500ms to avoid false triggering by instantaneous noise. Condition B: A preset control in the interactive area is detected to be triggered; The preset control includes physical buttons, touch screen buttons, or air gesture recognition areas, and when the control is triggered, it is accompanied by tactile or visual feedback signals to confirm the interruption intention. Condition C: A preset wake word was detected; The wake word is at least one custom keyword, which is identified by a locally deployed keyword wake-up model. After the model identifies the wake word, it outputs a confidence score. Only when the confidence score is higher than a preset threshold is it determined to be a valid interruption signal. Condition D: A visitor is detected to be continuously staring at the digital human area for more than a threshold duration; The gaze detection is based on the fusion of the face orientation and gaze direction captured by the camera. The threshold duration is 1.5 seconds to 3.0 seconds. When the visitor's gaze direction deviates from the digital human area by more than a preset angle, the gaze timer is reset to prevent accidental triggering.
3. The method for interrupting and resuming narration in a large-screen intelligent digital human guide as described in claim 1, characterized in that, In step S3, the navigation context snapshot also includes a snapshot identifier and a snapshot generation timestamp.
4. The method for interrupting and resuming narration in a large-screen intelligent digital human guide as described in claim 3, characterized in that, In the navigation context snapshot, the playback status of the associated navigation display material includes at least one of the following: whether the image has been displayed, the video playback position, and whether the video is in a loop playback state; The video playback position record is a triplet data structure containing a video stream timestamp, a video frame index, and an audio track level value, which is used to achieve precise video frame-level positioning during recovery. Furthermore, the navigation context snapshot also includes a playback speed indicator and a subtitle track indicator corresponding to the associated navigation display material. The playback speed indicator is used to maintain the original playback rate of the material during restoration, and the subtitle track indicator is used to reactivate the corresponding dynamic subtitle track during restoration. In step S6, when restoring the multimedia playback based on the navigation context snapshot, the method further includes: performing frame-level positioning of the video player based on the triplet data structure, restoring the video playback rate based on the playback speed identifier, restoring the subtitle display based on the subtitle track identifier, and controlling the display or hiding of the image display layer based on whether the image has been displayed.
5. The method for interrupting and resuming narration in a large-screen intelligent digital human guide as described in claim 1, characterized in that, In step S4, if the question-and-answer session times out, voice recognition fails, or the user actively cancels the question-and-answer session, the playback of the response information is skipped, and the process proceeds directly to step S5.
6. The method for interrupting and resuming narration in a large-screen intelligent digital human guide as described in claim 1, characterized in that, In step S5, the transition statement is selected from a preset template library and the digital human is driven to play a transition animation to complete the switch from a question-and-answer posture to a lecturing posture.
7. The method for interrupting and resuming narration in a large-screen intelligent digital human guide as described in claim 1, characterized in that, Step S6 specifically includes: Step S61: In response to the end signal of the question and answer, load the navigation context snapshot, parse and verify the integrity and validity of the global playback offset of the digital human driven script, the index of the current explanatory sentence, the playback duration within the sentence, and the playback status of the associated navigation display materials; Step S62: Based on the global playback offset of the digital human driving script, send a preloading instruction to the animation rendering pipeline. The preloading instruction instructs the animation rendering pipeline to pre-calculate and cache the driving frame data after the global playback offset of the digital human driving script within a preset number of frame periods, so as to eliminate the rendering delay at the beginning of the playback recovery stage. Step S63: Based on the current explanatory sentence index and the playback duration within the sentence, locate the corresponding sampling point of the speech decoder within the sentence, and fill the audio output buffer with the speech data after the location point using a silent or low-volume pre-decoding method. Step S64: Based on the playback status of the associated navigation display materials, restore the playback progress of the image display layer, video player and dynamic subtitle track respectively. For video playback, restore the video playback speed, audio track level and subtitle track selection status. Step S65: After the voice decoder, the animation rendering pipeline, and the multimedia player have all completed positioning and preloading, a synchronization trigger signal is generated based on the same global time base. The synchronization trigger signal is used to start simultaneously: Starting from the global playback offset of the digital human driving script, the animation frame sequence of lip movements, facial expressions, and gestures is restored for playback; Starting from the speech sampling points corresponding to the duration already played within the sentence, restore the audio output; Starting from the progress position corresponding to the playback status of the associated guide display materials, the multimedia display is restored; Step S66: Within the first preset monitoring window after the synchronous recovery is started, continuously monitor the time offset between the current playback frame of the digital human driving script and the current output sampling point of the voice decoder, and determine whether the time offset is within the preset tolerance range. Step S67: If the time offset exceeds the preset tolerance range, a dynamic correction operation is triggered: calculate the actual offset between the digital human driving script and the voice decoder, and adjust the playback progress of the digital human driving script by adjusting the playback rate or frame jump compensation based on the current playback position of the voice decoder, so that the two converge back to the preset tolerance range within the second preset monitoring window. Step S68: If the digital human driving script fails to recover or the driving frame data verification fails, switch to the downgrade recovery mode: pause the digital human animation playback, maintain the synchronous recovery of the voice playback and the multimedia playback, and overlay a preset static placeholder image or a preset waiting prompt message in the digital human display area. At the same time, continuously attempt to reload the digital human driving script in the background until the loading is successful or the navigation segment ends.
8. The method for interrupting and resuming narration in a large-screen intelligent digital human guide as described in claim 1, characterized in that, The method further includes: Step S7: After the recovery is complete, clear or archive the navigation context snapshot and continue the current navigation until the end of the navigation segment.
9. A method for interrupting and resuming narration in a large-screen intelligent digital human guided tour as described in any one of claims 1-8, characterized in that, In step S2, the operations of pausing voice playback, digital human animation playback, and multimedia playback are completed within the same control cycle as the operation of generating the navigation context snapshot, in order to prevent breakpoint drift.
10. A method for interrupting and resuming narration in a large-screen intelligent digital human guided tour as described in any one of claims 1-8, characterized in that, In step S4, during the question-and-answer state, the navigation context snapshot is in a read-only state and cannot be rewritten by the navigation progress.
