Intelligent exhibition hall multi-mode interactive digital human system and implementation method

Abstract

The invention relates to the technical field of computer data processing, and discloses an intelligent exhibition hall multi-modal interaction digital human system and an implementation method, which are used for solving the problem that multi-source data of multi-modal interaction lacks a unified clock and verifiable timestamp alignment mechanism in a traditional method. According to the method, a unified time domain and a time version are established on an edge side, terminal access is restrained, and an acquisition time mark, an access time mark and a serial number are written in data or a state; performing gating shunting according to a time version, performing de-duplication and out-of-order rearrangement based on a serial number and double time marks, and generating and solidifying a session window evidence index; locking a transaction window boundary according to the evidence index, generating a participation source list and a transaction number, establishing a fragment reference relationship and generating an alignment voucher; in the linkage stage, phase division issuing is carried out according to a preparation phase, an execution phase and a confirmation phase, backward reading verification is carried out according to an action sequence number, a transaction log is archived, and alignment, rechecking and playback of an interaction link are achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of computer data processing, and specifically provides a multi-modal interactive digital human system and implementation method for a smart exhibition hall. Background Art

[0002] With the development of smart exhibition halls and digital exhibition displays, the digital human interpretation and interaction system for audiences has gradually evolved from a single voice question-and-answer to a multi-modal interaction form that integrates voice, video, skeletal posture, and the status of exhibition hall equipment, so as to achieve functions such as audience intention recognition, content interpretation, exhibit triggering, and linkage of lighting, sound, and screen displays; in the prior art, for example, CN116430991A discloses a digital human interpretation and interaction implementation solution for exhibition hall scenarios, which collects audience interaction information through a terminal and drives the digital human to display and interpret; CN115167674A discloses an information processing and interaction method related to digital human multi-modal interaction, emphasizing the acquisition and processing of multi-modal interaction information to improve the interaction experience. The above solutions can support the interactive display of digital humans in exhibition halls to a certain extent, but there are still common key technical defects in the engineering deployment of real exhibition halls; Specifically, multi-modal interaction in exhibition halls often involves a variety of heterogeneous acquisition and control terminals, such as sound pickup and voice front-ends, cameras and skeletal recognition terminals, edge computing nodes, digital human rendering and playback terminals, as well as control devices such as lighting, sound, projection, and screen displays. These terminals are usually provided by different manufacturers and use different sampling frequencies, different system clocks, and different transmission links. Traditional methods mostly use independent timestamps or arrival times of each terminal as event markers, lacking a unified clock reference and a verifiable timestamp alignment mechanism, and also lacking effective calibration and consistency verification means for link delays such as acquisition delay, transmission jitter, and buffer queuing, resulting in difficulty in forming a consistent temporal correlation relationship between the same interaction event in the voice stream, video frames, skeletal key point sequences, and device status readback; thus, on the business side, problems such as inconsistent voice intentions and gesture directions, out-of-sync confirmation actions and question-and-answer outputs, misalignment or delay accumulation of exhibit triggering and lighting, sound, and linkage are likely to occur,进而 causing unstable interaction determination, unreliable linkage control, and difficult fault location, making it difficult to meet the comprehensive requirements of smart exhibition halls for real-time performance, accuracy, and traceability; Therefore, there is an urgent need for a technical solution that can establish a unified clock reference in a multi-modal interactive digital human system and achieve verifiable timestamp alignment to ensure the temporal consistency of interaction events among multi-source data, so as to achieve stable real-time determination and reliable linkage control. Summary of the Invention

[0003] Aiming at the deficiencies of the prior art, the present invention provides a multi-modal interactive digital human system and implementation method for a smart exhibition hall, which solves the problem of the lack of a unified clock and a verifiable timestamp alignment mechanism for multi-source data in multi-modal interaction in traditional methods.

[0004] To achieve the goal of ensuring the temporal consistency of interactive events across multiple data sources, as mentioned in the background section, this invention provides the following technical solution: A method for implementing a multimodal interactive digital human in a smart exhibition hall includes: S1: Establish a unified time domain on the edge of the exhibition hall, generate a time version identifier, and send registration parameters and alignment session identifiers to each acquisition terminal and linkage control terminal; S2: Each terminal completes time domain access and writes the acquisition time identifier, access time identifier, time version identifier, and segment sequence number into the output data or status; S3: At the edge, multi-source data is split according to time version identifier, and sorted in order according to segment number and dual time identifier to form a session window evidence index; S4: Initiate an interactive transaction based on the evidence index, determine the transaction window boundary and the list of participating sources, establish the transaction number and data reference relationship, and generate an aligned credential reference; S5: Generate a sequence of linked actions based on interactive transactions, and issue and read back the actions in phases according to the preparation phase, execution phase, and confirmation phase, and record the transaction log.

[0005] In a preferred embodiment, a unified time domain is established at the edge of the exhibition hall, a time version identifier is generated, and registration parameters and alignment session identifiers are sent to each acquisition terminal and linkage control terminal, including: At the edge, a unified time domain identifier and time version identifier are generated, and the old version failure boundary is set to form a session constraint set; Session constraint sets limit access phases, session state transitions, time field definitions, and mandatory message field verification points; On the edge side, register and catalog the voice acquisition terminal, video acquisition terminal, skeleton acquisition terminal and linkage control terminal according to the terminal template, and generate a registration parameter set; The registration parameter set defines the identity declaration, channel selection, session persistence and recovery rules, publishes the aligned session identifier, and registers the session registry.

[0006] In a preferred embodiment, each terminal completes time domain access and writes the acquisition time identifier, access time identifier, time version identifier, and segment sequence number into the output data or status, including: The edge side opens an access window, performs the application, verification, confirmation and receipt verification in sequence, and updates the terminal status to show that the session is active. The access request carries the alignment session identifier, time version identifier, terminal identifier and channel identifier. The acquisition end submits the fragment organization declaration, and the linkage end submits the readback field and readback trigger declaration. The terminal output writes the acquisition time identifier, access time identifier, time version identifier, and segment sequence number or action sequence number, and writes the sequence number reason mark. The edge side writes the access time identifier and isolates and registers the outputs that fail the access verification.

[0007] In a preferred embodiment, the edge side splits multi-source data according to time version identifiers, including: The edge side determines the set of valid versions based on the session constraint set and maintains a corresponding session buffer for each valid version. After the data is written to the access time identifier, it is routed to the corresponding buffer according to the time version identifier. Data of missing or invalid versions is isolated and version anomalies are registered.

[0008] When a version change occurs, a new session buffer is enabled, and the old session buffer is set to read-only collation and archiving status.

[0009] In a preferred embodiment, the evidence is organized sequentially according to segment number and dual time identifier to form a session window evidence index, including: The edge side maintains the status table of each modal sequence number, performs deduplication and random reordering based on the segment sequence number or action sequence number and in combination with the acquisition time identifier and access time identifier, checks the segment identifier and the range of the dual time identifier for duplicate segments and registers the sequence number reuse anomaly. Write missing placeholder references to the sequence number holes and record the boundary acceptance or reset. Generate cross-source alignment anchors according to window rules and solidify the evidence index. Assign immutable index identifiers and register the index registry and archive summary.

[0010] In a preferred embodiment, an interactive transaction is initiated based on the evidence index, and the transaction window boundaries and the list of participating sources are determined, including: The edge side receives trigger messages carrying an alignment session identifier, a time version identifier, a trigger source terminal identifier, a collection time identifier, and an access time identifier; On the edge side, the evidence index is selected from the index registration table according to the trigger access time identifier, and the transaction window boundary is locked. Based on the abnormal metadata of the evidence index, a list of participating sources is generated and the reference range of each participating source is determined.

[0011] In a preferred embodiment, establishing the relationship between transaction numbers and data references and generating aligned credential references includes: The edge side generates a transaction number and establishes a binding relationship between the transaction number and the multimodal segment reference. It registers the sequence number range and the collection time identifier range and access time identifier range of each segment, and registers the action sequence number field for the device status segment reference. The edge side generates an alignment credential reference and associates the alignment credential reference with the version identifier, session identifier, evidence index identifier, and terminal access status snapshot. Edge-side registration of transaction conflict rules based on the intersection relationship of device set identifiers.

[0012] In a preferred embodiment, generating a sequence of linked actions based on the interactive transaction includes: The edge side trims the set of controllable devices based on the transaction scope marker and the restricted source marker, generates the target device range marker, and generates a sequence of linked actions based on the target device range marker and assigns the action sequence number according to the monotonically increasing rule within the transaction. The edge side displays the grouping markers for the linkage action sequence registration equipment and the sequence view within each group.

[0013] In a preferred embodiment, the transaction is issued and read back in phases according to the preparation phase, execution phase, and confirmation phase, and a transaction log is recorded, including: The pre-phase reads the device status, generates a status snapshot, and initiates an interlock occupancy request. If the request fails, it is placed in a queued state, and a retry is triggered by an occupancy release event. The pre-phase reference time stamp is registered. The execution phase is grouped by action number to issue control commands and write access timestamps and phase markers. The confirmation phase is read back by action number to establish readback references and completes verification based on status snapshots, issued records and readback criteria. The transaction log registers phase records, exception pointers, alignment credential references, evidence index identifiers, completion markers and release of occupancy and updates the registration table.

[0014] On the other hand, the present invention provides a smart exhibition hall multimodal interactive digital human system, comprising: Time Domain and Session Publishing Module: Establishes a unified time domain and publishes aligned sessions on the edge side, completes terminal registration and cataloging and parameter distribution, and establishes a session registry to register permissions, status and readback criteria; Terminal access and field encapsulation module: used for terminal access and standardization, uniformly write time and serial number identifier, supplement access time on the edge side and isolate abnormal input; Version routing and evidence index construction module: It performs time-version-based flow control, combines fragment sequence number and dual time identifier to sort and deduplicate, generates session window evidence index and registers mapping for transaction reference; Interactive Transaction and Alignment Certificate Module: Selects evidence index and locks window based on trigger flag, generates participating source list and transaction number, establishes fragment reference and solidifies alignment certificate, and handles contract concurrent conflicts according to device set; Linked phase execution and transaction log module: Generates and numbers linked sequences under transaction constraints, issues back-read verification in phases, archives logs by transaction and associates vouchers and evidence indexes to achieve a replayable closed loop.

[0015] Compared with existing technologies, this invention provides a smart exhibition hall multimodal interactive digital human system and its implementation method, which has the following beneficial effects: 1. This invention, through access constraints of aligned sessions, terminal registration parameter configuration, and dual-time identifier encapsulation, enables voice segments, video frame segments, skeletal segments, and device status to have verifiable temporal references within the same session. At the edge, gating and routing are performed based on time version identifiers, and deduplication, reordering, and missing placeholder processing are completed using segment sequence numbers in conjunction with dual time identifiers, generating a referable session window evidence index to reduce the risk of cross-source misalignment in jitter and disconnection scenarios. Subsequently, the transaction window boundaries are solidified based on the evidence index, and a list of participating sources is generated, establishing a relationship between transaction numbers and segment references, while simultaneously generating aligned credential references, enabling subsequent... The judgment and linkage records can consistently point to the same window boundary and the same time caliber; the linkage phase organizes the actions into sequence numbers, and completes interlocking, grouping and distribution and back-read verification according to the preparation phase, execution phase and confirmation phase. Combined with the transaction log, the phase records, exception types and attribution pointers are structured and archived, thereby alleviating the problems of cross-modal association inconsistency, linkage instability and fault location difficulties caused by independent timestamps of multiple terminals, and forming a verifiable, replayable and auditable interactive linkage link. It solves the problem of lack of unified clock and verifiable timestamp alignment mechanism for multi-source data with multimodal interaction in traditional methods.

[0016] 2. This invention, by unifying the governance of time versions and session constraints, incorporates terminal access, channel usage, readback caliber, and session lifecycle into the same rule system, reducing the impact of differences in field semantics and state expression between heterogeneous terminals on the consistency of linkage semantics; using evidence indexes as the basis for transaction input, pre-fixing window boundaries, source integrity, and anomaly markers, ensuring that trigger information is only used to locate evidence windows, and that transaction boundaries and input ranges are locked by the index and can be verified; and by constraining the scope and coordinating queuing of device sets under transaction isolation and conflict reduction, reducing competition and miscontrol of concurrent transactions on the same device, enhancing control consistency and occupancy management controllability during cross-item and multi-session parallel operation, thereby improving the linkage determinism, operational controllability, and maintenance manageability in high-concurrency scenarios. Attached Figure Description

[0017] Figure 1 This is a flowchart of a method for realizing a multimodal interactive digital human in a smart exhibition hall according to the present invention; Figure 2 This is a schematic diagram of the structure of a smart exhibition hall multimodal interactive digital human system according to the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Example 1: Figure 1 A method for implementing multimodal interactive digital humans in smart exhibition halls is presented, including: S1: Establish a unified time domain on the edge of the exhibition hall, generate a time version identifier, and send registration parameters and alignment session identifiers to each acquisition terminal and linkage control terminal; S2: Each terminal completes time domain access and writes the acquisition time identifier, access time identifier, time version identifier, and segment sequence number into the output data or status; S3: At the edge, multi-source data is split according to time version identifier, and sorted in order according to segment number and dual time identifier to form a session window evidence index; S4: Initiate an interactive transaction based on the evidence index, determine the transaction window boundary and the list of participating sources, establish the transaction number and data reference relationship, and generate an aligned credential reference; S5: Generate a sequence of linked actions based on interactive transactions, and issue and read back the actions in phases according to the preparation phase, execution phase, and confirmation phase, and record the transaction log.

[0020] S1: Establish a unified time domain at the edge of the exhibition hall, generate a time version identifier, and send registration parameters and alignment session identifiers to each acquisition terminal and linkage control terminal. The specific implementation is as follows: During the exhibition hall operation preparation phase, the edge side establishes a unified time domain and publishes aligned session configurations, enabling data and status related to voice, video, skeleton, and linkage control within the exhibition hall to enter subsequent access, encapsulation, and processing processes under the same time caliber. The edge side generates a unified time domain identifier to identify the time caliber belonging to the exhibition hall and serves as the intra-domain routing basis for session configuration, terminal capability profiles, and data access links. Based on the unified time domain identifier, the edge side generates a time version identifier to mark the time field caliber and interaction message constraints used in this session. When session constraints change or the unified time domain is reinitialized, the edge side generates a new time version identifier and marks the old version as invalid, forming a version boundary that can be consistently identified by subsequent processes. After the time version identifier is generated, the edge side forms a session constraint set bound to that time version identifier. This session constraint set serves as a version-level universal rule, consistently effective for participating terminals. The session constraint set at least specifies the access phase, session state and its migration conditions, time field definition, and the set of mandatory fields and verification points for interaction messages. The access phase restricts the sequential relationship between the terminal and the session from session preparation and access confirmation to session effectiveness, ensuring that the terminal does not enter the effective state before completing definition confirmation. The session state is used to identify whether the session is in the publication, effectiveness, or expiration phase, and the migration conditions are used to define the transition from publication to effectiveness and from... The triggering rules for transitioning from active to inactive enable the edge side to manage session lifecycles using a unified standard and stop referencing the old standard in subsequent links after a session expires. The time field standard unifies the semantics and consistency requirements of the time field in the terminal output data and status, ensuring that different terminals have a consistent and executable interpretation of the same time field. The set of required fields and verification points specify the fields that must be carried in access request, access confirmation, session keep-alive, and status readback messages and their verification locations, enabling the edge side to complete session and terminal runtime registration accordingly and providing a rule basis for subsequent unified field writing and windowed organization. The edge side registers and catalogs the terminals participating in multimodal interaction within the exhibition hall, generating a registration and cataloging table as terminal asset information and capability files for session configuration reference and continuous maintenance. The registration and cataloging is categorized by terminal function role, dividing terminals into voice acquisition terminals, video acquisition terminals, skeleton acquisition terminals, and linkage control terminals, and establishing corresponding registration item templates for each type of terminal. The registration item templates require terminals to submit terminal identifiers, interface types, data output granularity, session access methods, and channel identification rules, enabling the edge side to determine the terminal access entry point and standardize channel expression. Linkage control terminals, during registration, also need to submit the controllable action range, status readback trigger method, and readback field caliber, allowing the edge side to read device status according to unified field semantics and form traceable readback records during subsequent linkage phase separation processes. The edge side writes the above information into the registration and cataloging table and establishes a binding relationship between capability files and time version identifiers, ensuring that session configurations clearly define the corresponding version caliber when referencing terminal capabilities, avoiding inconsistent capability interpretations for the same terminal under different caliber versions. After registration and cataloging are completed, the edge side generates a terminal-level registration parameter set for each terminal. This registration parameter set, as the execution configuration of the session constraint set on the terminal side, matches the terminal category and registration and cataloging capabilities, and is used to implement version-level general rules into terminal-executable access and channel behaviors. The registration parameter set includes at least a set of identity declaration fields, data channel selection rules, and session persistence and recovery declaration rules. The identity declaration field set includes at least a terminal identifier, an aligned session identifier, a time version identifier, and a channel identifier, ensuring that the terminal maintains consistency in identity and messaging when initiating access, continuous activation, and readback interactions, and facilitating field verification by the edge side. The data channel selection rules limit the terminal's... The methods of enabling, switching, and restoring channels within a session ensure that channel behavior remains bound to the aligned session identifier throughout the session, preventing cross-session drift. Session persistence and recovery declaration rules limit keep-alive and recovery declarations during the session, enabling terminals to complete access acceptance according to the same session criteria after short-term disconnection, reconnection, or channel switching, and to send the restored access status back to the edge side with standardized fields to update the session running status. The edge side binds and stores the registration parameter set with the terminal identifier, terminal category, unified time domain identifier, and time version identifier, and establishes a reference relationship with the registration catalog table, which facilitates the differentiated distribution and execution of configurations according to terminals and supports source traceability when subsequent sessions are published. Subsequently, the edge side generates an aligned session identifier and sends session publishing information to each terminal. The session publishing information carries at least the aligned session identifier, the time version identifier, and the registration parameter set matching the terminal. The edge side establishes a session registry based on the registration catalog table and registers the terminal runtime mapping information for this session in the session registry. The runtime mapping information includes at least the terminal access status, the whitelist of allowed output data categories, the whitelist of allowed controlled linkage action categories, and the reference to the readback field caliber. After completing the distribution and registration, the edge side opens the access window, puts the terminal in session ready state, waits for the terminal to initiate time domain access, and proceeds to the next step.

[0021] S2: Each terminal completes time domain access and writes the acquisition time identifier, access time identifier, time version identifier, and segment sequence number into the output data or status. The specific implementation is as follows: After the alignment session identifier and time version identifier are published, each terminal enters the time domain access and time field encapsulation process; the edge side opens the access window for this alignment session based on the session constraint set, and drives the terminal access state machine based on the session registry, so that the terminal access goes through access application, access verification, access confirmation and session activation in sequence; the edge side uses the session constraint set and session registry as the basis for judgment during the verification and state transition process, and synchronously updates the terminal runtime mapping information. During the access application phase, all types of terminals submit access application messages. These messages must carry at least an aligned session identifier, a time version identifier, a terminal identifier, and a channel identifier. Acquisition terminals also submit a fragment organization declaration, describing the fragment boundary identification method and applicable fragment sequence number type for data units during the session. The linkage control terminal simultaneously submits a readback field set declaration and a readback trigger method declaration; the former limits the semantic scope of device status readback fields within the session, while the latter limits the readback interaction type. After receiving the access application message, the edge side verifies the consistency between the terminal identifier and terminal category in the session registry and checks whether the channel identifier is within the allowed set for this session. Subsequently, it updates the fragment organization declaration based on the session constraint set. Alternatively, a consistency verification can be performed by reading back the declaration. After successful verification, the edge side generates and sends out access confirmation information. The access confirmation information includes at least a session validity condition flag, a field encapsulation rule reference, a fragment boundary expression caliber reference, and a sequence number rule reference, and provides a set of required fields for the access confirmation receipt. The access confirmation receipt carries at least an aligned session identifier, a time version identifier, a terminal identifier, and a channel identifier. After the edge side verifies the receipt, the running status of the terminal in the session registry is updated to session validity. Terminals that fail verification or do not receive a receipt remain in the access incomplete state. Data generated in this stage is processed as out-of-session data, without the attached aligned session identifier and time version identifier, and does not enter the edge side session link. After the terminal enters the session active state, each acquisition terminal outputs data units according to the fixed segment boundary criteria and generates segment numbers; the voice acquisition terminal outputs voice segments, and the segment number is used to mark the sequential position within the session; the video acquisition terminal outputs video frame segments or frame group segments, and its segment boundaries and segment numbers are generated according to the fixed criteria in the access confirmation information; the skeleton acquisition terminal outputs posture key point segments, and its segment boundaries and number rules are executed according to the fixed criteria; when reconnection recovery, channel switching or session switching occurs, the skeleton acquisition terminal outputs boundary markers for sequence number inheritance or sequence number reset according to the fixed recovery declaration method, so that the edge side can identify the recovery boundary; the linkage control terminal outputs device status readback information according to the fixed readback criteria and readback triggering method, and keeps the semantics of the readback fields consistent with the fields of the control command interaction within the current session; During the session, each terminal encapsulates the output data and status using unified fields. Acquisition terminals write an acquisition time identifier when generating data units, marking the time position at which the data unit becomes an output segment on the terminal side. They also write an alignment session identifier and a time version identifier to mark session and time caliber affixation, and a segment sequence number to mark the sequential position within the session. The segment sequence number is generated according to a fixed incrementing rule. In the event of reconnection recovery, channel switching, session switching, or terminal restart, it is processed according to fixed acceptance or reset rules, and a reset reason identifier is written. The reset reason identifier distinguishes between scenarios such as session switching, channel switching, reconnection recovery, and terminal restart, enabling subsequent processing to identify sequence number changes as boundary events. The linkage control terminal writes an acquisition time identifier, alignment session identifier, and time version identifier when acquiring device status, and writes an action sequence number as the status segment sequence number field. The action sequence number corresponds to the sequence of control commands issued by the edge side within the same session, establishing a traceable correspondence between status readback and control actions. When data enters the edge processing domain, the edge side writes an access time identifier for each data unit. The access time identifier is generated by a unified time domain time source on the edge side and is used to mark the time position of the data unit entering the same time domain. The acquisition time identifier is generated by the time source determined by the terminal during the access confirmation phase and carried with the packet. The edge side uses it as an in-session timing verification field for subsequent processing and does not use it as an admission basis. The acquisition time identifier and the access time identifier together constitute a dual time identifier, which is used to establish a verifiable correspondence between the terminal acquisition timing and the edge access timing. While writing the access time identifier, the edge side performs basic access verification, verifying whether the aligned session identifier and time version identifier match the current access window, and verifying whether the terminal identifier and channel identifier are within the allowed set registered in the session registry. For data units that fail the access verification, the edge side writes them into the isolation queue and records an exception flag. The exception flag is synchronously registered in the terminal runtime mapping information of the session registry, and an index relationship is established according to the terminal identifier and the aligned session identifier. To adapt to short-term connection loss and recovery scenarios of heterogeneous terminals, the terminal sends back a session liveness declaration and a channel stability declaration according to the fixed declaration elements during the session, and initiates a recovery request after reconnection. The recovery request carries identification elements used to define the recovery boundary, as well as the most recent sequence number position and reset reason flag. The edge side combines the recovery request with the most recent sequence state stored in the session registry to generate recovery confirmation information. The recovery confirmation information clarifies the sequence number succession method or sequence number reset method after recovery, and synchronously updates the terminal running state mapping information in the session registry. Through the access and field encapsulation process, data and status within the session form a unified and interpretable time and sequence number caliber; inputs outside the session are isolated and their boundary information is recorded in runtime metadata; thus, a clear and traceable input foundation can be provided for the next step of time version-based routing, sequential organization within the session window, and construction of evidence index.

[0022] S3: At the edge, multi-source data is split according to time version identifier, and then sorted sequentially by segment number and dual time identifier to form a session window evidence index. The specific implementation is as follows: After completing in-session admission and field encapsulation in the previous stage, the edge side enters the process of time version routing, multi-source sequential organization, and session window evidence index solidification. The continuous multi-source stream in the session is organized into a windowed evidence set before entering the interactive transaction. In this stage, version gating and version routing, deduplication and sequential organization, missing placeholders, and cross-source alignment anchor maintenance are performed on the data in the session. The window results are solidified and registered as evidence indexes that can be directly referenced in subsequent steps. The edge side first performs version gating and traffic splitting; the version release information recorded in the session constraint set is used to generate the valid version set for this session and establish a binding relationship with the alignment session identifier; the edge side only receives data belonging to the valid version set and maintains a corresponding session buffer for each valid version; the buffer is bound to the field caliber reference and session constraint set reference of the version to constrain the incoming data to be consistent in terms of field semantics, message elements and session rules. When a version change occurs in the session constraint set, the edge side activates a new session buffer and switches the original buffer to read-only organization and archiving state to form a clear version boundary and avoid mixing different versions in the same organization link. After the data enters the edge processing domain and is written with the access time identifier, the edge side reads its time version identifier and routes it to the corresponding buffer. For data units with missing time version identifiers, formats that do not conform to the constraint set definition, exceed the scope of the terminal registration parameter set declaration, point to invalid versions, or are in an invalid state, they are written to the isolation queue and a version anomaly mark is recorded. After the traffic is split, the edge side aggregates the data in the corresponding buffer according to the aligned session identifier, so that subsequent sequential organization and windowing processing are limited to the same session scope. The edge side performs sequential organization and deduplication of multi-source data within the same aligned session identifier range. The edge side maintains separate sequence number status tables for voice, video, skeleton, and device status. Voice, video, and skeleton data use segment sequence numbers as primary keys, while device status data uses action sequence numbers as the state segment sequence number field. The sequence number status table records received sequence number intervals, continuity markers, acceptance or reset boundaries, and their reason categories, and associates segment identifier references within the interval with the most recent boundary event reference, providing a basis for duplicate identification, out-of-order processing, missing placeholders, and boundary interpretation. After receiving a new segment, the edge side first determines whether its sequence number falls within a registered interval. If it does, a secondary check is performed, at least verifying whether the segment identifier and dual time identifier ranges are consistent with existing reference relationships. If they are consistent, it is written to the deduplication queue and a reference redundancy marker is recorded; otherwise, it is recorded as a sequence number reuse boundary event and a sequence number is generated. Sequence number reuse anomaly marker; segments not falling into the interval enter the sorting queue, and sorting and consistency verification are completed to identify out-of-order and sequence number jumps; segments judged to be out of order are rearranged and the sequence number status table is updated to stabilize the order position within the session; when a sequence number gap occurs, the gap is written to the missing marker table, and the missing placeholder reference is retained in the windowed output to make the coverage gap display referable; when a takeover or reset boundary is detected, the boundary marker is written to the sequence number status table and synchronized to the session running state mapping information so that the index can distinguish between normal switching boundaries and abnormal chain break boundaries; for device status data, the missing, jump, or boundary event of the action sequence number is combined with the status readback link and solidified into a status coverage gap or readback boundary marker, so that the control action and status readback maintain a corresponding and traceable relationship within the same window, avoiding cross-window mismatch of subsequent linkage confirmation phases; During the multimodal parallel processing, the edge side establishes a unified session window partitioning caliber based on window rule references in the session constraint set, and generates cross-source alignment anchors to form intra-window alignment relationships. Window rule references are registered in the session constraint set as rule entries, and each rule entry contains at least the trigger source type, trigger message field reference, boundary marker field reference, and solidification judgment field reference. During the processing, the edge side writes the matched trigger sources into the window metadata and drives window boundary advancement and solidification judgment according to the field references of the rule entries: when a boundary marker is detected, the window boundary is advanced; when the solidification judgment is satisfied, it is transferred to the index. The link is solidified; cross-source alignment anchors are generated in the version session buffer and incrementally updated as the window progresses; anchors use a unified time field at the session level as a common constraint to map the acquisition time identifier range and access time identifier range of each modality segment to the same window boundary; the edge side generates a relationship type between each segment and the window boundary and writes it into the coverage relationship record. The relationship type at least distinguishes between segments falling into the window, segments crossing the window boundary, and segments with gaps between the segment and the window boundary; the coverage relationship record is updated synchronously when missing placeholders are written, boundary events occur, or anchors are updated, so that the multimodal coverage relationship within the window is consistently expressed. When a window meets the conditions for solidification, the edge side generates a session window evidence index and completes index registration. The evidence index uses the session window as the smallest management unit and writes window ownership and boundary information, fragment reference information, and runtime status and exception metadata. The window ownership and boundary information includes the aligned session identifier, time version identifier, window start and end boundaries, and the sequence number range of each modality within the window, used to determine the session ownership, version ownership, and boundary positioning method of the indexed object. The fragment reference information includes a list of fragments for each modality within the window. The fragment list records the fragment identifier, sequence number field, collection time identifier range, and access time identifier range by reference, and retains missing placeholder references for explicit display. This indicates coverage gaps; for device status segments, the segment reference information retains both action sequence number references and readback boundary markers, enabling subsequent steps to establish a correspondence between control actions and status readbacks within the same window; runtime and abnormal metadata are extracted and solidified hierarchically according to their sources, where access status snapshots and recovery boundaries are taken from runtime mapping information in the session registry, sequence number continuity, missing placeholders, deduplication results, and sequence number reuse boundaries are taken from the sequence number status table, and version anomalies are taken from version gating records; abnormal metadata includes version anomaly markers, sequence number jump markers, missing placeholder markers, sequence number reuse anomaly markers, and reconnection boundary markers, used to describe the boundary conditions and availability information of window coverage; To ensure the stable referencing of evidence indexes in subsequent steps, the edge side generates an immutable index identifier for each evidence index and registers this index identifier, along with its corresponding alignment session identifier, time version identifier, window boundary, and fragment reference set, in the index registration table. The index registration table is used to establish a deterministic mapping between the index identifier, window boundary, and fragment reference set, enabling the direct reference of the index identifier to obtain transaction input boundaries, available information of participating sources, and fragment reference set when initiating interactive transactions. The edge side also sets lifecycle constraints on index objects to ensure that the index remains available and can be referenced by transactions during the session's validity period. After the session expires, the index identifier, window boundary, fragment reference set summary, and runtime and exception metadata summary are permanently saved according to the archiving rules of the session constraint set. This allows the transaction input basis to be restored and the reference relationship to remain consistent even after the buffer is reclaimed, based on the index registration table. Through version routing, sequence sorting and windowing solidification, a stable session window evidence index is generated on the edge side, and a deterministic mapping from the index identifier to the window boundary and fragment reference set is established through the index identifier and the index registration table.

[0023] S4: Initiate an interactive transaction based on the evidence index, determine the transaction window boundaries and the list of participating sources, establish the transaction number and data reference relationship, and generate aligned document references. The specific implementation is as follows: After forming the session window evidence index and completing the index registration in the previous stage, the edge side enters the interactive transaction initiation and alignment credential reference construction process; within the validity period of the alignment session, the edge side receives trigger tag messages from each terminal. The trigger tag message carries at least the alignment session identifier, time version identifier, trigger source terminal identifier, trigger collection time identifier, and trigger access time identifier. The edge side completes the evidence index location based on this and verifies the location result of the proposed window. Upon receiving the trigger flag message, the edge side initiates the evidence index selection process. This process uses the alignment of session identifier and time version identifier as a prerequisite constraint, retrieving registered evidence indexes only within the session version range consistent with the trigger flag message, and reading the start and end boundaries of the window corresponding to the index identifier from the index registration table as candidate transaction window boundaries. The edge side uses the trigger access time identifier for window positioning: when the trigger access time identifier falls within a certain window start and end boundary and the window is not marked as a boundary crossing relationship type, the evidence index corresponding to that window is selected as the basis for transaction input; when the trigger access time identifier does not fall within any window boundary, or falls within a window marked as a boundary crossing relationship type, ... The edge side locates the adjacent window index identifiers of the window through the index registration table, and reads the relationship type markers and abnormal metadata of the involved windows to generate boundary abnormal constraints. The abnormal metadata includes at least reconnection boundary markers, missing placeholder markers, sequence number jump markers, and version abnormal markers. The relationship type markers are used to distinguish the coverage relationship of fragments falling into the window, fragments crossing the window boundary, and fragments with gaps between the fragments and the window boundary. The edge side verifies the window location results with the trigger collection time marker and writes the verification marker into the transaction context. After selection, the edge side registers the selected evidence index identifier in the transaction context and solidifies the transaction window boundary fields according to the mapping results of the index registration table. After determining the transaction window boundaries, the edge side generates a list of participating sources based on the evidence index. This list is based on the modality coverage information, relation type markers, and anomaly metadata embedded in the evidence index, and does not automatically shrink or expand the input set simply because the trigger source belongs to a single modality. The edge side reads the list of modal fragments and missing placeholder references from the evidence index to determine the modality type to be referenced, the corresponding terminal identifier, and provides the range of fragments allowed to be referenced by each modality within the window. The fragment range is primarily limited by the fragment sequence number range, while the range of acquisition time identifiers and access time identifiers are written into the participating source list as consistency verification fields, enabling subsequent stages to verify the references without relying on external buffers. Whether the fragments belong to the same window boundary and the same version caliber; the edge side then performs restricted source marking and range pruning on the participating source list based on abnormal metadata: when a terminal has a reconnection boundary, sequence number jump, missing placeholder, or version abnormal mark during the window, the terminal is marked as a restricted source, and its referable range is pruned according to the fragment list, relation type mark, and missing placeholder reference, limiting it to the sub-interval within the window with continuous sequence numbers and not covered by missing placeholders, while retaining the abnormal mark reference and the pruning basis reference; when there are no abnormal marks that affect window coverage and version consistency, the participating source list retains the complete fragment range within the window, so that the transaction input range is determined once and traceable when initiated; After determining the transaction window boundaries and the list of participating sources, the edge side generates a transaction number and creates a transaction context bound to the transaction number. The transaction number uniquely identifies the current interaction processing unit and serves as the primary key for subsequent judgment, linkage, and archiving. The transaction context records the evidence index identifier, transaction window boundary field, alignment session identifier, time version identifier, and list of participating sources. It also registers the trigger source type and trigger source terminal identifier of the trigger marker message as auxiliary metadata to reconstruct the transaction initiation background and trigger chain afterward. Subsequently, the edge side establishes a data reference relationship between the transaction number and data references. This data reference relationship is built based on the evidence index and is fixed through the reference chain. The system converts transaction input without copying the original fragment data. On the edge side, transaction numbers are bound to voice fragment references, video fragment references, skeletal fragment references, and device status fragment references, respectively. The fragment identifier, fragment sequence number range, and acquisition time identifier range and access time identifier range are registered in the reference relationship, so that the transaction can independently verify whether its input belongs to the registered evidence index and transaction window boundary in subsequent stages. For device status fragments, the action sequence number field is further registered in the reference relationship so that the readback status is matched with the control action sequence number in the same transaction in the linkage confirmation phase, maintaining a consistent link between control actions and readback status at the reference level. After establishing data reference relationships, the edge side generates alignment credential references in the transaction context. These alignment credential references carry the time consistency and caliber consistency context upon which the transaction depends. They are registered as alignment pointers in the form of reference identifiers and do not copy context content in various output records. The alignment credential references are associated with at least the time version identifier, alignment session identifier, and evidence index identifier, and simultaneously with the terminal access status snapshot reference and the index anomaly view reference. The terminal access status snapshot is extracted from the session registry and solidified into a status view by the edge side at the time of transaction initiation. The status view includes the access status flags of each terminal, the allowed output data category flags, and the readback caliber flags, used for access verification in the transaction execution chain. The index anomaly view references point to the abnormal metadata and relationship type flags of the evidence index, used for coverage verification in the transaction execution chain. The edge side binds and stores the alignment credential references with the transaction number and registers them as mandatory items carried in the transaction output chain. Subsequent judgment outputs, linkage action issuance records, and status readback records related to this transaction all carry this alignment credential reference, enabling audits to uniformly point to the window boundaries, participating source list, and anomaly caliber set of the transaction. To adapt to concurrent interaction scenarios in the exhibition hall, the edge side performs transaction isolation and conflict control when initiating a transaction. The edge side determines the transaction scope based on the device set identifier corresponding to the device status fragment reference in the participating source list, and associates this scope with the transaction number as a scope marker. The edge side uses the existence of intersection of device set identifiers as the conflict determination condition: when the device sets of two transactions have an intersection, the later-arriving transaction enters a queuing state and waits for the earlier-arriving transaction to release. The queuing order is determined by the order of the trigger access time identifier. If the trigger access time identifiers are the same, they are determined by the order of the transaction window start boundary. When the device sets do not have an intersection, concurrent execution is allowed. The edge side writes the control result into the transaction context and registers the corresponding control marker in the alignment credential reference, so that subsequent phase-by-phase execution, readback archiving, and conflict review follow a unified control standard. After the transaction is initiated, the edge side outputs the transaction context and alignment credential reference bound to the transaction number, which serves as the unified entry point for the next stage of phase-by-phase distribution, status readback, and log archiving.

[0024] S5: Generate a sequence of linked actions based on interactive transactions, and issue and read back the actions in phases according to the preparation phase, execution phase, and confirmation phase, and record the transaction log. The specific implementation is as follows: The edge side determines the set of controllable devices that the transaction is allowed to reach based on the transaction scope marker and the list of participating sources in the transaction context. It then prunes this set of devices using the restricted source marker to obtain the target device scope marker. This target device scope marker is written into the linkage execution context and linked to the device identifier mapping in the registration catalog table and session registry, ensuring consistent interpretation of the target device scope throughout the transaction's lifecycle. Under the constraints of the target device scope marker, the edge side generates a linkage action sequence and stores it bound to the transaction number. Simultaneously, it assigns a unique and monotonically increasing action sequence number to each action within the sequence, establishing a verifiable correspondence between control issuance records, status readback records, and exception records. The edge side determines a device subset for each group of actions and registers the device grouping marker. Subsequently, it solidifies the sequence caliber within the group and registers it as an immutable sequence view to support subsequent group-based issuance and group-based aggregation readback. The edge side divides the coordinated execution into three phases: preparation phase, execution phase, and confirmation phase. Alignment credential reference serves as the shared alignment context pointer for all three phases, transaction number serves as the shared primary key, and time version identifier serves as the shared time caliber marker, ensuring that records in each phase point to the same transaction input basis and maintain the same time caliber. Upon entering the preparation phase, the edge side initiates a status read request to the coordinated control end, requesting the target device range marker and the readback field caliber reference. The readback field caliber reference is taken from the readback caliber registered and fixed by the coordinated control end during the access phase, used to define the field set boundaries, field type markers, and field meaning caliber of the device status, ensuring that the returned status can be interpreted under a unified semantic framework. The coordinated control end returns the device status and occupancy status, with the return message carrying the acquisition time identifier and access time identifier. Based on this, the edge side generates a device status snapshot reference and registers it in the coordinated execution context. The device status snapshot reference serves as the starting status view for this transaction and as the baseline view for the confirmation phase verification. Subsequently, the edge... The edge side initiates an interlock occupancy request based on the conflict protocol marker. The interlock occupancy request takes the target device range marker as input, uses the intersection relationship of device sets as the conflict determination criterion, and records the occupancy status in the transaction registration table. When the interlock occupancy request is approved, the edge side generates an interlock occupancy marker and writes it into the linked execution context, ensuring that the occupancy criterion for the target device range remains consistent during the execution phase and confirmation phase of this transaction. When the interlock occupancy request fails, the edge side puts this transaction into a queued state and registers a blocking reason reference. The blocking reason reference points to the transaction number of the occupancy source and the device set identifier where the intersection occurs. Queued transactions use an occupancy release event or an occupancy status change event as the re-evaluation trigger condition. After the event arrives, the transaction re-enters the preparation phase, updates the device status snapshot reference, and initiates the interlock occupancy request again, thus forming a queuing and wake-up closed loop. The preparation phase also registers a preparation phase time reference marker and binds and stores this reference marker with the transaction number, which is used to organize the timing traceability of the execution phase issuance record and the confirmation phase readback record. After the interlock occupancy request is approved and the linkage execution context is in an executable state, the edge side enters the execution phase. The edge side issues control commands to the linkage control terminal according to the action sequence number of the linkage action sequence. The control commands are written with the action sequence number, target device range marker, device group marker, and stage marker to clarify that the message belongs to the execution phase and distinguish the message semantics of different stages. When the control command enters the edge unified time domain, the edge side writes the access time identifier and binds the access time identifier with the action sequence number, device group marker, and group sequence marker to register as a distribution record, so that each distribution has a traceable record in terms of time position, execution order, and scope of affected devices. For multi-device collaborative actions, the edge side distributes the commands in groups according to the device group marker and determines the group sequence according to the fixed group sequence view in the linkage execution context, and distributes them in sequence to maintain a consistent sequence for the same action under link jitter conditions. The edge side synchronously registers the association between the action sequence number and the data reference slot within the transaction window boundary, so that the evidence window position corresponding to the action sequence number can be located by the transaction number during audit playback, and the correspondence between interactive evidence and linkage execution can be restored. After the execution phase is completed, the edge side enters the confirmation phase. The confirmation phase initiates a status readback request using the action sequence number as an index. The edge side sends a readback message to the linkage control terminal. The readback message carries the action sequence number and the readback field caliber reference to instruct the linkage control terminal to output the status field set according to the unified readback caliber and maintain the consistency of the field meanings. The linkage control terminal returns a status readback message, which carries the acquisition time identifier and the access time identifier. The edge side collects the readback status according to the action sequence number, device grouping mark, and group sequence mark, and establishes a readback reference object for each action sequence number. The readback reference object is bound to the transaction number, action sequence number, and readback field caliber reference, pointing to the field set and time identifier set of the corresponding readback message, so that the readback record can be located within the transaction context and form a verifiable association with the corresponding issued record. On the edge side, a consistency verification is performed on the confirmed phase. The consistency verification uses the device status snapshot reference as the starting state view, the execution phase issuance record as the action boundary basis, and the readback field caliber reference as the semantic constraint of the field set and field type. The completeness, correspondence, and semantic consistency of the readback record are verified. The edge side defines the abnormal boundary with the state machine and field caliber and adopts rule-based judgment: if a readback reference object corresponding to a certain action sequence number is not formed before the confirmed phase end condition is triggered, it is marked as readback missing; if a readback reference object has been formed but its field set does not meet the field set boundary or field type mark required by the readback field caliber reference, it is marked as caliber mismatch; if the readback reference object cannot establish a correspondence with the device group mark under the target device range mark or cannot establish a correspondence with the action sequence number, it is marked as a corresponding abnormality. The edge side registers the abnormality record with the abnormality type, corresponding action sequence number, corresponding device group mark, and corresponding stage mark, and registers the association pointer with the readback reference object or issuance record, so that the abnormality can be located to the specific action and specific device group and has verifiable evidence reference. After the phase confirmation is completed, the edge side will solidify the entire process of this transaction linkage execution into a transaction log and complete the archiving registration. The transaction log uses the transaction number as the primary key and writes the time version identifier, alignment certificate reference, and evidence index identifier to form a closed-loop reference relationship between the log and the transaction input basis. The transaction log consists of a log header, phase records, exception sets, and tail records. The log header registers the transaction window boundary, target device range marker, conflict protocol marker, and interlock occupancy marker or blocking reason reference, which is used to restore the transaction scope and concurrency protocol basis. The preparatory phase record registers the device status snapshot reference, occupancy status view, and interlock occupancy application result, which is used to trace the initial state and occupancy judgment caliber. The execution phase record registers the action sequence number, device group marker, group sequence marker, phase marker, access time identifier, and reference pointer associated with the data reference slot, which is used to restore the correspondence between the control access trajectory and the interactive evidence window. The confirmation phase record registers the readback reference object, readback field caliber reference, acquisition time identifier, and access time identifier, which is used to verify the semantic caliber and time position of the readback status. The anomaly set registers the anomaly type, corresponding action number, corresponding device grouping mark, and corresponding stage mark, and registers the attribution pointer. The attribution pointer is associated with the evidence index identifier and its anomaly view reference to distinguish whether the source of the anomaly is a coverage gap or inconsistency in the alignment link, or a readback missing, inconsistent, or corresponding anomaly in the linkage link, thereby forming a maintainable fault attribution path. The transaction log records the transaction completion status and interlock occupancy release mark, and synchronously updates the occupancy status in the transaction registration table, so that the application, maintenance, and release of device occupancy are closed within the same transaction link. After the transaction log is written to disk, the edge side updates the association between the transaction number and the evidence index identifier and the alignment voucher reference in the transaction registration table, so that the audit playback can obtain the session window evidence on which the transaction is based and the linkage execution trajectory of the transaction by using the transaction number as the entry point. At the edge, execution records and transaction logs are archived and linked by transaction number, and the logs are linked by aligning credential references to form a replayable transaction chain.

[0025] In this embodiment, a unified time domain is first established at the edge of the smart exhibition hall, and a time version identifier is generated. Registration and cataloging of the voice acquisition terminal, video acquisition terminal, skeleton acquisition terminal, and linkage control terminal are completed. Alignment session identifiers and registration parameter sets are published to each terminal, and the terminal access status, allowed output data types, allowed linkage action types, and readback field specifications are registered in the session registry. After access confirmation, each terminal enters the session active state, writing the acquisition time identifier, access time identifier, and time version identifier to voice segments, video frame segments, skeleton segments, and device status, and writing according to data unit type. The system inputs segment numbers or action numbers; at the edge, it routes data according to time version identifiers, and within the session, it uses segment numbers combined with dual time identifiers to perform deduplication and reordering, generating and solidifying the session window evidence index and the mapping relationship between the index identifier and the segment reference set; then, it locks the transaction window boundary based on the evidence index and forms a list of participating sources, generates a transaction number, establishes the relationship between the transaction number and data references, and generates an aligned credential reference; finally, it issues linked actions and reads back the status according to the preparation phase, execution phase, and confirmation phase, archives the transaction log, and forms an interactive linkage link that is aligned, verifiable, and replayable.

[0026] Example 2: Figure 2 This invention discloses a multimodal interactive digital human system for smart exhibition halls, comprising: Time Domain and Session Publishing Module: Establishes a unified time domain and publishes aligned sessions on the edge side, completes terminal registration and cataloging and parameter distribution, and establishes a session registry to register permissions, status and readback criteria; Terminal access and field encapsulation module: used for terminal access and standardization, uniformly write time and serial number identifier, supplement access time on the edge side and isolate abnormal input; Version routing and evidence index construction module: It performs time-version-based flow control, combines fragment sequence number and dual time identifier to sort and deduplicate, generates session window evidence index and registers mapping for transaction reference; Interactive Transaction and Alignment Certificate Module: Selects evidence index and locks window based on trigger flag, generates participating source list and transaction number, establishes fragment reference and solidifies alignment certificate, and handles contract concurrent conflicts according to device set; Linked phase execution and transaction log module: Generates and numbers linked sequences under transaction constraints, issues back-read verification in phases, archives logs by transaction and associates vouchers and evidence indexes to achieve a replayable closed loop.

[0027] It should be noted that this invention can be deployed on the device itself to realize embedded applications, or it can run on a PC or other terminal with a user interface, thereby meeting various hardware environments and usage requirements.

[0028] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wireless or wired transmission; wired transmission methods include optical fiber, twisted pair, coaxial cable, etc.; wireless transmission includes infrared, microwave, etc. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center containing one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0029] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0030] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0031] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0032] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0033] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0034] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0035] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for realizing a multimodal interactive digital human in a smart exhibition hall, characterized in that, include: S1: Establish a unified time domain on the edge of the exhibition hall, generate a time version identifier, and send registration parameters and alignment session identifiers to each acquisition terminal and linkage control terminal; S2: Each terminal completes time domain access and writes the acquisition time identifier, access time identifier, time version identifier, and segment sequence number into the output data or status; S3: At the edge, multi-source data is split according to time version identifier, and sorted in order according to segment number and dual time identifier to form a session window evidence index; S4: Initiate an interactive transaction based on the evidence index, determine the transaction window boundary and the list of participating sources, establish the transaction number and data reference relationship, and generate an aligned credential reference; S5: Generate a sequence of linked actions based on interactive transactions, and issue and read back the actions in phases according to the preparation phase, execution phase, and confirmation phase, and record the transaction log.

2. The method for realizing a multimodal interactive digital human in a smart exhibition hall according to claim 1, characterized in that, A unified time domain is established at the edge of the exhibition hall, a time version identifier is generated, and registration parameters and alignment session identifiers are sent to each acquisition terminal and linkage control terminal, including: At the edge, a unified time domain identifier and time version identifier are generated, and the old version failure boundary is set to form a session constraint set; Session constraint sets limit access phases, session state transitions, time field definitions, and mandatory message field verification points; On the edge side, register and catalog the voice acquisition terminal, video acquisition terminal, skeleton acquisition terminal and linkage control terminal according to the terminal template, and generate a registration parameter set; The registration parameter set defines the identity declaration, channel selection, session persistence and recovery rules, publishes the aligned session identifier, and registers the session registry.

3. The method for realizing a multimodal interactive digital human in a smart exhibition hall according to claim 1, characterized in that, Each terminal completes time domain access and writes the acquisition time identifier, access time identifier, time version identifier, and segment sequence number into the output data or status, including: The edge side opens an access window, performs the application, verification, confirmation and receipt verification in sequence, and updates the terminal status to show that the session is active. The access request carries the alignment session identifier, time version identifier, terminal identifier and channel identifier. The acquisition end submits the fragment organization declaration, and the linkage end submits the readback field and readback trigger declaration. The terminal output writes the acquisition time identifier, access time identifier, time version identifier, and segment sequence number or action sequence number, and writes the sequence number reason mark. The edge side writes the access time identifier and isolates and registers the outputs that fail the access verification.

4. The method for realizing a multimodal interactive digital human in a smart exhibition hall according to claim 1, characterized in that, The edge side distributes multi-source data by time version identifier, including: The edge side determines the set of valid versions based on the session constraint set and maintains a corresponding session buffer for each valid version. After data is written to the access time identifier, it is routed to the corresponding buffer according to the time version identifier. Data of missing or invalid versions is isolated and version anomalies are registered. When a version change occurs, a new session buffer is enabled, and the old session buffer is set to read-only collation and archiving status.

5. The method for realizing a multimodal interactive digital human in a smart exhibition hall according to claim 1, characterized in that, The evidence is organized sequentially by segment number and dual time identifier to form a session window evidence index, including: The edge side maintains the status table of each modal sequence number, performs deduplication and random reordering based on the segment sequence number or action sequence number and in combination with the acquisition time identifier and access time identifier, checks the segment identifier and the range of the dual time identifier for duplicate segments and registers the sequence number reuse anomaly. Write missing placeholder references to the sequence number holes and record the boundary acceptance or reset. Generate cross-source alignment anchors according to window rules and solidify the evidence index. Assign immutable index identifiers and register the index registry and archive summary.

6. The method for realizing a multimodal interactive digital human in a smart exhibition hall according to claim 1, characterized in that, Initiate an interactive transaction based on the evidence index, determine the transaction window boundaries and the list of participating sources, including: The edge side receives trigger messages carrying an alignment session identifier, a time version identifier, a trigger source terminal identifier, a collection time identifier, and an access time identifier; On the edge side, the evidence index is selected from the index registration table according to the trigger access time identifier, and the transaction window boundary is locked. Based on the abnormal metadata of the evidence index, a list of participating sources is generated and the reference range of each participating source is determined.

7. The method for realizing a multimodal interactive digital human in a smart exhibition hall according to claim 1, characterized in that, Establish the relationship between transaction numbers and data references and generate aligned document references, including: The edge side generates a transaction number and establishes a binding relationship between the transaction number and the multimodal segment reference. It registers the sequence number range and the collection time identifier range and the access time identifier range of each segment, and registers the action sequence number field for the device status segment reference. The edge side generates an alignment credential reference and associates the alignment credential reference with the version identifier, session identifier, evidence index identifier, and terminal access status snapshot. Edge-side registration of transaction conflict rules based on the intersection relationship of device set identifiers.

8. The method for realizing a multimodal interactive digital human in a smart exhibition hall according to claim 1, characterized in that, Generate a sequence of linked actions based on the interactive transaction, including: The edge side trims the set of controllable devices based on the transaction scope marker and the restricted source marker, generates the target device range marker, and generates a sequence of linked actions based on the target device range marker and assigns the action sequence number according to the monotonically increasing rule within the transaction. The edge side displays the grouping markers for the linkage action sequence registration equipment and the sequence view within each group.

9. The method for realizing a multimodal interactive digital human in a smart exhibition hall according to claim 1, characterized in that, The transaction is issued and read back in phases according to the preparatory phase, execution phase, and confirmation phase, and the transaction log is recorded, including: The pre-phase reads the device status, generates a status snapshot, and initiates an interlock occupancy request. If the request fails, it is placed in a queued state, and a retry is triggered by an occupancy release event. The pre-phase reference time stamp is registered. The execution phase is grouped by action number to issue control commands and write access timestamps and phase markers. The confirmation phase is read back by action number to establish readback references and completes verification based on status snapshots, issued records and readback criteria. The transaction log registers phase records, exception pointers, alignment credential references, evidence index identifiers, completion markers and occupancy releases and updates the registration table.

10. A smart exhibition hall multimodal interactive digital human system, used to implement the smart exhibition hall multimodal interactive digital human implementation method according to any one of claims 1-9, characterized in that, include: Time Domain and Session Publishing Module: Establishes a unified time domain and publishes aligned sessions on the edge side, completes terminal registration and cataloging and parameter distribution, and establishes a session registry to register permissions, status and readback criteria; Terminal access and field encapsulation module: used for terminal access and standardization, uniformly write time and serial number identifier, supplement access time on the edge side and isolate abnormal input; Version routing and evidence index construction module: It performs time-version-based flow control, combines fragment sequence number and dual time identifier to sort and deduplicate, generates session window evidence index and registers mapping for transaction reference; Interactive Transaction and Alignment Certificate Module: Selects evidence index and locks window based on trigger flag, generates list of participating sources and transaction number, establishes fragment reference and solidifies alignment certificate, and handles contract concurrency conflicts according to device set; Linked phase execution and transaction log module: Generates and numbers linked sequences under transaction constraints, issues back-read verification in phases, archives logs by transaction and associates vouchers and evidence indexes to achieve a replayable closed loop.

Images