Digital human companion interaction method based on near field communication triggering and related device

Abstract

The application relates to a digital human companion interaction method based on near field communication triggering and related devices. A terminal reads a near field communication tag on an entity guide carrier, initiates a scheduling request carrying identification information, triggering space-time information and a device fingerprint to a server; the server performs trusted triggering verification and generates trusted triggering credentials binding the terminal and the triggering space-time information, and issues digital human image resources and exclusive knowledge base access credentials; the terminal loads the image resources and establishes a real-time audio and video interaction channel; the server continuously performs presence continuity verification based on the trusted triggering credentials as a chain head to obtain a presence credential chain, thereby maintaining, downgrading or suspending the service, and providing guidance, shopping or question answering based on the exclusive knowledge base through the channel. The application binds one-time triggering and conversation presence into a continuous trust chain, has the advantages of inhibiting clone tag misuse and conversation moving and stealing, and accurately measuring the service and attribution to the real triggering source.

Description

Technical Field

[0001] This application relates to the field of human-computer interaction, and in particular to a digital human companion interaction method and related apparatus based on near-field communication. Background Technology

[0002] In recent years, the integration of digital human technology and near-field communication (NFC) technology has provided new interactive methods for guided tours and shopping services in scenic spots, shopping malls, exhibition halls, and other venues. Visitors can touch NFC tags deployed on physical guideposts such as signs and booths with their handheld devices to trigger the system to dispatch the corresponding digital human avatar. Through real-time audio and video channels, they can receive guided tours, shopping assistance, and Q&A services. Compared to traditional fixed audio guides and paper-based tour guides, this method of triggering digital human interaction with a single touch is more natural and can present different digital human avatars and exclusive knowledge content based on different physical guideposts. It has gradually become an important service model for smart tourism and smart retail.

[0003] In the operation of such services, services and revenues are typically attributed based on the triggering source. That is, based on the specific physical guide vehicle touched by the tourist, the resulting services such as guided tours and shopping assistance, as well as the corresponding commissions, are measured and attributed to the corresponding digital avatar and operating entity. Therefore, whether the trigger is genuine and credible, and whether the session established by the trigger is still held by the legitimate user who initiated the trigger during its existence, directly affects whether the service can be correctly measured and whether the revenue can be attributed to the genuine triggering source. This is the foundation for the stable operation of such services.

[0004] However, in existing near-field communication triggering services, the credentials used for triggering rely on readable data of physical tags. The identifiers and tokens of physical tags can be copied at low cost, and even their unique identifiers can be counterfeited by rewritable cards. Furthermore, after a successful trigger, the system usually does not continue to verify whether the legitimate user who initiated the trigger is still present and holding the session. Because the trigger is designed as a discrete, one-time event, and the trigger credentials are not continuously bound to the user's physical presence, cloned tags can be used to trigger in a different location, and sessions established by legitimate users can be moved midway and occupied by others. This leads to unauthorized use of the service, hijacking of commissions attributable to the trigger source, and makes it difficult for the operating entity to accurately measure the service and revenue to the true trigger source. Summary of the Invention

[0005] To address the problems in existing near-field communication (NFC) triggered services, such as trigger credentials being attached to easily cloned entity tags and a lack of continuous verification of session presence after successful triggering, which leads to unauthorized use of the service, hijacking of commissions based on the trigger source, and difficulty in accurately measuring services and revenue to the true trigger source, this application provides a digital human escort interaction method and related apparatus based on NFC triggering.

[0006] Firstly, this application provides a digital human companion interaction method based on near-field communication, which adopts the following technical solution: A digital human companion interaction method based on near-field communication includes the following steps: S1. The terminal reads the near-field communication tag deployed on the physical guide carrier through near-field communication, obtains the identification information carried by the near-field communication tag, and initiates a digital human scheduling request to the server. The digital human scheduling request carries the identification information, the terminal's triggering spatiotemporal information, and the terminal's device fingerprint. S2. The server performs trusted trigger verification on this trigger based on the identification information, trigger time and space information and device fingerprint. In response to the trusted trigger verification being passed, it generates a trusted trigger credential that binds the terminal and the trigger time and space information, and sends the digital human image resource corresponding to the identification information and the access credential of the exclusive knowledge base to the terminal. S3. The terminal loads digital human image resources and establishes a real-time audio and video interaction channel between the terminal and the server; S4. During the existence of the real-time audio and video interaction channel, the server continuously performs presence continuity verification on the terminal using a trusted trigger credential as the chain head, obtains the presence credential chain, and maintains, downgrades, or suspends the digital human companion interaction service carried by the real-time audio and video interaction channel based on the presence credential chain. S5. The server provides digital human companion interactive services such as tour guidance, shopping guidance, or Q&A to the terminal through a real-time audio and video interaction channel based on a dedicated knowledge base.

[0007] By adopting the above technical solution, the trusted trigger credentials generated by a single touch are used as the head of the presence credential chain that runs through the entire session. This allows the server to bind service authorization to the terminal that initiated the trigger and the trigger time and space at the triggering stage. During the session, the presence continuity is continuously verified starting from this head of the chain. This extends the one-time judgment of the authenticity of the trigger to a continuous judgment of the session's ownership. As a result, even if a cloned tag impersonates the trigger, it will be identified because it cannot continue the legitimate presence credential chain. When a legitimate session is moved midway, its presence credential chain breaks, triggering service degradation or suspension. This ensures that the navigation, shopping guide, and Q&A services and their attribution can be stably maintained and measured back to the true source of the trigger.

[0008] Optionally, the trusted trigger verification in S2 includes rolling token verification, which includes the following sub-steps: S21. The server extracts the scroll token currently carried in the writable area of ​​the near-field communication tag from the identification information, and verifies whether the scroll token is a valid token that was written after the most recent legitimate interaction of the near-field communication tag and has not been marked as used.

[0009] S22. In response to the successful verification of the rolling token, the server marks the rolling token as used and generates a new rolling token based on the hardware unique identifier of the near-field communication tag, the current triggering time and space information and the incrementing sequence number. The new rolling token is sent to the terminal along with the trusted triggering credentials.

[0010] S23. The terminal performs a write-back, writing the new rolling token into the writable area of ​​the near-field communication tag via near-field communication, which will serve as the token to be verified for the next trigger of the near-field communication tag.

[0011] Rolling token verification also includes clone tag verification. Cloning tag verification includes: the server performs consistency verification between the hardware unique identifier bound to the rolling token and the actual hardware unique identifier of the near-field communication tag that submitted the rolling token. If the hardware unique identifier bound to the rolling token is inconsistent with the actual hardware unique identifier, it is determined to be a clone tag and the trigger is rejected.

[0012] By adopting the above technical solution, the token is written back and marked as used after each trigger, which invalidates the old token that was copied to the ordinary tag. The clone tag verification further requires that the tag identifier bound to the token be consistent with the actual identifier of the submitted tag, thereby intercepting clone tags that only copy readable data.

[0013] Optionally, the server maintains a transition token pair containing the current token and the previous token for the near-field communication tag. In S21, the server compares the extracted rolling token with the transition token pair. If either the rolling token or the transition token pair matches, the rolling token is deemed to have passed the verification. If the terminal reports that the write-back is incomplete, or if the server detects that the near-field communication tag still carries the previous token in the next trigger, the server enters the transition fault tolerance window. Within the transition fault tolerance window, the server accepts the previous token, but requires that the spatiotemporal information and device fingerprint of this trigger be consistent with the first trigger. If the terminal successfully completes the write-back in the next interaction, the server converges the transition token pair into a transition token pair with the latest written token as the current token.

[0014] By adopting the above technical solution, the transition token formed by the current token and the previous token absorbs the misjudgment caused by near-field write-back out of step, and converges after successful write-back, so that the rolling token mechanism can still be used when write-back fails occasionally without rejecting legitimate triggers.

[0015] Optionally, the transition token binds to the device fingerprint and trigger spatiotemporal information of the device that first triggered the near-field communication tag. The trusted trigger verification also includes a clone diffusion decision executed by the server. The clone diffusion decision includes: in response to the rolling token with the same incrementing sequence number being submitted at multiple terminals with inconsistent device fingerprints, or being submitted under multiple trigger spatiotemporal information that are physically unreachable, the server determines that the rolling token has forked and the near-field communication tag has been cloned and diffused, freezes the triggering of the near-field communication tag and requires rebinding. The server also divides the trigger spatiotemporal information into three levels according to the deviation of the near-field communication tag's registration position: sensing neighborhood, scene neighborhood, and cross-scene. Triggers within the sensing neighborhood are allowed, triggers within the scene neighborhood require secondary on-site verification, and triggers across scenes are frozen.

[0016] By adopting the above technical solution, the clone spread of the tag is identified and frozen by detecting the submission of the same serial number token in inconsistent devices or unreachable time and space. Then, it is dealt with according to the trigger position deviation, so that the cloned card with the unique identifier of the counterfeit tag is adjudicated due to global time and space conflict.

[0017] Optionally, trusted trigger verification also includes relay protection. Relay protection includes: when the terminal triggers, it collects the environmental fingerprint of the location of the near-field communication tag and sends the environmental fingerprint along with the digital human scheduling request. The server compares the environmental fingerprint with the registered environmental fingerprint of the near-field communication tag. The server also determines whether the trigger is relayed based on the upper bound of the round-trip time between the terminal and the server for this trigger, combined with the physical distance constraint of near-field communication. If the environmental fingerprint is mismatched or the round-trip time exceeds the upper bound, the trigger is rejected.

[0018] By adopting the above technical solution, the relay forwarding is determined by the environmental fingerprint of the trigger location, the near-field physical action distance, and the upper limit of the round-trip delay, thus intercepting attacks that impersonate near-field triggering in other locations through relay.

[0019] Optionally, the presence credential chain in S4 uses a trusted trigger credential as its head. The trusted trigger credential carries the binding of trigger spatiotemporal information and device fingerprint. The presence continuity verification includes the following sub-steps: S41. The terminal continuously collects motion and environmental sensing continuous fingerprints during the real-time audio and video interaction channel.

[0020] S42. The server compares the sensor continuity fingerprint with the sensor continuity fingerprint bound to the previous presence credential in the presence credential chain.

[0021] S43. In response to the continuity of the sensor fingerprint, the server generates a new presence credential based on the previous presence credential, the current sensor fingerprint, and the binding carried by the chain head, and continues it to the presence credential chain, so that each presence credential in the presence credential chain can be traced back to the chain head. In response to the breakage of the sensor fingerprint, the server determines that the current session of the real-time audio and video interaction channel has been moved, invalidates the presence credential chain, and requires the terminal to reread the near-field communication tag to rebuild the chain head.

[0022] By adopting the above technical solution, the presence credential chain uses a trusted trigger credential as the chain head and carries the trigger time and space binding with the device. Each presence credential can be traced back to the chain head. When the session is moved and the sensing continuity is broken, the credential chain is invalidated and the chain head is required to be touched again to rebuild it, so that the ownership of the session presence can be traced continuously.

[0023] Optionally, the presence continuity verification uses the terminal's device hardware root as the invariant for relocation determination, and the sensor continuity fingerprint as a tolerable auxiliary criterion. In S43, the server distinguishes between normal terminal movement and session relocation based on the mutation rate of the sensor continuity fingerprint. Only when the device hardware root changes, or the mutation rate of the sensor continuity fingerprint exceeds the range of the corresponding normal movement and the presence continuity verification of the terminal is lost at the same time, is it determined that the current session has been relocated. In response to the interruption of the network on which the presence continuity verification depends, but the device hardware root and the sensor continuity fingerprint are still continuous, the terminal maintains the digital human companion interaction service based on the most recent presence credential cached locally, and replays the reconciliation to the server after the network is restored. The presence continuity verification is handled in stages according to the duration of the network interruption. The digital human companion interaction service is maintained within the first duration, the digital human companion interaction service is downgraded within the second duration which is longer than the first duration, and the real-time audio and video interaction channel is suspended and the presence continuity verification is required to be re-executed when the second duration is exceeded or the sensor continuity fingerprint is broken.

[0024] By adopting the above technical solution, the method takes the device hardware root as the invariant for relocation judgment, distinguishes between normal movement and relocation by the rate of change of sensor continuity, and maintains, downgrades or suspends services according to the duration of network interruption, and reconciles accounts after network restoration. This ensures that the method does not misjudge under normal user movement and weak network environment, and suspends services in a timely manner during actual relocation.

[0025] Optionally, in S5, for the measurement and attribution of attribution events in the digital human escort interaction service, the server requires that each attribution event carry the current presence credential from the presence credential chain and the fresh presence credential derived from the device hardware root when it occurs. The server traces the fresh presence credential back to the head of the presence credential chain, verifies that the attribution link of the attribution event is complete and uninterrupted from the head of the chain, and measures the attribution of the attribution event to the near-field communication tag corresponding to the head of the chain and the digital human image corresponding to the digital human image resource. In response to the attribution event being a transaction or payment-related key attribution event, the server requires the terminal to reread the near-field communication tag and perform strong presence verification for secondary environmental confirmation.

[0026] By adopting the above technical solution, each attribution event carries fresh credentials derived from the on-site credential chain and the device hardware root, and backtracks to the chain head to verify the integrity of the attribution chain. Services and commissions are measured to the corresponding tags and digital human images at the chain head. Key attribution events require re-touching strong verification, so that the revenue is accurately attributed to the real triggering source and is difficult to be impersonated.

[0027] Optionally, the triggering spatiotemporal information includes at least one of satellite positioning coordinates, floor or area identifier, and server-side time synchronization timestamp.

[0028] By adopting the above technical solution, the location and time of the triggering are described by multi-dimensional spatiotemporal elements, providing a more reliable spatiotemporal basis for trigger verification and cloning proliferation adjudication.

[0029] Optionally, the device fingerprint consists of at least two of the following: the terminal's hardware identifier, operating system characteristics, and near-field communication module characteristics.

[0030] By adopting the above technical solution, the distinguishability of device fingerprints is improved by combining multiple features, reducing the possibility of device fingerprints being easily counterfeited.

[0031] Optionally, the hardware unique identifier of the near-field communication tag is located in the read-only area of ​​the near-field communication tag, and the rolling token is located in the writable area of ​​the near-field communication tag. The read-only area and the writable area are physically isolated.

[0032] By adopting the above technical solution, the immutable identifier and the rewritable token are physically separated, so that the token can be updated on a rolling basis while the identifier is difficult to be arbitrarily rewritten.

[0033] Optionally, the rolling token is obtained by the server using the server key to perform a keyed hash on the hardware unique identifier, trigger time and space information and the incrementing sequence number. Two adjacent rolling tokens form a hash token chain.

[0034] By adopting the above technical solution, the hash token chain derived from the server key makes the tokens correlated and cannot be forged by the tag side.

[0035] Optionally, the incrementing sequence number is generated by the near-field communication tag or a monotonic counter on the server to prevent rollback.

[0036] By adopting the above technical solution, monotonic counting is used to prevent the token sequence number from being rolled back and replayed.

[0037] Optionally, the criteria for deviation grading include at least one of positioning distance, signal reception strength, and trigger time difference.

[0038] By adopting the above technical solutions, quantifiable criteria are provided for classifying sensing neighborhoods, scene neighborhoods, and cross-scenes.

[0039] Optionally, the upper bound of the response round-trip delay is adaptively calibrated by adding margin to the baseline round-trip delay of the terminal's historical legitimate triggers.

[0040] By adopting the above technical solution, the latency threshold for relay determination can be made adaptive to the actual network conditions of the terminal, thereby reducing misjudgments.

[0041] Optionally, environmental fingerprint matching allows for local mismatches below a set percentage.

[0042] By adopting the above technical solution, local mismatch can still be tolerated when there are slight changes in the environment, thus avoiding false rejection of legitimate triggers.

[0043] Optionally, the frozen near-field communication tag can be unfrozen after being reread by a legitimate terminal and verified through out-of-band identity verification.

[0044] By adopting the above technical solutions, a controlled unfreezing channel is provided for accidental freezing or legitimate rebinding.

[0045] Optionally, the sensing continuity fingerprint includes at least one of motion gait features based on an inertial measurement unit, barometric elevation features, and ambient light and sound features.

[0046] By adopting the above technical solutions, the continuity of presence can be characterized by multimodal sensing features, thereby improving the reliability of movement identification.

[0047] Optionally, the new presence credentials are obtained by the server using the previous presence credentials as the key to perform a keyed hash on the current sensing continuity fingerprint, and the presence credential chain constitutes the presence hash chain.

[0048] By adopting the above technical solution, the hash chain makes the present credentials related before and after, and the chain can be traced even if it breaks, making it difficult for forged credentials to be inserted midway.

[0049] Optionally, the comparison of continuous fingerprints can be based on a sequence similarity below a threshold as a breakage criterion.

[0050] By adopting the above technical solution, the boundary between continuity and discontinuity in the field is quantified by sequence similarity.

[0051] Optionally, the rate of change can be determined based on the continuity of acceleration and angular velocity from the motion sensor.

[0052] By adopting the above technical solution, the continuity of acceleration and angular velocity can be used to distinguish between normal user movement and session relocation.

[0053] Optionally, the digital human companion interaction service can be downgraded to suspending shopping guidance and transactions, and only retaining tours and Q&A.

[0054] By adopting the above technical solution, functions involving transactions can be shut down first when the credibility of the presence decreases, while basic services are preserved.

[0055] Optionally, the replay reconciliation after network recovery involves verifying the sequence of presence credentials cached locally during the terminal upload interruption one by one by the server.

[0056] By adopting the above technical solution, the presence credentials that are maintained locally on the terminal during weak network conditions can be re-verified by the server after the network is restored, thus maintaining the integrity of the home link.

[0057] Optionally, the device hardware root includes the key held by the terminal's security unit or trusted execution environment.

[0058] By adopting the above technical solutions, the hardware root of the device is carried by a secure unit or trusted execution environment, making it difficult to copy the invariants of the relocation determination.

[0059] Optionally, the attribution event includes at least one of the following: tour completion, shopping click, transaction payment, and content sharing.

[0060] By adopting the above technical solutions, multiple measurable attributable events in escort services can be covered.

[0061] Optionally, the attribution measurement is assigned at the first level to the near-field communication tag and digital human image corresponding to the chain head, and multi-level revenue sharing is performed according to the intermediate nodes through which the on-site credential chain passes.

[0062] By adopting the above technical solution, multi-level accounting along the chain of presence credentials can be supported based on attribution to the triggering source.

[0063] Optionally, fresh presence credentials are derived from current presence credentials, device hardware roots, and a summary of attribution events.

[0064] By adopting the above technical solution, the credentials of each attribution event are bound to the content of that event, making it difficult for them to be misappropriated to other events.

[0065] Optional, secondary environmental verification for strong presence verification includes visual and acoustic features of the location of the physical guide vehicle.

[0066] By adopting the above technical solution, secondary verification is performed at key attribution events using location and environmental characteristics, thereby enhancing the credibility of strong verification.

[0067] Optionally, the identification information is bound one-to-one with the digital human image resources and exclusive knowledge base, with different identification information corresponding to different digital human images and exclusive knowledge bases.

[0068] By adopting the above technical solutions, different physical guide carriers can trigger corresponding digital human images and exclusive knowledge content.

[0069] Optionally, digital human image resources include at least one of three-dimensional digital human models, two-dimensional digital humans, or voice digital humans.

[0070] By adopting the above technical solutions, it is possible to support the presentation of digital human images in various forms.

[0071] Optionally, the dedicated knowledge base includes tour guide knowledge, product knowledge, and frequently asked questions corresponding to the scene where the physical tour guide is located.

[0072] By adopting the above technical solutions, the content of the dedicated knowledge base can be matched with the triggering scenario.

[0073] Optionally, the terminal can be a smartphone, smart glasses, or a tablet.

[0074] By adopting the above technical solutions, various terminal forms with near-field communication capabilities can be supported.

[0075] Optionally, the server can be deployed on a cloud server or an edge computing node.

[0076] By adopting the above technical solutions, cloud or edge deployments can be supported to adapt to different latency and computing power requirements.

[0077] Optionally, the registration environment fingerprint of the near-field communication tag includes visual and acoustic features of the location of the physical guide carrier.

[0078] By adopting the above technical solution, a registration environment fingerprint is constructed using the visual and acoustic characteristics of the location, which is then used for relay protection comparison.

[0079] Secondly, this application provides a digital human companion interaction system based on near-field communication, which adopts the following technical solution: A digital human companion interaction system based on near-field communication includes a terminal and a server connected to each other via a communication network. The terminal includes a near-field triggering unit, an on-site sensing unit, and a digital human interaction unit. The near-field triggering unit is configured to read near-field communication tags deployed on the physical tour guide carrier via near-field communication and initiate a digital human scheduling request to the server, carrying identification information, triggering spatiotemporal information, and device fingerprints. The on-site sensing unit is configured to continuously collect the motion and environmental sensing continuity fingerprints of the terminal during the existence of the real-time audio-visual interaction channel. The digital human interaction unit is configured to load digital human image resources corresponding to the identification information, establish the real-time audio-visual interaction channel, and present the digital human through the channel to provide digital human companion interaction services such as tour guidance, shopping guidance, or Q&A. The server includes a trusted trigger verification unit, an on-site binding unit, and a digital human service unit. The trusted trigger verification unit is configured to perform trusted trigger verification based on the identification information, the trigger spatiotemporal information, and the device fingerprint. After the verification is successful, it generates a trusted trigger credential that binds the terminal and the trigger spatiotemporal information and sends access credentials for the digital human image resources and the exclusive knowledge base to the terminal. The on-site binding unit is configured to use the trusted trigger credential as the head of the chain, maintain the on-site credential chain based on the sensor continuity fingerprint, and maintain, downgrade, or suspend the digital human companion interaction service based on the on-site credential chain. The digital human service unit is configured to provide the guide, shopping guide, or Q&A content to the terminal through the channel based on the exclusive knowledge base. The digital human interaction unit interacts with the digital human service unit via the channel.

[0080] By adopting the above technical solution, the system uses functional units on both the terminal and the server to respectively undertake trigger collection, presence perception, image interaction and trigger verification, presence binding, and content supply. A data flow is formed between the units using trusted trigger credentials, sensor continuity fingerprints, and presence credential chains, so that the trusted triggering and session presence continuity trust chain of the aforementioned method can be implemented in the system structure. This enables the suppression of cloning and impersonation and session migration, as well as the measurement of service to the real trigger source, at the device level.

[0081] Optionally, the presence binding unit is also configured to use the terminal's device hardware root as the invariant for relocation determination and to distinguish between normal movement and session relocation by the mutation rate of the sensor continuity fingerprint. The system also includes an attribution metering unit. The presence binding unit provides the attribution metering unit with a presence credential chain. The attribution metering unit is configured to require each attribution event in the digital human escort interaction service to carry the current presence credential in the presence credential chain and the fresh presence credential derived from the device hardware root. After tracing the fresh presence credential back to the head of the presence credential chain and verifying the integrity of the attribution chain, the attribution metering of the attribution event is applied to the near-field communication tag corresponding to the head of the chain and the digital human image corresponding to the digital human image resource.

[0082] By adopting the above technical solution, an independent attribution metering unit is responsible for tracing and measuring attribution events back to their triggering sources, enabling the system to structurally possess the ability to accurately attribute services and revenues to their actual triggering sources.

[0083] In summary, this application includes at least one of the following beneficial technical effects: 1. This application uses a trusted trigger credential generated by a single touch as the head of the presence credential chain that runs through the session, extending the one-time judgment of the authenticity of the trigger to a continuous judgment of the session's ownership. This enables the near-field communication trigger-based digital human escort service to suppress cloned tag impersonation and session transfer and theft, and to stably measure the service and attribution to the true source of the trigger.

[0084] 2. This application improves the robustness of the triggering process against cloning and relay by binding the rolling token write-back with the token-tag identifier, dual-token transition fault tolerance, global clone diffusion adjudication, and relay protection, making the triggering credentials difficult to impersonate even when the physical tag can be copied, and keeping them usable even when near-field write-back fails occasionally.

[0085] 3. This application uses the relocation determination of device hardware root invariants and sensor continuity change rate, the service governance of weak network according to duration, and the revenue sharing mechanism of the head of the attribution event back-tracking presence credential chain to ensure that session presence is not misjudged during normal user movement and weak network environment, is suspended in time during real relocation, and realizes traceable multi-level attribution of revenue to the real triggering source. Attached Figure Description

[0086] Figure 1 This is a schematic diagram of the overall process of a digital human companion interaction method based on near-field communication in one embodiment of the present invention; Figure 2 This is a flowchart illustrating the trusted trigger verification, presence credential chain construction, and attribution process in one embodiment of the present invention. Figure 3 This is a schematic diagram of the composition of a digital human companion interaction system based on near-field communication in one embodiment of the present invention. Detailed Implementation

[0087] The present application will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the scope of the application.

[0088] This application discloses a digital human companion interaction method based on near-field communication.

[0089] This method uses the near-field communication (NFC) tag deployed on the physical guide carrier as the trigger entry point. The server performs a trusted trigger verification on this touch. After successful verification, the server generates a trusted trigger credential that binds the terminal and the trigger's spatiotemporal information, which serves as the head of the trust chain for the entire escort session. The server also issues the digital human image resource corresponding to the touched tag and access credentials for a dedicated knowledge base to the terminal. After the terminal loads the digital human image and establishes a real-time audio and video interaction channel with the server, the server uses this trust chain head as the starting point and continuously verifies whether the tourist is still present based on the sensor continuity fingerprint continuously reported by the terminal. This process sequentially generates a presence credential chain that can be traced back to the head of the chain, and uses this chain to maintain, downgrade, or suspend the digital human escort interaction service. When a tourist experiences an attribution event such as a guided tour, shopping assistance, or a transaction, the server verifies the attribution event along the presence credential chain and assigns it to the NFC tag and digital human image corresponding to the head of the chain. Thus, a single physical touch is extended into a continuous chain of trust that runs through the presence of the session, resolving the contradiction that trigger credentials can be cloned and that the presence of the session cannot be continuously verified after the trigger.

[0090] The following is a description of several terms used in the embodiments of this application.

[0091] Near Field Communication (NFC) tags are tags deployed on physical guide carriers for terminals to read via NFC. They carry identification information to identify the objects being touched and can be read or updated by the server.

[0092] A trusted trigger credential is a credential issued by the server for this trigger after the trusted trigger verification is passed. It binds the terminal that initiated this trigger with the trigger time and space information, carries the binding relationship between the trigger time and space information and the device fingerprint, and is used as the head of the present credential chain in subsequent sessions, hence it is also called the chain head.

[0093] The presence credential chain is a sequence of credentials generated by the server during the existence of the real-time audio and video interaction channel, with a trusted trigger credential as the head of the chain. Each presence credential in the chain can be traced back to the head of the chain. Once the continuity of presence is broken, the chain is invalidated.

[0094] Sensor continuity fingerprints are features that are continuously collected by the terminal during the real-time audio and video interaction channel, reflecting the movement status of the tourist holding the terminal and the continuity of the environment.

[0095] The device hardware root is a hardware-level key or identifier held by the terminal that is difficult to copy or migrate. In the embodiments of this application, it is used as an invariant to determine whether the current session has been moved.

[0096] The digital human image resources and the exclusive knowledge base are respectively used to present the digital human's image data on the terminal and the knowledge data to support the digital human's responses. Both correspond to the identification information of the near-field communication tag.

[0097] The application scenarios of this method are described below. This method can be applied to places with physical guide carriers, such as museums, scenic spots, exhibition halls, and commercial spaces. As an example scenario throughout the embodiments of this application, a visitor is touring a museum. The visitor's smartphone serves as the terminal, and a physical guide sign placed next to the exhibit serves as the physical guide carrier, with a near-field communication tag attached to it. When the visitor touches the guide sign with their smartphone, they can trigger a digital human guide corresponding to the exhibit. The digital human guide then provides the visitor with guided explanations of the exhibit, recommendations for related cultural and creative products, and Q&A services through a real-time audio and video interactive channel. Subsequent examples of each step follow this example scenario.

[0098] The overall flow of this method is described in detail below with reference to the accompanying drawings. This method includes steps S1 to S5.

[0099] S1. The terminal reads the near-field communication (NFC) tag deployed on the physical guide carrier via NFC, obtains the identification information carried by the NFC tag, and initiates a digital human scheduling request to the server. The digital human scheduling request carries the identification information, the terminal's trigger spatiotemporal information, and the terminal's device fingerprint. In this example scenario, a visitor touches the NFC tag on the guide sign next to the exhibit with their smartphone. The smartphone reads the identification information carried by the tag, which is used to identify the exhibit, and then initiates a digital human scheduling request to the server. In addition to the identification information, the request also carries the smartphone's trigger spatiotemporal information at the time of the touch and the device fingerprint used to identify the smartphone, so that the server can determine the credibility of the touch.

[0100] The aforementioned triggering spatiotemporal information may optionally include at least one of satellite positioning coordinates, floor or area identifier, and server-synchronized timestamp.

[0101] The aforementioned device fingerprint can be composed of at least two of the following: the terminal's hardware identifier, operating system characteristics, and near-field communication module characteristics. This allows the server to identify whether the terminal submitting the current trigger is consistent with the terminals that have been legally triggered in the past.

[0102] S2. The server performs a trusted trigger verification based on the identification information, trigger spatiotemporal information, and device fingerprint. Upon successful verification, the server generates a trusted trigger credential that binds the terminal and trigger spatiotemporal information, and sends the digital human avatar resource corresponding to the identification information and access credentials for the exclusive knowledge base to the terminal. In this example scenario, after receiving a digital human scheduling request from a smartphone, the server comprehensively determines whether the touch is a genuine touch initiated by the visitor at the exhibit site using their own terminal, based on the identification information, trigger spatiotemporal information, and device fingerprint carried in the request. If the determination is successful, the server issues a trusted trigger credential for the touch. This credential binds the smartphone that initiated the touch and the trigger spatiotemporal information of the touch, serving as the head of the subsequent session presence credential chain. The server then sends the avatar resource of the digital human guide corresponding to the exhibit and access credentials for the exhibit's exclusive knowledge base to the smartphone. In some embodiments, the server generates the trusted trigger credential using the following formula. :

[0103] in, The signing key held by the server. For identification information, To trigger spatiotemporal information, For device fingerprint, The timestamp for when the server issues the credential. This indicates bitwise concatenation. This represents a keyed signature operation. The resulting trusted triggering credential binds the triggering terminal and triggering time and space to the credential, preventing it from being used on other terminals or in other time and space sessions. The specific algorithm for the signature operation and the key length can be selected according to the security level.

[0104] In some embodiments, the trusted trigger verification of S2 includes rolling token verification, which includes sub-steps S21 to S23.

[0105] S21. The server extracts the scroll token currently carried in the writable area of ​​the near-field communication tag from the identification information, and verifies whether the scroll token is a valid token written after the most recent legitimate interaction of the near-field communication tag and has not been marked as used. In this example scenario, the server retrieves the scroll token read by the smartphone and sent along with the identification information, and compares it with the current token recorded by the server for the exhibit tag after the last legitimate touch, and whether the token has not yet been consumed.

[0106] S22. In response to the successful verification of the rolling token, the server marks the rolling token as used and generates a new rolling token based on the hardware unique identifier of the near-field communication tag, the current trigger spatiotemporal information, and the incrementing sequence number. The new rolling token is sent to the terminal along with the trusted trigger credentials. In some embodiments, the server generates the first rolling token using the following formula: A rolling token :

[0107] in, This serves as the unique hardware identifier for the near-field communication tag. For the spatiotemporal information triggered this time, For incrementing sequence numbers, This indicates a keyed hash operation; two consecutive rolling tokens. and Because of shared keys These tokens are linked sequentially by their recursive sequence numbers to form a hash token chain. The hash algorithm and key length can be selected based on the security level. In other embodiments, new rolling tokens can also be randomly generated by the server, which maintains a mapping table between near-field communication tag identifiers and the current token, without using hash derivation. This method also achieves the goal of unpredictably updating the token with each legitimate touch.

[0108] S23. The terminal performs a write-back, writing the new scrolling token to the writable area of ​​the near-field communication tag via near-field communication, which will serve as the token to be verified for the next trigger of the near-field communication tag. In this example scenario, the smartphone writes the new scrolling token issued by the server back to the writable area of ​​the exhibit tag at the same time as the session is established, so that the token carried by the tag is updated with each legitimate touch.

[0109] Scrolling token verification also includes clone tag verification. The server performs a consistency check between the hardware unique identifier bound to the scrolling token and the actual hardware unique identifier of the near-field communication tag that submitted the scrolling token. If the two are inconsistent, the tag is determined to be a clone and the trigger is rejected. For example, an attacker copies the current scrolling token in the writable area of ​​an exhibit tag, but the clone tag used has a different hardware unique identifier. During verification, the server finds that the scrolling token was originally generated based on the hardware unique identifier of the genuine tag, which does not match the actual hardware unique identifier of the tag submitted this time. Therefore, it determines that the tag is a clone and the trigger is rejected. The specific bit width and reading method of the hardware unique identifier depend on the tag model used.

[0110] The hardware unique identifier of the near-field communication (NFC) tag can optionally be located in the read-only area of ​​the NFC tag, while the rolling token is located in the writable area. The read-only area and the writable area are physically isolated, making the hardware unique identifier difficult to rewrite, while the rolling token can be updated sequentially. The aforementioned incrementing sequence number can be generated by a monotonic counter on the NFC tag or the server to prevent sequence number rollback; when an attacker attempts to replay with a token of an old sequence number, the attempt is rejected because the monotonic counter does not accept rolled-back sequence numbers.

[0111] In other embodiments, considering that near-field write-back may lose synchronization due to the terminal leaving too quickly or communication interruption, the server maintains a transition token pair containing the current token and the previous token for the near-field communication tag, and performs transition fault tolerance accordingly. In S21, the server compares the extracted rolling token with the transition token pair. If the rolling token matches any token in the transition token pair, it is determined that the verification has passed. If the terminal reports that the write-back is not completed, or if the server detects that the near-field communication tag still carries the previous token in the next trigger, the server enters the transition fault tolerance window, accepts the previous token in the transition fault tolerance window, but requires that the spatiotemporal information and device fingerprint of this trigger are consistent with the first trigger. If the terminal successfully completes the write-back in the next interaction, the server converges the transition token pair into a transition token pair with the most recently written token as the current token. For example, if a visitor touches the tag but leaves before completing the write-back, the tag will still carry the previous token upon the next touch. The server accepts the previous token within the transitional fault-tolerance window and verifies that the trigger time and device fingerprint match the initial one before allowing access. Once the write-back is successful, the transitional token pair is converged. The specific duration of the transitional fault-tolerance window can be set based on the typical time consumption of near-field write-backs, for example, on the order of several seconds. This prevents legitimate visitors from being continuously and falsely rejected due to a single write-back misalignment. In some embodiments, write-back misalignment can also be assisted by the terminal reporting a write-back receipt to the server after completion. The server determines whether to enter the transitional fault-tolerance window based on the presence and timing of the receipt. This method also achieves fault tolerance for write-back misalignment.

[0112] To address the scenario where near-field communication (NFC) tags are cloned and disseminated across multiple locations, the transition token is bound to the device fingerprint and trigger spatiotemporal information of the device that first triggers the NFC tag. Trusted trigger verification also includes a cloning and dissemination adjudication performed by the server. If a rolling token with the same incrementing sequence number is submitted at multiple terminals with inconsistent device fingerprints, or submitted under multiple physically unreachable trigger spatiotemporal information, the server determines that the rolling token has forked, the NFC tag has been cloned and disseminated, freezes the triggering of the NFC tag, and requires rebinding. The server also categorizes triggers into three levels based on their deviation from the NFC tag's registration location: sensing neighborhood, scene neighborhood, and cross-scene. Triggers within the sensing neighborhood are allowed; triggers within the scene neighborhood require secondary on-site verification; and cross-scene triggers are frozen. For example, if the same serial number token for an exhibit is submitted almost simultaneously in two locations far apart, and the device fingerprints of the two submissions are different, the server determines that the tag has been cloned and spread, and freezes its trigger. Similarly, touches occurring near the exhibit are allowed based on the sensing neighborhood, touches occurring in other exhibition areas within the same hall require secondary on-site verification based on the scene neighborhood, and touches occurring outside the hall are frozen as cross-scene. The specific distance thresholds for the three neighborhood levels can be set according to the scale of the venue; for example, the sensing neighborhood is based on the near-field physical distance, the scene neighborhood is based on the scale of a single hall, and anything exceeding this is considered cross-scene. The criteria for the above deviation grading can optionally include at least one of positioning distance, signal reception strength, and trigger time difference. Frozen near-field communication tags can be unfrozen after being reread by a legitimate terminal and verified through out-of-band identity verification. In other embodiments, the identification of cloning and spread can also be accomplished by the server performing offline reconciliation and abnormal risk control modeling of the trigger records of each tag in the background, and freezing tags exhibiting cloning and spread characteristics in batches. This method also achieves the identification and handling of cloning and spread.

[0113] The above describes a trusted trigger verification method using rolling token verification. In other embodiments, trusted trigger verification can also be implemented based on a digital certificate embedded in the near-field communication tag. Specifically, the tag contains a digital certificate and corresponding private key issued by the issuer. When the tag is touched, it signs a random challenge value issued by the server using its private key. The server verifies the signature using the certificate's public key. If the verification is successful, the tag is considered authentic and the touch is trusted. This method does not rely on the rolling write-back of tokens and still verifies the authenticity of the touch; therefore, it also belongs to the trusted trigger verification implementation method.

[0114] Trusted trigger verification may also include relay protection. Upon triggering, the terminal collects the environmental fingerprint of the near-field communication tag's location and uploads it along with the digital human scheduling request. The server compares the environmental fingerprint with the registered environmental fingerprint of the near-field communication tag. The server also determines whether the trigger was relayed based on the upper bound of the round-trip time between the triggering terminal and the server, combined with the physical distance constraints of near-field communication. If the environmental fingerprint mismatch or the round-trip time exceeds the upper bound, the trigger is rejected. For example, if an attacker in a different location relays a touch signal from an exhibit tag, and the uploaded environmental fingerprint does not match the visual and acoustic characteristics recorded during tag registration, and the relaying of the signal causes the round-trip time to significantly exceed the upper bound, the server determines that the trigger was relayed and rejects it. The specific value of the upper bound of the round-trip time and the comparison method with the environmental fingerprint can be determined according to the site conditions. The upper bound of the above response round-trip delay can be adaptively calibrated by adding a margin to the baseline round-trip delay of the terminal's historical legal triggers; the comparison of the above environmental fingerprint can allow local mismatches below a set ratio to tolerate normal fluctuations in the ambient lighting and acoustic environment; the registration environmental fingerprint of the above near-field communication tag may optionally include the visual and acoustic features of the location of the physical guide carrier.

[0115] The identification information is bound to digital human avatar resources and a dedicated knowledge base, with different identification information corresponding to different digital human avatars and dedicated knowledge bases. In this example scenario, the guide label for each exhibit carries its own identification information, which corresponds to the digital human guide avatar for that exhibit and the guide, merchandise, and Q&A knowledge for that exhibit. Thus, when visitors touch the guide labels for different exhibits, they are connected to different digital human guides and different knowledge content.

[0116] S3. The terminal loads the digital human avatar resource and establishes a real-time audio-visual interaction channel between the terminal and the server. In this example scenario, after receiving the digital human guide avatar resource and exclusive knowledge base access credentials from the server, the smartphone loads the guide avatar and establishes a real-time audio-visual interaction channel with the server to support subsequent real-time voice and video interaction between tourists and the digital human guide. The aforementioned digital human avatar resource may optionally include at least one of a 3D digital human model, a 2D digital human, or a voice digital human; for example, a 2D digital human or a voice digital human can be used on a terminal with limited computing power, while a 3D digital human model can be used on a terminal that supports 3D rendering.

[0117] S4. During the duration of the real-time audio and video interaction channel, the server continuously verifies the presence continuity of the terminal using a trusted trigger credential as the head of the chain, obtaining a presence credential chain. Based on this chain, the server maintains, downgrades, or suspends the digital human companion interaction service carried by the real-time audio and video interaction channel. The problem this step addresses is: after the touch verification is passed and the session is established, whether the terminal holder is still the same visitor who initiated the touch at the exhibit site, and whether they are still present, needs to be continuously confirmed throughout the session; otherwise, the session may be transferred or impersonated midway. Therefore, the server does not stop trust at the moment of touch, but instead uses the trusted trigger credential as the starting point to continuously verify the visitor's presence continuity during the session, obtaining a presence credential chain sequentially, and determining whether to maintain the full service, downgrade the service, or suspend the session based on this chain. In this example scenario, while a visitor stops to view an exhibit and interacts with the digital human guide, their presence is continuously confirmed, and the service continues normally. When the visitor gets up to move on to the next exhibit, the service continues as long as they remain present. Once the terminal is handed over to someone else or taken away, the continuity of presence is interrupted, and the service is downgraded or suspended accordingly. In other embodiments, presence continuity verification can also be achieved through a two-way heartbeat mechanism where the server periodically initiates challenges to the terminal during the session, and the terminal responds by combining its credentials and sensor status. The server determines whether presence continues based on the continuity of the heartbeats, which also achieves continuous confirmation of session presence.

[0118] The presence credential chain uses a trusted trigger credential as its head. The trusted trigger credential carries the binding of triggering spatiotemporal information and device fingerprint. In some embodiments, presence continuity verification includes sub-steps S41 to S43.

[0119] S41. During the real-time audio and video interaction channel, the terminal continuously collects motion and environmental sensing continuity fingerprints. This continuous sensing fingerprint may optionally include at least one of motion gait characteristics based on an inertial measurement unit, barometric pressure-elevation characteristics, and ambient light and sound characteristics, collected by the terminal at certain intervals as a time-progressing feature sequence.

[0120] S42. The server compares the sensor continuity fingerprint with the sensor continuity fingerprint bound to the previous presence credential in the presence credential chain. Optionally, this comparison uses a sequence similarity lower than a threshold as the break criterion. That is, when the similarity between the sensor sequence collected this time and the sensor sequence bound to the previous presence credential is lower than a set threshold, it is determined that the presence continuity has been broken.

[0121] S43. In response to a continuous sensor fingerprint, the server generates a new presence credential based on the previous presence credential, the current continuous sensor fingerprint, and the binding carried at the chain head, and continues it to the presence credential chain, so that each presence credential in the presence credential chain can be traced back to the chain head; in response to a broken continuous sensor fingerprint, the server determines that the current session of the real-time audio and video interaction channel has been moved, invalidates the presence credential chain, and requires the terminal to reread the near-field communication tag to rebuild the chain head. In this way, each presence credential is linked together with the chain head through the previous presence credential, and any link in the chain can be traced back to the touch that was verified by trusted trigger; once the continuity is broken, the entire chain is invalidated, and the session must be rebuilt by the visitor touching the exhibit tag again and re-verifying by trusted trigger. In some embodiments, the new presence credential is obtained by the server using the previous presence credential as the key and performing a keyed hash on the current sensing continuity fingerprint. This associates adjacent presence credentials, forming a presence hash chain, with the head of the chain being the trusted trigger credential. Any forgery or insertion of credentials in the chain will be detected because it does not match the hash relationship of the previous presence credential. The hash algorithm can be selected according to the security level. In other embodiments, the presence credential chain can also employ a signature chain with a server signature attached to each presence credential, or organize the presence credentials into a Merkle chain to achieve traceability and tamper-proofing, similarly achieving the goal of allowing each presence credential to be traced back to the head of the chain.

[0122] In some embodiments, the presence continuity verification uses the terminal's device hardware root as the invariant for relocation determination, and the sensor continuity fingerprint as a tolerable auxiliary criterion. The device hardware root may optionally include the key held by the terminal's security unit or trusted execution environment. Because it is difficult to copy or migrate, it is suitable as an invariant for determining whether the session is still on the original terminal. In S43, the server distinguishes between normal terminal movement and session relocation based on the mutation rate of the sensor continuity fingerprint. This mutation rate may optionally be based on the continuity determination of acceleration and angular velocity of motion sensors. The server determines that the current session has been moved only when the device hardware root changes, or the mutation rate of the sensor continuity fingerprint exceeds the range of the corresponding normal movement and the presence continuity verification of the terminal is lost at the same time. For example, when a visitor walks normally to the next exhibit, the device hardware root remains unchanged and the sensor mutation rate is within the normal movement range, so the service determines it as normal movement and maintains the service; however, when the terminal is taken away by someone else, the device hardware root changes, or the sensor exhibits a snatching-like mutation and the presence verification is lost at the same time, the service determines it as session relocation. The range of abrupt change rates that distinguishes between normal movement and transport can be calibrated by combining the sensor characteristics of actual walking and transport.

[0123] In response to a network outage on which the on-site continuity verification relies, but the device hardware root and sensor continuity fingerprint remain continuous, the terminal maintains the digital human companion interaction service based on the most recent on-site credential cached locally, and replays the reconciliation with the server after the network is restored. On-site continuity verification is handled in stages according to the duration of the network outage: the digital human companion interaction service is maintained for the first duration; for a second duration longer than the first, the digital human companion interaction service is downgraded; and if the second duration is exceeded or the sensor continuity fingerprint is broken, the real-time audio and video interaction channel is suspended and on-site continuity verification is required to be re-executed. The downgrading of the digital human companion interaction service can optionally involve suspending shopping guides and transactions, retaining only the guide and Q&A functions, prioritizing the closure of transaction-related functions when on-site credibility decreases; the replay reconciliation after network restoration can optionally involve the terminal uploading the on-site credential sequence cached locally during the outage, with the server verifying each credential one by one, to ensure that the attribution link of the on-site credential chain is not interrupted during weak network conditions. For example, if a visitor experiences a brief network interruption while in a signal dead zone of the exhibition hall, the terminal can maintain the guided tour for the first duration using locally cached presence credentials. If the interruption extends to a second duration, the service is downgraded to only retaining the guided Q&A session, suspending the introduction and purchase of cultural and creative products. After the network is restored, the terminal uploads the sequence of presence credentials cached during the dead zone for the server to verify one by one. The specific values ​​of the first and second durations can be set according to the tolerance for continuity in the application scenario. In other embodiments, whether the session has been moved can also be determined by combining the handover record of the base station cell accessed by the terminal or the fingerprint of the wireless access point at the location, as a supplementary criterion in addition to the sensing continuity fingerprint, thus also achieving the distinction between normal movement and session migration.

[0124] S5. The server, based on a dedicated knowledge base, provides digital human-assisted interactive services such as guided tours, product recommendations, or Q&A to the terminal via a real-time audio-visual interaction channel. In this example scenario, the digital human guide, based on the dedicated knowledge base corresponding to the exhibit it touches, provides real-time explanations of the exhibit, recommends related cultural and creative products, and answers tourists' questions via the real-time audio-visual interaction channel. The aforementioned dedicated knowledge base may optionally include guide knowledge, product knowledge, and frequently asked questions corresponding to the scene where the physical guide is located, thus limiting the digital human guide's responses to the knowledge scope corresponding to that exhibit. The overall flow of this method is as follows: Figure 1 As shown.

[0125] To ensure that revenue generated from digital human companion interaction services, such as guided tours, shopping guides, and transactions, is accurately attributed to the actual source of contact, in some embodiments, S5 measures and assigns attribution events in the digital human companion interaction service. The server requires each attribution event to carry, upon occurrence, the current presence credential from the presence credential chain and a fresh presence credential derived from the device hardware root. This fresh presence credential is traced back along the presence credential chain to the head, verifying that the attribution link of the attribution event is complete and uninterrupted from the head. Based on this, the attribution event is measured to the near-field communication tag corresponding to the head of the chain and the digital human image corresponding to the digital human image resource. In response to a key attribution event such as a transaction or payment, the server requires the terminal to reread the near-field communication tag and perform a strong presence verification for secondary environmental confirmation. The aforementioned attribution events may optionally include at least one of guided tour completion, shopping guide click, transaction payment, and content sharing. This fresh presence credential can optionally be derived from the current presence credential, the device hardware root, and a summary of the attribution event, binding the credential to the current attribution event and making it difficult to transfer to other events. The aforementioned secondary environmental confirmation of the key attribution event can optionally include the visual and acoustic characteristics of the physical guide vehicle's location; that is, before a transaction, visitors are required to touch the exhibit label again and verify the surrounding environment to confirm that the transaction indeed occurred at the exhibit's location.

[0126] Because each attribution event must be verified by tracing back along the presence credential chain, and this chain is interconnected from the head, the attribution event can only be attributed to the tag and digital human image corresponding to the genuine touch that has been verified through a trusted trigger. Even if others obtain the session content, they cannot claim the benefits for themselves. In terms of attribution measurement, in some embodiments, it is first-level attribution to the near-field communication tag and digital human image corresponding to the head of the chain, and multi-level revenue sharing is performed according to the intermediate nodes traversed by the presence credential chain. For example, after a visitor is guided by a digital human guide in front of an exhibit, and then completes a purchase of cultural and creative products on the terminal based on the guide's recommendation, the server requires that the transaction event carry a fresh presence credential derived from the current presence credential and the device hardware root. After tracing back along the chain to the head and verifying the completeness of the attribution chain, the transaction is first-level attributed to the guide sign tag of the exhibit and the corresponding digital human guide. If the visitor's visit involves several intermediate steps, such as touching the guide sign at the entrance before being guided to the exhibit, then multi-level revenue sharing is performed on the transaction according to the intermediate nodes traversed by the presence credential chain. The specific revenue sharing ratio for each level in a multi-level revenue sharing scheme can be set according to operational rules. In other embodiments, the attribution and revenue sharing of attribution events can also be recorded in blockchain notarization or trusted logs on the server side to provide tamper-proof attribution credentials, thus achieving the goal of accurately attributing attribution events to the true source of the touch. The process of trusted trigger verification, on-site credential chain construction, and attribution is as follows: Figure 2 As shown.

[0127] The methods of the embodiments of this application have been described above. Based on the same inventive concept, embodiments of this application also disclose a digital human companion interaction system triggered by near-field communication, the system including a terminal and a server connected to each other via a communication network.

[0128] The terminal includes a near-field triggering unit, a presence sensing unit, and a digital human interaction unit. The near-field triggering unit is configured to read near-field communication tags deployed on the physical tour guide carrier via near-field communication and initiate a digital human scheduling request to the server, carrying identification information, trigger spatiotemporal information, and device fingerprints, corresponding to S1 mentioned above. The presence sensing unit is configured to continuously collect the terminal's motion and environmental sensing continuity fingerprints during the existence of the real-time audio-visual interaction channel, corresponding to the terminal-side collection action in S41 mentioned above. The digital human interaction unit is configured to load digital human image resources corresponding to the identification information, establish a real-time audio-visual interaction channel, and present the digital human through the real-time audio-visual interaction channel to provide digital human companion interaction services such as tour guidance, shopping assistance, or Q&A, corresponding to S3 mentioned above and the interaction between tourists and the digital human.

[0129] The server-side includes a trusted trigger verification unit, an on-site binding unit, and a digital human service unit. The trusted trigger verification unit is configured to perform trusted trigger verification based on identification information, trigger spatiotemporal information, and device fingerprint. Upon successful verification, it generates a trusted trigger credential binding the terminal and trigger spatiotemporal information, and issues access credentials for the digital human's image resources and a dedicated knowledge base to the terminal, corresponding to S2 above. The on-site binding unit is configured to use the trusted trigger credential as the head of the chain, maintain the on-site credential chain based on sensor continuity fingerprints, and maintain, downgrade, or suspend the digital human companion interaction service based on the on-site credential chain, corresponding to S4 above. The digital human service unit is configured to provide the terminal with guided tours, shopping guides, or Q&A content via a real-time audio-visual interaction channel based on a dedicated knowledge base, corresponding to S5 above.

[0130] Among the aforementioned units, the trusted trigger verification unit provides trusted trigger credentials to the on-site binding unit, the on-site perception unit provides sensing continuity fingerprints to the on-site binding unit, and the digital human interaction unit interacts with the digital human service unit via a real-time audio-visual interaction channel. The composition of the above system is as follows: Figure 3 As shown. The specific actions and limitations performed by each unit are the same as those described in the corresponding method steps above, and will not be repeated here.

[0131] In some embodiments, the presence binding unit is further configured to use the terminal's device hardware root as an invariant for relocation determination and to distinguish between normal movement and session relocation by the mutation rate of the sensor continuity fingerprint. The system also includes an attribution measurement unit, which provides a presence credential chain to the attribution measurement unit. The attribution measurement unit is configured to require each attribution event in the digital human companion interaction service to carry a fresh presence credential derived from the current presence credential in the presence credential chain and the device hardware root. After tracing the fresh presence credential back to the head of the presence credential chain and verifying the integrity of the attribution chain, the attribution measurement of the attribution event is assigned to the near-field communication tag corresponding to the head of the chain and the digital human image corresponding to the digital human image resource. The specific limitations of its relocation determination and attribution measurement are the same as those described in the corresponding steps of the method above.

[0132] The aforementioned terminals can be smartphones, smart glasses, or tablets; the aforementioned servers can be deployed on cloud servers or edge computing nodes.

[0133] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0134] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A digital human companion interaction method based on near-field communication triggering, characterized in that, Includes the following steps: S1. The terminal reads the near-field communication tag deployed on the physical guide carrier through near-field communication, obtains the identification information carried by the near-field communication tag, and initiates a digital human scheduling request to the server. The digital human scheduling request carries the identification information, the triggering spatiotemporal information of the terminal, and the device fingerprint of the terminal. S2. The server performs a trusted trigger verification on the current trigger based on the identification information, the triggering time and space information and the device fingerprint. In response to the trusted trigger verification being passed, it generates a trusted trigger credential that binds the terminal and the triggering time and space information, and sends access credentials for the digital human image resources and the exclusive knowledge base corresponding to the identification information to the terminal. S3. The terminal loads the digital human image resource and establishes a real-time audio and video interaction channel between the terminal and the server; S4. During the existence of the real-time audio and video interaction channel, the server continuously performs presence continuity verification on the terminal using the trusted trigger credential as the chain head to obtain a presence credential chain, and maintains, downgrades, or suspends the digital human companion interaction service carried by the real-time audio and video interaction channel based on the presence credential chain; the trusted trigger credential carries the binding of the trigger spatiotemporal information and the device fingerprint, and the presence continuity verification includes the following sub-steps: S41. The terminal continuously collects motion and environmental sensing continuous fingerprints of the terminal during the existence of the real-time audio and video interaction channel; S42. The server compares the sensor continuity fingerprint with the sensor continuity fingerprint bound to the previous presence credential in the presence credential chain; S43. In response to the continuity of the sensing fingerprint, the server generates a new presence credential based on the previous presence credential, the current sensing fingerprint, and the binding carried by the chain head, and continues it to the presence credential chain, so that each presence credential in the presence credential chain can be traced back to the chain head; in response to the breakage of the sensing fingerprint, the server determines that the current session of the real-time audio and video interaction channel has been moved, invalidates the presence credential chain, and requests the terminal to reread the near-field communication tag to rebuild the chain head; S5. The server provides the digital human companion interactive service, which includes navigation, shopping guidance, or Q&A, to the terminal based on the exclusive knowledge base and through the real-time audio and video interaction channel.

2. The digital human companion interaction method based on near-field communication triggered according to claim 1, characterized in that, In step S2, the trusted trigger verification includes rolling token verification, which includes the following sub-steps: S21. The server extracts the scroll token currently carried in the writable area of ​​the near-field communication tag from the identification information, and verifies whether the scroll token is a valid token written after the most recent legitimate interaction of the near-field communication tag and not marked as used. S22. In response to the scrolling token passing the verification, the server marks the scrolling token as used and generates a new scrolling token based on the hardware unique identifier of the near-field communication tag, the current triggering spatiotemporal information and the incrementing sequence number. The new scrolling token is sent to the terminal along with the trusted triggering credential. S23. The terminal performs a write-back, writing the new rolling token into the writable area of ​​the near-field communication tag via near-field communication, as the token to be verified for the next trigger of the near-field communication tag; The server also performs a consistency check between the hardware unique identifier bound to the rolling token and the actual hardware unique identifier of the near-field communication tag. If the two are inconsistent, it is determined to be a cloned tag and the trigger is rejected.

3. The digital human companion interaction method based on near-field communication triggered according to claim 2, characterized in that, The server maintains a transition token pair containing the current token and the previous token for the near-field communication tag; in S21, the server compares the extracted rolling token with the transition token pair, and in response to the rolling token matching any token in the transition token pair, determines that the rolling token passes the verification. In response to the terminal reporting that the write-back is not complete, or the server detecting that the near-field communication tag still carries the previous token during the next trigger, the server enters a transitional fault-tolerant window, accepts the previous token within the transitional fault-tolerant window, but requires that the triggering spatiotemporal information and the device fingerprint be consistent with the first trigger; In response to the next successful interaction of the terminal to complete the write-back, the server will converge the transition token pair into a transition token pair with the most recently written token as the current token.

4. The digital human companion interaction method based on near-field communication triggered according to claim 3, characterized in that, The transition token is bound to the device fingerprint and the triggering spatiotemporal information of the first time the near-field communication tag is triggered; The trusted trigger verification also includes a clone diffusion decision executed by the server. The clone diffusion decision includes: in response to the rolling token with the same incrementing sequence number being submitted at multiple terminals with inconsistent device fingerprints, or being submitted under multiple physically unreachable trigger spatiotemporal information, the server determines that the rolling token has forked and the near-field communication tag has been cloned and diffused, freezes the trigger of the near-field communication tag, and requires rebinding; the server also classifies the trigger spatiotemporal information into three levels according to the deviation of the trigger spatiotemporal information from the registration position of the near-field communication tag: sensing neighborhood, scene neighborhood, and cross-scene, allowing triggers within the sensing neighborhood, requiring secondary on-site verification for triggers within the scene neighborhood, and freezing cross-scene triggers.

5. The digital human companion interaction method based on near-field communication triggered according to claim 2, characterized in that, The trusted trigger verification also includes relay protection, which includes: when the terminal triggers, it collects the environmental fingerprint of the location of the near-field communication tag and sends the environmental fingerprint along with the digital human scheduling request; the server compares the environmental fingerprint with the registered environmental fingerprint of the near-field communication tag; the server also determines whether the trigger is relayed based on the upper bound of the round-trip time between the terminal and the server for this trigger, combined with the physical distance constraint of near-field communication, and rejects the trigger if the environmental fingerprint is mismatched or the round-trip time exceeds the upper bound.

6. The digital human companion interaction method based on near-field communication triggered according to claim 1, characterized in that, The presence continuity verification uses the terminal's device hardware root as the invariant for relocation determination, and the sensor continuity fingerprint as a tolerant auxiliary criterion; in S43, the server distinguishes between normal terminal movement and session relocation based on the mutation rate of the sensor continuity fingerprint. Only when the device hardware root changes, or the mutation rate of the sensor continuity fingerprint exceeds the range of the corresponding normal movement and the presence continuity verification of the terminal is lost at the same time, is the current session determined to be relocated; in response to the network on which the presence continuity verification depends being interrupted but the device hardware root and the sensor continuity fingerprint are still continuous, the terminal maintains the digital human companion interaction service based on the most recent presence credential cached locally, and replays the reconciliation to the server after the network is restored; The presence continuity verification is handled in stages according to the duration of the network interruption. The digital human companion interaction service is maintained during the first duration, the digital human companion interaction service is downgraded during the second duration which is longer than the first duration, and the real-time audio and video interaction channel is suspended and the presence continuity verification is required to be re-executed when the second duration is exceeded or the sensor continuity fingerprint is broken.

7. The digital human companion interaction method based on near-field communication triggered according to claim 6, characterized in that, In step S5, for the measurement and attribution of attribution events in the digital human companion interaction service, the server requires each attribution event to carry the current presence credential from the presence credential chain and the fresh presence credential derived from the device hardware root when it occurs. The server traces the fresh presence credential back along the presence credential chain to the head of the chain, verifies that the attribution link of the attribution event is complete and uninterrupted from the head of the chain, and accordingly measures the attribution of the attribution event to the near-field communication tag corresponding to the head of the chain and the digital human image corresponding to the digital human image resource. In response to the attribution event being a transaction or payment-related key attribution event, the server requires the terminal to reread the near-field communication tag and perform a strong presence verification for secondary environmental confirmation.

8. A digital human companion interaction system based on near-field communication triggering, characterized in that, This includes terminals and servers connected to each other via a communication network; The terminal includes a near-field triggering unit, an on-site sensing unit, and a digital human interaction unit. The near-field triggering unit is configured to read near-field communication tags deployed on the physical tour guide carrier via near-field communication and initiate a digital human scheduling request to the server, carrying identification information, triggering spatiotemporal information, and device fingerprints. The on-site sensing unit is configured to continuously collect the motion and environmental sensing continuity fingerprints of the terminal during the existence of the real-time audio-visual interaction channel. The digital human interaction unit is configured to load digital human image resources corresponding to the identification information, establish the real-time audio-visual interaction channel, and present the digital human through the channel to provide digital human companion interaction services such as tour guidance, shopping guidance, or Q&A. The server includes a trusted trigger verification unit, an on-site binding unit, and a digital human service unit. The trusted trigger verification unit is configured to perform trusted trigger verification based on the identification information, the trigger spatiotemporal information, and the device fingerprint. Upon successful verification, it generates a trusted trigger credential binding the terminal and the trigger spatiotemporal information and issues access credentials for the digital human image resources and exclusive knowledge base to the terminal. The on-site binding unit is configured to use the trusted trigger credential as the head of a chain, maintain an on-site credential chain based on the sensor continuity fingerprint, and maintain, downgrade, or suspend the digital human companion interaction service based on the on-site credential chain. The on-site binding unit is also configured to... The fingerprint is compared with the sensor continuity fingerprint bound to the previous presence credential in the presence credential chain. In response to the continuity of the sensor continuity fingerprint, a new presence credential is generated based on the previous presence credential, the current sensor continuity fingerprint, and the binding carried by the chain head, and is continued to the presence credential chain, so that each presence credential in the presence credential chain can be traced back to the chain head. In response to the breakage of the sensor continuity fingerprint, it is determined that the current session has been moved, the presence credential chain is invalidated, and the terminal is required to reread the near-field communication tag to rebuild the chain head. The digital human service unit is configured to provide the guide, shopping guide, or Q&A content to the terminal through the channel based on the exclusive knowledge base. The digital human interaction unit interacts with the digital human service unit via the channel.

9. The digital human companion interaction system based on near-field communication triggering according to claim 8, characterized in that, The presence binding unit is further configured to use the device hardware root of the terminal as the invariant for relocation determination and to distinguish between normal movement and session relocation by the mutation rate of the sensing continuity fingerprint; the system also includes an attribution measurement unit, which is configured to require each attribution event in the digital human companion interaction service to carry a fresh presence credential derived from the current presence credential of the presence credential chain and the device hardware root. After tracing the fresh presence credential back to the head of the presence credential chain and verifying the integrity of the attribution link, the attribution measurement of the attribution event is assigned to the near-field communication tag corresponding to the head of the chain and the digital human image corresponding to the digital human image resource.

Images