A non-contact human-computer interaction method and system based on non-biological feature information data
By employing a non-contact human-computer interaction method based on non-biological feature data, combined with geometric ray projection and semantic pose recognition, the problems of privacy leakage and unreasonable device functions in AR/VR devices have been solved, achieving an efficient and secure interactive experience and device optimization.
Patent Information
- Application Number
- CN202611255666.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-18
- Publication Date
- 2026-11-06
AI Technical Summary
Existing AR/VR devices rely on biometric recognition for interaction, which poses risks of privacy leaks and has poor compliance. Furthermore, the division of functions within these devices is unreasonable, resulting in a poor user experience.
It uses non-biological feature information data for environmental perception and interaction, locks onto targets through geometric ray projection, combines semantic pose recognition and intent completion, and leverages the collaborative work of wearable devices and mobile communication devices to achieve contactless interaction, while processing user data locally to ensure privacy and security.
It achieves efficient, privacy-preserving, and secure contactless human-computer interaction, meets compliance requirements, improves user experience, and optimizes the division of device functions.
Abstract
Description
Technical Field
[0001] This invention relates to the field of human-computer interaction technology, and in particular to a non-contact human-computer interaction method and system based on non-biological feature information data, applicable to the fusion and coexistence of AR / VR wearable devices and mobile communication devices. Background Technology
[0002] With the development of AR / VR technology, devices such as smart glasses are emerging. However, current mainstream interaction methods mostly rely on biometric recognition technologies such as eye tracking, fingerprints, or facial recognition. These technologies pose privacy risks and have poor compliance in specific scenarios (such as healthcare and government / enterprise use). Furthermore, existing devices attempt to cram smartphone functions into glasses, resulting in a poor user experience. Therefore, there is an urgent need for a solution that can both protect user privacy and achieve efficient interaction. Summary of the Invention
[0003] This invention provides a non-contact human-computer interaction method and system based on non-biological feature information data to solve the problems of poor privacy and security and unreasonable division of labor among devices in the existing technology. The technical solution of the present invention is as follows: A non-contact human-computer interaction method based on non-biological feature information data includes: acquiring environmental perception data collected by a non-imaging environmental perception sensor on a wearable device for non-identity recognition purposes; based on the environmental perception data, performing geometric ray projection within a preset spatial display area to lock onto a target interaction object; identifying the user's semantic gestures towards the target interaction object; combining locally stored context information on the device with a preset behavior adaptation model to complete the semantic gestures and generate an interaction command; executing the interaction command and outputting feedback results through the spatial display area. A contactless human-computer interaction system includes a wearable device and a mobile communication device, which work together via a wireless protocol: the wearable device is responsible for collecting environmental perception data for non-identification purposes and performing geometric ray locking, semantic pose recognition, intent completion, and spatial display; the mobile communication device is responsible for heavy computing processing, privacy and security anchoring, and cellular gateway services. Beneficial effects: This invention uses non-biological features (such as millimeter waves, ToF, IMU, etc.) for environmental perception and interaction, avoiding the collection of sensitive biological information such as irises and faces, and meeting privacy compliance requirements; it achieves efficient contactless interaction by combining geometric rays and semantic gestures; and it achieves the integration and symbiosis of wearable devices and mobile communication devices by reordering their roles (the mobile phone becomes a security anchor and computing dock, and the glasses become the main interaction surface). Detailed Implementation This invention provides a non-contact human-computer interaction method and system based on non-biological feature information data, which will be described below in conjunction with specific implementation scenarios. The non-biological information data described in this application refers to biological characteristic data that is not collected, stored, or processed for uniquely identifying a user, including but not limited to iris, fingerprint, finger veins, voiceprint, facial geometry, gait, eye movement features, and genetic information; it also excludes feature templates extracted from blood, sweat, body odor, and breath gases for identity verification, as well as situations where physiological parameters such as heart rate and blood oxygen are linked to identity. The directional reference information described in this application is derived solely from environmental geometric relationships and posture joint coordinates. Implementation Method 1 (Basic Scheme): Wearable devices (such as AR glasses) collect environmental perception data of the surrounding environment for non-identification purposes through built-in non-imaging environmental perception sensors (including but not limited to millimeter-wave radar, ToF sensors, IMU inertial measurement units, or ultrasonic sensors). The system performs geometric ray projection within the spatial display area of the wearable device to lock onto the target interactive object that the user is looking at or pointing at. The user interacts with the locked target through specific semantic gestures (such as finger micro-movements, knuckle pinching, and other non-unique universal gestures). A lightweight behavior adaptation model stored locally on the device, combined with current context information (such as the application scenario being viewed and historical interaction records), completes the semantic gesture intent, generates interaction commands, executes them, and outputs the results through the spatial display area. Implementation Method Two (Multimodal Silent Input): Building upon Implementation Method 1, the system combines millimeter-wave gesture recognition with electromyography or ultrasound-based silent input technology. When users are unable to speak in public, the system uses millimeter-wave radar to capture subtle hand movements, combined with ultrasound lip movement recognition, to perform semantic analysis locally on the device, enabling silent text or command input without collecting sound or biometric data throughout the process. Implementation Method 3 (Device-side Local Personalization): Based on implementation method one or two, the behavior adaptation model performs online learning and personalized adjustments locally on the terminal. User chat habits, interaction preferences, and other data are all stored locally and not uploaded to the cloud, ensuring privacy and security. Implementation Method Four (Scenario Prediction and Minimalist Verification): The system uses an edge-side model to predict the current application scenario (e.g., if a user looks up in a certain direction, it predicts that the user may need to view relevant information). The system then displays candidate commands, and the user can complete the operation with a very simple semantic gesture (e.g., a single finger confirmation after ray lock), significantly reducing the input bandwidth requirements. Implementation Method 5 (Wearable and Mobile Phone Integrated Symbiotic System): This embodiment provides a system architecture. Wearable devices and mobile communication devices work collaboratively via a wireless protocol. The wearable device is equipped with a non-imaging environmental perception sensor, a spatial display module, and an edge processing module, responsible for collecting environmental perception data for non-identity recognition purposes, performing geometric ray locking, semantic pose recognition, intent completion, and spatial display. The mobile communication device is equipped with a heavy computing processing module and a security anchoring module, responsible for running heavy-load models, storing sensitive data, and providing cellular communication capabilities. The mobile communication device acts as a privacy and security anchor and computing hub, while the wearable device serves as the main interaction surface. The wearable device can be configured with an eSIM module for independent communication, while the mobile communication device degenerates into a private cloud drive and secure enclave for storing payment keys and identity information. The edge processing module runs a lightweight model for real-time completion, while the mobile communication device runs a heavier model for deep summarization and cross-application orchestration.
Claims
1. A non-contact human-computer interaction method based on non-biological feature information data, characterized in that, The steps include: acquiring environmental perception data collected by non-imaging environmental perception sensors on wearable devices for purposes other than identity recognition; Based on the environmental perception data, geometric ray projection is performed within a preset spatial display area to lock onto the target interactive object; the semantic gestures of the user toward the target interactive object are identified; the semantic gestures are completed by combining the context information stored locally on the device with a preset behavior adaptation model to generate an interaction command; the interaction command is executed, and the feedback result is output through the spatial display area.
2. The method according to claim 1, characterized in that, The non-imaging environmental sensing sensor includes at least one of millimeter-wave radar, ToF sensor, IMU or ultrasonic sensor.
3. The method according to claim 1, characterized in that, The semantic gestures include at least one non-biological gesture, such as hand micro-movements, knuckle tapping, or silent lip movements.
4. The method according to claim 1, characterized in that, The behavior adaptation model is a lightweight model that can be adjusted online or trained offline on the terminal device, and does not upload the original perception data to the cloud.
5. The method according to claim 1, characterized in that, Also includes: When the confidence level corresponding to the semantic gesture action is lower than a preset threshold, a degradation path is triggered, and the interaction task is transferred to the associated mobile communication device for processing.
6. The method according to claim 1, characterized in that, The context information includes at least one of the following: the current application scenario, historical interaction records, or environmental noise level.
7. The method according to claim 1, characterized in that, The process of generating the interaction instructions includes: predicting user needs based on a context prediction model and confirming zero-action actions in conjunction with the semantic gestures.
8. The method according to claim 1, characterized in that, The method is applied to a symbiotic system of wearable devices and mobile communication devices, wherein the wearable device is responsible for lightweight interaction and display, and the mobile communication device is responsible for heavy computing processing, privacy and security anchoring, and cellular gateway services.
9. A non-contact human-computer interaction system, characterized in that, The system includes wearable devices and mobile communication devices that work together via a wireless protocol. The wearable device is equipped with a non-imaging environmental perception sensor, a spatial display module, and an edge processing module to collect environmental perception data for non-identification purposes and perform geometric ray locking, semantic pose recognition, intent completion, and spatial display. The mobile communication device is equipped with a heavy computing processing module and a security anchoring module to run heavy-load models, store sensitive data, and provide cellular communication capabilities. The wearable device and the mobile communication device transmit data transparently through a non-biometric interaction protocol, with the mobile communication device acting as a privacy and security anchor and a computing hub, and the wearable device acting as the main interaction surface.
10. The system according to claim 9, characterized in that, The wearable device is equipped with an eSIM module for independent communication, and the mobile communication device degenerates into a private cloud drive and a secure enclave for storing payment keys and identity information.