Intelligent positioning terminal with multi-modal interaction function and system thereof

Abstract

The application discloses an intelligent positioning terminal with a multi-modal interaction function and a system thereof, and relates to the technical field of computers.The intelligent positioning terminal with the multi-modal interaction function comprises a main controller, a multi-modal interaction module, a multi-source positioning module, an environment sensing module and a power management module.The intelligent positioning terminal with the multi-modal interaction function integrates environment sensing, geographic fence and power monitoring data through the main controller, evaluates the trust level of an operation scene, and automatically calls a corresponding power consumption privacy strategy based on the trust level, thereby realizing intelligent enabling and disabling of different interaction units in the multi-modal interaction module and self-adaptive adjustment of the positioning accuracy of the multi-source positioning module; therefore, the deficiencies of static strategies in the prior art are solved, and the terminal can achieve the optimal balance between functional integrity, power consumption economy and privacy security according to an actual use scene.

Description

Technical Field

[0001] This invention relates to the field of computer technology, specifically to an intelligent positioning terminal and system with multimodal interaction capabilities. Background Technology

[0002] Currently, products integrating multiple interaction methods and positioning functions have emerged in the field of smart terminals; these products are usually equipped with voice interaction, touch operation and contactless operation, and integrate multiple positioning technologies to obtain location information.

[0003] Existing technical solutions generally adopt a preset static working mode, and their power management and function activation rules are fixed after device initialization. Although some improved solutions can switch modes simply based on the power level, their decision-making dimensions are singular and fail to comprehensively consider the differences in the actual environment in which the terminal is located. Therefore, existing technical products have the following problems during use: First, the static working mode cannot adapt to complex and ever-changing actual usage scenarios, resulting in the inability to fully utilize the device's functions in a safe environment, while posing unnecessary power consumption and privacy risks in public areas. Second, existing solutions lack emergency handling mechanisms in low power situations, often directly shutting down core functions, affecting the availability and reliability of the terminal in emergency situations. To address the shortcomings of existing technologies, this invention provides an intelligent positioning terminal and its system with multimodal interaction functions to solve the above problems. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides an intelligent positioning terminal and system with multimodal interaction capabilities. By integrating environmental perception, geofencing, and power monitoring data through a main controller, it assesses the trust level of the operating scenario and automatically invokes corresponding power consumption and privacy policies based on this assessment. This enables intelligent enabling and disabling of different interaction units within the multimodal interaction module, as well as adaptive adjustment of the positioning accuracy of the multi-source positioning module. Therefore, it overcomes the limitations of static policies in existing technologies, allowing the terminal to achieve an optimal balance between functional integrity, power efficiency, and privacy security based on actual usage scenarios, thereby improving the device's scenario adaptability and user experience.

[0005] To achieve the above objectives, the present invention provides the following technical solution: an intelligent positioning terminal with multimodal interaction capabilities, comprising:

[0006] Main controller;

[0007] A multimodal interaction module includes a voice interaction unit, a touch interaction unit, and at least one contactless interaction unit;

[0008] A multi-source positioning module is used to obtain the terminal's geographical location information;

[0009] The environmental sensing module includes environmental sensors for collecting environmental data;

[0010] The power management module is used to monitor the terminal's battery status;

[0011] The main controller is configured to: determine the current operating scenario trust level of the terminal based on the environmental data collected by the environmental perception module, the pre-stored geofence information, and the power status monitored by the power management module; based on the operating scenario trust level, call a set of corresponding policies from a set of preset power consumption privacy policies, the policies including rules for enabling and disabling different interaction units in the multimodal interaction module and rules for adjusting the positioning accuracy of the multi-source positioning module; and adjust the working state of the multimodal interaction module and the multi-source positioning module according to the called policies.

[0012] Preferably, the main controller is configured to identify whether the terminal has entered a preset safe area based on pre-stored geofence information, and set the trust level of the scene entering the safe area to a high trust level.

[0013] Preferably, when the trust level of the operating scenario is a high trust level corresponding to the safe zone, the main controller invokes the first strategy: to enable the voice interaction unit and the contactless interaction unit, and to adjust the positioning accuracy of the multi-source positioning module to the standard accuracy.

[0014] Preferably, when the trust level of the operating scenario is a low trust level corresponding to a public area, the main controller invokes a second strategy: disables the contactless interaction unit, sets the voice interaction unit to only respond to a preset specific wake word, and adjusts the positioning accuracy of the multi-source positioning module to a power-saving mode that is lower than the standard accuracy.

[0015] Preferably, the main controller is configured to forcibly switch the operating scenario trust level to the emergency level when the power management module detects that the power level is lower than the first threshold, and to invoke the third strategy: disable the voice interaction unit and the non-contact interaction unit in the multimodal interaction module, retain only the touch interaction unit, and control the multi-source positioning module to work in the lowest power consumption mode.

[0016] Preferably, the contactless interaction unit is a camera, and the main controller is configured to completely shut down the power of the camera when the second and third strategies are invoked.

[0017] Preferably, the environmental perception module includes a microphone, and the main controller is configured to analyze the environmental noise data collected by the microphone to help determine the trust level of the operating scene; when the environmental noise continuously exceeds a set decibel value, the trust level of the current scene is lowered.

[0018] Preferably, the smart positioning terminal further includes an accelerometer for detecting specific physical gestures; the main controller is configured to continuously monitor the signal output by the accelerometer even when the voice interaction unit and the contactless interaction unit are disabled, and when a specific signal sequence matching a pre-stored emergency gesture is identified, the multimodal interaction module is forcibly enabled and an alarm function is triggered.

[0019] Preferably, the main controller is configured to: when invoking a strategy to restrict the contactless interaction unit, if it receives a function instruction initiated by an unrestricted interaction unit that requires invoking the contactless interaction unit, before executing the instruction, output a privacy risk confirmation prompt through the touch interaction unit, and temporarily activate the contactless interaction unit after obtaining confirmation.

[0020] A second aspect of the present invention discloses an intelligent positioning system, including the intelligent positioning terminal with multimodal interaction function.

[0021] The technical effects and advantages of this invention are as follows:

[0022] 1. This intelligent positioning terminal with multimodal interaction capabilities integrates environmental perception, geofencing, and power monitoring data through a main controller to assess the trust level of the operating scenario. Based on this, it automatically invokes the corresponding power consumption and privacy policies, enabling intelligent enabling and disabling of different interaction units in the multimodal interaction module, as well as adaptive adjustment of the positioning accuracy of the multi-source positioning module. Therefore, it overcomes the shortcomings of existing static policies, enabling the terminal to achieve an optimal balance between functional integrity, power consumption economy, and privacy security according to the actual usage scenario, thereby improving the device's scenario adaptability and user experience.

[0023] 2. This intelligent positioning terminal with multimodal interaction function identifies safe areas through pre-stored geofence information and sets the scene to a high trust level, thereby invoking the policy of enabling voice and contactless interaction units; it realizes scenario-based function management based on precise location services, ensuring that users can obtain a complete and smooth multimodal interaction experience in a safe and reliable environment, while providing a reliable judgment benchmark for tightening the system's policy in unsafe areas.

[0024] 3. This intelligent positioning terminal with multimodal interaction function monitors the power level through the power management module and forcibly switches to the emergency level when the power level is lower than the threshold, calling a strategy to disable high-power interaction units while retaining the minimum positioning function; a hierarchical emergency response mechanism is established, which enhances the reliability and service continuity of the terminal in emergency situations by ensuring the continuous operation of the core positioning function under extreme conditions, thus providing protection for users. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a system architecture diagram of the present invention;

[0027] Figure 2 The trust level determination logic diagram for this invention;

[0028] Figure 3 This is a flowchart of the strategy invocation and working status adjustment of the present invention. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] This embodiment discloses an intelligent positioning terminal with multimodal interaction capabilities, according to the appendix. Figure 1 To be continued Figure 3 As shown, it includes:

[0031] Main controller;

[0032] A multimodal interaction module includes a voice interaction unit, a touch interaction unit, and at least one contactless interaction unit;

[0033] A multi-source positioning module is used to obtain the terminal's geographical location information;

[0034] The environmental sensing module includes environmental sensors for collecting environmental data;

[0035] The power management module is used to monitor the terminal's battery status;

[0036] The main controller is configured to: determine the current operating scenario trust level of the terminal based on the environmental data collected by the environmental perception module, the pre-stored geofence information, and the power status monitored by the power management module; based on the operating scenario trust level, call a set of corresponding policies from a set of preset power consumption privacy policies, the policies including rules for enabling and disabling different interaction units in the multimodal interaction module and rules for adjusting the positioning accuracy of the multi-source positioning module; and adjust the working state of the multimodal interaction module and the multi-source positioning module according to the called policies.

[0037] This intelligent positioning terminal with multimodal interaction capabilities integrates multiple functional modules through a main controller to achieve scene-based intelligent power consumption and privacy management. The main controller, as the terminal's central processing unit, is responsible for coordinating the work of each module. The multimodal interaction module includes a voice interaction unit, a touch interaction unit, and at least one contactless interaction unit, such as a camera or gesture sensor, which together provide diverse user interaction methods. The multi-source positioning module is used to acquire high-precision geographic location information, and can employ positioning technologies such as LTE, GPS, BeiDou, Wi-Fi, and Bluetooth. The environmental perception module includes environmental sensors, such as microphones, light sensors, and temperature sensors, for real-time collection of environmental data. The power management module monitors the terminal's battery status, including remaining battery power and power consumption trends.

[0038] The main controller's configuration analyzes environmental data collected by the environmental perception module, pre-stored geofence information, and power status monitored by the power management module. It employs a trust assessment algorithm to determine the trust level of the terminal's current operating scenario. This trust level reflects the security and reliability of the terminal's environment; for example, a high trust level corresponds to a secure area, such as a home or office, while a low trust level corresponds to a public area, such as a shopping mall or street. Based on the trust level, the main controller invokes a set of pre-defined power consumption and privacy policies. These policies include rules for enabling and disabling different interaction units in the multimodal interaction module, such as disabling contactless interaction units in low-trust scenarios, and rules for adjusting the positioning accuracy of the multi-source positioning module, such as reducing positioning accuracy to save power. By adjusting the module's operating state, the terminal optimizes power consumption and protects privacy while ensuring a good user experience.

[0039] This intelligent positioning terminal, featuring multimodal interaction capabilities, automatically identifies scenarios and adjusts its behavior through multi-dimensional analysis of environmental data, geofencing, and battery status, without requiring manual user intervention, thus enhancing convenience and security. Adjusting positioning accuracy and the use of interactive units significantly extends battery life, especially suitable for mobile devices. In low-trust scenarios, limiting unnecessary interactive units, such as cameras, prevents unauthorized data collection and reduces the risk of privacy breaches. The independent yet collaborative operation of each module improves system reliability and scalability.

[0040] Geofence information is typically stored locally or in a cloud database, including area boundary coordinates and attribute labels, such as setting location areas like "home" or "office." The main controller obtains its current location through a multi-source positioning module and matches it with the geofence information. If it enters a secure area, the scene trust level is set to high. Secure areas are generally considered low-risk environments where users tend to use full-featured interactions; therefore, a high trust level allows for more lenient power consumption and privacy policies.

[0041] This terminal utilizes geofencing technology to ensure the accuracy of scene judgment and avoid user experience degradation caused by misjudgment; without the need for users to manually switch modes, the terminal automatically upgrades the trust level, enables more functions, and enhances its intelligence.

[0042] When the scenario trust level is high, corresponding to a secure area, the main controller invokes the first strategy. This strategy includes enabling the voice interaction unit and the contactless interaction unit, and adjusting the positioning accuracy of the multi-source positioning module to standard accuracy, such as using GPS and Wi-Fi fusion positioning with an error within the meter range. Within the secure area, users typically require full interactive functions, such as voice assistants or gesture control, and power consumption is relatively low, thus allowing high-precision positioning and a full interactive mode.

[0043] It provides all interaction methods in a safe environment to meet diverse user needs; standard precision positioning ensures accuracy without excessive power consumption.

[0044] When the trust level of the operating scenario is the low trust level corresponding to a public area, the main controller invokes the second strategy. This strategy includes disabling the contactless interaction unit and setting the voice interaction unit to only respond to a preset specific wake word, such as "Hello, terminal," while adjusting the positioning accuracy of the multi-source positioning module to a power-saving mode. Public areas may pose privacy risks or high noise interference; limiting the interaction unit and positioning accuracy can reduce the risk of privacy leaks and power consumption.

[0045] Disable contactless units and limit voice interaction to prevent sensitive data from being accidentally collected; power-saving mode is designed to reduce power consumption and extend usage time.

[0046] The main controller is configured to forcibly switch the operating scenario trust level to emergency level and invoke a third strategy when the power management module detects that the battery level is below a first threshold (e.g., the first threshold can be set to 20% battery remaining). This strategy includes disabling the voice interaction unit and contactless interaction unit in the multimodal interaction module, retaining only the touch interaction unit as the minimum input method, and controlling the multi-source positioning module to operate in the lowest power consumption mode, such as using only satellite positioning and configuring it to update intermittently. In low power situations, the terminal prioritizes the continued operation of core functions, extending battery life by sacrificing unnecessary functions, and maximizing battery utilization by disabling high-power modules.

[0047] The contactless interaction unit is a camera, and the main controller is configured to completely power off the camera when the second and third strategies are invoked. Cameras are typically high-power and high-privacy-risk components; completely shutting them off in insecure scenarios or when the battery is low avoids unnecessary energy consumption and privacy issues.

[0048] The environmental perception module includes a microphone, and the main controller is configured to analyze ambient noise data collected by the microphone to help determine the trust level of the operating scene. For example, when the ambient noise continuously exceeds a set decibel value, such as 70 decibels, the trust level of the current scene is lowered. High-noise environments are often associated with public areas and may interfere with voice interaction; therefore, noise analysis can enhance the accuracy of scene recognition.

[0049] The smart positioning terminal also includes an accelerometer for detecting specific physical gestures; the main controller is configured to continuously monitor the accelerometer's output signal even when the voice interaction unit and the contactless interaction unit are disabled. When a specific signal sequence matching a pre-stored emergency gesture is detected, such as a gesture signal of rapidly shaking the terminal three times, the multimodal interaction module is forcibly enabled and an alarm function is triggered. The accelerometer has extremely low power consumption and can serve as an emergency trigger mechanism, providing a discreet wake-up method in the disabled state for emergency situations.

[0050] Therefore, this terminal provides a reliable emergency triggering method, enhancing the terminal's security; the accelerometer's continuous operation does not affect the overall power consumption, ensuring the availability of emergency functions.

[0051] The main controller is configured to: when invoking the policy of restricting contactless interaction units, if it receives a function instruction initiated through an unrestricted interaction unit that requires the use of contactless interaction units, such as a photo-taking request, before executing the instruction, output a privacy risk confirmation prompt through the touch interaction unit, pop up a dialog box explaining the privacy risks, and temporarily activate the contactless interaction unit after obtaining user confirmation. In restricted mode, users may still need to use contactless functions. The confirmation mechanism balances functional needs and privacy protection.

[0052] Therefore, this terminal allows users to use sensitive functions temporarily with their knowledge, increasing flexibility; and prompts ensure that users are aware of privacy risks.

[0053] A second aspect of the present invention discloses an intelligent positioning system, including the intelligent positioning terminal with multimodal interaction function.

[0054] Example 1: This example demonstrates the workflow of a user using a smart positioning terminal in a secure home area. The workflow is as follows:

[0055] The terminal obtains its current location through a multi-source positioning module, and the main controller compares it with the pre-stored geofence information to identify whether the terminal has entered the "home" safe zone.

[0056] The microphone detected ambient noise below 50 decibels, the light sensor detected suitable lighting, and the power management module reported sufficient battery power (80%).

[0057] Based on this data, the main controller determines that the trust level of the operating scenario is high trust level.

[0058] The main controller invokes the first strategy: enable the voice interaction unit and the contactless interaction unit, adjust the positioning accuracy of the multi-source positioning module to the standard accuracy, and use GPS and Wi-Fi fusion positioning.

[0059] Users can control smart home devices by using the voice command "turn on the lights," and the terminal will recognize and execute the command through the voice interaction unit. At the same time, users can switch modes by waving their hands in front of the camera, and the non-contact interaction unit will respond normally.

[0060] The terminal continuously monitors the environment, and if the user leaves the home area, the main controller reassesses the trust level and adjusts the policy.

[0061] Example 2: This example demonstrates the workflow of a user using a smart positioning terminal in a shopping mall's public area. The workflow is as follows:

[0062] The terminal obtains its current location through a multi-source positioning module, and the main controller compares the geofence information to identify whether the terminal has entered the public area of ​​the "shopping mall".

[0063] The environmental sensing module collects environmental data: the microphone detects that the ambient noise continuously exceeds 70 decibels; the power management module reports that the battery level is 50%.

[0064] Based on this data, the main controller determines that the trust level of the operating scenario is low.

[0065] The main controller invokes the second strategy: disables the contactless interaction unit and sets the voice interaction unit to only respond to the specific wake word "Hello, terminal". At the same time, it adjusts the positioning accuracy of the multi-source positioning module to power-saving mode and uses only base station positioning.

[0066] The user attempted to use voice commands, but the terminal only responded after the wake word was spoken; the camera remained off to prevent accidental recording.

[0067] If a user urgently needs to use the camera function, such as scanning a QR code, they can initiate a request through the touch interaction unit. The main controller will then output a privacy risk confirmation prompt, and the user can confirm before the camera is temporarily activated to complete the operation.

[0068] When the battery level drops to 20%, the main controller forcibly switches to emergency level and invokes the third strategy: disables voice and camera, retains only touch interaction, and the positioning module enters the lowest power consumption mode.

[0069] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A smart positioning terminal with multimodal interaction capabilities, characterized in that: include: Main controller; A multimodal interaction module includes a voice interaction unit, a touch interaction unit, and at least one contactless interaction unit; A multi-source positioning module is used to obtain the terminal's geographical location information; The environmental sensing module includes environmental sensors for collecting environmental data; The power management module is used to monitor the terminal's battery status; The main controller is configured to determine the current operating scenario trust level of the terminal based on the environmental data collected by the environmental perception module, the pre-stored geofence information, and the power status monitored by the power management module. Based on the trust level of the operating scenario, a set of corresponding policies is invoked from a set of preset power consumption and privacy policies. The policies include rules for enabling and disabling different interaction units in the multimodal interaction module and rules for adjusting the positioning accuracy of the multi-source positioning module. The working state of the multimodal interaction module and the multi-source positioning module is adjusted according to the invoked policies.

2. The intelligent positioning terminal with multimodal interaction function according to claim 1, characterized in that, The main controller is configured to identify whether a terminal has entered a preset safe zone based on pre-stored geofence information, and to set the trust level of the scene that has entered the safe zone to a high trust level.

3. The intelligent positioning terminal with multimodal interaction function according to claim 2, characterized in that, When the trust level of the operating scenario is the high trust level corresponding to the safe zone, the main controller invokes the first strategy: to enable the voice interaction unit and the contactless interaction unit, and to adjust the positioning accuracy of the multi-source positioning module to the standard accuracy.

4. The intelligent positioning terminal with multimodal interaction function according to claim 2, characterized in that, When the trust level of the operating scenario is a low trust level corresponding to a public area, the main controller invokes the second strategy: disables the contactless interaction unit, sets the voice interaction unit to only respond to a preset specific wake word, and adjusts the positioning accuracy of the multi-source positioning module to a power-saving mode that is lower than the standard accuracy.

5. The intelligent positioning terminal with multimodal interaction function according to claim 1, characterized in that, The main controller is configured to forcibly switch the operating scenario trust level to the emergency level when the power management module detects that the power level is lower than the first threshold, and to invoke the third strategy: disable the voice interaction unit and non-contact interaction unit in the multimodal interaction module, retain only the touch interaction unit, and control the multi-source positioning module to work in the lowest power consumption mode.

6. The intelligent positioning terminal with multimodal interaction function according to claim 5, characterized in that, The contactless interaction unit is a camera, and the main controller is configured to completely shut down the power of the camera when the second and third strategies are invoked.

7. The intelligent positioning terminal with multimodal interaction function according to claim 1, characterized in that, The environmental perception module includes a microphone, and the main controller is configured to analyze environmental noise data collected by the microphone to help determine the trust level of the operating scene. When the ambient noise continuously exceeds the set decibel value, the trust level of the current scene will be lowered.

8. The intelligent positioning terminal with multimodal interaction function according to claim 1, characterized in that, The smart positioning terminal also includes an accelerometer for detecting specific physical gestures; the main controller is configured to continuously monitor the signal output by the accelerometer even when the voice interaction unit and the contactless interaction unit are disabled, and when a specific signal sequence matching a pre-stored emergency gesture is identified, the multimodal interaction module is forcibly enabled and an alarm function is triggered.

9. The intelligent positioning terminal with multimodal interaction function according to claim 1, characterized in that, The main controller is configured to: when invoking a strategy to restrict the contactless interaction unit, if it receives a function instruction initiated by an unrestricted interaction unit that requires calling the contactless interaction unit, before executing the instruction, output a privacy risk confirmation prompt through the touch interaction unit, and temporarily activate the contactless interaction unit after obtaining confirmation.

10. An intelligent positioning system, characterized in that, Including the intelligent positioning terminal with multimodal interaction function as described in any one of claims 1-9.

Images