Multi-modal interaction system and method for old people based on dynamic biological recognition

Abstract

The invention discloses a multi-mode elderly-friendly interaction system and method based on dynamic biological recognition. The system comprises a user feature recognition and mode activation unit, a dynamic interaction guide system and a continuous learning optimization module. The user feature recognition and mode activation unit comprises a camera acquisition module and an age recognition model; the dynamic interactive guidance system comprises a visual guidance subsystem, a voice enhancement subsystem and a touch fault-tolerant subsystem. Through a technical closed loop of'biological recognition-intelligent guidance-sustained evolution ', the crossing of the old-age adaptation technology from passive adaptation to active service is realized, and the problems of'single old-age adaptation mode', 'lost operation ', 'high false touch rate' and'incapability of intelligent backtracking 'in the old-age adaptation process of the central control system are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of elderly-friendly interaction systems, and in particular to a multimodal elderly-friendly interaction system based on dynamic biometrics and a method thereof. Background Art

[0002] As the trend of population aging continues to deepen, the adaptation of smart home central control screens to the needs of the elderly faces severe challenges. In response to the operational difficulties of traditional smart devices caused by common problems such as decreased vision, hearing loss, and slow reaction among the elderly, as well as the high learning cost for the elderly due to the complex interaction of most existing central control screens, existing solutions mainly provide elderly-friendly interface design (large icon layout) and basic voice broadcast. From the actual use point of view, they still cannot meet the needs of the elderly and have the following problems: (1) The elderly-friendly mode is single and lacks personalization; (2) There is a lack of dynamic operation guidance and the false touch rate is high; (3) The fault tolerance mechanism is insufficient and intelligent backtracking is impossible; Based on the above problems, this application proposes a multimodal elderly-friendly interaction system and method based on dynamic biometrics. Summary of the Invention

[0003] Based on the technical problems existing in the background technology, the present invention proposes a multimodal elderly-friendly interaction system and method based on dynamic biometrics.

[0004] The present invention proposes a multimodal elderly-friendly interaction system based on dynamic biometric recognition, which includes a user feature recognition and mode activation unit, a dynamic interaction guidance system, and a continuous learning optimization module;

[0005] The user feature recognition and pattern activation unit includes a camera acquisition module and an age recognition model;

[0006] The dynamic interactive guidance system includes a visual guidance subsystem, a voice enhancement subsystem and a tactile fault tolerance subsystem.

[0007] Preferably, the camera acquisition module is used to capture the user's portrait. The camera acquisition module and the age recognition model are used in conjunction to complete the user identity and physiological state determination. The user identity determination is used to distinguish between new and old users, create new records / retrieve historical records / modify records. The physiological state determination is used to grade visual ability and motor ability, adjust interface parameters according to pupil reaction and viewing distance, and identify the amplitude of hand tremors.

[0008] Preferably, the age recognition model is used to automatically activate different levels of elderly-friendly modes, which are divided into standard elderly-friendly, mild impairment and severe impairment, and output user initial interaction parameters, wherein the setting mode of standard elderly recognition is based on large fonts and large icons, the setting mode of mild impairment is to add voice assistance enhancement and cursor visual guidance for key steps in addition to the standard elderly-friendly settings, and the setting mode of severe impairment is to add full voice and cursor navigation in addition to the standard elderly-friendly settings, and tremor compensation automatically extends the effective stay time of the button. The output user initial interaction parameters include new user saving records and old user status modification records at different times.

[0009] Preferably, the visual guidance subsystem is used to provide static operation demonstrations, and is also used to decide whether to provide dynamic cursor guidance based on the elderly-friendly mode. If necessary, a dynamic floating cursor is generated in the user interface and operation process to guide the user step by step, and a pressure-sensitive cursor magnification function is added. When the user clicks according to the preset process, the original cursor is enlarged to provide operation feedback.

[0010] Preferably, the speech enhancement subsystem is used to decide whether to turn on the speech assistance or full speech function according to the elderly-friendly mode, and is also used to add a fuzzy speech fault-tolerant engine and a context compensation algorithm to improve the success rate of speech recognition, wherein the context compensation algorithm is used to infer semantic intent, confirm voice broadcast, create a voice command memory library for continuous learning and optimization, and automatically complete high-frequency commands.

[0011] Preferably, the tactile fault-tolerant subsystem is used to provide pressure feedback and forced secondary pop-up confirmation. When multiple repeated clicks, continuous errors, and repeated returns to the previous step are detected, different fault-tolerant solutions are provided for different elderly-friendly modes and error levels.

[0012] Preferably, the continuous learning and optimization module is used to anonymously report errors that occur during user operations and continuously optimize the elderly-friendly model.

[0013] The present invention also proposes a multimodal elderly-friendly interaction method based on dynamic biometrics, comprising the following steps:

[0014] S1: User feature recognition and mode activation: The camera capture module captures the user's portrait and determines the user's identity and physiological status based on the age recognition model. Based on the results of the determination, the system automatically activates different levels of elderly-friendly modes and outputs the user's initial interaction parameters.

[0015] S2: Multimodal interactive guidance system starts: Its specific logical steps are as follows:

[0016] S201: Visual Guidance: A complete operation plan is preset through the visual guidance subsystem. A static operation demonstration is provided when the user first enters the visual guidance subsystem. During subsequent use, whether to provide dynamic cursor guidance is determined based on the elderly-friendly mode activated in S1. If required, a dynamic floating cursor is generated on the user interface and operation process to guide the user step by step. A pressure-sensitive cursor magnification function is also added. When the user clicks according to the preset process, the original cursor is magnified to provide operation feedback;

[0017] S202: Voice Enhancement: Determines whether to enable voice assistance or full voice function based on the elderly-friendly mode activated in S1. It activates anti-noise mode when a cough or wheeze is detected, automatically improves voice sensitivity, and provides dual-channel guidance of "voice broadcast + cursor guidance" for complex operations.

[0018] S203: Tactile Fault Tolerance: When multiple repeated clicks, continuous errors, or repeated returns to the previous step are detected during the operation, different fault tolerance solutions are provided based on different elderly-friendly modes and error levels;

[0019] S3: Continuous learning and optimization: Anonymously report errors that occur during user operations and continuously optimize the elderly-friendly model.

[0020] Preferably, in S204, the fault-tolerant solution includes automatic cursor correction, interface status snapshot recovery, voice + cursor guidance gradual rollback, direct return to the main interface and remote assistance options.

[0021] Compared with the existing technology, the beneficial effects of the present invention are:

[0022] 1. Dynamically adapt to the elderly mode through biometric recognition. Compared with the traditional single elderly mode, it realizes dynamic adaptation of three-level elderly mode and automatically changes the elderly mode level when the user feature data is refreshed;

[0023] 2. Through visual, auditory, and tactile multimodal dynamic guidance, using cursor + voice + pressure feedback, it solves the problems of "operation loss" and "learning difficulty";

[0024] 3. By setting up an intelligent fault-tolerant mechanism and creating a three-level error recovery mechanism, the success rate of error operation recovery is improved and the "error panic" problem is solved;

[0025] 4. Through continuous learning and optimization of the elderly-friendly model, an error reporting mechanism is created during operation to update the elderly-friendly model;

[0026] Through the technical closed loop of "biometric recognition-intelligent guidance-continuous evolution", the present invention has achieved the leap from passive adaptation to active service in elderly-friendly technology, and solved the problems of "single elderly-friendly mode", "lost operation", "high false touch rate" and "inability to intelligently backtrack" in the process of elderly-friendly central control system. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a block diagram of a multimodal elderly-friendly interaction system based on dynamic biometrics proposed by the present invention;

[0028] Figure 2 This is a flowchart of user feature recognition and mode activation in a multimodal elderly-friendly interaction method based on dynamic biometrics proposed by the present invention;

[0029] Figure 3 This is a flow chart of starting a multimodal interaction guidance system in a multimodal elderly-friendly interaction method based on dynamic biometrics proposed by the present invention;

[0030] Figure 4 This is a flowchart of continuous learning optimization in a multimodal elderly-friendly interaction method based on dynamic biometrics proposed by the present invention. DETAILED DESCRIPTION

[0031] The present invention will be further explained below with reference to specific embodiments.

[0032] Example

[0033] Reference Figure 1-4 This embodiment proposes a multimodal elderly-friendly interaction system based on dynamic biometrics, including a user feature recognition and mode activation unit, a dynamic interaction guidance system, and a continuous learning optimization module. The continuous learning optimization module is used to anonymously report errors that occur during user operations and continuously optimize the elderly-friendly model.

[0034] The user feature recognition and pattern activation unit includes a camera acquisition module and an age recognition model;

[0035] The camera acquisition module is used to capture the user's portrait. The camera acquisition module and the age recognition model are used to determine the user's identity and physiological status. The user identity determination is used to distinguish between new and old users and create new records / retrieve historical records / modify records. The physiological status determination is used to grade visual and motor abilities, adjust interface parameters based on pupil response and viewing distance, and identify hand tremor amplitude.

[0036] The age recognition model is used to automatically activate different levels of senior-friendly modes, which are divided into standard senior-friendly, mild impairment, and severe impairment, and outputs initial user interaction parameters. The standard senior-friendly setting mode is based on large fonts and large icons. The mild impairment setting mode adds voice assistance enhancements and cursor visual guidance for key steps in addition to the standard senior-friendly settings. The severe impairment setting mode adds full voice and cursor navigation in addition to the standard senior-friendly settings, and tremor compensation automatically extends the effective button dwell time. The output initial user interaction parameters include new user save records and old user status changes when the record is modified;

[0037] The dynamic interactive guidance system includes a visual guidance subsystem, a speech enhancement subsystem, and a tactile error tolerance subsystem;

[0038] The visual guidance subsystem provides static operation demonstrations and determines whether to provide dynamic cursor guidance based on the senior-friendly mode. If required, a dynamic floating cursor is generated on the user interface and operation process to guide the user step by step. A pressure-sensitive cursor magnification function is also added. When the user clicks according to the preset process, the original cursor is magnified to provide operation feedback.

[0039] The speech enhancement subsystem determines whether to enable voice assistance or full voice recognition based on the senior-friendly mode. It also adds a fuzzy speech error-tolerance engine and a context compensation algorithm to improve speech recognition success rates. The context compensation algorithm is used to infer semantic intent, confirm speech broadcasts, create a voice command memory library for continuous learning and optimization, and automatically complete high-frequency commands.

[0040] The tactile fault tolerance subsystem provides pressure feedback and forced secondary pop-up confirmation. When detecting multiple repeated clicks, continuous errors, or repeated returns to the previous step, it provides different fault tolerance solutions based on different elderly-friendly modes and error levels.

[0041] This embodiment also proposes a method for multimodal elderly-friendly interaction based on dynamic biometrics, including the following steps:

[0042] S1: User feature recognition and mode activation: The camera capture module captures the user's portrait and determines the user's identity and physiological status based on the age recognition model. Based on the results of the determination, the system automatically activates different levels of elderly-friendly modes and outputs the user's initial interaction parameters.

[0043] S2: Multimodal interactive guidance system starts: Its specific logical steps are as follows:

[0044] S201: Visual Guidance: A complete operation plan is preset through the visual guidance subsystem. A static operation demonstration is provided when the user first enters the visual guidance subsystem. During subsequent use, whether to provide dynamic cursor guidance is determined based on the elderly-friendly mode activated in S1. If required, a dynamic floating cursor is generated on the user interface and operation process to guide the user step by step. A pressure-sensitive cursor magnification function is also added. When the user clicks according to the preset process, the original cursor is magnified to provide operation feedback;

[0045] S202: Voice Enhancement: Determines whether to enable voice assistance or full voice function based on the elderly-friendly mode activated in S1. It activates anti-noise mode when a cough or wheeze is detected, automatically improves voice sensitivity, and provides dual-channel guidance of "voice broadcast + cursor guidance" for complex operations.

[0046] S203: Tactile Fault Tolerance: When multiple repeated clicks, continuous errors, or repeated returns to the previous step are detected during an operation, different fault tolerance solutions are provided for different senior-friendly modes and error levels. These solutions include automatic cursor correction, interface state snapshot recovery, voice + cursor guidance for gradual return, direct return to the main interface, and remote assistance options.

[0047] S3: Continuous learning and optimization: Anonymously report errors that occur during user operations and continuously optimize the elderly-friendly model;

[0048] This embodiment achieves the transition from passive adaptation to active service in elderly-friendly technology through the technical closed loop of "biometric recognition-intelligent guidance-continuous evolution", and solves the problems of "single elderly-friendly mode", "lost operation", "high false touch rate" and "inability to intelligently backtrack" in the process of elderly-friendly central control system.

[0049] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solution and inventive concept of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. A multimodal elderly-friendly interaction system based on dynamic biometrics, characterized by: It includes user feature recognition and pattern activation unit, dynamic interaction guidance system and continuous learning optimization module; The user feature recognition and pattern activation unit includes a camera acquisition module and an age recognition model; The dynamic interactive guidance system includes a visual guidance subsystem, a voice enhancement subsystem and a tactile fault tolerance subsystem.

2. The multimodal elderly-friendly interactive system based on dynamic biometrics according to claim 1, characterized in that: The camera acquisition module is used to capture the user's portrait. The camera acquisition module and the age recognition model are used to complete the user identity and physiological status determination. The user identity determination is used to distinguish between new and old users, create new records / retrieve historical records / modify records. The physiological status determination is used to grade visual ability and motor ability, adjust interface parameters according to pupil reaction and viewing distance, and identify the amplitude of hand tremors.

3. The multimodal elderly-friendly interactive system based on dynamic biometrics according to claim 1, characterized in that: The age recognition model is used to automatically activate different levels of elderly-friendly modes, which are divided into standard elderly-friendly, mild impairment and severe impairment, and output user initial interaction parameters. The setting mode of standard elderly recognition is based on large fonts and large icons. The setting mode of mild impairment is to add voice assistance enhancement and cursor visual guidance for key steps in addition to the standard elderly-friendly settings. The setting mode of severe impairment is to add full voice and cursor navigation in addition to the standard elderly-friendly settings, and tremor compensation automatically extends the effective stay time of the button. The output user initial interaction parameters include new user saving records and old user status modification records at different times.

4. The multimodal elderly-friendly interactive system based on dynamic biometrics according to claim 1, characterized in that: The visual guidance subsystem is used to provide static operation demonstrations, and is also used to decide whether to provide dynamic cursor guidance based on the elderly-friendly mode. If necessary, a dynamic floating cursor is generated in the user interface and operation process to guide the user step by step, and a pressure-sensitive cursor magnification function is added. When the user clicks according to the preset process, the original cursor is enlarged to provide operation feedback.

5. The multimodal elderly-friendly interactive system based on dynamic biometrics according to claim 1, characterized in that: The speech enhancement subsystem is used to decide whether to enable speech assistance or full speech function according to the elderly-friendly mode. It is also used to add a fuzzy speech fault-tolerant engine and a context compensation algorithm to improve the success rate of speech recognition. The context compensation algorithm is used to infer semantic intent, confirm voice broadcast, create a voice command memory library for continuous learning and optimization, and automatically complete high-frequency commands.

6. The multimodal elderly-friendly interactive system based on dynamic biometrics according to claim 1, characterized in that: The tactile fault-tolerant subsystem is used to provide pressure feedback and forced secondary pop-up confirmation. When multiple repeated clicks, continuous errors, and repeated returns to the previous step are detected, different fault-tolerant solutions are provided for different elderly-friendly modes and error levels.

7. The multimodal elderly-friendly interactive system based on dynamic biometrics according to claim 1, characterized in that: The continuous learning and optimization module is used to anonymously report errors that occur during user operations and continuously optimize the elderly-friendly model.

8. A multimodal elderly-friendly interaction method based on dynamic biometrics, characterized in that: The following steps are involved: S1: User feature recognition and mode activation: The camera capture module captures the user's portrait and determines the user's identity and physiological status based on the age recognition model. Based on the results of the determination, the system automatically activates different levels of elderly-friendly modes and outputs the user's initial interaction parameters. S2: Multimodal interactive guidance system starts: Its specific logical steps are as follows: S201: Visual Guidance: A complete operation plan is preset through the visual guidance subsystem. A static operation demonstration is provided when the user first enters the visual guidance subsystem. During subsequent use, whether to provide dynamic cursor guidance is determined based on the elderly-friendly mode activated in S1. If required, a dynamic floating cursor is generated on the user interface and operation process to guide the user step by step. A pressure-sensitive cursor magnification function is also added. When the user clicks according to the preset process, the original cursor is magnified to provide operation feedback; S202: Voice Enhancement: This function determines whether to enable voice assistance or full voice function based on the elderly-friendly mode activated in S1. It activates anti-noise mode when the user coughs or wheezes, automatically improves voice sensitivity, and provides dual-channel guidance for complex operations, including "voice broadcast + cursor guidance"; S203: Tactile Fault Tolerance: When multiple repeated clicks, continuous errors, or repeated returns to the previous step are detected during the operation, different fault tolerance solutions are provided based on different elderly-friendly modes and error levels; S3: Continuous learning and optimization: Anonymously report errors that occur during user operations and continuously optimize the elderly-friendly model.

9. The method of multimodal elderly-friendly interaction based on dynamic biometrics according to claim 8, characterized in that: In S204, the fault-tolerant solution includes automatic cursor correction, interface status snapshot recovery, voice + cursor guidance gradual rollback, direct return to the main interface and remote assistance options.

Images