Method for adaptive control of digital and physical environment for user with disabilities based on personal digital twin

Abstract

FIELD: assistive information technologies; medicine.SUBSTANCE: invention can be used for automatic adaptation of the digital and physical environment for users with disabilities. The technical result consists in increasing the safety and accessibility of user interaction with the digital and physical environment through the automatic adaptation of user interfaces, navigation tools, connected devices, smart home elements, and infrastructure services based on the user's personal digital twin, sensor infrastructure data, and contextual information, as well as in reducing the likelihood of hazardous situations associated with the deterioration of the user's functional state. To achieve this, a digital user profile is formed, a personal digital twin of the user is built and updated, data on the user's state and contextual information is obtained, the said data is analysed using the personal digital twin, an adaptation scenario is automatically selected, and at least one element of the digital environment, the physical environment, or the user's route of movement is adaptively controlled. If an increased risk of a security breach, disorientation, fall, overload, or crisis condition is detected, a notification is generated and transmitted to the user, accompanying person, and / or response service.EFFECT: increasing the safety and accessibility of user interaction with the digital and physical environment.11 cl

Description

[0001] The invention relates to the field of medical and assistive information technologies, life support systems for individuals with disabilities, digital platforms for monitoring user status, personal digital twin technologies, automatic control of user interfaces, smart home elements and infrastructure facilities.

[0002] The invention can be used in systems for supporting users with visual, hearing, musculoskeletal, cognitive, mental, and functional impairments, as well as combined functional limitations, when automatic adaptation of the digital and physical environment is required, taking into account the user's current state and contextual information.

[0003] Solutions are known for ensuring the accessibility of user interfaces and navigation for individuals with disabilities. In particular, publication US 20040218451 A1 discloses an accessible user system and navigation method that utilize various input and output modalities, including speech control, specialized input devices, and adapted information presentation for users with disabilities.

[0004] The disadvantage of this solution is that it is focused primarily on adapting the user's interaction with the device interface and does not disclose the method in which the adaptation of the digital environment, physical environment and travel route is carried out on the basis of the user's personal digital twin, sensor infrastructure data, the user's current functional state and contextual information about his location, workload and changes in risk level.

[0005] Solutions using digital twins in medical and related systems are also known. For example, in publication US 20220375621 A1, a digital twin is used as an object model that can be updated based on observation data and used for condition analysis and decision support.

[0006] The disadvantage of this solution is that it does not use a digital twin as a basis for adaptive accessibility management of the digital environment, user interfaces, smart home elements, and urban infrastructure specifically for users with disabilities, taking into account acceptable interaction modalities, routes, changes in functional status, and crisis scenarios.

[0007] Research and applied solutions are known to focus on intelligent smart homes for people with disabilities, personalized routing for vulnerable road users, and the use of digital twins to support people with disabilities. However, these solutions and publications typically describe either accessible home environments, specialized routing, or medical modeling of the user's condition, without revealing a unified method for adaptive control of the digital and physical environment based on the user's personal digital twin and sensor infrastructure data.

[0008] The method disclosed in US 20040218451 A1 was chosen as the prototype, as it is closest in purpose, as it aims to ensure accessible user interaction with the interface. A drawback of this prototype is that it does not provide for the creation and updating of a personal digital twin of the user, does not use data on the user's current functional state and contextual information to automatically select an adaptation scenario, and does not provide for joint adaptive control of the digital environment, the physical environment, and the user's route, including smart home elements and infrastructure.

[0009] The objective of the invention is to create a method for adaptive control of a digital and physical environment for a user with disabilities, which ensures automatic determination of the current functional state of the user and automatic selection of a scenario for adapting the means of interaction and the environment depending on the type of functional limitation, the context of movement, the level of load and the change in the level of risk.

[0010] The technical result of the invention is to increase the safety and accessibility of user interaction with the digital and physical environment by automatically selecting and implementing a scenario for adapting user interfaces, navigation tools, connected devices, smart home elements and infrastructure services based on the user's personal digital twin, sensor infrastructure data and contextual information, as well as reducing the likelihood of dangerous situations associated with the deterioration of the user's functional state.

[0011] The specified technical result is achieved by the fact that in the method of adaptive control of the digital and physical environment for a user with disabilities, a digital user profile is formed, containing information about the type of limitations of the user's functions, medical data, information about the assistive devices used, preferred interaction modalities and environmental parameters significant for the user's life activities.

[0012] Based on the specified digital profile, a personal digital twin of the user is formed and updated in the form of a dynamic model of the user's functional state and the conditions of his interaction with the digital and physical environment.

[0013] Receive from a plurality of sources, including at least wearable devices, user mobile devices, smart home devices, positioning devices and other sensory sources, data on the user's state and contextual information, including at least data on the user's location, route of movement, motor activity, environmental parameters, use of user interfaces and smart home elements.

[0014] The specified data is analyzed using the user's personal digital twin to determine the user's current functional state, changes in risk level, and the need to adapt the digital and / or physical environment. At least one adaptation scenario corresponding to the user's current functional state and risk level is then automatically selected and activated.

[0015] The adaptation scenario includes changing the user interface modality, navigation prompt parameters, alert issuing method, operating modes of connected smart home devices, room access parameters, help call parameters, and / or infrastructure routing parameters.

[0016] When an increased risk of safety violation, disorientation, fall, overload, or crisis condition is detected, a notification is automatically generated for the user, accompanying person, medical professional, and / or emergency services.

[0017] The claimed method is implemented using a digital platform containing computing resources, data storage and processing modules, and communication capabilities with wearables, mobile devices, smart home devices, and external infrastructure services. The platform can be implemented as a server-based, cloud-based, local, or distributed hardware and software solution.

[0018] In the first stage, a digital profile is created for the user, including medical information, information about the type of functional limitations, acceptable methods of interaction, assistive devices used, behavioral characteristics, routing restrictions, and preferred assistance scenarios.

[0019] In the second stage, a personal digital twin of the user is created based on the digital profile. The digital twin is a dynamic model that is updated as new data is received about the user's state, actions, reactions to interfaces, and environmental changes.

[0020] In the third stage, user and context data are obtained from multiple sources, including physiological data, location and route information, environmental parameter data, signals from smart home devices, mobile device data, and user interaction data with interfaces.

[0021] At the fourth stage, the specified data is analyzed using the user's personal digital twin, which determines the user's current functional state, the change in risk level, and the need to adapt the digital and / or physical environment, after which an adaptation scenario is automatically selected that corresponds to the user's current functional state and risk level.

[0022] At the fifth stage, the user's digital and / or physical environment is automatically controlled. Control may include changing the visual and acoustic parameters of the interface, enabling screen access, switching the type of prompts, adapting the frequency of notifications, changing the route, activating geo-tracking, controlling lighting, doors, household appliances, calling for help, and other environmental objects.

[0023] At the sixth stage, when a critical or crisis condition is detected, a notification is automatically generated for the user, accompanying person, medical personnel and / or response service.

[0024] Example 1. Visually impaired user

[0025] For a visually impaired user, the digital profile specifies acceptable interaction modalities, including audio prompts, vibration signals, and interaction with screen readers. Upon receiving data about the user's route, the density of surrounding objects, noise levels, changes in speed, and deviations in the user's actual behavior from the previously recorded behavioral profile, the digital twin detects an increased risk of disorientation. The platform automatically increases the frequency of navigation prompts, switches the mobile app interface to a larger, high-contrast view, activates audio navigation and vibration warnings about obstacles, and, if necessary, adjusts the route to a safer one.

[0026] Example 2. User with musculoskeletal impairment

[0027] For a wheelchair user, the digital profile specifies environmental accessibility parameters, surface inclination restrictions, permissible routes, and the preferred interface control mode. When the user's condition changes and signs of fatigue or deterioration of upper body stability are detected, the digital twin-based system selects a route accessible to the user, activates pre-opening of doors, calls the elevator, reduces the number of steps in the control interface, and increases the frequency of warning messages about difficult sections of the route.

[0028] Example 3. User with cognitive impairment or mental crisis

[0029] For users with cognitive impairments, autism spectrum disorders, or mental health conditions, the digital profile contains information about the acceptable level of sensory overload, the preferred prompting rate, the composition of trusted contacts, and crisis response rules. If these data detect signs of disorientation, panic, emotional overload, severe behavioral disturbance, or another crisis, the system reduces visual stimuli, switches to a simplified interface, increases the frequency of step-by-step prompts, activates geo-monitoring, and sends a notification to the trusted contact or specialist specified in the user's profile.

[0030] Example 4. Hearing-impaired user

[0031] For users with hearing impairments, the system utilizes visual and vibration notifications. Upon detecting events requiring the user's attention, including smart home device alarms, changes in environmental parameters, route alerts, or assistance service messages, the digital platform, depending on the current adaptation scenario, converts audio signals into text, graphic, and vibration notifications, and activates indicator lights in the room and / or on the wearable device.

[0032] The embodiments described confirm the possibility of implementing the invention with the achievement of the claimed technical result and show that the method can be implemented using known computing means, communication means, wearable devices, mobile devices, smart home devices and data analysis software modules.

Claims

1. A method for adaptive management of the digital and physical environment for a user with disabilities based on a personal digital twin, characterized in that: a digital profile of the user is formed, containing at least information on the type of limitations of the user's functions, medical data, data on the assistive devices used, preferred interaction modalities and environmental parameters significant for the user's life; based on the said digital profile, a personal digital twin of the user is formed and updated in the form of a dynamic model of the functional state of the user and the conditions of his interaction with the digital and physical environment;receive data from multiple sources about the user's state and contextual information, including at least data about the user's location, route of movement, motor activity, environmental parameters, use of user interfaces and / or smart home elements; analyze said data using a personal digital twin of the user and determine the current functional state of the user, changes in the level of risk and the need to adapt the digital and / or physical environment; automatically select, based on the results of the specified analysis, at least one scenario for adapting the digital and / or physical environment that corresponds to the current functional state of the user and the level of risk; carry out adaptive control of at least one element of the digital environment, the physical environment, or the user's route of movement by changing the interface modality, navigation parameters, warning parameters, operating modes of connected devices, access parameters to infrastructure facilities, and / or other parameters of user interaction with the digital and physical environment; When an increased risk of security breach, disorientation, fall, overload, or crisis condition is detected, a notification is automatically generated and transmitted to the user, accompanying person, and / or response service.

2. The method according to claim 1, characterized in that the user belongs to at least one of the groups: users with visual impairments, hearing impairments, musculoskeletal disorders, cognitive impairments, autism spectrum disorders, mental disorders and / or combined functional impairments.

3. The method according to claim 1, characterized in that the interaction methods used are at least voice control, gesture control, gaze control, single-button control, tactile interaction, vibration signals, screen access and / or a combination of the said modalities.

4. The method according to claim 1, characterized in that the adaptive control of the digital environment includes changing the structure and visual parameters of the user interface, information density, the method of presenting notifications, the frequency and form of prompts, enabling screen access and / or switching between visual, audio and tactile presentation of information.

5. The method according to claim 1, characterized in that the adaptive control of the physical environment includes automatic control of lighting, door locks, elevators, connected room devices, emergency signaling devices, geo-control devices, room positioning devices and / or help calling devices.

6. The method according to paragraph 1, characterized in that when the user’s stability, gait, coordination of movements, or balance deteriorates, the route of movement is automatically changed, the frequency of navigation prompts is increased, geo-control is activated, and / or access to dangerous areas is restricted.

7. The method according to paragraph 1, characterized in that upon detection of signs of cognitive overload, disorientation, panic reaction, severe behavioral disorder and / or other mental crisis state, the interaction scenario is automatically changed, the sensory load of the interface is reduced, a crisis response protocol is activated and a notification is initiated to a trusted person, a medical specialist and / or an emergency service.

8. The method according to claim 1, characterized in that for users with visual impairments, the adaptation includes audio navigation, tactile and / or vibration signals, increased contrast, changing the display scale, and interaction with screen access programs.

9. The method according to claim 1, characterized in that for users with hearing impairments, the adaptation includes visual and / or vibration notifications, conversion of sound events into text, graphic and / or vibration signals.

10. The method according to paragraph 1, characterized in that for users with musculoskeletal disorders, the adaptation includes the selection of routes accessible for a wheelchair, automatic control of doors and elevators, and switching of interfaces to a control mode with a minimum number of actions.

11. The method according to paragraph 1, characterized in that when performing analytical processing, personal data is used in a secure environment, and aggregated, pseudonymized and / or anonymized data is used for intersystem exchange and setting up analytical algorithms.