Interactive support device using non-request-based stage transition display for people with dementia

Abstract

This aims to create a 24-hour, permanently installed psychological support device that encourages gradual engagement in elderly individuals with dementia or those in the early stages of dementia, starting from unconscious responses and maintaining psychological stability, without requiring operational instructions or assessments. [Solution] The system begins with non-request-based visual change displays, progresses through the formation of successful experiences through tactile responses, and then through non-evaluative, playful interactions that do not include people, text, numbers, or evaluation elements, before displaying a selection screen only when predetermined conditions are met. Furthermore, it integrates with an external voice dialogue API to enable voice dialogue after psychological stability is achieved, and allows for the detection of subtle changes in cognitive function through temporal analysis of conversation records.

Description

Technical Field

[0001] The present invention relates to an interactive support device and a display control method thereof that form psychological stability starting from unconscious reactions and gradually promote participation for elderly people including dementia patients without giving operation instructions, evaluations, or attention calls.

Background Art

[0002] With the progress of an aging society, the need for support for dementia patients is increasing. Conventionally, dementia support has relied on medical diagnoses or human responses by caregivers. Especially in the early stages of cognitive decline, there has been no means for users to receive mental support that can be provided 24 hours a day.

[0003] In addition, many existing dialogue devices and applications require users to answer questions, receive operation instructions, evaluations, memory recall, or cognitive function measurement, which may cause psychological burdens or rejection reactions for users with reduced cognitive function. In particular, voice dialogue systems that assume dialogue ability are difficult to use for users with reduced speech ability, and form a psychological barrier by requiring conversation from the initial screen.

[0004] Furthermore, conventional care support systems mainly aim at physical assistance or watching and monitoring, and lack continuous support means for the mental aspects of users. In particular, there has been a need for a technology that allows users to maintain active participation while maintaining psychological stability even during periods when caregivers are absent.

Prior Art Documents

Patent Documents

[0005] None in particular

Non-Patent Documents

[0006] [[ID=​​​​​​​​

[0007] The problem that this invention aims to solve is to realize a 24-hour psychological support device that can be permanently installed in the living space of elderly people with dementia or those in the pre-dementia stage, without requiring operational instructions or evaluations. In particular, there is a need for a technology that does not presuppose the ability to converse, but gradually builds psychological stability starting from unconscious responses, and ultimately enables active participation, including voice dialogue. [Means for solving the problem]

[0008] The interactive support device of the present invention comprises a screen display means, a contact detection means, and a control means, the control means being A first display state that shows a visual change without providing the user with operating instructions, warnings, questions, or evaluations. A second display state in which the aforementioned visual change disappears due to a contact response to the aforementioned visual change. A third display state that provides non-evaluative playful interaction, And the fourth display state, which displays a selection screen when a predetermined condition is met, This system is characterized by controlling transitions in a stepwise manner in this order.

[0009] Furthermore, the device of the present invention is further equipped with voice input / output means, and when the user makes a predetermined selection in the fourth display state, it can initiate voice dialogue in cooperation with a voice dialogue system. This makes it possible to provide voice dialogue functionality only after psychological stability has been established, without requiring conversational ability in the initial stages. [Effects of the Invention]

[0010] According to the present invention, it is possible to gradually guide users from unconscious responses to active participation without causing them psychological burden or giving them an experimental impression. This realizes a 24-hour mental support device that can be naturally installed in living spaces from the pre-stage of dementia progression, enabling mental care support that does not require constant intervention from caregivers. [Brief explanation of the drawing]

[0011] [Figure 1] This flowchart shows the display state transitions in an embodiment of the present invention. It shows the transition conditions from the first display state to the fourth display state and the display content in each state. [Figure 2] This is a block diagram showing the hardware configuration in an embodiment of the present invention. It illustrates the tablet terminal, control device, audio input / output device, and interaction with an external API. [Modes for carrying out the invention] [Examples]

[0012] An embodiment of the present invention will be described with reference to Figures 1 and 2. This device comprises a touch panel display device (e.g., tablet terminal, touch panel compatible PC), a control device, and a storage device. The control device has a processor and memory and executes a display control program. In this embodiment, a control program implemented using VBA (Visual Basic for Applications) on Microsoft Excel is used, but implementation using other programming languages ​​is also possible. First display state

[0013] When the device is started, the control device displays a first display state. In the first display state, the entire screen is displayed in a single color (e.g., white, light blue), and a visual change is generated in a part of the screen (e.g., the lower right corner, the upper left corner). In this embodiment, a circular area with a diameter of about 50 to 100 pixels is provided, and the brightness of this area is slowly changed with a period of 5 to 10 seconds.

[0014] Brightness changes are made sinusoidally or linearly within a range of luminance values ​​from 100 to 150. The rate of change is set so that it is visible to the user but does not draw attention to it. Importantly, no text, audio, operation instructions, or button displays are shown at this stage. This allows the user to naturally notice the visual change without feeling any psychological burden of having to do something. Second display state

[0015] When the user touches the circular area, the control device compares the contact coordinates with the coordinates of the circular area and detects the contact if they match. When detecting the contact, the control device causes the circular area to disappear by fade-out display over a period of 0.3 seconds to 1.0 seconds. The disappearance speed is set to a level where the user can recognize that "a change has occurred due to their own operation".

[0016] As a result, the user obtains a successful experience of "disappearing when touched" and a recognition of predictability that "their actions produce results". After disappearance, after a waiting time of 3 seconds to 5 seconds, a visual change is displayed again at another position on the screen (for example, if the previous one was the lower right, the next one is the upper left). By changing the display position randomly or regularly, the user's eye movement and spatial recognition are gently promoted.

[0017] The control device records the cumulative number of contact operations. In this embodiment, when the cumulative number reaches 3 or more and 5 or more, it is determined that a part of the transition condition to the third display state is satisfied. The threshold value of the cumulative number can be adjusted according to the user's cognitive function level. The third display state

[0018] When the number of contact operations accumulates to 5 or more, the control device transitions to the third display state. In the third display state, a landscape image (for example, mountains, sea, forest, grassland) is displayed on the entire screen or most of the screen. Importantly, this image does not include people, characters, numerical values, time, or evaluation elements. This is to avoid giving the user a psychological burden of "being watched by someone" or "having to read something".

[0019] When the user touches an arbitrary position on the screen, the control device gently changes a part of the image according to the contact position. For example, when touching an empty area, the shape of the cloud is changed, and when touching a sea area, the movement of the waves is changed. The change occurs in a visible form within 0.5 seconds to 2.0 seconds from the user's operation and remains within a predictable range (for example, the cloud moves a little, the waves become a little larger).

[0020] This interaction has no scores, time limits, correct / incorrect judgments, or goal achievement requirements. The user can enjoy the playful experience of "something changes when touched" without the pressure of evaluation or competition. The control device records the stay time in the third display state and the number of contact operations, and enables the transition to the fourth display state when predetermined conditions (e.g., stay time of 20 seconds or more and number of contacts of 10 times or more, or stay time of 30 seconds or more) are met. The fourth display state

[0021] When the third display state continues for 20 seconds or more and 30 seconds or more, and the cumulative number of contact operations reaches 10 times or more, the control device transitions to the fourth display state. In the fourth display state, three options are displayed in a large button format at the lower part or the center of the screen. Examples of options include "Continue", "Talk", "Rest", "End", etc.

[0022] There is no right or wrong for these options, and they are not for the purpose of measuring cognitive functions. The options aim to provide the user with a sense of subjectivity of "being able to choose by oneself" and a sense of involvement of "one's own will being respected". The buttons are displayed in a sufficient size (e.g., 150 pixels × 80 pixels or more) and clear colors (e.g., blue, green, gray) so that the elderly can easily operate them.

Example

[0023] This device may further include voice input / output means (a microphone and a speaker). When the user selects "Talk" or "Continue" in the fourth display state, the control device transitions to the voice dialogue mode.

[0024] In the voice dialogue mode, the control device communicates with an external voice dialogue API (e.g., ChatGPT API of OpenAI, Claude API of Anthropic) and generates a response according to the user's speech. Specifically, the following processes are executed: (1) Obtain the user's voice from the microphone (2) Convert the voice to text by a voice recognition API (e.g., Whisper API of OpenAI) (3) Send the text to the voice dialogue API and obtain a response text (4) Convert the response text into speech using a speech synthesis API (e.g., Google Text-to-Speech API). (5) Output audio from the speaker

[0025] Importantly, the voice dialogue function is not presented initially, but only as an option when the fourth display state is reached. This enables a gradual approach that does not presuppose conversational ability, allowing users to participate in voice dialogue only after achieving psychological stability. [Examples]

[0026] This device may further include screen capture means and character recognition means. In voice dialogue mode, the control device captures the entire screen at predetermined intervals (e.g., every 5 seconds, every 10 seconds) and saves it to a storage device.

[0027] OCR (Optical Character Recognition) technology (e.g., Tesseract OCR) is applied to the saved images to extract the conversation text. The extracted text, along with date and time information, is recorded in a database or spreadsheet (e.g., Excel). In this embodiment, systematic dialogue recording based on a 27-step question branching logic is possible.

[0028] The recorded data allows for the analysis of the user's language patterns, response times, and topic changes over time. This makes it possible to detect subtle changes in cognitive function before a clinical diagnosis can be made. However, this device is not intended for diagnostic purposes, but rather for providing information to the user and their caregiver. [Examples]

[0029] The hardware configuration of this device will be explained based on Figure 2. This device comprises the following components: (1) Touch panel display: Resolution of 1024 x 768 pixels or higher, screen size of 10 to 15 inches (2) Control device: CPU (e.g., Intel Core i3 or higher, ARM Cortex-A53 or higher), memory 4GB or more (3) Storage device: SSD or HDD, capacity of 128GB or more (4) Audio input / output devices: microphone, speaker (5) Network interface: Wi-Fi or wired LAN

[0030] The control program can be implemented as a VBA macro in Microsoft Excel. In this case, a Windows PC or tablet (Windows 10 or later) is used. Implementation using other programming languages ​​such as Python and JavaScript is also possible, and the platform can be Windows, macOS, Linux, Android, or iOS. [Examples]

[0031] The visual changes in the first display state are not limited to changes in brightness, but may also include changes in color tone (e.g., a gradient from blue to green), changes in shape (e.g., from a circle to an ellipse), changes in position (e.g., a gentle movement from left to right), or a combination thereof.

[0032] The image in the third display state is not limited to landscapes; it may also include abstract patterns, geometric shapes, or photographs of animals (excluding people). The important point is that it does not require the user to make any evaluation or judgment.

[0033] The number of options in the fourth display state is not limited to three; it may be two or four or more. However, too many options can burden the user, so two to five options is considered appropriate.

[0034] This device may support simultaneous use by two people (e.g., a married couple, or friends). In this case, the control device can detect multiple contact points and independently record the operations of each user. The two-person use mode promotes social interaction and reduces the psychological barrier to solo use. [Industrial applicability]

[0035] This invention can be widely used as a psychological support device for people with dementia and the elderly in homes, nursing homes, medical institutions, community comprehensive support centers, etc. In particular, it serves as a psychological companion that can be installed 24 hours a day, contributing to reducing the burden on caregivers and improving the quality of life (QOL) of users.

Claims

1. An interactive support device that provides psychological support to users, including those with dementia, It comprises a screen display means and a control means, The control means is A first display state that displays a visual change in which at least one of the brightness, color tone, shape, or position changes gradually in a part of the screen without providing the user with operating instructions, warnings, questions, or evaluations. A second display state in which the visual change disappears in response to a tactile reaction to the aforementioned visual change. A third display state that provides a non-evaluative, playful interaction. And a fourth display state in which a selection screen is displayed when predetermined conditions are met based on the number of user touch operations and the duration of stay in the third display state, An interactive support device characterized by transitioning between stages in this order.

2. In the interactive support device according to claim 1, The visual changes in the first display state are: Not the entire screen, but a portion of the screen, An interactive support device characterized by a gradual change in at least one of its properties: brightness, color tone, shape, or position.

3. In the interactive support device according to Claim 2, In the second display state, By eliminating the visual change in response to the aforementioned contact reaction, An interactive support device characterized by providing users with a sense of success and psychological stability.

4. In the interactive support device according to claim 3, The third display state is, An interactive support device characterized by providing non-evaluative, playful interactions using display content that does not include people, characters, numbers, time, or evaluation elements.

5. In the interactive support device according to Claim 4, In the third display state, An interactive support device characterized in that a portion of the displayed content changes gradually within a predictable range in response to user touch operations.

6. In the interactive support device according to claim 5, The aforementioned predetermined conditions are: This condition is met when the cumulative number of contact operations by the user exceeds a predetermined number, and the duration of the third display state exceeds a predetermined time. The aforementioned predetermined number of times is 3 to 10 times. An interactive support device characterized in that the predetermined time is 20 seconds or more and 60 seconds or less.

7. A display control method that provides psychological support to users, including those with dementia, Computers A first step involves displaying a visual change in which at least one of the brightness, color tone, shape, or position changes gradually in a portion of the screen without providing the user with operating instructions, warnings, questions, or evaluations; A second step involves eliminating the aforementioned visual change in response to a tactile reaction to the visual change, A third step involves providing a non-evaluative, playful interaction, The fourth step involves displaying a selection screen when predetermined conditions are met based on the number of user touch operations and the duration of stay in the third step, A display control method characterized by being executed in this order.

8. In the interactive support device according to claim 6, Furthermore, it is equipped with audio input / output means, The control means is When the user makes a predetermined selection in the fourth display state, It initiates voice interaction by integrating with an external voice dialogue API. The user's voice is converted to text using a speech recognition API. The text is sent to the external voice dialogue API to obtain the response text. An interactive support device characterized by converting the aforementioned response text into speech using a speech synthesis API and outputting it.

9. In the interactive support device according to claim 8, Furthermore, it is equipped with screen capture means and character recognition means, The control means is During voice dialogue, the screen is captured at predetermined intervals. The aforementioned character recognition means extracts conversational text, The aforementioned conversation text is recorded along with date and time information. An interactive support device characterized by analyzing changes in the user's language patterns over time from recorded data.

10. In the interactive support device according to claim 9, The control means is Direct transition from the first display state to the fourth display state is prohibited. The transition to the fourth display state is as follows: An interactive support device characterized in that it is only possible when the first display state, the second display state, and the third display state are passed through in this order. That's all.

Images