Disability support system

Abstract

This invention aims to comprehensively evaluate linguistic (speech characteristics) and non-linguistic (facial expressions and behavioral characteristics) aspects by synchronously analyzing audio and video data. It also aims to quantitatively grasp the effectiveness of support and identify effective support methods. [Solution] The disability support system 1 connects a user terminal 3 and a disability support site 5 via a communication network 17, and provides support from the disability support site 5 to the user terminal 3. The user terminal 3 is equipped with a microphone 3c to acquire voice data of a person with a disability and a video camera 3b to acquire video data of a person with a disability in order to receive disability support. The disability support site 5 is equipped with a voice data acquisition and analysis unit 5e that acquires and analyzes voice data, a video data acquisition and analysis unit 5f that acquires and analyzes video data, and an integrated analysis unit 5g that performs integrated analysis of the analysis results of the voice data and the video data.

Description

Technical Field

[0001] The present invention relates to a disabled person support system.

Background Art

[0002] In recent years, the opportunities for disabled people to enter society have been increasing. Many disabled people also have special abilities, and exploring the suitability and potential abilities of disabled people is also included in the support for disabled people.

[0003] Patent Document 1 provides a disabled person support device that can provide an appropriate training method. The disabled person support device 1 includes an input unit that inputs information of a checklist on which a plurality of question contents related to health and life, movement and sensation, cognition and behavior, language and communication, and human relations and sociality are described, a diagnosis unit that diagnoses based on the information input to the input unit and outputs, as diagnosis results, items that need to be preferentially solved, and a support plan generation unit that generates an individual support plan based on the diagnosis results by the diagnosis unit. The plurality of question contents described in the checklist are respectively associated with the stages of sensation, perception, cognition, and behavior in the information processing process of the brain, and the diagnosis unit discloses a technique of outputting, in association with the information processing process of the brain, the reasons why the items that need to be preferentially solved cannot be achieved and the training methods.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, traditional assessments of the suitability of individuals with disabilities have relied solely on information gathered through interviews. Consequently, because only audio data is managed individually, comprehensive analysis and quantitative effectiveness measurement, such as combining audio and video data, have been difficult. Furthermore, traditional approaches have been limited to providing general support policies, making it difficult to propose specific support methods tailored to individual characteristics and to measure their effectiveness.

[0006] This invention aims to comprehensively evaluate linguistic (speech characteristics) and non-linguistic (facial expressions and behavioral characteristics) aspects by synchronously analyzing audio and video data. It also aims to quantitatively grasp the effectiveness of support and identify effective support methods. [Means for solving the problem]

[0007] The present invention has been made in view of the above problems, and the invention of claim 1 is a disability support system that connects a user terminal and a disability support site via a communication network and provides support from the disability support site to the user terminal, wherein the user terminal is equipped with a microphone for acquiring voice data of a person with a disability and a video camera for acquiring video data of a person with a disability, and the disability support site is equipped with a voice data acquisition and analysis unit for acquiring and analyzing the voice data, a video data acquisition and analysis unit for acquiring and analyzing the video data, and an integrated analysis unit for integrating and analyzing the analysis results of the voice data and the analysis results of the video data.

[0008] The invention according to claim 2 is a disability support system in which the audio data and the video data are multimodal data.

[0009] The invention according to claim 3 is a disability support system comprising a support plan generation unit that identifies an effective support pattern for an individual from the time-series data of the voice data and the video data, and proposes a support plan in order to generate an individually optimized support plan.

[0010] The invention according to claim 4 is a disability support system that continuously monitors the effectiveness of support and adjusts the support plan in accordance with the monitoring.

[0011] The invention according to claim 5 is a disability support system in which the voice data acquisition and analysis unit, the video data acquisition and analysis unit, and the integrated analysis unit perform analysis using an artificial intelligence unit.

[0012] The invention according to claim 6 is a disability support system in which the artificial intelligence unit is configured to communicate via an API (Application Programming Interface). [Effects of the Invention]

[0013] As described above, the present invention has the effect of enabling the integrated analysis of various data (audio data, video data, etc.) of persons with disabilities, thereby providing effective support. [Brief explanation of the drawing]

[0014] The drawings illustrate specific embodiments of the present invention, including not only essential components of the invention but also optional and preferred embodiments. [Figure 1] This is a schematic diagram of the disability support system. [Figure 2] This is a schematic diagram of the user terminal configuration. [Figure 3] This is a schematic diagram of the structure of a website supporting people with disabilities. [Figure 4] This is an explanatory diagram describing the artificial intelligence department. [Figure 5] This is a flowchart of a disability support system. [Modes for carrying out the invention]

[0015] The following describes each embodiment in detail with reference to the attached drawings. In this example, publicly known technologies are omitted from the explanation. Furthermore, the devices and methods described are illustrative examples for realizing the technical idea of ​​the invention, and the technical idea of ​​the present invention is not limited to those described below. The technical idea of ​​the present invention can be modified in various ways within the scope of the claims. In particular, it should be noted that the drawings are schematic and may differ from reality.

[0016] Figure 1 is a schematic diagram of the disability support system 1 of this embodiment. The disability support system 1 of this embodiment is a system that connects a user terminal 3 (a personal computer or smartphone, etc.), a disability support site 5 (a computer server system), an audio data analysis server 9 (a computer server system), a video data analysis server (a computer server system) 11, and an integrated analysis server 15 (a computer server system) via a communication network, which is the Internet network (Internet Protocol) 17.

[0017] The disability support site 5 is equipped with a server 5a, and the server 5a is equipped with a control unit 5b. Details of the control unit 5b will be described later.

[0018] The audio data analysis server 9 is configured to communicate with the server of the disability support site 5 via API (Application Programming Interface) 7. Similarly, the video data analysis server 11 is configured to communicate with the server 5a of the disability support site 5 via API (Application Programming Interface) 7. The integrated analysis server 15 is configured to communicate with the server 5a of the disability support site 5 via API (Application Programming Interface) 13.

[0019] Refer to FIG. 2. The user terminal 3 consists of a computer, and includes a CPU (Central Processing Unit) 3a, a ROM (Read-only Memory) 3g that stores a control program operating on the CPU 3a, various data, etc., and a RAM (Random Access Memory) 3f for temporarily storing various data. Further, for example, it includes a microphone 3c, a video camera 3b, and a control unit 3h.

[0020] Refer to FIG. 3. The control unit 5b of the server 5a of the disabled person support site 5 includes a storage unit 5d for storing voice data, a storage unit 5c for storing video data, a voice data acquisition and analysis unit 5e for acquiring and analyzing voice data, a video data acquisition and analysis unit 5f for acquiring and analyzing video data, an integrated analysis unit 5g for integrally analyzing the analyzed voice data and the analyzed video data, and a support plan generation unit 5h for generating an appropriate support plan from the analysis results of the integrated analysis unit 5g.

[0021] Refer to FIG. 4. The voice data acquisition and analysis unit 5e for acquiring and analyzing voice data, the video data acquisition and analysis unit 5f for acquiring and analyzing video data, the integrated analysis unit 5g for integrally analyzing the analyzed voice data and the analyzed video data, and the support plan generation unit 5h for generating an appropriate support plan from the analysis results of the integrated analysis unit 5g preferably include artificial intelligence units 5e, 5f, 5g, 5h. Specifically, it has a generation AI (artificial intelligence) function and outputs an answer to the "prompt" which is a question.

[0022] Explain the uniqueness of the disabled person support system 1 configured as described above.

[0023] Integrated accumulation and analysis of multimodal data. · Voice data and video data can be analyzed synchronously, and the language aspect (characteristics of speech) and non-language aspect (facial expressions and behavior characteristics) can be integrated and comprehensively evaluated as integrated data 20. • The effectiveness of support can be quantitatively assessed (e.g., 80% improvement in understanding through visual support), and effective support methods can be identified. In contrast, with conventional systems, data is managed individually, making such comprehensive analysis and quantitative effect measurement difficult.

[0024] Generating individually optimized support plans. • By analyzing data over time, it is possible to identify effective support patterns for individuals and propose specific support measures (such as visualizing work procedures and setting break times). • The effectiveness of support can be continuously monitored, and the plan can be flexibly adjusted. In contrast, conventional systems only provided general support policies, making it difficult to propose specific support methods tailored to individual characteristics or to measure their effectiveness.

[0025] The process for proposing appropriate support is explained below using a flowchart, referring to Figure 5. The optimal support proposal is generated by AI through the following steps.

[0026] First, in step S1, the control unit 5b performs multimodal data acquisition. The data acquisition scene and specific data are as follows. 1. Interview scene: Content of counseling, changes in facial expressions, interaction in the dialogue, etc. 2. Work / Activity Scenarios: Speech, actions, efficiency, etc., during work. 3. Everyday life situations: daily activities, interpersonal interactions, habits, etc. 4. Group activity settings: communication within the group, interaction with others, etc.

[0027] In step S2, the control unit 5b performs data storage. It then constructs a personal database, manages time-series change data, and stores quantitative evaluation data of the support effect.

[0028] In step S3, the control unit 5b performs data analysis. The analysis method is as follows: 1. Individual data analysis. • Audio: Speech content, emotion, and pattern. • Images: Facial expressions, posture, and behavioral characteristics. • Activities: work efficiency, interpersonal relationships, etc. 2. Integrated analysis. • Correlation analysis of multimodal data. • Analysis of change patterns over time. • Quantitative evaluation of support effectiveness.

[0029] In step S4, the control unit 5b utilizes the analysis data. In other words, it includes the following: Identifying patterns of change in individual characteristics. • Correlation analysis with support effects. • Identifying effective support methods.

[0030] In step S5, the control unit 5b generates the optimal support. In other words, it includes the following: • Development of a predictive support plan. • Proposal of individualized and specific support measures. • Setting goals in stages.

[0031] In step S6, the control unit 5b implements and adjusts the plan. In other words, it includes the following: • Continuous monitoring of support effectiveness • Real-time planning adjustments • Reflecting feedback

[0032] I will explain using specific examples. [Case 1: A man in his 20s with an intellectual disability] Control unit 5b revealed from the recorded data of the interview that the user spoke slowly, gave many single-word responses, and took a long time to respond to questions. Furthermore, video data of the work scene, also analyzed by control unit 5b, showed that the user's concentration decreased after about 30 minutes, and that they had particular difficulty with detailed tasks.

[0033] Based on past data, the control unit 5b has confirmed that introducing visual aids improves work comprehension and increases work efficiency in small groups of 3-4 people. Furthermore, analysis by time of day has shown that work efficiency is particularly high in the morning.

[0034] Control unit 5b indicates that this individual requires time to process information but can effectively utilize visual information. Therefore, support combining the introduction of a work procedure visualization tool, 30-minute breaks, and activities in small groups is considered optimal. In fact, this support has extended the duration of work from 30 minutes to 45 minutes, and the accuracy of understanding instructions has also improved.

[0035] [Case 2: A woman in her 30s with autism spectrum disorder] Control unit 5b revealed from the recorded data of the interview that the conversation was one-sided and frequently used specialized vocabulary. Video data also revealed a tendency to avoid eye contact. During work, the individual exhibited a strong fixation on specific tasks and maintained the same posture for extended periods.

[0036] Data from the past three months, as measured by control unit 5b, shows that adjusting the environment reduces panic, and stable activity is possible in situations where the work content and procedures are clearly structured. In particular, it has been confirmed that a high level of concentration can be maintained for more than two hours when working in one's area of ​​expertise.

[0037] Control unit 5b determined that this individual has strong sensory hypersensitivity and anxiety about schedule changes, but also possesses high expertise in a specific field. Therefore, support combining the provision of an individual workspace, the introduction of a visual schedule, and the assignment of roles that utilize their expertise is considered optimal. As a result of this support, the occurrence of panic attacks decreased from three times a week to once a month, and work efficiency improved by 20%. Furthermore, the appropriateness of speech improved from 40% to 70%.

[0038] Although each embodiment has been described in detail above, the invention is not limited to any particular embodiment, and various modifications and changes are possible within the scope described in the claims. Furthermore, it is possible to combine all or more of the components of the embodiments described above. [Explanation of Symbols]

[0039] 1. Support system for people with disabilities 3. User terminals 5. Websites supporting people with disabilities 7 API 9. Voice data analysis server 11. Video data analysis server 13 API 15. Integrated Analysis Server 17. Internet network (communication network)

Claims

1. A disability support system that connects a user terminal and a disability support site via a communication network, and provides support from the disability support site to the user terminal, The user terminal is equipped with a microphone to acquire voice data of a person with a disability and a video camera to acquire video data of a person with a disability in order to receive assistance from the person with a disability. The disability support site is a disability support system comprising: an audio data acquisition and analysis unit for acquiring and analyzing the audio data; a video data acquisition and analysis unit for acquiring and analyzing the video data; and an integrated analysis unit for integrating and analyzing the analysis results of the audio data and the video data.

2. The disability support system according to claim 1, wherein the audio data and the video data are multimodal data.

3. The disability support system according to claim 2, further comprising a support plan generation unit that identifies an effective support pattern for an individual from the time-series data of the audio data and the video data, and proposes a support plan in order to generate an individually optimized support plan.

4. The disability support system according to claim 3, comprising continuously monitoring the effectiveness of the support and adjusting the support plan in accordance with the monitoring.

5. The disability support system according to claim 1, wherein the voice data acquisition and analysis unit, the video data acquisition and analysis unit, and the integrated analysis unit perform analysis using an artificial intelligence unit.

6. The disability support system according to claim 5, wherein the artificial intelligence unit is configured to communicate via API (Application Programming Interface).

Images