Processing system, processing method, processing device, and processing program

Abstract

The system provides a processing system and the like that suppresses the occurrence of dangers caused by the actions of the subject. [Solution] The processing system comprises a terminal device 3 equipped with an imaging unit for capturing images of a subject and an output unit for outputting information to the subject, a processing device 1 that performs processing based on image information obtained by imaging by the terminal device 3, and a notification device that makes notifications regarding the subject. The processing system detects a first action of the subject based on image information obtained by imaging by the imaging unit of the terminal device 3 (S302), and if a first action of the subject is detected (S302:YES), the output unit of the terminal device 3 outputs a suppression output to suppress a second action different from the first action (S305). Furthermore, the processing system detects a second action of the subject based on image information obtained by imaging by the imaging unit of the terminal device 3 (S307), and if a second action of the subject is detected (S307:YES), the notification device makes a notification (S308).

Description

Technical Field

[0001] The present application relates to a processing system using a terminal device including an imaging unit for imaging a target person, a processing method executed by such a processing system, a processing device used in such a processing system, and a processing program for realizing such a processing device.

Background Art

[0002] Various systems for caring for or nursing target persons such as the elderly, patients, and care recipients have been realized. For example, the applicant of the present application has proposed a biological information monitoring system that enables prompt response by an operator such as a doctor operating a terminal device when an abnormal value exists in the biological information detected from the target person.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In addition to monitoring biological information, an important matter in caring for or nursing a target person is suppressing or assisting operations that may cause danger.

[0005] The present application has been made in view of such circumstances, and discloses a processing system that makes a notification based on the actions of a target person.

[0006] The present application also discloses a processing method using the processing system disclosed in the present application.

[0007] Furthermore, the present application discloses a processing device used in the processing system disclosed in the present application and a processing program for realizing such a processing device.

Means for Solving the Problems

[0008] To solve the above problems, the processing system disclosed in this application comprises a terminal device having an imaging unit for imaging a subject and an output unit for outputting to the subject; a processing unit that performs processing based on image information obtained by imaging by the terminal device; and a notification device that makes a notification regarding the subject, characterized in that it comprises a first detection means for detecting a first action of the subject based on image information obtained by imaging by the imaging unit of the terminal device; an output means that, when the first detection means detects a first action of the subject, outputs a suppression output from the output unit of the terminal device to suppress a second action different from the first action; a second detection means for detecting a second action of the subject based on image information obtained by imaging by the imaging unit of the terminal device; and a first notification means that, when the second detection means detects a second action of the subject, makes a notification via the notification device.

[0009] Furthermore, the processing system is characterized in that the first detection means detects a first action in which the subject transitions from a lying position to a sitting position on the bed, and the second detection means detects a second action in which the subject attempts to get out of bed.

[0010] Furthermore, in the processing system, the output means is characterized by providing suppression output of at least one of video and audio from the output unit of the terminal device.

[0011] Furthermore, the processing system is characterized by comprising: a quantification means that quantifies the physical condition of the subject based on image information obtained by imaging by the terminal device; and a second notification means that notifies the notification device when the numerical value obtained by quantifying the physical information falls outside a preset standard value.

[0012] Furthermore, the processing system is characterized in that the digitization means generates a numerical value representing HbAlc based on image information including the face of the subject.

[0013] Furthermore, the processing system is characterized in that the terminal device performs authentication processing using an information storage medium that stores specific information to identify the target person.

[0014] Furthermore, the processing method disclosed in this application is characterized by using a terminal device equipped with an imaging unit for imaging a subject and an output unit for outputting to the subject, a processing unit that performs processing based on image information obtained by imaging by the terminal device, and a notification device that makes a notification regarding the subject, wherein the processing method detects a first action of the subject based on image information obtained by imaging by the imaging unit of the terminal device, and when the first action of the subject is detected, the output unit of the terminal device outputs a suppression output to suppress a second action different from the first action, and when the second action of the subject is detected based on image information obtained by imaging by the imaging unit of the terminal device, the notification device makes a notification.

[0015] Furthermore, the processing device disclosed herein is a processing device that can communicate with a terminal device equipped with an imaging unit for imaging a subject and an output unit for outputting to the subject, and performs processing based on image information obtained by imaging the subject, and is characterized by comprising: means for detecting a first action of the subject based on image information obtained by imaging by the imaging unit of the terminal device; means for causing the output unit of the terminal device to output a suppression output to suppress a second action different from the first action when the first action of the subject is detected; means for detecting a second action of the subject based on image information obtained by imaging by the imaging unit of the terminal device; and means for performing notification processing when the second action of the subject is detected.

[0016] Furthermore, the processing program disclosed in this application is a processing program that causes a computer capable of communicating with a terminal device equipped with an imaging unit for imaging a subject and an output unit for outputting to the subject to perform processing based on image information obtained by imaging the subject, characterized in that the computer is caused to perform the following steps: to detect a first action of the subject based on image information obtained by imaging by the imaging unit of the terminal device; to cause the output unit of the terminal device to output a suppression output to suppress a second action different from the first action when the first action of the subject is detected; to detect a second action of the subject based on image information obtained by imaging by the imaging unit of the terminal device; and to perform notification processing when the second action of the subject is detected. [Effects of the Invention]

[0017] The processing system disclosed in this application detects the movements of a subject based on images of that subject and issues a deterrent output or notification. This has excellent effects, such as suppressing the occurrence of dangers caused by the subject's movements. [Brief explanation of the drawing]

[0018] [Figure 1] This is a schematic diagram illustrating an example of the configuration of the processing system disclosed in this application. [Figure 2] This is a block diagram schematically showing an example of the configuration of various devices used in the processing system disclosed in this application. [Figure 3] This flowchart shows an example of the authentication process for terminal devices included in the processing system disclosed in this application. [Figure 4] This flowchart shows an example of image transmission and reception processing for the processing device and terminal device included in the processing system disclosed in this application. [Figure 5] This flowchart shows an example of the operation monitoring process for the processing device and terminal device included in the processing system disclosed in this application. [Figure 6] This flowchart shows an example of the physical information detection process of the processing device included in the processing system disclosed in this application.

Mode for Carrying Out the Invention

[0019] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the following embodiments are an example of embodying the present invention and do not have the character of limiting the technical scope of the present invention.

[0020] <Outline of the System> FIG. 1 is a schematic diagram schematically showing a configuration example of the processing system disclosed in the present application. The processing system disclosed in the present application is a system that monitors the states of actions, physical conditions, etc. of target persons such as patients, elderly people, and care recipients indoors in buildings such as hospitals, nursing homes, and care facilities. By performing the monitoring itself with a processing device 1 using a computer, it is possible to ensure the personal information and dignity of the target person as compared with the case where a person monitors.

[0021] In the building, there are rooms such as hospital rooms and bedrooms where the target person sleeps, and in the room, a bed 2 used by the target person is installed. Near the bed 2, holders 20 such as tablet arms and tablet stands that hold terminal devices 3 such as tablet computers and high-function mobile phones (so-called smartphones) are installed.

[0022] Furthermore, in the same or a different building, an installed or portable notification device 4 that notifies notifications such as a nurse call is installed or carried by a nurse. Note that those who receive notifications by the notification device 4 are not limited to nurses, but may be any person who supports the actions of the target person, such as a caregiver, a doctor, or a relative of the target person (hereinafter referred to as a supporter). Therefore, the notification device 4 can be realized as a system or device other than the nurse call system.

[0023] The processing unit 1, terminal device 3, and notification device 4 can communicate with the processing unit 1 via wireless or wired communication methods through a communication network NW such as a LAN (Local Area Network), WAN (Wide Area Network), or the Internet. The processing unit 1 may be installed inside a building or on the Internet.

[0024] The individuals concerned possess an information storage medium 5 that stores specific information identifying them, such as a My Number Card or various other ID cards.

[0025] <Hardware configuration of various devices> Next, the hardware configuration of the various devices used in the processing system disclosed in this application will be described. Figure 2 is a block diagram schematically showing an example of the configuration of the various devices used in the processing system disclosed in this application. In the following description, a processing unit 1 using a single server computer will be described, but it is also possible to configure the processing unit 1 by linking multiple various computers. For example, the processing unit 1 disclosed in this application can be configured by linking multiple server computers, such as an authentication server used for authentication processing described later, a motion analysis server used for motion monitoring processing, and a reference value server that stores reference values ​​used for determining physical condition.

[0026] The processing unit 1 is a computer-based device such as a server computer, and comprises a control unit 10, a storage unit 11, a communication unit 12, and other components. The control unit 10 includes various circuits such as an information processing circuit, a timing circuit, and a register circuit, and is controlled by a CPU that executes processes to control each part of the device. This refers to processors such as Central Processing Units (CPAs).

[0027] The storage unit 11 is a storage unit composed of non-volatile memory such as hard disks, SSDs (Solid State Drives), RAID (Redundant Arrays of Inexpensive Disks), and flash memory, and volatile memory such as various types of RAM (Random Access Memory). The storage unit 11 stores information such as basic programs (OS: Operating System), application programs that run on the basic program, and various data. As application programs, various programs such as processing programs 110 for realizing the processing device 1 disclosed in this application are stored. Examples of various data include data that is temporarily generated necessary for processing by the processing device 1 disclosed in this application, as well as specific information that identifies an individual subject, various information used for operation monitoring processing, and reference value information that indicates a reference value. The various information used for operation monitoring processing also includes information such as a learning model that has been generated by learning the actions of the subject and other subjects in advance through machine learning or other processing using training data. It is also possible to store specific information, reference value information, and other information in a database server that can be accessed as appropriate from the processing device 1 and to configure it as part of the processing device 1.

[0028] The communication unit 12 is a communication device such as a LAN adapter and control circuit, which connects to the communication network NW and communicates with various devices.

[0029] A computer such as a server computer having the various configurations exemplified above reads various programs such as the processing program 110 stored in the storage unit 11 under the control of the control unit 10, and executes various steps contained in the read program. In this way, the computer operates as the processing unit 1 disclosed in this application.

[0030] Terminal device 3 is operated by the subject or by a supporter such as a nurse assisting the subject's movements. Terminal device 3 is configured using a computer such as a tablet computer or a high-function mobile phone. Terminal device 30 has various components such as a control unit 30, a storage unit 31, an imaging unit 32, an operation unit 33, an image output unit 34, an audio output unit 35, a long-range communication unit 36, and a short-range communication unit 37. The control unit 30 is a processor that controls the entire device. The storage unit 31 is a storage unit configured using non-volatile memory and volatile memory.

[0031] The imaging unit 32 is an input interface used for receiving light from a camera or other device using an image sensor, and generates image information such as a series of still images or videos through imaging. The operation unit 33 is an input interface that accepts operation inputs such as a touch panel or push buttons. The image output unit 34 is an output interface for displaying images such as a liquid crystal monitor. The operation unit 33 and the image output unit 34 can also be configured as an integrated liquid crystal touch panel by superimposing a touch panel and a liquid crystal monitor. The audio output unit 35 is an output interface such as a speaker that outputs sound. These input and output interfaces can also be configured as external devices to the terminal device 3.

[0032] The long-range communication unit 36 ​​is a communication interface that connects to the communication network NW via wired or wireless means and communicates with various devices. The short-range communication unit 37 is a communication interface for short-range communication such as RFID readers and Bluetooth®, and is capable of communicating with the information storage medium 5.

[0033] A computer such as a tablet computer having the various configurations exemplified above will operate as the terminal device 3 disclosed in this application by, for example, executing an application program for terminal device 3 downloaded from processing unit 1.

[0034] The notification device 4 comprises various components, including a control unit 40, a communication unit 41, and a notification unit 42. The control unit 40 is a processor that controls the entire device. The communication unit 41 is a communication unit that communicates with the processing unit 1 and the terminal device 3 via a communication network NW. The notification unit 42 is a notification unit that notifies caregivers such as nurses using output methods such as voice, images, and vibrations.

[0035] The information storage medium 5 comprises various components, including an information storage unit 50 and a communication unit 51. The information storage unit 50 is a storage unit such as an IC chip that stores specific information, such as a digital signature certificate and a digital user authentication certificate, which identify the target person. The communication unit 51 is a communication unit such as a short-range wireless antenna that transmits specific information based on the power generated by electromagnetic waves from the reader.

[0036] <Processing of various devices> Next, the processing of various devices in the processing system disclosed in this application will be explained. Figure 3 is a flowchart showing an example of the authentication process of terminal device 3 in the processing system disclosed in this application. The user starts the authentication process by bringing an information storage medium 5 containing specific information that identifies them close to terminal device 3.

[0037] The terminal device 3, under the control of the control unit 30, reads specific information stored in the nearby information storage medium 5 using the short-range communication unit 37 (S101), and performs authentication processing by comparing the read specific information with information stored in the storage unit 11 of the processing unit 1 (S102). If authentication is successful, the control unit 30 starts executing processes such as motion monitoring processing and physical information detection processing for the target person (S103). The authentication process can be performed using various methods, such as matching with an ID and password.

[0038] As described above, terminal device 3 performs the authentication process.

[0039] Figure 4 is a flowchart illustrating an example of the image transmission and reception processing of the processing system 1 and terminal device 3 provided in the processing system disclosed in this application. The processing system disclosed in this application performs motion monitoring processing to detect and notify actions such as when a subject lying on bed 2 independently gets off bed 2. The processing system 1 and terminal device 3 perform image transmission and reception processing as a prerequisite for the motion monitoring processing.

[0040] The control unit 30 of the terminal device 3 captures an image of the subject using the imaging unit 32 (S201), and transmits the image information obtained from the imaging to the processing unit 1 via the communication network NW from the long-distance communication unit 36 ​​(S202). The processing in steps S201 to S202 may be carried out by transmitting the image information obtained each time an image is captured, or by transmitting it as a stream.

[0041] The control unit 10 of the processing unit 1 receives image information via the communication unit 12 (S203).

[0042] As described above, the processing unit 1 and the terminal device 3 execute the image transmission and reception process. The image transmission and reception process is executed continuously and repeatedly while the operation monitoring process is running.

[0043] Figure 5 is a flowchart showing an example of the operation monitoring process for the processing unit 1 and terminal device 3 provided in the processing system disclosed in this application. The processing unit 1 of this application performs the operation monitoring process by executing each step included in the processing program 110 stored in the storage unit 11 under the control of the control unit 10.

[0044] The control unit 10 of the processing unit 1 analyzes the subject's actions based on the received image information (S301) and determines whether or not it has detected that the subject has performed the first action (S302). The analysis of the subject's actions in step S301 and subsequent processing involves analyzing the image information and storing it. This is performed by comparing various data stored in unit 11 and by processing using AI (Artificial Intelligence) based on the learning model stored in memory unit 11. The subject's first action in step S302 is set to be, for example, an action of transitioning from a supine position, which is the standard state, to a sitting position, that is, an action of the subject, who was lying in bed 2, getting up.

[0045] In step S302, if it is determined that the subject has performed the first action (S302:YES), the control unit 10 of the processing unit 1 transmits suppression information to the subject via the communication network NW from the communication unit 12 to the terminal device 3 (S303). The second action is a different action from the first action that is performed after the first action. For example, the action of getting off the bed 2 is set to be performed after the action of transitioning to a seated position.

[0046] The control unit 30 of the terminal device 3 receives suppression information via the long-range communication unit 36 ​​(S304), and based on the received suppression information, outputs a suppression signal from the image output unit 34 and / or the audio output unit 35 to suppress the second action (S305). The suppression signal in step S305 is, for example, a process that outputs a message such as "Please press the nurse call button when getting out of bed 2" as an image from the image output unit 34 and / or as audio from the audio output unit 35.

[0047] If it is determined in step S302 that the subject has not performed the first action (S302: NO), the control unit 10 of the processing unit 1 returns to step S301 and repeats the subsequent processing.

[0048] After transmitting the suppression information in step S303, the control unit 10 of the processing unit 1 further analyzes the subject's actions based on the received image information (S306) and determines whether or not it has detected that the subject has performed a second action (S307).

[0049] In step S307, if it is determined that the subject has performed a second action (S307: YES), the control unit 10 of the processing unit 1 executes a notification process to send a notification via the notification device 4 (S308). The notification process in step S308 is, for example, a process to notify a nurse or other support person by making a nurse call using a nurse call system. The nurse or other support person who receives the notification takes measures such as assisting the subject who is trying to get out of bed 2. Alternatively, by configuring the notification device 4 operated by the support person as a computer such as a tablet computer, or by providing a separate device, video communication may be initiated between the terminal device 2 and the notification device 4 when it is detected that the subject has performed a second action. By initiating video communication, the subject and the support person can communicate, and the support person can encourage the subject to take safe actions. Note that video communication between the terminal device 2 and the notification device 4 may be initiated when the first action is detected. Furthermore, the processing system disclosed in this application can be deployed in various forms, such as starting voice-only communication when a first action is detected, and starting voice and video communication when a second action is detected.

[0050] If, in step S307, it is determined that the subject has not performed the second action (S307: NO), the control unit 10 of the processing unit 1 further analyzes the subject's actions based on the received image information (S309) and determines whether or not it has detected that the subject has returned to the baseline state (S310).

[0051] If it is determined in step S310 that the subject has returned to the baseline state (S310:YES), the control unit 10 of the processing unit 1 returns to step S301 and repeats the subsequent processing.

[0052] If it is determined in step S310 that the subject has not returned to the baseline state (S310: NO), the control unit 10 of the processing unit 1 returns to step S306 and repeats the subsequent processing.

[0053] As described above, the processing unit 1 and the terminal device 3 perform operation monitoring processing. The processing system disclosed in this application monitors the actions of the subject by computer through operation monitoring processing and performs suppression output and notification processing as appropriate.

[0054] Figure 6 is a flowchart showing an example of the physical information detection process of the processing device 1 in the processing system disclosed in this application. The processing device 1 of this application performs the physical information detection process by executing each step included in the processing program 110 stored in the storage unit 11 under the control of the control unit 10. The physical information detection process is performed based on image information received by the image transmission and reception process, but it may also be performed by having the subject separately capture images of their own state and transmit the resulting image information to the processing device 1. Furthermore, the processing device 1 may start the physical information detection process when the subject or supporter inputs a predetermined operation using a device such as a terminal device 3. Moreover, in the motion monitoring process, it may be performed on image information based on video footage for video communication as an ancillary process in video communication that is started when a first or second motion is detected.

[0055] The control unit 10 of the processing device 1 quantifies physical information indicating the physical state of the subject based on the received image information (S401). The physical information of the subject in step S401 refers to information about the subject's body, such as heart rate, respiration, blood oxygen, stress level, blood pressure, and HbA1c. As for the quantification of physical information based on image information in step S401, for example, physical information such as heart rate and respiration can be quantified by reading the image of the face from the image information of the subject and quantifying the pulsation period of the face. Also, for example, physical information such as blood pressure can be quantified by comparing the color obtained from the image of the face with past states. Such techniques for obtaining physical information of a subject quantified from image information are introduced in non-patent literature such as "Shohei Koyama, Measurement of Life Data (Trends in Sensor Devices and Wearable Sensors), Measurement and Control, The Society of Instrument and Control Engineers, Vol. 59, No. 9, September 2020."

[0056] The control unit 10 determines whether the numerical value obtained by digitizing the physical information deviates from a reference value that is pre-set as reference value information stored in the memory unit 11 (S402). In step S402, deviation from the reference value means, for example, if the reference value indicates an upper limit, the control unit 10 determines whether the numerical value obtained by digitization exceeds the upper limit. If the reference value indicates a lower limit, the control unit 10 determines whether it falls below the lower limit. If the reference value indicates a range between the upper limit and the lower limit, the control unit 10 determines whether it falls outside the range.

[0057] In step S402, if the numerical value obtained by quantifying the physical information is determined to be outside the standard range (S402: YES), the control unit 10 executes a notification process to send a notification via the notification device 4 (S403). The notification process in step S403 is, for example, a notification to a support person such as a doctor or nurse.

[0058] In step 402, if it is determined that the numerical value is not outside the reference range (S402: NO), the control unit 10 of the processing unit 1 either terminates the process or returns to step S401 to repeat the process.

[0059] As described above, the processing unit 1 performs body information detection processing.

[0060] As illustrated above with the embodiments, the processing system disclosed in this application performs various processes such as authentication processing, motion monitoring processing, and physical information detection processing. For example, the motion monitoring processing detects a first action by the subject, such as getting up from a lying position in bed 2, and outputs a suppression signal through automated voice and image processing. When a second action, such as getting out of bed 2, is detected, it notifies a caregiver such as a nurse. As a result, the processing system disclosed in this application has excellent effects, such as being able to suppress the occurrence of dangers caused by the subject's actions, such as the danger of falling when getting out of bed 2. Moreover, the processing system disclosed in this application has excellent effects, such as reducing the burden on caregivers because they do not need to constantly monitor the subject, and protecting the subject's personal information and dignity because the monitoring is done by a computer.

[0061] Furthermore, the processing system disclosed in this application manages physical information based on image information, thus reducing the burden of examinations on the subjects, and thus exhibiting other excellent effects.

[0062] The present invention is not limited to the embodiments described above and can be implemented in various other forms. Therefore, the embodiments described above are merely illustrative in all respects and should not be interpreted restrictively. The scope of the present invention is defined by the claims and is not restricted in any way by the text of the specification. Furthermore, any modifications or changes falling within the equivalent scope of the claims are all within the scope of the present invention.

[0063] For example, in the above embodiment, the processing unit 1 performs processing to detect the first operation, but the processing system disclosed in this application is not limited to this, and processing can be appropriately divided, for example, with the terminal device 3 performing the processing from the detection of the first operation to the suppression output. In other words, the processing system disclosed in this application can be configured not only to construct the processing unit 1 by distributing it among multiple devices, but also to distribute various processing among various devices, including the processing unit 1 and the terminal device 3.

[0064] Furthermore, in the above embodiment, the first action was shown as transitioning from a lying position to a sitting position on the bed 2, and the second action as attempting to get off the bed 2. However, the processing system disclosed in this application allows for appropriate settings of the first action and the second action performed after the first action. For example, the processing system disclosed in this application can be expanded into various forms, such as preventing wandering by setting the first action as walking around inside the house and the second action as attempting to leave the house. [Explanation of Symbols]

[0065] 1 Processing Unit 10 Control Unit 11 Storage section 110 Processing Program 12 Communications Department 2 beds 20 Retainer 3 Terminal devices 30 Control Unit 31 Storage section 32 Imaging Unit 33 Operation section 34 Image output section 35 Audio output section 36 Long-distance communications department 37 Short-Range Communications Department 4 Notification device 40 Control Unit 41 Communications Department 42 Reporting Department 5 Information storage medium 50 Information storage section 51 Communications Department Network

Claims

1. A terminal device comprising an imaging unit for imaging a subject and an output unit for outputting information to the subject, A processing device that performs processing based on image information obtained by imaging by the terminal device, A notification device that notifies supporters of the aforementioned person of information about the aforementioned person, Equipped with, A first detection means for detecting the first action of the subject based on image information obtained by imaging by the imaging unit of the terminal device, When the first detection means detects the first action of the subject, the terminal device's output unit outputs a suppression output to suppress a second action that is different from the first action, A second detection means for detecting a second action of the subject based on image information obtained by imaging by the imaging unit of the terminal device, The first notification means, when the second detection means detects the second action of the subject, notifies the notification device. A processing system characterized by comprising the following:

2. A processing system according to claim 1, The first detection means detects a first action in which the subject transitions from a lying position to a sitting position on the bed. The second detection means detects a second action in which the subject attempts to get out of bed. A processing system characterized by the following:

3. A processing system according to claim 1 or claim 2, The suppression output means outputs suppression signals consisting of at least one of video and audio from the output unit of the terminal device. A processing system characterized by the following:

4. A processing system according to claim 1 or claim 2, A digitization means that quantifies the physical condition of the subject based on image information obtained by imaging by the terminal device, A second notification means that, when the numerical value obtained by quantifying the physical condition falls outside a predetermined standard value, notifies the system via the notification device. A processing system characterized by comprising the following:

5. The processing system according to claim 4, The digitization means generates a numerical value representing HbAlc based on image information including the subject's face. A processing system characterized by the following:

6. A processing system according to claim 1 or claim 2, The terminal device performs authentication processing using an information storage medium that stores specific information to identify the target person. A processing system characterized by the following:

7. A terminal device comprising an imaging unit for imaging a subject and an output unit for outputting information to the subject, A processing device that performs processing based on image information obtained by imaging by the terminal device, A notification device that notifies supporters of the aforementioned person of information about the aforementioned person, Using, Based on the image information obtained by imaging by the imaging unit of the terminal device, the first action of the subject is detected. When the first action of the subject is detected, the output unit of the terminal device outputs a suppression output to suppress a second action that is different from the first action. Based on the image information obtained by the imaging unit of the terminal device, the second action of the subject is detected. When the second action of the aforementioned subject is detected, the notification device will issue a notification. A processing method characterized by comprising:

8. A terminal device comprising an imaging unit for imaging a subject and an output unit for outputting information to the subject, and a processing device capable of communicating with a notification device for providing information about the subject to a supporter assisting the subject, and which performs processing based on image information obtained by imaging the subject, Means for detecting the first action of the subject based on image information obtained by imaging by the imaging unit of the terminal device, When the first action of the subject is detected, the terminal device's output unit provides a means to cause a suppression output to suppress a second action that is different from the first action, A means for detecting the second action of the subject based on image information obtained by imaging by the imaging unit of the terminal device, A means for causing the notification device to perform notification processing when the second action of the subject is detected. A processing apparatus characterized by comprising:

9. A processing program that causes a computer capable of communicating with an imaging unit for imaging a subject and an output unit for outputting information to the subject, and an information notification device for providing information about the subject to a supporter assisting the subject, to perform processing based on image information obtained by imaging the subject, To the aforementioned computer, The steps include detecting the first action of the subject based on image information obtained by imaging by the imaging unit of the terminal device, When the first action of the subject is detected, the terminal device's output unit outputs a suppression output that suppresses a second action different from the first action. A step of detecting the second action of the subject based on image information obtained by imaging by the imaging unit of the terminal device, The step of causing the notification device to perform notification processing when the second action of the subject is detected. A processing program characterized by causing the execution of a certain action.

Images