Information processing apparatus, evaluation method and control program

Abstract

To appropriately evaluate an action and a situation of a cared person.SOLUTION: An information processing apparatus 10 comprises: a first acquisition unit 111 that acquires first information and second information for a cared person obtained within a predetermined period; a second acquisition unit 112 that acquires recorded information recorded in relation to the cared person before or after the predetermined period; and an evaluation unit 114 that evaluates the cared person on the basis of the acquired first information, second information and recorded information.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an evaluation method, and a control program. [Background technology]

[0002] There are technologies that recognize human actions using images. Furthermore, there are technologies that use not only images but also sound and temperature to improve the accuracy of action recognition. For example, the technology disclosed in Patent Document 1 monitors a person's cooking actions in a kitchen space and recognizes the type of person's action from image, sound, and temperature information. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-101240 Summary of the Invention [Problem to be solved by the invention]

[0004] The technology of Patent Document 1 is expected to produce speech related to actions when the person being photographed is conscious of being recognized while being photographed, such as cooking in a kitchen. Therefore, although the technology of Patent Document 1 functions effectively, it is difficult to apply it to residents of facilities such as nursing homes.

[0005] Residents in facilities are not always aware that their actions are being recognized through photography, and there are cases where their actions and behaviors are unrelated to or contradict the content of their speech. For example, if a resident falls indoors and says, "I'm fine," image analysis alone will recognize the action and behavior of the fall, but when combined with the results of voice recognition, it can lead to an incorrect behavioral estimation, such as "I fell, but I'm safe."

[0006] The present invention has been made in view of the above circumstances, and aims to appropriately evaluate the behavior or situation of a care recipient. [Means for solving the problem]

[0007] The above object of the present invention can be achieved by the following means.

[0008] (1) a first acquisition unit that acquires first and second information of a care recipient obtained within a predetermined period; a second acquisition unit that acquires recorded information about the care recipient before or after the predetermined period; an evaluation unit that evaluates the care recipient based on the acquired first information, second information, and record information; An information processing device comprising:

[0009] (2) The information processing device described in (1) above, wherein the evaluation includes information related to the needs of the care recipient.

[0010] (3) The information processing device according to (1) or (2), wherein the second acquisition unit acquires the record information relating to the care recipient from a time period earlier than the predetermined time period.

[0011] (4) The information processing device according to (1) or (2) above, wherein the first and second information and the recorded information are multimodal information that combines information obtained from mutually different information formats.

[0012] (5) The first information includes at least one of image information and image analysis information obtained by analyzing the image information, the second information includes at least one of audio information and audio analysis information obtained by analyzing the audio information, The information processing device according to (1) or (2), wherein the recorded information includes at least one of text information and text analysis information obtained by analyzing the text information.

[0013] (6) The information processing device described in (2) above, wherein the evaluation unit includes a machine-learned model trained to output information related to the needs of the care recipient.

[0014] (7) The first information and the second information are simultaneously sensing data obtained by sensing the care recipient, or information obtained by analyzing the sensing data, The information processing device according to (1) or (2) above, wherein the recorded information is information recorded by a staff member providing care to the care recipient.

[0015] (8) The first information is behavioral information of the care recipient obtained by analyzing an image acquired from a camera that photographs the care recipient, The second information is voice analysis information obtained by analyzing voice acquired from microphones placed around the care recipient, The information processing device according to (7) above, wherein the evaluation unit modifies the behavioral information based on the voice analysis information and the recording information.

[0016] (9) Step (a) of acquiring first and second information of the care recipient obtained within a predetermined period of time; (b) obtaining recorded information recorded regarding the subject of care before or after the predetermined period of time; (c) evaluating the care recipient based on the first and second information and the recorded information obtained in steps (a) and (b); Evaluation methods including.

[0017] (10) Step (a) of acquiring first and second information of the care recipient obtained within a predetermined period of time; (b) obtaining recorded information recorded regarding the subject of care before or after the predetermined period of time; (c) evaluating the care recipient based on the first and second information and the recorded information obtained in steps (a) and (b); A control program for causing a computer to execute a process including the above. [Effects of the Invention]

[0018] The information processing device of the present invention includes a first acquisition unit that acquires first and second information about a care recipient obtained within a predetermined period of time, a second acquisition unit that acquires record information recorded about the care recipient before or after the predetermined period of time, and an evaluation unit that evaluates the care recipient based on the acquired first and second information and record information, thereby appropriately evaluating the behavior or situation of the care recipient. [Brief explanation of the drawings]

[0019] [Figure 1] FIG. 1 is a schematic diagram illustrating an information processing apparatus according to an embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram showing the configuration of a detection unit. [Figure 3] FIG. 2 is a block diagram showing the configuration of a control device. [Figure 4] FIG. 1 is a block diagram showing a configuration of an information processing device. [Figure 5] FIG. 2 is a schematic diagram showing a data flow. [Figure 6] 10 is a flowchart illustrating an evaluation method using an information processing device. [Figure 7] 7 is a subroutine flowchart showing the processing of step S03 in FIG. 6. [Figure 8A] 10 is an example of an evaluation result based on the first to third information. [Figure 8B] 10 is an example of an evaluation result based on the first to third information. DETAILED DESCRIPTION OF THE INVENTION

[0020] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, the scope of the present invention is not limited to the disclosed embodiments. In the description of the drawings, the same elements are denoted by the same reference numerals, and duplicate explanations will be omitted. Furthermore, the dimensional proportions in the drawings are exaggerated for the convenience of explanation and may differ from the actual proportions.

[0021] FIG. 1 is a schematic diagram illustrating an information processing system 1000. The information processing system 1000 includes an information processing device 10, a detection unit 20, and a control device 30, and these devices are connected to each other via a network 50. The information processing device 10 is a server or a personal computer (PC). When the information processing device 10 is configured as a server, it may be an on-premise server installed in a nursing care facility or a cloud server using a commercial cloud service. The control device 30 is a PC and is operated by users such as nursing care staff (hereinafter simply referred to as staff) and administrators at the nursing care facility. The control device 30 is installed in the nursing care facility and functions as an edge server. The detection unit 20 will be described first, followed by the control device 30 and the information processing device 10. In the information processing system 1000 illustrated in FIG. 1, a detection unit 20 and a control device 30 are installed in each of multiple rooms in the nursing care facility. An example is shown in which each device in the nursing care facility is connected to the information processing device 10 via a network 50. That is, the devices 20 and 30 in one nursing care facility are connected to the information processing device 10. However, this is not limited to this, and one information processing device 10 may be connected to multiple nursing care facilities, each having a control device 30 and multiple detection units 20.

[0022] (Detection unit 20) FIG. 2 is a block diagram showing the configuration of the detection unit 20. Referring to FIGS. 1 and 2, the detection unit 20 is configured to monitor the movements (behavior) of the care recipient 71 in an observation area (room) of a room where the care recipient 71 is located. The care recipient 71 is a person receiving care and a resident of the care facility (hereinafter also referred to as a "resident"). Care or nursing care is provided to the care recipient 71 by staff. The room is, for example, one room in a care facility where multiple care recipients 71 reside. Furthermore, the information processing device 10 is connected to detection units 20 provided in each of multiple rooms as shown in FIG. 1. The device ID of each detection unit 20 and the subject ID of the care recipient 71 to be monitored are associated and stored in a memory unit (memory unit 32 or memory unit 12 described below).

[0023] As shown in FIG. 2 , the detection unit 20 includes a control unit 21, a communication unit 22, a camera 23, a sensor 24, a care call unit 25, a microphone 26, and the like. The detection unit 20 generates sensing data (image data, audio data, etc.) by constantly sensing the care recipient 71 in the observation area (room) in real time, 24 hours a day, using a plurality of various sensors. The control unit 21 may also include a large-capacity memory. The camera 23, the sensor 24, and the microphone 26 are disposed in a main body of the detection unit 20. The main body is disposed on the ceiling or upper part of a wall of the room. The care call unit 25 is disposed separately from the main body and is communicatively connected to the main body via wired or short-range wireless communication. The microphone 26 may be provided in multiple locations, and may be disposed not only on the main body near the ceiling of the detection unit 20 but also next to the bed 81, for example. For example, the microphone 26 may also be provided inside the care call unit 25.

[0024] The control unit 21 is composed of a CPU, RAM, ROM, etc., and controls each part of the detection unit 20 and performs calculation processing according to a program. The communication unit 22 is an interface circuit (for example, a LAN card, etc.) for communicating with other devices such as the control device 30 via the network 50.

[0025] The camera 23 is placed, for example, on the ceiling or upper part of a wall of the room, and captures an overhead image of the room with the care recipient 71 directly below as an observation area (photographing area). The camera 23 is an example of a sensor that detects the movement of the care recipient 71 within the observation area. The room contains objects such as a bed 81 and a wheelchair 82. The camera 23 captures the care recipient 71, who is a person, as well as surrounding objects, and outputs the captured image data (video). The camera 23 is a near-infrared camera, but a visible light camera may be used instead, or a combination of these may be used. The camera 23 may also be a wide-angle camera.

[0026] The sensor 24 detects the movement of the care recipient 71. The sensor 24 includes various sensors other than the camera 23 and the microphone 26. For example, the sensor 24 may include at least one of a body movement sensor, a bed sensor, a mat sensor, a thermal sensor, and an infrared sensor. The "body movement sensor" may be a Doppler shift sensor that transmits and receives microwaves to and from the bed 81 and detects the Doppler shift of the microwaves caused by the body movement (e.g., breathing) of the care recipient 71. This body movement sensor detects chest movement (up and down movement of the chest) associated with the breathing of the care recipient 71. The period and amplitude of the body movement can be used to determine the sleep state or abnormal micro-movements (e.g., due to cardiac arrest). The "bed sensor" may be a sensor attached to the bed. For example, the bed sensor may detect weight and be placed on the bed 81 or on the floor at the exit of the bed 81 as an observation area to detect whether a person is standing on the sensor or to detect the sleep state. The "mat sensor" has the same function as the bed sensor, detecting the presence of a person in each divided area of the floor. The "infrared sensor," also known as a human presence sensor, detects whether a person is present in the room. For example, the infrared sensor is placed throughout the room or on the bed 81 as its observation area.

[0027] The care call unit 25 includes a push-button switch, and detects a care call when the care recipient 71 operates this switch. The microphone 26 (hereinafter referred to as the mic 26) collects sounds in the room and outputs audio data. The mic 26 is an example of a sensor that detects the movement of the care recipient 71 within the observation area.

[0028] (Control device 30) FIG. 3 is a block diagram showing the configuration of the control device 30. As shown in FIG. 3, the control device 30 includes a control unit 31, a memory unit 32, a communication unit 33, a display unit 34, and an operation input unit 35. The control unit 31 is composed of a CPU, RAM, ROM, etc., and controls each unit of the control device 30 and performs calculations according to a program. The control unit 31 functions as a data acquisition unit 311 in cooperation with the communication unit 33 or the operation input unit 35. The control unit 31 also functions as an image analysis unit 312 and a sound analysis unit 313. The memory unit 32 is composed of a hard disk or the like that stores various programs and various data. The communication unit 33 is an interface circuit for communicating with other devices via the network 50. The display unit 34 is composed of an LCD display, a touch panel, etc., and displays various information and operation screens. The operation input unit 35 is an input device such as a keyboard or a mouse, and accepts inputs from staff, staff leaders of units (also referred to as areas or groups in charge), and facility managers (hereinafter collectively referred to as managers, etc.).

[0029] (Data acquisition unit 311) The data acquisition unit 311 acquires image data from the camera 23, audio data from the microphone 26, and the like from the detection unit 20. The data acquisition unit 311 also accepts input of resident information, staff information, and LIFE data from a manager, etc. The data acquisition unit 311 also accepts input of nursing care records, rehabilitation records, and handover records from staff.

[0030] (Image analysis unit 312) The image analysis unit 312 analyzes the image sent from the detection unit 20 to generate image analysis information, behavior information, caption information, and the like.

[0031] (Image analysis information) The image analysis unit 312 analyzes images to perform human / object detection processing, joint point detection processing, etc. For example, the image analysis unit 312 may use a trained model such as OpenPose or DeepPose to detect a person and the person's joint points.

[0032] (Behavioral information) The image analysis unit 312 classifies behavioral scenes using a machine learning model. This machine learning model is trained through supervised learning using time-series information on person information (joint coordinates, head coordinates), object information (object position coordinates such as wheelchairs and beds), and correct labels for behavioral scenes. This training is performed using a recurrent neural network (RNN) or a hidden Markov model (HMM). For example, when a care recipient 71 is present in an image, the image analysis unit 312 classifies the image into behavioral scenes (states) such as independent walking, falling, standing, lying or supine on a bed, sitting on a chair, and banging on a wall based on the person information (joint coordinates, head coordinates). When classifying gait in a behavioral scene, the image analysis unit 312 determines whether the person is walking normally, unsteadily, or walking while holding on to nearby handrails or other obstacles. Furthermore, when the care recipient 71 is alone in the room, the image analysis unit 312 determines the frequency of independent walking from the person information (joint point coordinates, head coordinates), and if this is high, determines that the care recipient 71 has transferred to a wheelchair by himself. Furthermore, when there is staff in the room in addition to the care recipient 71, if the frequency of the care recipient 71 walking independently is low, it determines from the person information of the staff when the care recipient 71 transfers that the staff is nearby but is frequently watching over the care recipient 71 without directly providing assistance. Furthermore, when the staff is helping the care recipient 71 transfer, it classifies the assistance as partial assistance or full assistance depending on the frequency of assistance.

[0033] (Generating caption information) The image analysis unit 312 generates caption information using a trained caption generation model. In this process, the caption generation model first identifies people and objects in the image and understands their characteristics. Then, based on this information, it generates a description to describe the content of the image.

[0034] (Audio analysis unit 313) The voice analysis unit 313 generates voice analysis information and emotion information by analyzing the voice data. The voice analysis information includes text information and speaker information. The voice analysis unit 313 converts the speech contained in the voice data into text using well-known technology. The voice analysis unit 313 performs preprocessing on the voice data, such as removing noise and stationary sounds and adjusting the sampling rate. The voice analysis unit 313 then analyzes the features of the voice to identify the speaker. To identify the speaker, the voice analysis unit 313 may determine that the care recipient 71 is the speaker if there is only one person in the room based on image data taken at the same time. Alternatively, if there are multiple people present, the voice analysis unit 313 may determine whether the speaker is the care recipient 71 based on the voice feature data of the care recipient 71. If the result of the determination is that the speaker is not the care recipient 71, the voice analysis unit 313 may determine that the speaker is a staff member. The voice analysis unit 313 also uses well-known voice recognition technology to extract speech from the voice data and convert it into text.

[0035] Furthermore, the voice analysis unit 313 estimates the emotion of the speaker using the voice data. Technology for estimating the emotion of the speaker is publicly known and is disclosed, for example, in Japanese Patent Application Laid-Open No. 2021-108021. The voice analysis unit 313 estimates the emotion by classifying the emotion of the speaker using an emotion model.

[0036] (Storage unit 32) The storage unit 32 stores resident information, staff information, nursing care records, rehabilitation records, handover records, LIFE data, image data, audio data, trained models, etc. Note that the resident information, nursing care records, rehabilitation records, handover records, and LIFE data related to the care recipient 71 are collectively referred to as resident data (or subject data).

[0037] The "resident information" is information about the care recipient 71, including the room number in which the care recipient 71 resides, family information, and the age, gender, medical history, and medical conditions of the care recipient 71. The resident information also includes physical condition information and information about the care recipient's lifestyle. Examples of the physical condition information include the height, weight, and level of care required of the care recipient 71. Physical conditions may also include the level of independent walking and the level of ability to transfer from the bed to a care chair. The physical condition information may also include the resident's dementia level. The dementia level may be input by a facility manager or the like. Alternatively, the control device 30 may determine the level of unsteadiness of the care recipient 71 while moving from images captured by the camera 23, and use this information to evaluate and determine the dementia level. Information about the resident's lifestyle includes the resident's activity level (travel time, travel distance), sleep time (life rhythm), number of care calls, etc. Medical history is past information about medical events and health, such as illnesses, injuries, surgeries, treatments, and allergies. This includes experiencing hallucinations and delusions.

[0038] The resident information also includes information on the replacement (replacement time) and replenishment of welfare equipment and nursing care products used by the care recipient 71. Welfare equipment includes walking aids, wheelchairs, handrails, nursing beds, etc. Nursing care products include diapers, clothing, slippers, and other footwear.

[0039] "Staff information" includes information such as staff name, affiliated unit, and work schedule.

[0040] "Nursing care records" include nursing record sheets, nursing record sheets (medical charts, life records), and diagnostic charts that record the care and treatment provided by staff, caregivers, nurses, and doctors (hereinafter referred to as "staff, etc.") who provide care to the care recipient 71. These records may be entered as text by scanning sheets handwritten by the staff, etc., and performing OCR processing. Alternatively, they may be entered by the staff, etc., through a staff terminal such as a smartphone that the staff, etc., carries while on duty. Care records include the amount of medication administered to the care recipient 71, medication care related to medication dosage, the amount of food eaten by the care recipient 71, dietary and hydration care related to fluid intake, the amount and quality of excretion, vital values (body temperature, blood pressure, pulse rate, etc.), and the time of occurrence of these events.

[0041] The "rehabilitation record" (rehabilitation record) records the results of rehabilitation conducted on the care recipient 71 to improve their activities of daily living (ADL) and motor skills. The rehabilitation record records the evaluation results and progress of the care recipient 71. For example, the evaluation results include the care recipient 71's walking ability and level of motivation for rehabilitation. Examples of rehabilitation records include those related to walking (using a walker), walking (using a cane), and pedal wheelchairs. A pedal wheelchair is a rehabilitation device that ensures leg exercise by pedaling while seated in the wheelchair, and allows the user to move independently to a destination. A pedal wheelchair may be equipped with a function to detect and output the amount of exercise and range of movement.

[0042] The "handover record" records special notes about the care recipient 71 entered by staff or administrators. Special notes include those recorded by the staff in charge of the care recipient 71 via the staff terminal. For example, special notes include the following: · The care recipients 71 often forget things. · The care recipient 71 usually uses a remote control to adjust the bed height, changing the bed height to make it easier to transfer to a care chair. - He keeps his daily medication on top of the dresser and has trouble distinguishing between them. · They tend to feel lonely when they are alone, so they actively seek out communication. - Requires some assistance with walking independently, but frequently attempts to do so. -Able to pick up objects from the floor by themselves.

[0043] LIFE is a scientific nursing care information system promoted by the Ministry of Health, Labor and Welfare. "LIFE data" refers to input data related to this LIFE system. LIFE data includes many input items, such as medical history, ADL, and nutrition.

[0044] The image data and audio data are respectively acquired by the camera 23 and microphone 26 of the detector 20 in each room. These data are held for a predetermined period of time.

[0045] The trained model is used by the image analysis unit 312 as described above to help classify action scenes and generate caption information.

[0046] (Information processing device 10) The configuration of the information processing device 10 will be described below with reference to Figures 4 and 5. Figure 4 is a block diagram showing the configuration of the information processing device. Figure 5 is a schematic diagram showing the flow of each piece of data.

[0047] As shown in FIG. 4, the information processing device 10 includes a control unit 11, a storage unit 12, and a communication unit 13. The control unit 11 is composed of a CPU, RAM, ROM, etc., and controls each unit of the information processing device 10 and performs calculation processing according to a program. The control unit 11 functions as a first acquisition unit 111 and a second acquisition unit 112 in cooperation with the communication unit 13. The control unit 11 also functions as a language analysis unit 113, an evaluation unit 114, and an output unit 115. The storage unit 12 is composed of a hard disk or the like that stores various programs and various data. The communication unit 13 is an interface circuit for communicating with other devices via a network 50.

[0048] (Control unit 11) (First acquisition unit 111, second acquisition unit 112) The first acquisition unit 111 acquires the first information and the second information from the control device 30 or the storage unit 12. The second acquisition unit 112 acquires the third information from the control device 30 or the storage unit 12. The first information to the third information are multimodal information that combines information obtained from different information formats such as video (images), audio, and text input. That is, the first information is visual information such as images and videos, the second information is auditory information such as audio, and the third information is character information from text input.

[0049] The first information acquired by the first acquisition unit 111 includes image information of image data or video data, image analysis information or behavioral information obtained from image analysis of the image information, and caption information obtained by analyzing the image information.

[0050] The second information acquired by the first acquisition unit 111 includes voice information of the voice data, voice analysis information obtained by analyzing the voice information, and emotion information. The voice analysis information also includes recognized text information and speaker information.

[0051] The third information acquired by the second acquisition unit 112 includes text information and text analysis information. The third information is resident data and also record information recorded regarding the care recipient 71. The resident data includes resident information, nursing care records, rehabilitation records, handover records, and LIFE data regarding the care recipient 71, as described above.

[0052] Here, the text analysis information is generated by the language analysis unit 113 described below. The other image analysis information, image information, behavior information, caption information, audio analysis information, and emotion information are obtained by the analysis processing of the image analysis unit 312 or audio analysis unit 313, as described above.

[0053] (Language Analysis Department 113) The language analysis unit 113 analyzes the text information constituting the third information and generates text analysis information. For example, the language analysis unit 113 performs natural language processing using keywords such as words included in the text information as elements, analyzes the processing results, and generates text analysis information in which the elements are structured using semantic vectors or the like. The text analysis information may be, for example, an information network in which the relationships between elements included in the resident data related to the care recipient 71 are networked.

[0054] Natural language processing is a technology that uses a computer to recognize and generate natural language used by humans, and existing technologies such as BERT (Bidirectional Encoder Representations from Transformers) can be used. For example, the language analysis unit 113 performs natural language processing on the resident data to extract elements and relationships between elements using keywords included in the resident data as elements, and generate text analysis information.

[0055] (Evaluation section 114) The evaluation unit 114 evaluates the care recipient 71 based on the first to third information. This evaluation includes information related to the needs of the care recipient 71. Details of the evaluation method of the evaluation unit 114 will be described later (see Figs. 6 to 8B described later).

[0056] (output unit 115) The output unit 115 outputs the evaluation result generated by the evaluation unit 114. For example, the output unit 115 transmits data of the evaluation result to the control device 30 or displays it on the display unit 34 of the control device.

[0057] (Storage unit 12) The storage unit 12 stores a first information DB, a second information DB, a third information DB, and a trained model. The first to third information DBs store the first to third information. The trained models are used by the language analysis unit 113 and the evaluation unit 114. The trained models used by the evaluation unit 114 include trained models that have been trained to output information related to the needs of the care recipient. These trained models have been developed through machine learning, for example, using combinations of evaluations and needs as training data.

[0058] (Evaluation method using information processing device 10) The evaluation method performed by the information processing device 10 will be described below with reference to FIGS. 6 to 8B. FIG. 6 is a flowchart showing the evaluation method performed by the information processing device 10. FIG. 7 is a subroutine flowchart showing the processing of step S03 in FIG. 6. FIGS. 8A and 8B are examples of evaluation results including requests based on the first to third information, when the evaluation method of this embodiment shown in FIG. 6 is applied. The following description will be based on the first and fourth cases shown in FIGS. 8A and 8B.

[0059] (Step S01) The first acquisition unit 111 of the information processing device 10 acquires first and second information related to a certain care recipient 71 during a predetermined period a. The length of the predetermined period a is, for example, any of several tens of minutes to several days. The predetermined period a is, for example, from 1:00 PM to 2:00 PM on a certain day.

[0060] Here, the first and second information is at least one of the following: (a1) Image information and audio information generated by the camera 23 or microphone 26 during the same predetermined period a. (a2) Image analysis information and audio analysis information obtained by analyzing the image information and audio information of (a1), respectively.

[0061] (Step S02) The second acquisition unit 112 of the information processing device 10 acquires recorded information (third information) related to the care recipient 71 before and after the predetermined period a defined in step S01. This third information includes information recorded before and after the predetermined period a, as well as information recorded before the predetermined period a. For example, "before and after" information includes a case where the condition of the care recipient or the care content within the predetermined period a was recorded after the predetermined period a, or a record of changing clothing or footwear recorded before the predetermined period a, as in Case 1 described below. Furthermore, "past" information includes, for example, resident data (medical history) such as that mentioned in Case 4 described below, which may include a case where past experiences of hallucinations or delusions are recorded.

[0062] (Step S03) The evaluation unit 114 of the information processing device 10 evaluates the care recipient 71 based on the first to third information. Fig. 7 is a subroutine flowchart showing the evaluation process in step S03 of Fig. 6.

[0063] (Step S31) The information processing device 10 acquires an initial evaluation x. When the above-described (a1) image information is acquired in step S01, the control unit 11 generates behavioral information by analyzing the image information (hereinafter referred to as initial evaluation x). When the (a2) image analysis information is acquired in step S01, the behavioral information included in the image analysis information is the initial evaluation x. For example, in the case of Case 1 shown in FIG. 8A, if the image information captures a state in which a resident (care recipient 71) is unsteady while walking, the behavioral information obtained by analyzing this image information is "resident's unsteady gait (walking by crawling)." In addition, in Case 4 shown in FIG. 8B, the behavioral information obtained by analyzing image information captures a state in which a resident is hitting the wall of the room is "violent or aggressive behavior."

[0064] (Step S32) The information processing device 10 acquires information y. When (a1) voice information is acquired in step S01, the control unit 11 analyzes the voice information to acquire voice analysis information (hereinafter referred to as information y). When (a2) voice analysis information is acquired in step S01, the information y is recognized text information and speaker information included in the voice analysis information. For example, in the same case 1 shown in FIG. 8A, the utterance of the resident (care recipient 71) "These slippers are big" is detected. In case 4 of FIG. 8B, the utterance of "Get them out of here" is detected.

[0065] (Step S33) The language analysis unit 113 of the information processing device 10 extracts information z from the third information. Specifically, the language analysis unit 113 extracts information related to the initial rating x and the information y. For example, in case 1, the language analysis unit 113 extracts text information related to "unsteadiness" in the initial rating x and "slippers" in the information y. For example, the language analysis unit 113 extracts multiple pieces of text information related to unsteadiness and slippers from the third information. Furthermore, in case 4, the language analysis unit 113 detects multiple pieces of text information related to violent and aggressive behavior.

[0066] (Step S34) The evaluation unit 114 generates an evaluation by modifying the initial evaluation x from information y, information z, or information obtained by adding the first information to the information y and z. The evaluation unit 114 generates an evaluation using a rule-based algorithm. Alternatively, the evaluation unit 114 uses a trained demo to input the information y, z, and the first information and obtain an evaluation output. In case 1, the evaluation unit 114 generates an evaluation of slippers being the wrong size. In case 4, the evaluation unit 114 generates an evaluation by modifying the initial evaluation x to "statements of hallucinations and delusions as one symptom of BSPD."

[0067] In addition, in case 1, the evaluation unit 114 generates a request "I want my slippers replaced" as the evaluation result of the care recipient 71.

[0068] In this case 1, based on the image data of the first information alone, the initial evaluation x (unsteadiness) may lead to the possibility of examining the problem of unsteadiness while walking and whether a walker is necessary. However, by referring to the voice data and resident data of other modal information, it is confirmed that the unsteady gait (walking with hands) is not due to a problem with walking ability, but rather due to the slippers not fitting properly. Therefore, the evaluation unit 114 modifies the initial evaluation of this behavior as "deterioration in walking ability" to "mismatched slippers size."

[0069] In addition, in Case 4, it is not possible to determine the reason for the banging on the wall, such as stress or bad mood, from the image data of the first information alone. However, by referring to the voice data and resident data of other modal information, it is confirmed that the resident is exhibiting behavior due to some kind of hallucination or delusion. Therefore, the evaluation unit 114 modifies the initial assessment of this behavior as "violent or aggressive behavior" to "having a hallucination or delusion as a symptom of BPSD."

[0070] This completes the subroutine flowchart of FIG. 7, and the process returns to the main flowchart of FIG.

[0071] (Step S04) The output unit 115 outputs the evaluation result including the request generated in step S03 to the control device 30 or the like.

[0072] As described above, the information processing device according to this embodiment includes a first acquisition unit that acquires first and second information about a care recipient obtained within a predetermined period of time, a second acquisition unit that acquires record information recorded about the care recipient before or after the predetermined period of time, and an evaluation unit that evaluates the care recipient based on the acquired first and second information and record information. This allows the behavior or situation of the care recipient to be appropriately evaluated.

[0073] The configuration of the information processing device 10 described above is a main configuration for explaining the features of the above embodiment, but is not limited to the above configuration and can be modified in various ways within the scope of the claims. Furthermore, the configuration of a general information processing device is not excluded.

[0074] In the example shown in FIG. 1 etc., the control device 30 and the information processing device 10 are described as separate entities, but they may be integrated. For example, the control device 30 functions as the information processing device 10. Alternatively, the main functions of the control device 30, that is, the image analysis unit 312 and the audio analysis unit 313, may be provided on the control unit 11 side of the information processing device 10. Furthermore, when configured as a fixed unit, the information processing device 10 may process the process of FIG. 6 in real time.

[0075] Furthermore, the means and methods for performing various processes in the information processing device or information processing system according to the above-described embodiments can be realized by either a dedicated hardware circuit or a programmed computer. The program may be provided, for example, by a computer-readable recording medium such as a USB memory or a DVD (Digital Versatile Disc)-ROM, or may be provided online via a network such as the Internet. In this case, the program recorded on the computer-readable recording medium is typically transferred and stored in a storage unit such as a hard disk. The program may also be provided as standalone application software, or may be incorporated as a function into the software of a device such as a detection unit. [Explanation of symbols]

[0076] 1000 Information Processing Systems 10. Information processing equipment 11 Control section 111 First acquisition part 112 Second acquisition part 113 Language Analysis Department 114 Evaluation Department 115 Output section 20 Detector 21 Control Unit 22 Communications Department 23 Camera 24 sensors 25 Care Call Department 26. Mike 30 Control device 31 Control Unit 311 Data Acquisition Department 312 Image Analysis Unit 313 Audio Analysis Unit 32 Storage section 33 Communications Department 34 Display section 35 Operation input section

Claims

1. a first acquisition unit that acquires first and second information of the care recipient obtained within a predetermined period; a second acquisition unit that acquires record information recorded regarding the care recipient before or after the predetermined period; an evaluation unit that evaluates the care recipient based on the acquired first information, second information, and record information; An information processing device comprising:

2. The information processing device according to claim 1 , wherein the evaluation includes information relating to the needs of the care recipient.

3. The information processing device according to claim 1 , wherein the second acquisition unit acquires the record information relating to the care recipient from a time period earlier than the predetermined time period.

4. 3. The information processing apparatus according to claim 1, wherein the first information, the second information, and the recorded information are multimodal information that combines information obtained from mutually different information formats.

5. The first information includes at least one of image information and image analysis information obtained by analyzing the image information, the second information includes at least one of audio information and audio analysis information obtained by analyzing the audio information, The information processing device according to claim 1 , wherein the recorded information includes at least one of text information and text analysis information obtained by analyzing the text information.

6. The information processing device according to claim 2 , wherein the evaluation unit includes a machine-learned model that is trained to output information related to the needs of the care recipient.

7. The first information and the second information are simultaneously sensing data obtained by sensing the care recipient, or information obtained by analyzing the sensing data, The information processing device according to claim 1 , wherein the recorded information is information recorded by a staff member who provides care to the care recipient.

8. The first information is behavioral information of the care recipient obtained by analyzing an image acquired from a camera that photographs the care recipient, The second information is voice analysis information obtained by analyzing voice acquired from microphones arranged around the care recipient, The information processing device according to claim 7 , wherein the evaluation unit modifies the behavioral information based on the voice analysis information and the recording information.

9. (a) acquiring first and second information of a care recipient obtained within a predetermined period of time; (b) obtaining recorded information recorded regarding the subject of care before or after the predetermined period of time; (c) evaluating the care recipient based on the first information, the second information, and the recorded information obtained in steps (a) and (b); Evaluation methods including.

10. (a) acquiring first and second information of a care recipient obtained within a predetermined period of time; (b) obtaining recorded information recorded regarding the subject of care before or after the predetermined period of time; (c) evaluating the care recipient based on the first information, the second information, and the recorded information obtained in steps (a) and (b); A control program for causing a computer to execute a process including the above.

Images