Care-related information providing apparatus and care-related information providing method

Abstract

To provide a device or a method for enabling a family or the like of a person to be cared to easily grasp the situation of the person to be cared.SOLUTION: A nursing care related information providing device provides information of a person to be cared for who moves into a facility to the outside of the facility. The nursing-care related information providing device includes a processor, a sound output device, an input device, and a communication interface. The processor performs arithmetic processing. The audio output device outputs audio. The input device inputs, to the processor, first information including information for identifying the care receiver or information obtained by recording or measuring the care receiver. The processor causes the sound output device to output sound. The processor transmits the first information and second information related to the sound output by the sound output device to the outside of the facility via the communication interface.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present application relates to an apparatus or method for providing information about a care recipient outside a facility. [Background technology]

[0002] Patent Document 1 discloses a method for recording care provided by staff to a care recipient. In the method of Patent Document 1, a camera is installed in a facility to capture images including the staff and the care recipient. Next, a control device analyzes the images including the staff and the care recipient to detect and record that the staff has provided care to the care recipient. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 7354634 Summary of the Invention [Problem to be solved by the invention]

[0004] It may be difficult for a care recipient who has entered a facility to directly contact their family or other relatives due to time or other constraints. As a result, the family or other relatives of the care recipient may not be able to grasp the condition of the care recipient. The images taken in Patent Document 1 are taken for staff performance and management purposes, and are not provided to the family or other relatives of the care recipient. Therefore, the method in Patent Document 1 cannot address the issue of the family or other relatives of the care recipient being unable to grasp the condition of the care recipient.

[0005] The present application has been made in view of the above circumstances, and its main object is to provide an apparatus or method that allows a family member or the like of a care recipient to easily grasp the condition of the care recipient. [Means for solving the problem]

[0006] The problem to be solved by the present application is as described above. Next, the means for solving this problem and the effects thereof will be explained.

[0007] According to a first aspect of the present application, there is provided a care-related information providing device having the following configuration. That is, the care-related information providing device provides information about a care recipient residing in a facility to a location outside the facility. The care-related information providing device includes a processor, an audio output device, an input device, and a communication interface. The processor performs arithmetic processing. The audio output device outputs audio. The input device inputs first information to the processor, the first information including information for identifying the care recipient or information obtained by recording or measuring the care recipient. The processor causes the audio output device to output audio. The processor transmits the first information and second information related to the audio output by the audio output device to a location outside the facility via the communication interface.

[0008] According to a second aspect of the present application, there is provided the following method for providing care-related information. That is, in this method, information about a care recipient residing in a facility is provided to a location outside the facility. In this method, first information is obtained, which is information for identifying the care recipient or information obtained by recording or measuring the care recipient. In this method, an audio output device that outputs audio is controlled to output the audio. In this method, the first information and second information related to the audio output by the audio output device are transmitted to a location outside the facility via a communication interface. [Effects of the Invention]

[0009] According to the present application, the family members of the care recipient can easily understand the condition of the care recipient. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is an explanatory diagram showing a situation in which a care-related information providing device is used. [Figure 2] FIG. 2 is a block diagram of a care-related information providing device. [Figure 3] FIG. 3 is a sequence diagram showing the overall processing of the care-related information providing device. [Figure 4] 10 is a flowchart showing a process in which the collection robot collects first information and second information. [Figure 5] FIG. 4 is an explanatory diagram showing how the utterances and movements of the collection robot are controlled based on the first information. [Figure 6] 10 is a flowchart showing a process for determining a setting state to be executed by the collection robot. [Figure 7] Remote conversation sequence diagram. DETAILED DESCRIPTION OF THE INVENTION

[0011] Next, an embodiment of the present application will be described with reference to the drawings.

[0012] First, the terms used in this specification are defined. "Nursing care" means assisting a care recipient with any activity throughout their life. Nursing care can also be replaced with "care." A "care recipient" is a person who receives care, or in other words, a "care recipient." A "facility" is a facility where nursing care is provided as part of the work. Facilities are not limited to facilities dedicated to nursing care, and include, for example, hospitals or facilities for the disabled. "Staff" refers to people who work at a facility. Staff includes caregivers, nurses, physical therapists, or doctors who provide nursing care as part of their work. Staff includes people who clean, prepare meals, or groom at nursing care facilities.

[0013] Next, the care-related information providing device 1 will be described with reference to Fig. 1. The care-related information providing device 1 is a device that provides information related to care, particularly information related to the condition of a care recipient, to the family of the care recipient or the like.

[0014] The care-related information providing device 1 includes a collection robot 10 and an information processing device 20. The collection robot 10 and the information processing device 20 are connected via a communication network 50. The communication network 50 may include a local area network or a wide area network such as the Internet. In other words, the information processing device 20 may be located inside or outside the facility.

[0015] The collection robot 10 is placed in a facility. The collection robot 10 converses with a care recipient in the facility and collects audio information related to the conversation and image information of the care recipient. The audio information and image information collected by the collection robot 10 are stored in an information processing device 20.

[0016] A language processing server 40 is further connected to the collecting robot 10 and the information processing device 20. The language processing server 40 is connected to the information processing device 20 via, for example, the Internet. The language processing server 40 creates utterances of the collecting robot 10 when the collecting robot 10 converses with the care recipient.

[0017] The information processing device 20 transmits audio information and image information of the care recipient to an external terminal 30 owned by the care recipient's family. This allows the care recipient's family to easily understand the care recipient's condition. The external terminal 30 is not limited to a terminal owned by the care recipient's family, but may be a terminal owned by a friend or relative.

[0018] Next, the configurations of the collection robot 10 and the information processing device 20 will be described with reference to FIG.

[0019] The collection robot 10 includes a head 10a, a body 10b, and an arm 10c. The collection robot 10 has a built-in actuator such as a motor. The head 10a and the arm 10c are moved by using the power of the actuator. The collection robot 10 can express emotions such as nodding, shaking, and tilting its head by moving the head 10a. The collection robot 10 can express emotions using gestures by moving the arm 10c. The collection robot 10 may be able to assist a care recipient by moving the arm 10c.

[0020] The collection robot 10 further includes a moving device 10d. The moving device 10d includes a drive source such as a motor and wheels. The wheels are driven and rotated using the drive source, causing the collection robot 10 to move. This allows various devices possessed by the collection robot 10 to move as a unit.

[0021] The collection robot 10 includes the head 10a, the body 10b, the arm 10c, and the moving device 10d, but this is merely an example, and at least one of these components may be omitted. For example, the collection robot 10 may not include the moving device 10d, in other words, may be immobile. Furthermore, the collection robot 10 is not limited to a robot type as long as it can collect information about the care recipient. For example, the collection robot 10 may collect information about the care recipient using a tablet PC.

[0022] The collection robot 10 includes an audio output device 11 , a camera 12 , a microphone 13 , a vital sensor 14 , a display 15 , a computer 16 , and a communication interface 17 .

[0023] The audio output device 11 outputs audio during a conversation with the care recipient. The audio output device 11 is a device including a speaker. However, the audio output device 11 may transmit audio information to another speaker and output audio through the other speaker.

[0024] The camera 12, microphone 13, and vital sensor 14 correspond to "input devices." The input devices are devices that record or measure the care recipient. Information obtained by the input devices recording or measuring the care recipient is referred to as "first information." The first information is provided to the care recipient's family, etc.

[0025] The camera 12 takes an image of the care recipient and generates an image. In this specification, "image" includes still images and moving images. The camera 12 is placed on the head 10a. When the care recipient looks at the head 10a of the collection robot 10, an image can be generated in which the care recipient's line of sight is aligned. Image information showing the image generated by the camera 12 is output to the computer 16.

[0026] The microphone 13 acquires the voice of the care recipient. The microphone 13 is placed on the head 10a. When the care recipient speaks toward the head 10a of the collection robot 10, the voice of the care recipient can be easily acquired with high accuracy. Voice information indicating the voice acquired by the microphone 13 is output to the computer 16.

[0027] The vital sensor 14 acquires vital information of the care recipient. The vital information includes, for example, body temperature, blood pressure, pulse rate, or respiratory status. The vital sensor 14 may be a non-contact sensor. The vital sensor 14 may be a contact sensor disposed on the arm 10c or the like. The vital sensor 14 may be a device worn by the care recipient. The vital information acquired by the vital sensor 14 is output to the computer 16.

[0028] The display 15 is a liquid crystal display, an organic EL display, or the like, and displays information. The display 15 is placed on the head 10a. This makes it easy to guide the gaze of the care recipient to the head 10a. The display 15 displays, for example, information corresponding to the face of the collection robot 10, information requested by the care recipient, or information regarding notices created by staff. Note that the placement locations of the devices provided on the collection robot 10 are merely examples and may be changed.

[0029] The computer 16 includes a processor 16a, a main memory 16b, and a storage 16c. The processor 16a is, for example, a CPU, and performs arithmetic processing. The main memory 16b is, for example, a RAM, and temporarily stores data or programs involved in the arithmetic processing of the processor 16a. The storage 16c is, for example, an SSD, HDD, or flash memory. Information about the care recipients collected by the collection robot 10 is stored in the storage 16c.

[0030] The communication interface 17 is a wired or wireless communication module. By using the communication interface 17, the collection robot 10 communicates with the information processing device 20.

[0031] The information processing device 20 includes a processor 21, a main memory 22, a storage 23, and a communication interface 24. The processor 21 has the same configuration as the processor 16a, the main memory 22 has the same configuration as the main memory 16b, the storage 23 has the same configuration as the storage 16c, and the communication interface 24 has the same configuration as the communication interface 17. By using the communication interface 24, the information processing device 20 communicates with an external terminal 30 and a language processing server 40.

[0032] The processor 16a or the processor 21 corresponds to the "processor." Some of the processes described below as being executed by the processor 16a may be executed by the processor 21. Conversely, some of the processes described below as being executed by the processor 21 may be executed by the processor 16a. Furthermore, the functions of the information processing device 20 may be installed in the collection robot 10, and the information processing device 20 may be omitted.

[0033] The external terminal 30 is a smartphone, a tablet PC, a notebook PC, or a desktop PC. The external terminal 30 may be a cloud storage service contracted by a family member of the care receiver or the like.

[0034] Next, with reference to FIG. 3, an overview of a method for providing care-related information performed using the care-related information providing device 1 will be described.

[0035] As shown in FIG. 3, the collecting robot 10 records first information and second information of the care recipient (S1). As described above, the first information is information obtained by measuring the care recipient using the camera 12, microphone 13, and vital sensor 14. Note that image information or vital information may be omitted from the first information. The second information is audio information indicating the audio output by the collecting robot 10 to the care recipient using the audio output device 11. In other words, the second information is information indicating the content of the remarks made by the collecting robot 10 to the care recipient. The collecting robot 10 stores the first information and the second information in the storage 16c. The first information or the second information may include information obtained by converting audio information into text information.

[0036] Next, the collecting robot 10 transmits the recorded first information and second information to the information processing device 20 (S2). In detail, the processor 16a transmits the first information and second information stored in the storage 16c to the information processing device 20 using the communication interface 17. The collecting robot 10 may transmit the first information and second information to the information processing device 20 in real time. The collecting robot 10 may transmit the first information and second information to the information processing device 20 at a predetermined time.

[0037] Next, the information processing device 20 stores the received first information and second information in association with the identification information of the care recipient (S3). The first information includes personal data such as the face or voice of the care recipient. The personal data is information for identifying the care recipient. The care-related information providing device 1 has a database in which the personal data of the care recipient is associated with the identification information of the care recipient. The collection robot 10 or the information processing device 20 can identify the identification information of the care recipient based on the database. The processor 21 stores the first information, the second information, and the identification information of the care recipient in the storage 23.

[0038] When the family member or the like of the care recipient wishes to check the condition of the care recipient, the family member or the like performs a predetermined operation on the external terminal 30. In response to the operation by the family member or the like of the care recipient, the external terminal 30 transmits a command to transmit the first information and the second information of the care recipient to the information processing device 20 (S4).

[0039] The information processing device 20 extracts the first information and the second information of the care recipient corresponding to the received transmission command (S5). For example, the transmission command includes the identification information of the care recipient, and the processor 21 extracts the first information and the second information by using the identification information of the care recipient. The information processing device 20 transmits the extracted first information and the second information to the external terminal 30 (S6). The destination of the first information and the second information is the external terminal 30. For example, the processor 21 transmits the first information and the second information to the external terminal 30 using the communication interface 24, with the IP address or the email address as the destination.

[0040] The external terminal 30 displays the received first information and second information on the display (S7). The family of the care recipient can understand the condition of the care recipient by looking at the contents displayed on the display.

[0041] By executing the process shown in the sequence diagram of FIG. 3 , information about the care recipient can be easily provided outside the facility. Furthermore, it is technically possible to provide information about the care recipient outside the facility by installing a camera in the care recipient's room. However, this is difficult to achieve from the perspective of the care recipient's privacy. In this embodiment, the care recipient is photographed while conversing with the collection robot 10, so the care recipient's privacy is not an issue in principle. Therefore, the care-related information providing device 1 is easy to implement from the perspective of privacy. Furthermore, even if a staff member takes a photograph of the care recipient with a camera, it is possible to provide the photograph outside the care recipient's facility. However, this is difficult to implement because it places a heavy burden on the staff's time. In this embodiment, the collection robot 10 is used, so the staff's time is not restricted. Therefore, the care-related information providing device 1 is easy to implement from the perspective of the time burden on the staff.

[0042] The information processing device 20 may voluntarily transmit the first information and the second information to the external terminal 30 regardless of whether a transmission command is received. The information processing device 20 may store the first information and the second information in cloud storage, and a family member of the care recipient or the like may access the cloud storage.

[0043] Next, the collection of the first information and the second information by the collection robot 10 will be described in detail with reference to FIGS.

[0044] 4 shows the processing executed by the care-related information providing device 1, in other words, the processing executed by the processor 16a or the processor 21. In the following description, the processing executed by the processor 16a may be described as being executed by the collection robot 10, and the processing executed by the processor 21 may be described as being executed by the information processing device 20.

[0045] The collection robot 10 identifies the care recipient based on image information and audio information of the care recipient (S101). The identification method is as described in step S3. Next, the collection robot 10 moves closer to the identified care recipient (S102). The collection robot 10 controls the moving device 10d so as to get closer to the identified care recipient, thereby approaching the care recipient.

[0046] The collection robot 10 may control the moving device 10d so that it faces the care recipient directly in front of it. The collection robot 10 may approach the care recipient that it identifies first. The collection robot 10 may select a care recipient to approach so that the contact time with multiple care recipients is equal. The collection robot 10 may approach a predetermined care recipient. The predetermined care recipient is, for example, a care recipient who has signed up for this service. Furthermore, when multiple collection robots 10 are placed in a facility, a care recipient responsible for each collection robot 10 may be specified.

[0047] Next, the collecting robot 10 starts recording the first information (S103). The collecting robot 10 may talk to the care recipient by itself, or may wait for the care recipient to speak. The first word from the collecting robot 10 may be stored in advance in the storage 16c, for example.

[0048] The collecting robot 10 uses the language processing server 40 to carry out a conversation with the care recipient. Specifically, the collecting robot 10 transmits first information to the language processing server 40 (S104). As shown in FIG. 5, the first information includes image information, audio information, and vital sign information. The collecting robot 10 transmits the first information to the language processing server 40 via the information processing device 20. However, the first information may not be transmitted via the information processing device 20. Furthermore, the first information may be processed before being transmitted to the language processing server 40. The processing may involve, for example, converting audio information into text information.

[0049] The language processing server 40 generates third information based on the first information. The third information is information including the content of speech uttered by the collection robot 10. The third information may include speech features. Speech features are parameterized values ​​of pitch, voice quality, emotion, intonation, etc.

[0050] The language processing server 40 has a large-scale language model. Hereinafter, the large-scale language model will be referred to as the LLM. Because the first information includes audio information, by inputting a predetermined prompt, the LLM can generate a utterance that indicates a response to the utterance of the care recipient. By inputting image information into the LLM, the LLM can generate a utterance that takes into account the facial expression of the care recipient. For example, if the care recipient looks troubled in response to the utterance of the collection robot 10, the LLM may generate a utterance that rephrases the utterance in an easier-to-understand way. By inputting vital sign information into the LLM, the LLM can generate a utterance that takes into account the physical condition of the care recipient. For example, if the care recipient is in poor health, the LLM may generate a utterance that shows concern for the care recipient.

[0051] The generation of utterance content using the language processing server 40 can be realized by using, for example, an API service. That is, the information processing device 20 transmits the first information and a prompt requesting the generation of a response to the voice information or text information included in the first information to the language processing server 40 using the API. As a result, the information processing device 20 receives the third information from the language processing server 40 (S105). The collection robot 10 causes the voice output device 11 to output a voice based on the third information (S106).

[0052] Next, the collection robot 10 determines whether or not a termination condition for the conversation with the care recipient is satisfied (S107). The termination condition is, for example, a time condition. The collection robot 10 determines that the termination condition is satisfied when a predetermined time limit has elapsed. The time limit may be, for example, a value between 3 minutes and 10 minutes. The time limit may be a common time for each care recipient, or may be a different time for each care recipient.

[0053] During a conversation between the collection robot 10 and the care recipient, the collection robot 10 receives an actuator control command as shown in FIG. 5. The actuator control command is a command to move the head 10a or the arm 10c. The actuator control command is a command to operate the head 10a or the arm 10c according to the content of the utterance of the collection robot 10. The actuator control command may be a command to operate the head 10a or the arm 10c of the collection robot 10 that has received the utterance of the care recipient.

[0054] During a conversation between the collecting robot 10 and the care recipient, the display 15 receives avatar information, as shown in FIG. 5. The avatar information is information to be displayed on the display 15 of the head 10a. The avatar information is, for example, the facial expression of the collecting robot 10. The avatar information may be changed depending on the content of speech made by the collecting robot 10 or the care recipient.

[0055] The avatar information may include not only facial expressions but also the appearance of the character. A different character may be displayed as an avatar for each care recipient. Specifically, the information processing device 20 associates the identification information of the care recipient with the identification information of the character and stores the associated information in the storage 23 or the like. The information processing device 20 identifies the associated character based on the identification information of the care recipient and transmits avatar information corresponding to the identified character to the display 15.

[0056] The collecting robot 10 repeatedly executes steps S104 to S106 and converses with the care recipient until it determines that the termination condition is met. When it determines that the termination condition is met, the collecting robot 10 ends recording of the first information and the second information (S108).

[0057] Next, we will explain customizing LLM.

[0058] The LLM is customized by the service provider of the care-related information providing device 1. Customization methods include prompting, external referencing, or additional learning. Prompting involves including a predetermined prompt when transmitting the first information. External referencing involves the LLM generating output by referencing other data. External referencing can be achieved, for example, using a technology called RAG. Additional learning involves providing additional data to the LLM and having it learn, thereby adjusting the internal parameters of the LLM and specializing it for a specific task or tendency.

[0059] The customization includes common customization that is common to all care recipients and individual customization that differs for each care recipient.

[0060] A conversation between the care recipient and the collection robot 10 can be considered a type of recreation. Therefore, it is preferable that the common customization of the LLM realizes a conversation that the care recipient can enjoy. Therefore, the common customization may be, for example, a change to emphasize consideration over correctness of the conversation. The common customization may be a change to emphasize continuity over the validity of the conversation. The common customization may be a change to actively provide topics. This type of customization can be realized by setting a personality using prompts, or by additional learning of suitable utterance examples.

[0061] Individual customization of the LLM may involve, for example, including the name of the care recipient being spoken to in the utterance. Customization to include the name in the utterance can be achieved, for example, by adjusting the prompt. Individual customization may involve generating utterances based on past conversations with the care recipient. Customization that takes past conversations into account can be achieved by external reference. For example, the information processing device 20 vectorizes past conversation histories and stores them in a database. The information processing device 20 searches for and extracts similar past conversation histories according to the care recipient and incorporates them into the input to the LLM.

[0062] The LLM may be personalized by using characteristics of related parties of the care recipient. The related parties may be, for example, family members or close acquaintances of the care recipient. Preferably, the related parties are related parties outside the facility. The related parties' characteristics may include catchphrases, vocal characteristics, or speaking tendencies. Personal customization using the related parties' characteristics can be achieved by external reference or additional learning.

[0063] Next, with reference to FIG. 6, a process for selecting and executing a setting state according to each care recipient will be described.

[0064] First, the collecting robot 10 identifies the care recipient based on image information and audio information of the care recipient (S201). The processing of step S201 is the same as the processing of step S101. Next, the collecting robot 10 determines whether or not there is a setting state associated with the care recipient (S202). The setting state is set based on, for example, the degree of dementia of the care recipient. The staff operates a terminal connected to the care-related information providing device 1 to register a setting state according to the degree of dementia for each care recipient.

[0065] In the following, it is assumed that a first setting state or a second setting state is registered in association with a care recipient. The first setting state is a setting state associated with a care recipient who can understand and respond to the utterances of the collecting robot 10. The second setting state is a setting state associated with a care recipient who has difficulty responding to the collecting robot 10, or who is able to respond but has difficulty giving a meaningful response. The collecting robot 10 in the first setting state responds to the utterances of the care recipient based on the utterances of the care recipient. The collecting robot 10 in the second setting state continues to actively speak to the care recipient even if the care recipient does not speak. Here, in this specification, "conversation" also includes a situation in which one person is talking unilaterally to another. It is to be noted that three or more setting states may be registered in advance.

[0066] If there is a setting state associated with the care recipient, the collecting robot 10 realizes the associated setting state (S203). In the first setting state, the collecting robot 10 transmits the first information of the care recipient to the language processing server 40 and makes a statement based on the received third information. In the second setting state, the collecting robot 10 continues to make statements independently. The content of the statements made by the collecting robot 10 in the second setting state may be predetermined. The predetermined content of statements is, for example, menu information, reading information, and song information. Furthermore, in the second setting state, the head 10a continues to face the care recipient even if there is no reaction from the care recipient.

[0067] If there is no setting state associated with the care recipient, the collection robot 10 executes the third setting state or the fourth setting state. The third setting state is a state similar to the first setting state. The fourth setting state is a state similar to the second setting state. In the fourth setting state, the tolerance and acceptance of conversation is higher than in the third setting state.

[0068] High tolerance and acceptance of conversation means that there is a loose tolerance for the correctness of conversation. Specifically, high tolerance and acceptance of conversation means that the collecting robot 10 does not take into account past conversation history. Furthermore, the collecting robot 10 in the fourth setting state may take into account the history of one previous conversation. The collecting robot 10 in the fourth setting state tends to output positive answers. The collecting robot 10 in the fourth setting state does not use terms that are pre-registered in the information processing device 20 or the like.

[0069] If there is no setting state associated with the care recipient, the collecting robot 10 executes the third setting state (S204). Next, the collecting robot 10 determines whether the voice information of the care recipient satisfies a predetermined characteristic (S205). The predetermined characteristic is a characteristic for determining whether the care recipient corresponds to the target of the fourth setting state. For example, the collecting robot 10 determines that the care recipient satisfies the predetermined characteristic when the care recipient repeats the same story a predetermined number of times, when the collecting robot 10 is unable to acquire the voice of the care recipient for a predetermined time or more, when the care recipient speaks incomprehensibly, when the care recipient does not understand what the collecting robot 10 is saying, or when the LLM outputs a response that it cannot understand what the care recipient is saying a predetermined number of times. Then, the collecting robot 10 executes the fourth setting state (S206).

[0070] As a result, the collection robot 10 can take action according to the degree of dementia registered by the staff or the degree of dementia determined by the collection robot 10.

[0071] In this embodiment, the setting state of the collection robot 10 is switched based on the degree of dementia of the care recipient. Alternatively, the state of the collection robot 10 may be switched based on the personality or wishes of the care recipient.

[0072] Next, a remote conversation using the care-related information providing device 1 will be described with reference to FIG.

[0073] First, the care recipient or their family member performs a pre-setting for remote conversation. This enables communication between the collection robot 10 and the external terminal 30. Next, the family member of the care recipient speaks to the external terminal 30. The external terminal 30 creates voice information based on the family member's speech and image information based on the camera image, and transmits the collected voice information and image information of the family member to the robot 10 (S11).

[0074] The collecting robot 10 reproduces the received audio information and image information, and moves the head 10a and the arm 10c based on the audio information (S12). The reproduced audio information is output from the audio output device 11. The reproduced image information is output from the display 15. The movements of the head 10a and the arm 10c are executed using the actuator control commands described above. The movement of the head 10a and the arm 10c makes it easier to convey emotions, allowing the care recipient to enjoy conversations with their family more.

[0075] The care recipient speaks to the collecting robot 10. The collecting robot 10 creates audio information based on the care recipient's speech and image information based on the image captured by the camera 12, and transmits the audio information and image information of the care recipient to the external terminal 30 (S13). The external terminal 30 plays back the received audio information and image information (S14). By repeating the processes of steps S11 to S14, the care recipient and their family members can converse with each other even when they are apart. The collecting robot 10 can be used not only to collect the first information and the second information, but also to converse with their family members.

[0076] Furthermore, instead of using voice information, the family may be able to instruct the head 10a and arm 10c of the collecting robot 10 to operate based on the operation of the external terminal 30. For example, the family can change the direction of the camera 12 by moving the head 10a, thereby checking images of the situation around the care recipient. Furthermore, the transmission of image information of the care recipient and family may be omitted. In this case, a pre-registered photo of the family may be displayed on the display 15 of the collecting robot 10, and a photo of the care recipient may be displayed on the display of the external terminal 30.

[0077] Remote conversation can be used not only for conversations between a care recipient and their family, but also for conversations between a care recipient and staff, allowing staff to talk to the care recipient from outside the facility.

[0078] The preferred embodiment of the present application has been described above, but the above configuration can be modified, for example, as follows. Each modification may be made alone, or multiple modifications may be made in any combination.

[0079] Processes and configurations that are not essential for providing the first information and the second information to the family of the care recipient may be omitted. For example, the process of moving the head 10a and the arm 10c in response to an actuator control command may be omitted.

[0080] The LLM provided by the language processing server 40 may be realized in a local environment within a facility.

[0081] The flowcharts shown in the above embodiments are merely examples, and some processes may be omitted, the contents of some processes may be changed, or new processes may be added.

[0082] The functionality of the elements disclosed herein may be implemented using one or more circuits or processing circuits, including general-purpose processors, special-purpose processors, integrated circuits, ASICs (Application Specific Integrated Circuits), FPGAs (Field Programmable Gate Arrays), and / or conventional circuitry. The functionality of the elements disclosed herein may be implemented using one or more circuits or processing circuits, including combinations of general-purpose processors, special-purpose processors, integrated circuits, ASICs, FPGAs, and conventional circuitry. The one or more circuits or processing circuits may be programmed using one or more programs stored together or separately in one or more memories or otherwise configured to perform the disclosed functions. A processor is considered a processing circuit or circuitry because it includes transistors and other circuits. A processor may also be a programmed processor that executes a program stored in a memory. In this disclosure, a circuit, unit, or means is hardware that performs the recited functions alone or in combination with each other, or hardware that is programmed to perform the recited functions alone or in combination with each other. The hardware may be any hardware disclosed herein that is programmed or configured to perform the recited functions. A computer program containing computer instructions is stored in memory. The computer instructions provide the logic and routines that enable the hardware to perform the methods disclosed herein. The hardware may include, for example, processing circuits or circuitry. The computer program may be implemented in a known format in a computer-readable storage medium, a computer program product, a memory device, a recording medium such as a CD-ROM or DVD, and / or the memory of FPGAs or ASICs.

[0083] The above-described embodiment is a specific example of the following aspects. [Aspect 1] A care-related information providing device that provides information about a care recipient residing in a facility to an outside of the facility, a processor for performing arithmetic processing; an audio output device that outputs audio; an input device that inputs first information including information for identifying the care recipient or information obtained by recording or measuring the care recipient into the processor; a communication interface; Equipped with the processor causes the audio output device to output audio; The processing unit transmits the first information and second information related to the voice output by the voice output device to an outside of the facility via the communication interface. Effect: The first and second information can be provided outside the facility, allowing the family members of the care recipient to understand the condition of the care recipient. [Aspect 2] The care-related information providing device according to aspect 1, the input device includes a microphone; The first information includes information on the voice of the care recipient responding to the voice output from the voice output device. Effect: The device allows conversations between the care recipient and people outside the facility, allowing the care recipient's family and others to understand the care recipient's condition in more detail. [Aspect 3] The care-related information providing device according to aspect 1 or 2, the input device includes a camera; The first information includes an image of the care recipient listening to the audio output from the audio output device. Effect: In addition to audio, images or video sequences can be provided outside the facility, allowing family members and others to visually grasp the condition of the person receiving care. [Aspect 4] The care-related information providing device according to any one of aspects 1 to 3, the input device includes a vital sensor; The first information includes vital sign information of the care recipient, the audio of which is output from the audio output device. Effect: In addition to audio, vital signs can be provided to those outside the facility, allowing the family members of the care recipient to have more information about the care recipient's physical condition. [Aspect 5] The care-related information providing device according to any one of aspects 1 to 4, the input device includes a microphone; The care-related information providing device, wherein the processor causes the audio output device to output the audio when the first information indicating the audio of the care recipient is input from the input device. Effect: The care-related information providing device can start or continue a conversation with the care recipient. [Aspect 6] The care-related information providing device according to any one of aspects 1 to 5, The processing unit identifies the care recipient based on the first information input from the input device, and actively causes the audio output device to output the audio to the identified care recipient even if the care recipient does not approach the processing unit. Effect: The care-related information providing device can proactively start a conversation with the care recipient, or speak to the care recipient who cannot speak. [Aspect 7] The care-related information providing device according to any one of aspects 1 to 6, a moving device for moving the facility; the audio output device moves integrally with the mobile device; The processor identifies the care recipient based on the first information input from the input device, and controls the mobile device so that the audio output device approaches the identified care recipient. Effect: When the moving device approaches the care recipient, the audio output device can actively output audio to the care recipient. [Aspect 8] The care-related information providing device according to any one of aspects 1 to 7, Equipped with a display, The processor is a care-related information providing device that identifies the care recipient based on the first information input from the input device and displays an avatar associated with the identified care recipient on the display. Effect: The avatar that corresponds to the care recipient can be displayed on the screen, making the care recipient feel more familiar with the care-related information providing device. In addition, the avatar can be automatically changed depending on the care recipient, making it easy to accommodate multiple care recipients. [Aspect 9] The care-related information providing device according to any one of aspects 1 to 8, Equipped with a display, the input device includes a camera; The care-related information providing device, wherein the camera is disposed adjacent to the display. Effect: By having the care recipient look at the display and talk, the care-related information providing device can create an image in which the care recipient's gaze is in contact. [Aspect 10] The care-related information providing device according to aspect 9, a robot having a head, a body, and an actuator for moving the head; the input device includes a microphone; the display is disposed on the head; The processing unit controls the actuator to move the head in response to the first information including audio input from the microphone or audio output from the audio output device. Effect: The robot moves its head while talking to the care recipient, making it easier for the care recipient to become familiar with the robot. [Aspect 11] The care-related information providing device according to any one of aspects 1 to 10, The processing device inputs the first information into a model created by machine learning, and outputs audio from the audio output device based on third information including the content of the utterance output by the model. Effect: The care-related information providing device can make statements that correspond to the care recipient or the condition of the care recipient. In particular, by using a model created by machine learning, the condition of the care recipient can be estimated with high accuracy. [Aspect 12] 12. The care-related information providing device according to aspect 11, A care-related information providing device in which the model has learned or is able to refer to the characteristics of those related to the person being cared for, and the third information acquired by the processor includes statements created in accordance with the characteristics of those related. Effect: The person receiving care can have a simulated conversation with someone involved. [Aspect 13] The care-related information providing device according to any one of aspects 1 to 12, the input device includes a microphone; The processor selects and executes a first setting state or a second setting state; In the first setting state, the processor causes the audio output device to output audio in response to first information including audio input from the input device; In the second setting state, the processor actively causes the audio output device to continue to output audio even if no audio is input from the input device. Effect: It is possible to switch between conversation mode and speech-only mode depending on the condition of the person being cared for or other circumstances. [Aspect 14] The care-related information providing device according to aspect 13, The processing unit identifies the care recipient based on the first information input from the input device, and executes the first setting state or the second setting state depending on the identified care recipient. Effect: The mode can be automatically switched depending on the person being cared for, making it easy to accommodate multiple people being cared for. [Aspect 15] The care-related information providing device according to any one of aspects 1 to 14, the input device includes a microphone; The processor selects and executes the third setting state or the fourth setting state; The content of the audio that the processor causes the audio output device to output is different between the third setting state and the fourth setting state, The care-related information providing device, wherein the processor switches from the third setting state to the fourth setting state when the voice input from the input device in the third setting state satisfies a predetermined characteristic. Effect: The care-related information providing device can switch to a mode that suits the condition of the care recipient in response to the care recipient's comments. [Aspect 16] The care-related information providing device according to aspect 15, The care-related information providing device, wherein the fourth setting state is more tolerant and accepting of conversation than the third setting state. Effect: For example, by responding to a dementia patient in the fourth setting state, the care-related information providing device can have a conversation appropriate for the dementia patient. [Aspect 17] The care-related information providing device according to any one of aspects 1 to 16, The processing unit changes the audio output from the audio output device in accordance with vital information of the care recipient. Effect: The care-related information providing device can output a voice corresponding to the vital signs of the care recipient from the voice output device. [Aspect 18] The care-related information providing device according to any one of aspects 1 to 17, the processor stores the first information and the second information in a storage in association with identification information of the care recipient; The processing device transmits the first information and the second information stored in the storage to a destination included in a transmission command via the communication interface in response to a transmission command given from outside via the communication interface. Effect: Information about the condition of the person receiving care can be provided upon request by family members, etc. [Aspect 19] A method for providing care-related information to a facility outside the facility, the method comprising: acquiring first information that is information for identifying the care recipient or information obtained by recording or measuring the care recipient; controlling an audio output device that outputs audio to output audio; The method for providing care-related information includes transmitting the first information and second information related to the voice output by the voice output device to an outside of the facility via a communication interface. [Explanation of symbols]

[0084] 1. Care-related information providing device 10 Collection Robot 11 Audio output devices 12 Camera 13 Microphone 14 Vital Sensor 16a processor 20 Information processing equipment 21 processors

Claims

1. A care-related information providing device that provides information about a care recipient residing in a facility to an outside of the facility, a processor for performing arithmetic processing; an audio output device that outputs audio; an input device that inputs first information, including information for identifying the care recipient or information obtained by recording or measuring the care recipient, to the processor; a communication interface; Equipped with the processor causes the audio output device to output audio; The processing unit transmits the first information and second information related to the voice output by the voice output device to an outside of the facility via the communication interface.

2. The care-related information providing device according to claim 1, the input device includes a microphone; The first information includes information on the voice of the care recipient responding to the voice output from the voice output device.

3. The care-related information providing device according to claim 1, the input device includes a camera; The first information includes an image of the care recipient listening to the audio output from the audio output device.

4. The care-related information providing device according to claim 1, the input device includes a vital sensor; The first information includes vital sign information of the care recipient, the audio of which is output from the audio output device.

5. The care-related information providing device according to claim 1, the input device includes a microphone; The processing unit causes the audio output device to output the audio when the first information indicating the audio of the care recipient is input from the input device.

6. The care-related information providing device according to claim 1, The processing unit identifies the care recipient based on the first information input from the input device, and actively causes the audio output device to output the audio to the identified care recipient even if the care recipient does not approach the processing unit.

7. The care-related information providing device according to claim 1, a moving device for moving the facility; the audio output device moves integrally with the mobile device; The processing device is a care-related information providing device that identifies the care recipient based on the first information input from the input device and controls the mobile device so that the audio output device approaches the identified care recipient.

8. The care-related information providing device according to claim 1, Equipped with a display, The processing unit identifies the care recipient based on the first information input from the input device, and displays an avatar associated with the identified care recipient on the display.

9. The care-related information providing device according to claim 1, Equipped with a display, the input device includes a camera; The care-related information providing device, wherein the camera is disposed adjacent to the display.

10. The care-related information providing device according to claim 9, a robot having a head, a body, and an actuator for moving the head; the input device includes a microphone; the display is disposed on the head; The processing device controls the actuator to move the head in accordance with the first information including audio input from the microphone or audio output from the audio output device.

11. The care-related information providing device according to claim 1, The processing device inputs the first information into a model created by machine learning, and outputs audio from the audio output device based on third information including the content of the utterance output by the model.

12. The care-related information providing device according to claim 11, A care-related information providing device in which the model has learned or is able to refer to the characteristics of those related to the person being cared for, and the third information acquired by the processor includes statements created in accordance with the characteristics of those related.

13. The care-related information providing device according to claim 1, the input device includes a microphone; The processor selects and executes a first setting state or a second setting state; In the first setting state, the processor causes the audio output device to output audio in response to first information including audio input from the input device; In the second setting state, the processor actively causes the audio output device to continue to output audio even if no audio is input from the input device.

14. The care-related information providing device according to claim 13, The processing unit identifies the care recipient based on the first information input from the input device, and executes the first setting state or the second setting state depending on the identified care recipient.

15. The care-related information providing device according to claim 1, the input device includes a microphone; The processor selects and executes a third setting state or a fourth setting state; The content of the audio that the processor causes the audio output device to output is different between the third setting state and the fourth setting state, The processing unit switches from the third setting state to the fourth setting state when the voice input from the input device in the third setting state satisfies a predetermined characteristic.

16. The care-related information providing device according to claim 15, The fourth setting state is more tolerant and accepting of conversation than the third setting state.

17. The care-related information providing device according to claim 1, The processing unit changes the audio output from the audio output device in accordance with vital information of the care recipient.

18. The care-related information providing device according to claim 1, the processor stores the first information and the second information in a storage in association with identification information of the care recipient; The processing device transmits the first information and the second information stored in the storage to a destination included in a transmission command via the communication interface in response to a transmission command given from outside via the communication interface.

19. A method for providing care-related information to a facility outside the facility, the method comprising: acquiring first information that is information for identifying the care recipient or information obtained by recording or measuring the care recipient; controlling an audio output device that outputs audio to output audio; A method for providing care-related information, comprising transmitting the first information and second information relating to the audio output by the audio output device to an outside of the facility via a communication interface.

Images