Robot, information processing system, information processing method, and program

Abstract

Provided are a robot, an information processing system, an information processing method, and a program that are capable of estimating emotions of a plurality of care recipients and utilizing the estimation results for care planning and implementation. An information processing system 1000 comprises: a vital data acquisition unit 11 that acquires vital data and facial expression data of a care recipient; an emotion estimation unit 31 that estimates a quantified emotion of the care recipient on the basis of the vital data and the facial expression data; a care proposal generation unit 32 that generates a proposal regarding care implementation on the basis of the quantified emotion; and a display unit 51 that displays at least one of the quantified emotion and the proposal.

Description

Robot, information processing system, information processing method and program

[0001] The present invention relates to a robot, an information processing system, an information processing method and a program, and in particular to a method for predicting the emotions of a care recipient and utilizing the prediction results in planning and implementing care.

[0002] Caregivers (including caregivers and care managers) who provide care to multiple care recipients (including care recipients) plan and provide care while checking the condition of the care recipients. For example, they may assign an order to multiple care recipients and provide care in sequence. However, care recipients may refuse to receive care due to their emotional state or other reasons. In such cases, the care provider must change the care plan, for example, the order of care, which increases the time required for care and places a greater burden on the care provider.

[0003] Therefore, it is desirable to provide an apparatus, system, method, and program that can estimate the emotions of multiple care recipients and use the estimation results to help plan and implement care.

[0004] Prior art related to this problem includes Patent Documents 1 and 2. Patent Document 1 discloses a method for estimating a subject's emotions based on first data including at least a heart rate and second data including at least a facial expression. Patent Document 2 discloses a system for displaying a menu of multiple activities to be performed in a care facility on a display device and displaying the progress of the activities.

[0005] Patent No. 6985005 JP 2021-051340 A

[0006] The inventions described in Patent Documents 1 and 2 do not provide a mechanism for estimating the emotions of multiple care recipients and utilizing the estimation results in planning and implementing care.

[0007] An object of one embodiment of the present invention is to provide a robot, an information processing system, an information processing method, and a program that assist care by predicting emotions.

[0008] In one embodiment, the information processing system includes a vital data acquisition unit that acquires vital data and facial expression data of a care recipient, a feeling estimation unit that estimates a quantified feeling of the care recipient based on the vital data and the facial expression data, a care suggestion generation unit that generates a suggestion regarding the provision of care based on the quantified feeling, and a display unit that displays at least one of the quantified feeling or the suggestion. In one embodiment, the feeling estimation unit calculates a relaxation level based on the vital data, calculates a smile level based on the facial expression data, and estimates the quantified feeling based on the relaxation level and the smile level. In one embodiment, if the relaxation level or the smile level cannot be calculated, the feeling estimation unit estimates the quantified feeling using information that the relaxation level or the smile level is unknown. In one embodiment, the suggestion includes information regarding whether care can be provided to the care recipient, how easy it is to provide it, or the order in which it should be provided. In one embodiment, the vital data acquisition unit includes a vital sensor installed in a cavity drilled in the outer surface of the robot, and the vital sensor senses the finger of the care recipient and measures the vital signs when the finger is inserted into the cavity. In one embodiment, the vital data acquisition unit further includes a clamping mechanism that starts an operation to clamp the finger of the care recipient when the finger of the care recipient is inserted into the cavity, and the sensor senses the finger and measures the vital signs in parallel with the clamping operation. This allows the care recipient to enjoy and relax during the measurement. In one embodiment, the vital data acquisition unit includes multiple cameras that capture the facial expressions of the care recipient from different angles. In one embodiment, the robot includes a vital data acquisition unit that acquires vital data and facial expression data of the care recipient, and a communication unit that transmits the vital data and the facial expression data to an emotion estimation unit, wherein the emotion estimation unit estimates a quantified emotion of the care recipient based on the vital data and the facial expression data, and the care suggestion generation unit generates a suggestion regarding the implementation of care based on the quantified emotion.By using the robot, facial expressions and vital signs close to their normal states can be acquired without harassing the care recipient. In one embodiment, the information processing method includes a vital data acquisition step of acquiring vital data and facial expression data of the care recipient, and a communication step of transmitting the vital data and the facial expression data to an emotion estimation unit, wherein the emotion estimation unit estimates a quantified emotion of the care recipient based on the vital data and the facial expression data, and a care suggestion generation unit generates a suggestion regarding the implementation of care based on the quantified emotion. Note that the suggestion can also be generated when the vital data or facial expression data is unknown. This is because care is necessary even when data is difficult to measure, and it is desirable to be able to suggest the condition, care order, etc. of the care recipient. In one embodiment, a program causes a computer to execute the above method.

[0009] The present invention can provide a robot, an information processing system, an information processing method, and a program that can estimate the emotions of multiple care recipients and use the estimation results to help plan and implement care.

[0010] Furthermore, the person receiving care will not feel aversion to the measuring equipment, and will be able to enjoy the measurement in a state close to normal.

[0011] 1 is a block diagram showing an example of the system configuration of an information processing system 1000. FIG. 1 is a diagram showing an example of the appearance of a measuring robot 1. FIG. 2 is a diagram showing an example of the appearance of a measuring robot 1. FIG. 3 is a diagram showing an example of the appearance of a measuring robot 1. FIG. 4 is a diagram showing an example of the appearance of a measuring robot 1. FIG. 5 is a diagram showing an example of the appearance of a measuring robot 1. FIG. 6 is a diagram showing an example of the internal structure of a measuring robot 1. FIG. 7 is a diagram showing an example of the shape of a hollow portion 105. FIG. 8 is a block diagram showing an example of the functional configuration of a measuring robot 1. FIG. 9 is a block diagram showing an example of the functional configuration of a server 3. FIG. 10 is a diagram showing an example of an emotion estimation method. FIG. 11 is a diagram showing an example of a proposal generation method. FIG. 12 is a block diagram showing an example of the functional configuration of a terminal device 5. FIG. 13 is a flowchart showing an example of the operation of the information processing system 1000. FIG. 14 is a flowchart showing an example of the operation of a measuring robot 1.

[0012] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, specific embodiments to which the present invention is applied will be described in detail with reference to the drawings.

[0013] 1 is a block diagram showing a system configuration of an information processing system 1000 according to a first embodiment of the present invention. The information processing system 1000 includes a measuring robot 1, a server 3, and a terminal device 5.

[0014] It should be noted that with regard to the system configuration of the information processing system 1000, components may be added, changed, or deleted within the scope of the present invention.

[0015] For example, in the above embodiment, the server 3 and the terminal device 5 are assumed to be independent devices, but the server 3 and the terminal device 5 may be integrated into a single information processing device. In this case, communication between the server 3 and the terminal device 5 is treated as communication within the integrated information processing device.

[0016] Furthermore, the measuring robot 1 may have at least some of the functions of the server 3 and the terminal device 5. For example, functions of the server 3, such as emotion estimation and suggestion generation (described later), may be implemented in the measuring robot 1.

[0017] The measuring robot 1 acquires facial expressions and vital signs of each care recipient. In this case, since the measuring robot 1 does not cause any feelings of disgust or fear to the care recipient, facial expressions and vital signs close to those in their normal state can be acquired.

[0018] 2A to 2F are diagrams showing an example of the appearance of the measuring robot 1. FIG. 2A is a front view, FIG. 2B is a right side view, FIG. 2C is a left side view, FIG. 2D is a rear view, FIG. 2E is a top view, and FIG. 2F is a bottom view. In this example, the measuring robot 1 has a rounded outer shape that makes it easy for the care recipient to stroke or hold, and is covered with a texture such as brushed fur that is pleasant to the touch. It also has parts that evoke the image of a living thing, such as eyes, a mouth, and arms, evoking a sense of familiarity and security in the care recipient. Note that the overall shape, size, materials, types and arrangement of parts of the measuring robot 1 shown in FIG. 2 are merely examples, and are not limited to these. Any shape, size, materials, types and arrangement of parts may be adopted.

[0019] 3 is a diagram showing an example of the internal structure beneath the covering of the measuring robot 1. The measuring robot 1 includes an exoskeleton 10. A camera 101, a microphone 102, a speaker 103, and a display unit 104 are attached to the exoskeleton 10. A hollow portion 105 is drilled in a part of the exoskeleton 10, and a finger detection unit 106, a clamping mechanism 107, and a vital sensor 108 are attached surrounding the hollow portion 105. A control device 110 is also provided inside the exoskeleton 10.

[0020] The exoskeleton 10 is a structural component that forms the skeleton of the measuring robot 1, and is made of metal, plastic resin, or the like. Preferably, the exoskeleton 10, excluding the camera 101 and hollow portion 105 (described later), is covered with a texture that is pleasant to the touch, such as brushed fabric. This allows the care recipient to feel a soft and familiar impression when touching the measuring robot 1.

[0021] The camera 101 photographs the care recipient in accordance with instructions from the control device 110 and outputs image data (including still images and videos) to the control device 110. Typically, the camera 101 is attached to a position corresponding to the eye of the measurement robot 1. If there is a covering, an opening is provided at a position corresponding to the camera 101.

[0022] Preferably, the measurement robot 1 has a plurality of cameras 101. For example, the plurality of cameras 101 are built in at separate positions, such as at both eyes, so that the face of the care recipient can be photographed from a plurality of different angles.

[0023] The facial expression of the care recipient can be acquired from the facial image. In addition, the facial color and vascular movement of the care recipient can be identified from the facial image, and vital data such as heart rate can be acquired based on this.

[0024] The microphone 102 collects sounds around the measuring robot 1 in accordance with commands from the control device 110, and outputs the sound data to the control device 110. The microphone 102 is attached to the measuring robot 1 at any position.

[0025] The speaker 103 outputs sound in accordance with commands from the control device 110. The speaker 103 is attached to the measuring robot 1 at any position.

[0026] The display unit 104 outputs information visible to the care recipient in accordance with commands from the control device 110. For example, one form of the display unit 104 is an LED lamp 104-1 that emits light in a predetermined color or pattern. Alternatively, another form of the display unit 104 is a display 104-2 that can display any text or image. In the example of FIG. 3 , the LED lamp 104-1 serving as the display unit 104 is attached to the tip of a protrusion resembling a horn or hat that is provided in a position corresponding to the head of the measuring robot 1. When attaching the display 104-2 to the measuring robot 1, the display 104-2 can be attached to the abdomen or torso as shown in FIG. 3 , or to a position corresponding to the face or back.

[0027] The cavity 105 is a hole drilled in the surface of the measuring robot 1. Figures 4A and 4B are diagrams showing an example of the shape of the cavity 105, with (a) being a view of the cavity 105 from the front of the measuring robot 1 and (b) being a cross-sectional view of the cavity 105 from the right side of the measuring robot 1. As shown in Figures 4A and 4B, the shape of the opening corresponding to the entrance of the cavity 105 can be approximately oval, approximately rectangular, or the like, but any shape is acceptable as long as it allows the insertion of an index finger. The depth D of the cavity 105 is preferably approximately midway between the first and second joints of a human index finger, for example, an adult male's, typically around 3 cm to 5 cm. The inner diameter R of the cavity 105 is preferably large enough to allow the insertion of a human index finger with ease, typically 2 cm or more. Furthermore, the inner diameter of the cavity 105 preferably gradually decreases from the entrance (R1) to the innermost portion (R2) (R1 > R2), i.e., it is preferably tapered. This makes it easier for a finger to fit inside the cavity 105. The inner diameter R of the cavity 105 can be reduced by the action of the clamping mechanism 107, which will be described later. The above-mentioned size of the inner diameter is the size of the inner diameter before the clamping mechanism 107 is operated. The inner wall surface of the cavity 105 is preferably made of a soft (easily deformable) material such as cloth, rubber, or elastomer. Typically, the cavity 105 is perforated at a position corresponding to the mouth of the measuring robot 1. If there is a covering, an opening is provided at a position corresponding to the cavity 105. The mouth is typically provided at a position below the parts that resemble eyes.

[0028] The finger detection unit 106 detects the insertion of the care recipient's finger (typically the index finger) into the cavity 105 and notifies the control device 110 of the detection. For example, as shown in FIGS. 4A and 4B , a pressure sensor, photoelectric sensor, heat sensor, infrared sensor, or the like may be provided at the end of the innermost portion of the cavity 105 to detect the care recipient's fingertip being abutted against it. Alternatively, as shown in FIGS. 4A and 4B , a photoelectric sensor, heat sensor, infrared sensor, or the like may be provided at any position on the wall surface of the cavity 105 (excluding the innermost portion) to detect the care recipient's fingertip. In this case, it is preferable that the finger detection unit 106 determines whether the finger has been inserted sufficiently far, in other words, whether the finger has been inserted to a position where vital signs can be measured reliably. For example, the finger detection unit 106 can measure the outer diameter of the finger and, when the outer diameter measured at a certain depth (distance from the entrance) exceeds a predetermined threshold, determine that the finger is inserted sufficiently deep and output a detection notification. Alternatively, the finger detection unit 106 can measure the position of the fingertip and, when the position of the fingertip reaches a predetermined depth (distance from the entrance), i.e., the threshold, determine that the finger is inserted sufficiently deep and output a detection notification. This threshold can be set, for example, based on whether the vital sensor 108 overlaps a desired measurement position on the finger (typically the pad of the finger, i.e., the area from the fingertip to the first joint). To set this threshold, the finger detection unit 106 may have a calibration function that measures the outer diameter and position of the fingertip while the vital sensor 108 is overlapped with the desired measurement position on the finger and sets the threshold. Note that the vital sensor 108 is preferably positioned at a position corresponding to the pad of the finger when, for example, an adult male inserts his finger to a depth of more than the first joint but less than the second joint.

[0029] Clamping mechanism 107 is a mechanism that reduces and restores inner diameter R of cavity 105 in accordance with commands from control device 110. In this embodiment, inner diameter R of cavity 105 indicates the shortest distance between the inner walls of cavity 105 at a certain position (distance from the entrance), and does not necessarily indicate that the cross section of cavity 105 is circular.

[0030] For example, as shown in FIGS. 4A and 4B , a clamping plate 1071 is provided at a position outside the inner wall surface of cavity 105 so as to sandwich or encase the finger, and an actuator moves clamping plate 1071 in a direction that reduces the inner diameter R of the cavity. Alternatively, clamping plate 1071 may be moved by any reciprocating mechanism. Alternatively, the inner diameter R may be reduced by inflating an airbag 1071 provided at a position outside the inner wall surface of cavity 105 so as to sandwich or encase the finger. This clamping action (the action of reducing the inner diameter R of cavity 105) causes the care recipient's finger inserted into cavity 105 to be sandwiched or encased by the inner wall surface of cavity 105. Clamping mechanism 107 may stop the clamping action (maintaining the inner diameter R of cavity 105 constant) when it detects a certain load or reaction force. This allows the care recipient to pinch or encase the finger with a comfortable strength without feeling any pain. Furthermore, the clamping mechanism 107 may repeat the clamping and release operations (expanding the inner diameter R of the cavity 105 and releasing the finger) at a constant rhythm or randomly. The strength of the clamping operation (the magnitude of pressure) may be varied at a constant rhythm or randomly. This allows the care recipient to feel a comfortable sensation, as if their finger is being massaged. Note that the various mechanisms shown here are examples of the clamping mechanism 107, and the present invention is not limited to these.

[0031] In accordance with commands from the control device 110, the vital sensor 108 senses the finger of the care recipient inserted into the cavity 105, measures vital data, and notifies the control device 110 of the measurement results. The vital data measured by the vital sensor 108 typically includes, but is not limited to, pulse (heart rate), body temperature, blood pressure, blood oxygen concentration, etc. As shown in Figures 4A and 4B, it is preferable to install the vital sensor 108 on the surface (inside) or outside of the inner wall surface of the cavity 105, at a position corresponding to the pad of the finger when, for example, an adult male inserts his finger to a depth of more than the first joint but less than the second joint.

[0032] The control device 110 is an information processing device including a processing device, a storage device, an input / output device, a communication device, etc. The control device 110 is communicatively connected to the camera 101, the microphone 102, the speaker 103, the display unit 104, the finger detection unit 106, the clamping mechanism 107, and the vital sensor 108, and controls the operations of these devices.

[0033] In this way, when the care recipient places a finger in the cavity 105, the clamping mechanism 104 moves, providing a sensation similar to a gentle bite or a comfortable feeling as if the fingertip is being wrapped around the fingertip, thereby avoiding feelings of aversion or anxiety and eliminating psychological resistance to measurement. Furthermore, since no complicated steps such as attaching a sensor are required, psychological resistance to measurement is not induced. Therefore, negative effects on vital data (e.g., increases in heart rate, body temperature, blood pressure, etc.) due to psychological resistance can be suppressed. Using such vital data to perform emotion estimation can suppress discrepancies between actual emotions and emotion estimation results.

[0034] Furthermore, by providing the vital sensor 108 inside the hollow portion 105, it is possible to prevent light from entering from the outside world. This improves measurement accuracy when an optical sensor (e.g., an infrared sensor, a green sensor, etc.) is used as the vital sensor 108.

[0035] Furthermore, the measuring robot 1 encourages the care recipient to voluntarily measure their vital signs, which makes it possible to reduce the working time of the measurer who manages a large number of care recipients.

[0036] The measuring robot 1 also performs a clamping operation on the finger inserted into the cavity 105 and then starts measuring the vital signs. This stabilizes the position of the finger during vital sign measurement. Furthermore, the finger can be held in approximately the same position no matter how many times measurements are performed. This improves the precision, accuracy, and stability of measurement accuracy.

[0037] Furthermore, the cavity 105 of the measuring robot 1 has an inner diameter that can easily accommodate the index finger of an adult male, which is statistically assumed to be the thickest finger, and the clamping mechanism 107 reduces the inner diameter of the cavity 105. This allows stable vital sign measurement to be performed regardless of the thickness of the finger.

[0038] The measuring robot 1 also has multiple cameras 101. This allows the care recipient to be photographed from multiple different angles, increasing the opportunities to photograph the care recipient's facial expressions from the front. The multiple cameras also allow the distance between the measuring robot 1 and the care recipient to be measured. Distance information can be used as a parameter to improve measurement accuracy, for example, when remotely measuring vital data (heart rate) using images captured by the cameras 101.

[0039] The measuring robot 1 may also be equipped with a contact-type vital sensor (not shown). For example, if the contact-type vital sensor is placed in the part of the palm of the care recipient's hand when the care recipient holds the measuring robot 1 (such as the part of the measuring robot 1 that touches the armpit), the measurement can be performed without the care recipient being aware of it.

[0040] 5 is a block diagram showing the functional configuration of the measuring robot 1. The measuring robot 1 includes a vital data acquisition unit 11, a facial expression data acquisition unit 12, and a communication unit 13.

[0041] The vital data acquisition unit 11 acquires vital data of the care recipient. The vital data is typically heart rate. The heart rate can be acquired by a vital sensor 108. Alternatively, the heart rate can be acquired by analyzing images captured by the camera 101. The method of measuring heart rate by image analysis is well known, and therefore a description thereof will be omitted.

[0042] The facial expression data acquisition unit 13 acquires facial expression data of the care recipient. The facial expression data may be image data of a photograph of the care recipient's face, or may be feature quantities extracted from the image data. It is preferable that the facial expression data be acquired approximately simultaneously, i.e., in parallel, with the above-mentioned vital data.

[0043] The communication unit 13 transmits the vital data acquired by the vital data acquisition unit 11 and the facial expression data acquired by the facial expression data acquisition unit 13 to the server 3 .

[0044] It should be noted that components of the measuring robot 1 can be added, changed, or deleted within the scope of the present invention.

[0045] For example, the control device 110 can prompt the care recipient to take regular measurements using the speaker 103 or the display unit 104. Typically, when a preset measurement timing arrives, a voice, text, image, or the like such as "Mr. / Ms. X, let's measure your vital signs" is output. This can prompt the care recipient to take measurements voluntarily. It can also reduce forgetting to measure and variations in measurement times.

[0046] For example, the control device 110 can notify the care recipient of the start or end of measurement via the speaker 103 or the display unit 104. Typically, when measurement starts, it outputs a voice, text, image, or the like such as "Mr. / Ms. XX, please wait as we will now measure you" and when measurement ends, it outputs a voice, text, image, or the like such as "Mr. / Ms. XX, measurement is complete."

[0047] For example, the control device 110 may have a function of acquiring a facial image or voice audio data of the care recipient using the camera 101 or the microphone 102 and identifying the care recipient (identifying the individual). Typically, the care recipient can be identified by determining whether the similarity between the feature amounts of the facial image or voice audio data of a pre-registered care recipient and the feature amounts of the acquired facial image or voice audio data is equal to or greater than a predetermined threshold. In this case, the control device 110 can output a pair of information identifying the care recipient (such as a personal identifier) ​​and the measurement results of the care recipient.

[0048] For example, the control device 110 can accumulate measurement results over a predetermined period of time. It can also calculate statistics of the measurement results, compare measurement results or statistical values ​​at any multiple points in time, and output the comparison results. For example, it can calculate and output the average, maximum, and minimum values ​​of measurement values ​​over a predetermined period of time. It can also output past (e.g., n days ago, n weeks ago, n months ago) measurements alongside current measurements, or output the differences. Typically, in the measurement result output step (S105), the progress and comparison of the measurement results can be displayed in text or graphs on the display 104-2 of the display unit 104, or audibly notify the user by an audio message such as, "Your blood pressure appears higher than last month." If the device also has a function for identifying the care recipient (individual), it can accumulate measurement results for each individual and output the progress of the measurement results.

[0049] The server 3 analyzes the facial expressions and vital signs of multiple care recipients to estimate their emotions. Based on the estimated emotions, the server 3 also generates suggestions that are useful for planning and implementing care.

[0050] The server 3 is an information processing device equipped with a processing device, a storage device, an input / output device, a communication device, etc., and is typically a server computer. Note that the server 3 does not necessarily have to be a single information processing device, and may be, for example, a virtual information processing device such as a cloud computing environment realized by the cooperation of multiple information processing devices.

[0051] 6 is a block diagram showing the functional configuration of the server 3. The server 3 includes a feeling estimation unit 31, a care suggestion generation unit 32, and a communication unit 33.

[0052] The emotion estimation unit 31 estimates the emotion of the care recipient based on the vital data and facial expression data received from the measurement robot 1. The emotion estimation algorithm will be described below.

[0053] The feeling estimation unit 31 measures (quantifies) the relaxation level based on the vital data. The relaxation level is calculated, for example, by the following procedure.

[0054] (1) Calculate a valid RRI (R-R Interval) from the heartbeat data. The RRI is a value indicating the time from one beat (ventricular excitation) to the next beat (ventricular excitation). Preferably, 20 to 40 consecutive RRIs (time series data) are acquired.

[0055] (2) Calculate pNN (Percentage of Normal to Normal Intervals) from the time series data of RRI. pNN is an index showing the percentage of times when the difference (absolute value) between adjacent RRIs exceeds a threshold, and indicates the dominance of activity of the parasympathetic nervous system.

[0056] (3) The relaxation level is determined according to the pNN value. For example, if pNN is 30 or less, the relaxation level is 30; if pNN is more than 30 but less than 60, the relaxation level is 60; if pNN is more than 60 but less than 100, the relaxation level is 100. If pNN cannot be calculated, the relaxation level is treated as unknown. In other words, in this example, the relaxation level of the care recipient is classified into four categories: three levels + unknown.

[0057] The emotion estimation unit 31 measures (quantifies) the smile level based on the facial expression data. The smile level is calculated, for example, by the following procedure. The smile level is calculated, for example, by the following procedure.

[0058] (1) Calculate the degree of similarity (%) between the facial expression data of the care recipient and a reference image. The reference image is facial expression data that serves as a standard for determining the smile degree, and is assumed to be stored in advance by the emotion estimation unit 31.

[0059] (2) The smile level is determined according to the similarity. For example, if the similarity is 30% or less, the smile level can be set to 30; if the similarity is more than 30% but less than 60%, the smile level can be set to 60; and if the similarity is more than 60% but less than 100%, the smile level can be set to 100. If the similarity cannot be calculated, the smile level is treated as unknown. In other words, in this example, the smile level of the care recipient is classified into four categories: three levels + unknown.

[0060] The emotion estimation unit 31 measures (quantifies) emotions based on the relaxation level and smile level. The emotions are calculated, for example, by the following procedure.

[0061] An emotion matrix with the relaxation level and smile level on the vertical and horizontal axes is defined in advance. Fig. 7 shows an example of the emotion matrix. In the example of Fig. 7, the relaxation level is classified into four categories (three levels + unknown), and the smile level is classified into four categories (three levels + unknown). Emotions quantified according to the combination of the relaxation level and smile level categories are defined. For example, if the relaxation level is 30 and the smile level is 100, the emotion is determined to be "1." In the example of Fig. 7, emotions are quantified as integers between 1 and 9.

[0062] This embodiment is characterized in that emotions are defined even when the relaxation level or smile level is unknown. In the example of Fig. 7, when the relaxation level is unknown and the smile level is 100, the emotion is determined to be "2".

[0063] When the relaxation levels and smile levels for multiple care recipients are received, the emotion estimation unit 31 estimates the emotion for each of the care recipients.

[0064] The care suggestion generation unit 32 generates suggestions that can be used as reference by a caregiver when providing care, based on the emotion of the care recipient estimated by the emotion estimation unit 31. The suggestions are generated, for example, by the following procedure.

[0065] The care suggestion generation unit 32 previously stores a mechanism for generating suggestions regarding the necessity of providing care, the ease of providing care, the order of providing care, etc., depending on whether the emotional state is suitable for providing care and the degree of suitability. For example, FIG. 8 shows suggestions corresponding to quantified emotions defined in advance in the form of a table. In the example of FIG. 8, emotions are divided into multiple ranges, and suggestions corresponding to each range are defined. The suggestions defined are: "Proactively and promptly provide care" for emotions 1, 2, or 4; "Normally provide care" for emotions 3, 5, or 7; and "Postpone care, if possible, avoid care" for emotions 6, 8, or 9.

[0066] The care suggestion generating unit 32 generates such suggestions for each of the multiple care recipients.

[0067] The communication unit 33 receives vital data and facial expression data of the care recipient from the multiple measurement robots 1. Typically, one care recipient uses one measurement robot 1, so the vital data and facial expression data of multiple care recipients are collected at any time in the server 3. The communication unit 33 also transmits the emotions of the care recipients estimated by the emotion estimation unit 31 and the suggestions generated by the care suggestion generation unit 32 to the terminal device 5.

[0068] According to this embodiment, the emotion estimation unit 31 classifies the smile level into multiple categories, including cases where it is unknown, and the relaxation level into multiple categories, including cases where it is unknown, and quantifies the emotion. Therefore, even when the smile level or relaxation level is unknown, the emotion can be quantified. This allows the emotion to be determined immediately without remeasurement or reanalysis. This also reduces the burden on the person making the measurement.

[0069] It should be noted that components of the server 3 can be added, changed, or deleted within the scope of the present invention.

[0070] For example, in this embodiment, the relaxation level is classified into four categories (three levels + unknown), but the number of levels may be arbitrary. Similarly, the smile level is classified into four categories (three levels + unknown), but the number of levels may be arbitrary.

[0071] In addition, although the relaxation level is calculated based on pNN in this embodiment, the relaxation level may be calculated based on vital data using any other known method. Similarly, the smile level is calculated based on the similarity to the reference image, but the smile level may be calculated based on facial expression data using any other known method.

[0072] Furthermore, although in this embodiment, the quantified emotion is estimated based on an emotion matrix, the emotion may be estimated by any other method as long as the relaxation level and smile level are used. For example, the quantified emotion can be obtained by applying the relaxation level and smile level to a given mathematical formula (including a statistical model). Furthermore, the quantified emotion can be obtained by inputting the calculated relaxation level and smile level into a machine-learned model that includes the relaxation level and smile level as explanatory variables.

[0073] In addition, although in this embodiment, suggestions regarding care planning and implementation are estimated using a predefined table, any other method may be used to estimate emotions as long as quantified emotions are used. For example, appropriate suggestions can be identified by applying quantified emotions to a given mathematical formula (including a statistical model). Furthermore, suggestions can be output by inputting calculated emotions into a machine-learned model that includes quantified emotions as explanatory variables.

[0074] The terminal device 5 presents information useful for planning and implementing care to the care provider.

[0075] The terminal device 5 is an information processing device equipped with a processing device, a storage device, an input device, a display device, a speaker, a communication device, etc., and is typically a personal computer (PC), a tablet computer, a smartphone, etc.

[0076] 9 is a block diagram showing the functional configuration of the terminal device 5. The terminal device 5 includes a display unit 51 and a communication unit 52.

[0077] The display unit 51 displays the emotions of the multiple care recipients and suggestions regarding care implementation received from the server 3 on the display device. For example, if there are multiple care recipients, suggestions such as a ranking of ease of implementation and a recommended implementation order can be displayed. The care provider plans and implements care based on the displayed emotions and suggestions. For example, the care provider can switch the order of care recipients in a good emotional state with care recipients in an unwell state, or adjust the implementation schedule.

[0078] The communication unit 52 receives the emotion of the care recipient estimated by the emotion estimation unit 31 and the suggestion generated by the care suggestion generation unit 32 from the server 3.

[0079] Conventionally, a caregiver provides care sequentially while checking the condition of multiple care recipients. However, according to the present embodiment, the display unit 51 displays the emotions of the care recipients, suggestions regarding the necessity of providing care, the ease of providing care, the order in which care should be provided, and the like. The caregiver can refer to this information and take more appropriate measures, such as providing care to those who do not refuse care first. This allows care to be provided more smoothly than before, reducing the psychological and time burden on the caregiver. Furthermore, since the care recipient can avoid or postpone care when they are in an emotional state where they do not want care, they are less likely to feel uncomfortable with the caregiver and are able to build a trusting relationship with the caregiver.

[0080] FIG. 10 is a flowchart showing an example of the operation of the information processing system 1000.

[0081] S1: Acquisition of vital data and facial expression data The measuring robot 1 acquires the vital data and facial expression data of the care recipient and transmits them to the server 3.

[0082] S2: Emotion Estimation The emotion estimation unit 31 of the server 3 calculates the relaxation level based on the vital data received in step S1 and the smile level based on the facial expression data. In addition, the emotion of the care recipient is estimated based on the relaxation level and the smile level.

[0083] S3: Proposal Generation The care proposal generation unit 32 of the server 3 generates a proposal that the care provider can refer to when providing care, based on the emotion of the care recipient estimated in step S2.

[0084] S4: Display of Emotion and Suggestion The server 3 transmits the emotion estimated in step S2 and / or the suggestion generated in step S3 to the terminal device 5. The terminal device 5 displays the received emotion and / or suggestion on a display device.

[0085] FIG. 11 is a flowchart showing an example of the operation of the measuring robot 1 in step S1.

[0086] S101: Finger insertion detection The care recipient inserts a fingertip into the hollow portion 105 drilled in the measuring robot 1. For example, if the measuring robot 1 has an outer shape that resembles an animal and the hollow portion 105 is drilled at a position corresponding to the animal's mouth, the care recipient will perform an action as if inserting a fingertip into the animal's mouth.

[0087] The finger detection unit 106 detects that the finger of the care recipient has been inserted into the cavity 105, and notifies the control device 110 of the detection.

[0088] S102: Start of clamping operation The control device 110 commands the clamping mechanism 107 to perform the clamping operation. The clamping mechanism 107 starts the clamping operation.

[0089] S103: Vital Sign Measurement The control device 110 commands the vital sensor 108 to measure vital data. The vital sensor 108 senses the fingertip of the care recipient to measure the vital data.

[0090] The clamping mechanism 107 can continue to clamp the finger of the care recipient during measurement. This prevents the fingertip from moving during measurement, allowing for stable measurement. The clamping mechanism 107 can also repeat clamping and release operations at any interval, or vary the strength of the clamping action. This gives the care recipient a comfortable sensation, similar to receiving a massage, and they will try to maintain this position without pulling out their fingertip, allowing for stable measurement.

[0091] S104: End of clamping operation The vital sensor 108 notifies the control device 110 of the measurement result.

[0092] The control device 110 instructs the clamping mechanism 107 to end the clamping operation. The clamping mechanism 107 ends the clamping operation and restores the inner diameter R of the cavity 105.

[0093] S105: Photographing Facial Expressions The camera 101 photographs the facial expressions of the care recipient and acquires facial expression data.

[0094] S106: Output The control device 110 outputs the vital data and facial expression data. That is, the vital data and facial expression data are transmitted to the server 3 via the communication device. For example, by using multiple measurement robots 1 to acquire vital data of multiple care recipients, the server 3 can easily aggregate the vital data of multiple care recipients.

[0095] Although one embodiment of the present invention has been described above, the present invention is not limited to this, and components can be added, changed, or deleted within the scope that does not depart from the spirit of the present invention.

[0096] Furthermore, some of the processes described in the above embodiments as being executed by hardware can also be logically realized by having a CPU (Central Processing Unit) execute a computer program. In this case, the computer program can be stored on various types of non-transitory computer-readable media and supplied to a computer. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memories (e.g., mask ROMs, programmable ROMs (PROMs), erasable PROMs (EPROMs), flash ROMs, and random access memories (RAMs)). The program may also be supplied to a computer by various types of transient computer-readable media. Examples of transient computer-readable media include electric signals, optical signals, and electromagnetic waves. The transient computer-readable medium can supply the program to a computer via a wired communication path such as an electric wire or optical fiber, or via a wireless communication path.

[0097] It should be noted that within the scope of the present invention, the embodiments may be freely combined, or any of the components of the embodiments may be modified, or any of the components of the embodiments may be omitted.

[0098] 1 Measuring robot 10 Exoskeleton 101 Camera 102 Microphone 103 Speaker 104 Display unit 104-1 LED lamp 104-2 Display 105 Cavity 106 Finger detection unit 107 Clamping mechanism 1071 Clamping plate, airbag 108 Vital sensor 110 Control device 11 Vital data acquisition unit 12 Facial expression data acquisition unit 13 Communication unit 3 Server 31 Emotion estimation unit 32 Care suggestion generation unit 33 Communication unit 5 Terminal device 51 Display unit 52 Communication unit

Claims

1. An information processing system comprising: a vital data acquisition unit that acquires vital data and facial expression data of a care recipient; an emotion estimation unit that estimates a quantified emotion of the care recipient based on the vital data and the facial expression data; a care suggestion generation unit that generates suggestions regarding the implementation of care based on the quantified emotion; and a display unit that displays at least one of the quantified emotion or the suggestion.

2. The information processing system according to claim 1, wherein the emotion estimation unit calculates a relaxation level based on the vital data, calculates a smile level based on the facial expression data, and estimates the quantified emotion based on the relaxation level and smile level.

3. The information processing system according to claim 1, wherein, when the relaxation level or the smile level cannot be calculated, the emotion estimation unit estimates the quantified emotion using information that the relaxation level or the smile level is unknown.

4. The information processing system according to claim 1, wherein the proposal includes information regarding whether or not care can be provided to the care recipient, the ease of providing care, or the order in which care should be provided.

5. The information processing system of claim 1, wherein the vital data acquisition unit includes a vital sensor installed in a hollow portion drilled in the outer surface of the robot, and the vital sensor senses the finger of the person being cared for and measures the vital signs when the finger is inserted into the hollow portion.

6. The information processing system of claim 5, wherein the vital data acquisition unit further has a clamping mechanism that begins clamping the finger when the finger of the care recipient is inserted into the hollow portion, and the sensor senses the finger and measures the vital signs in parallel with the clamping operation.

7. The information processing system according to claim 1, wherein the vital data acquisition unit includes a plurality of cameras that capture the facial expressions of the care recipient from different angles.

8. A robot including: a vital data acquisition unit that acquires vital data and facial expression data of a care recipient; and a communication unit that transmits the vital data and the facial expression data to an emotion estimation unit, wherein the emotion estimation unit estimates a quantified emotion of the care recipient based on the vital data and the facial expression data; and a care suggestion generation unit that generates suggestions regarding the implementation of care based on the quantified emotion.

9. An information processing method comprising: a vital data acquisition step of acquiring vital data and facial expression data of a care recipient; and a communication step of transmitting the vital data and the facial expression data to an emotion estimation unit, wherein the emotion estimation unit estimates a quantified emotion of the care recipient based on the vital data and the facial expression data, and a care suggestion generation unit generates a suggestion regarding the implementation of care based on the quantified emotion.

10. A program for causing a computer to execute the method according to claim 9.

Images