Information processing device, mutual monitoring method, control program, and mutual monitoring system

Abstract

To provide an information processing device and the like, which allow for supporting mutual aid among group members (by optimizing the form of mutual aid).SOLUTION: An information processing device is provided, comprising a target detection unit configured to detect a target member in need of care by other members of a group on the basis of the condition of the members of the group estimated from a captured image of the members, and a notification unit for notifying each member of the group of the detected target member.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, a mutual monitoring method, a control program, and a mutual monitoring system. [Background technology]

[0002] By analyzing a person's face image, it is possible to estimate various conditions of the person. For example, the following Patent Document 1 discloses a patient monitoring device that estimates the emotion of a patient using a face image and calculates a stress value indicating the degree of stress of the patient based on the emotion estimation result and the patient's medical information. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-146345 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the patient monitoring device described in Patent Document 1, the roles are divided so that the examinee (patient) is the one being monitored, and the medical facility staff is the one doing the monitoring. However, rather than a fixed division of roles in which one of the group members is the one watching over and the other is the one being watched over, a flat form of mutual assistance may be preferable. For example, when a member of a group is not in good mental or physical condition, it may be preferable for the care to be given by someone who is able to care for that member at that time, rather than a predetermined member of the group.

[0005] The present disclosure has been made in consideration of the above problems, and an exemplary purpose thereof is to provide an information processing device and the like that supports mutual assistance among members of a group. [Means for solving the problem]

[0006] An information processing device according to an exemplary aspect of the present disclosure includes a target detection means for detecting a target member of a group who requires care from other members based on a condition of the member estimated from an image of the member, and a notification means for notifying each member of the group of the detected target member.

[0007] In a mutual monitoring method according to an exemplary aspect of the present disclosure, at least one processor executes a target detection process to detect a target member of the group who needs care from other members based on a condition of the member estimated from an image of the member, and a notification process to notify each member of the group of the detected target member.

[0008] A control program according to an exemplary aspect of the present disclosure causes a computer to function as a target detection means for detecting a target member of a group who needs care from other members based on a condition of the member estimated from an image taken of the member, and as a notification means for notifying each member of the group of the detected target member. Effect of the Invention

[0009] According to an exemplary aspect of the present disclosure, an exemplary effect is provided in that a technology for supporting mutual assistance among members of a group can be provided. [Brief description of the drawings]

[0010] [Figure 1] 1 is a block diagram showing a configuration of an information processing device according to the present disclosure. [Diagram 2] 1 is a flow diagram showing the flow of a mutual monitoring method according to the present disclosure. [Diagram 3] 1 is a diagram illustrating a configuration example of a mutual monitoring system according to the present disclosure. [Figure 4] FIG. 13 is a block diagram showing a configuration of another information processing device according to the present disclosure. [Diagram 5] FIG. 2 is a block diagram illustrating a configuration example of a terminal device according to the present disclosure. [Figure 6] 13A and 13B are diagrams illustrating examples of UI screens before and after notifying a target member. [Figure 7] 13A to 13C are diagrams illustrating examples of UI screens when a report is presented, when data is presented, and when a video call is made. [Figure 8] FIG. 13 illustrates an example of a UI screen presenting providers for the care of a target member. [Figure 9] 13 is a diagram showing an example of a UI screen for notifying a discrepancy between the result of self-assessment of a state and the result of estimation of a state. FIG. [Figure 10] 5 is a flow chart showing an example of processing executed by the information processing device shown in FIG. 4. [Figure 11] 6 is a flow chart showing an example of a process executed by the terminal device shown in FIG. 5. [Figure 12] FIG. 1 is a block diagram showing a configuration of a computer that functions as an information processing device according to the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] The following are examples of embodiments of the present invention. However, the present invention is not limited to the exemplary embodiments shown below, and various modifications are possible within the scope of the claims. For example, an embodiment obtained by appropriately combining the technical means employed in each of the exemplary embodiments shown below may also be included in the scope of the present invention. In addition, an embodiment obtained by appropriately omitting a part of the technical means employed in each of the exemplary embodiments shown below may also be included in the scope of the present invention. In addition, the effects mentioned in each of the exemplary embodiments shown below are examples of effects expected in the exemplary embodiments, and do not define the scope of the present invention. In other words, an embodiment that does not exhibit the effects mentioned in each of the exemplary embodiments shown below may also be included in the scope of the present invention.

[0012] First Exemplary Embodiment A first exemplary embodiment, which is an example of an embodiment of the present invention, will be described in detail with reference to the drawings. This exemplary embodiment is the basic form of each exemplary embodiment described later. The scope of application of each technical means adopted in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technical means adopted in this exemplary embodiment can be adopted in other exemplary embodiments included in this disclosure to the extent that no particular technical obstacle occurs. In addition, each technical means shown in the drawings referred to for explaining this exemplary embodiment can also be adopted in other exemplary embodiments included in this disclosure to the extent that no particular technical obstacle occurs.

[0013] [Configuration of information processing device 1] The configuration of the information processing device 1 will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of the information processing device 1. As shown in Fig. 1, the information processing device 1 includes an object detection unit 101 and a notification unit 102.

[0014] The target detection unit 101 detects target members of the group who need care from other members based on the state of the group member estimated from an image of the member. When the state of each of a plurality of members is estimated from an image of each of the members, the target detection unit 101 detects the target member from among the members. When the state of one of the members is estimated from an image of the member, the target detection unit 101 determines whether the member is a target member.

[0015] Here, "care" refers to any action aimed at maintaining or improving the condition of a member. For example, calling a member who is not feeling well to check on their condition, or calling out to or sending a message to a member who is feeling depressed or unstable, are also included in the category of "care." "Care" can also be expressed as consideration, attention, or care.

[0016] It is only necessary to predefine what state a member needs to be in when "care" is required. For example, a member whose stress level is estimated to be high from an image, or a member whose stress level is showing a tendency to increase, may be determined as a member who needs "care." In this case, the target detection unit 101 detects target members based on the stress level of each member. The stress level is an index value indicating the magnitude of stress.

[0017] The notification unit 102 notifies each member of the group of the target members detected by the target detection unit 101. The manner of notification is not particularly limited. For example, the notification unit 102 may notify the target members via a terminal device used by each member, or may notify the target members via a display device (e.g., a television) shared by each member. Note that the "members" to which the notification is to be sent are members other than the target member. All members other than the target member may be the notification destinations, or some of them may be the notification destinations.

[0018] As described above, the information processing device 1 includes a target detection unit 101 that detects target members of the group who need care from other members based on the state of the group members estimated from an image of the group members, and a notification unit 102 that notifies each member of the group of the detected target members. According to this configuration, it is possible to notify each member of the target members who need care from other members, and to encourage mutual assistance between the members. Therefore, according to the information processing device 1, it is possible to obtain an effect that it is possible to support mutual assistance between the members of the group. Alternatively, according to the information processing device 1, it is possible to optimize the manner of mutual assistance between the members of the group.

[0019] [Control program configuration] The above-mentioned functions of the information processing device 1 can also be realized by a program. The control program according to this exemplary embodiment is configured to make a computer function as a target detection means for detecting a target member of the group who needs care from other members based on the state of the group member estimated from an image of the member, and as a notification means for notifying each member of the group of the detected target member. Therefore, the control program according to this exemplary embodiment has an effect of supporting mutual assistance between the members of the group.

[0020] [Mutual monitoring method flow] The flow of the mutual monitoring method will be described with reference to Fig. 2. Fig. 2 is a flow diagram showing the flow of the mutual monitoring method. Note that the execution subject of each step in this mutual monitoring method may be a processor included in the information processing device 1, or may be a processor included in another device, or each step may be executed by a processor provided in a different device.

[0021] In S1, at least one processor executes a target detection process to detect target members of the group who require care from other members based on a state of the group member estimated from an image taken of the member.

[0022] In S2, at least one processor executes a notification process for notifying each member of the group of the target member detected in S1.

[0023] As described above, the mutual monitoring method employs a configuration in which at least one processor executes a target detection process for detecting a target member who needs care from other members of the group based on the state of the group member estimated from an image of the group member, and a notification process for notifying each member of the group of the detected target member. Therefore, the mutual monitoring method has the effect of supporting mutual assistance between the members of the group.

[0024] Second Exemplary Embodiment A second exemplary embodiment, which is an example of an embodiment of the present invention, will be described in detail with reference to the drawings. Components having the same functions as those described in the above exemplary embodiment will be given the same reference numerals, and their description will be omitted as appropriate. The scope of application of each technical means adopted in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technical means adopted in this exemplary embodiment can be adopted in other exemplary embodiments included in this disclosure, as long as no particular technical hindrance occurs. In addition, each technical means shown in each drawing referred to for explaining this exemplary embodiment can be adopted in other exemplary embodiments included in this disclosure, as long as no particular technical hindrance occurs.

[0025] (Configuration of Mutual Monitoring System 3) FIG. 3 is a diagram showing a configuration example of a mutual monitoring system 3. As shown in the figure, the mutual monitoring system 3 includes an information processing device 1A and terminal devices 2a to 2f. The mutual monitoring system 3 is a system that supports mutual monitoring among members of a group. For example, as shown in the figure, when a group is formed by users of terminal devices 2a to 2f, the information processing device 1A supports mutual monitoring among the users. In the following, when there is no need to distinguish between the terminal devices 2a to 2f, they are simply referred to as "terminal devices 2". Also, while FIG. 3 shows six terminal devices 2, the mutual monitoring system 3 may include at least two terminal devices 2.

[0026] The information processing device 1A is a device that supports mutual monitoring among group members. Similar to the above-mentioned information processing device 1, the information processing device 1A detects target members of the group who need care from other members based on the state of the group members estimated from an image of the group members. Then, the information processing device 1A notifies each member of the group of the detected target members.

[0027] The terminal device 2 is a device used by each user of the mutual monitoring system 3 to use the mutual monitoring system 3. For example, the terminal device 2 notifies its own user (i.e., a member of the same group as the target member) of the notified target member by displaying information indicating the target member notified from the information processing device 1A. FIG. 3 shows an example in which the terminal device 2 is a smartphone. However, the terminal device 2 is not limited to a smartphone as long as it can realize functions such as receiving user input, transmitting the received input to the information processing device 1A, and presenting various information notified from the information processing device 1A. In addition, the terminal device 2 may be a portable device or a stationary device.

[0028] As described above, the mutual monitoring system 3 includes an information processing device 1A that detects target members of a group who need care from other members based on the state of the group members estimated from an image of the group members and notifies each member of the group of the detected target members, and a terminal device 2 that presents the target members to the member using the terminal device 2 based on the notification from the information processing device 1A. This provides the effect of supporting mutual assistance between the members of the group.

[0029] Furthermore, in the mutual monitoring system 3, fixed roles are not assigned to group members, so it is easy to maintain a relationship of mutual support even if the number of members increases or decreases. However, since it is conceivable that some members do not require care from other members, it is also possible to allow each member to set whether or not to receive notifications. Furthermore, since the target members are presented to each member of the group, transparent operation can be achieved.

[0030] It should be noted that the members that make up a group can be determined arbitrarily. For example, the mutual monitoring system 3 can support mutual assistance between members of a group made up of family units, and can also support mutual assistance between members of groups made up of various members, such as colleagues at work, sports teams, and various hobby clubs.

[0031] [Configuration of information processing device 1A] The configuration of the information processing device 1A will be described with reference to Fig. 4. Fig. 4 is a block diagram showing an example of the configuration of the information processing device 1A. As shown in the figure, the information processing device 1A includes a control unit 10A that controls each unit of the information processing device 1A and a storage unit 11A that stores various data used by the information processing device 1A. The information processing device 1A also includes a communication unit 12A that allows the information processing device 1A to communicate with other devices, an input unit 13A that accepts input of various data to the information processing device 1A, and an output unit 14A that allows the information processing device 1A to output various data. The control unit 10A also includes a target detection unit 101A, a notification unit 102A, a data acquisition unit 103A, a state estimation unit 104A, a priority setting unit 105A, an execution detection unit 106A, a deviation detection unit 107A, a trader detection unit 108A, and an assistant control unit 109A.

[0032] 1, the target detection unit 101A detects target members of the group who need care from other members based on the state of the group member estimated from an image of the group member. More specifically, the target detection unit 101A detects a member as a target member when the state estimated for the member by the state estimation unit 104A described later satisfies a predetermined condition.

[0033] The predetermined condition may be determined in advance according to the condition estimated by the condition estimation unit 104A. Alternatively, the predetermined condition may be determined for each member. For example, if a group includes a member with high blood pressure, the condition estimation unit 104A may estimate the blood pressure of the member, and the predetermined condition may be that the blood pressure falls outside the normal range. In this way, when the estimated blood pressure of the member falls outside the normal range, the other members can be notified of this and be prompted to take care.

[0034] 1, the notification unit 102A notifies each member in the same group as the target member of the target member detected by the target detection unit 101A. Notification by the notification unit 102A will be described later with reference to FIG. 6 etc.

[0035] The data acquisition unit 103A acquires information necessary for estimating the state of the members. Specifically, the data acquisition unit 103A acquires images of the members of the group. For example, the data acquisition unit 103A may acquire images for a video call that are taken while the members are making a video call using the terminal device 2 or the like.

[0036] The state estimation unit 104A uses an image acquired by the data acquisition unit 103A to estimate the state of the member depicted in the image. The state to be estimated may be a state that can be estimated from the image and may be used as a criterion for determining whether or not care is required by other members. For example, the state estimation unit 104A may estimate at least one of stress level, concentration level, cognitive function, emotion, arousal level, and tension level. In addition, when estimating these states, the state estimation unit 104A may detect at least one of a change in facial color, facial movement, gaze, blinking, pupil diameter, iris movement, and facial expression. It is also possible to apply well-known techniques as a method for estimating these states from an image.

[0037] Furthermore, the state estimation unit 104A may determine whether or not a member has a disease or a symptom thereof based on the above estimation result. For example, the state estimation unit 104A may determine whether or not a member has at least one symptom of dementia, mental illness, high blood pressure, heart disease, diabetes, and cancer. When determining whether or not a member has a disease, it is preferable to notify not only the group members but also medical professionals such as doctors.

[0038] The state estimation unit 104A may also use information other than images to estimate the member's state. For example, if the member uses a wearable device, the state estimation unit 104A may acquire various vital data and activity data measured by the wearable device and use the data for state estimation. The state estimation unit 104A may also estimate the state by taking into account the member's medical history, chronic illnesses, health check results, and the like.

[0039] The priority setting unit 105A sets a priority when the notification unit 102A notifies the target member. This priority is a priority for taking care of the target member. In other words, the notification unit 102A notifies the target member so that a member with a high priority set by the priority setting unit 105A takes care of the target member first. For example, the notification unit 102A may notify each member in descending order of priority. Also, for example, the notification unit 102A may notify the members including a message according to the priority. For example, the notification unit 102A may notify a message including information indicating the priority, or may notify the members with a high priority including a message strongly recommending them to take care of the target member.

[0040] In addition, when it is not possible to confirm that the target member has been cared for even after a predetermined time has elapsed after notifying each member of the target member, the notification unit 102A may notify each member of a reminder. The notification unit 102A may determine whether or not to notify the target member or the frequency of the reminder notification according to the priority set by the priority setting unit 105A.

[0041] The method of setting the priority is not particularly limited. For example, the priority setting unit 105A may set the priority designated by the member via the communication unit 12A or the input unit 13A. Also, for example, the priority setting unit 105A may obtain information indicating the current status of each member (e.g., during working hours, outside working hours, etc.) and set the priority according to the status indicated by the information. This enables the notification unit 102A to prioritize notifications to members outside working hours over notifications to members during working hours.

[0042] Also, for example, the priority setting unit 105A may set priorities based on the history of care provided by each member in the past. For example, the priority setting unit 105A may set the priority of a member who has provided care relatively few times or frequency in the past higher than the priority of a member who has provided care relatively many times. This makes it possible to avoid the burden of care being concentrated on a specific member by preferentially notifying a member who has provided care relatively few times or frequency in the past. Also, for example, the priority setting unit 105A may increase the possibility of providing effective care by preferentially notifying a member who has provided effective care in the past. Note that if the condition of the target member who was provided care by a certain member improves after the care is provided by that member, it is inferred that the care by that member was effective. For this reason, the priority setting unit 105A can identify the member who provided effective care by referring to the care history and the condition of the target member after the care.

[0043] In addition, for example, the priority setting unit 105A may set priorities based on the location information of each member, the relationships between members, the state estimated by the state estimation unit 104A, and the characteristics of each member (qualifications, age, sex, special skills, etc.), etc. This allows notifications to be given priority to members who are close to the target member, members who are close to the target member, members who can provide care according to the target member's state, etc.

[0044] In this way, the information processing device 1A may include a priority setting unit 105A that sets a priority for caring for the target member for each of the members other than the target member. Then, the notification unit 102A may notify each member based on the priority. In this way, in addition to the effect of the information processing device 1, it is possible to obtain an effect that it is possible to provide an appropriate notification that takes into account the differences between members.

[0045] The execution detection unit 106A detects that a member of the group has taken care of the target member. The method of detecting that care has been taken is not particularly limited. For example, the execution detection unit 106A may detect that care has been taken when a notification that care has been taken is received from the member who has taken care or the target member who has been taken care of. In addition, for example, care may be taken in the form of sending a message to the target member using the terminal device 2, or making a video call or voice call with the target member using the terminal device 2. In such a case, the execution detection unit 106A may detect that care has been taken when a message is sent to the target member or a call with the target member is made.

[0046] In this way, the information processing device 1A may include an execution detection unit 106A that detects that a member of the group has taken care of the target member. The notification unit 102A may end notification regarding the target member on the condition that it is detected that the target member has been taken care of. In this way, in addition to the effect of the information processing device 1, it is possible to obtain an effect that each member can be informed that the target member has been taken care of and that other members no longer need to take care of the target member.

[0047] The deviation detection unit 107A detects that there is a deviation between the result of the member's self-assessment of the member's state and the state of the member estimated by the state estimation unit 104A. The result of the member's self-assessment may be input by the member via, for example, the terminal device 2. In addition, it may be determined in advance what relationship between the result of the self-assessment and the estimated state is to determine that there is a deviation between them. For example, when the state estimation unit 104A evaluates the stress level on a five-point scale from 1 to 5, the member may also be made to self-assess the stress level on the same five-point scale. In this case, the deviation detection unit 107A may detect that there is a deviation between them when, for example, there is a difference between the stages of the stress level.

[0048] The provider detection unit 108A detects a provider related to the care of the target member from among the providers included in the group members. Note that the term "provider" here includes natural persons and corporations that conduct business in general. In this way, the mutual monitoring system 3 may allow a provider to be registered as a member of the group. For example, by registering a caregiver dispatching company, a hospital, a doctor, a medical professional, or an insurance company as a member of the group, monitoring including these providers or their employees can be realized.

[0049] The service provider detection unit 108A may detect a service provider related to the care of the target member based on at least one of the target member and the condition of the target member. For example, when the target member is elderly, the service provider detection unit 108A may detect a service provider related to care from among the service providers registered as members of the group. Also, for example, when the condition of the target member requires treatment, the service provider detection unit 108A may detect a service provider related to medical care from among the service providers registered as members of the group.

[0050] The assistant control unit 109A is for assisting the use of the mutual monitoring system 3. More specifically, the assistant control unit 109A controls the operation of a virtual assistant (hereinafter referred to as a virtual assistant) that interacts with the members who use the mutual monitoring system 3, thereby assisting the members in using the mutual monitoring system 3.

[0051] For example, the assistant control unit 109A may assist the member by inputting a query to a generation AI (Artificial Intelligence) to generate a message and presenting the generated message to the member by displaying it on the terminal device 2, for example. The generation AI used may be, for example, a language model that has been machine-learned to learn the arrangement of components (such as words) in a sentence or the arrangement of sentences in a piece of writing. By using such a language model, the assistant control unit 109A can generate a message to be presented to the member or generate an answer to a question input by the member.

[0052] [Configuration of terminal device 2] The configuration of the terminal device 2 will be described with reference to FIG. 5. FIG. 5 is a block diagram showing an example of the configuration of the terminal device 2. As shown in the figure, the terminal device 2 includes a control unit 20 that controls each unit of the terminal device 2, and a storage unit 21 that stores various data used by the terminal device 2. The terminal device 2 also includes a communication unit 22 that allows the terminal device 2 to communicate with other devices, an input unit 23 that accepts input of various data to the terminal device 2, a display unit 24 that displays images, and a photographing unit 25 that photographs images (still images or moving images). The control unit 20 also includes a reception unit 201, a presentation unit 202, an execution detection unit 203, a call control unit 204, and a messaging control unit 205. For example, a control program for causing a computer to function as each of these units can be installed in a general-purpose computer to cause the computer to function as the terminal device 2.

[0053] The reception unit 201 receives various inputs related to the mutual monitoring system 3. For example, the reception unit 201 receives an input from a member using the terminal device 2 and notifies the information processing device 1A of the contents of the input. In addition, for example, the reception unit 201 receives various notifications (e.g., notifications of target members) from the information processing device 1A.

[0054] The presentation unit 202 presents various information related to the mutual monitoring system 3. For example, when the reception unit 201 receives a notification of a target member, the presentation unit 202 may notify the member using the terminal device 2 of the reception of the notification by turning on an indicator (not shown) included in the terminal device 2 or by outputting a notification sound from an audio output unit (not shown).

[0055] Furthermore, the presentation unit 202 presents various information related to the mutual monitoring system 3 to the members using the terminal devices 2 by displaying various UI screens on the display unit 24. The UI screen may be acquired from the information processing device 1 and displayed by the presentation unit 202, or may be generated by the presentation unit 202. The UI screen will be described later with reference to Figs. 6 to 9.

[0056] The execution detection unit 203 detects that the target member has been cared for by a member using the terminal device 2. When the execution detection unit 203 detects that the target member has been cared for, it notifies the information processing device 1A of that fact. The detection method is not particularly limited. For example, the execution detection unit 203 may detect that the target member has been cared for when a call with the target member via the call control unit 204 described below, or a message to the target member via the messaging control unit 205 described below, has been transmitted. In addition to this, for example, the execution detection unit 203 may detect that the target member has been cared for when the reception unit 201 receives an input operation indicating that the target member has been cared for.

[0057] The call control unit 204 performs control for making a video call between members. More specifically, the call control unit 204 makes a video call to a member in response to an input received by the reception unit 201. Furthermore, the call control unit 204 performs a process for ending the video call and a process for switching the audio input and output to a speaker mode (a mode for hands-free calling) in response to an input received by the reception unit 201. The call control unit 204 can also perform a call using only audio without displaying video.

[0058] The messaging control unit 205 performs control for exchanging messages between members. More specifically, the messaging control unit 205 performs a process of displaying each message previously transmitted and received between members on a timeline, a process of accepting input of a new message, a process of adding an input message to the timeline, and the like. Note that messages are not limited to those displayed on the timeline. For example, the messaging control unit 205 may notify each member of a message addressed to that member.

[0059] [UI (User Interface) screen example: Notification of target members] An example of a UI screen displayed on the terminal device 2 in the mutual monitoring system 3 will be described with reference to Fig. 6. Fig. 6 is a diagram showing an example of a UI screen before and after notifying a target member. Specifically, Img1 shown in Fig. 6 is an example of a UI screen when a target member has not been notified, and Img2 is an example of a UI screen when a target member has been notified.

[0060] A member who uses the mutual monitoring system 3 logs in to the mutual monitoring system 3, for example, by inputting his / her own ID (IDentification: identification information) into the terminal device 2. The presentation unit 202 identifies the member from the ID and displays a UI screen for the member on the display unit 24. Img1 and 2 are examples of UI screens for individual members that are displayed in this manner. The process of generating Img1 and 2 may be performed by the information processing device 1A or the terminal device 2. In the following, a member who views a UI screen after logging in as described above is referred to as a viewing member.

[0061] Img1 displays eight icons i1 to i8. Among these, i8 is an icon of a viewing member. Also, i7 is an icon indicating the above-mentioned virtual assistant. And, i2 to i6 are icons of members belonging to the same group as the member. More specifically, among the icons i2 to i6, the icons i2 to i5 indicate people registered as members, and the icon i6 indicates a business registered as a member. While the icons i2 to i5 and i8 all display images of members, the icon i6 displays a mark indicating a business. In this way, the presentation unit 202 of the terminal device 2 may display the business icon and the person icon in a distinguishable manner.

[0062] In Img1, the icons i2 to i6 of each member are displayed above the line L1. The icon i8 of the viewing member and the message m1 are displayed below the line L2. The message m1 is a message for the viewing member generated by the generation AI and is displayed as a message from the virtual assistant.

[0063] In addition, in Img1, icon i7 is placed between icon i1 and icons i2 to 6. This allows members to recognize that the virtual assistant represented by icon i7 is an intermediary between the viewing members and each member of the group represented by icons i2 to 6.

[0064] Also displayed in Img1 are a button a1 for displaying the top screen and a button a2 for starting a self-check. Note that the top screen here is the Img1 screen on which the icons of each member are displayed in a list.

[0065] A self-check means that a browsing member has his / her own state judged by the information processing device 1A. When the reception unit 201 receives an operation on the button a2, it starts the photographing unit 25 to photograph the member and transmits the photographed image to the information processing device 1A. Then, in the information processing device 1A that receives the image, the data acquisition unit 103A acquires the image, and the state estimation unit 104A analyzes the image to estimate the state of the browsing member. The estimation result is notified to the terminal device 2, and the presentation unit 202 of the terminal device 2 presents the notified estimation result to the browsing member by, for example, displaying it on the display unit 24.

[0066] 6, the display position of icon i4 among member icons i2 to 6 has been changed to the area between lines L1 and L2. Icon i4 indicates a target member detected by target detection unit 101A of information processing device 1A. Such a change in display position is made based on a notification from notification unit 102A.

[0067] In this way, the presenting unit 202 may present the target member to the viewing members by changing the display position of the icon i4 of the target member from the area above the line L1, which is the original display position, to a position within a predetermined display area between the lines L1 and L2, based on the notification from the notification unit 102A. This allows the viewing members to intuitively recognize that the target member needs care.

[0068] Also, the badge a3 is displayed in association with the icon i4. This allows the browsing member to recognize that there is information to be confirmed regarding the target member corresponding to the icon i4. For example, the presentation unit 202 may terminate the display of the badge a3 in response to the browsing member confirming the report regarding the status of the target member.

[0069] Furthermore, Img2 displays a message m2 explaining the state of the target member (Mr. B). The message m2 may be generated by inputting the name of the target member into a template prepared in advance for each state. Alternatively, the message m2 may be generated by the generation AI generated by the assistant control unit 109A. In other words, the notification unit 102A may notify a message explaining the state of the target member, which is generated using the generation AI, i.e., a machine-learned language model.

[0070] Also, in Img2, a button a4 is displayed for displaying a report analyzing the state of the target member. When the browsing member performs an input operation on the button a5, the presentation unit 202 displays the report on the display unit 24. The contents of the report will be described with reference to FIG. 7.

[0071] Also, in Img2, a button a5 for making a video call with the target member is displayed. When the browsing member performs an input operation on the button a5, the call control unit 204 makes a video call to the target member. This allows the browsing member to smoothly start a video call with the target member.

[0072] As described above, the notification unit 102A may change the display position of the icon of the target member among the icons corresponding to the members displayed on the display screen viewable by each member of the group to a position within a predetermined display area set on the display screen. This provides an effect that each member can easily recognize that a target member has appeared among the members of the group, in addition to the effect provided by the information processing device 1.

[0073] As described above, the notification unit 102A may notify a message explaining the state of the target member, which is generated using a machine-learned language model. This provides an effect that the state of the target member can be easily conveyed to other members, in addition to the effect provided by the information processing device 1.

[0074] The above-mentioned predetermined display area can be any area. For example, as in the UI screen example shown in Fig. 6, the above-mentioned predetermined display area may be set between the area displaying the icons of the members and the area displaying the icons of the browsing members. This gives the impression that the target member is requesting care from the browsing members, and encourages the browsing members to provide care.

[0075] The notification unit 102A may generate a UI screen in which the display position of the icon of the target member is changed, and transmit the generated UI screen to the terminal device 2, thereby causing the UI screen to be displayed on the terminal device 2. The UI screen may also be generated by the presentation unit 202 of the terminal device 2. In this case, in accordance with the notification from the notification unit 102A, the presentation unit 202 changes the display position of the icon of the target member, among the icons corresponding to each member to be displayed on a display screen viewable by each member of the group, to a position within a predetermined display area set on the display screen.

[0076] When the execution detection unit 106A detects that a member of the group has taken care of the target member, the notification unit 102A ends the notification regarding the target member. For example, when the notification unit 102A detects that the target member has been taken care of while the UI screen is being displayed, the notification unit 102A may generate a new UI screen and transmit the generated UI screen to the terminal device 2 to update the UI screen. Here, the new UI screen is a UI screen in which the display position of the icon of the target member has returned to the display area that displays the icons of each member (the area above the line L1 in the example of FIG. 6).

[0077] Furthermore, for a member who was not displaying a UI screen when it was detected that the target member had been cared for, the notification unit 102A displays a UI screen in which the display position of the icon of the target member has returned to the display area displaying the icons of each member when the UI screen is displayed next time or later. Note that such updating and generation of the UI screen may also be performed by the presentation unit 202 of the terminal device 2.

[0078] [UI screen example: Report, data, video call] 7 is a diagram showing examples of UI screens when a report is presented, when data is presented, and when a video call is made. Specifically, Img3 shown in FIG. 7 is an example of a UI screen that presents a report, Img4 is an example of a UI screen that presents data that supports the report, and Img5 is an example of a UI screen during a video call.

[0079] Img3 shows a report on the status of B, a target member. Img3 displays B's icon i4, as well as a message m3 explaining B's status, a message m4 explaining precautions regarding B's status, and objects b2 and b3 showing data on which B was detected as a target member. Messages m3 and m4 may be standard messages according to the status of the target member, or may be messages generated by the generation AI by the assistant control unit 109A.

[0080] Img3 also displays the date the report was created, information b1 indicating the members who contacted Mr. B, a button b4 for displaying past reports, a button b5 for making a video call with Mr. B, and a button b6 for sending a message to Mr. B.

[0081] When button b4 is operated, the presentation unit 202 displays past reports. When button b5 is operated, the call control unit 204 makes a video call to the target member. When button b6 is operated, the messaging control unit 205 accepts input of a message to the target member by, for example, displaying an input acceptance screen for a message to the target member.

[0082] Img4 shows an example of a UI screen presenting data supporting the report of Img3. Specifically, Img4 displays a graph showing a time series change of "mood" shown in object b2 of Img3 and a graph showing a time series change of "activity amount" shown in object b3 of Img3. "Mood" and "activity amount" are estimated by the state estimation unit 104A. By storing the estimation result of the state estimation unit 104A in the storage unit 11A or the like, the notification unit 102A can generate a UI screen including such a graph using the stored estimation result. Note that the notification unit 102A may notify the terminal device 2 of the estimation result, and the presentation unit 202 of the terminal device 2 may generate a UI screen including a graph based on the notified estimation result.

[0083] Img5 shows an example of a UI screen during a video call. Img5 displays an image b10 of a browsing member, an image b11 of the call partner (in this example, the target member), information b12 to b14 showing the result of state estimation by the state estimation unit 104A, a mute button b15 for muting speech from the browsing member, a call end button b16 for ending the video call, and a switch button b17 for switching audio input and output to speaker mode.

[0084] Here, the data acquisition unit 103A may acquire time-series images (or video images) taken during the video call for the video call. Then, the state estimation unit 104A may estimate the state of the member from each acquired image (or each frame image extracted from the video image). This allows the notification unit 102A to update the information b12 to b14 to be displayed on the UI screen in real time.

[0085] As described above, the state estimation unit 104A may estimate the state of a member from an image taken during a video call by the member for the video call. This eliminates the need for the member to take an image for state estimation. In addition, for example, it is possible to record the transition of the state by simply having the members periodically make video calls.

[0086] As described above, the state estimation unit 104A may repeatedly perform state estimation during a video call to generate a time-series state estimation result. In this case, the target detection unit 101A may determine whether the member corresponds to a target member by using a representative value of the time-series estimation result by the state estimation unit 104A. Examples of the representative value include an average value, a maximum value, a minimum value, a median value, and a mode value.

[0087] Here, the state estimation result, etc. are personal information of the members. Therefore, each member may be allowed to set the disclosure range and disclosure target. In this case, the notification unit 102A or the presentation unit 202 notifies or presents the state estimation result, etc. within the set range.

[0088] [UI screen example: vendor presentation] Fig. 8 is a diagram showing an example of a UI screen presenting a provider for caring for a target member. Img6 shown in Fig. 8 is displayed after a viewing member (member indicated by icon i8) presented with a UI screen such as Img2 in Fig. 6 has cared for the target member (member indicated by icon i4).

[0089] Img6 differs from Img2 in FIG. 6 in that the display mode of icon i6 has changed, badge a3 that was displayed in association with icon i4 has disappeared, the message has changed from m2 to m5, and button c1 is displayed in place of buttons a4 and a5.

[0090] Icon i6 is an icon indicating a provider related to the care of the target member, that is, a provider detected by the provider detection unit 108A. In Img6, the provider corresponding to icon i6 is an insurance company called "C Insurance", and the target member B is a policyholder of the insurance of this insurance company. As in the example of FIG. 8, the notification unit 102A may notify the viewing member of the provider by changing the display mode of the icon indicating the provider detected by the provider detection unit 108A. Note that how the display mode of the icon is changed is arbitrary.

[0091] Message m5 is a message about a provider detected by provider detection unit 108A. Specifically, message m5 is a message notifying target member B that there is a service available through a special clause in the insurance contract he has, and urging him to contact the person in charge of the insurance (D). In this way, notification unit 102A may also notify viewing members of information about the provider detected by provider detection unit 108A (particularly information useful for caring for the target member). In this way, it is possible to support viewing members in making decisions about dealing with B early.

[0092] The button c1 is a button for sending a message to the agent (more precisely, the agent's person in charge) detected by the agent detection unit 108A. This allows the browsing member to smoothly contact the agent. The notification unit 102A may display a button for starting a call (which may be a video call or a voice-only call) with the agent or the agent in charge of the agent.

[0093] As described above, the members of the group may include a provider. The information processing device 1A may also include a provider detection unit 108A that detects a provider related to the care of the target member from among the providers included in the members. The notification unit 102A may then notify the member who has provided care for the target member of the detected provider. This provides the effect of facilitating the use of a provider to care for the target member, in addition to the effect provided by the information processing device 1.

[0094] [UI screen example: Notification regarding discrepancy with self-assessment] As described above, the deviation detection unit 107A detects a deviation between the result of the member's self-evaluation of the member's state and the state of the member estimated by the state estimation unit 104A. In addition, the notification unit 102A issues a notification to the member for whom a deviation has been detected, to call attention to the deviation. A specific example of this notification will be described with reference to FIG.

[0095] Fig. 9 is a diagram showing an example of a UI screen for notifying a discrepancy between the result of self-assessment of a state and the result of estimation of a state. More specifically, Img8 in Fig. 9 shows an example of a UI screen for notifying a discrepancy. Img7 is an example of a UI screen for self-assessment of a state, and Img9 is an example of a UI screen for presenting a report of the result of estimation of a state.

[0096] In Img7, a message m6 asking about the viewing member's status is displayed, along with status candidates d1. The candidates d1 include four options ranging from "bad" to "good." The viewing member can input the result of self-assessment by selecting the option from the options d1 that is closest to the status he or she recognizes. In this way, the notification unit 102A may present a plurality of status candidates to the viewing member. In this case, the deviation detection unit 107A can identify the result of the self-assessment by the viewing member from the selection result of the viewing member. In the example of Img7, the option "good" is highlighted, which indicates that the option was selected, that is, the viewing member's self-assessment of the status was good.

[0097] The message m6 may be a standard message or may be a message generated by the assistant control unit 109A using a language model. After the self-evaluation, the notification unit 102A may prompt the browsing member to take a picture of himself, the data acquisition unit 103A may acquire the captured image, and the state estimation unit 104A may estimate the state of the browsing member using the acquired image. This allows the deviation detection unit 107A to detect deviations using the estimation result of the state at the time of self-evaluation. When deviations are detected, the notification unit 102A may alert the browsing member by displaying a UI screen such as Img8, for example.

[0098] Img8 displays a message m7 indicating that there is a discrepancy between the result of self-assessment of the state and the result of analysis of the state, i.e., the result of estimation by the state estimation unit 104A. The message m7 may be a standard message, or may be a message generated by the language model by the assistant control unit 109A. In the latter case, the assistant control unit 109A may generate a query instructing the creation of a message informing that there is a discrepancy between the self-assessment and the analysis result, and input the query to the language model.

[0099] Also, in Img8, a button d2 for displaying a report of the state estimation results is displayed. When a browsing member performs an input operation on button d2, a UI screen such as that shown in Img9 is displayed.

[0100] Img9 shows a report on the status of A, a viewing member. Img9 displays a message m8 explaining A's status and a message m9 explaining precautions regarding A's status. These messages may also be prepared in advance and displayed according to the status or the degree of deviation, or the assistant control unit 109A may generate them in the language model.

[0101] Also displayed in Img9 are objects d3 and d4 indicating the data on which A was detected as a target member. Also displayed in Img9 are a button d5 for displaying past reports, a button d6 for consulting with a specific consultee, a button d7 for making a video call with the specific consultee, and a button d8 for sending a message to the specific consultee.

[0102] The predetermined consultation destination may be a consultation destination determined in advance, or may be a consultation destination determined according to the browsing member and its status. For example, the notification unit 102A may present, as a consultation destination, a member who has frequently had a dialogue (a call or a message exchange) with the browsing member among the members of the group. Also, for example, the notification unit 102A may present, as a consultation destination, a business related to the browsing member's status (e.g., a family doctor, etc.) among businesses registered as members of the group.

[0103] As described above, the information processing device 1A includes a deviation detection unit 107A that detects a deviation between the result of the member's self-evaluation of the state of the member and the state of the member estimated from the image. Then, the notification unit 102A issues a notification to the member for whom a deviation is detected to call attention. In this way, in addition to the effect of the information processing device 1, an effect is obtained in which a member who may be in such a dangerous state that he or she cannot make a proper self-evaluation can be made aware of the state. The notification unit 102A may also notify each member of the member for whom a deviation is detected as a target member, so that the entire group can take care of a member who may be in such a dangerous state that he or she cannot make a proper self-evaluation.

[0104] [Processing flow: information processing device 1A] The flow of the process executed by the information processing device 1A will be described with reference to Fig. 10. Fig. 10 is a flow diagram showing an example of the process executed by the information processing device 1A. The flow of Fig. 10 includes a mutual monitoring method of this exemplary embodiment.

[0105] In S11, the data acquisition unit 103A acquires an image of the member. For example, the data acquisition unit 103A may acquire an image taken by the terminal device 2 when the member makes a video call, or may acquire an image taken by the terminal device 2 when the member performs a self-check. Furthermore, the data acquisition unit 103A acquires the member's ID in addition to the member's image. The data acquisition unit 103A may also acquire data that can be used to estimate the member's condition, such as the member's vital data.

[0106] In S12, the state estimation unit 104A estimates the state of the member appearing in the image acquired in S11 using the image. If data other than the image is also acquired in S11, the state estimation unit 104A estimates the state of the member by using the data.

[0107] In S13, the target detection unit 101A judges whether or not the member whose condition is to be estimated needs care based on the estimation result in S12. As described above, the condition that is judged to need care is specified in advance. If the result of S13 is NO, the process of FIG. 10 ends. On the other hand, if the result of S13 is YES, the process proceeds to S14.

[0108] In S14, the target detection unit 101A detects the members appearing in the image acquired in S11 as target members. In this way, the target detection unit 101A executes a target detection process to detect target members who need care from other members of the group, based on the state of the group members estimated from the image capturing the members.

[0109] In S15, priority setting unit 105A sets a priority for caring for each member in the same group as the target member detected in S14. The method for setting the priority is as described above, and therefore the description will not be repeated here.

[0110] In S16, the notification unit 102A notifies each member of the same group as the target member of the target member detected in S14, according to the priority set in S15. In this way, the notification unit 102A executes a notification process to notify each member of the group of the target member detected by the target detection unit 101A. The notification may include the ID of the target member. Note that, as described above, the notification may be performed in the order of the priority set by the priority setting unit 105A.

[0111] In S17, the execution detection unit 106A judges whether or not the target member has been cared for by a member of the group. If the result of S17 is NO, the process of S17 is performed again after a predetermined time. On the other hand, if the result of S17 is YES, the process proceeds to S18. Note that when the number of times the process of S17 is repeated reaches a predetermined upper limit, the process of FIG. 10 may be terminated, or the notification unit 102A may re-notify.

[0112] In S18, the company detection unit 108A detects a company related to the care of the target member from among the companies included as members in the same group as the target member. In S19, the notification unit 102A notifies the member who performed the care of the target member of the company detected in S18. Note that the notification unit 102A may also notify other members of the company. In addition, if a company related to the care of the target member is not registered, the processes of S18 and S19 are omitted.

[0113] In S20, the notification unit 102A ends the notification regarding the target member detected in S14, and the process in Fig. 10 ends. For example, if the notification unit 102A has generated and displayed a UI screen such as Img2 in Fig. 6, the notification unit 102A updates this UI screen to a UI screen such as Img1. On the other hand, if the presentation unit 202 of the terminal device 2 generates such a UI screen, the notification unit 102A transmits the ID of the target member to the terminal device 2 and instructs it to change the display position of the icon of the member identified by the ID.

[0114] [Processing flow: Terminal device 2] The flow of the process executed by the terminal device 2 will be described with reference to Fig. 11. Fig. 11 is a flow diagram showing an example of the process executed by the terminal device 2.

[0115] In S21, the reception unit 201 judges whether or not a notification indicating that a target member has been generated has been received. This notification is sent from the information processing device 1A in S16 of Fig. 10. If the judgment in S21 is YES, the process proceeds to S22, and if the judgment in S21 is NO, the judgment in S21 is performed again after a predetermined time.

[0116] In S22, the presentation unit 202 presents the target member to the member using the terminal device 2 based on the notification received in S21. The presentation may be in any manner. For example, if the member using the terminal device 2 is viewing the UI screen of the mutual monitoring system 3 (i.e., if the member is a viewing member), the presentation unit 202 may present the target member by changing the display position of the icon of the target member (see Img2 in FIG. 6).

[0117] Furthermore, when the notification is received while the UI screen is not displayed, the presenting unit 202 may first prompt the user to check the UI screen, and then display a UI screen such as Img2 in response to an operation to display the UI screen to present the target member. The method of prompting the user to check the UI screen is not particularly limited. For example, the presenting unit 202 may prompt the user to check the UI screen by generating at least one of light, sound, and vibration from the terminal device 2, or by displaying a message or object on the display unit 24.

[0118] In S23, the execution detection unit 203 judges whether or not care has been performed by the browsing member viewing the above-mentioned UI screen. For example, when a UI screen such as Img2 in FIG. 6 is presented to the browsing member and an operation to select button a5 is performed, a video call is made between the browsing member and the target member via the call control unit 204. The execution detection unit 203 may judge that care has been performed by the browsing member when such a call is made or when a message is sent from the browsing member to the target member via the messaging control unit 205. If NO is judged in S23, the process returns to S21. On the other hand, if YES is judged in S23, the process proceeds to S24.

[0119] In S24, the execution detection unit 203 notifies the information processing device 1A that the viewing member has performed care for the target member. This notification may include the ID of the viewing member and the ID of the target member. In the information processing device 1A that receives this notification, YES is determined in S17 of Fig. 10, and the processes of S18 and S19 are performed. As a result, the terminal device 2 is notified of the provider related to the care for the target member.

[0120] In S25, the reception unit 201 receives the notification indicating the provider for the care of the target member. If the information processing device 1A does not detect the provider, the processes of S25 and S26 are not performed, and the illustrated process ends.

[0121] In S26, the presentation unit 202 presents the business indicated in the notification received in S25 to the viewing members. This ends the processing in Fig. 11. For example, the presentation unit 202 may present the business by highlighting the icon of the business on the UI screen, as in the example of Img6 in Fig. 8. The presentation unit 202 may also display messages related to the business, buttons for contacting the business or the person in charge at the business, and the like on the UI screen.

[0122] [Modifications] The execution subject of each process described in each of the above exemplary embodiments is arbitrary and is not limited to the above examples. In other words, as long as each process described in each of the above exemplary embodiments can be executed, the devices constituting the mutual monitoring system 3 can be appropriately changed. For example, some of the functions of the information processing device 1A may be provided in another server, or some of the functions of the information processing device 1A may be provided in the terminal device 2. In the flow diagrams shown in Figs. 10 and 11, the processing of each step may be executed by one device (which may also be referred to as a processor) or by multiple devices (which may also be referred to as processors). In other words, the execution subject of the processes shown in Figs. 10 and 11 may be one processor or multiple processors.

[0123] [Software implementation example] Some or all of the functions of the information processing devices 1, 1A and the terminal device 2 may be realized by hardware such as an integrated circuit (IC chip), or may be realized by software.

[0124] In the latter case, the information processing device 1, 1A or the terminal device 2 is realized by, for example, a computer that executes instructions of a program, which is software that realizes each function. An example of such a computer (hereinafter, referred to as computer C) is shown in Fig. 12. Fig. 12 is a block diagram showing the hardware configuration of the computer C that functions as the information processing device 1, 1A or the terminal device 2.

[0125] The computer C includes at least one processor C1 and at least one memory C2. The memory C2 stores a program P (control program) for operating the computer C as the information processing device 1, 1A, or the terminal device 2. In the computer C, the processor C1 reads and executes the program P from the memory C2, thereby realizing each function of the information processing device 1, 1A, or the terminal device 2.

[0126] The processor C1 may be, for example, a central processing unit (CPU), a graphic processing unit (GPU), a digital signal processor (DSP), a micro processing unit (MPU), a floating point number processing unit (FPU), a physics processing unit (PPU), a tensor processing unit (TPU), a quantum processor, a microcontroller, or a combination thereof. The memory C2 may be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination thereof.

[0127] The computer C may further include a RAM (Random Access Memory) for expanding the program P during execution and for temporarily storing various data. The computer C may further include a communication interface for transmitting and receiving data to and from other devices. The computer C may further include an input / output interface for connecting input / output devices such as a keyboard, a mouse, a display, and a printer.

[0128] Furthermore, the program P can be recorded on a non-transitory tangible recording medium M that can be read by the computer C. Such a recording medium M can be, for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer C can obtain the program P via such a recording medium M. Furthermore, the program P can be transmitted via a transmission medium. Such a transmission medium can be, for example, a communication network or broadcast waves. The computer C can also obtain the program P via such a transmission medium.

[0129] [Appendix A] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0130] (Appendix A1) An information processing device comprising: a target detection means for detecting a target member among the members of a group who requires care from other members based on the condition of the member estimated from an image taken of the member of the group; and a notification means for notifying each member of the group of the detected target member.

[0131] (Appendix A2) An information processing device as described in Appendix A1, comprising an execution detection means for detecting that a member of the group has taken care of the target member, and the notification means terminates notification regarding the target member on the condition that it is detected that care has been taken of the target member.

[0132] (Appendix A3) The information processing device described in Appendix A1 or A2, wherein the notification means changes the display position of the icon of the target member among the icons corresponding to each member displayed on a display screen viewable by each member of the group, to a position within a predetermined display area set on the display screen.

[0133] (Appendix A4) An information processing device according to any one of appendices A1 to A3, further comprising a deviation detection means for detecting a deviation between the member's self-evaluation of his / her condition and the member's condition estimated from the image, and the notification means issues a notification to the member for whom the deviation is detected, to warn him / her to be careful.

[0134] (Appendix A5) The members include vendors, and the information processing device is provided with a vendor detection means for detecting vendors related to the care of the target member from among the vendors included in the members, and the notification means notifies the member who provided care for the target member of the detected vendor.

[0135] (Appendix A6) The information processing device according to any one of appendices A1 to A5, further comprising: a state estimation means for estimating a state of the member from an image for the video call taken during the video call by the member.

[0136] (Appendix A7) An information processing device as described in any of Appendices A1 to A6, further comprising a priority setting means for setting a priority for caring for the target member for each of the other members, and the notification means notifies each member based on the priority.

[0137] (Appendix A8) The information processing device according to any one of appendices A1 to A7, wherein the notification means notifies a message explaining the state of the target member, the message being generated using a machine-learned language model.

[0138] (Appendix A9) A mutual monitoring system including: an information processing device according to any one of appendices A1 to A7; and a terminal device that presents the target member to a member using the terminal device based on a notification from the information processing device.

[0139] [Appendix B] (Appendix B1) A mutual monitoring method including: a target detection process in which at least one processor detects a target member among the members of a group who needs care from other members based on a condition of the member estimated from an image taken of the member; and a notification process in which the at least one processor notifies each member of the group of the detected target member.

[0140] (Appendix B2) A mutual monitoring method as described in Appendix B1, wherein the at least one processor includes an execution detection process for detecting that care has been taken of the target member by a member of the group, and in the notification process, the at least one processor terminates notification regarding the target member on the condition that it is detected that care has been taken of the target member.

[0141] (Appendix B3) The mutual monitoring method described in Appendix B1 or B2, in which, in the notification processing, the at least one processor changes the display position of the icon of the target member among the icons corresponding to each member displayed on a display screen viewable by each member of the group, to a position within a specified display area set on the display screen.

[0142] (Appendix B4) A mutual monitoring method described in any of Appendices B1 to B3, wherein the at least one processor includes a discrepancy detection process in which the member detects a discrepancy between the member's self-evaluation of his / her condition and the member's condition estimated from the image, and in the notification process, the at least one processor issues a notification to the member in respect of whom the discrepancy is detected, to warn him / her to be careful.

[0143] (Appendix B5) A mutual monitoring method described in any of Appendices B1 to B4, wherein the members include a provider, the at least one processor includes a provider detection process for detecting a provider related to the care of the target member from among the providers included in the members, and in the notification process, the at least one processor notifies the member who provided care for the target member of the detected provider.

[0144] (Appendix B6) A mutual monitoring method described in any one of Appendices B1 to B5, wherein the at least one processor includes a state estimation process for estimating the state of the member from an image for the video call taken during the video call by the member.

[0145] (Appendix B7) A mutual monitoring method described in any of Appendices B1 to B6, wherein the at least one processor includes a priority setting process for setting a priority for caring for the target member for each of the other members, and in the notification process, the at least one processor notifies each member based on the priority.

[0146] (Appendix B8) A mutual monitoring method described in any of Appendices B1 to B7, wherein the at least one processor includes a notification process for notifying a message explaining the status of the target member, the message being generated using a machine-learned language model.

[0147] [Appendix C] (Appendix C1) A control program that causes a computer to function as an information processing device, the control program causing the computer to function as a target detection means that detects target members of a group who require care from other members based on the condition of the members estimated from an image of the members of the group, and a notification means that notifies each member of the group of the detected target members.

[0148] (Appendix C2) A control program as described in Appendix C1, which causes the computer to function as an execution detection means for detecting that a member of the group has taken care of the target member, and the notification means terminates notification regarding the target member on the condition that it is detected that care has been taken of the target member.

[0149] (Appendix C3) The control program described in Appendix C1 or C2, wherein the notification means changes the display position of the icon of the target member among the icons corresponding to each member displayed on a display screen viewable by each member of the group, to a position within a specified display area set on the display screen.

[0150] (Appendix C4) A control program as described in any of appendices C1 to C3, which causes the computer to function as a deviation detection means for detecting a deviation between the member's self-evaluation of his / her condition and the member's condition estimated from the image, and the notification means issues a notification to the member for whom the deviation is detected, to warn him / her to be careful.

[0151] (Appendix C5) The members include providers, and the computer functions as a provider detection means for detecting providers related to the care of the target member from among the providers included in the members, and the notification means notifies the member who provided care for the target member of the detected provider, in a control program described in any of Appendices C1 to C4.

[0152] (Appendix C6) The control program according to any one of appendices C1 to C5, which causes the computer to function as a state estimation means for estimating a state of the member from an image for the video call taken by the member during the video call.

[0153] (Appendix C7) A control program as described in any of appendices C1 to C6, which causes the computer to function as a priority setting means for setting a priority for caring for the target member for each of the other members, and the notification means notifies each member based on the priority.

[0154] (Appendix C8) The control program according to any one of appendices C1 to C7, wherein the notification means notifies a message explaining the state of the target member, the message being generated using a machine-learned language model.

[0155] [Appendix D] (Appendix D1) An information processing device having at least one processor that executes a target detection process to detect target members of a group who require care from other members based on a condition of the members estimated from an image of the members of the group, and a notification process to notify each member of the group of the detected target members.

[0156] (Appendix D2) An information processing device as described in Appendix D1, which executes an execution detection process to detect that a member of the group has taken care of the target member, and in the notification process, the at least one processor terminates notification regarding the target member on the condition that it is detected that care has been taken of the target member.

[0157] (Appendix D3) The information processing device described in Appendix D1 or D2, wherein in the notification processing, the at least one processor changes the display position of the icon of the target member among the icons corresponding to each member displayed on a display screen viewable by each member of the group, to a position within a specified display area set on the display screen.

[0158] (Appendix D4) An information processing device described in any of Appendices D1 to D3, which executes a deviation detection process to detect a deviation between the member's self-evaluation of his / her condition and the member's condition estimated from the image, and in the notification process, the at least one processor issues a notification to the member in respect of whom the deviation is detected, to warn him / her to be careful.

[0159] (Appendix D5) The information processing device described in any of Appendices D1 to D4, wherein the members include a provider, and a provider detection process is executed to detect a provider related to the care of the target member from among the providers included in the members, and in the notification process, the at least one processor notifies the member who provided care for the target member of the detected provider.

[0160] (Appendix D6) The information processing device according to any one of appendices D1 to D5, wherein the at least one processor executes a state estimation process to estimate a state of the member from an image for the video call taken during the video call by the member.

[0161] (Appendix D7) An information processing device described in any of appendices D1 to D6, which executes a priority setting process to set a priority for caring for the target member for each of the other members, and in the notification process, the at least one processor notifies each member based on the priority.

[0162] (Appendix D8) A mutual monitoring method described in any of Appendices D1 to D7, wherein the at least one processor executes a notification process to notify a message describing the status of the target member, the message being generated using a machine-learned language model.

[0163] [Appendix E] The information processing device may further include a memory, and the memory may store a control program for causing the at least one processor to execute each of the processes.

[0164] (Appendix E1) A non-temporary recording medium having recorded thereon a control program that causes a computer to function as an information processing device, the control program causing the computer to execute a target detection process that detects target members of the group who require care from other members based on the condition of the members estimated from an image of the members of the group, and a notification process that notifies each member of the group of the detected target members. [Explanation of symbols]

[0165] 1. Information processing device 101 Object detection unit (object detection means) 102 Notification section (notification means) 1A Information processing equipment 101A Object detection unit (object detection means) 102A Notification section (notification means) 104A State Estimation Unit (State Estimation Means) 105A Priority setting section (priority setting means) 106A Execution detection unit (execution detection means) 107A Deviation detection unit (deviation detection means) 108A Business operator detection unit (business operator detection means) 2 Terminal Device 3 Mutual monitoring system

Claims

1. a target detection means for detecting a target member of the group who needs care from other members based on a state of the member estimated from an image of the member; and a notification means for notifying each member of the group of the detected target member.

2. an execution detection means for detecting that the member of the group has performed care for the target member; The information processing apparatus according to claim 1 , wherein the notification means terminates notification regarding the target member on condition that it is detected that care for the target member has been provided.

3. The information processing device according to claim 1 or 2, wherein the notification means changes the display position of the icon of the target member among the icons corresponding to each member displayed on a display screen viewable by each member of the group, to a position within a predetermined display area set on the display screen.

4. a deviation detection means for detecting a deviation between a result of the self-evaluation of the member's condition by the member and the state of the member estimated from the image, The information processing apparatus according to claim 1 , wherein the notification means issues a notification to alert the member in respect of which the deviation has been detected.

5. The members include merchants; a service provider detection means for detecting a service provider related to the care of the target member from among the service providers included in the members; 3. The information processing apparatus according to claim 1, wherein the notification means notifies the member who has provided care for the target member of the detected trader.

6. 3 . The information processing apparatus according to claim 1 , further comprising: a state estimation unit for estimating a state of the member from an image taken during a video call by the member for the video call.

7. a priority setting means for setting a priority for each of the other members in caring for the target member; 3. The information processing apparatus according to claim 1, wherein said notification means notifies each member based on the priority.

8. The information processing device according to claim 1 , wherein the notification means notifies a message explaining a state of the target member, the message being generated using a machine-learned language model.

9. At least one processor a target detection process for detecting a target member who needs care from other members among the members of the group based on a state of the member estimated from an image of the member; and a notification process of notifying each member of the group of the detected target member.

10. Computer, a target detection means for detecting a target member of the group who needs care from other members based on a state of the member estimated from an image of the member; and a control program that functions as a notification means for notifying each member of the group of the detected target member;

11. An information processing device according to claim 1 or 2; A mutual monitoring system including a terminal device that presents the target members to a member using the terminal device based on a notification from the information processing device.

Images