Information processing device, terminal, and information processing method

Abstract

To facilitate communication between a user and a medical professional when conducting an interview via a communication network. [Solution] The information processing device 1 communicates with a user terminal 2 used by the user and also with a medical worker terminal 3 used by a medical worker who interviews the user online, and has a relay unit 131 that transmits image data of the user received from the user terminal 2 to the medical worker terminal 3, a determination unit 132 that determines whether the state of the image shown in the image data satisfies predetermined conditions, and an acquisition unit 133 that acquires supplementary information, which is information to supplement the image data, when the determination unit 132 determines that the state of the image shown in the image data satisfies the predetermined conditions, and the relay unit 131 transmits the supplementary information to the medical worker terminal 3 when the determination unit 132 determines that the state of the image shown in the image data satisfies the predetermined conditions.

Description

[Technical Field]

[0001] The present invention relates to an information processing device, a terminal, and an information processing method. [Background technology]

[0002] A system has been disclosed in which a medical professional remotely conducts consultations with a user for medical treatment or the like via a communication network such as the Internet (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-179961 Summary of the Invention [Problem to be solved by the invention]

[0004] When a user and a medical professional have an interview via a communication network, there is a problem that communication is more difficult than when the interview is actually face-to-face.

[0005] The present invention has been made in consideration of these points, and aims to make it easier for users and medical professionals to communicate when conducting interviews via a communication network. [Means for solving the problem]

[0006] An information processing device according to a first aspect of the present invention has a relay unit that communicates with a user terminal used by a user and a medical staff terminal used by a medical staff who conducts an online interview with the user, and transmits image data of the user received from the user terminal to the medical staff terminal, a determination unit that determines whether the state of the image represented by the image data satisfies predetermined conditions, and an acquisition unit that acquires supplementary information, which is information to supplement the image data, when the determination unit determines that the state of the image represented by the image data satisfies the predetermined conditions, and the relay unit transmits the supplementary information to the medical staff terminal when the determination unit determines that the state of the image represented by the image data satisfies the predetermined conditions.

[0007] When the determination unit determines that the state of the image indicated by the image data satisfies the specified condition, the acquisition unit may request the user terminal to acquire the supplementary information, and acquire the supplementary information generated in the user terminal from the user terminal.

[0008] The acquisition unit may input the image data into a large-scale language model, instruct the large-scale language model to generate the supplemental information corresponding to the image data, and acquire the supplemental information output from the large-scale language model.

[0009] The relay unit may communicate with the user terminal via a first communication network and with the medical staff terminal via a second communication network, thereby transmitting the image data received from the user terminal to the medical staff terminal, and the specified condition may include that the state of the image indicated by the image data indicates that the communication quality of the first communication network is below the specified quality.

[0010] The acquisition unit may acquire the supplemental information as explanatory information that explains the state of the user based on the image data. The determination unit determines whether the state of the image indicated by the image data received from the medical staff terminal satisfies predetermined conditions, and when the determination unit determines that the state of the image indicated by the image data received from the medical staff terminal satisfies the predetermined conditions, the acquisition unit identifies a predetermined area contained in the image indicated by the image data received from the user terminal and processes the image data to obtain processed image data that includes the identified predetermined area and has a smaller data volume than the image data, and the relay unit may transmit the processed image data obtained by the acquisition unit as the image data to the medical staff terminal.

[0011] The acquisition unit may identify the specified area included in the image represented by the image data based on questionnaire information indicating the user's questionnaire, medical history information indicating the user's medical history, or interview history information indicating the history of the interview.

[0012] The image data may indicate an image including an image of the user's affected area, and the acquisition unit may identify the user's affected area based on the acquired questionnaire information, medical history information, or interview history information, and identify the specified area indicating the user's affected area included in the image indicated by the image data.

[0013] When transmitting the processed image data as the image data to the medical staff terminal, the relay unit may transmit information indicating that the image data has been processed to the medical staff terminal.

[0014] The relay unit relays information related to the interview between the user terminal and a related party terminal used by a related party of the user, and the medical staff terminal, and when the determination unit determines that the state of the image indicated by the image data satisfies the specified condition, it may reduce the amount of data received from the related party terminal and transmit the data to at least one of the medical staff terminal and the user terminal.

[0015] When the determination unit determines that the state of the image indicated by the image data satisfies the predetermined condition, the acquisition unit may acquire the supplemental information having a smaller data amount than the image data. When the determination unit determines that the state of the image indicated by the image data satisfies the predetermined condition, the relay unit may transmit the image data and the supplemental information to the medical staff terminal.

[0016] A terminal according to a second aspect of the present invention is a terminal used by a user, and comprises: a medical professional terminal used by a medical professional who conducts an online interview with the user; a transmission unit that transmits image data of the user to an information processing device that relays communication with the terminal; a judgment unit that determines whether the state of the image represented by the image data satisfies predetermined conditions; and an acquisition unit that acquires supplementary information, which is information to supplement the image data, when the judgment unit determines that the state of the image represented by the image data satisfies the predetermined conditions; and when the judgment unit determines that the state of the image represented by the image data satisfies the predetermined conditions, the transmission unit transmits the supplementary information to the information processing device.

[0017] An information processing method according to a third aspect of the present invention is executed by a computer, and includes the steps of communicating with a user terminal used by a user and a medical staff terminal used by a medical staff who conducts an online interview with the user, thereby transmitting image data of the user received from the user terminal to the medical staff terminal; determining whether the state of the image represented by the image data satisfies predetermined conditions; and, if it is determined that the state of the image represented by the image data satisfies the predetermined conditions, acquiring supplementary information that is information to supplement the image data. In the transmitting step, if the computer determines that the state of the image represented by the image data satisfies the predetermined conditions, it transmits the supplementary information to the medical staff terminal. [Effects of the Invention]

[0018] According to the present invention, it is possible to facilitate communication between a user and a medical professional when conducting an interview via a communication network. [Brief explanation of the drawings]

[0019] [Figure 1] FIG. 1 is a diagram illustrating an overview of a communication system. [Figure 2] FIG. 2 is a diagram illustrating a functional configuration of an information processing device. [Figure 3] 10 is a flowchart showing a process flow in which the information processing device transmits image data received from a user terminal to a medical staff terminal. [Figure 4] FIG. 10 is a diagram illustrating an example of a medical interview including a person related to the user. DETAILED DESCRIPTION OF THE INVENTION

[0020] [Communication System S Overview] 1 is a diagram illustrating an overview of a communication system S. The communication system S has an information processing device 1, a user terminal 2, and a medical staff terminal 3, and is a system for realizing an online medical consultation between a user using the user terminal 2 and a medical staff using the medical staff terminal 3.

[0021] The information processing device 1 is a computer for providing an online medical interview service. The user terminal 2 is a wireless communication-capable terminal such as a personal computer or smartphone used by a user who uses the online medical service to conduct online interviews with medical professionals. The user terminal 2 can conduct a medical interview with a medical professional using the medical professional terminal 3, for example, by executing an interview application program that enables the user to conduct a medical interview with the medical professional.

[0022] The medical staff terminal 3 is a wireless communication-capable terminal such as a personal computer or smartphone used by a medical staff member who provides online medical services. The medical staff terminal 3 can conduct a medical interview with the user of the user terminal 2, for example, by executing an interview application program.

[0023] A medical interview is, for example, an online medical consultation conducted by a doctor as a medical professional for a user as a patient, or online medication guidance for prescription drugs or over-the-counter drugs conducted by a pharmacist for a user. In the following description, a medical interview is referred to as a medical interview, and an application for the interview is also referred to as an interview app.

[0024] The information processing device 1 relays communication between a user terminal 2 related to a medical consultation and a medical staff terminal 3. That is, the information processing device 1 receives image data of a user, audio data indicating the user's voice, etc. transmitted from the user terminal 2, and transmits them to the medical staff terminal 3. The information processing device 1 also receives image data of a medical staff member, audio data indicating the voice of the medical staff, etc. transmitted from the medical staff terminal 3, and transmits them to the user terminal 2.

[0025] The information processing device 1 determines whether the state of the image shown by the image data of the user satisfies a predetermined condition. The predetermined condition is, for example, that the state of the image shown by the image data indicates that the communication quality of the communication between the user terminal 2 and the medical staff terminal 3 is equal to or lower than a predetermined quality. When the information processing device 1 determines that the predetermined condition is satisfied, it acquires supplemental information that is information for supplementing the image data of the user and transmits the supplemental information to the medical staff terminal 3.

[0026] In this way, the information processing device 1 can make it easier for the medical professional to check the image using the supplementary information even in a situation where the communication quality between the user terminal 2 and the medical professional terminal 3 has deteriorated and it is difficult for the medical professional to check the image of the user. As a result, the information processing device 1 can make it easier for the user and the medical professional to communicate when having an interview via a communication network.

[0027] [Functional configuration of information processing device 1] Next, a description will be given of the functional configuration of the information processing device 1. Fig. 2 is a diagram showing the functional configuration of the information processing device 1. The information processing device 1 has a communication unit 11, a storage unit 12, and a control unit 13.

[0028] The communication unit 11 is a communication interface for transmitting and receiving data to and from the user terminal 2, the medical staff terminal 3, and external devices such as a communication device that manages a communication network. The storage unit 12 is a storage medium that stores various types of data, and includes a ROM, a RAM, a hard disk, etc. The storage unit 12 stores a program executed by the control unit 13. The storage unit 12 stores a program that causes the control unit 13 to function as a relay unit 131, a determination unit 132, and an acquisition unit 133.

[0029] The control unit 13 is, for example, a CPU. The control unit 13 executes the programs stored in the storage unit 12, thereby functioning as a relay unit 131, a determination unit 132, and an acquisition unit 133.

[0030] The relay unit 131 relays information related to a medical interview between the user terminal 2 and the medical staff terminal 3. The relay unit 131 communicates with the user terminal 2 via a first communication network, and also communicates with the medical staff terminal 3 used by the medical staff who conducts an online medical interview with the user via a second communication network, thereby relaying information related to the interview between the user terminal 2 and the medical staff terminal 3.

[0031] For example, when the relay unit 131 receives from the user terminal 2 a request to start a medical interview between a user and a medical professional using the medical professional terminal 3, and also receives from the medical professional terminal 3 a request to start a medical interview between the medical professional and the user, the relay unit 131 starts relaying information related to the medical interview between the user terminal 2 and the medical professional terminal 3.

[0032] For example, the relay unit 131 receives image data indicating an image of a user captured by the user terminal 2 and audio data indicating the user's voice collected by the user terminal 2 via a first communication network, and transmits the image data and audio data via a second communication network to the medical staff terminal 3. Similarly, the relay unit 131 receives image data indicating an image of a medical staff captured by the medical staff terminal 3 and audio data indicating the medical staff's voice collected by the medical staff terminal 3 via the second communication network, and transmits the image data and audio data to the user terminal 2 via the first communication network.

[0033] When the relay unit 131 receives a request to end the medical interview from the user terminal 2 or the medical staff terminal 3, it ends the relay of information related to the medical interview between the user terminal 2 and the medical staff terminal 3.

[0034] The determination unit 132 determines whether the communication quality of the communication between the user terminal 2 and the medical staff terminal 3 is equal to or lower than a predetermined quality. For example, the determination unit 132 determines whether the communication quality of the communication between the user terminal 2 and the medical staff terminal 3 is equal to or lower than a predetermined quality by determining whether at least one of the audio state indicated by the user's audio data received from the user terminal 2 and the image state indicated by the image data capturing the user satisfies a first condition as a predetermined condition. Here, the first condition is a condition that at least one of the audio state indicated by the user's audio data received from the user terminal 2 and the image state indicated by the image data indicates that the communication quality of the first communication network is equal to or lower than a predetermined quality.

[0035] Specifically, first, the determination unit 132 analyzes an image represented by image data received from the user terminal 2. Then, the determination unit 132 identifies the degree of blurring of edges included in the image using a known analysis technique, and if the identified degree of blurring exceeds a predetermined threshold, determines that the communication quality of the first communication network is equal to or lower than a predetermined quality, i.e., determines that the first condition is satisfied. Similarly, the determination unit 132 analyzes the audio and images represented by the user's audio data and image data received from the user terminal 2 using a known analysis technique. Then, if the amount of noise per unit time included in the audio and images represented by the analyzed audio or image data is equal to or higher than a predetermined amount, if the number of interruptions in the audio or image per unit time is a predetermined number, or if the size of the audio or image is equal to or smaller than a predetermined amount, the determination unit 132 determines that the communication quality of the first communication network is equal to or lower than a predetermined quality, i.e., determines that the first condition is satisfied. The judgment unit 132 judges that the communication quality of the communication between the user terminal 2 and the medical staff terminal 3 is below a predetermined quality when at least one of the voice state indicated by the user's voice data received from the user terminal 2 and the image state indicated by the image data capturing the user satisfies a first condition.

[0036] The determination unit 132 may determine that the communication quality of the communication between the user terminal 2 and the medical staff terminal 3 is equal to or lower than a predetermined quality by determining whether at least one of the voice state indicated by the voice data of the medical staff received from the medical staff terminal 3 and the image state indicated by the image data capturing the medical staff satisfies a second condition. In this case, the second condition is that at least one of the voice state indicated by the voice data of the medical staff received from the medical staff terminal 3 and the image state indicated by the image data indicates that the communication quality of the second communication network is equal to or lower than a predetermined quality.

[0037] Specifically, the determination unit 132 analyzes the image represented by the image data received from the medical staff terminal 3 and identifies the degree of blurring of edges included in the image. If the identified degree of blurring exceeds a predetermined threshold, the determination unit 132 determines that the communication quality of the second communication network is equal to or lower than a predetermined quality, i.e., determines that the second condition is satisfied. Similarly, the determination unit 132 analyzes the audio and images represented by the audio data and image data of the medical staff received from the medical staff terminal 3 using a known analysis technique. If the amount of noise per unit time included in the audio and images represented by the analyzed audio or image data is equal to or higher than a predetermined amount, if the number of audio or image interruptions per unit time is a predetermined number, or if the size of the audio or image is equal to or smaller than a predetermined amount, the determination unit 132 determines that the communication quality of the second communication network is equal to or lower than a predetermined quality, i.e., determines that the second condition is satisfied. The judgment unit 132 judges that the communication quality of the communication between the user terminal 2 and the medical worker terminal 3 is below a predetermined quality when at least one of the voice state indicated by the voice data of the medical worker received from the medical worker terminal 3 and the image state indicated by the image data capturing the medical worker satisfies a second condition.

[0038] In addition, the judgment unit 132 may judge that the communication quality of the communication between the user terminal 2 and the medical staff terminal 3 is below a predetermined quality if the voice data received and analyzed from the medical staff terminal 3 includes voice that indicates poor quality of communication related to the medical interview, such as "it's hard to see" or "the image is moving around."

[0039] When the determination unit 132 determines that the communication quality of the communication between the user terminal 2 and the medical staff terminal 3 is equal to or lower than a predetermined quality, the acquisition unit 133 acquires supplemental information. Specifically, when the determination unit 132 determines that the state of the image indicated by the image data received from the user terminal 2 satisfies a first condition, the acquisition unit 133 acquires supplemental information that is information for supplementing the image data. Furthermore, when the determination unit 132 determines that the state of the image indicated by the image data received from the medical staff terminal 3 satisfies a second condition, the acquisition unit 133 acquires supplemental information that is information for supplementing the image data received from the user terminal 2.

[0040] Here, the supplemental information is, for example, information with a smaller data volume than the image data, and is explanatory information that explains the user's condition based on the image data, such as the user's appearance, mood, emotions, or medical condition shown in the image data, or the condition of the user's affected area. For example, when the determination unit 132 determines that the condition of the image shown in the image data received from the user terminal 2 satisfies a first condition, or when the determination unit 132 determines that the condition of the image shown in the image data received from the medical staff terminal 3 satisfies a second condition, the acquisition unit 133 requests the user terminal 2 to acquire the supplemental information.

[0041] When the user terminal 2 receives a request to acquire supplemental information, it acquires the supplemental information by analyzing the image data that it transmits to the information processing device 1. The user terminal 2 generates, for example, the image data and a prompt that instructs the large-scale language model to generate explanatory information that explains the user's condition based on the image data, and inputs the prompt to the large-scale language model. The user terminal 2 may include information indicating the user's symptoms that the user terminal 2 has received in advance from the user in the prompt, and input the prompt to the large-scale language model.

[0042] Then, the user terminal 2 generates supplemental information by acquiring the explanatory information output from the large-scale language model. The user terminal 2 transmits the generated supplemental information to the information processing device 1. The acquisition unit 133 acquires the supplemental information generated in the user terminal 2 from the user terminal 2.

[0043] Note that the acquisition unit 133 acquires the supplemental information from the user terminal 2, but this is not limiting. The acquisition unit 133 may input image data acquired from the user terminal 2 to the large-scale language model, instruct the large-scale language model to generate supplemental information corresponding to the image data, and acquire the supplemental information output from the large-scale language model.

[0044] When the determination unit 132 determines that the communication quality of the communication between the user terminal 2 and the medical staff terminal 3 is below a predetermined quality, i.e., when the determination unit 132 determines that the state of the image indicated by the image data acquired from the user terminal 2 satisfies a first condition, or when the determination unit 132 determines that the state of the image indicated by the image data received from the medical staff terminal 3 satisfies a second condition, the relay unit 131 transmits the supplementary information acquired by the acquisition unit 133 to the medical staff terminal 3.

[0045] For example, when the determination unit 132 determines that the communication quality of the communication between the user terminal 2 and the medical staff terminal 3 is equal to or lower than a predetermined quality, the relay unit 131 transmits the image data and supplemental information to the medical staff terminal 3.

[0046] Note that the relay unit 131 may transmit the supplemental information acquired by the acquisition unit 133 to the user terminal 2 instead of transmitting the image data. Furthermore, when the acquisition unit 133 acquires the supplemental information from the user terminal 2, the relay unit 131 may transmit instruction information instructing the user terminal 2 to send the supplemental information instead of sending the image data, and may receive the supplemental information from the user terminal 2 but not receive the image data. In this way, the information processing device 1 can reduce the load on the first communication network.

[0047] Furthermore, when the determination unit 132 determines that the communication quality of the communication between the user terminal 2 and the medical staff terminal 3 is equal to or lower than a predetermined quality, and then the determination unit 132 determines that the communication quality of the communication between the user terminal 2 and the medical staff terminal 3 exceeds the predetermined quality, the relay unit 131 may transmit the image data to the medical staff terminal 3 without transmitting the supplemental information. In this case, the acquisition unit 133 may not acquire the supplemental information.

[0048] Furthermore, if the medical interview is related to diagnosing a rash or injury on the user's skin, the image data may represent an image including an image of the user's affected area. In this case, when the determination unit 132 determines that the state of the image represented by the image data received from the medical staff terminal 3 satisfies a first condition, or when the determination unit 132 determines that the state of the image represented by the image data received from the medical staff terminal 3 satisfies a second condition, the acquisition unit 133 may identify a predetermined area included in the image represented by the image data received from the user terminal 2. Here, the predetermined area may be an area representing the image of the user's affected area included in the image represented by the image data.

[0049] For example, at the start of or during a medical interview, the acquisition unit 133 acquires medical questionnaire information indicating the user's medical questionnaire, medical history information indicating the user's medical history, or interview history information indicating the history of the interview. Then, the acquisition unit 133 identifies the user's affected area based on the acquired medical questionnaire information, medical history information, or interview history information, and identifies a predetermined area indicating the image of the user's affected area included in an image indicated by image data received from the user terminal 2. For example, the acquisition unit 133 generates a prompt that instructs the large-scale language model to identify a predetermined area from the image data based on, for example, the medical questionnaire information, medical history information, or interview history information, and inputs the prompt, the medical questionnaire information, the medical history information, or the interview history information, and the image data to the large-scale language model. Then, the acquisition unit 133 identifies the predetermined area based on the information indicating the predetermined area output from the large-scale language model.

[0050] When the acquisition unit 133 identifies the predetermined area, it acquires processed image data that includes the identified predetermined area and has a smaller data volume than the image data by processing the image data received from the user terminal 2. The acquisition unit 133 may extract the predetermined area from the image indicated by the image data received from the user terminal 2 and use the image data indicating the extracted predetermined area as the processed image data, or may generate the processed image data by reducing the data volume of an area of the image indicated by the image data received from the user terminal 2 that is different from the predetermined area.

[0051] Then, the relay unit 131 transmits the processed image data acquired by the acquisition unit 133 to the medical staff terminal 3 as image data received from the user terminal 2. When transmitting the processed image data as image data to the medical staff terminal 3, the relay unit 131 may transmit information indicating that the image data has been processed as supplemental information to the medical staff terminal 3. In this way, the information processing device 1 can make the medical staff recognize that the image data has been processed.

[0052] [Operation flow] Next, we will explain the flow of processing related to the application of the communication preferential service in the information processing device 1. Fig. 3 is a flowchart showing the flow of processing in which the information processing device 1 transmits image data received from the user terminal 2 to the medical staff terminal 3. The processing shown in Fig. 3 starts in response to the execution of an interview app on the user terminal 2.

[0053] The relay unit 131 receives image data of a user captured from the user terminal 2 (S1). Next, the determination unit 132 determines whether the communication quality of the communication between the user terminal 2 and the medical staff terminal 3 is equal to or lower than a predetermined quality (S2). If the determination unit 132 determines that the communication quality is equal to or lower than the predetermined quality (YES in S2), the process proceeds to S3, and if the determination unit 132 determines that the communication quality is not equal to or lower than the predetermined quality (NO in S2), the process proceeds to S5.

[0054] In S3, the acquisition unit 133 acquires supplemental information for the image data received from the user terminal 2. Subsequently, the relay unit 131 transmits the image data and supplemental information received from the user terminal 2 to the medical staff terminal 3 (S4).

[0055] In S5, the relay unit 131 transmits the image data received from the user terminal 2 to the medical staff terminal 3. Next, the determination unit 132 determines whether the medical interview has ended (S6). If the determination unit 132 determines that the medical interview has ended (YES in S6), it ends the processing according to this flowchart, and if it determines that the medical interview has not ended (NO in S6), it returns to S1.

[0056] [Variation 1] In the above embodiment, the medical interview is conducted between the user of the user terminal 2 and the medical worker using the medical worker terminal 3, but this is not limiting, and the medical interview may be conducted between the user of the user terminal 2 and a related person of the user, and the medical worker using the medical worker terminal 3. A related person of the user is, for example, the user's caregiver or the user's guardian.

[0057] 4 is a diagram showing an example of a medical interview including a person related to the user. As shown in FIG. 4, the relay unit 131 relays communications between the user terminal 2, the medical staff terminal 3, and the related party terminal 4 used by the person related to the user. That is, the relay unit 131 transmits data indicating the captured image and audio of the user received from the user terminal 2 to the medical staff terminal 3 and the related party terminal 4. The relay unit 131 also transmits data indicating the captured image and audio of the medical staff received from the medical staff terminal 3 to the user terminal 2 and the related party terminal 4.

[0058] Furthermore, the relay unit 131 transmits data indicating the captured images and audio of the related parties received from the related party terminal 4 to the user terminal 2 and the medical staff terminal 3. When the determination unit 132 determines that the communication quality of the communication between the user terminal 2 and the medical staff terminal 3 is equal to or lower than a predetermined quality, the relay unit 131 reduces the amount of data received from the related party terminal 4 that is transmitted to at least one of the user terminal 2 and the medical staff terminal 3.

[0059] For example, the relay unit 131 degrades the data received from the related party terminal 4, i.e., reduces the data volume of the data, and transmits it to at least one of the user terminal 2 and the medical staff terminal 3. The relay unit 131 may also reduce the volume of data transmitted to at least one of the user terminal 2 and the medical staff terminal 3 by not transmitting the data received from the related party terminal 4 to at least one of the user terminal 2 and the medical staff terminal 3. In this way, during a medical interview in which a related party participates, it is possible to prevent the communication capacity from being overwhelmed by data received from the related party terminal 4 of a third party related party, and to prevent a decrease in communication quality.

[0060] [Variation 2] In the above-described embodiment, the relay unit 131 transmits the supplemental information to the medical staff terminal 3 when the determination unit 132 determines that the communication quality of the communication between the user terminal 2 and the medical staff terminal 3 is equal to or lower than a predetermined quality, but this is not limiting. The relay unit 131 may transmit the supplemental information to the medical staff terminal 3 when the communication quality of the communication between the user terminal 2 and the medical staff terminal 3 is not equal to or lower than the predetermined quality.

[0061] In this case, for example, the determination unit 132 determines whether the state of the image indicated by the image data satisfies a predetermined condition. The predetermined condition is, for example, that the state of the image indicated by the image data is easy for medical personnel to see, specifically, that the brightness of the image indicated by the image data is equal to or greater than a first threshold and less than a second threshold. In this way, the information processing device 1 can output supplemental information to the medical personnel terminal 3 when the image data received from the user terminal 2 is too bright or too dark and difficult for medical personnel to see. This allows the information processing device 1 to make it easier for medical personnel to understand the state of the user's affected area, etc., shown in the user's image indicated by the image data.

[0062] [Variation 3] In the above-described embodiment, the determination unit 132 determines whether the communication quality of the communication between the user terminal 2 and the medical staff terminal 3 is equal to or lower than a predetermined quality based on the image data received from the user terminal 2, but this is not limiting. The determination unit 132 may, for example, identify the communication quality, such as the throughput, between the user terminal 2 and the information processing device 1, and the communication quality, such as the throughput, between the medical staff terminal 3 and the information processing device 1. Then, the determination unit 132 may determine whether the communication quality of the communication between the user terminal 2 and the medical staff terminal 3 is equal to or lower than the predetermined quality by determining whether at least one of the identified communication qualities is equal to or lower than the predetermined quality.

[0063] In this case, for example, the determination unit 132 determines whether the state of the image indicated by the image data satisfies a predetermined condition. The predetermined condition is, for example, that the state of the image indicated by the image data is easy for medical personnel to see, specifically, that the brightness of the image indicated by the image data is equal to or greater than a first threshold and less than a second threshold. In this way, the information processing device 1 can output supplemental information to the medical personnel terminal 3 when the image data received from the user terminal 2 is too bright or too dark and difficult for medical personnel to see. This allows the information processing device 1 to make it easier for medical personnel to understand the state of the user's affected area, etc., shown in the user's image indicated by the image data.

[0064] [Variation 4] In the above-described embodiment, the information processing device 1 functions as the determination unit 132 and determines whether the state of the image indicated by the image data transmitted by the user terminal 2 satisfies a predetermined condition, but the present invention is not limited to this.

[0065] For example, the control unit of the user terminal 2 may function as the determination unit 132. The user terminal 2 may then determine whether the state of the image shown in the image data of the user that the user terminal 2 transmits to the information processing device 1 satisfies a predetermined condition. The control unit of the user terminal 2 may also determine whether the state of the image shown in the image data received from the medical staff terminal 3 satisfies a predetermined condition. The control unit of the user terminal 2 may then acquire supplementary information for the image data of the user when it determines that the predetermined condition is satisfied. This reduces the load on the information processing device 1 and makes it easier for the user and medical staff to communicate with each other when they have an interview via a communication network.

[0066] [Effects of information processing device 1] As described above, the information processing device 1 according to this embodiment communicates with the user terminal used by the user and also communicates with the medical staff terminal 3 used by the medical staff who will have an online interview with the user, thereby transmitting image data of the user received from the user terminal 2 to the medical staff terminal 3, and when it determines that the state of the image represented by the image data satisfies a predetermined condition, acquiring supplementary information that is information for supplementing the image data, and transmitting it to the medical staff terminal 3. In this way, the information processing device 1 can facilitate communication when the user and the medical staff have an interview via a communication network.

[0067] Furthermore, this invention will make it possible to contribute to Goal 9 of the United Nations' Sustainable Development Goals (SDGs), which is "Build resilient infrastructure, promote inclusive and sustainable industrialization, and promote innovation and resilience."

[0068] The present invention has been described above using embodiments, but the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. For example, all or part of the device can be configured by functionally or physically distributing or integrating any unit. Furthermore, new embodiments resulting from any combination of multiple embodiments are also included in the embodiments of the present invention. The effects of the new embodiments resulting from the combination also have the effects of the original embodiments. [Explanation of symbols]

[0069] 1. Information processing equipment 2. User terminal 3 Medical staff terminals 4. Stakeholder terminals 11 Communications Department 12 Storage section 13 Control Unit 131 Relay Section 132 Judgment section 133 Acquisition Department

Claims

1. a relay unit that communicates with a user terminal used by a user and also communicates with a medical staff terminal used by a medical staff who has an online interview with the user, and transmits image data of the user received from the user terminal to the medical staff terminal; a determination unit that determines whether the state of the image represented by the image data satisfies a predetermined condition; an acquisition unit that, when the determination unit determines that the state of the image indicated by the image data satisfies the predetermined condition, requests the user terminal to acquire supplemental information that is information for supplementing the image data, and acquires the supplemental information generated in the user terminal from the user terminal; and When the determination unit determines that the state of the image indicated by the image data satisfies the predetermined condition, the relay unit transmits the supplemental information to the medical staff terminal. Information processing device.

2. a relay unit that communicates with a user terminal used by a user and also communicates with a medical staff terminal used by a medical staff who has an online interview with the user, and transmits image data of the user received from the user terminal to the medical staff terminal; a determination unit that determines whether the state of the image represented by the image data satisfies a predetermined condition; an acquisition unit that, when the determination unit determines that the state of the image indicated by the image data satisfies the predetermined condition, inputs the image data into a large-scale language model, instructs the large-scale language model to generate supplemental information that corresponds to the image data and is information for supplementing the image data, and acquires the supplemental information output from the large-scale language model; and When the determination unit determines that the state of the image indicated by the image data satisfies the predetermined condition, the relay unit transmits the supplemental information to the medical staff terminal. Information processing device.

3. the relay unit communicates with the user terminal via a first communication network and with the medical staff terminal via a second communication network, thereby transmitting the image data received from the user terminal to the medical staff terminal; the predetermined condition includes a condition of an image represented by the image data indicating that the communication quality of the first communication network is equal to or lower than a predetermined quality; 3. The information processing device according to claim 1 or 2.

4. the acquisition unit acquires the supplemental information as explanatory information that explains the state of the user based on the image data.

3. The information processing device according to claim 1 or 2.

5. When the determination unit determines that the state of the image indicated by the image data satisfies the predetermined condition, the acquisition unit acquires the supplemental information as explanatory information that explains the state of the user and has a data amount smaller than that of the image data. The information processing device according to claim 4 .

6. the determination unit determines whether a state of an image indicated by the image data received from the medical staff terminal satisfies a predetermined condition; When the determination unit determines that the state of the image represented by the image data received from the medical staff terminal satisfies the predetermined condition, the acquisition unit identifies a predetermined area included in the image represented by the image data received from the user terminal based on medical questionnaire information representing the user's medical questionnaire, medical history information representing the user's medical history, or consultation history information representing the history of the consultation, and processes the image data to acquire processed image data that includes the identified predetermined area and has a smaller data volume than the image data; The relay unit transmits the processed image data acquired by the acquisition unit as the image data to the medical staff terminal.

3. The information processing device according to claim 1 or 2.

7. the image data indicates an image including an image of an affected area of the user, the acquiring unit identifies the affected area of the user based on the acquired medical questionnaire information, medical history information, or interview history information, and identifies the predetermined area indicating the affected area of the user included in the image represented by the image data. The information processing device according to claim 6 .

8. When transmitting the processed image data as the image data to the medical staff terminal, the relay unit transmits information indicating that the image data has been processed to the medical staff terminal. The information processing device according to claim 6 .

9. the relay unit relays information related to the interview between the user terminal and a related party terminal used by a related party of the user, and the medical staff terminal, and when the determination unit determines that the state of the image indicated by the image data satisfies the predetermined condition, reduces the amount of data received from the related party terminal and transmits the data to at least one of the medical staff terminal and the user terminal; 3. The information processing device according to claim 1 or 2.

10. When the determination unit determines that the state of the image indicated by the image data satisfies the predetermined condition, the relay unit transmits the image data and the supplemental information to the medical staff terminal.

3. The information processing device according to claim 1 or 2.

11. A terminal used by a user, a transmission unit that transmits image data of the user to a medical staff terminal used by the medical staff who has an online interview with the user and an information processing device that relays communication with the terminal; a determination unit that determines whether the state of the image represented by the image data satisfies a predetermined condition; an acquisition unit that, when the determination unit determines that the state of the image indicated by the image data satisfies the predetermined condition, inputs the image data into a large-scale language model, instructs the large-scale language model to generate supplemental information that corresponds to the image data and is information for supplementing the image data, and acquires the supplemental information output from the large-scale language model; and When the determination unit determines that the state of the image indicated by the image data satisfies the predetermined condition, the transmission unit transmits the supplemental information acquired by the acquisition unit to the information processing device. Terminal.

12. The computer executes a step of communicating with a user terminal used by the user and also communicating with a medical staff terminal used by a medical staff who will be conducting an online interview with the user, thereby transmitting image data of the user received from the user terminal to the medical staff terminal; determining whether the state of the image represented by the image data satisfies a predetermined condition; When it is determined that the state of the image indicated by the image data satisfies the predetermined condition, requesting the user terminal to acquire supplemental information that is information for supplementing the image data, and acquiring the supplemental information generated in the user terminal from the user terminal; and In the transmitting step, when the computer determines that the state of the image represented by the image data satisfies the predetermined condition, the computer transmits the supplemental information to the medical staff terminal. Information processing methods.

13. The computer executes a step of communicating with a user terminal used by the user and also communicating with a medical staff terminal used by a medical staff who will be conducting an online interview with the user, thereby transmitting image data of the user received from the user terminal to the medical staff terminal; determining whether the state of the image represented by the image data satisfies a predetermined condition; a step of inputting the image data into a large-scale language model when it is determined that the state of the image represented by the image data satisfies the predetermined condition, instructing the large-scale language model to generate supplemental information that corresponds to the image data and is information for supplementing the image data, and acquiring the supplemental information output from the large-scale language model; and In the transmitting step, when the computer determines that the state of the image represented by the image data satisfies the predetermined condition, the computer transmits the supplemental information to the medical staff terminal. Information processing methods.

Images