Image processing method, program, and terminal device

Abstract

This invention provides an image processing method that allows for greater flexibility in configuring the appearance of images indicating participants, which are displayed on the terminal device of the communication partner. [Solution] An image processing method comprising: receiving an input to select one display mode from a selection of multiple display modes that define the appearance of an image of a participant using a first terminal device displayed on a second terminal device; and transmitting an image of the participant having the appearance defined by the one display mode to the second terminal device, wherein the selection includes at least one of the following display modes: a first mode which is a display mode that reduces the visibility of a predetermined body part among the images of the participant's body included in the captured image acquired by the first terminal device, without reducing the visibility of other body parts; and a second mode which is a display mode that reduces the visibility of a predetermined type of object among the images of objects included in the captured image, without reducing the visibility of other types of objects.

Description

Technical Field

[0001] The present invention relates to an image processing method, a program, and a terminal device.

Background Art

[0002] Conventionally, an image processing method for performing processing for privacy protection on a captured image has been known. For example, Patent Document 1 discloses an image processing method in which a human face is detected from a captured image, and mask processing such as mosaic is performed on the detected face or an image of its background.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the image processing method of Patent Document 1, since mask processing is performed on the entire image of the face or its background detected from the captured image, for example, it is not possible to show or not show only a part of the body other than the face or a part of the background to the communication partner. For this reason, it has been desired to increase the degree of freedom of the image displayed on the partner terminal device.

Means for Solving the Problems

[0005] The image processing method of the present disclosure includes receiving an input to select one display mode from a selection of multiple display modes that define the appearance of an image showing a participant using the first terminal device displayed on a second terminal device which is communicably connected to the first terminal device, and transmitting an image showing the participant having the appearance defined by one of the display modes to the second terminal device, wherein the selection includes at least one of a first mode which is a display mode that reduces the visibility of a predetermined body part among the images of the participant's body included in the captured image acquired by the first terminal device, without reducing the visibility of other body parts, and a second mode which is a display mode that reduces the visibility of a predetermined type of object among the images of objects included in the captured image, without reducing the visibility of other types of objects.

[0006] The program of this disclosure is a program that causes a computer to receive input to select one display mode from a selection of display modes that define the appearance of an image of a participant using the first terminal device displayed on a second terminal device which is communicably connected to the first terminal device, and to transmit an image of the participant having the appearance defined by one of the display modes to the second terminal device, wherein the selection includes at least one of two display modes: a first mode which is a display mode that reduces the visibility of a predetermined body part of the image of the participant's body included in an image captured by the first terminal device without reducing the visibility of other body parts, and a second mode which is a display mode that reduces the visibility of a predetermined type of object without reducing the visibility of other types of objects among images of objects included in the image captured.

[0007] The terminal device of this disclosure includes an input device that receives input for selecting one display mode from a selection of multiple display modes that define the appearance of an image of a participant displayed on another terminal device that is a communication partner in a remote dialogue, and a communication device that transmits an image of the participant having the appearance defined by the one display mode to the other terminal device, wherein the selection includes at least one of the following display modes: a first mode which is a display mode that reduces the visibility of a predetermined body part of the image of the participant's body included in an image captured as an image of the participant, without reducing the visibility of other body parts, and a second mode which is a display mode that reduces the visibility of a predetermined type of object without reducing the visibility of other types of objects among the images of objects included in the image captured. [Brief explanation of the drawing]

[0008] [Figure 1] A diagram showing the configuration of the remote dialogue system according to the embodiment. [Figure 2] An explanatory diagram showing an overview of how the remote dialogue system works. [Figure 3] A block diagram showing the configuration of the first terminal device. [Figure 4] A block diagram showing the configuration of the second terminal device. [Figure 5] A block diagram showing the server configuration. [Figure 6] A flowchart illustrating the processing steps of an image processing method. [Figure 7] A diagram showing an example of a remote dialogue screen displayed on the second terminal device. [Figure 8] A diagram showing an example of display modes. [Figure 9] A diagram showing an example of the display mode selection screen. [Figure 10] A diagram showing an example of a target information input screen. [Modes for carrying out the invention]

[0009] [1. Configuration of the Remote Dialogue System] This embodiment will be described below with reference to the drawings. Figure 1 shows the configuration of the remote dialogue system according to this embodiment. Remote Dialogue System 1 is a system in which multiple participants engage in remote dialogue using terminal devices. Here, "remote dialogue" refers to so-called remote communication, where multiple people who are not physically in the same location communicate using voice and images through tools such as terminal devices that are connected to each other.

[0010] The remote dialogue system 1 includes a first terminal device 2A used by the first participant U1, a second terminal device 2B used by the second participant U2, and a server 3. The first terminal device 2A, the second terminal device 2B, and the server 3 are connected to each other via a communication network 4.

[0011] In the following, when the first terminal device 2A and the second terminal device 2B are not distinguished, they will be referred to as terminal device 2. Also, in the following, when the first participant U1 and the second participant U2 are not distinguished, they will be referred to as participant U.

[0012] The first terminal device 2A is installed at site S1, and the second terminal device 2B is installed at site S2. In the following, if sites S1 and S2 are not distinguished, they will be referred to as site S.

[0013] The remote dialogue system 1 allows each participant U to participate in a remote dialogue by sharing audio and images with other participants U using their respective terminal devices 2. The images may include images captured within the base S where the terminal devices 2 are installed, or predetermined images. The images captured within the base S may include an image of the participant U using the terminal device 2 and its background image. The predetermined images may be, for example, a logo, icon, avatar, or animation representing the participant U using the terminal device 2.

[0014] The first participant U1 is a participant who desires to be able to intentionally display or not display at the second terminal device 2B at least a part of a person, at least a part of a background, or at least a part of an object at the base S1, etc. that appears in the captured image transmitted by the first terminal device 2A. One first terminal device 2A may be used by a plurality of first participants U1, and one second terminal device 2B may be used by a plurality of second participants U2.

[0015] Here, the first terminal device 2A corresponds to the terminal device in the present disclosure, and the second terminal device 2B corresponds to another terminal device that is a communication partner of the terminal device.

[0016] As an example, the base S1 where the first terminal device 2A is installed is the living room of the house where the medical care child C is. Also, the first participant U1 is the medical care child C and the person N who cares for the medical care child C. Also, the second participant U2 who uses the second terminal device 2B is a nursing student who learns about medical care, and the base S2 where the second terminal device 2B is installed is a conference room in the nursing school that those nursing students attend, etc.

[0017] The terminal device 2 is a computer having a communication function. Specifically, the terminal device 2 is a desktop PC (Personal Computer), a tablet PC, a smartphone, etc. The server 3 may be one computer, may be composed of a plurality of computers, or may be a cloud server.

[0018] The communication network 4 may be a LAN (Local Area Network) or a WAN (Wide Area Network). It may also be a global network including a dedicated line, a public switched telephone network, the Internet, etc.

[0019] The first terminal device 2A consists of a first processing unit 21a and a plurality of devices connected to the first processing unit 21a by wire or wireless means. The devices connected to the first processing unit 21a include a display device 22a, a keyboard 23a, a mouse 24a, a camera 25a, a microphone 26a, a speaker 27a, and a communication device 28a. At least one of these devices may be integrated into the housing that houses the first processing unit 21a.

[0020] The display device 22a is, for example, a liquid crystal display, an organic EL (Electro-Luminescence) display, or a plasma display. The keyboard 23a and mouse 24a are input devices used by the first participant U1 for input operations.

[0021] Camera 25a captures an image of base S1 where the first terminal device 2A is installed. As described above, in this embodiment, base S1 is the room of child C requiring medical care. The image of base S1 captured by camera 25a may include an image of child C requiring medical care, who is the first participant U1, and the caregiver N, as well as a background image. The background image may be an image of the room. The background image may include an image of the medical equipment used for the medical care of child C, which is placed in the room, as well as an image of the room's interior and furniture.

[0022] The second terminal device 2B consists of a second processing unit 21b and a plurality of devices connected to the second processing unit 21b by wire or wireless means. The devices connected to the second processing unit 21b include a display device 22b, a keyboard 23b, a mouse 24b, a camera 25b, a microphone 26b, a speaker 27b, and a communication device 28b. At least one of these devices may be integrated into the housing that houses the second processing unit 21b.

[0023] The display device 22b is, for example, a liquid crystal display, an organic EL (Electro-Luminescence) display, or a plasma display. Keyboard 23b and mouse 24b are input devices used by the second participant U2 for input operations. Camera 25b captures images of base S2 where the second terminal device 2B is installed. The images of base S2 captured by camera 25b may include images of the second participant U2 and background images.

[0024] Figure 2 is an explanatory diagram illustrating the overview of the operation of the remote dialogue system 1. In the remote dialogue system 1, participants U view and hear each other's images and sounds using their respective terminal devices 2. The terminal devices 2 send and receive image and sound data, as well as information about inputs and other operations performed on each terminal device 2, via the server 3.

[0025] Specifically, the first terminal device 2A and the second terminal device 2B transmit terminal data D11 and D12, respectively, to the server 3. The server 3 also transmits server data D21 and D22, respectively, to the first terminal device 2A and the second terminal device 2B. In the following, terminal data D11 and D12 will be referred to as terminal data D1 if they are not distinguished. Similarly, server data D21 and D22 will be referred to as server data D2 if they are not distinguished.

[0026] Terminal data D11 includes image data, audio data, and operation data. Image data is data of an image captured by camera 25a or data of a predetermined image. Audio data includes audio data collected by microphone 26a. Operation data may include data representing input operations or selection operations performed by keyboard 23a or mouse 24a.

[0027] Similarly, terminal data D12 includes image data, audio data, and operation data. Image data is data of an image captured by camera 25b or data of a predetermined image. Audio data includes audio data collected by microphone 26b. Operation data may also include data representing input operations or selection operations performed by keyboard 23b or mouse 24b.

[0028] Server 3 generates display data, including the captured image or predetermined image, based on the image data contained in the terminal data D1 received from each terminal device 2. Server 3 also generates integrated audio data, including the audio collected by microphones 26a and 26b, based on the audio data contained in the terminal data D1 received from each terminal device 2. Server 3 also generates setting data, which integrates the information contained in the operation data, based on the operation data contained in the operation data received from each terminal device 2. Server 3 generates server data D2, which includes the generated display data, integrated audio data, and setting data.

[0029] Furthermore, in order to prevent feedback at site S, server 3 may use different integrated audio data for transmission to the first terminal device 2A and the second terminal device 2B. For example, server 3 transmits server data D21, which does not include the audio picked up by microphone 26a, to the first terminal device 2A. In this case, speaker 27a does not output the audio picked up by microphone 26a. As a result, the first participant U1 does not hear their own voice from speaker 27a, thus avoiding any discomfort for the first participant U1. Server 3 may also perform similar control on server data D22.

[0030] The first terminal device 2A receives server data D21 and outputs sound from speaker 27a based on the integrated voice data contained in server data D21. The first terminal device 2A also displays the display data contained in server data D21 on display device 22a. Similarly, the second terminal device 2B outputs sound from speaker 27b and displays information on display device 22b based on server data D22. As a result, all participants U using the remote dialogue system 1 can view captured images or predetermined images and audio transmitted by other participants U's terminal devices 2 using their own terminal devices 2, and communicate with each other.

[0031] Figure 7 shows an example of a remote dialogue screen with the first terminal device 2A, as displayed on the display device 22b of the second terminal device 2B. In the example shown, a participant display area IAa, indicating the first participant U1 using the first terminal device 2A, is displayed in the middle of the screen. In addition, a participant display area IAb, indicating the second participant U2 using the second terminal device 2B, is displayed in the lower right of the screen. In Figure 7, a predetermined image containing the text "Mr. A's house" indicating the first participant U1 is displayed in participant display area IAa, and a predetermined image containing the text "Suzuki" indicating the second participant U2 is displayed in participant display area IAb. At the top of the screen, a camera button 60 for turning the camera 25b on / off, a microphone button 61 for turning the microphone 26b on / off, and an end button 62 for instructing the second terminal device 2B to end the current remote dialogue are displayed. The sizes of the participant display areas IAa and IAb can be changed by dragging the border lines of the corresponding areas with the mouse 24b.

[0032] The display device 22a of the first terminal device 2A also displays a remote dialogue screen similar to the remote dialogue screen shown in Figure 7. In the remote dialogue screen displayed on the display device 22a of the first terminal device 2A, for example, a participant display area IAb indicating the second participant U2 is displayed in the middle of the screen, and a participant display area IAa indicating the first participant U1 is displayed in the lower right of the screen.

[0033] Here, the image displayed in the participant display area IAa of the display device 22b of the second terminal device 2B, which shows the first participant U1, corresponds to the image of the first participant U1 using the first terminal device 2A displayed on the second terminal device 2B in this disclosure. Hereinafter, the image of the first participant U1 using the first terminal device 2A displayed on the second terminal device 2B will also be referred to as the "first participant display image".

[0034] In the remote dialogue system 1 of this embodiment, in particular, input is received to select one display mode from a set of options that include multiple display modes defining the appearance of the first participant display image displayed on the second terminal device 2B. This input for selecting a display mode is made, for example, by the first participant U1 using the keyboard 23a or mouse 24a of the first terminal device 2A. That is, the keyboard 23a or mouse 24a of the first terminal device 2A is an example of an input device in this disclosure that receives input to select one display mode from the above options.

[0035] Then, the first participant display image, having the appearance defined by the selected display mode, is transmitted from the communication device 28a of the first terminal device 2A to the second terminal device 2B. The communication device 28a is an example of a communication device in this disclosure that transmits an image of a participant, having the appearance defined by the display mode selected in the input, to another terminal device.

[0036] The above options include at least one of the first and second modes shown below as display modes. The first mode is a display mode in which, among the images of the body of the first participant U1 included in the captured image acquired by the first terminal device 2A, the visibility of a predetermined body part is reduced while the visibility of other body parts is not reduced. The second mode is a display mode that reduces the visibility of other types of objects in the image of objects included in the captured image acquired by the first terminal device 2A, without reducing the visibility of a predetermined type of object.

[0037] This allows the first participant U1 to set the appearance of the first participant display image displayed on the second terminal device 2B to a desired appearance by appropriately selecting one of the display mode options. For example, when the first participant U1 uses the image captured by camera 25a as the first participant display image, they can select which parts of the person in the captured image to display or hide on the second terminal device 2B by appropriately selecting the display mode. Also, for example, the first participant U1 can select which types of objects in the captured image to display or hide on the second terminal device 2B by selecting the display mode. Therefore, compared to conventional systems that apply masking to the entire face image or background image of a person detected from an captured image, the remote dialogue system 1 can improve the degree of freedom in setting the appearance of the first participant display image displayed on the second terminal device 2B, which is the communication partner.

[0038] [2. Configuration of the first terminal device] Figure 3 is a block diagram showing the functional configuration of the first terminal device 2A that constitutes the remote dialogue system 1. As described above, the first terminal device 2A corresponds to the terminal device in this disclosure. The first terminal device 2A has a first processing unit 21a, to which a display device 22a, a keyboard 23a, a mouse 24a, a camera 25a, a microphone 26a, a speaker 27a, and a communication device 28a are connected.

[0039] The keyboard 23a and mouse 24a are input devices used by the first participant U1 for input operations. In addition to the keyboard 23a and mouse 24a, the first terminal device 2A may also be equipped with a touch sensor (not shown) as an input device, which is mounted on top of the display panel of the display device 22a. The communication device 28a is a transceiver that communicates with the server 3 via the communication network 4 using wired or wireless communication.

[0040] The first processing unit 21a is a computer comprising a first processor 40 and a first memory 46. The first memory 46 is composed of a magnetic memory device, a semiconductor memory element such as flash ROM (Read Only Memory), or other types of non-volatile memory devices. The first memory 46 may also include RAM (Random Access Memory) which constitutes the work area of ​​the first processor 40.

[0041] The first memory 46 stores the first program 47 and display mode information 48. The display mode information 48 will be described later.

[0042] The first processor 40 is composed of, for example, a CPU (Central Processing Unit), an MPU (Micro-processing unit), or other integrated circuits. The first processor 40 includes, as functional elements or functional units, a communication control unit 41, a selection and acquisition unit 42, a selection determination unit 43, an image generation unit 44, and a registration unit 45.

[0043] These functional elements of the first processor 40 are realized, for example, by the first processor 40 executing a first program 47 stored in the first memory 46. The first program 47 can be stored in any computer-readable storage medium. Alternatively, all or part of the above functional elements of the first processor 40 can be configured by hardware, each including one or more electronic circuit components.

[0044] The communication control unit 41 communicates with the second terminal device 2B via the server 3, enabling remote dialogue between the first terminal device 2A and the second terminal device 2B using voice and images.

[0045] Specifically, the communication control unit 41 converts the sound collected by the microphone 26a into digital audio data and generates audio data based on the digital audio data. The communication control unit 41 receives input from the first participant U1 using the input device, which is the keyboard 23a or mouse 24a, and generates operation data based on the received input. The communication control unit 41 receives image data of the first participant display image displayed on the second terminal device 2B from the image generation unit 44. The communication control unit 41 transmits the terminal data D11, which includes the audio data, operation data, and image data, to the server 3 via the communication device 28a.

[0046] The communication control unit 41 also receives server data D21 transmitted by server 3 via the communication device 28a. Based on the display data contained in the server data D21, the communication control unit 41 controls the display device 22a to display a remote dialogue screen on the display device 22a. The remote dialogue screen displayed on the display device 22a may include a participant display area IAb that displays an image of the second participant U2 using the second terminal device 2B. The communication control unit 41 also outputs audio from the speaker 27a based on the audio data contained in the server data D21.

[0047] The selection and acquisition unit 42 receives an input to select one display mode from a set of options that include multiple display modes that define the appearance of the first participant display image displayed on the second terminal device 2B.

[0048] Specifically, for example, the selection acquisition unit 42 displays the above options on the display device 22a, and the first participant U1 can input a selection of one display mode from the above options using the keyboard 23a or mouse 24a. The selection acquisition unit 42 receives the above input via the keyboard 23a or mouse 24a. The keyboard 23a or mouse 24a is an example of an input device that receives input to select one display mode from the above options in this disclosure.

[0049] As described above, the above options include at least one of the first mode and the second mode shown below as display modes. The first mode is a display mode in which, among the images of the body of the first participant U1 included in the captured image acquired by the first terminal device 2A, the visibility of a predetermined body part is reduced while the visibility of other body parts is not reduced. The second mode is a display mode that reduces the visibility of other types of objects in the image of objects included in the captured image acquired by the first terminal device 2A, without reducing the visibility of a predetermined type of object.

[0050] The above options are determined by the option determination unit 43. The selection determination unit 43 determines the above selection from a group of display modes, including a plurality of predefined display modes that define the appearance of the image representing the participant, based on the purpose information transmitted from the second terminal device 2B. The purpose information is information indicating the purpose of the call made using the first terminal device 2A and the second terminal device 2B, and in this embodiment, it is input to the second terminal device 2B by the second participant U2 and notified to the first terminal device 2A. The purpose information will be described in more detail later.

[0051] Figure 8 shows an example of a group of display modes that includes a predefined set of multiple display modes. The group of display modes shown in Figure 8 can be predetermined and stored in the first memory 46 as display mode information 48. Figure 8 shows the display modes included in the display mode group in a table format. The columns in the table shown in Figure 8 are numbered 1st, 2nd, ..., 10th column, starting from the leftmost column and moving to the right.

[0052] The first column is the display mode identification number, and the contents of display mode "0", display mode "1", display mode "2", ..., display mode "9" are shown in columns 2 through 10 of the corresponding row.

[0053] The second column defines the type of image used for the first participant display image displayed in the second terminal device 2B. As described above, the first participant display image is an image representing the first participant U1 displayed in the second terminal device 2B, and in this embodiment, as shown in Figure 7, it is displayed in the participant display area IAa on the display device 22b of the second terminal device 2B. In the second column, "captured image" refers to an image of the area S1 captured by camera 25a. "Predetermined image" may be a predetermined image such as a logo, icon, avatar, or animation representing the first participant U1.

[0054] Columns 3 through 10 specify the visibility of captured images when "captured images" are specified in column 2. Specifically, columns 3 through 10 specify whether or not the visibility of the captured image is partially reduced.

[0055] Columns 3 through 6 define the visibility of images of the first participant U1, namely the "caregiver" and the "child receiving care," from the images captured by camera 25a. Here, "child receiving care" refers to child C, who is one of the first participants U1. "Caregiver" refers to another first participant U1, namely person N, who provides care for child C, for example, a member of child C's family.

[0056] The third and fourth columns specify the "maintained portion" and "reduced portion" of the image of caregiver N, respectively. Similarly, the fifth and sixth columns specify the "maintained portion" and "reduced portion" of the image of medically fragile child C. The "maintained portion" specifies the part of the captured image that will be maintained without a decrease in visibility, while the "reduced portion" specifies the part of the image that will have reduced visibility. In other words, the "maintained portion" shown in the third column is a predetermined part of the image of the first participant U1 whose visibility will not be reduced.

[0057] For example, in display mode "1", the third and fourth columns specify that, in the image of caregiver N's ​​body, the visibility of parts other than the face, which are designated as such, should be maintained, while the visibility of the face should be reduced. Also, in display mode "7", the fifth and sixth columns specify that, in the image of medically fragile child C's body, the visibility of the face and designated body parts should not be reduced, while the visibility of other body parts should be reduced.

[0058] Here, among the display modes shown in Figure 8, display modes "1", "6", "7", and "8" that reduce the visibility of a part of the image of the body of the first participant U1, which is the caregiver N or the child requiring medical care C, while not reducing the visibility of other body parts, correspond to the first mode.

[0059] In this embodiment, the display modes corresponding to the first mode include, as in display modes "6" and "8", those in which a predetermined body part of the image of the first participant U1 that does not reduce visibility includes a part of the first participant U1 other than the face.

[0060] Columns 7 and 8 define the visibility of objects captured in the image from camera 25a, respectively. Column 7, "Types to Maintain," specifies the types of objects that should be displayed without reducing their visibility. Column 8, "Types to Reduce," specifies the types of objects that should be displayed with reduced visibility. For example, in display mode "2," columns 7 and 8 specify that the visibility of objects belonging to the toy category in the captured image should be reduced, while the visibility of other types of objects should be maintained.

[0061] Here, among the display modes shown in Figure 8, display modes "2", "6", and "8", which do not reduce the visibility of a predetermined type of object among multiple objects included in the captured image, but reduce the visibility of other types of objects, correspond to the second mode in this disclosure. That is, display modes "6" and "8" correspond to both the first mode and the second mode in this disclosure.

[0062] Columns 9 and 10 define the "maintained portion" and the "reduced portion" of the background captured by the camera 25a, respectively. For example, in display mode "3", columns 9 and 10 specify that the visibility of the background portion in a predetermined direction within the captured image is reduced, while the visibility of the background portion in other directions is maintained.

[0063] Referring to Figure 3, the selection determination unit 43 determines a mode selection that includes at least one display mode corresponding to the second mode described above, for example, when the purpose of the call indicated by the purpose information transmitted from the second terminal device 2B relates to the use of an object. Specifically, for example, when the purpose of the call relates to the use of care devices, such as "learning the operation procedure of the device," the selection determination unit 43 determines a mode selection that includes a second mode that does not reduce the visibility of the "care devices." Such a second mode that does not reduce the visibility of the "care devices" corresponds, for example, to display modes "6" and "8" in the group of display modes shown in Figure 7. This ensures that mode options, including display modes, are appropriately determined based on the purpose of the call related to the use of the object.

[0064] Figure 9 shows an example of a display mode selection screen that the selection acquisition unit 42 displays on the display device 22a when it receives input to select one display mode from the options determined by the option determination unit 43. Figure 9 is an example of a display mode selection screen when the purpose of the call, indicated by the purpose information transmitted from one of the second terminal devices 2B, is "to learn the operating procedures of the device".

[0065] In the illustrated example, the selection determination unit 43 determines a mode selection that includes three display modes: display modes "6" and "8" corresponding to the second mode described above, plus display mode "9". Display mode "9" is a display mode that does not display the captured image and can always be included in the mode selection. This allows the first participant U1 to prevent the second participant U2 from seeing the captured image from camera 25a by selecting display mode "9" if they do not want to select any of the other display modes in the mode selection.

[0066] In the input screen shown in Figure 9, the selection acquisition unit 42 displays a list 70 containing the three display modes, "6", "8", and "9", which are included in the mode selection determined by the selection determination unit 43. On this input screen, the first participant U1 can input the selection of the desired display mode by selecting one desired display mode from the list 70 using the keyboard 23a or mouse 24a and pressing the OK button 71. As a result, the selection acquisition unit 42 accepts the input of the selection of the desired display mode.

[0067] Alternatively, if the first participant U1 does not accept any of the displayed mode options, they can press the "Not Selectable" button 72. When the "Not Selectable" button 72 is pressed, the selection acquisition unit 42 sends a "Not Selectable" notification to the second terminal device 2B. The "Not Selectable" notification may be sent to the server 3 as part of the operation data of the terminal data D11, and then sent from the server 3 to the second terminal device 2B.

[0068] The communication control unit 51 of the second terminal device 2B, described later, displays a notification on the display device 22b indicating that the "unselectable" button 72 was pressed in the first terminal device 2A, upon receiving the "unselectable" notification. As described later, in this case, the second participant U2 can take action such as changing the purpose of the call.

[0069] Referring to Figure 3, the image generation unit 44 generates a first participant display image to be displayed on the second terminal device 2B. In this embodiment, in particular, the image generation unit 44 generates an image with an appearance defined by the selected display mode as the first participant display image, based on the input of a selection of one display mode received by the selection acquisition unit 42.

[0070] The image generation unit 44 sends the image data of the generated image to the communication control unit 41. The communication control unit 41 transmits the image data from the communication device 28a to the second terminal device 2B via the server 3. As a result, the display device 22b of the second terminal device 2B displays the first participant display image with the appearance defined by the selected display mode.

[0071] For example, when the selected display mode is the first mode, the image generation unit 44 performs image processing on the image captured by the camera 25a to generate an image of the first participant U1 in which the visibility of a predetermined body part is reduced while the visibility of other body parts is not reduced. The image generation unit 44 then sends the image data of the generated image to the communication control unit 41, and the image data is transmitted from the communication control unit 41 to the second terminal device 2B via the server 3.

[0072] Furthermore, for example, when the selected display mode is the second mode, the image generation unit 44 performs image processing on the image captured by the camera 25a to generate an image that reduces the visibility of other types of objects without reducing the visibility of a predetermined type of object. The image generation unit 44 then sends the image data of the generated image to the communication control unit 41, and the image data is transmitted from the communication control unit 41 to the second terminal device 2B via the server 3.

[0073] Furthermore, for example, if the selected display mode is one that does not use images captured by the camera 25a, such as display mode "9" shown in Figure 7, the image generation unit 44 generates image data of a predetermined image. The image generation unit 44 then sends the generated image data to the communication control unit 41, and the image data is transmitted from the communication control unit 41 to the second terminal device 2B via the server 3. As described above, the predetermined image may be an image such as a logo, icon, avatar, or animation representing the first participant U1.

[0074] Furthermore, the identification of human body parts and specific types of objects within captured images can be performed using image processing methods such as pattern matching and feature point matching, as well as machine learning. Machine learning methods may include, for example, R-CNN (Region-based Convolutional Neural Networks) and YOLO (You Only Look Once). Alternatively, machine learning methods may include, for example, SSD (Single Shot Multibox Detector). Furthermore, processing that reduces the visibility of an captured image could, for example, involve lowering the resolution of the image portion being processed and blurring it. Alternatively, processing that reduces visibility could involve superimposing an avatar or substitute image onto the image portion being processed.

[0075] Prior to the start of remote dialogue between the first terminal device 2A and the second terminal device 2B, the registration unit 45 creates an acceptable list, which is a list of display modes that the user who will become the first participant U1 may allow in the remote dialogue.

[0076] For example, when the registration unit 45 receives instructions from a user who will be the first participant U1 via the keyboard 23a or mouse 24a, it displays a registration screen on the display device 22a. The registration screen may include a list of all display modes included in the display mode group and a message prompting the user to select an acceptable display mode. The user inputs the selection of at least one acceptable display mode from the list of display modes displayed on the display device 22a using the keyboard 23a or mouse 24a. Based on the above input, the registration unit 45 generates an acceptable list that includes the acceptable display mode selected by the user.

[0077] The registration unit 45 sends the created allow list to the server 3. The registration management unit 33 of the server 3, which will be described later, receives the allow list and stores the received allow list in association with the communication address of the first terminal device 2A, which is the source of the list, and the identification information of the user who will be the first participant U1. As a result, the allow list is registered in the server 3 as user information 37, as will be described later.

[0078] The registered user information 37 may be used to match a user who will be a second participant U2 with a user who can be a first participant U1 who is capable of providing remote communication in line with the purpose of the call desired by that user.

[0079] [3. Configuration of the second terminal device] Figure 4 is a block diagram showing the functional configuration of the second terminal device 2B. The second terminal device 2B has a second processing unit 21b, to which a display device 22b, a keyboard 23b, a mouse 24b, a camera 25b, a microphone 26b, a speaker 27b, and a communication device 28b are connected.

[0080] The keyboard 23b and mouse 24b are input devices used by the second participant U2 for input operations. In addition to the keyboard 23b and mouse 24b, the second terminal device 2B may also be equipped with a touch sensor (not shown) as an input device, which is mounted on top of the display panel of the display device 22b.

[0081] The communication device 28b is a transceiver that communicates with the server 3 via the communication network 4 using wired or wireless communication.

[0082] The second processing unit 21b is a computer comprising a second processor 50 and a second memory 54. The second memory 54 is composed of a magnetic memory device, a semiconductor memory element such as flash ROM, or other types of non-volatile memory devices. The second memory 54 may also include RAM that constitutes the work area of ​​the second processor 50.

[0083] The second memory 54 stores the second program 55 and the objective information 56. The objective information 56 will be described later.

[0084] The second processor 50 is composed of, for example, a CPU, an MPU, or other integrated circuit. The second processor 50 includes, as functional elements or functional units, a communication control unit 51, a target acquisition unit 52, and an extraction unit 53.

[0085] These functional elements of the second processor 50 are realized, for example, by the second processor 50 executing a second program 55 stored in the second memory 54. The second program 55 can be stored in any computer-readable storage medium. Alternatively, all or part of the above functional elements of the second processor 50 can be configured by hardware, each including one or more electronic circuit components.

[0086] The communication control unit 51 communicates with the first terminal device 2A via the server 3 to realize remote dialogue with the first terminal device 2A using voice and images.

[0087] Specifically, the communication control unit 51 converts the sound collected by the microphone 26b into digital audio data and generates audio data based on the digital audio data. The communication control unit 51 receives input from the input device, which is the keyboard 23b or mouse 24b, and generates operation data based on the received input. The communication control unit 51 generates image data for an image to be output in the participant display area IAb, which displays the second participant U2 on the first terminal device 2A. The image output in the participant display area IAb may be an image captured by the camera 25a or a predetermined image. The predetermined image may be an image such as a logo, icon, avatar, or animation representing the second participant U2. The communication control unit 41 transmits the terminal data D12, which includes the voice data, operation data, and image data, to the server 3 via the communication device 28b.

[0088] The communication control unit 51 also receives server data D22 transmitted by server 3 via the communication device 28b. Based on the display data contained in the server data D22, the communication control unit 51 controls the display device 22b to display a remote dialogue screen as shown in Figure 7. The remote dialogue screen displayed on the display device 22b may include a participant display area IAa that displays a first participant display image, which is an image representing the first participant U1. The communication control unit 51 also outputs audio from speaker 27b based on the audio data contained in the server data D22.

[0089] When the communication control unit 51 receives a notification of "unavailable" from the first terminal device 2A, it displays a notification screen on the display device 22b that includes a notification that the "unavailable" button 72 has been pressed on the first terminal device 2A. This notification screen may display an end button to indicate the end of the current remote conversation with the first terminal device 2A, and a continue button to indicate the continuation of the current remote conversation. The second participant U2 presses either the end button or the continue button to instruct the second terminal device 2B on the next action. When the continue button is pressed, the objective acquisition unit 52, which will be described later, displays an input screen for objective information on the display device 22b.

[0090] The purpose acquisition unit 52 acquires purpose information, which is information indicating the purpose of the call that the second participant U2, using the second terminal device 2B, makes with the first terminal device 2A. When the objective acquisition unit 52 receives an instruction from the second participant U2 via the keyboard 23b or mouse 24b to receive objective information, or when the aforementioned continue button is pressed, it displays an objective information input screen, including a list of objective information options, on the display device 22b. For example, when the second participant U2 starts a remote interaction with the first terminal device 2A, it instructs the objective acquisition unit 52 to accept objective information. The objective information options displayed on the objective information input screen may be predetermined and stored in the second memory 54 as objective information 56.

[0091] The second participant U2 uses the keyboard 23b or mouse 24b to select one desired objective from the list of options included in the objective information input screen. The objective acquisition unit 52 then acquires the objective information requested by the second participant U2.

[0092] Figure 10 shows an example of a target information input screen displayed on the display device 22b by the target acquisition unit 52. In the illustrated example, a target list 75 showing five predetermined target information items is displayed as a list of target information options. On this input screen, the second participant U2 can input the selection of the desired target information item by selecting one desired target information item from the target list 75 using the keyboard 23b or mouse 24b and pressing the OK button 76. The target acquisition unit 52 then accepts the input of the selection of the desired target information item.

[0093] The objective acquisition unit 52 transmits the acquired objective information to the first terminal device 2A via the server 3. Specifically, the objective acquisition unit 52 sends the acquired objective information to the communication control unit 51. The communication control unit 51 generates operation data indicating the objective information and transmits terminal data D12, which includes the generated operation data and the aforementioned voice data and image data, to the server 3 via the communication device 28b. Based on the terminal data D12, the server 3 generates server data D21 which includes the selected objective information as operation data and transmits the generated server data D21 to the first terminal device 2A.

[0094] Referring to Figure 4, the extraction unit 53 extracts users who can become the first participant U1 and who can provide the target information desired by the user who will become the second participant U2, prior to the start of the remote dialogue. Specifically, the user who will become the second participant U2 first inputs an extraction instruction using the keyboard 23b or mouse 24b. The extraction instruction can be input, in accordance with the prior art, for example, by pressing a predefined shortcut key on the keyboard 23b to display a menu screen, and then inputting from the displayed menu screen.

[0095] In response to the input of an extraction instruction, the extraction unit 53 displays a list of target information on the display device 22b, similar to the screen shown in Figure 10, and obtains the target information desired by the user who will be the second participant U2. The extraction unit 53 sends an extraction request to the server 3 that includes the obtained target information as extraction conditions. The information provision unit 34 of the server 3, described later, refers to the user information 37 stored in the third memory 35, extracts users whose display modes that can correspond to the above target information are included in the acceptable list, and returns a list of the extracted users. The extraction unit 53 displays the list of extracted users returned by the server 3 on the display device 22b.

[0096] The second participant U2 can select one user from the list displayed on the display device 22b and send an invitation email for remote dialogue to the selected user's first terminal device 2A. If the selected user accepts participation in the remote dialogue, the user who is the second participant U2 can smoothly conduct a remote dialogue with the selected user using the second terminal device 2B and the first terminal device 2A.

[0097] [4. Server Configuration] Figure 5 is a block diagram showing the functional configuration of server 3. Server 3 comprises a third processing unit 30 and a communication device 38. The communication device 38 is a transceiver that communicates with the first terminal device 2A and the second terminal device 2B via the communication network 4 using wired or wireless communication.

[0098] The third processing unit 30 is a computer comprising a third processor 31 and a third memory 35. The third memory 35 is composed of a magnetic memory device, a semiconductor memory element such as flash ROM, or other types of non-volatile memory devices. The third memory 35 may also include RAM that constitutes the work area of ​​the third processor 31. The third memory 35 stores the third program 36 and user information 37.

[0099] The third processor 31 is composed of, for example, a CPU, an MPU, or other integrated circuit. The third processor 31 includes, as functional elements or functional units, a communication control unit 32, a registration management unit 33, and an information provision unit 34.

[0100] These functional elements of the third processor 31 are realized, for example, by the third processor 31 executing the third program 36 stored in the third memory 35. The third program 36 can be stored in any computer-readable storage medium. Alternatively, all or part of the above functional elements of the third processor 31 can be configured by hardware, each including one or more electronic circuit components.

[0101] The communication control unit 32 communicates with the first terminal device 2A and the second terminal device 2B via the communication device 38, and realizes remote dialogue using voice and images between the first terminal device 2A and the second terminal device 2B.

[0102] Specifically, the communication control unit 32 receives terminal data D1 transmitted by the terminal device 2 via the communication device 38. Based on the image data contained in the terminal data D1 received from each terminal device 2, the communication control unit 32 generates display data that includes all of the captured images or predetermined images contained in each image data. The display data includes image data displayed in the participant display areas IAa and IAb on each terminal device 2.

[0103] Furthermore, Server 3 generates integrated voice data based on the voice data contained in the terminal data D1 received from each terminal device 2. Additionally, Communication Control Unit 32 generates configuration data by integrating the information contained in the operation data contained in each operation data, based on the operation data contained in the terminal data D1 received from each terminal device 2. The configuration data includes the target information acquired by the second terminal device 2B.

[0104] The communication control unit 32 generates server data D2, which includes the generated display data, integrated voice data, and setting data. The communication control unit 32 transmits the generated server data D2 to each terminal device 2 via the communication device 38. As described above, the integrated voice data included in each server data D2 transmitted to each terminal device 2 does not need to include the voice acquired at the terminal device 2 to which it is transmitted. Furthermore, the target information acquired at the second terminal device 2B only needs to be included in the server data D21 transmitted to the first terminal device 2A, and does not need to be included in the server data D22 transmitted to the second terminal device 2B.

[0105] The registration management unit 33 receives the above-mentioned acceptable lists from multiple first terminal devices 2A that can be communication partners for remote dialogue with the second terminal device 2B. As described above, the acceptable lists are created by the registration unit 45 of each first terminal device 2A. The acceptable list is a list of display modes that the user using that first terminal device 2A can accept in remote dialogue. The registration management unit 33 stores user information 37, which associates the received acceptable list with the communication address of the first terminal device 2A that sent it and the user's identification information, in the third memory 35. As a result, the received acceptable lists are registered with the server 3.

[0106] In response to receiving an extraction request from the second terminal device 2B, the information provision unit 34 extracts users who can become the first participant U1 corresponding to the target information included in the received extraction request from the user information 37 stored in the third memory 35. Specifically, the information provision unit 34 extracts users from the user information 37 whose display modes corresponding to the target information are included in the acceptable list. Then, it sends the list of extracted users to the second terminal device 2B, which is the source of the extraction request.

[0107] [5. Operation of the Remote Dialogue System] Next, we will explain the operation of the remote dialogue system 1. Figure 6 is a flowchart showing the processing steps of the image processing method executed by the remote dialogue system 1. The image processing method shown in Figure 6 is executed collaboratively by the first processing unit 21a of the first terminal device 2A, the second processing unit 21b of the second terminal device 2B, and the third processing unit 30 of the server 3, all of which are computers included in the remote dialogue system 1.

[0108] The process shown in Figure 6 begins when the power to the first terminal device 2A, the second terminal device 2B, and the server 3 is turned on. When processing begins, the objective acquisition unit 52 of the second terminal device 2B first determines whether or not an instruction to receive objective information has been input from the second participant U2 (S100). If no instruction to receive objective information has been input (S100, NO), the objective acquisition unit 52 returns to step S100 and repeats the process, waiting for an instruction to receive objective information to be input. On the other hand, if an instruction to receive objective information has been input (S100, YES), the objective acquisition unit 52 acquires the objective information input by the second participant U2 (S102). The objective acquisition unit 52 transmits the acquired objective information to the first terminal device 2A via the server 3, and the first terminal device 2A receives the objective information.

[0109] Next, the selection determination unit 43 of the first terminal device 2A determines a display mode from the group of display modes shown in Figure 7 based on the received objective information (S104). As described above, each of the display modes defines the appearance of the first participant display image displayed on the second terminal device 2B.

[0110] The selection acquisition unit 42 of the first terminal device 2A displays a display mode selection screen, including a list of options, on the display device 22a, as shown in Figure 9. The selection acquisition unit 42 then determines whether or not the "unselectable" button 72 has been pressed on the display mode selection screen (S106).

[0111] Then, when the OK button on the display mode selection screen is pressed, that is, when the unselectable button 72 is not pressed (S106, NO), the selection acquisition unit 42 accepts the input to select one display mode on the display mode selection screen (S108). Specifically, the selection of the above one display mode is performed when the first participant U1 selects one of the display modes included in the selection determination unit 43 determined in step S104 and presses the OK button.

[0112] Next, the image generation unit 44 of the first terminal device 2A determines whether the selected display mode specifies that the captured image should be used as the display image for the first participant (S110). If the selected display mode specifies that the captured image should be used (S110, YES), the image generation unit 44 starts generating an image with the appearance defined by the selected display mode (S112).

[0113] For example, when a display mode corresponding to the first mode is selected in step S108, the image generation unit 44 performs image processing on the captured image to generate an image that reduces the visibility of a predetermined part of the first participant U1's body while not reducing the visibility of other body parts. Furthermore, for example, when a display mode corresponding to the second mode is selected in step S108, the image generation unit 44 performs image processing on the captured image to generate an image in which the visibility of a predetermined type of object is not reduced, while the visibility of other types of objects is reduced.

[0114] On the other hand, in step S110, if the selected display mode specifies the use of a predetermined image, that is, if it does not specify the use of an captured image (S110, NO), the image generation unit 44 generates a predetermined image (S114). As described above, the predetermined image may be an image such as a logo, icon, avatar, or animation representing the first participant U1. Here, the avatar or animation may be one in which the character's face or mouth moves in sync with the voice of the first participant U1 using prior art.

[0115] Next, the communication control unit 41 of the first terminal device 2A starts transmitting the image generated in step S112 or S114 to the second terminal device 2B (S116). As a result, the captured image with the appearance defined by the selected display mode is displayed as the first participant display image in the participant display area IAa on the display device 22b of the second terminal device 2B.

[0116] Next, the communication control unit 41 determines whether or not an instruction to terminate the remote dialogue has been given (S118). In accordance with the prior art, an instruction to terminate the remote dialogue is given, for example, by pressing the terminate button 62 on the remote dialogue screen displayed on the first terminal device 2A and the second terminal device 2B, as shown in Figure 7.

[0117] Then, when an instruction to terminate the remote dialogue is received (S118, YES), the communication control unit 41 and the image generation unit 44 terminate the transmission and generation of the image (S120), notify the first terminal device 2A of the termination of the remote dialogue, and then terminate this process. On the other hand, when there is no instruction to terminate the remote dialogue (S118, NO), the communication control unit 41 returns to step S116 and repeats the process, waiting for the termination of the current remote dialogue.

[0118] On the other hand, if the unselectable button 72 is pressed in step S106 (S106, YES), the selection acquisition unit 42 sends an unselectable notification to the second terminal device 2B. As described above, in response to receiving the unselectable notification, the communication control unit 51 of the second terminal device 2B displays a notification screen including an end button and a continue button on the display device 22b.

[0119] The communication control unit 51 of the second terminal device 2B determines whether the end button has been pressed and whether the current remote dialogue with the first terminal device 2A has been instructed to end (S122). If the end button has been pressed and the remote dialogue has been instructed to end (S122, YES), the communication control unit 51 notifies the first terminal device 2A of the termination of the remote dialogue and terminates this process.

[0120] On the other hand, when the "Continue" button is pressed on the notification screen, that is, when the termination of the remote dialogue is not instructed (S122, NO), the communication control unit 51 returns to step S102. This allows the second participant U2 to change the objective information in step S102. Subsequently, in step S104, the display mode selection corresponding to the changed objective information is determined.

[0121] [6. Effects of the Embodiment] As described above, the remote dialogue system 1 executes an image processing method. The image processing method includes receiving an input to select one display mode from a set of options that define the appearance of a first participant display image displayed on a second terminal device 2B which is communicably connected to the first terminal device 2A. Here, the first participant display image refers to an image showing the first participant U1 using the first terminal device 2A. The image processing method also includes transmitting an image showing the first participant U1 with the appearance defined by the one display mode selected in the input to the second terminal device 2B. The options include at least one of the following first and second modes as display modes. The first mode is a display mode that reduces the visibility of a predetermined body part while not reducing the visibility of other body parts in the image of the first participant U1's body included in the captured image acquired by the first terminal device 2A. The second mode is a display mode that reduces the visibility of other types of objects while not reducing the visibility of a predetermined type of object in the image of objects included in the captured image acquired by the first terminal device 2A.

[0122] This allows, for example, the first participant U1 to appropriately select one of the display mode options to hide only a specific part of the first participant U1's body or only a specific type of object in the background of the first participant display image. In other words, the remote dialogue system 1 offers greater flexibility in setting the appearance of the first participant display image displayed on the second terminal device 2B compared to conventional systems that uniformly reduce the visibility of the participant's face or other parts of their body.

[0123] Furthermore, when the above-mentioned image processing method includes the first mode and the selected display mode is the first mode, the above-mentioned image processing method includes transmitting to the second terminal device 2B an image obtained by applying image processing to the captured image, which reduces the visibility of a predetermined part of the image of the first participant U1's body while not reducing the visibility of other body parts.

[0124] According to this, by processing the captured image, it is possible to easily generate an image that reduces the visibility of a predetermined part of the body of the first participant U1 while not reducing the visibility of other body parts, and transmit it to the second terminal device 2B.

[0125] Furthermore, when the above-mentioned image processing method includes the second mode and the selected display mode is the second mode, the above-mentioned image processing method includes transmitting to a second terminal device an image obtained by applying image processing to the captured image, which reduces the visibility of other types of objects without reducing the visibility of a predetermined type of object.

[0126] According to this, by processing the captured image, it is possible to easily generate an image that reduces the visibility of other types of objects without reducing the visibility of a predetermined type of object, and transmit it to the second terminal device 2B.

[0127] The above image processing method further includes acquiring purpose information, which is the purpose of a call made using the first terminal device 2A and the second terminal device 2B. The above image processing method also includes determining the above choice from a group of display modes, which includes a predefined set of multiple display modes, that define the appearance of the image representing the participant, based on the above purpose information.

[0128] This allows the display mode selection to be determined according to the target information, enabling the first participant U1 to quickly select the appropriate display mode that aligns with the target information.

[0129] Furthermore, in the above image processing method, if the purpose indicated by the above purpose information is related to the use of the object, an option including the second mode is determined.

[0130] According to this, the options, including the display mode, are appropriately determined according to the purpose information related to the use of the object.

[0131] Furthermore, in the image processing method described above, the predetermined body parts of the image of the first participant U1 that do not reduce visibility in the first mode include parts of the first participant U1's body other than the face.

[0132] This allows for greater flexibility in configuring the appearance of the first participant display image shown on the second terminal device, compared to conventional systems that distinguish between the participant's face and other parts of their body and uniformly reduce the visibility of the other parts.

[0133] Furthermore, the first program 47 is a program executed by the first terminal device 2A, which is a computer. The first program 47 causes the first terminal device 2A to accept input to select one display mode from a selection of multiple display modes that define the appearance of the first participant display image displayed on the second terminal device 2B, which is connected to the first terminal device 2A in a communicative manner. Here, the first participant display image refers to an image representing the first participant U1 using the first terminal device 2A. The first program 47 also causes the first terminal device 2A to transmit the image with the appearance defined by the display mode selected in the above input to the second terminal device 2B.

[0134] Furthermore, the above options include at least one of the following first and second modes as display modes. The first mode is a display mode in which, among the images of the body of the first participant U1 included in the captured image acquired by the first terminal device 2A, the visibility of a predetermined body part is reduced without reducing the visibility of other body parts. The second mode is a display mode in which, among the images of objects included in the captured image acquired by the first terminal device 2A, the visibility of a predetermined type of object is not reduced while the visibility of other types of objects is reduced.

[0135] This allows, for example, the first participant U1 to appropriately select one of the display mode options to hide only a specific part of the first participant U1's body or only a specific type of object in the background of the first participant display image. In other words, the remote dialogue system 1 offers greater flexibility in setting the appearance of the first participant display image displayed on the second terminal device 2B compared to conventional systems that uniformly reduce the visibility of the participant's face or other parts of their body.

[0136] Furthermore, the first terminal device 2A is a terminal device used by the first participant U1 to conduct remote dialogue. The first terminal device 2A communicates with another terminal device, the second terminal device 2B. The first terminal device 2A includes an input device that accepts input to select one display mode from a selection of multiple display modes that define the appearance of the first participant display image displayed on the second terminal device 2B. Here, the first participant display image refers to an image showing the first participant U1 using the first terminal device 2A. The input device may be, for example, a keyboard 23a or a mouse 24a. The first terminal device 2A also includes a communication device 28a that transmits an image showing the first participant U1 with the appearance defined by the display mode selected in the input to the second terminal device 2B.

[0137] Furthermore, the above options include at least one of the following first and second modes as display modes. The first mode is a display mode in which, among the images of the body of the first participant U1 included in the captured image acquired by the first terminal device 2A, the visibility of a predetermined body part is reduced without reducing the visibility of other body parts. The second mode is a display mode in which, among the images of objects included in the captured image acquired by the first terminal device 2A, the visibility of a predetermined type of object is not reduced while the visibility of other types of objects is reduced.

[0138] This allows, for example, the first participant U1 to appropriately select one of the display mode options to hide only a specific part of the first participant U1's body or only a specific type of object in the background of the first participant display image. In other words, the remote dialogue system 1 offers greater flexibility in setting the appearance of the first participant display image displayed on the second terminal device 2B compared to conventional systems that uniformly reduce the visibility of the participant's face or other parts of their body.

[0139] [7. Other Embodiments] The above embodiments illustrate a specific example of the application of the present invention, and the present invention is not limited thereto. In the following, components similar to those described in the embodiments are denoted by the same reference numerals, and their detailed descriptions are omitted.

[0140] In the embodiment described above, the first terminal device 2A is equipped with one display device 22a. In another embodiment, the first terminal device 2A may be equipped with an additional display device that displays at least a second participant display image. For example, the caregiver N, who is the first participant U1, may use the display device 22a, and the medically fragile child C may use the additional display device. If the medically fragile child C is bedridden or otherwise unable to move freely, the additional display device is preferably a projector that can project an image onto a wall or ceiling that is easily visible to the medically fragile child C.

[0141] In the embodiment described above, the selection acquisition unit 42 accepts the input to select one display mode that defines the appearance of the first participant display image from among the mode options determined by the selection determination unit 43 based on the objective information. In another embodiment, the selection acquisition unit 42 may display a list of all display modes included in the display mode group on the display device 22a and accept the input to select one display mode from among all of the above display modes.

[0142] In another embodiment, the display mode selected in the first terminal device 2A may be notified to the second terminal device 2B, and the content defined by the display mode may be displayed on the display device 22b.

[0143] In another embodiment, when the selection acquisition unit 42 receives input for selecting a display mode, it may display a submenu for specifying details about the selected display mode and accept input for one or more specifications regarding the details. For example, if display mode "7" shown in Figure 8 is selected, a submenu may be displayed for specifying a "predetermined area" defined in the "maintenance area" of "child in care". Also, for example, if display mode "4" or "5" shown in Figure 8 is selected, a submenu may be displayed for specifying the range of a "wide area" or "narrow area" defined in the "reduction area" of "background". This makes it possible to specify in detail the variations in the image appearance defined by a single display mode.

[0144] In the embodiment described above, the selection determination unit 43 and the image generation unit 44 are provided by the first terminal device 2A. In another embodiment, the selection determination unit 43 may be provided by the second terminal device 2B or the server 3, and the mode selection determined by the selection determination unit 43 may be notified to the first terminal device 2A. Also in another embodiment, the image generation unit 44 may be provided by the second terminal device 2B or the server 3. The first terminal device 2A can notify the second terminal device 2B or the server 3, which is equipped with the image generation unit 44, of the selected display mode acquired by the selection acquisition unit 42. This reduces the processing load on the first terminal device 2A, allowing the first participant U1 to easily participate in remote conversations using a smartphone or other device with a CPU that does not have high processing power as the first terminal device 2A.

[0145] In the embodiment described above, user information 37, which is information about a user who may be the first participant, is stored in the server 3. In another embodiment, user information 37 may be stored in the second terminal device 2B. This allows the extraction unit 53 of the second terminal device 2B to quickly extract users who can correspond to the selected target information.

[0146] In the embodiment described above, the user information 37 stored by the server 3 includes an acceptable list, which is a list of display modes that a user who may become a first participant may accept. In another embodiment, the user information 37 may include, in addition to the acceptable list, conditions for the date and time when the user is available to make a call. The conditions for the date and time when a call is possible may be, for example, information on the days of the week or time slots when a call is possible. The registration unit 45 of the first terminal device 2A can obtain the above date and time conditions from the user who will become the first participant U1 via the keyboard 23a or mouse 24a when creating the acceptable list. As a result, the user who will become the second participant U2 can extract from the users registered in the user information 37 a user who can handle the desired purpose information and can handle the desired date and time for a call, thereby enabling more appropriate matching between users.

[0147] In the embodiment described above, as an example, the remote dialogue system 1 is used to facilitate communication between a family with a child requiring medical care and a nursing student. The applications of the remote dialogue system 1 of the present invention are not limited to those described above, but are suitable for applications where it is desirable to reduce the visibility of images of specific objects or specific body parts of people that are of interest in remote dialogue, while reducing the visibility of other parts of the image captured by the first terminal device 2A. Such applications may include, for example, online medical consultations, various counseling sessions, rehabilitation and other training, exercises, and user testing of equipment. The counseling described above may be, for example, online consultations on the health of people or animals, interior design of houses, plant growth, etc., by sharing images of the subject. In particular, the remote dialogue system of the present invention is suitable for applications where there are situations in which the use of equipment or tools is instructed, confirmed, or monitored online.

[0148] In the embodiment described above, the first participants U1 using the first terminal device 2A consist of two people: a child requiring medical care C and a caregiver N. However, depending on the purpose of communication, there may be one or three or more participants. Furthermore, the appearance of the images defined by the display mode can be arbitrarily defined according to the purpose, objective, or type of communication conducted by the remote dialogue system 1. In addition, if there are two or more first participants U1, the display mode may be defined to maintain or reduce the visibility of images of various body parts for each first participant U1.

[0149] Furthermore, the operational step units shown in Figure 6 are divided according to the main processing content in order to facilitate understanding of the operation of the remote dialogue system 1, and this disclosure is not limited by the way the processing units are divided or the names of the processing units. Depending on the processing content, it may be divided into many step units. Alternatively, it may be divided so that one step unit includes many processes. Also, the order of the steps may be changed as appropriate, as long as it does not hinder the intent of this disclosure.

[0150] [8. Summary of this disclosure] A summary of this disclosure is provided below.

[0151] (Note 1) Image processing method comprising: receiving an input to select one display mode from a selection of multiple display modes that define the appearance of an image showing a participant using the first terminal device displayed on a second terminal device which is communicably connected to the first terminal device; and transmitting an image showing the participant having the appearance defined by one of the display modes to the second terminal device, wherein the selection includes at least one of a first mode which is a display mode that reduces the visibility of a predetermined body part but not other body parts among the images of the participant's body included in an image captured by the first terminal device, and a second mode which is a display mode that reduces the visibility of a predetermined type of object but not other types of objects among the images of objects included in the image captured.

[0152] This allows participants using the first terminal device to appropriately select one display mode from the options, thereby hiding only specific body parts or only specific types of objects in the background of the image representing the participant displayed on the second terminal device. In other words, compared to conventional systems that uniformly reduce the visibility of the participant's face or other parts of their body, this increases the degree of freedom in configuring the appearance of the image representing the participant displayed on the second terminal device.

[0153] (Note 2) The image processing method according to Appendix 1, wherein the option includes the first mode, and when the one display mode is the first mode, the method includes transmitting to the second terminal device an image obtained by applying image processing to the captured image, which reduces the visibility of a predetermined body part of the participant's body while not reducing the visibility of other body parts.

[0154] This makes it possible to easily generate an image by processing the captured image that reduces the visibility of a specific part of the participant's body while not reducing the visibility of other parts of the body, and then transmit it to a second terminal device.

[0155] (Note 3) The image processing method according to Appendix 1 or 2, wherein the option includes the second mode, and when one of the display modes is the second mode, the method includes transmitting to the second terminal device an image obtained by applying image processing to the captured image, which does not reduce the visibility of a predetermined type of object but reduces the visibility of other types of objects.

[0156] This makes it possible to easily generate an image by processing the captured image that reduces the visibility of other types of objects without reducing the visibility of a predetermined type of object, and then transmit it to a second terminal device.

[0157] (Note 4) An image processing method according to any one of the appendices 1 to 3, comprising: acquiring purpose information indicating the purpose of a call made using the first terminal device and the second terminal device; and determining the selection from a group of display modes, which includes a plurality of predefined display modes that define the appearance of an image representing a participant, based on the purpose information.

[0158] This allows the display mode selection to be determined according to the purpose of the call, enabling participants to quickly select the appropriate display mode for their call.

[0159] (Note 5) The image processing method according to Appendix 4, wherein the option including the second mode is determined when the purpose indicated by the purpose information relates to the use of an object.

[0160] This ensures that the appropriate options, including display modes, are determined based on the purpose of the call related to the use of the object.

[0161] (Note 6) The image processing method according to any one of the appendices 1 to 5, wherein in the first mode, the predetermined body part includes a part of the participant's body other than their face.

[0162] This allows for greater flexibility in configuring the appearance of the participant image displayed on the second terminal device, compared to conventional systems that distinguish between the participant's face and other parts of their body, uniformly reducing the visibility of the other parts.

[0163] (Note 7) A program to be executed by a computer, the program to receive input to select one display mode from a selection of display modes that define the appearance of an image of a participant using the first terminal device displayed on a second terminal device which is communicably connected to the first terminal device, and to transmit an image of the participant having the appearance defined by one of the display modes to the second terminal device, wherein the selection includes at least one of the following display modes: a first mode which is a display mode that reduces the visibility of a predetermined body part of an image of the participant's body included in an image captured by the first terminal device without reducing the visibility of other body parts, and a second mode which is a display mode that reduces the visibility of a predetermined type of object without reducing the visibility of other types of objects among images of objects included in the image captured.

[0164] This allows participants using the first terminal device to select a display mode from the options available, thereby hiding only specific body parts of the participant or only specific types of objects in the background of the image representing the participant. In other words, compared to conventional systems that uniformly reduce the visibility of the participant's face or other parts of their body, this system offers greater flexibility in configuring the appearance of the image representing the participant displayed on the second terminal device.

[0165] (Note 8) A terminal device comprising: an input device that accepts input for selecting one display mode from a selection of multiple display modes that define the appearance of an image of a participant displayed on another terminal device that is a communication partner in a remote dialogue; and a communication device that transmits an image of the participant having the appearance defined by one of the display modes to the other terminal device, wherein the selection includes at least one of two display modes: a first mode which is a display mode that reduces the visibility of a predetermined body part of the image of the participant's body included in an image captured as an image of the participant, without reducing the visibility of other body parts; and a second mode which is a display mode that reduces the visibility of a predetermined type of object without reducing the visibility of other types of objects among images of objects included in the image captured.

[0166] This allows participants to select a display mode from a set of options, thereby hiding only specific body parts of themselves or only certain types of objects in the background of the image displayed on the other terminal device of their communication partner. In other words, compared to conventional systems that uniformly reduce the visibility of a participant's face or other parts of their body, this system offers greater flexibility in configuring the appearance of the image representing the participant displayed on other terminal devices. [Explanation of Symbols]

[0167] 1...Remote dialogue system, 2...Terminal device, 2A...First terminal device, 2B...Second terminal device, 3...Server, 4...Communication network, 21a...First processing unit, 21b...Second processing unit, 22a, 22b...Display device, 23a, 23b...Keyboard, 24a, 24b...Mouse, 25a, 25b...Camera, 26a, 26b...Microphone, 27a, 27b...Speaker, 28a, 28b, 38...Communication device, 30...Third processing unit, 31...Third processor, 32, 41, 51...Communication control unit, 33...Registration management unit, 34...Information provision unit, 35...Third memory, 36...Third program, 37...User information, 40...First processor, 42...Selection acquisition unit, 43 ...Choice determination unit, 44...Image generation unit, 45...Registration unit, 46...First memory, 47...First program, 48...Display mode information, 50...Second processor, 52...Purpose acquisition unit, 53...Extraction unit, 54...Second memory, 55...Second program, 56...Purpose information, 60...Camera button, 61...Microphone button, 62...Exit button, 70...List, 71, 76...OK button, 72...Unselectable button, 75...Purpose list, C...Child requiring medical care, D1, D11, D12...Terminal data, D2, D21, D22...Server data, IAa, IAb...Participant display area, N...Caregiver, S, S1, S2...Base, U1...First participant, U2...Second participant.

Claims

1. The system accepts input to select one display mode from a set of options that include multiple display modes defining the appearance of an image showing a participant using the first terminal device, which is displayed on a second terminal device that is communicatively connected to the first terminal device. Transmitting an image representing the participant, having the appearance defined by one of the aforementioned display modes, to the second terminal device; Includes, The above options are, A first mode is a display mode in which, among the images of the participant's body included in the captured image acquired by the first terminal device, the visibility of a predetermined body part is reduced while the visibility of other body parts is not reduced, and A second mode is a display mode in which, among the images of objects included in the captured image, the visibility of a predetermined type of object is not reduced while the visibility of other types of objects is reduced. Including at least one of the display modes, Image processing methods.

2. The aforementioned options include the first mode, When the one display mode is the first mode, an image obtained by applying image processing to the captured image, which reduces the visibility of a predetermined body part of the participant's body while not reducing the visibility of other body parts, is transmitted to the second terminal device. including, The image processing method according to claim 1.

3. The aforementioned options include the second mode, When the first display mode is the second mode, an image obtained by applying image processing to the captured image, which reduces the visibility of other types of objects without reducing the visibility of a predetermined type of object, is transmitted to the second terminal device. including, The image processing method according to claim 1.

4. To acquire purpose information indicating the purpose of a call made using the first terminal device and the second terminal device, Based on the aforementioned objective information, the selection is determined from a group of display modes, which includes a predefined set of multiple display modes that define the appearance of the image representing the participant. including, The image processing method according to claim 1.

5. If the purpose indicated by the purpose information relates to the use of the object, the selection including the second mode is determined. The image processing method according to claim 4.

6. In the first mode, the predetermined body part includes a part of the participant's body other than their face. The image processing method according to any one of claims 1 to 5.

7. A program that is executed by a computer, To the aforementioned computer, The system accepts input to select one display mode from a set of options that include multiple display modes defining the appearance of an image showing a participant using the first terminal device, which is displayed on a second terminal device that is communicatively connected to the first terminal device. Transmitting an image representing the participant, having the appearance defined by one of the aforementioned display modes, to the second terminal device; Make it run, The above options are, A first mode is a display mode in which, among the images of the participant's body included in the captured image acquired by the first terminal device, the visibility of a predetermined body part is reduced while the visibility of other body parts is not reduced, and A second mode is a display mode in which, among the images of objects included in the captured image, the visibility of a predetermined type of object is not reduced while the visibility of other types of objects is reduced. Including at least one of the display modes, program.

8. An input device that accepts input to select one display mode from a set of options including multiple display modes that define the appearance of an image of a participant displayed on another terminal device that is the communication partner in a remote dialogue, A communication device that transmits an image representing the participant, having the appearance defined by one of the aforementioned display modes, to the other terminal device; Equipped with, The above options are, A first mode is a display mode in which, among the images of the participant's body included in the captured image used as an image to represent the participant, the visibility of a predetermined body part is reduced while the visibility of other body parts is not reduced, and A second mode is a display mode in which, among the images of objects included in the captured image, the visibility of a predetermined type of object is not reduced while the visibility of other types of objects is reduced. Including at least one of the display modes, Terminal device.

Images