Telepresence system, image processing method, and image processing program

The image processing device enhances telepresence by synthesizing real-time rear and foreground images to improve the sense of realism and presence in virtual interactions.

JP7589608B2Active Publication Date: 2024-11-26TOYOTA JIDOSHA KK
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
JP2021046730
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2021-03-22
Publication Date
2024-11-26
Estimated Expiration
2041-03-22

AI Technical Summary

Technical Problem

Existing telepresence technology lacks realism as it displays the background of the person at the second base, detracting from the sense of presence.

Method used

An image processing device that acquires and synthesizes real-time image information from the rear direction of the display unit with the foreground image of the person at the second base, creating a composite image that enhances the sense of presence.

Benefits of technology

Improves the sense of realism by aligning the background with the viewer's line of sight, providing a more immersive virtual presence experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007589608000001
    Figure 0007589608000001
  • Figure 0007589608000002
    Figure 0007589608000002
  • Figure 0007589608000003
    Figure 0007589608000003
Patent Text Reader

Abstract

To provide an image processing device capable of improving a sense of reality in a virtual state in which a person at a second base is at a first base.SOLUTION: An image processing device (7) according to an embodiment of the present disclosure that processes an image displayed on a display unit of a telepresence robot (2) located at a first base (H1) includes a background image acquisition unit (71) that acquires in real time image information obtained by photographing the back direction of the display unit, a person image acquiring unit (72) that acquires in real time image information including a person at a second base (H2) remote from a first base (H1), an extraction unit (73) that extracts an area in which the person is reflected from the image information including the person, and a synthesizing unit (74) that synthesizes the image information of the extracted person and the image information of the back direction of the display unit.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to an image processing device, a telepresence system, an image processing method, and an image processing program, and more particularly to an image processing device, a telepresence system, an image processing method, and an image processing program that process images displayed on a display unit of a telepresence robot, for example. [Background technology]

[0002] 2. Description of the Related Art In recent years, telepresence technology has become common, enabling a conversation to be carried out by making one person feel as if he or she is in the same place as the other person, even though the other person is in a remote location.

[0003] Such telepresence technology, as disclosed in Patent Document 1, for example, displays an image including a person at a second location that is remote from the first location on the display unit of a telepresence robot placed at a first location, thereby creating a virtual state in which the person at the second location appears to be at the first location. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2016-208416 A Summary of the Invention [Problem to be solved by the invention]

[0005] The applicant has found the following problem. The telepresence technology of Patent Document 1 displays an image of a person at a second location, as well as the background of the person, on the display unit of a telepresence robot. Therefore, in a virtual state in which a person at the second location is in the first location, there is a problem that the sense of realism is poor.

[0006] The present disclosure has been made in consideration of such problems, and realizes an image processing device, a telepresence system, an image processing method, and an image processing program that can improve the sense of realism in a virtual state in which a person at a second location appears to be at a first location. [Means for solving the problem]

[0007] An image processing device according to an embodiment of the present disclosure is an image processing device that processes an image displayed on a display unit of a telepresence robot disposed at a first base, a background image acquisition unit that acquires image information obtained by photographing a back surface of the display unit in real time; a person image acquisition unit that acquires image information including a person at a second location that is remote from the first location in real time; an extraction unit that extracts an area in which the person is reflected from image information including the person; a synthesis unit that synthesizes the extracted image information of the person with image information in a rear direction of the display unit; Equipped with.

[0008] A telepresence system according to one aspect of the present disclosure includes: The image processing device described above; a person photographing unit that photographs an area including a person at the second location; a telepresence robot provided with the display unit; a background photographing unit that photographs a back surface of the display unit; Equipped with.

[0009] The above-mentioned telepresence system preferably includes, as the display unit, a first display unit and a second display unit that displays an image facing in a different direction from the first display unit.

[0010] It is preferable that the above-mentioned telepresence system includes, as the background capture unit, a first capture unit that captures the rear direction of the first display unit and a second capture unit that captures the rear direction of the second display unit.

[0011] The above-mentioned telepresence system preferably includes a 360-degree camera as the background photographing unit.

[0012] An image processing method according to an embodiment of the present disclosure is an image processing method for processing an image displayed on a display unit of a telepresence robot disposed at a first base, comprising: acquiring image information obtained by photographing a rear surface of the display unit in real time; acquiring image information including a person at a second location remote from the first location in real time; extracting an area in which the person is captured from image information including the person; a step of synthesizing the extracted image information of the person with image information of the rear direction of the display unit; Equipped with.

[0013] An image processing program according to an embodiment of the present disclosure is an image processing program for processing an image displayed on a display unit of a telepresence robot disposed at a first base, A process of acquiring image information obtained by photographing a rear surface of the display unit in real time; A process of acquiring image information including a person at a second location remote from the first location in real time; A process of extracting an area in which the person is reflected from image information including the person; A process of synthesizing the extracted image information of the person and image information in the rear direction of the display unit; to be executed by the computer. Effect of the Invention

[0014] According to the present disclosure, it is possible to realize an image processing device, a telepresence system, an image processing method, and an image processing program that can improve the sense of realism in a virtual state in which a person at a second location appears to be at a first location. [Brief description of the drawings]

[0015] [Figure 1]1 is a diagram showing a state in which an image is displayed on a display unit of a telepresence robot using the telepresence system of embodiment 1. FIG. [Diagram 2] FIG. 2 is a block diagram showing a control system of the telepresence system of the first embodiment. [Diagram 3] FIG. 4 is a flowchart showing the flow of image processing executed by the image processing device according to the first embodiment. [Figure 4] FIG. 11 is a block diagram showing a control system of the telepresence system of the second embodiment. [Diagram 5] FIG. 11 is a front perspective view showing a telepresence robot according to a second embodiment. [Figure 6] FIG. 11 is a rear perspective view showing a telepresence robot according to a second embodiment. [Figure 7] FIG. 11 is a block diagram showing a control system of the telepresence system of the third embodiment. [Figure 8] FIG. 11 is a front perspective view showing a telepresence robot according to a third embodiment. [Figure 9] FIG. 2 is a diagram illustrating an example of a hardware configuration included in the image processing apparatus. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] Hereinafter, specific embodiments to which the present disclosure is applied will be described in detail with reference to the drawings. However, the present disclosure is not limited to the following embodiments. In addition, the following description and drawings are appropriately simplified for clarity of explanation.

[0017] <Embodiment 1> Fig. 1 is a diagram showing a state in which an image is displayed on a display unit of a telepresence robot using the telepresence system of the present embodiment Fig. 2 is a block diagram showing a control system of the telepresence system of the present embodiment.

[0018] As shown in FIGS. 1 and 2, a telepresence system 1 according to the present embodiment includes a telepresence robot 2, a person photographing unit 3, a microphone 4, a speaker 5, a display unit 6, and an image processing device .

[0019] 1, the telepresence robot 2 is placed at a first base H1 and connected to a network 10. Here, the network 10 is, for example, the Internet, and is constructed by a telephone line network, a wireless communication path, Ethernet (registered trademark), etc.

[0020] 1 and 2, the telepresence robot 2 includes a moving unit 21, a display unit 22, a foreground photographing unit 23, a background photographing unit 24, a microphone 25, a speaker 26, and a control unit 27. The moving unit 21 includes a dolly 21a, a pair of driving wheels 21b and driven wheels 21c rotatably provided on the dolly 21a, and a pair of driving mechanisms 21d that rotationally drive the driving wheels 21b.

[0021] The driving mechanism 21d includes a motor, a reducer, etc. The driving mechanism 21d is driven based on control information received from the control unit 27 to rotate the driving wheels 21b, thereby enabling the telepresence robot 2 to move forward, move backward, and rotate.

[0022] This allows the telepresence robot 2 to move to any position. Here, the telepresence robot 2 may move autonomously based on environmental information acquired by an environmental camera (not shown), based on an external operation, or based on the movement of a person P2 at a second location H2 that is remote from the first location H1.

[0023] The above-described configuration of the moving unit 21 is merely an example, and is not limited thereto. For example, the number of driving wheels 21b and driven wheels 21c of the moving unit 21 may be arbitrary, and any known mechanism may be used as long as the telepresence robot 2 can be moved to an arbitrary position.

[0024] The display unit 22 includes a display device such as a liquid crystal display or an organic EL (Electro Luminescence) display. The display unit 22 displays a composite image of image information of a person P2 at the second location H2 and image information of a rear view of the display unit 22. Here, in FIG. 1, the image of the rear view of the display unit 22 is simplified and shown by hatching.

[0025] The display unit 22 is fixed to the moving unit 21 so as to be able to display an image toward the front of the telepresence robot 2. In other words, the display surface of the display unit 22 is disposed facing toward the front of the telepresence robot 2.

[0026] In this case, the display surface of the display unit 22 may be a curved surface that protrudes toward the front of the telepresence robot 2. In addition, the display unit 22 may have a size that allows at least the upper body of a person P2 located at the second base H2 to be displayed in approximately actual size.

[0027] The foreground photographing unit 23 includes a camera having an imaging element such as a CMOS (Complementary Metal Oxide Semiconductor). The foreground photographing unit 23 is fixed to the moving unit 21 so as to be able to photograph the front direction of the display unit 22 (i.e., the display direction of the image) in real time. In other words, the foreground photographing unit 23 has a photographing range that is the area in front of the telepresence robot 2. In this case, it is preferable that the foreground photographing unit 23 is disposed near the display unit 22.

[0028] The background photographing unit 24 includes a camera having an imaging element such as a CMOS. The background photographing unit 24 is fixed to the moving unit 21 so as to be able to photograph the rear direction of the display unit 22 (i.e., the opposite direction to the display direction of the image) in real time. In other words, the background photographing unit 24 has a photographing range that is the rear area of ​​the telepresence robot 2.

[0029] The microphone 25 collects sounds inside the first location H1. The speaker 26 outputs sounds inside the second location H2. The microphone 25 and the speaker 26 are provided in the moving unit 21. However, it is sufficient that the microphone 25 and the speaker 26 are disposed at the first location H1. The control unit 27 controls the drive mechanism 21d of the moving unit 21, the display unit 22, the foreground photographing unit 23, the background photographing unit 24, the microphone 25, and the speaker 26.

[0030] 1, the person photographing unit 3 is disposed at the second site H2 so as to be able to photograph the interior of the second site H2. Therefore, when a person P2 is present in the photographing area of ​​the person photographing unit 3, the person photographing unit 3 photographs an image including the person P2.

[0031] The microphone 4 is disposed inside the second location H2. The microphone 4 collects sounds inside the second location H2. The speaker 5 is disposed inside the second location H2. The speaker 5 outputs sounds inside the first location H1. The display unit 6 includes a display device such as a liquid crystal display or an organic EL display.

[0032] The display unit 6 is disposed inside the second site H2, and displays an image of the front direction of the display unit 22 at the first site H1 captured by the foreground capturing unit 23 of the telepresence robot 2. In this case, it is preferable that the display unit 6 is disposed near the person capturing unit 3. The person capturing unit 3, microphone 4, speaker 5, and display unit 6 are connected to a network 10.

[0033] 2, the image processing device 7 includes a background image acquisition unit 71, a person image acquisition unit 72, an extraction unit 73, and a synthesis unit 74, and is connected to the network 10. The background image acquisition unit 71 acquires in real time image information of the rear view of the display unit 22 captured by the background capture unit 24 of the telepresence robot 2. However, it is sufficient for the background image acquisition unit 71 to acquire image information of the rear view of the display unit 22, and may include, for example, the background capture unit 24.

[0034] Person image acquisition unit 72 acquires in real time image information including person P2 located at second location H2 photographed by person photographing unit 3. However, person image acquisition unit 72 only needs to be able to acquire image information including person P2 located at second location H2, and may be equipped with person photographing unit 3, for example.

[0035] The extraction unit 73 extracts an area in which the person P2 at the second location H2 is reflected from image information including the person P2 at the second location H2. The extraction unit 73 can extract an area in which the person P2 is reflected by, for example, applying chromakey processing to the image information including the person P2 at the second location H2. However, the method for extracting an area in which the person P2 at the second location H2 is reflected from the image information including the person P2 at the second location H2 is not limited, and any known method can be used.

[0036] The synthesis unit 74 synthesizes the extracted image information of the person P2 with the image information in the rear direction of the display unit 22. At this time, the synthesis unit 74 may synthesize the mutual image information such that the extracted image of the person P2 is disposed at a preset position with respect to the image in the rear direction of the display unit 22.

[0037] Here, the synthesis unit 74 can synthesize the extracted image information of person P2 and the image information of the rear direction of the display unit 22, for example, by chromakey synthesis, but the method of synthesizing the mutual image information is not limited, and any known method can be used.

[0038] Next, a flow of image processing executed by the image processing device 7 of this embodiment will be described. Fig. 3 is a flow chart showing the flow of image processing executed by the image processing device of this embodiment.

[0039] First, the background image acquisition unit 71 acquires in real time image information of the rear direction of the display unit 22 captured by the background photographing unit 24 of the telepresence robot 2. At the same time, the person image acquisition unit 72 acquires in real time image information including the person P2 located at the second site H2 captured by the person photographing unit 3 (S1).

[0040] Next, the extraction unit 73 extracts an area in which the person P2 at the second location H2 is reflected from image information including the person P2 (S2). Then, the synthesis unit 74 synthesizes the extracted image information of the person P2 with image information in the rear direction of the display unit 22 (S3). This makes it possible to generate a synthetic image of the extracted image information of the person P2 and image information in the rear direction of the display unit 22.

[0041] Next, a state will be described in which a person P1 at a first location H1 and a person P2 at a second location H2 are having a conversation using the telepresence system 1 of this embodiment. The person P2 at the second location H2 looks at an image including the person P1 at the first location H1 displayed on the display unit 6, listens to the sound inside the first location H1 output from the speaker 5, and moves and speaks toward the display unit 6.

[0042] At this time, at the second location H2, the person photographing unit 3 photographs an image including the person P2 present at the second location H2, and the microphone 4 collects sounds within the second location H2 including the voice of the person P2.

[0043] A person P1 at a first location H1 looks at a composite image displayed on a display unit 22 of a telepresence robot 2, hears sounds inside the second location H2 including the voice of a person P2 output from a speaker 26, and moves and speaks toward the display unit 22.

[0044] At this time, at the first site H1, the foreground photographing unit 23 photographs an image including the person P1 at the first site H1, and the background photographing unit 24 photographs an image in the rear direction of the display unit 22. In addition, the microphone 25 collects sounds within the first site H1, including the voice of the person P1.

[0045] This allows the person P1 at the first site H1 and the person P2 at the second site H2 to converse with each other in a virtual state in which the person P2 at the second site H2 is at the first site H1. At this time, a composite image in which the extracted image information of the person P2 and the image information in the rear direction of the display unit 22 are combined is displayed on the display unit 22 of the telepresence robot 2.

[0046] In this way, the telepresence system 1, image processing device 7, and image processing method of this embodiment display a composite image on the display unit 22 of the telepresence robot 2 by combining the extracted image information of the person P2 and the image information of the rear direction of the display unit 22.

[0047] Therefore, in a virtual state in which a person P2 at a second location H2 is at a first location H1, the background of the person P2 in the composite image can be made substantially the same as the background in the line of sight of the person P1 at the first location H1, thereby improving the sense of realism.

[0048] Furthermore, image information of the rear view of the display unit 22 of the telepresence robot 2 and image information including the person P2 at the second base H2 are acquired in real time, so the sense of realism can be further improved.

[0049] Here, if the display surface of the display unit 22 of the telepresence robot 2 is a curved surface that protrudes toward the front of the telepresence robot 2, the person P2 is displayed on the display surface in a rounded manner, further enhancing the sense of realism.

[0050] In addition, the display unit 22 of the telepresence robot 2 in this embodiment is disposed so as to display an image toward the front of the telepresence robot 2, but the direction in which the display unit 22 displays an image is not limited, and may be, for example, to the side of the telepresence robot 2 or to the rear of the telepresence robot 2. In this case, the position of the background photographing unit 24 may be changed as appropriate according to the position of the display unit 22.

[0051] <Embodiment 2> Fig. 4 is a block diagram showing a control system of the telepresence system of this embodiment. Fig. 5 is a front perspective view showing the telepresence robot of this embodiment. Fig. 6 is a rear perspective view showing the telepresence robot of this embodiment.

[0052] The telepresence system 101 of this embodiment has substantially the same configuration as the telepresence system 1 of the first embodiment, so duplicated explanations will be omitted and the same elements will be described using the same reference numerals.

[0053] The telepresence system 101 differs from the telepresence system 1 of the first embodiment in that the telepresence robot 102 includes a second display unit 103 in addition to the display unit (first display unit) 22.

[0054] The second display unit 103 includes a display device such as a liquid crystal display or an organic EL display. The second display unit 103 displays a composite image of image information of a person P2 at the second location H2 and image information of a rear view of the second display unit 103.

[0055] The second display unit 103 is fixed to the moving unit 21 so as to be able to display an image toward the rear of the telepresence robot 2. Therefore, the display surface of the second display unit 103 is disposed facing toward the rear of the telepresence robot 2.

[0056] The synthesis unit 105 of the image processing device 104 synthesizes the extracted image information of the person P2 at the second base H2 with the image information of the rear direction of the first display unit 22 captured by the background capture unit 24 to generate a composite image to be displayed on the first display unit 22 of the telepresence robot 2.

[0057] In addition, the synthesis unit 105 of the image processing device 104 synthesizes the extracted image information of the person P2 at the second base H2 with the image information of the rear direction of the second display unit 103 captured by the foreground capturing unit 23 to generate a composite image to be displayed on the second display unit 103 of the telepresence robot 2.

[0058] As a result, the first display unit 22 of the telepresence robot 2 can display a composite image of the extracted image information of the person P2 at the second base H2 and the image information of the rear direction of the first display unit 22 captured by the background capture unit 24.

[0059] In addition, the second display unit 103 can display a composite image of the extracted image information of the person P2 at the second location H2 and the image information of the rear direction of the second display unit 103 captured by the foreground capture unit 23.

[0060] Therefore, a person P1 at the first location H1 can interact with a person P2 at the second location H2 not only from the front of the telepresence robot 2 but also from the rear of the telepresence robot 2, and for example, multiple people can interact with a person P2 at the second location H2 from both the front and rear of the telepresence robot 2.

[0061] Here, the display unit 6 of the second location H2 may display both the image captured by the foreground photographing unit 23 and the image captured by the background photographing unit 24, or may display only the image in which the person P1 at the first location H1 is reflected.

[0062] In addition, the second display unit 103 of the telepresence robot 102 in this embodiment is positioned so as to display an image toward the rear of the telepresence robot 102, but it is sufficient that the direction in which the second display unit 103 displays an image is different from the direction in which the first display unit 22 displays an image.

[0063] In addition, the telepresence robot 102 may be provided with a foreground photographing unit 23 and a background photographing unit 24 for interacting using the first display unit 22, and may be provided with another foreground photographing unit and a background photographing unit for interacting using the second display unit 103.

[0064] <Embodiment 3> Fig. 7 is a block diagram showing a control system of the telepresence system of this embodiment. Fig. 8 is a front perspective view showing the telepresence robot of this embodiment. The telepresence system 201 of this embodiment has substantially the same configuration as the telepresence system 101 of the second embodiment, so duplicated explanations will be omitted and the same reference numerals will be used for the same elements.

[0065] As shown in FIG. 7, the telepresence system 201 differs from the telepresence system 101 of the second embodiment in that the telepresence robot 202 includes a 360-degree camera 203 instead of the foreground photographing unit 23 and the background photographing unit 24.

[0066] 8, the 360-degree camera 203 is fixed to the upper end of the first display unit 22 and the second display unit 103. A general 360-degree camera can be used as the 360-degree camera 203, and for example, a dual-lens type 360-degree camera can be used.

[0067] This makes it possible to obtain image information of the entire area surrounding the telepresence robot 202, and using this image information, the synthesis unit 205 of the image processing device 204 can generate a synthetic image to be displayed on the first display unit 22 by synthesizing the extracted image information of the person P2 at the second location H2 with the image information of the rear direction of the first display unit 22 captured by the 360-degree camera 203.

[0068] In addition, the synthesis unit 205 of the image processing device 204 can generate a synthetic image by synthesizing the extracted image information of the person P2 at the second base H2 with the image information of the rear view of the second display unit 103 captured by the 360-degree camera 203 as an image to be displayed on the second display unit 103 of the telepresence robot 2.

[0069] In such a telepresence system 201, the number of image capturing units can be reduced compared to the telepresence system 101 of the second embodiment, and the telepresence system 201 can be configured at low cost.

[0070] <Other embodiments> The image processing device according to the above embodiment may have the following hardware configuration. Fig. 9 is a diagram showing an example of the hardware configuration included in the image processing device. As described above in the various embodiments, the processing procedure in the image processing device, the present disclosure may also be embodied as an image processing method.

[0071] The device shown in Fig. 9 includes a processor 301 and a memory 302, together with an interface 303. A portion of the configuration of the image processing device described in the above embodiment is realized by the processor 301 reading and executing a program stored in the memory 302. In other words, this program is a program for causing the processor 301 to function as a portion of the configuration of the image processing device. It can be said that this program is a program for causing the image processing device to execute processing in its configuration or in a portion of it.

[0072] The above-mentioned program can be stored using various types of non-transitory computer readable media and supplied to a computer (including a computer including an information notification device). The non-transitory computer readable medium includes various types of tangible storage media. Examples of the non-transitory computer readable medium include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives) and magneto-optical recording media (e.g., magneto-optical disks). Further, examples include CD-ROM (Read Only Memory), CD-R, and CD-R / W. Further, examples include semiconductor memory (e.g., mask ROM, PROM, EPROM, flash ROM, RAM). The program may also be supplied to a computer by various types of transitory computer readable media. Examples of the transitory computer readable medium include electric signals, optical signals, and electromagnetic waves. The transitory computer readable medium can supply the program to a computer via a wired communication path such as an electric wire and an optical fiber, or via a wireless communication path.

[0073] The present disclosure is not limited to the above-described embodiment, and can be modified as appropriate without departing from the spirit and scope of the present disclosure. [Explanation of symbols]

[0074] 1. Telepresence System 2. Telepresence Robot 21 moving part, 21a carriage, 21b driving wheel, 21c driven wheel, 21d driving mechanism 22 Display section 23 Foreground Photography Section 24 Background Photography Department 25. Mike 26 Speaker 27 Control Unit 3. Portrait Photography Section 4. Microphone 5 Speakers 6 Display section 7 Image Processing Device 71 Background image acquisition section 72 Person image acquisition unit 73 Extraction part 74 Synthesis section 10 Network 101 Telepresence System 102 Telepresence Robot 103 Second display unit 104 Image Processing Device 105 Synthesis section 201 Telepresence System 202 Telepresence Robot 203 360 degree camera 204 Image Processing Device 205 Synthesis Section 301 Processor 302 Memory 303 Interface H1 First base H2 Second base P1, P2 people

Claims

1. An image processing device for processing images displayed on a display unit of a telepresence robot located at a first base, comprising: a background image acquisition unit for acquiring image information captured in the rear direction of the display unit in real time; a person image acquisition unit for acquiring image information including a person at a second base that is remote from the first base in real time; an extraction unit for extracting an area in which the person is reflected from the image information including the person; and a synthesis unit for synthesizing the extracted image information of the person with image information in the rear direction of the display unit; a person photographing unit that photographs an area including a person at the second location; a telepresence robot provided with the display unit; a background photographing unit that photographs a back surface of the display unit; A telepresence system comprising: The display unit includes a first display unit and a second display unit that displays an image in a direction different from that of the first display unit, A telepresence system having, as the background capture unit, a first capture unit that captures an image in the rear direction of the first display unit, and a second capture unit that captures an image in the rear direction of the second display unit.

2. An image processing device for processing images displayed on a display unit of a telepresence robot arranged at a first base, comprising: a background image acquisition unit for acquiring image information captured in the rear direction of the display unit in real time; a person image acquisition unit for acquiring image information including a person at a second base that is remote from the first base in real time; an extraction unit for extracting an area in which the person is reflected from the image information including the person; and a synthesis unit for synthesizing the extracted image information of the person with image information in the rear direction of the display unit; a person photographing unit that photographs an area including a person at the second location; a telepresence robot provided with the display unit; a background photographing unit that photographs a back surface of the display unit; A telepresence system comprising: The display unit includes a first display unit and a second display unit that displays an image in a direction different from that of the first display unit, A telepresence system having a 360-degree camera as the background photographing unit.

3. An image processing method for processing an image displayed on a display unit of a telepresence robot disposed at a first base, comprising: acquiring image information in real time obtained by capturing a rear view of a first display unit of the telepresence robot by a first capturing unit of the telepresence robot; acquiring, in real time, image information captured by a second imaging unit of the telepresence robot, of a rear direction of a second display unit that displays an image facing a different direction from the first display unit of the telepresence robot; acquiring image information including a person at a second location remote from the first location in real time; extracting an area in which the person is captured from image information including the person; a step of combining the extracted image information of the person and image information of the rear direction of the first display unit and displaying the combined image on the first display unit; a step of combining the extracted image information of the person with image information of the rear direction of the second display unit and displaying the combined image on the second display unit; An image processing method comprising:

4. An image processing program for processing an image displayed on a display unit of a telepresence robot disposed at a first base, A process of acquiring image information of a rear direction of a first display unit of the telepresence robot in real time based on image information of the entire area around the telepresence robot acquired by a 360-degree camera of the telepresence robot; A process of acquiring in real time image information of a rear direction of a second display unit that displays an image in a different direction from the first display unit of the telepresence robot based on image information of the entire area around the telepresence robot; A process of acquiring image information including a person at a second location remote from the first location in real time; A process of extracting an area in which the person is reflected from image information including the person; a process of synthesizing the extracted image information of the person and image information of the rear direction of the first display unit and displaying the synthesized image on the first display unit; a process of synthesizing the extracted image information of the person and image information of the second display unit in the rear direction thereof and displaying the synthesized image on the second display unit; An image processing program that causes a computer to execute the above.

Citation Information

Patent Citations

  • Video communication apparatus

    JP2006333301A

  • Video image communication system

    JP2006352749A

  • Apparatus for extracting and synthesizing subject image

    JP2014175837A

  • Communication system and control method therefor

    JP2016208416A

  • Image composition device and image composition program

    JP2017143376A