Terminal equipment

Abstract

To provide a terminal apparatus that improves reality in virtual face-to-face communication.SOLUTION: In a call system, a terminal apparatus includes a communication section, a transparent panel that displays an image and accepts a drawing, a display section superimposed on the transparent panel, an imaging section disposed near the display section to capture an image of a user, and a control section that communicates by the communication section. The control section receives, from another terminal apparatus, information for displaying a model image representing another user on the basis of a captured image of another user who uses another terminal apparatus, and information on a drawn image drawn by another user using a drawing tool on a transparent panel of another terminal apparatus, and displays the model image on the display section of the terminal apparatus and a hand side image of the hand of another user holding the drawing tool and the drawn image on the transparent panel of the terminal apparatus.SELECTED DRAWING: Figure 5B

Description

[Technical field]

[0001] The present disclosure relates to a terminal device. [Background technology]

[0002] There is known a device that displays images and the like on a transparent touch panel, outputs various information to a user, and accepts input of various information. A technology has been proposed for using such a device as a videophone terminal that enables video calls via a network. For example, Patent Document 1 discloses a videophone device that projects video light onto a large holographic screen, displays full-color or monochrome moving or still images, and functions as an information terminal. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2003-005617 A Summary of the Invention [Problem to be solved by the invention]

[0004] 2. Description of the Related Art In a technology in which users communicate with each other by exchanging captured images, drawings, and the like using a terminal device having a transparent touch panel, there is room for improving the reality of communication.

[0005] The present disclosure provides a terminal device etc. that enables improved reality in communication using a transparent touch panel. [Means for solving the problem]

[0006] A terminal device in the present disclosure is a terminal device having a communication unit, a transparent panel that displays images and accepts drawings, a display unit that is overlaid on the transparent panel, an imaging unit that is positioned near the display unit and images the user, and a control unit that communicates via the communication unit, wherein the control unit receives from the other terminal device information for displaying a model image representing another user based on an image captured by the other user using the other terminal device, and information on a drawing image that the other user draws with a drawing tool on the transparent panel of the other terminal device, and displays the model image on the display unit of the terminal device, and an image of the other user's hand holding the drawing tool and the drawing image on the transparent panel of the terminal device. Effect of the Invention

[0007] According to the terminal device and the like of the present disclosure, it is possible to improve the realism of communication using a transparent panel. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a telephone system. [Diagram 2] FIG. 2 is a diagram showing a state of a user using a terminal device. [Figure 3A] FIG. 13 is a diagram showing an example of a display on a terminal device. [Figure 3B] FIG. 13 is a diagram showing an example of a display on a terminal device. [Figure 4] FIG. 2 is a sequence diagram showing an example of the operation of the call system. [Figure 5A] FIG. 11 is a flowchart illustrating an example of the operation of the terminal device. [Figure 5B] FIG. 11 is a flowchart illustrating an example of the operation of the terminal device. [Figure 6A] FIG. 13 is a diagram showing an example of a display on a terminal device. [Figure 6B] FIG. 13 is a diagram showing an example of a display on a terminal device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] The following describes the embodiments.

[0010] 1 is a diagram showing an example of the configuration of a call system 1 according to an embodiment. The call system 1 includes a server device 10 and a plurality of terminal devices 12, which are connected to each other so as to be able to communicate information with each other via a network 11. The call system 1 is a system that enables users to carry out virtual face-to-face communication (hereinafter referred to as virtual face-to-face communication) by sending and receiving images, sounds, and the like using the terminal devices 12.

[0011] The server device 10 is, for example, a server computer that belongs to a cloud computing system or other computing system and functions as a server that implements various functions. The server device 10 may be composed of two or more server computers that are communicably connected and operate in cooperation with each other. The server device 10 transmits and receives information and processes information required to provide virtual face-to-face communication.

[0012] The terminal device 12 is an information processing device having a communication function and an input / output function for images, audio, etc., and is used by a user. The terminal device 12 includes an information processing device having a communication function and a display function for images, etc., and a transparent touch panel. The terminal device 12 may be a device dedicated to virtual face-to-face communication, or may be configured by combining, for example, a smartphone, a tablet terminal, a personal computer, digital signage, or the like with a transparent touch panel.

[0013] Network 11 may be, for example, the Internet, but may also include an ad-hoc network, a Local Area Network (LAN), a Metropolitan Area Network (MAN), or other network, or any combination thereof.

[0014] In this embodiment, the terminal device 12 receives information for displaying a model image representing another user based on a captured image of the other user using the other terminal device 12, and information on a drawn image that the other user draws with a drawing tool on the transparent touch panel of the other terminal device 12, from the other terminal device 12, and displays the model image on its own display unit, and an image of the hand holding the drawing tool and the drawn image on its own transparent touch panel. Even if the hand of the other user holding the drawing tool is not visible in the captured image due to the position and angle of view of the imaging unit that captures the captured image, the reality of the displayed model image can be improved by supplementing the image of the hand.

[0015] The configurations of the server device 10 and the terminal device 12 will now be described in detail.

[0016] The server device 10 includes a communication unit 101, a storage unit 102, a control unit 103, an input unit 105, and an output unit 106. When the server device 10 is configured with two or more server computers, these components are appropriately arranged in the two or more computers.

[0017] The communication unit 101 includes one or more communication interfaces. The communication interface is, for example, a LAN interface. The communication unit 101 receives information used in the operation of the server device 10, and transmits information obtained by the operation of the server device 10. The server device 10 is connected to a network 11 by the communication unit 101, and communicates information with a terminal device 12 via the network 11.

[0018] The storage unit 102 includes, for example, one or more semiconductor memories, one or more magnetic memories, one or more optical memories, or a combination of at least two of them, functioning as a main storage device, an auxiliary storage device, or a cache memory. The semiconductor memory is, for example, a random access memory (RAM) or a read only memory (ROM). The RAM is, for example, a static RAM (SRAM) or a dynamic RAM (DRAM). The ROM is, for example, an electrically erasable programmable ROM (EEPROM). The storage unit 102 stores information used in the operation of the server device 10 and information obtained by the operation of the server device 10.

[0019] The control unit 103 includes one or more processors, one or more dedicated circuits, or a combination of these. The processor is, for example, a general-purpose processor such as a CPU (Central Processing Unit), or a dedicated processor such as a GPU (Graphics Processing Unit) specialized for a specific process. The dedicated circuit is, for example, an FPGA (Field-Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), etc. The control unit 103 executes information processing related to the operation of the server device 10 while controlling each unit of the server device 10.

[0020] The input unit 105 includes one or more input interfaces. The input interface is, for example, a physical key, a capacitive key, a pointing device, a touch panel integrated with a display, or a microphone that accepts voice input. The input unit 105 accepts an operation to input information used in the operation of the server device 10, and sends the input information to the control unit 103.

[0021] The output unit 106 includes one or more output interfaces. The output interface is, for example, a display or a speaker. The display is, for example, an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display. The output unit 106 outputs information obtained by the operation of the server device 10.

[0022] The functions of the server device 10 are realized by a processor included in the control unit 103 executing a control program. The control program is a program for causing a computer to function as the server device 10. Also, some or all of the functions of the server device 10 may be realized by a dedicated circuit included in the control unit 103. Also, the control program may be stored in a non-transitory recording / storage medium readable by the server device 10, and read by the server device 10 from the medium.

[0023] The terminal device 12 includes a communication unit 111 , a storage unit 112 , a control unit 113 , an input / output unit 115 , and an imaging unit 117 .

[0024] The communication unit 111 has a communication module compatible with wired or wireless LAN standards, a module compatible with mobile communication standards such as LTE, 4G, 5G, etc. The terminal device 12 is connected to the network 11 by the communication unit 111 via a nearby router device or a mobile communication base station, and performs information communication with the server device 10, etc. via the network 11.

[0025] The memory unit 112 includes one or more semiconductor memories, one or more magnetic memories, one or more optical memories, or a combination of at least two of these. The semiconductor memories are, for example, RAM or ROM. The RAM is, for example, SRAM or DRAM. The ROM is, for example, EEPROM. The memory unit 112 functions as, for example, a main memory device, an auxiliary memory device, or a cache memory. The memory unit 112 stores information used in the operation of the control unit 113 and information obtained by the operation of the control unit 113.

[0026] The control unit 113 has, for example, one or more general-purpose processors such as a CPU, an MPU (Micro Processing Unit), or one or more dedicated processors such as a GPU specialized for a specific process. Alternatively, the control unit 113 may have one or more dedicated circuits such as an FPGA, an ASIC, or the like. The control unit 113 generally controls the operation of the terminal device 12 by operating according to a control / processing program, or operating according to an operating procedure implemented as a circuit. The control unit 113 transmits and receives various information to and from the server device 10, etc. via the communication unit 111, and executes the operation according to this embodiment.

[0027] The functions of the control unit 113 are realized by a processor included in the control unit 113 executing a control program. The control program is a program for causing the processor to function as the control unit 113. Also, some or all of the functions of the control unit 113 may be realized by a dedicated circuit included in the control unit 113. Also, the control program may be stored in a non-transitory recording / storage medium readable by the terminal device 12, and read by the terminal device 12 from the medium.

[0028] The input / output unit 115 includes a transparent touch panel, a display, and one or more input and output interfaces. The input / output unit 115 detects the input of a drawn image based on the displacement of the contact position of a finger, a pointing device, or the like on the transparent touch panel, and sends the detected information to the control unit 113. The transparent touch panel includes a transparent display and displays information such as an image sent from the control unit 113 and an image corresponding to the contact of a pointing device, or the like. The display is, for example, an LCD or an organic EL display, and displays information such as an image sent from the control unit 113. The input interface includes, for example, a physical key, a capacitance key, and a pointing device. The input interface includes a microphone that accepts voice input. Furthermore, the input interface may include a scanner or camera that scans an image code, and an IC card reader. The output interface includes, for example, a speaker. The input / output unit 115 accepts an operation to input information used in the operation of the control unit 113, sends the input information to the control unit 113, and outputs information obtained by the operation of the control unit 113.

[0029] The imaging unit 117 includes a camera that captures an image of a subject using visible light, and a distance measurement sensor that measures the distance to the subject to obtain a distance image. The camera captures images of the subject at, for example, 15 to 30 frames per second to generate a video consisting of continuous captured images. The distance measurement sensor includes a ToF (Time Of Flight) camera, LiDAR (Light Detection And Ranging), and a stereo camera, and generates a distance image of the subject including distance information. The imaging unit 117 sends the captured image and the distance image to the control unit 113.

[0030] FIG. 2 shows an example of the arrangement of the transparent touch panel, display, and camera of the imaging unit 117 of the input / output unit 115. The transparent touch panel 21 is located between the display 22 and the user 23. The user 23 can see the model image of the other user displayed on the display 22 through the transparent touch panel 21, and draws an image on the transparent touch panel 21 with a drawing tool 24. By providing the display 22 from the user 23 through the transparent touch panel 21 and displaying the model image of the other user thereon, it is possible to improve the reality of the user experience, such as communicating with the other user while drawing through the transparent touch panel 21. With this configuration, it is possible to create a reality with depth, for example, compared to the case where the model image of the other user is displayed on the transparent touch panel 21. The camera 20 is provided near the display 22, for example, above the display 22. If the camera 20 is provided at a position superimposed on the display 22, there is a risk that the camera 20 will block the image displayed on the display 22, or that the image drawn on the transparent touch panel 21 will block the image of the user 23 captured by the camera 20. In this regard, by providing the camera 20 above the display 22, it becomes possible to capture an image of the user 23 through the transparent touch panel 21 without blocking the display or capture of the image.

[0031] In the terminal device 12 configured as described above, the control unit 113 acquires a captured image and a distance image of the user 23 by the imaging unit 117. The control unit 113 also collects the speech of the user 23 by the microphone of the input / output unit 115. Furthermore, the control unit 113 acquires information of a drawn image drawn by the user 23 on the transparent touch panel 21 of the input / output unit 115 from the input / output unit 115. The control unit 113 generates encoded information by encoding the captured image and distance image of the user 23 for generating a model image of the user 23, the drawn image drawn by the user 23, and audio information for reproducing the audio of the user 23. The model image is, for example, a 3D model, a 2D model, etc., but hereinafter, a 3D model is taken as an example. The control unit 113 may perform any processing (for example, resolution change, trimming) on ​​the captured image, etc., when encoding. Here, because the camera 20 is located above the display 22, the hand 25 of the user 23 holding the drawing tool 24 may not be included in the range 26 of the angle of view of the camera 20. Therefore, the control unit 113 omits the hand 25 from the 3D model. The control unit 113 also derives the position of the drawing image for the user 23 based on the captured image of the user 23. For example, the position of the drawing image for the user 23 is derived based on the positional relationship between the camera 20 and the transparent touch panel 21, the positional relationship of the user 23 to the camera 20, and the position of the drawing image on the transparent touch panel 21. Then, the control unit 113 determines the position at which the drawing image is displayed on the 3D model of the user 23 so as to correspond to the derived position. Such position information is also included in the coded information. The control unit 113 sends the coded information to another terminal device 12 via the server device 10 by the communication unit 111.

[0032] Moreover, the control unit 113 receives encoded information sent from the other terminal device 12 via the server device 10 by the communication unit 111. When the control unit 113 decodes the encoded information received from the other terminal device 12, the control unit 113 generates a 3D model representing the other user using the other terminal device 12 by using the decoded information. When generating the 3D model, the control unit 113 generates a polygon model using a distance image of the other user, and applies texture mapping to the polygon model using the captured image of the other user to generate the 3D model of the other user. However, the generation of the 3D model is not limited to the example shown here, and any method can be adopted. The control unit 113 generates a rendering image in which a virtual space including the 3D model is viewed from a virtual viewpoint. The virtual viewpoint is, for example, the position of the eyes of the user 23. The control unit 113 derives the spatial coordinates of the eyes relative to an arbitrary reference from the captured image of the user 23, and associates them with the spatial coordinates in the virtual space. The arbitrary reference is, for example, the position of the camera 20. The 3D model of the other user is placed at a position and angle that allows eye contact with the virtual viewpoint. At this time, a model of the other user's hands 25 is omitted. The control unit 113 causes the rendering image to be displayed on the display 22, and also causes the transparent touch panel 21 to display an image of the hand holding the drawing tool and the drawing image. The image of the hand is an image of the hand holding the drawing tool including an image of the drawing tool, and is stored in advance in the storage unit 112. The control unit 113 causes the display / output unit 116 to display the image for display, and also outputs the speech of the other user based on the voice information of the other user.

[0033] FIG. 3A shows an example in which a 3D model of another user is displayed. A 3D model 30 of another user is displayed on a display 22 located behind a transparent touch panel 21. Meanwhile, an image of the user's hand 33 and a drawing image 32 are displayed on the transparent touch panel 21. As shown in an enlarged view in FIG. 3B, the image of the user's hand 33 is an image showing a drawing tool and a part of the user's wrist holding the drawing tool, observed from the tip side of the drawing tool or the palm side. The image of the user's hand 33 and the drawing image 32 are displayed on the transparent touch panel 21 in front of the user, so that the 3D model 30 of the other user has a more three-dimensional appearance. This makes it possible to improve the reality of the displayed 3D model.

[0034] 4 is a sequence diagram for explaining the operation procedure of the call system 1. This sequence diagram shows the procedure related to the cooperative operation of the server device 10 and the multiple terminal devices 12 (for convenience, when distinguishing between them, they are referred to as terminal devices 12A and 12B). This procedure is the procedure when the terminal device 12A calls the terminal device 12B. When the multiple terminal devices 12B are called, the operation procedure related to the terminal device 12B shown here is executed by each of the multiple terminal devices 12B, or by each of the multiple terminal devices 12B and the server device 10.

[0035] 4 are executed by the respective control units 103 and 113. Moreover, the steps relating to the transmission and reception of various information in the server device 10 and the terminal device 12 are executed by the respective control units 103 and 113 transmitting and receiving information to each other via the communication units 101 and 111, respectively. In the server device 10 and the terminal device 12, the control units 103 and 113 store the information to be transmitted and received in the storage units 102 and 112, respectively, as appropriate. Moreover, the control unit 113 of the terminal device 12 accepts input of various information through the input / output unit 115, and outputs various information through the display / output unit 116.

[0036] In step S400, the terminal device 12A accepts input of setting information from its user. The setting information includes a call schedule, a list of call partners, etc. The list includes the user names of the call partners and the email addresses of each user. Then, in step S401, the terminal device 12A sends the setting information to the server device 10. The server device 10 receives the information sent from the terminal device 12A. For example, the terminal device 12A obtains an input screen for setting information from the server device 10 and displays the input screen to the user. Then, the user inputs the setting information into the input screen, and the setting information is sent to the server device 10.

[0037] In step S402, the server device 10 identifies the other party based on the setting information. The control unit 103 stores the setting information and the information of the other party in the storage unit 102 in association with each other.

[0038] In step S403, the server device 10 sends authentication information to the terminal device 12B. The authentication information is information such as an ID and a passcode for identifying and authenticating a call partner using the terminal device 12B. For example, this information is sent as an attachment to an e-mail. The terminal device 12B receives the information sent from the server device 10.

[0039] In step S405, the terminal device 12B sends the authentication information and authentication application information received from the server device 10 to the server device 10. The call recipient operates the terminal device 12B to apply for authentication using the authentication information sent from the server device 10. For example, the terminal device 12B accesses a site for calls provided by the server device 10, obtains an input screen for authentication information and information for authentication application, and displays the input screen to the call recipient. Then, the terminal device 12B accepts the information input by the call recipient and sends it to the server device 10.

[0040] In step S406, the server device 10 authenticates the other party of the call. The storage unit 102 stores the identification information of the terminal device 12B and the identification information of the other party in association with each other.

[0041] In steps S408 and S409, the server device 10 sends a call start notification to the terminal devices 12A and 12B, respectively. Upon receiving the information sent from the server device 10, the terminal devices 12A and 12B each start capturing an image of the user and collecting the spoken voice.

[0042] In step S410, virtual face-to-face communication including a telephone call between the users is executed by the terminal devices 12A and 12B via the server device 10. The terminal devices 12A and 12B transmit and receive information for displaying a 3D model representing each user, a drawn image, and information on a spoken voice to each other via the server device 10. In addition, the terminal devices 12A and 12B output, to each user, an image including a 3D model representing the other user and the spoken voice of the other user.

[0043] 5A and 5B are flow charts for explaining the operation procedure of the terminal device 12 for executing virtual face-to-face communication. The procedure shown here is common to the terminal devices 12A and 12B, and will be explained without distinguishing between the terminal devices 12A and 12B.

[0044] FIG. 5A relates to an operation procedure of the control unit 113 when each terminal device 12 transmits information for displaying a 3D model of the user using that terminal device 12.

[0045] In step S502, the control unit 113 acquires a visible light image, a distance image, a drawn image, and collects audio. The control unit 113 captures a visible light image and a distance image of the user at an arbitrarily set frame rate using the imaging unit 117. The control unit 113 also acquires a drawn image using the input / output unit 115. Furthermore, the control unit 113 collects the voice of the user speaking using the input / output unit 115.

[0046] In step S503, the control unit 113 determines the image of the hand. The control unit 113 performs any image processing including pattern matching on the captured image to estimate the attributes of the user. The attributes are handedness, sex, age, etc. Handedness is the distinction between the left and right hand holding the drawing tool. The storage unit 112 stores images of the left and right hands in advance. The images of the hands include images of the hands for each sex or age group. Such images of the hands are created in advance based on images of hands of models with different sexes or age groups. The control unit 113 selects and determines the image of the hands corresponding to the estimated attributes. Alternatively, the control unit 113 may extract the image of the hands from a past captured image. In the drawing action of the user, the image of the hand holding the drawing tool may be included in the captured image by including the drawing tool and the part of the hand holding it within the range of the angle of view of the camera 20. The control unit 113 may detect such captured images and extract the image of the hands from the captured images.

[0047] In step S504, the control unit 113 encodes the captured image, the distance image, the image at hand, the drawn image, and the audio information to generate encoded information.

[0048] In step S 506 , the control unit 113 causes the communication unit 111 to packetize the encoded information and transmit it to the server device 10 toward the other terminal devices 12 .

[0049] When the control unit 113 acquires information input in response to an operation by the user himself / herself to interrupt image capture or sound collection or to exit virtual face-to-face communication (Yes in S508), it ends the processing procedure in Fig. 5A, and while it does not acquire information corresponding to the operation for interruption or exit (No in S508), it executes steps S502 to S506 and transmits information for displaying a 3D model representing the user himself / herself, a drawn image, and information for outputting sound to the server device toward the other terminal device 12. Note that the determination of the image at hand in step S503 may be executed for each processing cycle of steps S502 to S506, or may be executed at any timing such as every several cycles.

[0050] Fig. 5B relates to an operation procedure of the control unit 113 when the terminal device 12 outputs another user's 3D model image, hand image, drawing image, and sound. When the control unit 113 receives, via the server device 10, a packet sent by another terminal device 12 executing the procedure of Fig. 5A, the control unit 113 executes steps S510 to S513.

[0051] In step S510, the control unit 113 decodes the encoded information contained in the packet received from the other terminal device 12 to obtain the captured image, distance image, close-up image, drawn image, and audio information.

[0052] In step S511, the control unit 113 sets an image of hands when displaying a 3D model of another user. The control unit 113 sets the image of hands sent from the other terminal device 12 as an image to be displayed.

[0053] In step S512, the control unit 113 generates a 3D model representing the user of the other terminal devices 12 based on the captured image and the distance image. When receiving information from a plurality of other terminal devices 12, the control unit 113 executes steps S510 to S512 for each of the other terminal devices 12 and generates a 3D model of each user.

[0054] In step S513, the control unit 113 places a 3D model representing another user in the virtual space. Coordinate information of the virtual space and information on the coordinates at which the 3D model should be placed for each other user, for example, according to the order of authentication, are stored in advance in the storage unit 112. The control unit 113 places the generated 3D model at the coordinates in the virtual space.

[0055] In step S514, the control unit 113 generates a display image. The control unit 113 generates a rendering image by capturing an image of the 3D model placed in the virtual space from a virtual viewpoint.

[0056] In step S516, the control unit 113 displays the display image, the image at hand, and the drawing image through the input / output unit 115, and outputs a sound. The control unit 113 displays the display image on the display 22, and the image at hand and the drawing image on the transparent touch panel 21. At this time, the control unit causes the display position of the image at hand to correspond to the position of the hand of the 3D model included in the image for display. The control unit 113 also rotates the angle of the image at hand so that it matches the angle of the forearm of the 3D model. For example, as shown in FIG. 6A, the control unit 113 matches the angle θ of the inclination of the image at hand 33 with respect to the horizontal direction with the angle θ of the forearm 60 of the 3D model with respect to the horizontal direction. As shown in FIG. 6B, when the angle θ of the inclination of the forearm 60 changes to a different angle θ' due to the movement of another user, the control unit 113 rotates the image at hand 33 so that the inclination of the image at hand with respect to the horizontal direction becomes angle θ'. This allows the user to view the 3D model and the image of their hands in a more natural way.

[0057] By repeatedly executing steps S510 to S516 by the control unit 113, the user can hear the voice of the other user while watching a video including the other user's 3D model and an image drawn by the 3D model. Note that the setting of the image at hand in step S511 may be performed for each processing cycle of steps S510 to S516, or may be performed at any timing such as every few cycles.

[0058] In the modified example, instead of determining the image at hand in step S503 in Fig. 5A, the image at hand is determined and set in step S511 in Fig. 5B. For example, the control unit 113 acquires the attributes of the other user estimated in step S503 in the other terminal device 12, and in step S512, determines the image at hand by extracting an image at hand that matches the attributes of the other user from among the images at hand stored in advance in the storage unit 112. Then, the control unit 113 can set the determined image at hand.

[0059] As described above, according to this embodiment, it is possible to improve the reality of communication using a transmissive touch panel.

[0060] In the above example, the terminal device 12 receives information for generating a 3D model of another user from another terminal device 12, i.e., a captured image, a distance image, etc., and then generates a 3D model and generates a rendering image in which the 3D model is placed in a virtual space. However, the processes of generating a 3D model and generating a rendering image, etc., may be appropriately distributed among the terminal devices 12. For example, a 3D model of another user may be generated based on a captured image, etc. in another terminal device 12, and the terminal device 12 that receives the information on the 3D model may generate a rendering image using the 3D model.

[0061] In the above example, the model image is a 3D model. However, the model image may be a 2D model. The three-dimensional structure in which the 2D model of another user is displayed on the display and the image of the other user's hand is displayed on the transparent touch panel in front of it makes it possible to express depth with a relatively simple configuration.

[0062] In the above, the embodiment has been described based on the drawings and examples, but it should be noted that those skilled in the art can easily make various modifications and corrections based on the present disclosure. Therefore, it should be noted that these modifications and corrections are included in the scope of the present disclosure. For example, the functions included in each means, each step, etc. can be rearranged so as not to be logically inconsistent, and multiple means, steps, etc. can be combined into one or divided into one. [Explanation of symbols]

[0063] 1. Call System 10. Server device 11 Network 12 Terminal Equipment 101, 111 Communications Department 102, 112 Storage section 103, 113 Control section 105 Input section 106 Output section 115 Input / output section 117 Imaging unit

Claims

1. The Communications Department and a transparent panel for displaying an image and accepting drawings; a display unit overlapping the transmissive panel; an imaging unit that is disposed near the display unit and captures an image of a user; A terminal device having a control unit that performs communication by the communication unit, the control unit receives, from the other terminal device, information for displaying a model image representing the other user based on a captured image of the other user using the other terminal device, and information on a drawn image that the other user draws with a drawing tool on a transparent panel of the other terminal device, and causes the control unit to display the model image on the display unit of the terminal device, and a hand image of the other user's hand holding the drawing tool and the drawn image on the transparent panel of the terminal device. Terminal device.

2. In claim 1, The control unit causes the image of the hand to correspond to an angle of the arm of the model image. Terminal device.

3. In claim 1, the control unit uses the image of the hand corresponding to the dominant hand of the other user in the model image. Terminal device.

4. In claim 1, The control unit uses the image at hand corresponding to the attribute of the other user in the model image. Terminal device.

5. In claim 1, The control unit receives the hand image extracted from the captured image from the other terminal device, and uses the hand image. Terminal device.

Images