Terminal equipment

Abstract

To provide a terminal apparatus that improves reality in virtual face-to-face communication.SOLUTION: In a call system 1 having a server apparatus 10 and a plurality of terminal apparatuses 12 that are connected via a network 11 to enable communication of information with each other, the terminal apparatus has a communication section, a display section, an imaging section that captures an image of a user, and a control section that communicates by the communication section. The control section receives information for displaying a 3D model representing another user on the basis of a captured image of another user who uses another terminal apparatus and a supplementary image that includes an invisible portion in the captured image, and controls the display section to display a 3D model in which the invisible portion in the 3D model based on the captured image is supplemented by the 3D model based on the supplementary image.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a terminal device. [Background technology]

[0002] There is a known technology that allows users of computers connected via a network to communicate with other users by sending and receiving images and voices to each other. For example, Patent Document 1 discloses a two-way communication system that displays on a display unit an image of the communication partner's space acquired from the communication partner. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2021-141346 A Summary of the Invention [Problem to be solved by the invention]

[0004] In a technology that allows users to communicate virtually face-to-face by sending and receiving images and voices to each other, there is room for improving the reality of the communication.

[0005] The present disclosure provides a terminal device etc. that enables improved realism in virtual face-to-face communication. [Means for solving the problem]

[0006] A terminal device in the present disclosure is a terminal device having a communication unit, a display unit, an imaging unit that images a user, and a control unit that communicates via the communication unit, wherein the control unit receives information for displaying a 3D model representing another user using another terminal device based on an image of the other user and a complementary image including invisible portions in the image, and causes the display unit to display a 3D model in which the invisible portions in the 3D model based on the image are supplemented with a 3D model based on the complementary image. Effect of the Invention

[0007] According to the terminal device etc. of the present disclosure, it is possible to improve the realism and convenience of virtual face-to-face communication. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a telephone system. [Figure 2A] FIG. 2 is a diagram showing a usage mode of a terminal device. [Figure 2B] FIG. 2 is a diagram showing an aspect of image processing in a terminal device. [Figure 2C] FIG. 2 is a diagram showing a usage mode of a terminal device. [Diagram 3] FIG. 2 is a sequence diagram showing an example of the operation of the call system. [Figure 4A] FIG. 11 is a flowchart illustrating an example of the operation of the terminal device. [Figure 4B] FIG. 11 is a flowchart illustrating an example of the operation of the terminal device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] The following describes the embodiments.

[0010] 1 is a diagram showing an example of the configuration of a call system 1 according to an embodiment. The call system 1 includes a server device 10 and a plurality of terminal devices 12, which are connected to each other so as to be able to communicate information with each other via a network 11. The call system 1 is a system that enables users to carry out virtual face-to-face communication (hereinafter referred to as virtual face-to-face communication) by sending and receiving images, sounds, and the like using the terminal devices 12.

[0011] The server device 10 is, for example, a server computer that belongs to a cloud computing system or other computing system and functions as a server that implements various functions. The server device 10 may be composed of two or more server computers that are communicably connected and operate in cooperation with each other. The server device 10 transmits and receives information and processes information required to provide virtual face-to-face communication.

[0012] The terminal device 12 is an information processing device having a communication function and an input / output function for images, audio, etc., and is used by a user. The terminal device 12 is, for example, a smartphone, a tablet terminal, a personal computer, a digital signage, etc.

[0013] Network 11 may be, for example, the Internet, but may also include an ad-hoc network, a Local Area Network (LAN), a Metropolitan Area Network (MAN), or other network, or any combination thereof.

[0014] In this embodiment, the terminal device 12 receives information for displaying a 3D model representing another user based on a captured image of the other user using the other terminal device 12 and a complementary image including an invisible part in the captured image, and displays a 3D model in which the invisible part in the 3D model based on the captured image is supplemented with a 3D model based on the complementary image. When a user of the terminal device 12 (hereinafter referred to as the "own user") performs virtual face-to-face communication with another user of the other terminal device 12 (hereinafter referred to as the "other user"), a part that is blocked from the imaging unit and becomes invisible when the other user draws a drawing image such as a character or a figure on a touch panel by a hand holding a drawing tool is supplemented with a 3D model based on the complementary image. Thus, the own user sees a more natural 3D model of the other user and experiences a reality as if he or she were performing face-to-face communication with the other user through a transparent panel while drawing on the transparent panel. This makes it possible to improve the reality in virtual face-to-face communication.

[0015] The configurations of the server device 10 and the terminal device 12 will now be described in detail.

[0016] The server device 10 includes a communication unit 101, a storage unit 102, a control unit 103, an input unit 105, and an output unit 106. When the server device 10 is configured with two or more server computers, these components are appropriately arranged in the two or more computers.

[0017] The communication unit 101 includes one or more communication interfaces. The communication interface is, for example, a LAN interface. The communication unit 101 receives information used in the operation of the server device 10, and transmits information obtained by the operation of the server device 10. The server device 10 is connected to a network 11 by the communication unit 101, and communicates information with a terminal device 12 via the network 11.

[0018] The storage unit 102 includes, for example, one or more semiconductor memories, one or more magnetic memories, one or more optical memories, or a combination of at least two of them, functioning as a main storage device, an auxiliary storage device, or a cache memory. The semiconductor memory is, for example, a random access memory (RAM) or a read only memory (ROM). The RAM is, for example, a static RAM (SRAM) or a dynamic RAM (DRAM). The ROM is, for example, an electrically erasable programmable ROM (EEPROM). The storage unit 102 stores information used in the operation of the server device 10 and information obtained by the operation of the server device 10.

[0019] The control unit 103 includes one or more processors, one or more dedicated circuits, or a combination of these. The processor is, for example, a general-purpose processor such as a CPU (Central Processing Unit), or a dedicated processor such as a GPU (Graphics Processing Unit) specialized for a specific process. The dedicated circuit is, for example, an FPGA (Field-Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), etc. The control unit 103 executes information processing related to the operation of the server device 10 while controlling each unit of the server device 10.

[0020] The input unit 105 includes one or more input interfaces. The input interface is, for example, a physical key, a capacitive key, a pointing device, a touch panel integrated with a display, or a microphone that accepts voice input. The input unit 105 accepts an operation to input information used in the operation of the server device 10, and sends the input information to the control unit 103.

[0021] The output unit 106 includes one or more output interfaces. The output interface is, for example, a display or a speaker. The display is, for example, an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display. The output unit 106 outputs information obtained by the operation of the server device 10.

[0022] The functions of the server device 10 are realized by a processor included in the control unit 103 executing a control program. The control program is a program for causing a computer to function as the server device 10. Also, some or all of the functions of the server device 10 may be realized by a dedicated circuit included in the control unit 103. Also, the control program may be stored in a non-transitory recording / storage medium readable by the server device 10, and read by the server device 10 from the medium.

[0023] The terminal device 12 includes a communication unit 111 , a storage unit 112 , a control unit 113 , an input unit 115 , a display / output unit 116 , an imaging unit 117 , and a detection unit 118 .

[0024] The communication unit 111 has a communication module compatible with wired or wireless LAN standards, a module compatible with mobile communication standards such as LTE, 4G, 5G, etc. The terminal device 12 is connected to the network 11 by the communication unit 111 via a nearby router device or a mobile communication base station, and performs information communication with the server device 10, etc. via the network 11.

[0025] The memory unit 112 includes one or more semiconductor memories, one or more magnetic memories, one or more optical memories, or a combination of at least two of these. The semiconductor memories are, for example, RAM or ROM. The RAM is, for example, SRAM or DRAM. The ROM is, for example, EEPROM. The memory unit 112 functions as, for example, a main memory device, an auxiliary memory device, or a cache memory. The memory unit 112 stores information used in the operation of the control unit 113 and information obtained by the operation of the control unit 113.

[0026] The control unit 113 has, for example, one or more general-purpose processors such as a CPU, an MPU (Micro Processing Unit), or one or more dedicated processors such as a GPU specialized for a specific process. Alternatively, the control unit 113 may have one or more dedicated circuits such as an FPGA, an ASIC, or the like. The control unit 113 generally controls the operation of the terminal device 12 by operating according to a control / processing program, or operating according to an operating procedure implemented as a circuit. The control unit 113 transmits and receives various information to and from the server device 10, etc. via the communication unit 111, and executes the operation according to this embodiment.

[0027] The input unit 115 includes a touch panel that is integral with the display and one or more input interfaces. The input unit 15 detects the input of a drawn image based on the displacement of the contact position of a finger, a pointing device, or the like on the touch panel, and sends the detected information to the control unit 113. The input interface includes, for example, a physical key, a capacitive key, and a pointing device. The input interface also includes a microphone that accepts voice input. Furthermore, the input interface may include a scanner or camera that scans an image code, and an IC card reader. The input unit 115 accepts an operation to input information used in the operation of the control unit 113, and sends the input information to the control unit 113.

[0028] The display / output unit 116 includes a display that displays images and one or more output interfaces. The display is, for example, an LCD or an organic EL display. The output interface includes, for example, a speaker. The display / output unit 116 outputs information obtained by the operation of the control unit 113.

[0029] The imaging unit 117 includes a camera that captures an image of a subject using visible light, and a distance measurement sensor that measures the distance to the subject to obtain a distance image. The camera captures images of the subject at, for example, 15 to 30 frames per second to generate a video consisting of continuous captured images. The distance measurement sensor includes a ToF (Time Of Flight) camera, LiDAR (Light Detection And Ranging), and a stereo camera, and generates a distance image of the subject including distance information. The imaging unit 117 sends the captured image and the distance image to the control unit 113.

[0030] The detection unit 118 has one or more sensors or an interface with the sensors that detect each part of the terminal device 12 or the situation around the terminal device 12, and sends information indicating the detection results by the sensors to the control unit 33. The sensors include a proximity sensor that detects the approach of an object to any part of the terminal device 12 using electromagnetic waves, sound waves, etc. The proximity sensor detects, for example, the approach of a finger or the like to a touch panel of the input unit 115, and sends the detection result to the control unit 113.

[0031] The functions of the control unit 113 are realized by a processor included in the control unit 113 executing a control program. The control program is a program for causing the processor to function as the control unit 113. Also, some or all of the functions of the control unit 113 may be realized by a dedicated circuit included in the control unit 113. Also, the control program may be stored in a non-transitory recording / storage medium readable by the terminal device 12, and read by the terminal device 12 from the medium.

[0032] 2A to 2C show how a user performs face-to-face communication using a terminal device 12. FIG.

[0033] 2A shows the state of the user himself / herself using the terminal device 12. The user himself / herself 20 makes a call while drawing characters, patterns, etc. on the touch panel of the input unit 115 superimposed on the display of the display / output unit 116. The display / output unit 116 displays information such as images corresponding to contact with a pointing device or the like. The imaging unit 117 is provided at a position where at least the upper half of the user himself / herself 20's body can be imaged, such as above the display, or behind the display when the display is configured as a transparent display.

[0034] The control unit 113 acquires a captured image and a distance image of the user 20 by the imaging unit 117. The control unit 113 also collects the speech of the user 20 by the microphone of the input unit 115. The control unit 113 also acquires information on a drawn image drawn by the user 20 on the touch panel of the input unit 115 from the input unit 115. The control unit 113 generates encoded information by encoding the captured image and distance image of the user 20 for generating a 3D model of the user 20, the drawn image drawn by the user 20, and audio information for playing the audio of the user 20. When encoding, the control unit 113 may perform any processing (e.g., changing the resolution, trimming, complementing, etc.) on the captured image, etc.

[0035] FIG. 2B is a diagram for explaining the processing process. Depending on the position of the imaging unit 117, when the user 20 draws, an occlusion area 27 that is not visible from the imaging unit 117 occurs due to occlusion by objects such as the drawing image 23 and the hand 24 holding the drawing tool in the range 26 of the angle of view of the imaging unit 117. If the arm 25 of the user 21, for example, included in the occlusion area 27, is not imaged, a visible light image and a distance image related to the arm 25 are not acquired, and only a 3D model missing the arm 25 is generated. Therefore, in this embodiment, the control unit 113 uses a complement image stored in the storage unit 112 in advance to complement the missing part. The control unit 113 also encodes the information of the complement image and makes it a part of the encoded information.

[0036] Furthermore, the control unit 113 derives the position of the drawn image relative to the user 20 based on the captured image of the user 20. For example, the position of the drawn image relative to the user 20 is derived based on the positional relationship between the imaging unit 117 and the touch panel, and the position of the user 20 and the drawn image relative to the imaging unit 117. Then, the control unit 113 determines the position at which to superimpose the drawn image on the 3D model of the user 20 so as to correspond to the derived position.

[0037] Then, the control unit 113 transmits the encoded information to the other terminal device 12 via the server device 10 through the communication unit 111 .

[0038] 2C shows an aspect of another user displayed on the terminal device 12. A rendering image 22 including a 3D model 21 of the other user is displayed on the display of the display / output unit 116 together with a drawing image 23 drawn by the other user 21.

[0039] The control unit 113 receives encoded information sent from the other terminal device 12 via the server device 10 by the communication unit 111. When the control unit 113 decodes the encoded information received from the other terminal device 12, the control unit 113 generates a 3D model 21 representing the other user using the other terminal device 12 by using the decoded information. When generating the 3D model 21, the control unit 113 generates a polygon model using a distance image of the other user, and applies texture mapping to the polygon model using a captured image of the other user to generate the 3D model 21 of the other user. However, the generation of the 3D model 21 is not limited to the example shown here, and any method can be adopted. When generating the 3D model 21, the control unit 113 generates a complementary 3D model 28 for complementing the invisible arm 25 with a complementary image, and complements the 3D model 21. Then, the control unit 113 generates a rendering image 22 in which the virtual space including the 3D model 21 is viewed from a virtual viewpoint. The virtual viewpoint is, for example, the position of the eyes of the user 20. The control unit 113 derives spatial coordinates of the eyes relative to an arbitrary reference from the captured image of the user 20, and associates them with spatial coordinates in the virtual space. The arbitrary reference is, for example, the position of the imaging unit 117. The 3D model of the other user 21 is placed at a position and angle where eye contact can be made with respect to the virtual viewpoint. Furthermore, the control unit 113 generates an image for display by superimposing the drawing image 23 on the rendering image 22. The drawing image 23 is placed so as to correspond to the position of the hand holding the drawing tool of the 3D model 21. The control unit 113 causes the display / output unit 116 to display the image for display and output the speech of the other user 21 based on the speech information of the other user 21.

[0040] 3 is a sequence diagram for explaining the operation procedure of the call system 1. This sequence diagram shows the procedure related to the cooperative operation of the server device 10 and the multiple terminal devices 12 (for convenience, when distinguishing between them, they are referred to as terminal devices 12A and 12B). This procedure is the procedure when the terminal device 12A calls the terminal device 12B. When the multiple terminal devices 12B are called, the operation procedure related to the terminal device 12B shown here is executed by each of the multiple terminal devices 12B, or by each of the multiple terminal devices 12B and the server device 10.

[0041] 3 are executed by the respective control units 103 and 113. Moreover, the steps relating to the transmission and reception of various information in the server device 10 and the terminal device 12 are executed by the respective control units 103 and 113 transmitting and receiving information to each other via the communication units 101 and 111, respectively. In the server device 10 and the terminal device 12, the control units 103 and 113 store the information to be transmitted and received in the storage units 102 and 112, respectively, as appropriate. Moreover, the control unit 113 of the terminal device 12 accepts input of various information through the input unit 115, and outputs various information through the display / output unit 116.

[0042] In step S300, the terminal device 12A accepts input of setting information from its user. The setting information includes a call schedule, a list of call partners, etc. The list includes the user names of the call partners and the email addresses of each user. Then, in step S301, the terminal device 12A sends the setting information to the server device 10. The server device 10 receives the information sent from the terminal device 12A. For example, the terminal device 12A obtains an input screen for setting information from the server device 10 and displays the input screen to the user. Then, the user inputs the setting information into the input screen, and the setting information is sent to the server device 10.

[0043] In step S302, the server device 10 identifies the other party based on the setting information. The control unit 103 stores the setting information and the information of the other party in the storage unit 102 in association with each other.

[0044] In step S303, the server device 10 sends authentication information to the terminal device 12B. The authentication information is information such as an ID and a passcode for identifying and authenticating a call partner using the terminal device 12B. For example, this information is sent as an attachment to an e-mail. The terminal device 12B receives the information sent from the server device 10.

[0045] In step S305, the terminal device 12B sends the authentication information and authentication application information received from the server device 10 to the server device 10. The call recipient operates the terminal device 12B to apply for authentication using the authentication information sent from the server device 10. For example, the terminal device 12B accesses a site for calls provided by the server device 10, obtains an input screen for authentication information and information for authentication application, and displays the input screen to the call recipient. Then, the terminal device 12B accepts the information input by the call recipient and sends it to the server device 10.

[0046] In step S306, the server device 10 authenticates the other party of the call. The storage unit 102 stores the identification information of the terminal device 12B and the identification information of the other party in association with each other.

[0047] In steps S308 and S309, the server device 10 sends a call start notification to the terminal devices 12A and 12B, respectively. Upon receiving the information sent from the server device 10, the terminal devices 12A and 12B each start capturing an image of the user and collecting the spoken voice.

[0048] In step S310, virtual face-to-face communication including a telephone call between users is executed by the terminal devices 12A and 12B via the server device 10. The terminal devices 12A and 12B transmit and receive information for generating a 3D model representing each user, a drawn image, and spoken voice information to each other via the server device 10. In addition, the terminal devices 12A and 12B output an image including a 3D model representing the other user and the spoken voice of the other user to each user.

[0049] 4A and 4B are flow charts for explaining the operation procedure of the terminal device 12 for executing virtual face-to-face communication. The procedure shown here is common to the terminal devices 12A and 12B, and will be explained without distinguishing between the terminal devices 12A and 12B.

[0050] FIG. 4A relates to an operation procedure of the control unit 113 when each terminal device 12 transmits information for generating a 3D model of the user who is using that terminal device 12.

[0051] In step S402, the control unit 113 acquires a visible light image, a distance image, a drawing image, and collects audio. The control unit 113 captures a visible light image and a distance image of the user at an arbitrarily set frame rate using the imaging unit 117. The control unit 113 also acquires a drawing image using the input unit 115. Furthermore, the control unit 113 collects the voice of the user speaking using the input unit 115.

[0052] In step S403, the control unit 113 determines a complement image. The complement image is an image that corresponds to an invisible portion in the captured image.

[0053] First, the control unit 113 identifies an invisible part. The invisible part is the upper arm or forearm that is located behind the hand and is shielded from the camera of the imaging unit 117 when the user extends the hand holding the drawing tool forward to draw on the touch panel. The control unit 113 extracts an image of the hand holding the drawing tool by performing any image processing such as pattern matching on the captured image, and identifies the forearm and upper arm connected to the hand as the invisible part when the image of the hand shows an arbitrarily determined size that indicates the approach to the imaging unit 117. The control unit 113 may determine the approach of the hand to the imaging unit 117 via the position of the touch panel when the distance from the touch panel to the hand detected by the detection unit 118 is equal to or less than an arbitrary reference.

[0054] When the control unit 113 specifies the invisible part, it selects a complementary image for complementing the invisible part from complementary image samples stored in the storage unit 112. The complementary image includes distance information required for generating a 3D model. Various complementary image samples are stored in the storage unit 112. For example, samples of images of the forearm or upper arm generated in advance for each hand, gender, skin color, type of clothing (short sleeve or long sleeve), and color of clothing are stored. The control unit 113 selects the most suitable image sample by, for example, pattern matching for the hand, gender, skin color, type of clothing, color of clothing, and the like of the user in the captured image. Alternatively, the control unit 113 may detect an image in which an invisible part is captured from the history of captured images, and create a complementary image for complementing the invisible part from the captured image and a distance image corresponding to the captured image.

[0055] In step S404, the control unit 113 encodes the captured image including the complementary image, the distance image, the drawn image, and the audio information to generate encoded information.

[0056] In step S 406 , the control unit 113 causes the communication unit 111 to packetize the encoded information and transmit it to the server device 10 toward the other terminal devices 12 .

[0057] When the control unit 113 acquires information input corresponding to an operation by the user himself / herself to interrupt imaging or sound collection or to exit virtual face-to-face communication (Yes in S408), it terminates the processing procedure of FIG. 4A, and while it does not acquire information corresponding to an operation for interrupting or exiting (No in S408), it executes steps S402 to S407 and sends information for generating a 3D model representing the user himself / herself, information for outputting a drawn image, and information for outputting sound to the server device for other terminal devices 12.

[0058] Fig. 4B relates to an operation procedure of the control unit 113 when the terminal device 12 outputs an image of a 3D model, a drawn image, and sound of another user. When the control unit 113 receives, via the server device 10, a packet sent by the other terminal device 12 executing the procedure of Fig. 4A, the control unit 113 executes steps S410 to S413.

[0059] In step S410, the control unit 113 decodes the encoded information contained in the packet received from the other terminal device 12 to obtain the captured image, distance image, drawn image, and audio information.

[0060] In step S412, the control unit 113 generates a 3D model representing the user of the other terminal devices 12 based on the captured image and the distance image. When receiving information from a plurality of other terminal devices 12, the control unit 113 executes steps S410 to S412 for each of the other terminal devices 12 to generate a 3D model for each user. At this time, the control unit 113 generates a complementary 3D model from the complementary image for each 3D model, and complements missing parts of the 3D model with the complementary 3D model.

[0061] In step S413, the control unit 113 places a 3D model representing another user in the virtual space. Coordinate information of the virtual space and information on the coordinates at which the 3D model should be placed for each other user, for example, according to the order of authentication, are stored in advance in the storage unit 112. The control unit 113 places the generated 3D model at the coordinates in the virtual space.

[0062] In step S414, the control unit 113 generates a display image. The control unit 113 generates a rendering image by capturing an image of a 3D model placed in a virtual space from a virtual viewpoint. The control unit 113 then superimposes a drawing image at a position corresponding to the 3D model on the rendering image to generate the display image.

[0063] In step S416, the control unit 113 causes the display / output unit 116 to display a display image and output sound.

[0064] By repeatedly executing steps S410 to S416 by the control unit 113, the user can hear the voice of the other user's speech while watching a video including the other user's 3D model and an image drawn by the 3D model. At that time, the missing part of the 3D model is complemented by the complementary 3D model, so the user can see a more natural image of the other user, improving reality.

[0065] Instead of step S403 in FIG. 4A, in step S412 in FIG. 4B, the control unit 113 may identify an invisible portion from a captured image received from another terminal device 12, determine a complementary image, and generate a complementary 3D model.

[0066] In the above example, the terminal device 12 receives information for generating a 3D model of another user from another terminal device 12, i.e., a captured image, a distance image, etc., and then generates a 3D model and generates a rendering image in which the 3D model is placed in a virtual space. However, processes such as generating a 3D model and its complement, and generating a rendering image may be appropriately distributed among the terminal devices 12. For example, the other terminal device 12 may generate a 3D model of another user and a complementary 3D model based on a captured image, etc., and the terminal device 12 that receives information on these 3D models may generate a rendering image using the 3D models.

[0067] In the above, the embodiment has been described based on the drawings and examples, but it should be noted that those skilled in the art can easily make various modifications and corrections based on the present disclosure. Therefore, it should be noted that these modifications and corrections are included in the scope of the present disclosure. For example, the functions included in each means, each step, etc. can be rearranged so as not to be logically inconsistent, and multiple means, steps, etc. can be combined into one or divided into one. [Explanation of symbols]

[0068] 1. Call System 10. Server equipment 11 Network 12 Terminal Equipment 101, 111 Communications Department 102, 112 Storage section 103, 113 Control section 105, 115 Input section 106 Output section 116 Display and output section 117 Imaging unit 118 Detection unit

Claims

1. A communication unit, A display unit; An imaging unit that images a user; A control unit that performs communication via the communication unit; In a terminal device having the control unit receives information for displaying a 3D model representing another user using another terminal device based on a captured image of the other user and a complementary image including an invisible portion in the captured image, and causes the display unit to display a 3D model in which the invisible portion in the 3D model based on the captured image is supplemented with a 3D model based on the complementary image; An input unit having a touch panel superimposed on the display unit, the invisible portion is an arm portion of the other user that is blocked from the imaging unit by the hand holding a drawing tool when the other user draws on the touch panel with the hand. Terminal device.

2. A communication unit, A display unit; An imaging unit that images a user; A control unit that performs communication via the communication unit; In a terminal device having the control unit receives information for displaying a 3D model representing another user using another terminal device based on a captured image of the other user and a complementary image including an invisible portion in the captured image, and causes the display unit to display a 3D model in which the invisible portion in the 3D model based on the captured image is supplemented with a 3D model based on the complementary image; A storage unit for storing a sample of the complementary image, the control unit receives from the other terminal device information on a complementary image corresponding to the invisible portion selected from the sample of complementary images, or selects from the sample of complementary images the complementary image corresponding to the invisible portion in the captured image received from the other terminal device. Terminal device.

3. A communication unit, A display unit; An imaging unit that images a user; A control unit that performs communication via the communication unit; In a terminal device having the control unit receives information for displaying a 3D model representing another user using another terminal device based on a captured image of the other user and a complementary image including an invisible portion in the captured image, and causes the display unit to display a 3D model in which the invisible portion in the 3D model based on the captured image is supplemented with a 3D model based on the complementary image; the control unit receives information on the complementary image acquired from a history of the captured image from the other terminal device, or acquires the complementary image from a history of the captured image received from the other terminal device. Terminal device.

Images