Terminal device

Abstract

To provide terminal devices that contribute to user convenience by adjusting for environmental differences.SOLUTION: In a communication system having one or more server devices and multiple terminal devices that are connected to each other so that they can communicate via a network, the terminal device has a communication part and a control part that performs communication via the communication part. The control part receives, from another terminal device, information for displaying to the user a first image drawn in a first drawing area by another user using the other terminal device, and sends, to the other terminal device, information for displaying a second image drawn by the user in a second drawing area superimposed on the displayed first image while being superimposed on the first drawing area.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present disclosure relates to a terminal device.

Background Art

[0002] A technology is known in which computers at multiple locations communicate via a network, and users of each computer make voice calls or exchange information such as images drawn on a touch panel with each other. Patent Document 1 discloses a technology related to input / output of images.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] If the input or output environment of an image or the like of each computer is different and it causes an obstacle to the input / output ability of the image, it is inconvenient for the user.

[0005] In view of the above, a terminal device and the like that can adjust the differences in the environment and contribute to user convenience will be disclosed below.

Means for Solving the Problems

[0006] The terminal device in the present disclosure is a terminal device having a communication unit and a control unit that communicates via the communication unit. The control unit receives information from the other terminal device for displaying, for the user, a first image drawn in a first drawing area by another user using the other terminal device, and sends information for displaying, in a second drawing area that overlaps the displayed first image, a second image drawn by the user in the second drawing area so as to overlap the first drawing area, to the other terminal device.

Effects of the Invention

[0007] The terminal devices described in this disclosure make it possible to adjust for differences in the environment between terminal devices, thereby contributing to user convenience. [Brief explanation of the drawing]

[0008] [Figure 1] This is a diagram illustrating an example of a communication system configuration. [Figure 2] This is a diagram illustrating how the terminal device is used. [Figure 3] This is a flowchart illustrating an example of terminal device operation. [Figure 4] This diagram illustrates how to change the display magnification for display images and transmission images. [Modes for carrying out the invention]

[0009] The embodiments will be described below.

[0010] First, using Figures 1 and 2, we will explain the system configuration including the terminal device in this embodiment and how the terminal device is used.

[0011] Figure 1 shows an example of the configuration of a communication system in this embodiment. The communication system 1 has one or more server devices 10 and multiple terminal devices 12 that are connected to each other via a network 11 so as to be able to communicate information with each other. The communication system 1 is a system that enables multiple users to make calls and exchange images with each other using their respective terminal devices 12.

[0012] The server device 10 is, for example, a server computer belonging to a cloud computing system or other computing system, and functions as a server that implements various functions. The server device 10 may be composed of two or more server computers that are connected and operate in cooperation with each other. The server device 10 performs call and image relay processing. The server device 10 has a communication unit 101, a storage unit 102, and a control unit 103. These components are appropriately arranged on two or more computers when the server device 10 is composed of two or more server computers.

[0013] The terminal device 12 is an information processing device equipped with communication functions and voice and image input / output functions, and is used by each user. The terminal device 12 is, for example, an information processing terminal such as a smartphone or tablet terminal, or an information processing device such as a personal computer. The terminal device 12 has a communication unit 121, a storage unit 122, a control unit 123, an input unit 125, and an output unit 126.

[0014] Network 11 is, for example, the Internet, but also includes ad-hoc networks, LANs (Local Area Networks), MANs (Metropolitan Area Networks), or other networks, or any combination thereof.

[0015] Figure 2 shows how terminal device 12 is used. Figure 2 schematically shows how user 20 and other users 20 (referred to as users 20a and 20b when distinguishing between them) exchange drawn images 22 (referred to as images 22a and 22b when distinguishing between users 20a and 20b when distinguishing between them using terminal devices 12, respectively) on the drawing area 21 (referred to as drawing area 21a and 21b when distinguishing between users 20a and 20b, respectively). Here, the drawing area 21 is an area of ​​any size for user 20 to draw, and includes paper, a touch panel, etc. The drawn image (hereinafter referred to as the drawn image) 22 is any character, symbol, picture, etc. drawn with a writing instrument, stylus pen, fingertip, etc. Furthermore, the drawn image 22 is acquired by the terminal device 12 either by being captured by the camera included in the input unit 125 or by being acquired by the touch panel included in the input unit 125. The other party's drawn image 22 is then displayed superimposed on the drawing area 21 by a display included in the output unit 126 and superimposed on the touch panel, or by projection from a projector included in the output unit 126.

[0016] In this embodiment, terminal device 12a receives information from terminal device 12b for displaying a drawn image 22b drawn in the drawing area 21b by another user 20b using another terminal device 12b to user 20a, and sends information to the other terminal device 12b for displaying a drawn image 22a drawn by user 20a superimposed on the displayed drawn image 22b in the drawing area 21a. In this way, users 20 of terminal devices 12 can exchange additional drawn images 22 while viewing the other party's drawn image 22 in the drawing area 21. At that time, as will be described later, the other party's drawn image 22 is displayed in the drawing area 21 with appropriate scaling. Therefore, it is possible to adjust the differences in the image input / output environment of each terminal device 12 to improve the convenience of the user 20.

[0017] Next, the configuration of the server device 10 will be described in detail.

[0018] The communication unit 101 includes one or more communication interfaces. The communication interface is, for example, a LAN interface. The communication unit 101 receives information used for the operation of the server device 10 and transmits information obtained by the operation of the server device 10. The server device 10 is connected to the network 11 by the communication unit 101 and communicates with the terminal device 12 via the network 11.

[0019] The storage unit 102 includes, for example, one or more semiconductor memories that function as a main memory device, an auxiliary storage device, or a cache memory, one or more magnetic memories, one or more optical memories, or a combination of at least two of these. The semiconductor memory is, for example, a RAM (Random Access Memory) or a ROM (Read Only Memory). The RAM is, for example, a SRAM (Static RAM) or a DRAM (Dynamic RAM). The ROM is, for example, an EEPROM (Electrically Erasable Programmable ROM). The storage unit 102 stores information used for the operation of the server device 10 and information obtained by the operation of the server device 10.

[0020] The control unit 103 includes one or more processors, one or more dedicated circuits, or a combination of these. The processor is, for example, a general-purpose processor such as a CPU (Central Processing Unit) or a dedicated processor such as a GPU (Graphics Processing Unit) specialized for specific processing. The dedicated circuit is, for example, an FPGA (Field-Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), etc. The control unit 103 executes information processing related to the operation of the server device 10 while controlling each part of the server device 10.

[0021] The functions of the server device 10 are realized by a processor included in the control unit 103 executing a control program. The control program is a program for causing a computer to function as the server device 10. Also, some or all of the functions of the server device 10 may be realized by a dedicated circuit included in the control unit 103. Further, the control program may be stored in a non-transitory recording and storage medium readable by the server device 10, and the server device 10 may read it from the medium.

[0022] Next, the configuration of the terminal device 12 will be described in detail.

[0023] The communication unit 121 includes a communication module corresponding to a wired or wireless LAN standard, a module corresponding to a mobile communication standard such as LTE, 4G, 5G, etc. The terminal device 12 is connected to the network 11 via the communication unit 121 through a nearby router device or a base station for mobile communication, and performs information communication with the server device 10, etc. via the network 11.

[0024] The storage unit 122 includes one or more semiconductor memories, one or more magnetic memories, one or more optical memories, or a combination of at least two of these. The semiconductor memory is, for example, a RAM or a ROM. The RAM is, for example, a SRAM or a DRAM. The ROM is, for example, an EEPROM. The storage unit 122 functions as, for example, a main storage device, an auxiliary storage device, or a cache memory. The storage unit 122 stores information used for the operation of the control unit 123 and information obtained by the operation of the control unit 123.

[0025] The control unit 123 has, for example, one or more general-purpose processors such as a CPU or MPU (Micro Processing Unit), or one or more dedicated processors such as a GPU specialized for specific processing. Alternatively, the control unit 123 may have one or more dedicated circuits such as FPGAs or ASICs. The control unit 123 comprehensively controls the operation of the terminal device 12 by operating according to a control and processing program, or by operating according to an operating procedure implemented as a circuit. The control unit 123 then sends and receives various information with the server device 10, etc., via the communication unit 121 and executes the operations according to this embodiment.

[0026] The input unit 125 includes one or more input interfaces. The input interfaces include, for example, physical keys, capacitive keys, pointing devices, and touch panels integrated with the display. The input interface also includes one or more cameras for capturing images. The cameras are configured separately from the terminal device 12 and may be configured to communicate with the terminal device 12 directly or via the network 11. The cameras are positioned and angled to capture the drawing area 21 when the user 20 is drawing. The cameras may also be configured to capture an area including, for example, the face of the user 20 who is drawing. Furthermore, the input interface may include a microphone for receiving audio input. The input unit 125 receives operations to input information used for the operation of the control unit 123, and performs actions such as capturing images and collecting sound, and sends the input information, captured images, and audio to the control unit 123.

[0027] The output unit 126 includes one or more output interfaces. The output interfaces include, for example, a display, a projector, and a speaker. The display is, for example, an LCD or an organic EL display. The display may be configured as a touch panel and integrated with the input interface of the input unit 125. The projector may be configured to communicate directly with the terminal device 12, or to communicate via the network 11. The output unit 126 outputs information obtained by the operation of the control unit 123.

[0028] The functions of the control unit 123 are realized by the execution of a control program by the processor included in the control unit 123. The control program is a program that causes the processor to function as the control unit 123. In addition, some or all of the functions of the control unit 123 may be realized by a dedicated circuit included in the control unit 123. Furthermore, the control program may be stored in a non-transient recording / storage medium readable by the terminal device 12, and the terminal device 12 may read it from the medium.

[0029] In this embodiment, the control unit 123 uses the input unit 125 to capture an image of the user 20's face and the drawing area 21 while simultaneously collecting the user 20's speech and receiving the drawn image 22. The control unit 123 also encodes the output image and the audio information for reproducing the user's voice to generate encoded information. The output image has any number of pixels, for example, so-called 4K (3840 x 2160) or more. The control unit 123 may perform any processing on the output image during encoding (for example, resolution change and cropping). The control unit 123 sends the encoded information to other terminal devices 12 via the server device 10 using the communication unit 121.

[0030] Furthermore, the control unit 123 receives encoded information sent from other terminal devices 12 via the server device 10 through the communication unit 121. After decoding the encoded information, the control unit 123 uses the decoded information to output an output image via the display or projector of the output unit 126, or to output sound via the speaker. The display or projector for displaying the other party's image 22 has any output resolution, for example, so-called 4K or higher. The image of the other party user 20 may be represented in 2D or as a 3D model. The display for displaying the 3D model of the other party user 20 may be, for example, a light field display that makes 3D images visible.

[0031] Figure 3 is a flowchart illustrating the operation procedure of terminal device 12. Here, the operation of terminal device 12a when it exchanges information with terminal device 12b is used as an example. The procedure in Figure 3 is executed by the control unit 123 of terminal device 12a at a period of several milliseconds to tens of milliseconds, which can be set arbitrarily.

[0032] In step S300, the control unit 123 captures an image of the user 20a including its face, acquires a drawn image 22a via capture or touch input, and collects spoken audio. The control unit 123 acquires the captured image, drawn image 22a, and audio information of the user 20a from the input unit 125.

[0033] In step S302, the control unit 123 receives the captured image, drawn image 22b, and audio information of user 20b from the terminal device 12b. The terminal device 12a receives the captured image 40, including the drawn image 22b drawn in the drawing area 21b, from the terminal device 12b, for example as shown in Example 401 in Figure 4.

[0034] In step S304, the control unit 123 generates a display image. The display image is an image that is superimposed on the drawing area 21a and includes a drawing image 22b. For example, as shown in example 402 of Figure 4, a display image 41 is generated for superimposition on the drawing area 21a. The display image 41 includes a drawing image 22b.

[0035] In step S306, the control unit 123 generates a transmission image. The transmission image is an image that is superimposed on the drawing area 21b, and if drawing is performed in the drawing area 21a, it includes the drawn image 22a.

[0036] In step S308, the control unit 123 determines whether the display magnification of the display image has been changed. The display magnification is changed by user 20a performing actions on the input unit 125 of the terminal device 12a, such as tapping or swiping on the touch panel, or specifying the magnification using a pointing device or key. If the display magnification has been changed as a result of such an operation (Yes), the control unit 123 proceeds to steps S310 and S312. On the other hand, if the display magnification has not been changed (No), the control unit 123 skips steps S310 and S312 and proceeds to step S314.

[0037] In step S310, the control unit 123 adjusts the display magnification of the display image. The control unit 123 adjusts the display magnification of the display image according to the changed display magnification. If the display magnification is changed from 1x to mx (where m is a positive real number other than 1), the control unit 123 adjusts the display image by multiplying by m the range of the display image that includes at least a part of the drawn image 22b sent from the terminal device 12b. For example, as shown in example 403 of Figure 4, the display image 41 for superimposing on the drawing area 21a is adjusted and the drawn image 22b is enlarged.

[0038] In step S312, the control unit 123 adjusts the display magnification of the image to be transmitted. The control unit 123 adjusts the display magnification of the image to be transmitted by multiplying the drawn image 22a drawn by the user 20a in the terminal device 12a by (1 / m) according to the changed display magnification of m times. For example, in example 404 of Figure 4, the drawn image 22a is shown in the drawing area 21a relative to the enlarged drawn image 22b. In this case, since the drawn image 22b is enlarged, the image to be transmitted 42 is adjusted and the drawn image 22a is reduced, as shown in example 405. The image to be transmitted 42 is generated in step S306 and adjusted in step S312.

[0039] In step S314, the control unit 123 displays a display image and outputs sound. The display of the display image includes displaying the user 20b's captured image and displaying the drawn image 22b in the drawing area 21a. For example, the display image 41 shown in example 403 of Figure 4 is displayed.

[0040] In step S316, the control unit 123 sends out image and audio information for transmission. The control unit 123 encodes the image and audio information for transmission and sends the encoded information to the terminal device 12b.

[0041] The procedure in Figure 3 is performed similarly in terminal device 12b. By executing step S314 in terminal device 12b, the display image 43 is superimposed on the drawing area 21b, as shown in example 406 in Figure 4. Here, the drawing image 22a added by terminal device 12a to the initially drawn drawing image 22b is included in the display image 41.

[0042] In this way, terminal devices 12a and 12b perform the procedures shown in Figure 3 to exchange drawn images 22 and to make calls. Therefore, users 20 of terminal devices 12 can exchange additional drawn images 22 while viewing the other party's drawn image 22 in the drawing area 21. At that time, the other party's drawn image 22 is displayed with appropriate scaling, improving the visibility of the other party's drawn image 22 and making it easier to add additional images. Thus, it is possible to adjust the differences in the image input / output environment of each terminal device 12 to improve the convenience of the users 20.

[0043] In this embodiment, the terminal device 12 stores the image 22 of the other party to be displayed, or additional images 22 to be drawn, in the storage unit 122, and is configured to display them as needed in response to user 20's operation. Furthermore, when terminal device 12a communicates with terminal device 12c used by another user 20c, together with terminal device 12b, terminal device 12a may be configured to switch between image 22b and image 22c as needed in response to user 20a's operation. In addition, terminal device 12a may be able to set the display time for each of the images 22b and 22c in response to user 20a's operation.

[0044] As described above, embodiments have been explained based on various drawings and examples, but it should be noted that those skilled in the art will find it easy to make various modifications and alterations based on this disclosure. Therefore, it should be noted that these modifications and alterations are within the scope of this disclosure. For example, the functions, etc., included in each means, each step, etc., can be rearranged in a logically consistent manner, and multiple means, steps, etc., can be combined into one or divided. [Explanation of Symbols]

[0045] 1. Communication System 10 Server devices 11 Network 12 Terminal devices 20 users 21 Drawing area 22 Images 101, 121 Communications Department 102, 122 Storage section 103, 123 Control Unit 105 Input section 106 Output section

Claims

1. Communications Department and, In a terminal device having a control unit that performs communication via the aforementioned communication unit, The control unit receives information from another terminal device for displaying a first image drawn in a first drawing area by another user using another terminal device to the user, and sends information to the other terminal device for displaying a second image drawn by the user superimposed on the first drawing area in a second drawing area that superimposes on the displayed first image. When the control unit enlarges the first image and displays it superimposed on the second drawing area, it sends information to the other terminal device for reducing the second image and displaying it superimposed on the first drawing area. Terminal device.

2. In claim 1, The first image is reduced in size and superimposed on the second drawing area, and the second image is enlarged and superimposed on the first drawing area. Terminal device.

3. In claim 1, The control unit is capable of switching between displaying and hiding the first image. Terminal device.

4. In claim 3, The control unit receives information from another terminal device for displaying a third image drawn in a third drawing area by yet another user using that terminal device to the user, and sends information to the yet another terminal device for displaying a fourth image drawn by the user in a fourth drawing area that superimposes on the third image displayed when the first image is hidden, superimposed on the third drawing area. Terminal device.

Images