Terminal device

Abstract

To provide terminal devices that contribute to user convenience by adjusting for environmental differences.SOLUTION: In a communication system having one or more server devices and multiple terminal devices that are connected to each other so that they can communicate via a network, the terminal device has a communication part and a control part that performs communication via the communication part. The control part receives, from another terminal device, information for displaying to the user a first image drawn in a first drawing area by another user using the other terminal device, and sends, to the other terminal device, information for displaying a second image drawn by the user in a second drawing area superimposed on the displayed first image while being superimposed on the first drawing area.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present disclosure relates to a terminal device. [Background technology]

[0002] There is known a technology that enables computers at multiple locations to communicate over a network, and enables users of the computers to have voice calls with each other and to exchange information such as images drawn on a touch panel. Patent Document 1 discloses a technology related to the input and output of images. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2004-109402 A Summary of the Invention [Problem to be solved by the invention]

[0004] If the input or output environment of images, etc. of each computer differs and this causes problems in inputting and outputting images, it will be inconvenient for the user.

[0005] In view of the above, a terminal device etc. capable of adjusting to differences in environment and contributing to user convenience is disclosed below. [Means for solving the problem]

[0006] A terminal device in the present disclosure is a terminal device having a communication unit and a control unit that communicates via the communication unit, wherein the control unit receives information from another terminal device for displaying to a user a first image drawn in a first drawing area by another user using another terminal device, and sends information to the other terminal device for displaying a second image drawn by the user in a second drawing area superimposed on the displayed first image, superimposed on the first drawing area. Effect of the Invention

[0007] According to the terminal device etc. of the present disclosure, it is possible to adjust for differences in environments between terminal devices, thereby contributing to user convenience. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 illustrates an example of the configuration of a communication system. [Diagram 2] FIG. 2 is a diagram for explaining a usage mode of a terminal device. [Diagram 3] FIG. 11 is a flowchart illustrating an example of the operation of the terminal device. [Figure 4] 11A and 11B are diagrams illustrating a change in display magnification of an image for display and an image for transmission. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] The following describes the embodiments.

[0010] First, the configuration of a system including a terminal device in this embodiment and a usage mode of the terminal device will be described with reference to Figs.

[0011] 1 is a diagram showing an example of the configuration of a communication system in this embodiment. The communication system 1 has one or more server devices 10 and multiple terminal devices 12 that are connected to each other so that they can communicate with each other via a network 11. The communication system 1 is a system that enables multiple users to talk to each other and send and receive images using each of the terminal devices 12.

[0012] The server device 10 is, for example, a server computer belonging to a cloud computing system or other computing system, and functions as a server implementing various functions. The server device 10 may be composed of two or more server computers that are communicatively connected and operate in cooperation with each other. The server device 10 executes relay processing of calls and images. The server device 10 has a communication unit 101, a storage unit 102, and a control unit 103. When the server device 10 is composed of two or more server computers, these components are appropriately arranged in the two or more computers.

[0013] The terminal device 12 is an information processing device equipped with a communication function and an input / output function of voice and image, and is used by each user. The terminal device 12 is, for example, an information processing terminal such as a smartphone or a tablet terminal, or an information processing device such as a personal computer. The terminal device 12 has a communication unit 121, a storage unit 122, a control unit 123, an input unit 125, and an output unit 126.

[0014] Network 11 may be, for example, the Internet, but may also include an ad-hoc network, a Local Area Network (LAN), a Metropolitan Area Network (MAN), or other network, or any combination thereof.

[0015] FIG. 2 shows a usage mode of the terminal device 12. FIG. 2 shows a schematic mode in which a user 20 exchanges a drawn image 22 (referred to as an image 22a or 22b when associated with the user 20a or 20b) drawn in a drawing area 21 (referred to as an image 21a or 21b when associated with the user 20a or 20b) with another user 20 (referred to as an image 22a or 22b when associated with the user 20a or 20b) using a terminal device 12 (referred to as an image 12a or 12b when associated with the user 20a or 20b). Here, the drawing area 21 is an area of ​​any size for the user 20 to draw, and includes a paper surface, a touch panel, or the like. The drawn image (hereinafter referred to as a drawn image) 22 is any character, symbol, picture, or the like drawn with a writing implement, a stylus pen, a fingertip, or the like. Furthermore, the drawing image 22 is acquired by the terminal device 12 by being photographed by a camera included in the input unit 125 or by being acquired by a touch panel included in the input unit 125. The drawing image 22 of the other party is then displayed superimposed on the drawing area 21 by a display included in the output unit 126 and provided superimposed on the touch panel, or by projection by a projector included in the output unit 126.

[0016] The terminal device 12a of this embodiment receives information from the terminal device 12b for displaying a drawing image 22b drawn in the drawing area 21b by another user 20b using another terminal device 12b to the user 20a, and sends information to the other terminal device 12b for displaying the drawing image 22a drawn by the user 20a in the drawing area 21b superimposed on the displayed drawing image 22b. In this way, the users 20 of the terminal devices 12 can exchange additional drawing images 22 while viewing the drawing image 22 of the other user in the drawing area 21. At that time, the drawing image 22 of the other user is displayed in the drawing area 21 with appropriate enlargement or reduction, as described later. Therefore, it is possible to adjust the difference in the input / output environment of the image of each terminal device 12, thereby contributing to the convenience of the user 20.

[0017] Next, the configuration of the server device 10 will be described in detail.

[0018] The communication unit 101 includes one or more communication interfaces. The communication interface is, for example, a LAN interface. The communication unit 101 receives information used in the operation of the server device 10, and transmits information obtained by the operation of the server device 10. The server device 10 is connected to a network 11 by the communication unit 101, and communicates information with a terminal device 12 via the network 11.

[0019] The storage unit 102 includes, for example, one or more semiconductor memories, one or more magnetic memories, one or more optical memories, or a combination of at least two of them, functioning as a main storage device, an auxiliary storage device, or a cache memory. The semiconductor memory is, for example, a random access memory (RAM) or a read only memory (ROM). The RAM is, for example, a static RAM (SRAM) or a dynamic RAM (DRAM). The ROM is, for example, an electrically erasable programmable ROM (EEPROM). The storage unit 102 stores information used in the operation of the server device 10 and information obtained by the operation of the server device 10.

[0020] The control unit 103 includes one or more processors, one or more dedicated circuits, or a combination of these. The processor is, for example, a general-purpose processor such as a CPU (Central Processing Unit), or a dedicated processor such as a GPU (Graphics Processing Unit) specialized for a specific process. The dedicated circuit is, for example, an FPGA (Field-Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), etc. The control unit 103 executes information processing related to the operation of the server device 10 while controlling each unit of the server device 10.

[0021] The functions of the server device 10 are realized by a processor included in the control unit 103 executing a control program. The control program is a program for causing a computer to function as the server device 10. Also, some or all of the functions of the server device 10 may be realized by a dedicated circuit included in the control unit 103. Also, the control program may be stored in a non-transitory recording / storage medium readable by the server device 10, and read by the server device 10 from the medium.

[0022] Next, the configuration of the terminal device 12 will be described in detail.

[0023] The communication unit 121 has a communication module compatible with a wired or wireless LAN standard, a module compatible with a mobile communication standard such as LTE, 4G, 5G, etc. The terminal device 12 is connected to the network 11 by the communication unit 121 via a nearby router device or a mobile communication base station, and performs information communication with the server device 10, etc. via the network 11.

[0024] The memory unit 122 includes one or more semiconductor memories, one or more magnetic memories, one or more optical memories, or a combination of at least two of these. The semiconductor memories are, for example, RAM or ROM. The RAM is, for example, SRAM or DRAM. The ROM is, for example, EEPROM. The memory unit 122 functions as, for example, a main memory device, an auxiliary memory device, or a cache memory. The memory unit 122 stores information used in the operation of the control unit 123 and information obtained by the operation of the control unit 123.

[0025] The control unit 123 has, for example, one or more general-purpose processors such as a CPU, an MPU (Micro Processing Unit), or one or more dedicated processors such as a GPU specialized for a specific process. Alternatively, the control unit 123 may have one or more dedicated circuits such as an FPGA, an ASIC, or the like. The control unit 123 generally controls the operation of the terminal device 12 by operating according to a control / processing program, or operating according to an operating procedure implemented as a circuit. The control unit 123 transmits and receives various information to and from the server device 10, etc. via the communication unit 121, and executes the operation according to this embodiment.

[0026] The input unit 125 includes one or more input interfaces. The input interfaces include, for example, physical keys, electrostatic capacitance keys, a pointing device, and a touch panel that is integral with the display. The input interface also includes one or more cameras that capture captured images. The camera may be configured separately from the terminal device 12 and configured to be able to communicate with the terminal device 12 directly or via the network 11. The camera is installed at a position and angle that allows it to capture an image of the drawing area 21 when the user 20 draws at hand. The camera may also be provided so as to be able to capture an image of an area including, for example, the face of the user 20 who is drawing. Furthermore, the input interface may include a microphone that accepts voice input. The input unit 125 accepts an operation to input information used in the operation of the control unit 123, captures images, collects sounds, etc., and sends information such as input information, captured images, and sounds to the control unit 123.

[0027] The output unit 126 includes one or more output interfaces. The output interfaces include, for example, a display, a projector, and a speaker. The display is, for example, an LCD or an organic EL display. The display may be configured as a touch panel integrally with the input interface of the input unit 125. The projector may be configured to be able to communicate directly with the terminal device 12, or may be configured to be able to communicate with the terminal device 12 via the network 11. The output unit 126 outputs information obtained by the operation of the control unit 123.

[0028] The functions of the control unit 123 are realized by a processor included in the control unit 123 executing a control program. The control program is a program for causing the processor to function as the control unit 123. A part or all of the functions of the control unit 123 may be realized by a dedicated circuit included in the control unit 123. The control program may be stored in a non-transitory recording / storage medium readable by the terminal device 12, and may be read by the terminal device 12 from the medium.

[0029] In this embodiment, the control unit 123 collects the speech of the user 20 while capturing an image of the drawing area 21 and an area including the face of the user 20 through the input unit 125, and receives the drawing image 22. The control unit 123 also generates encoded information by encoding the image for output and audio information for playing the user's voice. The image for output has an arbitrary number of pixels, for example, so-called 4K (3840×2160) or more. When encoding, the control unit 123 may perform arbitrary processing (for example, changing the resolution and trimming) on ​​the image for output. The control unit 123 sends the encoded information to another terminal device 12 via the server device 10 through the communication unit 121.

[0030] In addition, the control unit 123 receives encoded information sent from another terminal device 12 via the server device 10 through the communication unit 121. When the control unit 123 decodes the encoded information, the control unit 123 uses the decoded information to output an image for output through the display or projector of the output unit 126, or to output sound through a speaker. The display or projector for displaying the drawing image 22 of the other party has any output resolution, for example, so-called 4K or higher. The image of the other party user 20 may be expressed in 2D or may be expressed by a 3D model. The display for displaying the 3D model of the other party user 20 may be, for example, a light field display that makes a 3D image visible.

[0031] Fig. 3 is a flow chart for explaining the operation procedure of the terminal device 12. Here, the operation of the terminal device 12a when the terminal device 12a exchanges information with the terminal device 12b is taken as an example. The procedure of Fig. 3 is executed by the control unit 123 of the terminal device 12a at an arbitrarily set cycle of several milliseconds to several tens of milliseconds, for example.

[0032] In step S300, the control unit 123 captures an image of an area including the face of the user 20a, acquires a drawn image 22a by imaging or touch input, and collects a spoken voice. The control unit 123 acquires the captured image, drawn image 22a, and voice information of the user 20a from the input unit 125.

[0033] In step S302, the control unit 123 receives the captured image, the drawn image 22b, and the voice information of the user 20b from the terminal device 12b. The terminal device 12a receives the captured image 40 including the drawn image 22b drawn in the drawing area 21b from the terminal device 12b, for example, as shown in an example 401 in FIG.

[0034] In step S304, the control unit 123 generates a display image. The display image is an image to be displayed superimposed on the drawing area 21a, and includes the drawing image 22b. For example, as shown in an example 402 in Fig. 4, a display image 41 to be displayed superimposed on the drawing area 21a is generated. The display image 41 includes the drawing image 22b.

[0035] In step S306, the control unit 123 generates an image for transmission. The image for transmission is an image that is displayed superimposed on the drawing area 21b, and includes the drawn image 22a when drawing is performed in the drawing area 21a.

[0036] In step S308, the control unit 123 determines whether the display magnification of the display image has been changed. The display magnification is changed by the user 20a tapping or swiping on the touch panel, or by specifying the magnification using a pointing device or a key, performed on the input unit 125 of the terminal device 12a. If the display magnification has been changed in response to such an operation (Yes), the control unit 123 proceeds to steps S310 and S312. On the other hand, if the display magnification has not been changed (No), the control unit 123 skips steps S310 and S312 and proceeds to step S314.

[0037] In step S310, the control unit 123 adjusts the display magnification of the display image. The control unit 123 adjusts the display magnification of the display image according to the changed display magnification. When the display magnification is changed from 1 to m (m is a positive real number other than 1), the control unit 123 adjusts the display image by multiplying a range of the display image that includes at least a part of the drawing image 22b sent from the terminal device 12b by m. For example, as shown in an example 403 in Fig. 4, the display image 41 to be displayed superimposed on the drawing area 21a is adjusted, and the drawing image 22b is enlarged.

[0038] In step S312, the control unit 123 adjusts the display magnification of the image for transmission. In response to the changed display magnification of m times, the control unit 123 adjusts the display magnification of the image for transmission by multiplying the drawn image 22a drawn by the user 20a on the terminal device 12a by (1 / m). For example, in example 404 of FIG. 4, the drawn image 22a drawn on the enlarged drawn image 22b in the drawing area 21a is shown. In this case, since the drawn image 22b is enlarged, the image for transmission 42 is adjusted to reduce the drawn image 22a as shown in example 405. Note that the image for transmission 42 is generated in step S306 and adjusted in step S312.

[0039] In step S314, the control unit 123 displays a display image and outputs a sound. The display image includes displaying a captured image of the user 20b and displaying a drawn image 22b in the drawing area 21a. For example, the display image 41 shown in the example 403 of FIG. 4 is displayed.

[0040] In step S316, the control unit 123 transmits the image and audio information to be transmitted. The control unit 123 encodes the image and audio information to be transmitted, and transmits the encoded information to the terminal device 12b.

[0041] The procedure of Fig. 3 is similarly executed in the terminal device 12b. By executing step S314 in the terminal device 12b, a display image 43 is displayed superimposed on the drawing area 21b as shown in example 406 in Fig. 4. Here, the drawing image 22a added by the terminal device 12a to the drawing image 22b initially drawn is included in the display image 41.

[0042] In this way, the terminal devices 12a and 12b each execute the procedure of Fig. 3 to exchange the drawing image 22 and to make a call. Thus, the users 20 of the terminal devices 12 can exchange additional drawing images 22 while viewing the drawing image 22 of the other party in the drawing area 21. At that time, the drawing image 22 of the other party is appropriately enlarged or reduced for display, improving the visibility of the drawing image 22 of the other party and facilitating additional drawing. Thus, it is possible to adjust for differences in the image input / output environment of each terminal device 12 and contribute to the convenience of the users 20.

[0043] In the embodiment, the terminal device 12 is configured to store the drawing image 22 of the other party to be displayed or the drawing image 22 to be additionally drawn in the storage unit 122, and to display the drawing image 22 at any time in response to the operation of the user 20. Furthermore, when the terminal device 12a communicates with the terminal device 12c used by the other user 20c in addition to the terminal device 12b, the terminal device 12a may be configured to be able to switch the image to be displayed in response to the operation of the user 20a between the drawing image 22b and the drawing image 22c at any time. Furthermore, the terminal device 12a may be able to set the display time of each of the drawing image 22b and the drawing image 22c in response to the operation of the user 20a.

[0044] In the above, the embodiment has been described based on the drawings and examples, but it should be noted that those skilled in the art can easily make various modifications and corrections based on the present disclosure. Therefore, it should be noted that these modifications and corrections are included in the scope of the present disclosure. For example, the functions included in each means, each step, etc. can be rearranged so as not to be logically inconsistent, and multiple means, steps, etc. can be combined into one or divided into one. [Explanation of symbols]

[0045] 1. Communication Systems 10. Server device 11 Network 12 Terminal Equipment 20 users 21 Drawing area 22 Drawing images 101, 121 Communications Department 102, 122 Storage section 103, 123 Control section 105 Input section 106 Output section

Claims

1. The Communications Department and A terminal device having a control unit that performs communication by the communication unit, the control unit receives information from another terminal device for displaying a first image drawn in a first drawing area by another user using the other terminal device to the user, and sends information to the other terminal device for displaying a second image drawn by the user in a second drawing area superimposed on the displayed first image, the second image being superimposed on the first drawing area; Terminal device.

2. In claim 1, the first image is enlarged and displayed so as to be superimposed on the second drawing area, and the second image is reduced and displayed so as to be superimposed on the first drawing area. Terminal device.

3. In claim 1, the first image is reduced and displayed so as to be superimposed on the second drawing area, and the second image is enlarged and displayed so as to be superimposed on the first drawing area. Terminal device.

4. In claim 1, The control unit is capable of switching between displaying and not displaying the first image. Terminal device.

5. In claim 4, the control unit receives information for displaying a third image drawn in a third drawing area by a still further user using the still further terminal device to the still further user, and sends information to the still further terminal device for displaying a fourth image drawn by the user in a fourth drawing area superimposed on the third image displayed when the first image is not displayed, the fourth image being drawn by the user in a fourth drawing area superimposed on the third image displayed when the first image is not displayed; Terminal device.

Images