Terminal device
By realizing information exchange and environmental differences between terminal devices in the terminal device, the convenience problem caused by different image input and output environments is solved, and the convenience and visibility of users when exchanging images is improved.
Patent Information
- Application Number
- CN202410681984.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-06-05
- Filing Date
- 2024-05-29
- Publication Date
- 2024-12-06
AI Technical Summary
In the prior art, due to the different image input or output environments of each computer, the input and output of images are inconvenient to the user.
A terminal device is designed to realize information exchange and environmental difference adjustment between terminal devices through communication components and control components, allowing users to display and send images drawn by another user in another area in one depicted area, and according to User operation adjusts the display magnification to adapt to environmental differences.
In this way, the user can conveniently exchange and display images between different terminal devices, adjust the input and output environment of the images, and improve the convenience and visibility of the user.
Smart Images

Figure CN119094824A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to a terminal device. Background Art
[0002] There is known a technology in which computers at multiple locations communicate via a network, and users of each computer communicate with each other by voice or exchange information such as images drawn on a touch panel. Patent Document 1 discloses a technology related to image input and output. Prior art literature Patent Literature
[0003] Patent Document 1: Japanese Patent Application Publication No. 2004-109402 Summary of the invention Technical problem to be solved by the invention
[0004] If the input and output capabilities of images are hindered due to the difference in the input and output environments of images and the like between computers, it is inconvenient for users.
[0005] In view of the above circumstances, a terminal device and the like that can adjust to differences in environments and improve user convenience are disclosed below. Means for solving technical problems
[0006] The terminal device in the present disclosure is a terminal device having a communication unit and a control unit that communicates through the communication unit, wherein the control unit receives information from other terminal devices for displaying to a user a first image drawn in a first drawing area by another user using the other terminal device, and sends to the other terminal devices information for displaying a second image drawn by the user in a second drawing area overlapping the displayed first image so as to overlap with the first drawing area. Effects of the Invention
[0007] According to the terminal device and the like in the present disclosure, it is possible to adjust for differences in environments between terminal devices to improve user convenience. BRIEF DESCRIPTION OF THE DRAWINGS
[0008] Figure 1 It is a diagram showing a configuration example of a communication system. Figure 2 This is a diagram for explaining how to use a terminal device. Figure 3 This is a flowchart showing an example of the operation of the terminal device. Figure 4 A diagram for explaining a change in display magnification of a display image and a transmission image. DETAILED DESCRIPTION
[0009] Hereinafter, embodiments will be described.
[0010] First, use Figure 1 , Figure 2 , the structure of the system including the terminal device in this embodiment and the usage of the terminal device are explained.
[0011] Figure 1 1 is a diagram showing an example of the configuration of a communication system in the present embodiment. The communication system 1 includes one or more server devices 10 and a plurality of terminal devices 12 connected to each other via a network 11 so as to be able to perform information communication. The communication system 1 is a system for enabling a plurality of users to communicate with each other or to send and receive images using the terminal devices 12, respectively.
[0012] The server device 10 belongs to a cloud computing system or other computing system, for example, and is a server computer that functions as a server for implementing various functions. The server device 10 may also be composed of two or more server computers that are connected and work in coordination for information communication. The server device 10 performs relay processing of calls and images. The server device 10 has a communication unit 101, a storage unit 102, and a control unit 103. In the case where the server device 10 is composed of two or more server computers, these components are appropriately configured in the two or more computers.
[0013] The terminal device 12 is an information processing device with a communication function and a voice and image input and output function, and is used by each user. The terminal device 12 is, for example, an information processing terminal such as a smartphone or a tablet terminal, an information processing device such as a personal computer, etc. The terminal device 12 has a communication unit 121, a storage unit 122, a control unit 123, an input unit 125, and an output unit 126.
[0014] The network 11 is, for example, the Internet, but includes an ad hoc network, a LAN (Local Area Network), a MAN (Metropolitan Area Network), or other networks, or any combination thereof.
[0015] Figure 2 The usage of the terminal device 12 is shown. Figure 2, schematically shows the manner in which a user 20 and another user 20 (when distinguished from each other, referred to as user 20a, 20b) use a terminal device 12 (when distinguished from the user 20a, 20b, referred to as terminal device 12a, 12b, respectively) to transmit and receive a drawing image 22 (when distinguished from the user 20a, 20b, referred to as image 22a, 22b, respectively) drawn in a drawing area 21 (when distinguished from the user 20a, 20b, referred to as drawing area 21a, 21b, respectively). Here, the drawing area 21 is an area of any size for the user 20 to draw, including a paper surface, a touch panel, etc. In addition, the drawn image (hereinafter referred to as a drawing image) 22 is any text, symbol, painting, etc. drawn by a writing instrument, a stylus, a fingertip, etc. In addition, the drawing image 22 is captured by a camera included in the input unit 125 or obtained by the touch panel included in the input unit 125, thereby being obtained by the terminal device 12. Then, the partner's drawn image 22 is displayed in a superimposed manner in the drawing area 21 through a display provided overlapping the touch panel included in the output unit 126 or through projection by a projector included in the output unit 126 .
[0016] The terminal device 12a of this embodiment receives information for displaying the image 22b drawn by the other user 20b using the terminal device 12b in the drawing area 21b to the user 20a, and transmits information for displaying the image 22a drawn by the user 20a in the drawing area 21a overlapping with the displayed image 22b to the other terminal device 12b so as to overlap the drawing area 21b. In this way, the users 20 of the terminal devices 12 can exchange additional images 22 while observing each other's images 22 in the drawing area 21. At this time, as described later, the other party's images 22 are displayed in the drawing area 21 with appropriate scale. Therefore, the difference in the input and output environment of the images of the terminal devices 12 can be adjusted, which is beneficial to the convenience of the users 20.
[0017] Next, the configuration of the server device 10 will be described in detail.
[0018] The communication unit 101 includes one or more communication interfaces. The communication interface is, for example, a LAN interface. The communication unit 101 receives information used in the operation of the server device 10 and sends information obtained by the operation of the server device 10. The server device 10 is connected to the network 11 through the communication unit 101, and performs information communication with the terminal device 12 via the network 11.
[0019] The storage unit 102 includes, for example, one or more semiconductor memories, one or more magnetic memories, one or more optical memories, or a combination of at least two of them that function as a main storage device, an auxiliary storage device, or a buffer memory. The semiconductor memory is, for example, a RAM (Random Access Memory) or a ROM (Read Only Memory). The RAM is, for example, an SRAM (Static RAM) or a DRAM (Dynamic RAM). The ROM is, for example, an EEPROM (Electrically Erasable Programmable ROM). The storage unit 102 stores information used in the operation of the server device 10 and information obtained through the operation of the server device 10.
[0020] The controller 103 includes one or more processors, one or more dedicated circuits, or a combination thereof. The processor is, for example, a general-purpose processor such as a CPU (Central Processing Unit) or a dedicated processor such as a GPU (Graphics Processing Unit) dedicated to specific processing. The dedicated circuit is, for example, an FPGA (Field-Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), etc. The control unit 103 performs information processing related to the operation of the server device 10 while controlling each unit of the server device 10.
[0021] The functions of the server device 10 are realized by executing a control program by a processor included in the control unit 103. The control program is a program for causing a computer to function as the server device 10. In addition, a part or all of the functions of the server device 10 may be realized by a dedicated circuit included in the control unit 103. In addition, the control program may be stored in a non-transitory recording / storage medium readable by the server device 10, and read from the medium by the server device 10.
[0022] Next, the configuration of the terminal device 12 will be described in detail.
[0023] The communication unit 121 has a communication module corresponding to a wired or wireless LAN standard, a module corresponding to a mobile communication standard such as LTE, 4G, and 5G, etc. The terminal device 12 is connected to the network 11 via a nearby router device or a mobile communication base station through the communication unit 121, and performs information communication with the server device 10, etc. via the network 11.
[0024] The storage unit 122 includes one or more semiconductor memories, one or more magnetic memories, one or more optical memories, or a combination of at least two of them. The semiconductor memory is, for example, a RAM or a ROM. The RAM is, for example, an SRAM or a DRAM. The ROM is, for example, an EEPROM. The storage unit 122 functions as, for example, a main storage device, an auxiliary storage device, or a buffer memory. The storage unit 122 stores information used in the operation of the control unit 123 and information obtained by the operation of the control unit 123.
[0025] The control unit 123 has, for example, one or more general-purpose processors such as a CPU or an MPU (Micro Processing Unit), or one or more dedicated processors such as a GPU dedicated to specific processing. Alternatively, the control unit 123 may have one or more dedicated circuits such as an FPGA or an ASIC. The control unit 123 operates according to a control / processing program, or according to an action step implemented as a circuit, thereby centrally controlling the actions of the terminal device 12. In addition, the control unit 123 sends and receives various information with the server device 10, etc. via the communication unit 121, and performs the actions involved in this embodiment.
[0026] The input unit 125 includes one or more input interfaces. The input interface includes, for example, a physical keyboard, a capacitive keyboard, a pointing device, and a touch panel integrally provided with the display. In addition, the input interface includes one or more cameras for importing captured images. The camera may also be configured as a different entity from the terminal device 12, and configured to be able to communicate with the terminal device 12 directly or via the network 11. The camera is configured to be able to capture the position and angle of the drawing area 21 when the user 20 is drawing at hand. In addition, the camera may also be configured to be able to capture the range of the user 20 who is drawing, for example, including the face. In addition, the input interface may also include a microphone for receiving voice input. The input unit 125 receives an operation to input information used in the operation of the control unit 123, or performs shooting, sound collection, etc., and sends the input information, captured images, voice and other information to the control unit 123.
[0027] The output unit 126 includes one or more output interfaces. The output interface includes, for example, a display, a projector, and a speaker. The display is, for example, an LCD or an organic EL display. The display may also be integrally formed as a touch panel with the input interface of the input unit 125. The projector may be configured to be able to communicate directly with the terminal device 12 or to be able to communicate via the network 11. The output unit 126 outputs information obtained by the operation of the control unit 123.
[0028] The functions of the control unit 123 are realized by executing a control program by a processor included in the control unit 123. The control program is a program for causing the processor to function as the control unit 123. In addition, a part or all of the functions of the control unit 123 may be realized by a dedicated circuit included in the control unit 123. In addition, the control program may be stored in a non-transitory recording / storage medium readable by the terminal device 12, and read from the medium by the terminal device 12.
[0029] In the present embodiment, the control unit 123 captures the range including the face of the user 20 and the depicted area 21 through the input unit 125, collects the uttered voice of the user 20, and receives the depicted image 22. In addition, the control unit 123 encodes the output image and the voice information for playing the user's voice to generate encoding information. The output image has an arbitrary number of pixels, such as the so-called 4K (3840×2160) or more. The control unit 123 may also perform arbitrary processing (such as resolution change and trimming) on the output image during encoding. The control unit 123 sends the encoding information to other terminal devices 12 via the server device 10 through the communication unit 121.
[0030] In addition, the control unit 123 receives the coded information transmitted from the other terminal device 12 via the server device 10 through the communication unit 121. After decoding the coded information, the control unit 123 uses the decoded information to output the output image through the display or projector of the output unit 126, or output the voice through the speaker. The display or projector for displaying the depicted image 22 of the other party has an arbitrary output resolution, such as so-called 4K or more. The image of the other party user 20 can be expressed in 2D or in 3D model. The display for displaying the 3D model of the other party user 20 can be, for example, a light field display that allows the 3D image to be visually recognized.
[0031] Figure 3 1 is a flowchart for explaining the operation procedure of the terminal device 12. Here, the operation of the terminal device 12a when the terminal device 12a and the terminal device 12b transmit and receive information is taken as an example. Figure 3 The steps are executed by the control unit 123 of the terminal device 12a at a period of several milliseconds to several tens of milliseconds which is arbitrarily set, for example.
[0032] In step S300 , the control unit 123 captures a range including the face of the user 20 a , obtains the depicted image 22 a based on the capture or touch input, and collects the uttered voice. The control unit 123 obtains the captured image, the depicted image 22 a , and the voice information of the user 20 a from the input unit 125 .
[0033] In step S302, the control unit 123 receives the captured image, the drawn image 22b, and the voice information of the user 20b from the terminal device 12b. Figure 4 As shown in example 401, a captured image 40 including a drawing image 22b drawn in a drawing area 21b is received from a terminal device 12b.
[0034] In step S304, the control unit 123 generates a display image. The display image is an image that is displayed in a superimposed manner in the drawing area 21a, and includes the drawing image 22b. Figure 4 As shown in example 402, a display image 41 is generated to be displayed in a superimposed manner in the drawing area 21a. The display image 41 includes the drawing image 22b.
[0035] In step S306, the control unit 123 generates a transmission image. The transmission image is an image that is superimposed and displayed in the drawing area 21b, and includes the drawing image 22a when drawn in the drawing area 21a.
[0036] In step S308, the control unit 123 determines whether the display magnification of the display image has been changed. The display magnification is changed by the user 20a clicking, sliding, etc. on the touch panel of the input unit 125 of the terminal device 12a, or by specifying the magnification based on a pointing device or a keyboard. When the display magnification is changed by accepting the relevant operation ("Yes"), the control unit 123 proceeds to steps S310 and S312. On the other hand, when the display magnification is not changed ("No"), the control unit 123 omits steps S310 and S312 and proceeds to step S314.
[0037] In step S310, the control unit 123 adjusts the display magnification of the display image. The control unit 123 adjusts the display magnification of the display image according to the changed display magnification. When the display magnification is changed from 1 to m (m is a positive real number other than 1), the control unit 123 adjusts the display image by making the range of the display image including at least a part of the depicted image 22b sent from the terminal device 12b become m times. For example, Figure 4 As shown in example 403, the display image 41 for superimposed display in the drawing area 21a is adjusted to enlarge the drawing image 22b.
[0038] In step S312, the control unit 123 adjusts the display magnification of the image for transmission. The control unit 123 adjusts the display magnification of the image for transmission by making the image 22a drawn by the user 20a in the terminal device 12a (1 / m) times according to the changed display magnification m. Figure 4In example 404, the depicted image 22a is depicted in the depicted area 21a relative to the enlarged depicted image 22b. In this case, since the depicted image 22b is enlarged, the transmission image 42 is adjusted so that the depicted image 22a is reduced as shown in example 405. In addition, the transmission image 42 is generated in step S306 and adjusted in step S312.
[0039] In step S314, the control unit 123 displays a display image and outputs a voice. The display of the display image includes displaying the captured image of the user 20b and displaying the drawing image 22b in the drawing area 21a. For example, the display Figure 4 The display image 41 is shown in example 403.
[0040] In step S316, the control unit 123 sends out the video and audio information for transmission. The control unit 123 encodes the video and audio information for transmission and sends out the encoded information to the terminal device 12b.
[0041] Figure 3 The steps of are also performed in the terminal device 12b. By performing step S314 in the terminal device 12b, as shown in FIG. Figure 4 The display image 43 as shown in the example 406 is superimposed and displayed in the drawing area 21b. Here, the drawing image 22a added by the terminal device 12a to the drawing image 22b drawn initially is included in the display image 41.
[0042] Thus, the terminal devices 12a and 12b execute Figure 3 The steps of sending and receiving the depicted image 22 and the communication are performed. Therefore, the users 20 of the terminal devices 12 can send and receive additional depicted images 22 while observing the depicted image 22 of the other party in the depicted area 21. At this time, since the depicted image 22 of the other party is displayed in an appropriately scaled manner, the visibility of the depicted image 22 of the other party is improved, and the additional drawing becomes easy. Therefore, the difference in the input and output environment of the images of the respective terminal devices 12 can be adjusted, which is beneficial to the convenience of the users 20.
[0043] In the embodiment, the terminal device 12 is configured to store the depicted image 22 of the other party or the additionally depicted depicted image 22 to be displayed in advance in the storage unit 122, and to be displayed at any time in response to the operation of the user 20. In addition, when the terminal device 12a and the terminal device 12b communicate with the terminal device 12c used by another user 20c, the terminal device 12a may be configured to switch the image displayed in response to the operation of the user 20a between the depicted image 22b and the depicted image 22c at any time. In addition, the terminal device 12a may be configured to set the display time of the depicted image 22b and the depicted image 22c in response to the operation of the user 20a.
[0044] Although the embodiments have been described above according to the drawings and examples, it should be noted that those skilled in the art can easily make various modifications and corrections according to the present disclosure. Therefore, it should be noted that these modifications and corrections are included in the scope of the present disclosure. For example, each unit, each step or the functions included in each step, etc. can be reconfigured in a logically non-contradictory manner, and multiple units, steps, etc. can be combined into one or divided. [Explanation of Reference Numerals]
[0045] 1. Communication System 10 Server Device 11 Network 12 Terminal Devices 20 users 21 Drawing area 22 Depicting 101, 121 Department of Communications 102, 122 Storage Department 103, 123 control unit 105 Input section 106 Output
Claims
1. A terminal device, comprising: Department of Communications; and A control unit that communicates via the communication unit, The control unit receives information from another terminal device for displaying to a user a first image drawn in a first drawing area by another user using the other terminal device, and sends to the other terminal device information for displaying a second image drawn by the user in a second drawing area overlapping the displayed first image so as to overlap with the first drawing area.
2. The terminal device according to claim 1, wherein: The first image is enlarged and displayed in an overlapping manner in the second drawing area, and the second image is reduced and displayed in an overlapping manner in the first drawing area.
3. The terminal device according to claim 1, wherein: The first image is reduced and displayed in an overlapping manner in the second drawing area, and the second image is enlarged and displayed in an overlapping manner in the first drawing area.
4. The terminal device according to claim 1, wherein: The control unit is capable of switching between display and non-display of the first image.
5. The terminal device according to claim 4, wherein: The control unit receives information from another terminal device for displaying a third image drawn in a third drawing area by another user using the other terminal device to the user, and sends information to the other terminal device for displaying a fourth image drawn in a fourth drawing area by the user overlapping the third drawing area, wherein the fourth drawing area overlaps the third image displayed when the first image is set to non-display.
