COMMUNICATION TERMINAL, SCREEN SHARING METHOD, AND PROGRAM

Abstract

To provide a communication terminal that even when screen resolution of a display unit of the communication terminal is lower than screen resolution of a shared image, displays the shared image without reducing its definition.SOLUTION: A communication terminal for receiving, via a network, a shared image having screen resolution and aspect ratio displayed by a communication terminal of a sharing source and displaying the shared image on a display unit includes display control means that when the aspect ratio of the received shared image differs from an aspect ratio of the display unit, cuts out a predetermined region of the shared image on the basis of the number of pixels in at least one of longitudinal and lateral directions of the display unit and makes the cut predetermined region of the shared image be displayed on the display unit.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present invention relates to a communication terminal, screen sharing method, and Program To Regarding. [Background technology]

[0002] In recent years, with the spread of the Internet, cloud computing has come to be used in various fields. Cloud computing is a form of service usage in which a user uses a communication terminal connected to the Internet to use a service (cloud service) provided by a server on the Internet and pays for that service.

[0003] For example, when a plurality of communication terminals share a screen, a technique for displaying the screen in accordance with the communication terminal even if the resolution is different has been conventionally known (see, for example, Patent Document 1). Summary of the Invention [Problem to be solved by the invention]

[0004] For example, in a screen sharing system in which multiple communication terminals share a screen, each communication terminal displays a shared image according to the screen resolution of its display. Screen resolution is the total number of pixels displayed on a display unit such as a display. Therefore, in conventional screen sharing systems, when the screen resolution of a shared image is greater than the screen resolution of the display unit of a communication terminal, the shared image is reduced for display, resulting in a problem of reduced definition. Note that the technology described in Patent Document 1 does not solve the above-mentioned problem.

[0005] An embodiment of the present invention aims to provide a communication terminal that displays a shared image without reducing resolution, even if the screen resolution of the display unit of the communication terminal is smaller than the screen resolution of the shared image. [Means for solving the problem]

[0006] In order to achieve the above object, claim 1 of the present application provides a communication terminal that receives a shared image having a screen resolution and aspect ratio displayed by a sharing communication terminal via a network and displays the image on a display unit, and when the aspect ratio of the received shared image differs from the aspect ratio of the screen of the display unit, the communication terminal includes a display control means that cuts out a predetermined area of ​​the shared image based on the number of pixels in at least one of the vertical and horizontal directions of the screen of the display unit and displays the cut-out predetermined area of ​​the shared image on the display unit. and when the number of pixels in the horizontal direction of the shared image is greater than the number of pixels in the horizontal direction of the screen of the display unit and the number of pixels in the vertical direction of the shared image is less than the number of pixels in the vertical direction of the screen of the display unit, the display control means cuts out a predetermined area of ​​the shared image based on the number of pixels in the horizontal direction of the screen of the display unit. It is characterized by: Effect of the Invention

[0007] According to an embodiment of the present invention, even if the screen resolution of the display unit of the communication terminal is smaller than the screen resolution of the shared image, the shared image can be displayed without reducing the resolution. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a system configuration diagram of an example of a screen sharing system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of a communication terminal and an extension terminal according to the embodiment. [Diagram 3] FIG. 2 is a diagram illustrating an example of a hardware configuration of a computer according to the present embodiment. [Figure 4] FIG. 2 is a functional block diagram of an example of a communication terminal according to the present embodiment. [Diagram 5] 11 is a conceptual diagram of an example of a process for displaying a shared image on a plurality of communication terminals via a distribution management device. [Figure 6] 11 is a conceptual diagram of an example of a process for displaying a shared image on a plurality of communication terminals via a distribution management device. [Figure 7] 11 is a sequence diagram illustrating an example of a synchronization process of a shared image in the screen sharing system according to the embodiment. FIG. [Figure 8] 11A and 11B are diagrams illustrating an example of a process for cutting out a predetermined area of ​​a shared image; [Figure 9] 11A and 11B are diagrams illustrating an example of a process for cutting out a predetermined area of ​​a shared image; [Figure 10] 11A and 11B are diagrams illustrating an example of a process for cutting out a predetermined area of ​​a shared image; [Figure 11] 11A and 11B are diagrams illustrating an example of a process for cutting out a predetermined area of ​​a shared image; [Figure 12] 11A and 11B are diagrams illustrating an example of a process for cutting out a predetermined area of ​​a shared image; [Figure 13] 11A and 11B are diagrams illustrating an example of a process for cutting out a predetermined area of ​​a shared image; [Figure 14] 11 is a sequence diagram illustrating an example of a synchronization process of a shared image in the screen sharing system according to the embodiment. FIG. [Figure 15] 11 is a sequence diagram illustrating an example of a synchronization process of a shared image in the screen sharing system according to the embodiment. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0010] [First embodiment] <System configuration> Fig. 1 is a system configuration diagram of an example of a screen sharing system according to this embodiment. In the screen sharing system 1 shown in Fig. 1, a distribution management device 2, a communication terminal 5a, a communication terminal 5b, an extension terminal 6, and a web server 8 communicate with each other via a network 9. The communication terminals 5a and 5b may be collectively referred to as a communication terminal 5. Although Fig. 1 shows an example in which there are two communication terminals 5, there may be three or more communication terminals.

[0011] The communication terminal 5 is an example of an information processing terminal operated by a user who receives a service of the screen sharing system 1. The multiple communication terminals 5 receive the shared image via the network 9 and display it on a display, which is an example of a display unit.

[0012] For example, the communication terminals 5a and 5b are electronic whiteboards having a function of displaying a shared image and a function of accepting an operation by an operator such as a user using a finger or an electronic pen. The communication terminal 5a is connected to the extension terminal 6 in a controllable manner. The communication terminal 5a has an extension mode in which the display of the communication terminal 5a and the display of the extension terminal 6 are displayed as one large screen. The extension terminal 6 may be any electronic device that can realize the extension mode with the communication terminal 5a, and may be, for example, an electronic whiteboard similar to the communication terminal 5a. The communication terminal 5a displays a shared image by using the display of the communication terminal 5a and the display of the extension terminal 6 as one large screen. Also, for example, the communication terminal 5b displays a shared image on the display of the communication terminal 5a.

[0013] The communication terminal 5 is not limited to an electronic whiteboard, and may be, for example, a projector, a video conference terminal, a PC (Personal Computer), a smartphone, or a tablet terminal.

[0014] The network 9 is a communication network capable of data communication such as the Internet or a LAN (Local Area Network), etc. The network 9 also includes a wireless communication network such as 3G (3rd Generation), WiMAX (Worldwide Interoperability for Microwave Access), or LTE (Long Term Evolution).

[0015] The distribution management device 2 has a so-called cloud browser (hereinafter referred to as "browser") as a web browser existing on the cloud, and obtains a content image by rendering web content on the cloud. The distribution management device 2 can distribute the obtained content image such as H.264 or MPEG-4 to the communication terminal 5. The content image can be used as a shared image to be shared by the communication terminal 5 of the screen sharing system 1 in FIG. 1. In addition, the distribution management device 2 provides a real-time communication / collaboration (RTC) function. A web server 8 provides web content to the browser of the distribution management device 2.

[0016] <Hardware configuration of communication terminal and extension terminal> The hardware configuration of the communication terminal 5 and the extension terminal 6 according to this embodiment will be described with reference to Fig. 2. Fig. 2 is a hardware configuration diagram of an example of a communication terminal and an extension terminal according to this embodiment. Fig. 2 describes an example of a communication terminal 5a connected to the extension terminal 6. The hardware configuration of the communication terminal 5b is the same as that of the communication terminal 5a.

[0017] 2 has a display 300, a controller 350, and a display input unit 358. The display 300 has a touch unit 302, a panel unit 304, a video input terminal 306, a power linkage unit 308, and a controller slot 310. The controller 350 has a communication unit 352, a control unit 354, and a storage unit 356.

[0018] The touch unit 302 detects operations performed by an operator using a finger or an electronic pen. The panel unit 304 displays video (images). The video input terminal 306 accepts video input from the outside. For example, the video input terminal 306 accepts an HDMI (registered trademark) input, a VGA input, or an internal input from a controller 350.

[0019] The power interlocking unit 308 coordinates power control when connected to the extension terminal 6. The power interlocking unit 308 coordinates, for example, the turning on or off of the power of the communication terminal 5a and the extension terminal 6. The power interlocking unit 308 coordinates, for example, the transition to or return from an energy saving mode of the communication terminal 5a and the extension terminal 6. The controller slot 310 is used to connect to the controller 350.

[0020] The display 300 of the communication terminal 5a can function as an extension terminal by itself. For example, as shown in FIG. 2, the display 300 of the communication terminal 5a and the display 200 of the extension terminal 6 have the same configuration. The display 200 of the extension terminal 6 has a touch unit 202, a panel unit 204, a video input terminal 206, a power supply interlock unit 208, and a controller slot 210.

[0021] The display 200 can function as a display unit of the extension terminal 6 by connecting a controller 350 to the controller slot 210. The display 300 can function as a display unit of the communication terminal 5a by connecting a controller 350 to the controller slot 310.

[0022] The controller 350 also has a communication unit 352, a control unit 354, and a storage unit 356. The communication unit 352 is connected to the network 9. The control unit 354 performs various input / output control or calculations. The storage unit 356 is an SSD (Solid State Drive) or an HDD (Hard Disk Drive) and stores data. The storage unit 356 acquires target data to be screen shared from the distribution management device 2 and stores it.

[0023] The control unit 354 includes, for example, a central processing unit (CPU), a random access memory (RAM), and a read only memory (ROM). The control unit 354 realizes a functional configuration described below by the CPU using the RAM as a work area and executing a program stored in the ROM or the storage unit 356.

[0024] When the extension terminal 6 is connected, the control unit 354 of the communication terminal 5a can acquire position information (described later) from both the touch unit 302 of the communication terminal 5a and the touch unit 202 of the extension terminal 6. In addition, when the extension terminal 6 is connected, the control unit 354 can output video to both the video input terminal 306 of the panel unit 304 of the communication terminal 5a and the video input terminal 206 of the panel unit 204 of the extension terminal 6.

[0025] <Hardware configuration of distribution management device and web server> The distribution management device 2 and web server 8 shown in Fig. 1 are realized, for example, by a computer 500 having a hardware configuration shown in Fig. 3. Moreover, the communication terminals 5a and 5b shown in Fig. 1 may be realized, for example, by a computer 500 having a hardware configuration shown in Fig. 3. Fig. 3 is a diagram showing an example of the hardware configuration of a computer according to this embodiment.

[0026] The computer 500 includes a CPU 501, a ROM 502, a RAM 503, a HD 504, a HDD controller 505, a display 506, an external device connection I / F 508, a network I / F 509, a data bus 510, a keyboard 511, a pointing device 512, a DVD-RW (Digital Versatile Disk Rewritable) drive 514, and a media I / F 516.

[0027] The CPU 501 controls the overall operation of the computer 500 according to a program. The ROM 502 stores programs such as an IPL used to drive the CPU 501. The RAM 503 is used as a work area for the CPU 501. The HD 504 stores various data such as programs. The HDD controller 505 controls the reading and writing of various data from and to the HD 504 according to the control of the CPU 501.

[0028] The display 506 displays various information such as a cursor, a menu, a window, characters, or an image. The external device connection I / F 508 is an interface for connecting various external devices. An external device is, for example, a USB memory. The network I / F 509 is an interface for data communication using the network 9. The data bus 510 is an address bus, a data bus, or the like for electrically connecting each component such as the CPU 501.

[0029] The keyboard 511 is a type of input means having a plurality of keys for inputting characters, numbers, various instructions, etc. The pointing device 512 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, etc. The DVD-RW drive 514 controls reading and writing of various data from a DVD-RW 513 as an example of a removable recording medium. Note that the medium is not limited to a DVD-RW, and may be a DVD-R or the like. The media I / F 516 controls reading and writing (storing) of data from a recording medium 515 such as a flash memory.

[0030] <Functional configuration of communication terminal> Next, functions of the communication terminal 5 will be described with reference to Fig. 4. Fig. 4 is a functional block diagram of an example of a communication terminal according to this embodiment. Each function of the communication terminal 5 is a function or means realized by operating any of the components shown in Fig. 2 by an instruction from a CPU according to a program deployed in a work area.

[0031] The communication terminal 5 in Fig. 4 includes a receiving unit 20, a transmitting unit 22, a display control unit 24, and a detecting unit 26. Note that Fig. 4 omits illustration of functions that are not necessary for explaining this embodiment.

[0032] The receiving unit 20 communicates with the communication unit 352 and receives data from the communication unit 352. For example, the receiving unit 20 receives information specifying target data to be screen-shared with other communication terminals 5, such as conference materials, and notifies the display control unit 24. The transmitting unit 22 communicates with the communication unit 352 and transmits data to the communication unit 352. The receiving unit 20 and the transmitting unit 22 are realized, for example, by the CPU of the control unit 354 shown in FIG. 2 using the RAM as a work area to execute processing according to a program.

[0033] The display control unit 24 controls display such as drawing processing and image output. The display control unit 24 acquires target data to be shared on the screen from the storage unit 356 and uses it for control of display such as drawing processing and image output. The display control unit 24 may have a configuration including a drawing processing unit and an image output unit. The display control unit 24 is realized, for example, by an instruction from the CPU of the control unit 354 shown in FIG. 2, the panel unit 304, etc. The detection unit 26 detects position information of an operation by the operator's finger or electronic pen on the touch unit 302. The detection unit 26 obtains coordinate data of a position where the operator's finger or electronic pen touches or approaches the touch unit 302, and detects handwritten input or operation by the operator from the obtained coordinate data. The detection unit 26 is realized, for example, by an instruction from the CPU of the control unit 354 shown in FIG. 2, the touch unit 302, etc.

[0034] <Example of shared image display> 5 is a conceptual diagram of an example of a process for displaying a shared image on a plurality of communication terminals via a distribution management device. At a first location (left side of FIG. 5), a communication terminal 5a, an extension terminal 6, and a communication terminal 7a serving as a video conference terminal are used. At a second location (right side of FIG. 5), a communication terminal 5b and a communication terminal 7b serving as a video conference terminal are used.

[0035] At the first location, an electronic pen P2 is used to display operation data such as stroke characters on a communication terminal 5a. At the second location, an electronic pen P1 is used to display operation data such as stroke characters on a communication terminal 5b.

[0036] At the first location, the video and audio data acquired by the communication terminal 7a is encoded by the encoding unit 40 and then transmitted to the distribution management device 2. The video and audio data is then decoded by the decoding unit 36 ​​of the distribution management device 2 and input to the browser 30. In addition, operation data indicating strokes and the like drawn on the communication terminal 5a by the electronic pen P2 is transmitted to the distribution management device 2 and input to the browser 30.

[0037] Meanwhile, also in the second base, the video and audio data acquired by the communication terminal 7b is encoded by the encoding unit 40 and then transmitted to the distribution management device 2. Then, the video and audio data is decoded by the decoding unit 36 ​​of the distribution management device 2 and input to the browser 30. In addition, operation data such as strokes drawn on the communication terminal 5b by the electronic pen P1 is transmitted to the distribution management device 2 and input to the browser 30.

[0038] The browser 30, for example, acquires web content data [A] of background images to be displayed at the first location and the second location from the web server 8. The browser 30 acquires operation data [p2] and video and audio data [E2] from the first location. The browser 30 also acquires operation data [p1] and video and audio data [E1] from the second location.

[0039] The browser 30 combines and renders the web content data [A], the operation data [p1] and [p2], and the video and audio data [E1] and [E2]. The browser 30 generates a shared image with the screen resolution and aspect ratio of area b of the screen in extended mode realized by the communication terminal 5a and the extended terminal 6 at the first location, for example. The screen resolution is the total number of pixels on the screen. For example, the screen resolution and aspect ratio of area b of the screen in extended mode realized by the communication terminal 5a and the extended terminal 6 at the first location are the screen resolution and aspect ratio of the screens of the communication terminal 5a and the extended terminal 6 treated as one large screen.

[0040] The encoder bridge unit 32 encodes the shared image generated by the browser 30, and the distribution management device 2 distributes it to each base. At the first base, the shared image in the area b is displayed on the screen in the extended mode using the display 300 of the communication terminal 5a and the display 200 of the extended terminal 6, and sound based on the video and audio data [E1] of the second base is output from the communication terminal 7a.

[0041] Meanwhile, at the second location, the shared image of area a is displayed on the screen of the display 300 of the communication terminal 5b, and sound based on the video and audio data [E2] of the first location is output from the communication terminal 7b. The screen sharing system 1 according to this embodiment is effective for remote conferences, etc., since it is possible to share the same information in real time between remote locations such as the first location and the second location.

[0042] In the example of Fig. 5, the shared image in area b is displayed in accordance with the landscape screen in extended mode at the first location. On the other hand, the display 300 of the communication terminal 5b at the second location has a portrait screen. Therefore, in the example of Fig. 5, if it is to be displayed with the same aspect ratio as the shared image in area b displayed at the first location, it is necessary to reduce the shared image to area a so that it fits on the display 300 of the communication terminal 5b. Therefore, the shared image is reduced and displayed at the second location, and the definition of the shared image decreases.

[0043] Therefore, in this embodiment, as shown in Fig. 6, the second location cuts out area d from the shared image of area b displayed at the first location as described below, and displays area d of the cut-out shared image on display 300 of communication terminal 5b. Fig. 6 is a conceptual diagram of an example of a process for displaying a shared image on multiple communication terminals via a distribution management device. As shown in Fig. 6, when the screen resolution and aspect ratio of the shared image differ from the screen resolution and aspect ratio of display 300 of communication terminal 5b, and the entire shared image cannot be displayed without being reduced, communication terminal 5b at the second location performs the following process.

[0044] Communications terminal 5b cuts out a predetermined area of ​​the shared image based on the number of pixels in at least one of the vertical and horizontal directions of display 300 of communications terminal 5b, and displays area d of the cut-out shared image on display 300 of communications terminal 5b. In the example of Fig. 6, since the number of pixels in the horizontal direction of the shared image is greater than the number of pixels in the horizontal direction of display 300 of communications terminal 5b, area d of the shared image is cut out and displayed based on the number of pixels in the horizontal direction of display 300 of communications terminal 5b.

[0045] Furthermore, if the number of vertical pixels of the shared image is greater than the number of vertical pixels of display 300 of communication terminal 5b, communication terminal 5b cuts out the area of ​​the shared image based on the number of vertical pixels of display 300 and displays it.

[0046] Furthermore, if the number of vertical and horizontal pixels of the shared image is greater than the number of vertical and horizontal pixels of the display 300 of the communication terminal 5b, the communication terminal 5b cuts out and displays an area of ​​the shared image based on the number of vertical and horizontal pixels of the display 300.

[0047] In this way, communication terminal 5b receives the entire shared image with the screen resolution and aspect ratio of area b displayed at the first location, and cuts out and displays a predetermined area of ​​the shared image to be displayed on display 300 from the received shared image data. Furthermore, the predetermined area of ​​the shared image cut out by communication terminal 5b changes according to an operation (such as a touch operation or a drag operation) with the operator's finger or electronic pen detected by detection unit 26. When the operator wishes to display an area other than the predetermined area of ​​the shared image displayed on display 300 of communication terminal 5b, he or she can change the display range by operating with a finger or electronic pen.

[0048] Fig. 7 is a sequence diagram of an example of a synchronization process of a shared image in the screen sharing system according to this embodiment. In the following description, an example will be described in which the communication terminal 5a at the first location is a presenter terminal operated by a presenter, and the communication terminal 5b at the second location is a participant terminal operated by a participant. Although Fig. 7 shows one participant terminal, multiple participants may be used.

[0049] In step S10, the presenter operates the shared screen of the communication terminal 5a, such as zooming in, zooming out, and page forwarding. In step S12, the detection unit 26 of the communication terminal 5a detects a change in the display range. The operation on the display screen is, for example, a scroll operation, a pinch-in / pinch-out operation, a swipe operation, a flick operation, a double-tap operation, and the like. In step S14, the detection unit 26 notifies the transmission unit 22 of the change in the display range.

[0050] In step S16, the transmission unit 22 transmits a page synchronization message to the conference server device 101. The page synchronization message includes information specifying a display range. For example, the transmission and reception of a screen sharing synchronization message such as a page synchronization message may use a communication protocol such as XMPP (Extensible Messaging and Presence Protocol) or another communication protocol.

[0051] In step S18, the distribution management device 2 transfers the received page synchronization message to one or more communication terminals 5b. In step S20, the receiving unit 20 of the communication terminal 5b notifies the display control unit 24 of reception of the page synchronization message. In step S22, the display control unit 24 requests the display area of ​​the display unit from the detection unit 26. In step S24, the display control unit 24 receives a response of the display area of ​​the display unit from the detection unit 26.

[0052] In step S26, the display control unit 24 requests the target data for screen sharing from the storage unit 356. In step S28, the display control unit 24 acquires the target data for screen sharing from the storage unit 356. Note that in a case where the target data for screen sharing does not exist in the storage unit 356, or in other cases where it is necessary to acquire new target data for screen sharing, a process of receiving the target data for screen sharing from the distribution management device 2 may be performed.

[0053] In step S30, the display control unit 24 of the communication terminal 5b cuts out an area d to be displayed on the display 300 from the shared image of the area b displayed at the first location, as shown in Fig. 6. In step S32, the display control unit 24 displays on the display 300 the area d of the shared image cut out in step S30.

[0054] In step S34, the display control unit 24 performs enlargement or reduction processing so that the shared image in area b displayed at the first location is displayed in, for example, area c, which is the display area of ​​the display 300 of the communication terminal 5b shown in Fig. 6. In the example of Fig. 6, reduction processing is performed so that the shared image in area b displayed at the first location can be displayed in area c of the display 300 of the communication terminal 5b.

[0055] In step S36, the display control unit 24 superimposes, for example, display area information onto the area c so that it is possible to recognize which part of the shared image the area d corresponds to. The display area information is, for example, information in which an area corresponding to the area d is represented by a rectangle or the like on the reduced shared image displayed in the area c. In step S38, the display control unit 24 displays the shared image superimposed with the display area information in the area c.

[0056] The process of step S30 for cutting out area d to be displayed on display 300 from the shared image of area b displayed at the first location is performed, for example, as follows: Figures 8 to 13 are explanatory diagrams of an example of the process of cutting out a predetermined area of ​​a shared image.

[0057] Fig. 8 shows an example of a shared image, a camera image of the sharing source, a camera image of the sharing destination, and a screen of a display unit at a second location, which are necessary for explaining the process of step S30. Fig. 8(a) shows a shared image displayed on the display unit of the sharing source, which performs screen sharing. Fig. 8(b) shows a camera image (acquired video) taken by a communication terminal 7a at a first location, which is the sharing source of the shared image. Fig. 8(c) shows a camera image taken by a communication terminal 7b at a second location, which is the sharing destination of the shared image. Fig. 8(d) shows a display screen (display area) of a display 300 of a communication terminal 5b at a second location, which is the sharing destination of the shared image.

[0058] In the example of the shared image, the camera image from which the image is shared, the camera image of the sharer, and the screen of the display unit of the second location shown in Figure 8, in the processing of step S30, a specified area of ​​the shared image is cut out, for example as shown in Figure 9, and displayed on the display 300 of the communication terminal 5b.

[0059] Fig. 9 shows an example in which the number of horizontal pixels (horizontal size) of a shared image for screen sharing is greater than the number of horizontal pixels of display 300 of sharing destination communication terminal 5b. The display screen of display 300 of sharing destination communication terminal 5b shown in Fig. 9(a) has fewer pixels in the horizontal direction than the shared image, as indicated by the arrow in Fig. 9(b).

[0060] Therefore, communication terminal 5b cuts out a predetermined area of ​​the shared image based on the number of pixels in the horizontal direction of display 300 of communication terminal 5b, as indicated by the arrow in Fig. 9(b), and displays area d of the cut-out shared image on the display screen of display 300 of communication terminal 5b. Also, in Fig. 9(b), the camera image of the sharing source and the camera image of the sharing destination are displayed below area d of the cut-out shared image.

[0061] In addition, in the example of the shared image, the camera image from which the image is shared, the camera image of the destination of the image, and the screen of the display unit of the second location shown in FIG. 8, in the processing of step S30, a predetermined area of ​​the shared image may be cut out, for example, as shown in FIG. 10, and displayed on the display 300 of the communication terminal 5b.

[0062] Fig. 10 shows an example in which the number of pixels in the horizontal direction (horizontal size) of a shared image for screen sharing is greater than the number of pixels in the horizontal direction (horizontal size) of the display 300 of the communication terminal 5b as the sharing destination. Fig. 10(b) shows a state similar to Fig. 9(b). There is unused display area on the display screen in Fig. 10(b).

[0063] Therefore, communication terminal 5b expands a specified area of ​​the shared image in the direction in which the unused display area decreases, as indicated by the arrow in Figure 10 (c), and then cuts it out based on the number of pixels in the horizontal direction of display 300 of communication terminal 5b, and displays the cut-out area d of the shared image on the display screen of display 300 of communication terminal 5b.

[0064] Fig. 11 shows an example in which the number of vertical pixels (horizontal size) of a shared image for screen sharing is the same as the number of vertical pixels of display 300 of sharing destination communication terminal 5b. The display screen of display 300 of sharing destination communication terminal 5b shown in Fig. 11(a) has the same number of vertical pixels as the shared image, as shown in Fig. 11(b). Also, the display screen of display 300 of sharing destination communication terminal 5b shown in Fig. 11(a) has fewer horizontal pixels than the shared image, as shown in Fig. 11(b).

[0065] Therefore, communication terminal 5b cuts out a predetermined area of ​​the shared image based on the number of pixels in the vertical and horizontal directions of display 300 of communication terminal 5b shown in Fig. 11(b), and displays area d of the cut-out shared image on the display screen of display 300 of communication terminal 5b. In Fig. 11(b), the camera image of the sharing source and the camera image of the sharing destination are displayed superimposed on area d of the cut-out shared image.

[0066] Fig. 12 shows an example of a shared image, a camera image of the sharing source, a camera image of the sharing destination, and a screen of a display unit at a second location, which are necessary for explaining the processing of step S30. Fig. 12(a) shows a shared image displayed on the display unit of the sharing source, which performs screen sharing. Fig. 12(b) shows a camera image taken by a communication terminal 7a at a first location, which is the sharing source of the shared image. Fig. 12(c) shows a camera image taken by a communication terminal 7b at a second location, which is the sharing destination of the shared image. Fig. 12(d) shows the display screen of the display 300 of a communication terminal 5b at a second location, which is the sharing destination of the shared image.

[0067] In the example of the shared image, the camera image from which the image is shared, the camera image of the destination of the image, and the screen of the display unit of the second location shown in Figure 12, in the processing of step S30, a specified area of ​​the shared image is cut out, for example as shown in Figure 13, and displayed on the display 300 of the communication terminal 5b.

[0068] Fig. 13 shows an example in which the number of vertical pixels (vertical size) of a shared image for screen sharing is more (larger) than the number of vertical pixels (vertical size) of display 300 of sharing destination communication terminal 5b. The display screen of display 300 of sharing destination communication terminal 5b shown in Fig. 13(a) has fewer vertical pixels than the shared image, as indicated by the arrow in Fig. 13(b).

[0069] Therefore, communication terminal 5b cuts out a predetermined area of ​​the shared image based on the number of pixels in the vertical direction of display 300 of communication terminal 5b, as indicated by the arrow in Fig. 13(b), and displays area d of the cut-out shared image on the display screen of display 300 of communication terminal 5b. In Fig. 13(b), the camera image of the sharing source and the camera image of the sharing destination are displayed to the right of area d of the cut-out shared image.

[0070] In this way, the process of step S30 for cutting out area d to be displayed on display 300 from the shared image of area b displayed at the first location cuts out and displays a predetermined area of ​​the shared image based on the number of pixels in at least one of the vertical and horizontal directions of the display unit of the sharing destination. Since communication terminal 5b, which is the sharing destination of the shared image, cuts out the predetermined area of ​​the shared image and displays it on the display unit without reducing the shared image, it is possible to display the shared image without reducing definition even if the screen resolution of the display unit of the sharing destination is smaller than the screen resolution of the shared image.

[0071] 14 is a sequence diagram of an example of a synchronization process of a shared image in the screen sharing system according to the present embodiment. In step S50, a participant performs a touch operation such as a drag operation on the area c or d displayed on the display screen of the communication terminal 5b.

[0072] In step S52, the detection unit 26 detects a change in the display range based on an operation performed by the participant on area c or d displayed on the display screen. In step S54, the detection unit 26 notifies the display control unit 24 of the detected change in the display range. In step S56, the display control unit 24 calculates the movement distance (movement amount) of the display area based on the change in the display range notified by the detection unit 26.

[0073] In step S58, the display control unit 24 requests the target data to be screen-shared from the storage unit 356. In step S60, the display control unit 24 acquires the target data to be screen-shared from the storage unit 356. In step S62, the display control unit 24 of the communication terminal 5b cuts out an area d to be displayed on the display 300 from the shared image. In step S64, the display control unit 24 displays the area d of the shared image cut out in step S62 on the display 300.

[0074] In step S66, the display control unit 24 performs a process of reducing the shared image in the area b displayed at the first location so that it can be displayed in the area c of the display 300 of the communication terminal 5b shown in FIG. 6, for example.

[0075] In step S68, the display control unit 24 combines the display area information with the area c so that it is possible to recognize which part of the shared image the area d corresponds to. In step S70, the display control unit 24 displays the shared image combined with the display area information in the area c.

[0076] 15 is a sequence diagram of an example of a synchronization process of a shared image in the screen sharing system according to the present embodiment. In step S80, a participant performs a touch operation such as a pinch-in or pinch-out operation on the area c or d displayed on the display screen of the communication terminal 5b.

[0077] In step S82, the detection unit 26 detects a change in the display range based on an operation performed by the participant on area c or d displayed on the display screen. In step S84, the detection unit 26 notifies the display control unit 24 of the detected change in the display range. In step S86, the display control unit 24 calculates the enlargement or reduction rate (size) of the display area based on the change in the display range notified by the detection unit 26.

[0078] In step S88, the display control unit 24 requests the target data to be screen shared from the storage unit 356. In step S90, the display control unit 24 acquires the target data to be screen shared from the storage unit 356. In step S92, the display control unit 24 of the communication terminal 5b cuts out an area d to be displayed on the display 300 from the shared image. In step S94, the display control unit 24 displays the area d of the shared image cut out in step S92 on the display 300.

[0079] In step S96, the display control unit 24 performs a process of reducing the shared image in the area b displayed at the first location so that it can be displayed in the area c of the display 300 of the communication terminal 5b shown in FIG. 6, for example.

[0080] In step S98, the display control unit 24 combines the display area information with the area c so that it is possible to recognize which part of the shared image the area d corresponds to. In step S100, the display control unit 24 displays the shared image combined with the display area information in the area c.

[0081] <Other application examples> The above describes the best mode for carrying out the present invention using examples, but the present invention is not limited to these examples in any way, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention.

[0082] For example, in this embodiment, the process of cutting out a predetermined area from a shared image is performed in the sharing destination communication terminal 5b, but this may be performed in the distribution management device 2 or in a server other than the distribution management device 2.

[0083] Each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, and a device such as an ASIC (Application Specific Integrated Circuit), a DSP (digital signal processor), an FPGA (field programmable gate array), or a conventional circuit module designed to execute each function described above. [Explanation of symbols]

[0084] 1. Screen sharing system 2. Distribution Management Device 5, 5a, 5b Communication terminal 6. Extended Terminal 8 Web Server 9 Network 20 Receiving section 22 Transmitter 24 Display control section 26 Detection unit 200, 300 Display 202, 302 Touch section 204, 304 Panel section 350 Controller 354 Control Unit 356 Storage Department 358 Display and input section [Prior art documents] [Patent documents]

[0085] [Patent Document 1] JP 2015-60292 A

Claims

1. A communication terminal that receives a shared image having a screen resolution and aspect ratio displayed by a sharing communication terminal via a network and displays the shared image on a display unit, a display control means for, when an aspect ratio of the received shared image differs from an aspect ratio of a screen of the display unit, cropping a predetermined area of ​​the shared image based on the number of pixels in at least one of the vertical and horizontal directions of the screen of the display unit and displaying the cropped predetermined area of ​​the shared image on the display unit; having When the number of pixels in the horizontal direction of the shared image is greater than the number of pixels in the horizontal direction of the screen of the display unit and the number of pixels in the vertical direction of the shared image is less than the number of pixels in the vertical direction of the screen of the display unit, the display control means cuts out a predetermined area of ​​the shared image based on the number of pixels in the horizontal direction of the screen of the display unit. A communication terminal characterized by:

2. A communication terminal that receives a shared image having a screen resolution and aspect ratio displayed by a sharing source communication terminal via a network and displays the shared image on a display unit, a display control means for, when an aspect ratio of the received shared image differs from an aspect ratio of a screen of the display unit, cropping a predetermined area of ​​the shared image based on the number of pixels in at least one of the vertical and horizontal directions of the screen of the display unit and displaying the cropped predetermined area of ​​the shared image on the display unit; having When the number of pixels in the vertical direction of the shared image is greater than the number of pixels in the vertical direction of the screen of the display unit and the number of pixels in the horizontal direction of the shared image is less than the number of pixels in the horizontal direction of the screen of the display unit, the display control means cuts out a predetermined area of ​​the shared image based on the number of pixels in the vertical direction of the screen of the display unit. A communication terminal characterized by:

3. When an aspect ratio of the received shared image differs from an aspect ratio of a screen of the display unit, the display control means enlarges the shared image and then cuts out a predetermined area of ​​the shared image based on the number of pixels in the vertical or horizontal direction of the screen of the display unit.

2. The communication terminal according to claim 1,

4. The display control means changes a predetermined area to be cut out from the shared image based on an operation on a display screen received from an operator. A communication terminal according to any one of claims 1 to 3, characterized in that

5. The display control means further causes display area information indicating which portion of the shared image the predetermined area displayed on the display unit corresponds to to be displayed on the display unit. A communication terminal according to any one of claims 1 to 4, characterized in that

6. The screen resolution and aspect ratio of the shared image are the screen resolution and aspect ratio of an image displayed as a single display screen by one or more communication terminals that are sharing sources using multiple display units.

6. A communication terminal according to claim 1, wherein:

7. A screen sharing method for a communication terminal, which receives a shared image having a screen resolution and aspect ratio displayed by a sharing source communication terminal via a network and displays the shared image on a display unit, a display control step of, when an aspect ratio of the received shared image differs from an aspect ratio of a screen of the display unit, cropping a predetermined area of ​​the shared image based on the number of pixels in at least one of the vertical and horizontal directions of the screen of the display unit, and displaying the cropped predetermined area of ​​the shared image on the display unit; having The display control step includes, when a number of pixels in a horizontal direction of the shared image is greater than a number of pixels in a horizontal direction of the screen of the display unit and a number of pixels in a vertical direction of the shared image is less than a number of pixels in a vertical direction of the screen of the display unit, cutting out a predetermined area of ​​the shared image based on the number of pixels in a horizontal direction of the screen of the display unit. A screen sharing method characterized by:

8. A screen sharing method for a communication terminal, which receives a shared image having a screen resolution and aspect ratio displayed by a sharing source communication terminal via a network and displays the shared image on a display unit, a display control step of, when an aspect ratio of the received shared image differs from an aspect ratio of a screen of the display unit, cropping a predetermined area of ​​the shared image based on the number of pixels in at least one of the vertical and horizontal directions of the screen of the display unit, and displaying the cropped predetermined area of ​​the shared image on the display unit; having The display control step includes, when a number of pixels in a vertical direction of the shared image is greater than a number of pixels in a vertical direction of the screen of the display unit and a number of pixels in a horizontal direction of the shared image is less than a number of pixels in a horizontal direction of the screen of the display unit, cutting out a predetermined area of ​​the shared image based on the number of pixels in a vertical direction of the screen of the display unit. A screen sharing method characterized by:

9. A communication terminal that receives a shared image having a screen resolution and aspect ratio displayed by a sharing source communication terminal via a network and displays the shared image on a display unit, a display control means for, when an aspect ratio of the received shared image differs from an aspect ratio of a screen of the display unit, cropping a predetermined area of ​​the shared image based on the number of pixels in at least one of the vertical and horizontal directions of the screen of the display unit and displaying the cropped predetermined area of ​​the shared image on the display unit; Function as a When the number of pixels in the horizontal direction of the shared image is greater than the number of pixels in the horizontal direction of the screen of the display unit and the number of pixels in the vertical direction of the shared image is less than the number of pixels in the vertical direction of the screen of the display unit, the display control means cuts out a predetermined area of ​​the shared image based on the number of pixels in the horizontal direction of the screen of the display unit. A program characterized by.

10. A communication terminal that receives a shared image having a screen resolution and aspect ratio displayed by a sharing source communication terminal via a network and displays the shared image on a display unit, a display control means for, when an aspect ratio of the received shared image differs from an aspect ratio of a screen of the display unit, cropping a predetermined area of ​​the shared image based on the number of pixels in at least one of the vertical and horizontal directions of the screen of the display unit and displaying the cropped predetermined area of ​​the shared image on the display unit; Function as a When the number of pixels in the vertical direction of the shared image is greater than the number of pixels in the vertical direction of the screen of the display unit and the number of pixels in the horizontal direction of the shared image is less than the number of pixels in the horizontal direction of the screen of the display unit, the display control means cuts out a predetermined area of ​​the shared image based on the number of pixels in the vertical direction of the screen of the display unit. A program characterized by.

Images