Communication terminal, display method, program, and image communication system

Abstract

To solve the problem of a conventional method in which a prohibited region is designated in a wide-range image, such as a panorama image, so as not to be transmitted or displayed from the viewpoint of security to display an image in as wide range as possible in a permitted region, while a receiver of image data has a need to, however roughly, grasp the overall status of a transmitter.SOLUTION: A communication terminal (3) transmits data of an omnidirectional image so as to be received by other communication terminals, the communication terminal being configured to: (S103) transmit a display restriction parameter so as to be received by the other communication terminals, the display restriction parameter restricting a display region within a predetermined region which is a part of the omnidirectional image when the other communication terminals (7, 9) display the omnidirectional image; and (S125, S203, S223) transmit display restriction lifting instruction information indicating an instruction to lift the restriction on the display region so as to be received by the other communication terminals.SELECTED DRAWING: Figure 30

Description

[[Technical Field]]

[0001] The present disclosure relates to a communication terminal, Display method, program , and Image communication system . [[Background Art]]

[0002] Conference systems that hold remote conferences with remote locations via communication networks such as the Internet have become widespread. In this conference system, in a conference room where one of the parties such as attendees attending the remote conference is located, a communication terminal of the remote conference system is used to photograph and collect images of the conference room including the conference parties and audio such as speeches, converts these into digital data, and transmits the digital data to the communication terminal of the other party. This enables image display on the display of the other party's conference room and audio output via a speaker to perform a video call, so a technique for holding a conference between remote locations in a state close to an actual conference is known (see Patent Document 1).

[0003] There is also a known technique in which an imaging device capable of acquiring omnidirectional panoramic images in real time is connected to a communication terminal, the omnidirectional (panoramic) image acquired from the imaging device is distributed to each communication terminal of the other party, and each communication terminal of the other party sequentially performs image conversion processing from the omnidirectional image to a rectangular planar image and displays the image on a display or the like (see Patent Document 2).

[0004] Furthermore, a system is known in which a communication terminal and a device capable of acquiring image data in real time are placed in an office (living room) where an employee normally works, and a connection is established between a communication terminal placed at a remote employee's home or the like, so that the office situation can always be grasped, and at the same time, the office can also always grasp the status of the employee working remotely (see Patent Document 3).

[0005] However, when the transmitted and received image data is 360-degree spherical image data, the receiving site of the 360-degree spherical image data can display a wide area of ​​the transmitting site, which means that even highly confidential information at the transmitting site will be displayed. Therefore, conventionally, a method has been disclosed in which the transmitting site can specify prohibited areas from a wide-area image, such as a panoramic image, that it does not want to transmit or display from security standpoints, and display the widest possible area of ​​the image within the displayable area (see Patent Document 4). [Overview of the project] [Problems that the invention aims to solve]

[0006] However, conventional methods present a challenge: they cannot meet the need for the receiving site of image data to understand the overall situation at the transmitting site, even if only roughly. [Means for solving the problem]

[0007] The invention according to claim 1 includes a determination means for determining whether the type of received image data is a special image, and if the image data is determined to be a special image, determining whether it is subject to display restrictions, and if it is determined that it is subject to display restrictions 、 From the aforementioned image data , the above Image corresponding to a portion of a special image and the image based on the display restriction parameter that gives the display restriction. A communication terminal having a display control means for displaying a predetermined area image. [Effects of the Invention]

[0008] As explained above, the present invention has the effect of meeting the need for the receiving site of image data to grasp the overall situation of the transmitting site, even if it is only a rough overview. [Brief explanation of the drawing]

[0009] [Figure 1] (a) is a left side view of the imaging device, (b) is a front view of the imaging device, and (c) is a top view of the imaging device. [Figure 2] This is a diagram illustrating the use of the imaging device. [Figure 3] (a) is a hemispherical image (front) taken with the imaging device, (b) is a hemispherical image (back) taken with the imaging device, and (c) is an image represented using the Mercator projection. [Figure 4] (a) A conceptual diagram showing the state of the sphere being covered by a Mercator image, and (b) A diagram showing a panoramic image of the entire sky. [Figure 5] This diagram shows the positions of a virtual camera and a predetermined area when a full-spherical panoramic image is treated as a three-dimensional sphere. [Figure 6] (a) is a stereoscopic perspective view of Figure 5, (b) is a diagram showing the state in which an image of a predetermined area is displayed on the display of a communication terminal, and (c) is a diagram showing a new predetermined area image when a part of Figure 6(b) is enlarged. [Figure 7] This figure shows the relationship between information from a predetermined region and an image of a predetermined region T. [Figure 8] This is a diagram showing a point in three-dimensional Euclidean space using spherical coordinates. [Figure 9] This is a schematic diagram of an image communication system according to an embodiment of the present invention. [Figure 10] This is a hardware configuration diagram of the imaging device. [Figure 11] This is a hardware configuration diagram for a video conferencing terminal. [Figure 12] This is a hardware configuration diagram of the communication management system and PC. [Figure 13] This is a hardware configuration diagram of a smartphone. [Figure 14] This is a functional block diagram of an image communication system. [Figure 15] This is a conceptual diagram showing the image type management table. [Figure 16] This is a conceptual diagram showing the management table for the imaging equipment. [Figure 17] This is a conceptual diagram showing the display restriction management table. [Figure 18] This is a conceptual diagram showing the display restriction management table. [Figure 19] It is a conceptual diagram showing a session management table. [Figure 20] It is a conceptual diagram showing an image type management table. [Figure 21] It is a conceptual diagram showing a full display restriction management table. [Figure 22] It is a sequence diagram showing processing for participating in a specific communication session. [Figure 23] It is a diagram showing a selection screen for a communication session (virtual meeting room). [Figure 24] It is a sequence diagram showing management processing for image type information. [Figure 25] It is a sequence diagram showing communication processing of image data in a video call. [Figure 26] It is a flowchart showing creation processing of a predetermined area image. [Figure 27] (a) shows a case where the imaging device 1a is not used, and (b) shows a case where the imaging device 1a is used. [Figure 28] It shows an example screen of a video call at a base B; (a) shows a case where image data sent from imaging devices 1a and 1b is displayed as it is without creating an omnidirectional panoramic image or a predetermined area image, and FIG. 28(b) shows a case where an omnidirectional panoramic image and a predetermined area image are created from the image data sent from imaging devices 1a and 1b. [Figure 29] It is a sequence diagram showing processing for setting a display restriction parameter. [Figure 30] It is a sequence diagram showing processing for canceling the setting of a display restriction parameter in response to a request from the restricted side. [Figure 31] It is a sequence diagram showing processing for canceling the setting of a display restriction parameter in response to a request from the restricting side. [Figure 32] It is an example screen of a video call including a notification screen. [Figure 33] It is an example screen of a video call including an approval request screen. [Figure 34] It is an example screen of a video call including a "Cancel display restriction" button. [Figure 35] This sequence diagram shows the process of changing the display restriction parameter settings based on instructions from the restricting side. [Figure 36] This is an example of a video call screen, including a settings change screen. [Modes for carrying out the invention]

[0010] Embodiments of the present invention will be described below with reference to the drawings.

[0011] <<Outline of the Embodiment>> <Method for generating a full-spherical panoramic image> The method for generating a full-sphere panoramic image will be explained using Figures 1 to 8.

[0012] First, the external appearance of the imaging device 1 will be described using Figure 1. The imaging device 1 is a digital camera used to obtain the image that will be the basis for a 360° panoramic image. Figure 1(a) is a left side view of the imaging device, Figure 1(b) is a front view of the imaging device, and Figure 1(c) is a top view of the imaging device.

[0013] As shown in Figure 1(a), the imaging device 1 is small enough to be held in one hand. Also, as shown in Figures 1(a), 1(b), and 1(c), the imaging device 1 has an image sensor 103a on the front side and an image sensor 103b on the rear side. These image sensors 103a and 103b are used in conjunction with an optical component (for example, a fisheye lens 102a or 102b, which will be described later) that can capture hemispherical images (angle of view of 180° or more). Also, as shown in Figure 1(b), an operating section 115, such as a shutter button, is provided on the side of the imaging device 1 opposite to the front side.

[0014] Next, the usage of the imaging device 1 will be explained using Figure 2. Figure 2 is an illustrative diagram of the imaging device in use. As shown in Figure 2, the imaging device 1 is used, for example, by a user holding it in their hand to photograph subjects around the user. In this case, two hemispherical images can be obtained by capturing images of subjects around the user using the image sensors 103a and 103b shown in Figure 1.

[0015] Next, using Figures 3 and 4, we will outline the process from the image captured by the imaging device 1 to the creation of a full-sphere panoramic image. Figure 3(a) shows the hemispherical image (front) captured by the imaging device, Figure 3(b) shows the hemispherical image (back) captured by the imaging device, and Figure 3(c) shows the image represented by the Mercator projection (hereinafter referred to as the "Mercator image"). Figure 4(a) is a conceptual diagram showing the state in which the sphere is covered by the Mercator image, and Figure 4(b) shows the full-sphere panoramic image.

[0016] As shown in Figure 3(a), the image obtained by the image sensor 103a becomes a curved hemispherical image (front side) by the fisheye lens 102a described later. Similarly, as shown in Figure 3(b), the image obtained by the image sensor 103b becomes a curved hemispherical image (rear side) by the fisheye lens 102b described later. The hemispherical image (front side) and the 180-degree inverted hemispherical image (rear side) are then combined by the imaging device 1 to create a Mercator image, as shown in Figure 3(c).

[0017] Then, by using OpenGL ES (Open Graphics Library for Embedded Systems), the Mercator image is superimposed to cover the sphere, as shown in Figure 4(a), creating a full-sphere panoramic image as shown in Figure 4(b). In this way, the full-sphere panoramic image is represented as an image where the Mercator image is oriented towards the center of the sphere. OpenGL ES is a graphics library used to visualize 2D (2-Dimensional) and 3D (3-Dimensional) data. The full-sphere panoramic image can be a still image or a video.

[0018] As described above, a full-sphere panoramic image is an image pasted over a sphere, which can cause discomfort to the human eye. Therefore, by displaying a predetermined area of ​​the full-sphere panoramic image (hereinafter referred to as the "predetermined area image") as a flat image with less curvature, it is possible to display it in a way that does not cause discomfort to the human eye. This will be explained using Figures 5 and 6.

[0019] Figure 5 shows the positions of the virtual camera and the predetermined region when the 360-degree panoramic image is treated as a three-dimensional sphere. The virtual camera IC corresponds to the viewpoint of the user viewing the 360-degree panoramic image, which is displayed as a three-dimensional sphere. Figure 6(a) is a stereoscopic perspective view of Figure 5, Figure 6(b) shows the predetermined region image as it appears on a display, and Figure 6(c) shows a new predetermined region image when a portion of Figure 6(b) is enlarged.

[0020] In Figure 6(a), the 360-degree panoramic image shown in Figure 4 is represented as a three-dimensional solid sphere CS. If the generated 360-degree panoramic image is a solid sphere CS, then, as shown in Figure 5, the virtual camera IC is located outside the 360-degree panoramic image. A predetermined region T in the 360-degree panoramic image is the shooting area of ​​the virtual camera IC and is identified by predetermined region information indicating the shooting direction and field of view of the virtual camera IC in the three-dimensional virtual space including the 360-degree panoramic image.

[0021] Then, the predetermined region image, which is an image of the predetermined region T shown in Figure 6(a), is displayed on a predetermined display as an image of the shooting area of ​​the virtual camera IC, as shown in Figure 6(b). The image shown in Figure 6(b) is a predetermined region image represented by the initially set (default) predetermined region information (display parameters). The following explanation will use the shooting direction (ea, aa) and field of view (α) of the virtual camera IC.

[0022] The relationship between the predetermined region information and the image of the predetermined region T will be explained using Figure 7. Figure 7 is a diagram showing the relationship between the predetermined region information and the image of the predetermined region T. As shown in Figure 7, "ea" is the elevation angle, "aa" is the azimuth angle, and "α" is the field of view (Angle). That is, the orientation of the virtual camera IC will be changed so that the point of fixation of the virtual camera IC, indicated by the shooting direction (ea, aa), becomes the center point CP of the predetermined region T, which is the shooting area of ​​the virtual camera IC. The predetermined region image Q is the image of the predetermined region T in the 360-degree spherical image CE. f is the distance from the virtual camera IC to the center point CP. L is the distance between any vertex of the predetermined region T and the center point CP (2L is the diagonal). And, in Figure 7, the trigonometric function shown in (Equation 1) below generally holds true.

[0023] Lf = tan(α / 2) ... (Equation 1) Furthermore, when a portion of the predetermined region image shown in Figure 6(b) is enlarged, a new predetermined region image is displayed, as shown in Figure 6(c).

[0024] Figure 8 shows a point in a three-dimensional Euclidean space using spherical coordinates. Let (r, θ, φ) be the position coordinates of the center point CP when expressed in spherical polar coordinates. (r, θ, φ) are the radial, polar angle, and azimuth angle, respectively. The radial r is equal to f because it is the distance from the origin of the three-dimensional virtual space containing the panoramic image to the center point CP. Figure 8 illustrates these relationships. Hereafter, we will use the position coordinates (r, θ, φ) of the virtual camera IC for explanation.

[0025] <Overview of the image communication system> Next, the general configuration of the image communication system of this embodiment will be explained using Figure 9. Figure 9 is a schematic diagram of the configuration of the image communication system of this embodiment.

[0026] As shown in Figure 9, the image communication system of this embodiment consists of a camera 1a, 1b, a video conferencing terminal 3, a communication management system 5, a PC (Personal Computer) 7, a camera 8, and a smartphone 9, and can communicate via a communication network 100 such as the Internet. The connection method of the communication network 100 may be either wireless or wired.

[0027] Of these, the shooting devices 1a and 1b are special digital cameras used to capture subjects, landscapes, etc., and obtain two hemispherical images that form the basis of a full-sphere panoramic image, as described above. On the other hand, the shooting device 8 is a general-purpose digital camera used to capture subjects, landscapes, etc., and obtain a general planar image.

[0028] The video conferencing terminal 3 is a terminal dedicated to video conferencing and displays the video call image on the display 4 via a wired cable such as a USB (Universal Serial Bus) cable. Normally, the video conferencing terminal 3 captures images with the camera 312 described later, but when connected via a wired cable to the cradle 2a to which the camera 1a is attached, the camera 1a takes priority, and two hemispherical images that form the basis of a full-spherical panoramic image can be obtained. When using a wired cable, the cradle 2a not only handles communication between the camera 1a and the video conferencing terminal 3, but also supplies power to the camera 1a and supports the camera 1a. Here, the camera 1a, cradle 2a, video conferencing terminal 3, and display 4 are all located at the same location, location A. Also, four users A1, A2, A3, and A4 are participating in the video call at location A.

[0029] The communication management system 5 manages communication between the video conferencing terminal 3, PC 7, and smartphone 9, and manages the types of image data transmitted and received (general images and special images). In this case, special images are 360-degree panoramic images. The communication management system 5 is installed in service companies that provide video communication services. The communication management system 5 may be constructed using a single computer, or it may be constructed using multiple computers, each part (function, means, or storage unit) divided and assigned as appropriate.

[0030] PC7 becomes capable of video calls when the camera 8 is attached to it. In this case, both PC7 and camera 8 are located at the same location, location C. Additionally, one user, C, is participating in the video call at location C.

[0031] Smartphone 9 displays the video call image on a display 917 located on its own device, as described later. Normally, smartphone 9 captures images using a CMOS (Complementary Metal Oxide Semiconductor) sensor 905 located on its own device, as described later. However, it can also acquire two hemispherical image data, which form the basis of the 360-degree panoramic image obtained by the imaging device 1b, using wireless communication technologies such as Wi-Fi (Wireless Fidelity) or Bluetooth (registered trademark). When using wireless communication technology, cradle 2b only serves to supply power to imaging device 1b and support it. In this case, imaging device 1b, cradle 2b, and smartphone 9 are all located at the same location, location B. Two users, B1 and B2, are participating in the video call at location B.

[0032] Furthermore, the video conferencing terminal 3, PC 7, and smartphone 9 are examples of communication terminals. Each communication terminal has OpenGL ES installed and can create predetermined region information that indicates a portion of a 360-degree panoramic image, or create predetermined region images from 360-degree panoramic images sent from other communication terminals.

[0033] Note that the arrangement of terminals, devices, and users shown in Figure 9 is just one example, and other arrangements are possible. For example, at base C, a shooting device capable of capturing 360-degree panoramic images may be used instead of shooting device 8. Communication terminals also include digital televisions, smartwatches, car navigation devices, etc. Furthermore, from now on, any shooting device among shooting devices 1a and 1b will be referred to as "shooting device 1".

[0034] <<Hardware configuration of the embodiment>> Next, the hardware configuration of the camera 1, video conferencing terminal 3, communication management system 5, PC 7, and smartphone 9 of this embodiment will be described in detail with reference to Figures 10 to 13. Note that the camera 8 is a standard camera, so a detailed explanation will be omitted.

[0035] <Hardware configuration of imaging device 1> First, the hardware configuration of the imaging device 1 will be explained using Figure 10. Figure 10 is a hardware configuration diagram of the imaging device 1. In the following, the imaging device 1 will be assumed to be a 360-degree (omnidirectional) imaging device using two image sensors, but it may have more than two image sensors. Furthermore, it is not necessarily required to be a device dedicated to omnidirectional imaging; a regular digital camera or smartphone can be fitted with an aftermarket omnidirectional imaging unit to achieve essentially the same functionality as the imaging device 1.

[0036] As shown in Figure 10, the imaging device 1 consists of an imaging unit 101, an image processing unit 104, an imaging control unit 105, a microphone 108, a sound processing unit 109, a CPU (Central Processing Unit) 111, a ROM (Read Only Memory) 112, an SRAM (Static Random Access Memory) 113, a DRAM (Dynamic Random Access Memory) 114, an operation unit 115, a network interface 116, a communication unit 117, and an antenna 117a.

[0037] Of these, the imaging unit 101 includes wide-angle lenses (so-called fisheye lenses) 102a and 102b, each having a field of view of 180° or more for forming hemispherical images, and two image sensors 103a and 103b, each corresponding to the wide-angle lens. The image sensors 103a and 103b include image sensors such as CMOS (Complementary Metal Oxide Semiconductor) sensors and CCD (Charge Coupled Device) sensors that convert the optical image from the fisheye lenses 102a and 102b into electrical signal image data and output it, timing generation circuits that generate horizontal or vertical synchronization signals and pixel clocks for these image sensors, and a group of registers in which various commands and parameters necessary for the operation of these image sensors are set.

[0038] The image sensors 103a and 103b of the imaging unit 101 are each connected to the image processing unit 104 via a parallel I / F bus. On the other hand, the image sensors 103a and 103b of the imaging unit 101 are connected separately from the imaging control unit 105 via a serial I / F bus (such as an I2C bus). The image processing unit 104 and the imaging control unit 105 are connected to the CPU 111 via bus 110. Furthermore, ROM 112, SRAM 113, DRAM 114, operation unit 115, network I / F 116, communication unit 117, and electronic compass 118 are also connected to bus 110.

[0039] The image processing unit 104 receives image data output from image sensors 103a and 103b via a parallel I / F bus, performs predetermined processing on each image data, and then combines these image data to create Mercator image data as shown in Figure 3(c).

[0040] The imaging control unit 105 generally uses the I2C bus to set commands and other information in the registers of the image sensors 103a and 103b, with the imaging control unit 105 acting as the master device and the image sensors 103a and 103b as slave devices. It receives the necessary commands and other information from the CPU 111. The imaging control unit 105 also uses the I2C bus to acquire status data and other information from the registers of the image sensors 103a and 103b and send it to the CPU 111.

[0041] Furthermore, the imaging control unit 105 instructs the image sensors 103a and 103b to output image data when the shutter button on the operation unit 115 is pressed. Depending on the imaging device 1, there may also be functions to display a preview or video on a display (for example, the display of the video conferencing terminal 3). In this case, the image data output from the image sensors 103a and 103b is performed continuously at a predetermined frame rate (frames / minute).

[0042] Furthermore, as will be described later, the imaging control unit 105 also functions as a synchronization control means that works in cooperation with the CPU 111 to synchronize the output timing of image data from the image sensors 103a and 103b. In this embodiment, the imaging device 1 is not provided with a display unit, but a display unit may be provided.

[0043] Microphone 108 converts sound into sound (signal) data. Sound processing unit 109 receives the sound data output from microphone 108 through the I / F bus and performs predetermined processing on the sound data.

[0044] The CPU 111 controls the overall operation of the imaging device 1 and executes necessary processing. The ROM 112 stores various programs for the CPU 111. The SRAM 113 and DRAM 114 are work memories that store programs executed by the CPU 111 and data in progress. In particular, the DRAM 114 stores image data in progress and processed Mercator image data from the image processing unit 104.

[0045] The control unit 115 is a collective term for various operation buttons, a power switch, a shutter button, and a touch panel that combines display and operation functions. The user inputs various shooting modes and shooting conditions by operating the operation buttons.

[0046] The network interface 116 is a general term for interface circuits (such as USB interfaces) that connect to external media such as SD cards or personal computers. The network interface 116 can be wireless or wired. The Mercator image data stored in the DRAM 114 is recorded to external media via this network interface 116, or transmitted to external devices such as the video conferencing terminal 3 via the network interface 116 as needed.

[0047] The communication unit 117 communicates with external devices such as the video conferencing terminal 3 via an antenna 117a provided on the imaging device 1, using short-range wireless technologies such as Wi-Fi or NFC (Near Field Communication). This communication unit 117 can also transmit Mercator image data to the external device of the video conferencing terminal 3.

[0048] The electronic compass 118 calculates the orientation and tilt (roll angle) of the imaging device 1 from the Earth's magnetic field and outputs orientation and tilt information. This orientation and tilt information is an example of related information (metadata) in accordance with Exif, and is used for image processing such as image correction of captured images. The related information also includes the date and time the image was taken and the data size of the image data.

[0049] <Hardware configuration of video conferencing terminals> Next, the hardware configuration of the video conferencing terminal 3 will be explained using Figure 11. Figure 11 is a hardware configuration diagram of the video conferencing terminal. As shown in Figure 11, the video conferencing terminal 3 is equipped with a CPU 301, ROM 302, RAM 303, flash memory 304, SSD 305, media I / F 307, operation buttons 308, power switch 309, bus line 310, network I / F 311, camera 312, image sensor I / F 313, microphone 314, speaker 315, sound input / output I / F 316, display I / F 317, external device connection I / F 318, short-range communication circuit 319, and antenna 319a of the short-range communication circuit 319.

[0050] Of these components, the CPU 301 controls the overall operation of the video conferencing terminal 3. The ROM 302 stores programs used to drive the CPU 301, such as the IPL (Initial Program Loader). The RAM 303 is used as the work area for the CPU 301. The flash memory 304 stores various data, such as communication programs, image data, and sound data. The SSD (Solid State Drive) 305 controls the reading or writing of various data to the flash memory 304 according to the control of the CPU 301. Note that an HDD may be used instead of the SSD. The media I / F 307 controls the reading or writing (storage) of data to the recording media 306, such as the flash memory. The operation button 308 is used to select a destination on the video conferencing terminal 3, etc. The power switch 309 is a switch for turning the power of the video conferencing terminal 3 ON / OFF.

[0051] Furthermore, the network I / F 311 is an interface for data communication using a communication network 100 such as the Internet. The camera 312 is a type of built-in imaging means that captures an image of a subject and obtains image data according to the control of the CPU 301. The image sensor I / F 313 is a circuit that controls the driving of the camera 312. The microphone 314 is a type of built-in sound collection means that inputs sound. The sound input / output I / F 316 is a circuit that processes the input and output of sound signals between the microphone 314 and the speaker 315 according to the control of the CPU 301. The display I / F 317 is a circuit that transmits image data to an external display 320 according to the control of the CPU 301. The external device connection I / F 318 is an interface for connecting various external devices. The short-range communication circuit 319 is a communication circuit such as NFC (registered trademark) or Bluetooth (registered trademark).

[0052] Furthermore, the bus line 310 is an address bus, data bus, etc., for electrically connecting each component, such as the CPU 301 shown in Figure 11.

[0053] Display 4 is a type of display means composed of liquid crystal or organic EL that displays images of the subject, operation icons, etc. Display 4 is connected to display I / F 317 by cable 4c. This cable 4c may be a cable for analog RGB (VGA) signals, a cable for component video, or a cable for HDMI (High-Definition Multimedia Interface) (registered trademark) or DVI (Digital Video Interactive) signals.

[0054] The camera 312 includes a lens and a solid-state image sensor that converts light into electric charge to create an image (video) of the subject. CMOS sensors and CCD sensors are used as the solid-state image sensor. External devices such as external cameras, external microphones, and external speakers can be connected to the external device connection I / F 318 via USB (Universal Serial Bus) cables, etc. When an external camera is connected, it is driven in priority over the built-in camera 312, according to the control of the CPU 301. Similarly, when an external microphone or external speaker is connected, it is driven in priority over the built-in microphone 314 and built-in speaker 315, respectively, according to the control of the CPU 301.

[0055] Furthermore, the recording medium 306 is configured to be detachable from the video conferencing terminal 3. In addition, any non-volatile memory that reads or writes data according to the control of the CPU 301 may be used, not limited to flash memory 304, but also including EEPROM (Electrically Erasable and Programmable ROM), etc.

[0056] <Communication management system, PC hardware configuration> Next, we will explain the hardware configuration of the communication management system 5 and PC 7 using Figure 12. Figure 12 is a hardware configuration diagram of the communication management system and PC. Since both the communication management system 5 and PC 7 are computers with the same configuration, we will explain the configuration of the communication management system 5 below, and omit the explanation of the configuration of PC 7.

[0057] The communication management system 5 includes a CPU 501 that controls the overall operation of the communication management system 5, a ROM 502 that stores programs used to drive the CPU 501 such as an IPL, a RAM 503 used as the work area of ​​the CPU 501, an HD 504 that stores various data such as programs for the communication management system 5, an HDD (Hard Disk Drive) 505 that controls the reading or writing of various data to the HD 504 according to the control of the CPU 501, a media drive 507 that controls the reading or writing (storage) of data to a recording medium 506 such as flash memory, a display 508 that displays various information such as cursors, menus, windows, characters, or images, a network I / F 509 for data communication using the communication network 100, a keyboard 511 with multiple keys for inputting characters, numbers, and various instructions, a mouse 512 for selecting and executing various instructions, selecting processing targets, moving the cursor, etc., and a CD-RW (Compact) as an example of a removable recording medium. The system includes a CD-RW drive 514 that controls the reading or writing of various data to the Disc-ReWritable) 513, and bus lines 510 such as an address bus and a data bus for electrically connecting the above components as shown in Figure 12.

[0058] <Smartphone Hardware Configuration> Next, the hardware of the smartphone will be described using Figure 13. Figure 13 is a hardware configuration diagram of the smartphone. As shown in Figure 13, the smartphone 9 is equipped with a CPU 901, ROM 902, RAM 903, EEPROM 904, CMOS sensor 905, accelerometer / compass sensor 906, media I / F 908, and GPS receiver 909.

[0059] Of these, the CPU 901 controls the overall operation of the smartphone 9. The ROM 902 stores programs used to drive the CPU 901, such as the IPL. The RAM 903 is used as the work area for the CPU 901. The EEPROM 904 reads or writes various data, such as smartphone programs, according to the control of the CPU 901. The CMOS (Complementary Metal Oxide Semiconductor) sensor 905 captures images of a subject (mainly a self-portrait) and obtains image data according to the control of the CPU 901. The acceleration / direction sensor 906 is a variety of sensors, such as an electronic magnetic compass, gyrocompass, and acceleration sensor that detect the Earth's magnetic field. The media I / F 908 controls the reading or writing (storage) of data to or from the recording medium 907, such as flash memory. The GPS receiver 909 receives GPS signals from GPS satellites.

[0060] The smartphone 9 also includes a long-range communication circuit 911, a camera 912, an image sensor interface 913, a microphone 914, a speaker 915, an audio input / output interface 916, a display 917, an external device connection interface 918, a short-range communication circuit 919, an antenna 919a for the short-range communication circuit 919, and a touch panel 921.

[0061] Of these, the long-range communication circuit 911 is a circuit that communicates with other devices via the communication network 100. The camera 912 is a type of built-in imaging means that captures an image of a subject and obtains image data according to the control of the CPU 901. The image sensor interface 913 is a circuit that controls the driving of the camera 912. The microphone 914 is a type of built-in sound collection means that inputs sound. The sound input / output interface 916 is a circuit that processes the input and output of sound signals between the microphone 914 and the speaker 915 according to the control of the CPU 901. The display 917 is a type of display means such as a liquid crystal or organic EL that displays images of the subject and various icons. The external device connection interface 918 is an interface for connecting various external devices. The short-range communication circuit 919 is a communication circuit such as NFC or Bluetooth. The touch panel 921 is a type of input means that allows the user to operate the smartphone 9 by pressing the display 917.

[0062] Furthermore, the smartphone 9 is equipped with a bus line 910. The bus line 910 is an address bus, data bus, etc., for electrically connecting various components such as the CPU 901.

[0063] Furthermore, recording media such as CD-ROMs on which the above programs are stored, as well as hard drives on which these programs are stored, may be provided domestically or internationally as program products.

[0064] <<Embodiment Functional Configuration>> Next, the functional configuration of this embodiment will be described using Figures 14 to 20. Figure 14 is a functional block diagram of the imaging devices 1a and 1b, the video conferencing terminal 3, the communication management system 5, the PC 7, and the smartphone 9, which constitute a part of the image communication system of this embodiment.

[0065] <Functional configuration of imaging device 1a> As shown in Figure 14, the imaging device 1a includes a reception unit 12a, an imaging unit 13a, a sound collection unit 14a, a communication unit 18a, and a storage / reading unit 19a. Each of these units is a function or means realized by any of the components shown in Figure 10 operating according to instructions from the CPU 111 that follow a program for the imaging device deployed on SRAM 113 to DRAM 114.

[0066] Furthermore, the imaging device 1 has a storage unit 1000a constructed from ROM 112, SRAM 113, and DRAM 114 as shown in Figure 10. The storage unit 1000a stores the GUID (Globally Unique Identifier) ​​of the device itself.

[0067] (Functional configuration of each part of the imaging device 1a) Next, the functional configurations of the imaging device 1a will be explained in more detail using Figures 10 and 14.

[0068] The reception unit 12a of the imaging device 1a is mainly implemented by the operation unit 115 and the CPU 111 shown in Figure 10, and receives operation input from the user.

[0069] The imaging unit 13a is mainly realized by the imaging unit 101, image processing unit 104, and imaging control unit 105 shown in Figure 10, as well as the processing of the CPU 111, to capture images of landscapes, etc., and obtain captured image data.

[0070] The sound collection unit 14a is realized by the microphone 108 and sound processing unit 109 shown in Figure 10, as well as the processing of the CPU 111, and collects sounds from the surrounding area of ​​the imaging device 1.

[0071] The communication unit 18a is primarily implemented by the processing of the CPU 111 and can communicate with the communication unit 38 of the video conferencing terminal 3 using short-range wireless communication technologies such as NFC, Bluetooth, and Wi-Fi.

[0072] The storage and reading unit 19a is mainly implemented by the processing of the CPU 111 shown in Figure 10, and stores various data (or information) in the storage unit 1000a and reads various data (or information) from the storage unit 1000a.

[0073] <Functional Configuration of Imaging Device 1b> The imaging device 1b includes a reception unit 12b, an imaging unit 13b, a sound collection unit 14b, a communication unit 18b, and a storage / reading unit 19b. These have the same functions as the reception unit 12a, imaging unit 13a, sound collection unit 14a, communication unit 18a, and storage / reading unit 19a in the imaging device 1a, so their explanation is omitted. The imaging device 1 also has a storage unit 1000b constructed from ROM 112, SRAM 113, and DRAM 114 as shown in Figure 10. The GUID of the device is stored in the storage unit 1000b.

[0074] <Functional Configuration of Video Conferencing Terminal 3> As shown in Figure 14, the video conferencing terminal 3 includes a transmitting / receiving unit 31, a receiving unit 32, an image / sound processing unit 33, a display control unit 34, a determination unit 35, a creation unit 36, a communication unit 38, and a storage / reading unit 39. Each of these units is a function or means realized by any of the components shown in Figure 11 operating according to instructions from the CPU 301 that follow the program for the video conferencing terminal 3 deployed from the flash memory 304 onto the RAM 303.

[0075] Furthermore, the video conferencing terminal 3 has a storage unit 3000 constructed from ROM 302, RAM 303, and flash memory 304 as shown in Figure 11. This storage unit 3000 contains an image type management DB 3001, a shooting device management DB 3002, a display restriction management DB 3003, and a display restriction management DB 3004. Of these, the image type management DB 3001 is composed of the image type management table shown in Figure 15. The shooting device management DB 3002 is composed of the shooting device management table shown in Figure 16. The display restriction management DB 3003 is composed of the display restriction management table shown in Figure 17. The display restriction management DB 3004 is composed of the display restriction management table shown in Figure 18.

[0076] (Image type management table) Figure 15 is a conceptual diagram showing the image type management table. In this image type management table, the image data ID, the IP address (an example of the destination of the sending terminal), and the source name are stored and managed in association with each other. Of these, the image data ID is an example of image data identification information used to identify image data when performing video communication. Image data sent from the same sending terminal is assigned the same image data ID. This allows the receiving terminal (the receiving communication terminal) to identify the sending terminal of the received image data. The IP address of the sending terminal indicates the IP address of the communication terminal that sends the image data indicated by the associated image data ID. The source name is a name used to identify the imaging device that outputs the image data indicated by the associated image data ID, and is an example of image type information. This source name is a name created by each communication terminal, such as the video conferencing terminal 3, according to a predetermined naming convention.

[0077] For example, the three communication terminals with IP addresses "1.2.1.3", "1.2.2.3", and "1.3.1.3" are shown to be transmitting image data indicated by image data IDs "RS001", "RS002", and "RS003", respectively. Furthermore, the image types indicated by the source names of each communication terminal are "Video_Theta", "Video", and "Video_Theta", which indicate that the image types are "Special Image", "General Image", and "Special Image", respectively. In this case, the special image is a 360-degree panoramic image.

[0078] Furthermore, data other than image data may also be managed in association with image data IDs. Examples of data other than image data include audio data and document data used during screen sharing.

[0079] (Imaging equipment management table) Figure 16 is a conceptual diagram showing the imaging device management table. This imaging device management table stores and manages the vendor ID and product ID of the GUID of the imaging device that can obtain the two hemispherical images that form the basis of a full-sphere panoramic image. As GUIDs, for example, the vendor ID (VID) and product ID (PID) used in USB devices can be used. These vendor ID and product ID are stored from the time of factory shipment of communication terminals such as video conferencing terminals, but they may also be stored after factory shipment.

[0080] (Display restriction management table) Figure 17 is a conceptual diagram showing the display restriction management table. This display restriction management table manages display restriction parameters that indicate the restrictions on the display of a predetermined area of ​​an image for each IP address of the communication terminal (destination terminal) to which the data of the two hemispherical images that form the basis of the full-sphere panoramic image is sent. Specifically, the display restriction parameters indicate the restrictions imposed on each element of radial angle (r), polar angle (θ), azimuth angle (φ), and field of view angle (α), as shown in Figures 7 and 8. For example, for the communication terminal with the IP address "1.3.1.2", the settings are managed such that the field of view angle α must exceed 120° (it cannot be less than 120°).

[0081] (Display Restriction Management Table) Figure 18 is a conceptual diagram showing the display restriction management table. This display restriction management table manages display restriction parameters that indicate the restrictions imposed (received) on the display of a predetermined area of ​​image for each IP address of the communication terminal (source terminal) that transmits the data for the two hemispherical images that form the basis of the full-spherical panoramic image. These display restriction parameters are the same as the display restriction parameters managed in the display restriction management table.

[0082] (Functional configuration of each video conferencing terminal) Next, we will explain the functional configuration of the video conferencing terminal 3 in more detail using Figures 11 and 14.

[0083] The transmitting / receiving unit 31 of the video conferencing terminal 3 is mainly realized by the processing of the network interface 311 and CPU 301 shown in Figure 11, and transmits and receives various data (or information) with the communication management system 5 via the communication network 100.

[0084] The reception unit 32 is primarily implemented by the operation buttons 308 and the CPU 301, and accepts various selections or inputs from the user. In addition to the operation buttons 308, other input means such as a touch panel may also be used.

[0085] The image and sound processing unit 33 is implemented by instructions from the CPU 301 shown in Figure 11, and performs image processing on the image data obtained when the camera 312 captures a subject. In addition, the image and sound processing unit 33 performs audio processing on the sound data related to the audio signal after the user's voice is converted into an audio signal by the microphone 314.

[0086] Furthermore, the image / sound processing unit 33 performs image processing on image data received from other communication terminals based on image type information such as the source name, in order for the display control unit 34 to display the image on the display 4. Specifically, if the image type information indicates that it is a special image, the image / sound processing unit 33 creates a full-sphere panoramic image data by converting the image data (for example, the data of each hemispheric image as shown in Figures 3(a) and (b)) into a full-sphere panoramic image data as shown in Figure 4(b), and further creates a predetermined region image as shown in Figure 6(b). In addition, the image / sound processing unit 33 outputs an audio signal related to the audio data received from other communication terminals via the communication management system 5 to the speaker 315, and outputs audio from the speaker 315.

[0087] The display control unit 34 is mainly implemented by the processing of the display I / F 317 and the CPU 301, and controls the display of various images, characters, etc. on the display 4.

[0088] The determination unit 35 is mainly implemented by the processing of the CPU 301 and, for example, determines the type of image related to the image data received from the imaging device 1a.

[0089] The creation unit 36 ​​is mainly implemented by the CPU 301 and, based on the determination by the judgment unit 35 that it is a general image or a special image (in this case, a full-sphere panoramic image), creates a source name, which is an example of image type information, according to the naming rules described above. For example, if the judgment unit 35 determines that it is a general image, the creation unit 36 ​​creates a source name "Video" that indicates it is a general image. On the other hand, if the judgment unit 35 determines that it is a special image, the creation unit 36 ​​creates a source name "Video_Theta" that indicates it is a special image.

[0090] The communication unit 38 is mainly realized by the processing of the short-range communication circuit 319, antenna 318a, and CPU 301, and can communicate with the communication unit 18a of the imaging device 1a using short-range wireless technology such as NFC, Bluetooth, or Wi-Fi. Although the communication unit 38 and the transmitting / receiving unit 31 have been described as having separate communication units, they may also be in a shared configuration.

[0091] The storage and reading unit 39 is mainly implemented by the processing of the CPU 301 shown in Figure 11, and stores various data (or information) in the storage unit 3000 and reads various data (or information) from the storage unit 3000.

[0092] <Functional Configuration of Communication Management System> Next, the functional configurations of the communication management system 5 will be described in detail using Figures 12 and 14. The communication management system 5 includes a transmitting / receiving unit 51, a determination unit 55, a generation unit 56, and a storage / reading unit 59. Each of these units is a function or means realized by any of the components shown in Figure 12 operating according to instructions from the CPU 501 that follow the program for the communication management system 5 deployed from the HD 504 onto the RAM 503.

[0093] Furthermore, the communication management system 5 has a storage unit 5000 constructed from the RAM 503 and HD 504 shown in Figure 12. This storage unit 5000 contains a session management DB 5001, an image type management DB 5002, and a full display restriction management DB 5003. Of these, the session management DB 5001 is composed of the session management table shown in Figure 19. The image type management DB 5002 is composed of the image type management table shown in Figure 20. The full display restriction management DB 5003 is composed of the full display restriction management table shown in Figure 25.

[0094] (Session management table) Figure 19 is a conceptual diagram showing the session management table. In this session management table, the session ID and the IP address of the participating communication terminal are stored and managed in association. The session ID is an example of session identification information used to identify a communication session that enables video calls, and is generated for each virtual conference room. The session ID is also managed by each communication terminal, such as video conferencing terminal 3, and is used when selecting a communication session at each communication terminal. The IP address of the participating communication terminal indicates the IP address of the communication terminal that joined the virtual conference room indicated by the associated session ID.

[0095] (Image type management table) Figure 20 is a conceptual diagram showing the image type management table. The image type management table shown in Figure 20 manages the same information as the image type management table shown in Figure 15, as well as the same session ID as the session ID managed in the session management table. Here, it is shown that three communication terminals with IP addresses "1.2.1.3", "1.2.2.3", and "1.3.1.3" are participating in the virtual conference room indicated by the same session ID "se101". Note that the reason why the communication management system 5 manages the same image data ID, source terminal IP address, and image type information for communication terminals such as the video conferencing terminal 3 is to send image type information etc. to the communication terminal already in a video call and the newly joined communication terminal when a new communication terminal enters the virtual conference room. This eliminates the need to send and receive image type information etc. between the communication terminal already in a video call and the newly joined communication terminal.

[0096] (Full display restriction management table) Figure 21 is a conceptual diagram showing the overall display restriction management table. This overall display restriction management table is a combined table of the display restriction table (see Figure 17) and the display restriction table (see Figure 18). In other words, the overall display restriction management table manages display restriction parameters that indicate the restrictions imposed on the display of a predetermined area of ​​image for each IP address of the communication terminal that transmits the data of the two hemispherical images that form the basis of the full-spherical panoramic image (source terminal) and the communication terminal that receives the data (destination terminal). This allows the communication management system 5 to manage what restrictions the image data source is imposing on each recipient.

[0097] (Functional configuration of each function in the communication management system) Next, we will explain in detail the functional configurations of the communication management system 5 using Figures 12 and 14.

[0098] The transmission / reception unit 51 of the communication management system 5 is mainly realized by the processing of the network I / F 509 and the CPU 501 shown in FIG. 12, and transmits and receives various types of data (or information) to and from the video conference terminal 3 or the PC 7 via the communication network 100.

[0099] The determination unit 55 is mainly realized by the processing of the CPU 501, and performs various types of determination.

[0100] The generation unit 56 is mainly realized by the processing of the CPU 501, and generates image data IDs.

[0101] The storage / readout unit 59 is mainly realized by the processing of the HDD 505 and the CPU 501 shown in FIG. 12, and stores various types of data (or information) in the storage unit 5000 and reads out various types of data (or information) from the storage unit 5000.

[0102] <Functional Configuration of PC> Next, the functional configuration of the PC 7 will be described in detail with reference to FIGS. 12 and 14. The PC 7 basically has the same functions as the video conference terminal 3. That is, as shown in FIG. 14, the PC 7 includes a transmission / reception unit 71, a reception unit 72, an image and sound processing unit 73, a display control unit 74, a determination unit 75, a creation unit 76, a communication unit 78, and a storage / readout unit 79. Each of these units is a function or means realized when any of the components shown in FIG. 12 operates in accordance with an instruction from the CPU 501 according to the program for the PC 7 expanded from the HD 504 onto the RAM 503.

[0103] The PC 7 also includes a storage unit 7000 constructed by the ROM 502, the RAM 503, and the HD 504 shown in FIG. 12. An image type management DB 7001 and an imaging device management DB 7002 are constructed in this storage unit 7000. Note that the image type management DB 7001 and the imaging device management DB 7002 have the same configurations as the image type management DB 3001 and the imaging device management DB 3002, respectively, so descriptions thereof are omitted.

[0104] (The functional configuration of a PC) The transmit / receive unit 71 of PC7 is mainly implemented by the processing of the network interface 509 and CPU 501 shown in Figure 12, and performs the same functions as the transmit / receive unit 31.

[0105] The reception unit 72 is mainly implemented by the keyboard 511, mouse 512, and CPU 501, and performs the same functions as the reception unit 32. The image / sound processing unit 73 is mainly implemented by instructions from the CPU 501, and performs the same functions as the image / sound processing unit 33. The display control unit 74 is mainly implemented by the CPU 501, and performs the same functions as the display control unit 34. The judgment unit 75 is mainly implemented by the CPU 501, and performs the same functions as the judgment unit 35. The creation unit 76 is mainly implemented by the CPU 501, and performs the same functions as the creation unit 36. The communication unit 78 is mainly implemented by the CPU 501, and performs the same functions as the communication unit 38. The storage / reading unit 79 is implemented by the CPU 501, and stores various data (or information) in the storage unit 7000 and reads various data (or information) from the storage unit 7000.

[0106] <Smartphone Functionality Configuration> Next, the functional configuration of the smartphone 9 will be explained in detail using Figures 13 and 14. The smartphone 9 basically has the same functions as the video conferencing terminal 3. That is, as shown in Figure 14, the smartphone 9 has a transmitting / receiving unit 91, a receiving unit 92, an image / sound processing unit 93, a display control unit 94, a judgment unit 95, a creation unit 96, a communication unit 98, and a storage / reading unit 99. Each of these units is a function or means realized by any of the components shown in Figure 13 operating according to instructions from the CPU 901 that follow the smartphone 9 program deployed from the EEPROM 904 onto the RAM 903.

[0107] Furthermore, the smartphone 9 has a storage unit 9000 constructed from ROM 902, RAM 903, and EEPROM 904 as shown in Figure 13. This storage unit 9000 contains an image type management DB 9001 and a shooting device management DB 9002. Note that the image type management DB 9001 and the shooting device management DB 9002 have the same configuration as the image type management DB 3001 and the shooting device management DB 3002, respectively, so their explanation is omitted.

[0108] (Smartphone function configurations) The transmitting / receiving unit 91 of the smartphone 9 is mainly realized by the processing of the long-distance communication circuit 911 and CPU 901 shown in Figure 13, and performs the same functions as the transmitting / receiving unit 31.

[0109] The reception unit 92 is mainly implemented by the processing of the touch panel 921 and the CPU 901, and performs the same functions as the reception unit 32.

[0110] The image and sound processing unit 93 is mainly implemented by instructions from the CPU 901 and performs the same functions as the image and sound processing unit 33.

[0111] The display control unit 94 is mainly implemented by the processing of the CPU 901 and performs the same functions as the display control unit 34.

[0112] The decision unit 95 is mainly implemented by the processing of the CPU 901 and performs the same functions as the decision unit 35.

[0113] The creation unit 96 is mainly implemented by the processing of the CPU 901 and performs the same functions as the creation unit 36.

[0114] The communication unit 98 is primarily implemented by the processing of the CPU 901 and performs the same functions as the communication unit 38.

[0115] The memory / read unit 99 is implemented by the CPU 901 and stores various data (or information) in the memory unit 9000 and reads various data (or information) from the memory unit 9000.

[0116] <<Processing or operation of the embodiment>> <Processing of participation> Next, the processing or operation of this embodiment will be described using Figures 22 to 27. First, the process of joining a specific communication session will be described using Figures 22 and 23. Figure 22 is a sequence diagram showing the process of joining a specific communication session. Figure 23 is a diagram showing the selection screen for a communication session (virtual conference room).

[0117] First, when a user at site A (for example, user A1) performs an operation on the video conferencing terminal 3 to display the selection screen for a communication session (virtual meeting room), the reception unit 32 receives the operation to display the selection screen, and the display control unit 34 displays the selection screen shown in Figure 23 on the display 4 (step S21). This selection screen displays selection buttons b1, b2, b3, etc., which indicate the virtual meeting rooms R1, R2, R3, etc. that are the target of selection. In addition, each selection button b1, etc., is associated with each session ID.

[0118] Here, when user A1 selects the desired selection button for the virtual meeting room (in this case, selection button b1), the reception unit 32 accepts the selection of the communication session (step S22). Then, the transmitting / receiving unit 31 sends a request to join the virtual meeting room to the communication management system 5 (step S23). This request includes the session ID indicating the communication session accepted in step S22, and the IP address of the requesting terminal, the video conferencing terminal 3. As a result, the transmitting / receiving unit 51 of the communication management system 5 receives the request to join.

[0119] Next, the storage / reading unit 99 performs the process of joining the communication session by adding the IP address received in step S23 to the field of the participating terminal IP address in the record with the same session ID as the session ID received in step S23 in the session management DB 5001 (see Figure 19) (step S24). Then, the transmitting / receiving unit 51 sends a join request response to the video conferencing terminal 3 (step S25). This join request response includes the session ID received in step S23 and the result of the join process. As a result, the transmitting / receiving unit 31 of the video conferencing terminal 3 receives the join request response. The following describes the case when the join process is successful.

[0120] <Image type information management process> Next, we will explain the image type information management process using Figure 24. Figure 24 is a sequence diagram showing the image type information management process.

[0121] First, when a user at site A (for example, user A1) connects the USB cable of the cradle 2a, with the camera 1a attached, to the video conferencing terminal 3, the memory / read unit 19a of the camera 1a reads the GUID of its own device (camera 1a) stored in the memory unit 1000a, and the communication unit 18a transmits the GUID of its own device to the communication unit 38 of the video conferencing terminal 3 (step S51). As a result, the communication unit 38 of the video conferencing terminal 3 receives the GUID of the camera 1a.

[0122] Next, the determination unit 35 of the video conferencing terminal 3 determines the image type (step S52) by checking whether the same vendor ID and product ID as those in the GUID received in step S51 are managed in the shooting device management DB 3002 (see Figure 16). Specifically, if the same vendor ID and product ID are managed in the shooting device management DB 3002, the determination unit 35 determines that the shooting device 1a is a shooting device that captures special images (in this case, a 360-degree panoramic image). Conversely, if the same vendor ID and product ID are not managed in the shooting device management DB 3002, the determination unit 35 determines that the shooting device 1a is a shooting device that captures general images.

[0123] Next, the storage / reading unit 39 stores the IP address of the source terminal (video conferencing terminal 3) and the image type information, which is the result of the determination made in step S52, in association with the image type management DB 3001 (see Figure 15) (step S53). In this state, the image data ID is not associated. The image type information is, for example, the source name determined according to a predetermined naming convention, or the image type (general image, special image).

[0124] Next, the transmitting / receiving unit 31 sends a request to the communication management system 5 for additional image type information (step S54). This request for additional image type information includes the IP address of the transmitting terminal, which is the local terminal, and the image type information, which were stored in step S53. As a result, the transmitting / receiving unit 51 of the communication management system 5 receives the request for additional image type information.

[0125] Next, the storage and reading unit 59 of the communication management system 5 uses the IP address of the source terminal received in step S54 as a search key to search the session management DB 5001 (see Figure 19) and read the corresponding session ID (step S55).

[0126] Next, the generation unit 56 generates a unique image data ID (step S56). Then, the storage / reading unit 59 stores the session ID read in step S55, the image data ID generated in step S56, and the IP address and image type information of the source terminal received in step S54 as a new record in the image type management DB 5002 (see Figure 20) (step S57). Then, the transmitting / receiving unit 51 transmits the image data ID generated in step S56 to the video conferencing terminal 3. As a result, the transmitting / receiving unit 31 of the video conferencing terminal 3 receives the image data ID (step S58).

[0127] Next, the memory and retrieval unit 39 of the video conferencing terminal 3 stores the image data ID received in step S58 in the image type management DB 3001 (see Figure 15), associating it with the IP address of the source terminal, the local terminal (video conferencing terminal 3), and the image type information stored in step S53 (step S59).

[0128] Meanwhile, the transmitting / receiving unit 51 of the communication management system 5 sends an additional notification of image type information to another communication terminal, the smartphone 9 (step S60). This additional notification of image type information includes the image data ID generated in step S56, as well as the IP address of the source terminal, the local terminal (video conferencing terminal 3), and the image type information stored in step S53. As a result, the transmitting / receiving unit 91 of the smartphone 9 receives the additional notification of image type information. The destination of the transmitting / receiving unit 51 is another IP address associated with the same session ID as the IP address of the video conferencing terminal 3 in the session management DB 5001 (see Figure 19). In other words, the destination is another communication terminal that is in the same virtual conference room as the video conferencing terminal 3.

[0129] Next, the memory and retrieval unit 99 of the smartphone 9 stores the image data ID, the IP address of the sending terminal, and the image type information received in step S60 as a new record in the image type management DB 9001 (see Figure 15) (step S61). Similarly, additional communication regarding image type information is sent to another communication terminal, PC 7, and is also stored in the image type management DB 7001 (see Figure 15) on PC 7. Thus, each communication terminal can share the same information in its respective image type management DBs 3001, 7001, and 9001.

[0130] <Image data communication processing> Figures 25 to 28 will be used to explain the communication processing of image data in video calls. Figure 25 is a sequence diagram showing the communication processing of image data in video calls.

[0131] First, the communication unit 18a of the shooting device 1a transmits image data obtained by shooting a subject or landscape to the communication unit 38 of the video conferencing terminal 3 (step S71). In this case, since the shooting device 1a is a device that can obtain two hemispherical images that will become the basis of a full-sphere panoramic image, the image data consists of data from the two hemispherical images, as shown in Figures 3(a) and (b). As a result, the communication unit 38 of the video conferencing terminal 3 receives the image data.

[0132] Next, the transmitting / receiving unit 31 of the video conferencing terminal 3 transmits the image data sent from the imaging device 1a to the communication management system 5 (step S72). This transmission includes an image data ID to identify the image data to be transmitted. The IP address of the video conferencing terminal 3, which is the source of the transmission, is also transmitted. As a result, the transmitting / receiving unit 51 of the communication management system 5 receives the image data, image data ID, and IP address.

[0133] Next, the transmitting / receiving unit 51 of the communication management system 5 transmits the image data sent from the video conferencing terminal 3 to the smartphone 9 (step S73). This transmission includes an image data ID to identify the image data to be transmitted. The IP address of the source video conferencing terminal 3 is also transmitted. As a result, the transmitting / receiving unit 51 of the smartphone 9 receives the image data, image data ID, and IP address.

[0134] Next, in the smartphone 9, the storage / reading unit 99 searches the image type management DB 9001 (see Figure 15) using the image data ID received in step S73 as a search key, and reads the corresponding image type information (source name) (step S74). Then, the smartphone 9 performs the process of creating a predetermined region image (step S75). Here, the process of step S75 will be explained in detail using Figure 26. Figure 26 is a flowchart showing the process of creating a predetermined region image.

[0135] First, the determination unit 95 of the smartphone 9 determines, based on the image type information read out in step S74, whether the image type of the image data received in step S73 is a special image (in this case, a 360-degree panoramic image) (step S71-1). If it is a special image (YES), the storage / reading unit 99 searches for the corresponding display restriction parameter using the IP address of the source communication terminal (in this case, the video conferencing terminal 3) as the search key (step S71-2).

[0136] Next, the determination unit 95 determines whether there is a display restriction parameter, that is, whether the display is restricted (step S71-3). If the display is restricted (YES), the determination unit 95 further determines whether the display size is restricted by the display restriction parameter more than the default display parameter (step S71-4). If the determination unit 95 determines that it is restricted (YES), the image / sound processing unit 93 creates a full-sphere panoramic image from the image data received in step S73, and further creates a predetermined region image based on the display restriction parameter (step S71-5). In this case, the image / sound processing unit 93 synthesizes an icon 191, which indicates that it is a full-sphere panoramic image (described later), onto the predetermined region image based on the image type information indicating that it is a special image (Video_Theta).

[0137] On the other hand, if in step S71-3 the determination unit 95 determines that there are no display restrictions (NO), and in step S71-4 the determination unit 95 determines that there are no restrictions (NO), the image / sound processing unit 93 creates a full-sphere panoramic image from the image data received in step S73, and further creates a predetermined region image based on default display parameters (step S71-6). In this case, the image / sound processing unit 93 synthesizes an icon 191 indicating that it is a full-sphere panoramic image (described later) onto the predetermined region image based on image type information indicating that it is a special image (Video_Theta).

[0138] Furthermore, in step S71-1, if the determination unit 95 determines that the image type of the image data is not a special image (NO), the process in step S71 is terminated.

[0139] Next, returning to Figure 25, the display control unit 94 displays a predetermined area image including the icon 191 on the display 917 of the smartphone 9 (step S76). If the image type information indicates that it is a general image (Video), the image / sound processing unit 93 does not create a full-sphere panoramic image from the image data received in step S73, and the display control unit 94 displays a general image that does not include the icon 191.

[0140] Next, we will explain the state of the video call using Figure 27. Figure 27 is an illustrative diagram showing the state of the video call. Of these, Figure 27(a) shows the case when the camera 1a is not used, and Figure 27(b) shows the case when the camera 1a is used.

[0141] First, as shown in Figure 27(a), if the camera 312 (see Figure 11) pre-installed on the video conferencing terminal 3 is used instead of the shooting device 1a, the field of view is 125 degrees horizontally and 70 degrees vertically. Therefore, the video conferencing terminal 3 must be placed at the edge of the desk where each user A1, etc., is visible. As a result, each user A1, etc., must face the video conferencing terminal 3 when speaking. Also, because each user A1, etc., is facing the video conferencing terminal 3, the display 4 must also be placed on the side of the video conferencing terminal 3. Consequently, users A2 and A4, who are far from the video conferencing terminal 3, are far from the microphone 314 (see Figure 11), so they have to speak relatively loudly, and the content displayed on the display 4 is difficult to see.

[0142] In contrast, as shown in Figure 27(b), when using the imaging device 1a, two hemispherical images that form the basis of the full-spherical panoramic image can be obtained, allowing the video conferencing terminal 3 and display 4 to be placed relatively close to the center of the desk. As a result, each user A1, etc., is closer to the microphone 314, allowing them to speak in a relatively low voice, and the content displayed on the display 4 is also easier to see.

[0143] Next, Figure 28 will be used to explain an example of the display on the display 917 at site B. Figure 28 shows an example of a video call screen at site B. Of these, Figure 28(a) shows the case where the image data sent from video conferencing terminal 3 (shooting device 1a) and shooting device 1b is displayed as is without creating a full-spherical panoramic image and a predetermined area image, and the case where the image data sent from PC 7 (shooting device 8) is displayed as is. Figure 28(b) shows the case where a full-spherical panoramic image and a predetermined area image are created from the image data sent from video conferencing terminal 3 (shooting device 1a) and shooting device 1b, and the case where the image data sent from PC 7 (shooting device 8) is displayed as is. Note that the image of site A is displayed in the upper left display area of ​​the display 917, the image of site B (own site) is displayed in the upper right display area, and the image of site C is displayed in the lower left display area. Also, in this example, simultaneous video calls are being made between three sites, so nothing is displayed in the lower right display area.

[0144] When the image data sent from the imaging devices 1a and 1b, which are capable of capturing full-spherical panoramic images, is displayed as is, it appears as shown in Figure 28(a). In this case, the upper left point A and the upper right point B (the local point) are displayed as the front hemisphere image and the rear hemisphere image, respectively, as shown in Figures 3(a) and (b).

[0145] In response, the image and sound processing unit 93 creates a full-sphere panoramic image from the image data sent from the imaging devices 1a and 1b, which can obtain two hemispherical images that will form the basis of the full-sphere panoramic image. Furthermore, it creates a predetermined region image, and as shown in Figure 28(b), the predetermined region image, which is a planar image, is displayed. Note that since base C is equipped with an imaging device 8 that obtains a general image, a general image is displayed in both Figures 28(a) and (b).

[0146] Furthermore, an icon 191 indicating that it is a 360-degree panoramic image is displayed in the upper left corner of each image of base A and base B (the user's own base). Note that the display position of icon 191 is not limited to the upper left, but can be anywhere, such as the upper right, lower left, or lower right. Also, the type of icon 191 is not limited to that shown in Figure 28(b). In addition, instead of icon 191, it may be text such as "360-degree image," or a combination of icon and text. As shown in Figure 28(b), icon 192 is similarly displayed in the display area of ​​base B.

[0147] Furthermore, users can change a predetermined region of the image within the same 360-degree panoramic image. Specifically, users B1 and B2 can place their fingers on the touch panel 921 of the smartphone 9 and move them, allowing the reception unit 92 to receive the finger movement, and the display control unit 94 to shift, rotate, shrink, or enlarge the predetermined region image. As a result, as shown in Figure 28(b), even if only users A1 and A2 of site A are displayed in the initial settings (default), users A4 and A3 can be displayed by shifting the predetermined region image.

[0148] <Setting display restriction parameters> Next, we will explain the process of setting display restriction parameters using Figure 29. Figure 29 is a sequence diagram showing the process of setting display restriction parameters.

[0149] First, at the source communication terminal (in this case, video conferencing terminal 3), user A1 or the like uses the operation button 308 to set the initial value of the display restriction parameter, and the reception unit 32 accepts the initial value (step S101). For example, the initial value could be α (field of view) > 120°, which means that only magnified display is allowed. Next, the storage / reading unit 99 stores the initial display restriction parameter in the display restriction management DB 3003 (step S102). After that, the video conferencing terminal 3 notifies all communication terminals at all locations participating in the conference of the initial display restriction parameter, and the transmitting / receiving unit 31 sends display restriction setting information indicating the display restriction setting to the communication management system 5 (step S103). This display restriction setting information includes the IP address of the source communication terminal (in this case, video conferencing terminal 3), a list of IP addresses of the destination communication terminals, and the display restriction parameter set in step S101. The list of IP addresses includes the IP addresses of all communication devices (in this case, PC7 and smartphone9) other than the user's own device participating in the meeting (in this case, the video conferencing device). As a result, the transmitting / receiving unit 51 of the communication management system 5 receives the display restriction setting information.

[0150] Next, in the communication management system 5, the storage / reading unit 99 stores the display restriction parameters in the total display restriction management DB 5003 for each IP address of the source communication terminal and the destination communication terminal received in step S103, as shown in Figure 21 (step S105). In this case, if display restriction parameters are already stored for the same set of IP addresses of the source communication terminal and the destination communication terminal, the storage / reading unit 99 overwrites and updates them.

[0151] Next, the transmitting / receiving unit 51 of the communication management system 5 transmits display restriction setting information to the smartphone 9 (step S106). This display restriction setting information includes the IP address of the source communication terminal (in this case, the video conferencing terminal 3) and the display restriction parameters set in step S101. As a result, the transmitting / receiving unit 91 of the smartphone 9 receives the display restriction setting information. Then, the storage / reading unit 99 of the smartphone 9 associates the IP address of the source communication terminal and the display restriction parameters received in step S105 and stores them in the display restriction management DB 9004 (step S106). This completes the setting of the display restriction parameters requested by the video conferencing terminal 3 on the smartphone 9.

[0152] Similarly, the transmitting / receiving unit 51 of the communication management system 5 transmits display restriction setting information to the PC 7, indicating the display restriction settings (step S107). This display restriction setting information includes the IP address of the source communication terminal (in this case, the video conferencing terminal 3) and the display restriction parameters set in step S101. As a result, the transmitting / receiving unit 71 of the PC 7 receives the display restriction setting information. Then, the storage / reading unit 79 of the PC 7 associates the IP address of the source communication terminal and the display restriction parameters received in step S107 and stores them in the display restriction management DB 7004 (step S108). This completes the setting of the display restriction parameters requested by the video conferencing terminal 3 on the PC 7.

[0153] This completes the process of setting the display restriction parameters.

[0154] <Process to remove display restriction parameters (restricted side)> Next, using Figure 30, we will explain the process of removing the display restriction parameter settings at the request of the restricted party. Figure 30 is a sequence diagram showing the process of removing the display restriction parameter settings at the request of the restricted party.

[0155] For example, if the user of the restricted smartphone 9 wants to remove the display restriction parameters that are being restricted by the video conferencing terminal 3, the reception unit 92 of the smartphone 9 will accept the removal of the display restriction when the user of the smartphone 9 performs the operation to remove the display restriction (step 121). For example, user B1 of the smartphone 9 double-tap the display area of ​​the video for which they want the display restriction removed (in this case, the upper left display area where user A1 etc. is displayed) on the screen shown in Figure 28(b).

[0156] Next, the transmitting / receiving unit 91 transmits display restriction release request information to the communication management system 5, indicating a request to release the display restriction (step S122). This display restriction release request information includes the IP address of the requesting communication terminal (in this case, smartphone 9) and the IP address of the receiving communication terminal (in this case, video conferencing terminal 3). The requesting communication terminal is the receiving communication terminal that received the display restriction setting. The receiving communication terminal is the sending communication terminal that set the display restriction. As a result, the transmitting / receiving unit 51 of the communication management system 5 receives the display restriction release request information. The transmitting / receiving unit 51 then transmits the display restriction release request information received in step S122 to the video conferencing terminal 3 (step S123). This display restriction release request information includes the IP address of the requesting communication terminal (in this case, smartphone 9) received in step S122. As a result, the transmitting / receiving unit 31 of the video conferencing terminal 3 receives the display restriction release request information.

[0157] Next, the video conferencing terminal 3 performs a process to temporarily release the display restriction (step S124). For example, the storage / reading unit 39 temporarily deletes the corresponding display restriction parameter by searching the display restriction management DB 3003 using the IP address of the requesting (destining) communication terminal (in this case, smartphone 9) received in step S123 as the search key. In this case, the image / sound processing unit 33 may output a notification sound or display a notification screen 351 as shown in Figure 32 to notify user A1, etc., that the release of the display restriction has been requested. Figure 32 is an example of a video call screen including a notification screen. Alternatively, the video conferencing terminal 3 may output a notification sound and, after a certain period of time (e.g., 10 seconds), the storage / reading unit 39 may automatically temporarily delete the display restriction parameter. Furthermore, the temporarily deleted display restriction parameter may be restored after a predetermined time (e.g., 10 minutes) from the time of deletion, or restored by a restoration instruction from the communication terminal that instructed the display restriction.

[0158] Furthermore, as part of the processing in step S124, the display control unit 34 may display an approval request screen 352 to request approval for the removal of the display restriction from user A1 of the requesting communication terminal (in this case, video conferencing terminal 3), as shown in Figure 33. Figure 33 is an example of a video call screen including the approval request screen. This approval request screen 352 includes a "Yes" button to be pressed when approving and a "No" button to be pressed when not approving. When approval is granted, the storage / reading unit 39 searches the display restriction management DB 3003 using the IP address of the requesting (destining) communication terminal (in this case, smartphone 9) received in step S123 as the search key, and temporarily deletes the corresponding display restriction parameter. On the other hand, when approval is not granted, the transmitting / receiving unit 31 notifies the requesting communication terminal (in this case, smartphone 9) via the communication management system 5 that approval is not granted (denied).

[0159] Next, the transmitting / receiving unit 31 of the video conferencing terminal 3 sends display restriction release instruction information to the communication management system 5 indicating that the display restriction has been released (step S125). This display restriction release instruction information includes the IP address of the requesting (sender) communication terminal, which is the local terminal (in this case, the video conferencing terminal 3), and the IP address of the requesting (destination) terminal. As a result, the transmitting / receiving unit 51 of the communication management system 5 receives the display restriction release instruction information.

[0160] Next, in the communication management system 5, the storage / reading unit 59 searches the full display restriction management DB 5003 using the pair of IP addresses received in step S125 as a search key, and performs the restriction removal process by temporarily deleting the corresponding display restriction parameter (step S126). Then, the transmitting / receiving unit 51 sends display restriction removal instruction information to the smartphone 9, indicating that the removal of the display restriction is complete. This display restriction removal instruction information includes the IP address of the requesting (sender's) communication terminal received in step S125. As a result, the transmitting / receiving unit 91 of the smartphone 9 receives the display restriction removal instruction information.

[0161] Next, the smartphone 9 performs a process to remove the display restriction (step S128). For example, the storage / reading unit 99 temporarily deletes the corresponding display restriction parameter by searching the display restriction management DB 3004 using the IP address of the request destination (sender) communication terminal (in this case, the video conferencing terminal 3) received in step S127 as the search key. As a result, the smartphone 9 determines that it is not subject to a display restriction in step 71-3 shown in Figure 26, and the image / sound processing unit 93 can display a predetermined area image on the display 917 according to the operations of users B1 and B2.

[0162] With the above steps completed, the process of removing the display restriction parameter (on the restricted side) is finished.

[0163] <Process to remove display restriction parameters (restriction side)> Next, using Figure 35, we will explain the process of individually removing the display restriction parameter settings at the instruction of the restricting party. Figure 31 is a sequence diagram showing the process of removing the display restriction parameter settings at the request of the restricting party.

[0164] For example, during a video call, the display control unit 34 displays the "Remove Display Restriction" buttons 361 and 362 as shown in Figure 34. If user A1 of the restricting video conferencing terminal 3 wants to remove the display restriction only for site B of site B and C, user A1 presses the "Remove Display Restriction" button 361, and the reception unit 32 of the video conferencing terminal 3 accepts the removal of the display restriction for site B (step 201). As a result, the storage / reading unit 39 searches the display restriction management DB 3003 using the IP address of the selected site B communication terminal as the search key, and removes the display restriction by temporarily deleting the corresponding display restriction parameter (step S202). Then, the video conferencing terminal 3 transmits display restriction removal information to the communication management system 5 via the transmitting / receiving unit 31 to notify the communication terminal of site B participating in the conference (in this case, a smartphone 9) that the display restriction has been removed (step S203). This display restriction removal information includes the IP address of the source communication terminal (in this case, video conferencing terminal 3) and the IP address of the destination communication terminal to which the display restriction is to be removed. As a result, the transmitting and receiving unit 51 of the communication management system 5 receives the display restriction removal information.

[0165] Next, in the communication management system 5, the storage / reading unit 99 performs a restriction removal process by temporarily deleting the display restriction parameters corresponding to the pair of IP addresses of the source communication terminal and the destination communication terminal received in step S203 from the total display restriction management DB 5003 (step S204).

[0166] Next, the transmitting / receiving unit 51 of the communication management system 5 sends display restriction release information to the smartphone 9, indicating that the display restriction has been released (step S205). This display restriction release information includes the IP address of the source communication terminal (in this case, the video conferencing terminal 3) that released the display restriction. As a result, the transmitting / receiving unit 91 of the smartphone 9 receives the display restriction release information. Then, the storage / reading unit 99 of the smartphone 9 searches the display restriction management DB 9004 using the IP address of the source communication terminal received in step S205 as a search key, and performs a restriction release process that temporarily deletes the corresponding display restriction parameter (step S206). As a result, the display restriction requested by the video conferencing terminal 3 is released on the smartphone 9, and the restriction remains in place on the PC 7, and the display restriction parameter setting release process (restriction side) ends.

[0167] <Processing to change display restriction parameters (restriction side)> Next, using Figure 35, we will explain the process of releasing the display restriction parameter settings at the instruction of the restricting side. Figure 35 is a sequence diagram showing the process of changing the display restriction parameter settings at the instruction of the restricting side.

[0168] Figure 36 shows an example of a video call screen including a change screen. For example, during a video call, when user A1 operates the operation button 308, the display control unit 34 displays the display restriction change screen 371 as shown in Figure 36. Then, when user A1 selects whether to restrict the display for locations B and C using the operation button 308 and presses the "OK" button, the reception unit 32 of the video conferencing terminal 3 accepts whether or not to restrict the display for each location (step 221). Here, we show the case where the display is restricted for location B and not for location C. As a result, the storage / reading unit 39 searches the display restriction management DB 3003 using the IP address of the selected communication terminal at location C as the search key, and temporarily deletes the corresponding display restriction parameter to perform a batch change of whether or not to restrict the display (step S222). Note that since the display restriction will continue for location B, the storage / reading unit 39 does not temporarily delete the display restriction parameter for location B and keeps it.

[0169] Next, the transmitting / receiving unit 31 transmits display restriction change information to the communication management system 5 (step S223). This display restriction change information includes the IP address of the source communication terminal (in this case, the video conferencing terminal 3), a list of IP addresses including the IP address of the destination communication terminal to which the display restriction change is to be sent, display restriction release instruction information indicating an instruction to release the display restriction, and display restriction reset instruction information indicating an instruction to reset the display restriction. In this case, either the display restriction release instruction information or the display restriction reset instruction information is associated with each IP address of the destination communication terminal. As a result, the transmitting / receiving unit 51 of the communication management system 5 receives the display restriction change information.

[0170] Next, in the communication management system 5, for the destination communication terminal associated with the display restriction release instruction information, the storage / reading unit 99 performs a restriction change process by temporarily deleting the display restriction parameter corresponding to the pair of the IP address of the source communication terminal and the IP address of the destination communication terminal received in step S223 from the total display restriction management DB 5003 (step S224). However, for the destination communication terminal associated with the display restriction release instruction information, the storage / reading unit 99 does not temporarily delete the display restriction parameter from the total display restriction management DB 5003 and keeps it.

[0171] Next, the transmitting / receiving unit 51 of the communication management system 5 sends display restriction change information to the smartphone 9, including an instruction to reset the display restrictions (step S225). This display restriction setting information includes the IP address of the source communication terminal (in this case, the video conferencing terminal 3). As a result, the transmitting / receiving unit 91 of the smartphone 9 receives the display restriction change information. Then, the storage / reading unit 99 of the smartphone 9 performs a process to reset the display restriction parameters in the display restriction management DB 9004 for the IP address of the source communication terminal received in step S205 (step S226). Note that in the case of resetting, the communication management system 5 does not need to send the display restriction change information to the communication terminal.

[0172] Furthermore, the transmitting / receiving unit 51 of the communication management system 5 sends display restriction change information to the PC 7, including an instruction to lift the display restriction (step S227). This display restriction change information includes the IP address of the source communication terminal (in this case, the video conferencing terminal 3) that is the source of the lifting of the display restriction. As a result, the transmitting / receiving unit 71 of the PC 7 receives the display restriction change information. Then, the storage / reading unit 79 of the PC 7 searches the display restriction management DB 7004 using the IP address of the source communication terminal received in step S227 as a search key, and performs a restriction lifting process that temporarily deletes the corresponding display restriction parameter (step S228). As a result, the display restriction instructed by the video conferencing terminal 3 is lifted on the PC 7, and the display restriction parameter setting change process (restriction side) is completed.

[0173] <<Main Effects of This Embodiment>> As described above, according to this embodiment, a communication terminal such as the video conferencing terminal 3 can create a full-spherical panoramic image based on the image data ID sent along with the image data, and further create a predetermined region image based on the corresponding image type information. This has the effect of preventing the display of the front hemisphere image and the rear hemisphere image, as shown in Figure 28(a).

[0174] Furthermore, a communication terminal can lift display restrictions upon request from other communication terminals (see steps S121-S128), proactively lift display restrictions for specific other communication terminals from its own terminal (steps S201-S206), or proactively reset (or maintain) or lift display restrictions for each other communication terminal from its own terminal (steps S221-S228). This allows for flexible control of display restrictions and lifting, so that a user on another communication terminal that has been subjected to a display restriction can lift the restriction if they want to get a general idea of ​​the location of the communication terminal. [Explanation of Symbols]

[0175] 1a Imaging device 1b Imaging device 3. Video conferencing terminal (an example of a communication terminal) 5. Communication Management System 7. PC (an example of a communication terminal) 8. Imaging device 9. Smartphone (an example of a communication device) 31 Transmitting and Receiving Unit (Example of transmitting means, example of receiving means) 32 Reception area (an example of reception method) 34 Display Control Unit (An example of display control means) 71 Transmitting and Receiving Unit (Example of transmitting means, example of receiving means) 72 Reception area (an example of reception procedures) 74 Display Control Unit (An example of display control means) 91 Transmitting and Receiving Unit (Example of transmitting means, example of receiving means) 92 Reception area (an example of reception procedures) 94 Display Control Unit (An example of display control means) [Prior art documents] [Patent Documents]

[0176] [Patent Document 1] Japanese Patent Publication No. 2012-178135 [Patent Document 2] Japanese Patent Publication No. 2011-223076 [Patent Document 3] Japanese Patent Publication No. 2002-034009 [Patent Document 4] Japanese Patent Publication No. 2001-148850

Claims

1. A determination means for determining whether the received image data is a special image, and if the image data is determined to be a special image, for determining whether it is subject to display restrictions, When it is determined that the aforementioned display restriction is in effect, a display control means for displaying a predetermined region image from the image data, which is an image corresponding to a part of the special image and is based on the display restriction parameter that imposes the aforementioned display restriction, A communication terminal.

2. The communication terminal according to claim 1, wherein the determination means does not determine whether or not the image data is subject to display restrictions if the image data is not a special image.

3. The communication terminal according to claim 1 or 2, wherein the display control means displays a general image if the image data is not a special image.

4. The determination means, if display restrictions are in place, determines whether the display restriction parameter restricts the display size more than the default display parameter. The communication terminal according to any one of claims 1 to 3, wherein the display control means determines that the display size is more restricted than the default display parameter, and based on the display restriction parameter, displays the predetermined area image of the special image from the image data.

5. The communication terminal according to claim 4, wherein the display control means determines that the display limit parameter does not restrict the display size more than the default display parameter, and then displays the predetermined area image of the special image from the image data based on the default display parameter.

6. The communication terminal according to any one of claims 1 to 5, wherein the display control means changes a portion of the area in response to user operation to display the predetermined area image.

7. The communication terminal according to any one of claims 1 to 6, wherein the special image is a full-sphere panoramic image.

8. A display method executed on a communication terminal, A determination step to determine whether the received image data is a special image, and if the image data is determined to be a special image, to determine whether it is subject to display restrictions, If it is determined that the aforementioned display restriction is applied, a display control step is performed to display a predetermined region image from the image data, which is an image corresponding to a part of the special image and is an image based on the display restriction parameter that applies the aforementioned display restriction. Display methods that include this.

9. Computers, A determination means for determining whether the received image data is a special image, and if the image data is determined to be a special image, for determining whether it is subject to display restrictions, When it is determined that the aforementioned display restriction is in effect, a display control means for displaying a predetermined region image from the image data, which is an image corresponding to a part of the special image and is based on the display restriction parameter that imposes the aforementioned display restriction, A program that makes it function as such.

10. An image communication system having a communication terminal and a communication management system capable of communicating with the communication terminal, A determination means for determining whether the received image data is a special image, and if the image data is determined to be a special image, for determining whether it is subject to display restrictions, When it is determined that the aforementioned display restriction is in effect, a display control means for displaying a predetermined region image from the image data, which is an image corresponding to a part of the special image and is based on the display restriction parameter that imposes the aforementioned display restriction, An image communication system having [a specific feature / feature].

Images