COMMUNICATION TERMINAL, DISPLAY METHOD, PROGRAM, AND IMAGE COMMUNICATION SYSTEM

Abstract

To solve the problem of a conventional method in which a prohibited region is designated in a wide-range image, such as a panorama image, so as not to be transmitted or displayed from the viewpoint of security to display an image in as wide range as possible in a permitted region, while a receiver of image data has a need to, however roughly, grasp the overall status of a transmitter.SOLUTION: A communication terminal (3) transmits data of an omnidirectional image so as to be received by other communication terminals, the communication terminal being configured to: (S103) transmit a display restriction parameter so as to be received by the other communication terminals, the display restriction parameter restricting a display region within a predetermined region which is a part of the omnidirectional image when the other communication terminals (7, 9) display the omnidirectional image; and (S125, S203, S223) transmit display restriction lifting instruction information indicating an instruction to lift the restriction on the display region so as to be received by the other communication terminals.SELECTED DRAWING: Figure 30

Description

[Technical field]

[0001] The present disclosure relates to a communication terminal, a communication system, a communication method, and a program. [Background technology]

[0002] Conference systems that hold remote conferences between remote locations via a communication network such as the Internet are becoming widespread. In this conference system, in a conference room where one of the parties, such as attendees, is present, images of the conference room of the parties and audio such as remarks are captured and collected using a communication terminal of the remote conference system, and these are converted into digital data and transmitted to the communication terminal of the other party. As a result, images can be displayed on a display and audio can be output from a speaker in the other party's conference room to conduct a video call, and a technology is known for holding a conference between remote locations in a state close to an actual conference (see Patent Document 1).

[0003] Also, a technology is known in which a communication terminal is connected to an image capturing device capable of acquiring a celestial sphere panoramic image in real time, the celestial sphere (panoramic) image acquired from the image capturing device is distributed to each communication terminal of the other party, and each communication terminal of the other party sequentially converts the celestial sphere image into a rectangular planar image and displays it on a display or the like (see Patent Document 2).

[0004] Furthermore, a system is known in which a communications terminal and a device capable of acquiring image data in real time are placed in the office (room) where employees normally work, and a connection is established between the communications terminal placed in the employee's home or other location, allowing the situation in the office to be constantly monitored, and at the same time the status of the remote working employee to be constantly monitored from the office (see Patent Document 3).

[0005] However, when the image data transmitted and received is spherical image data, the receiving site of the spherical image data can display the transmitting site over a wide area, and therefore, even highly confidential information at the transmitting site is displayed. Therefore, a method has been disclosed in which the transmitting site specifies a prohibited area that is not to be transmitted or displayed from the viewpoint of security, etc., from an image captured over a wide area, such as a panoramic image, and displays as wide an area of ​​the image as possible within the displayable area (see Patent Document 4). Summary of the Invention [Problem to be solved by the invention]

[0006] However, the conventional method has a problem in that it is not possible to meet the need for the receiving site of the image data to grasp the overall situation of the sending site, even if only roughly. [Means for solving the problem]

[0007] An invention according to claim 1 is a communication terminal that transmits data of a celestial sphere image to be received by another communication terminal, the communication terminal including a transmitting means that transmits a display restriction parameter for restricting a display area to a predetermined area that is a part of the celestial sphere image when the other communication terminal displays the celestial sphere image to be received by the other communication terminal, the transmitting means transmitting display restriction release instruction information indicating an instruction to release the restriction on the display area to be received by the other communication terminal. Effect of the Invention

[0008] As described above, according to the present invention, it is possible to meet the need for a receiving site of image data to grasp the overall situation of the sending site, even if only roughly. [Brief description of the drawings]

[0009] [Figure 1] 1A is a left side view of the imaging device, FIG. 1B is a front view of the imaging device, and FIG. [Diagram 2] FIG. 13 is an image of the imaging device in use. [Diagram 3] (a) is a hemispherical image (front) taken by the photographing device, (b) is a hemispherical image (back) taken by the photographing device, and (c) is an image represented by the Mercator projection. [Figure 4] (a) A conceptual diagram showing how a sphere is covered with a Mercator image, and (b) a diagram showing a spherical panoramic image. [Diagram 5] FIG. 13 is a diagram showing the positions of a virtual camera and a predetermined area when a celestial sphere panoramic image is a three-dimensional solid sphere. [Figure 6] 6(a) is a three-dimensional oblique view of FIG. 5, (b) is a diagram showing the state in which an image of a specified area is displayed on the display of a communication terminal, and (c) is a diagram showing a new image of the specified area when a portion of FIG. 6(b) is enlarged. [Figure 7] 13 is a diagram showing the relationship between predetermined region information and an image of a predetermined region T. FIG. [Figure 8] This is a diagram showing points in three-dimensional Euclidean space in spherical coordinates. [Figure 9] 1 is a schematic diagram of an image communication system according to an embodiment of the present invention. [Figure 10] FIG. 2 is a diagram illustrating a hardware configuration of the imaging device. [Figure 11] FIG. 2 is a hardware configuration diagram of a video conference terminal. [Figure 12] FIG. 2 is a hardware configuration diagram of a communication management system and a PC. [Figure 13] FIG. 1 is a hardware configuration diagram of a smartphone. [Figure 14] FIG. 1 is a functional block diagram of an image communication system. [Figure 15] FIG. 13 is a conceptual diagram illustrating an image type management table. [Figure 16] FIG. 13 is a conceptual diagram illustrating a photographing device management table. [Figure 17] FIG. 13 is a conceptual diagram illustrating a display restriction management table. [Figure 18] FIG. 13 is a conceptual diagram illustrating a display restriction management table. [Figure 19] FIG. 13 is a conceptual diagram illustrating a session management table. [Figure 20] FIG. 13 is a conceptual diagram illustrating an image type management table. [Figure 21] FIG. 13 is a conceptual diagram illustrating a full display restriction management table. [Figure 22] FIG. 11 is a sequence diagram showing a process of joining a specific communication session. [Figure 23] FIG. 13 is a diagram showing a selection screen for a communication session (virtual conference room). [Figure 24] FIG. 11 is a sequence diagram showing a management process of image type information. [Diagram 25] FIG. 11 is a sequence diagram showing a communication process of image data in a video call. [Figure 26] 13 is a flowchart showing a process of creating a predetermined area image. [Figure 27] 1A shows the case where the photographing device 1a is not used, and FIG. 1B shows the case where the photographing device 1a is used. [Figure 28] 28A shows an example of a screen of a video call at site B, where FIG. 28A shows a case where the image data transmitted from the photographing devices 1a and 1b is displayed as is without creating a celestial sphere panoramic image and a predetermined area image, and FIG. 28B shows a case where the celestial sphere panoramic image and a predetermined area image are created from the image data transmitted from the photographing devices 1a and 1b. [Figure 29] FIG. 11 is a sequence diagram showing a process for setting a display restriction parameter. [Diagram 30] 13 is a sequence diagram showing a process of canceling the setting of a display restriction parameter in response to a request from a restricted side. FIG. [Diagram 31] 13 is a sequence diagram showing a process of canceling the setting of a display restriction parameter in response to a request from the restricting side. FIG. [Diagram 32] 13 is an example of a video call screen including a notification screen. [Diagram 33] 13 is a diagram showing an example of a screen during a video call including an approval request screen. [Diagram 34] 13 is an example of a video call screen including a "Remove Display Restrictions" button. [Diagram 35] 11 is a sequence diagram showing a process of changing the setting of a display restriction parameter in response to an instruction from the restricting side. FIG. [Diagram 36] 13 is a screen example of a video call including a change screen. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0011] <<Outline of the embodiment>> <How to generate spherical panoramic images> A method for generating a spherical panoramic image will be described with reference to FIGS.

[0012] First, the appearance of the imaging device 1 will be described with reference to Fig. 1. The imaging device 1 is a digital camera for obtaining captured images that are the basis of a full celestial sphere (360°) panoramic image. Fig. 1(a) is a left side view of the imaging device, Fig. 1(b) is a front view of the imaging device, and Fig. 1(c) is a plan view of the imaging device.

[0013] As shown in FIG. 1(a), the photographing device 1 is sized so that a person can hold it in one hand. Also, as shown in FIG. 1(a), FIG. 1(b), and FIG. 1(c), an image sensor 103a is provided on the front side (front side) and an image sensor 103b is provided on the back side (rear side) at the top of the photographing device 1. These image sensors (image sensors) 103a and 103b are used in conjunction with optical members (for example, fisheye lenses 102a and 102b described later) capable of photographing hemispherical images (angle of view of 180° or more). Also, as shown in FIG. 1(b), an operation unit 115 such as a shutter button is provided on the surface opposite to the front side of the photographing device 1.

[0014] Next, the usage of the photographing device 1 will be described with reference to Fig. 2. Fig. 2 is an image diagram of the photographing device in use. As shown in Fig. 2, the photographing device 1 is used, for example, by a user holding it in his / her hand to photograph subjects around the user. In this case, two hemispherical images can be obtained by capturing images of subjects around the user using the image sensor 103a and the image sensor 103b shown in Fig. 1, respectively.

[0015] Next, an outline of the process of creating a celestial sphere panoramic image from an image captured by the image capture device 1 will be described with reference to Fig. 3 and Fig. 4. Fig. 3(a) is a diagram showing a hemispherical image (front side) captured by the image capture device, Fig. 3(b) is a diagram showing a hemispherical image (rear side) captured by the image capture device, and Fig. 3(c) is a diagram showing an image expressed by the Mercator projection (hereinafter referred to as a "Mercator image"). Fig. 4(a) is a conceptual diagram showing a state in which a sphere is covered with a Mercator image, and Fig. 4(b) is a diagram showing a celestial sphere panoramic image.

[0016] As shown in Fig. 3(a), the image obtained by the image sensor 103a becomes a hemispherical image (front side) curved by a fisheye lens 102a described below. Also, as shown in Fig. 3(b), the image obtained by the image sensor 103b becomes a hemispherical image (rear side) curved by a fisheye lens 102b described below. Then, the hemispherical image (front side) and the hemispherical image (rear side) flipped 180 degrees are combined by the photographing device 1 to create a Mercator image as shown in Fig. 3(c).

[0017] Then, by using OpenGL ES (Open Graphics Library for Embedded Systems), the Mercator image is pasted to cover the sphere as shown in FIG. 4(a), and a celestial sphere panoramic image as shown in FIG. 4(b) is created. In this way, the celestial sphere panoramic image is represented as an image in which the Mercator image faces the center of the sphere. Note that OpenGL ES is a graphics library used to visualize 2D (2-Dimensions) and 3D (3-Dimensions) data. Note that the celestial sphere panoramic image may be a still image or a video.

[0018] As described above, since a celestial sphere panoramic image is an image pasted so as to cover a spherical surface, it gives a sense of incongruity to humans when viewed. Therefore, by displaying a predetermined region (hereinafter referred to as a "predetermined region image") of a celestial sphere panoramic image as a planar image with little curvature, it is possible to display the image in a way that does not give a sense of incongruity to humans. This will be described with reference to Figs. 5 and 6.

[0019] Fig. 5 is a diagram showing the positions of the virtual camera and the predetermined area when the omnidirectional panoramic image is a three-dimensional sphere. The virtual camera IC corresponds to the position of the viewpoint of a user who views the omnidirectional panoramic image displayed as a three-dimensional sphere. Fig. 6(a) is a three-dimensional perspective view of Fig. 5, Fig. 6(b) is a diagram showing the predetermined area image when displayed on a display, and Fig. 6(c) is a diagram showing a new predetermined area image when a part of Fig. 6(b) is enlarged.

[0020] In Fig. 6(a), the omnidirectional panoramic image shown in Fig. 4 is represented by a three-dimensional stereoscopic sphere CS. If the omnidirectional panoramic image generated in this manner is a stereoscopic sphere CS, then the virtual camera IC is located outside the omnidirectional panoramic image as shown in Fig. 5. A predetermined area T in the omnidirectional panoramic image is a shooting area of ​​the virtual camera IC, and is specified by predetermined area information indicating the shooting direction and angle of view of the virtual camera IC in a three-dimensional virtual space including the omnidirectional panoramic image.

[0021] Then, the predetermined area image, which is an image of the predetermined area T shown in Fig. 6(a), is displayed on a predetermined display as an image of the shooting area of ​​the virtual camera IC, as shown in Fig. 6(b). The image shown in Fig. 6(b) is a predetermined area image represented by the initial setting (default) predetermined area information (display parameters). The following description will be given using the shooting direction (ea, aa) and the angle of view (α) of the virtual camera IC.

[0022] The relationship between the predetermined area information and the image of the predetermined area T will be described with reference to FIG. 7. FIG. 7 is a diagram showing the relationship between the predetermined area information and the image of the predetermined area T. As shown in FIG. 7, "ea" indicates the elevation angle, "aa" indicates the azimuth angle, and "α" indicates the angle of view (Angle). That is, the attitude of the virtual camera IC is changed so that the gaze point of the virtual camera IC indicated by the shooting direction (ea, aa) becomes the center point CP of the predetermined area T, which is the shooting area of ​​the virtual camera IC. The predetermined area image Q is an image of the predetermined area T in the omnidirectional image CE. f is the distance from the virtual camera IC to the center point CP. L is the distance between any vertex of the predetermined area T and the center point CP (2L is the diagonal). In FIG. 7, the trigonometric function generally shown in the following (Equation 1) holds.

[0023] Lf = tan(α / 2) (Eq. 1) Moreover, when a part of the predetermined area image shown in FIG. 6(b) is enlarged, a new predetermined area image is displayed as shown in FIG. 6(c).

[0024] FIG. 8 is a diagram showing points in a three-dimensional Euclidean space in spherical coordinates. The position coordinates of the center point CP when expressed in a spherical polar coordinate system are (r, θ, φ). (r, θ, φ) are the radius, polar angle, and azimuth angle, respectively. The radius r is the distance from the origin of the three-dimensional virtual space including the spherical panoramic image to the center point CP, and is therefore equal to f. FIG. 8 is a diagram showing these relationships. Hereinafter, the explanation will be given using the position coordinates (r, θ, φ) of the virtual camera IC.

[0025] <Outline of the image communication system> Next, an outline of the configuration of the image communication system of this embodiment will be described with reference to Fig. 9. Fig. 9 is a schematic diagram of the configuration of the image communication system of this embodiment.

[0026] 9, the image communication system of this embodiment is composed of photographing devices 1a and 1b, a video conference terminal 3, a communication management system 5, a PC (Personal Computer) 7, a photographing device 8, and a smartphone 9, and can communicate via a communication network 100 such as the Internet. The connection form of the communication network 100 may be either wireless or wired.

[0027] Of these, the photographing devices 1a and 1b are special digital cameras for photographing subjects, scenery, etc. to obtain two hemispherical images that are the basis of a celestial sphere panoramic image, as described above. On the other hand, the photographing device 8 is a general digital camera for photographing subjects, scenery, etc. to obtain a general planar image.

[0028] The video conference terminal 3 is a terminal dedicated to video conferences, and displays images of a video call (video call) on the display 4 via a wired cable such as a USB (Universal Serial Bus) cable. The video conference terminal 3 usually captures images with a camera 312 described later, but when the video conference terminal 3 is connected to a cradle 2a to which the photographing device 1a is attached via a wired cable, the photographing device 1a is given priority, and two hemispherical images that are the basis of a celestial sphere panoramic image can be obtained. When a wired cable is used, the cradle 2a not only performs communication between the photographing device 1a and the video conference terminal 3, but also plays a role of supplying power to the photographing device 1a and supporting the photographing device 1a. Here, the photographing device 1a, the cradle 2a, the video conference terminal 3, and the display 4 are located at the same base, base A. Also, at base A, four users A1, A2, A3, and A4 are participating in the video call.

[0029] The communication management system 5 manages the communications between the video conference terminals 3, the PCs 7, and the smartphones 9, and manages the types of image data (classification of general images and special images) transmitted and received. Here, the special images are spherical panoramic images. The communication management system 5 is installed in a service company that provides video communication services. The communication management system 5 may be constructed by a single computer, or may be constructed by multiple computers in which each section (function, means, or storage section) is divided and arbitrarily assigned.

[0030] The PC 7 is capable of video calling by having the photographing device 8 attached thereto. In this example, the PC 7 and the photographing device 8 are located at the same location, site C. Also, at site C, one user C is participating in the video call.

[0031] The smartphone 9 displays an image of the video call on a display 917 provided in the smartphone 9, which will be described later. The smartphone 9 usually captures images using a CMOS (Complementary Metal Oxide Semiconductor) sensor 905 (described later) provided in the smartphone 9, but can obtain two hemispherical image data that are the basis of the omnidirectional panoramic image obtained by the image capture device 1b by using wireless communication technology such as Wi-Fi (Wireless Fidelity) or Bluetooth (registered trademark). When using wireless communication technology, the cradle 2b only serves to supply power to the image capture device 1b and support the image capture device 1b. Here, the image capture device 1b, the cradle 2b, and the smartphone 9 are located at the same base, base B. At base B, two users B1 and B2 are participating in the video call.

[0032] The video conference terminal 3, the PC 7, and the smartphone 9 are examples of communication terminals. OpenGL ES is installed in each communication terminal, and the communication terminals can create predetermined area information indicating a partial area of ​​a celestial sphere panoramic image and create a predetermined area image from a celestial sphere panoramic image transmitted from another communication terminal.

[0033] The arrangement of the terminals, devices, and users shown in FIG. 9 is one example, and other examples may be used. For example, at site C, instead of the camera device 8, a camera device capable of capturing images related to a celestial sphere panoramic image may be used. The communication terminal also includes a digital television, a smart watch, a car navigation device, and the like. In the following description, any one of the camera devices 1a and 1b will be referred to as "camera device 1."

[0034] <<Hardware configuration of the embodiment>> 10 to 13, the hardware configurations of the image capturing device 1, the video conference terminal 3, the communication management system 5, the PC 7, and the smartphone 9 of this embodiment will be described in detail. Note that the image capturing device 8 is a general camera, and therefore a detailed description thereof will be omitted.

[0035] <Hardware configuration of the imaging device 1> First, the hardware configuration of the photographing device 1 will be described with reference to Fig. 10. Fig. 10 is a hardware configuration diagram of the photographing device 1. In the following, the photographing device 1 is assumed to be an omnidirectional (all-directional) photographing device using two imaging elements, but the number of imaging elements may be two or more. In addition, the photographing device 1 does not necessarily have to be a device dedicated to omnidirectional photographing, and an omnidirectional photographing unit may be attached to a normal digital camera, smartphone, or the like to have substantially the same functions as the photographing device 1.

[0036] As shown in FIG. 10, the photographing device 1 is composed of an imaging unit 101, an image processing unit 104, an imaging control unit 105, a microphone 108, a sound processing unit 109, a CPU (Central Processing Unit) 111, a ROM (Read Only Memory) 112, an SRAM (Static Random Access Memory) 113, a DRAM (Dynamic Random Access Memory) 114, an operation unit 115, a network I / F 116, a communication unit 117, and an antenna 117a.

[0037] Of these, the imaging unit 101 includes wide-angle lenses (so-called fisheye lenses) 102a and 102b each having an angle of view of 180° or more for forming a hemispherical image, and two imaging elements 103a and 103b provided corresponding to each wide-angle lens. The imaging elements 103a and 103b include an image sensor such as a CMOS (Complementary Metal Oxide Semiconductor) sensor or a CCD (Charge Coupled Device) sensor that converts an optical image by the fisheye lenses 102a and 102b into image data of an electrical signal and outputs it, a timing generation circuit that generates horizontal or vertical synchronization signals and pixel clocks for the image sensors, and a group of registers in which various commands and parameters necessary for the operation of the imaging elements are set.

[0038] The imaging elements 103a and 103b of the imaging unit 101 are each connected to the image processing unit 104 via a parallel I / F bus. Meanwhile, the imaging elements 103a and 103b of the imaging unit 101 are connected to a serial I / F bus (such as an I2C bus) separately from the imaging control unit 105. The image processing unit 104 and the imaging control unit 105 are connected to a CPU 111 via a bus 110. Furthermore, ROM 112, SRAM 113, DRAM 114, an operation unit 115, a network I / F 116, a communication unit 117, an electronic compass 118, and the like are also connected to the bus 110.

[0039] The image processing unit 104 takes in the image data output from the image sensors 103a and 103b via a parallel I / F bus, performs a predetermined processing on each image data, and then synthesizes the image data to create Mercator image data such as that shown in Figure 3(c).

[0040] The imaging control unit 105 generally sets commands and the like in the register groups of the imaging elements 103a and 103b using an I2C bus, with the imaging control unit 105 acting as a master device and the imaging elements 103a and 103b acting as slave devices. Necessary commands and the like are received from the CPU 111. The imaging control unit 105 also uses the I2C bus to retrieve status data and the like from the register groups of the imaging elements 103a and 103b and send it to the CPU 111.

[0041] Furthermore, the imaging control unit 105 instructs the imaging elements 103a and 103b to output image data when the shutter button of the operation unit 115 is pressed. Some imaging devices 1 have a preview display function or a function corresponding to video display on a display (for example, a display of the video conference terminal 3). In this case, the image data is output from the imaging elements 103a and 103b continuously at a predetermined frame rate (frames / minute).

[0042] As described later, the imaging control unit 105 also functions as a synchronization control means for synchronizing the output timing of image data from the imaging elements 103a and 103b in cooperation with the CPU 111. Note that, in this embodiment, the photographing device 1 is not provided with a display unit, but may be provided with a display unit.

[0043] The microphone 108 converts sound into sound (signal) data. The sound processing unit 109 takes in the sound data output from the microphone 108 via the I / F bus, and performs a predetermined process on the sound data.

[0044] The CPU 111 controls the overall operation of the photographing device 1 and executes necessary processing. The ROM 112 stores various programs for the CPU 111. The SRAM 113 and the DRAM 114 are work memories, and store programs executed by the CPU 111, data in the middle of processing, etc. In particular, the DRAM 114 stores image data in the middle of processing by the image processing unit 104 and processed Mercator image data.

[0045] The operation unit 115 is a collective term for various operation buttons, a power switch, a shutter button, a touch panel that combines display and operation functions, etc. A user operates the operation buttons to input various shooting modes, shooting conditions, and the like.

[0046] The network I / F 116 is a general term for an interface circuit (such as a USB I / F) with an external medium such as an SD card or a personal computer. The network I / F 116 may be wireless or wired. The Mercator image data stored in the DRAM 114 is recorded in an external medium via the network I / F 116, or transmitted to an external device such as the video conference terminal 3 via the network I / F 116 as necessary.

[0047] The communication unit 117 communicates with an external device such as the video conference terminal 3 by short-range wireless technology such as Wi-Fi or NFC (Near Field Communication) via an antenna 117a provided in the image capturing device 1. The communication unit 117 can also transmit Mercator image data to the external device of the video conference terminal 3.

[0048] The electronic compass 118 calculates the direction and tilt (Roll rotation angle) of the image capturing device 1 from the earth's magnetism, and outputs the direction and tilt information. This direction and tilt information is an example of related information (metadata) according to Exif, and is used for image processing such as image correction of the captured image. The related information also includes data such as the date and time the image was captured, and the data size of the image data.

[0049] <Hardware configuration of video conferencing terminal> Next, the hardware configuration of the video conference terminal 3 will be described with reference to Fig. 11. Fig. 11 is a diagram showing the hardware configuration of the video conference terminal. As shown in Fig. 11, the video conference terminal 3 includes a CPU 301, a ROM 302, a RAM 303, a flash memory 304, an SSD 305, a media I / F 307, an operation button 308, a power switch 309, a bus line 310, a network I / F 311, a camera 312, an image sensor I / F 313, a microphone 314, a speaker 315, an audio input / output I / F 316, a display I / F 317, an external device connection I / F 318, a short-range communication circuit 319, and an antenna 319a of the short-range communication circuit 319.

[0050] Of these, the CPU 301 controls the overall operation of the video conference terminal 3. The ROM 302 stores programs used to drive the CPU 301, such as an IPL (Initial Program Loader). The RAM 303 is used as a work area for the CPU 301. The flash memory 304 stores various data, such as communication programs, image data, and sound data. The SSD (Solid State Drive) 305 controls reading and writing of various data from the flash memory 304 under the control of the CPU 301. Note that a HDD may be used instead of the SSD. The media I / F 307 controls reading and writing (storing) of data from a recording medium 306, such as a flash memory. The operation button 308 is a button that is operated when selecting a destination of the video conference terminal 3, etc. The power switch 309 is a switch for switching the power of the video conference terminal 3 ON / OFF.

[0051] The network I / F 311 is an interface for data communication using the communication network 100 such as the Internet. The camera 312 is a type of built-in imaging means for capturing an image of a subject under the control of the CPU 301 to obtain image data. The image sensor I / F 313 is a circuit for controlling the driving of the camera 312. The microphone 314 is a type of built-in sound collection means for inputting sound. The sound input / output I / F 316 is a circuit for processing input / output of sound signals between the microphone 314 and the speaker 315 under the control of the CPU 301. The display I / F 317 is a circuit for transmitting image data to an external display 320 under the control of the CPU 301. The external device connection I / F 318 is an interface for connecting various external devices. The short-range communication circuit 319 is a communication circuit such as NFC (registered trademark) or Bluetooth (registered trademark).

[0052] 11. The bus line 310 is an address bus, a data bus, or the like for electrically connecting the various components such as the CPU 301 shown in FIG.

[0053] The display 4 is a type of display means configured with liquid crystal or organic electroluminescence (EL) for displaying an image of a subject, operation icons, etc. The display 4 is also connected to the display I / F 317 by a cable 4c. This cable 4c may be a cable for analog RGB (VGA) signals, a cable for component video, or a cable for HDMI (High-Definition Multimedia Interface) (registered trademark) or DVI (Digital Video Interactive) signals.

[0054] The camera 312 includes a lens and a solid-state imaging element that converts light into electric charges to digitize an image (video) of a subject, and a CMOS sensor, CCD sensor, or the like is used as the solid-state imaging element. External devices such as an external camera, an external microphone, and an external speaker can be connected to the external device connection I / F 318 via a USB (Universal Serial Bus) cable, or the like. When an external camera is connected, the external camera is driven in preference to the built-in camera 312 under the control of the CPU 301. Similarly, when an external microphone or an external speaker is connected, the external microphone and the external speaker are driven in preference to the built-in microphone 314 and the built-in speaker 315, respectively, under the control of the CPU 301.

[0055] Moreover, the recording medium 306 is configured to be detachable from the video conference terminal 3. Moreover, as long as it is a non-volatile memory that reads or writes data under the control of the CPU 301, it is not limited to the flash memory 304, and an EEPROM (Electrically Erasable and Programmable ROM) or the like may be used.

[0056] <Communication management system, PC hardware configuration> Next, the hardware configuration of the communication management system 5 and the PC 7 will be described with reference to Fig. 12. Fig. 12 is a hardware configuration diagram of the communication management system and the PC. Note that since the communication management system 5 and the PC 7 are both computers and have the same configuration, the following describes the configuration of the communication management system 5, and the description of the configuration of the PC 7 will be omitted.

[0057] The communication management system 5 includes a CPU 501 for controlling the overall operation of the communication management system 5, a ROM 502 storing a program used to drive the CPU 501, such as an IPL, a RAM 503 used as a work area for the CPU 501, a HD 504 for storing various data such as programs for the communication management system 5, a HDD (Hard Disk Drive) 505 for controlling the reading or writing of various data from the HD 504 under the control of the CPU 501, a media drive 507 for controlling the reading or writing (storage) of data from a recording medium 506, such as a flash memory, a display 508 for displaying various information such as a cursor, a menu, a window, characters, or an image, a network I / F 509 for data communication using the communication network 100, a keyboard 511 having multiple keys for inputting characters, numbers, various instructions, etc., a mouse 512 for selecting and executing various instructions, selecting a processing target, moving the cursor, etc., a CD-RW (Compact The device is equipped with a CD-RW drive 514 that controls the reading and writing of various data from and to a CD-ReWritable disc 513, and bus lines 510 such as an address bus and a data bus for electrically connecting the above components as shown in FIG. 12.

[0058] <Hardware configuration of smartphone> Next, the hardware of the smartphone will be described with reference to Fig. 13. Fig. 13 is a hardware configuration diagram of the smartphone. As shown in Fig. 13, the smartphone 9 includes a CPU 901, a ROM 902, a RAM 903, an EEPROM 904, a CMOS sensor 905, an acceleration / direction sensor 906, a media I / F 908, and a GPS receiver 909.

[0059] Among these, the CPU 901 controls the overall operation of the smartphone 9. The ROM 902 stores programs used to drive the CPU 901, such as an IPL. The RAM 903 is used as a work area for the CPU 901. The EEPROM 904 reads or writes various data, such as smartphone programs, under the control of the CPU 901. The CMOS (Complementary Metal Oxide Semiconductor) sensor 905 captures an object (mainly a self-portrait) and obtains image data under the control of the CPU 901. The acceleration / direction sensor 906 is an electronic magnetic compass that detects geomagnetism, a gyrocompass, an acceleration sensor, or other such sensors. The media I / F 908 controls the reading or writing (storage) of data from or to a recording medium 907, such as a flash memory. The GPS receiver 909 receives GPS signals from GPS satellites.

[0060] The smartphone 9 also has a long-distance communication circuit 911, a camera 912, an image sensor I / F 913, a microphone 914, a speaker 915, an audio input / output I / F 916, a display 917, an external device connection I / F 918, a short-distance communication circuit 919, an antenna 919a of the short-distance communication circuit 919, and a touch panel 921.

[0061] Among these, the long-distance communication circuit 911 is a circuit that communicates with other devices via the communication network 100. The camera 912 is a type of built-in imaging means that captures an image of a subject and obtains image data under the control of the CPU 901. The image sensor I / F 913 is a circuit that controls the driving of the camera 912. The microphone 914 is a type of built-in sound collection means that inputs sound. The sound input / output I / F 916 is a circuit that processes input and output of sound signals between the microphone 914 and the speaker 915 under the control of the CPU 901. The display 917 is a type of display means such as a liquid crystal or organic EL that displays an image of a subject, various icons, etc. The external device connection I / F 918 is an interface for connecting various external devices. The short-distance communication circuit 919 is a communication circuit such as NFC or fBluetooth. The touch panel 921 is a type of input means that a user operates the smartphone 9 by pressing the display 917.

[0062] The smartphone 9 also includes a bus line 910. The bus line 910 is an address bus, a data bus, or the like for electrically connecting the components such as the CPU 901.

[0063] Incidentally, the recording media such as CD-ROMs on which the above-mentioned programs are stored, and the HD on which these programs are stored, can be provided domestically or internationally as program products.

[0064] <<Functional configuration of the embodiment>> Next, the functional configuration of this embodiment will be described with reference to Fig. 14 to Fig. 20. Fig. 14 is a functional block diagram of the photographing devices 1a and 1b, the video conference terminal 3, the communication management system 5, the PC 7, and the smartphone 9, which constitute a part of the image communication system of this embodiment.

[0065] <Functional configuration of the imaging device 1a> 14, the photographing device 1a has a reception unit 12a, an imaging unit 13a, a sound collection unit 14a, a communication unit 18a, and a memory / readout unit 19a. Each of these units is a function or means realized by operating any of the components shown in FIG. 10 in response to an instruction from a CPU 111 in accordance with a program for the photographing device loaded from the SRAM 113 onto the DRAM 114.

[0066] The photographing device 1 also has a storage unit 1000a that is configured by the ROM 112, the SRAM 113, and the DRAM 114 shown in Fig. 10. The storage unit 1000a stores a GUID (Globally Unique Identifier) ​​of the photographing device 1 itself.

[0067] (Functional configuration of the imaging device 1a) Next, each functional configuration of the image capturing device 1a will be described in more detail with reference to FIG. 10 and FIG.

[0068] The reception unit 12a of the photographing device 1a is realized mainly by the processing of the operation unit 115 and the CPU 111 shown in FIG. 10, and receives operation input from the user.

[0069] The imaging section 13a is mainly realized by the imaging unit 101, image processing unit 104, imaging control unit 105, and processing by the CPU 111 shown in FIG. 10, and captures images of scenery and the like to obtain captured image data.

[0070] The sound collection section 14a is realized by the microphone 108 and sound processing unit 109 shown in FIG.

[0071] The communication unit 18a is mainly realized by the processing of the CPU 111, and can communicate with the communication unit 38 of the video conference terminal 3 by short-range wireless communication technology such as the NFC standard, BlueTooth, or Wi-Fi.

[0072] The storage / reading unit 19a is realized mainly by the processing of the CPU 111 shown in FIG. 10, and stores various data (or information) in the storage unit 1000a and reads various data (or information) from the storage unit 1000a.

[0073] <Functional configuration of the imaging device 1b> The photographing device 1b has a reception unit 12b, an imaging unit 13b, a sound collection unit 14b, a communication unit 18b, and a memory / readout unit 19b. These have the same functions as the reception unit 12a, the imaging unit 13a, the sound collection unit 14a, the communication unit 18a, and the memory / readout unit 19a in the photographing device 1a, respectively, and therefore the description thereof will be omitted. The photographing device 1 also has a memory unit 1000b constructed by the ROM 112, the SRAM 113, and the DRAM 114 shown in Fig. 10. The memory unit 1000b stores the GUID of the own device.

[0074] <Functional configuration of videoconferencing terminal 3> 14, the video conference terminal 3 has a transmission / reception unit 31, a reception unit 32, an image / sound processing unit 33, a display control unit 34, a determination unit 35, a creation unit 36, a communication unit 38, and a storage / readout unit 39. Each of these units is a function or means realized by any of the components shown in FIG. 11 operating in response to an instruction from the CPU 301 in accordance with the program for the video conference terminal 3 expanded from the flash memory 304 onto the RAM 303.

[0075] The video conference terminal 3 also has a storage unit 3000 constructed by a ROM 302, a RAM 303, and a flash memory 304 shown in Fig. 11. An image type management DB 3001, a photographing device management DB 3002, a display restriction management DB 3003, and a display restricted management DB 3004 are constructed in the storage unit 3000. Of these, the image type management DB 3001 is constructed by an image type management table shown in Fig. 15. The photographing device management DB 3002 is constructed by a photographing device management table shown in Fig. 16. The display restriction management DB 3003 is constructed by a display restriction management table shown in Fig. 17. The display restricted management DB 3004 is constructed by a display restricted management table shown in Fig. 18.

[0076] (Image type management table) FIG. 15 is a conceptual diagram showing an image type management table. In this image type management table, image data ID, an IP address which is an example of the destination of the transmission source terminal, and a source name are stored and managed in association with each other. Among these, the image data ID is an example of image data identification information for identifying image data when performing video communication. The same image data ID is added to image data transmitted from the same transmission source terminal. This allows the transmission destination terminal (receiving communication terminal) to identify the transmission source terminal of the received image data. The IP address of the transmission source terminal indicates the IP address of the communication terminal that transmits the image data indicated by the associated image data ID. The source name is a name for identifying the photographing device that outputs the image data indicated by the associated image data ID, and is an example of image type information. This source name is a name created by each communication terminal such as the video conference terminal 3 according to a predetermined naming rule.

[0077] For example, it is shown that three communication terminals with IP addresses "1.2.1.3", "1.2.2.3", and "1.3.1.3" are respectively transmitting image data indicated by image data IDs "RS001", "RS002", and "RS003". Furthermore, the types of images indicated by the source names of the communication terminals are "Video_Theta", "Video", and "Video_Theta", which indicate that the image types are "special image", "normal image", and "special image", respectively. Note that the special image here is a spherical panoramic image.

[0078] Note that data other than image data may also be managed in association with the image data ID. Data other than image data may be, for example, audio data or document data for screen sharing.

[0079] (Radiation device management table) 16 is a conceptual diagram showing an image capturing device management table. In this image capturing device management table, the vendor ID and product ID of the GUID of an image capturing device capable of obtaining two hemispherical images that are the basis of a celestial sphere panoramic image are stored and managed. As the GUID, for example, a vendor ID (VID) and product ID (PID) used in a USB device can be used. The vendor ID and product ID are stored in a communication terminal such as a video conference terminal when the terminal is shipped from the factory, but may be additionally stored after the shipment from the factory.

[0080] (Display restriction management table) Fig. 17 is a conceptual diagram showing a display restriction management table. In this display restriction management table, display restriction parameters indicating the contents of the restriction on the display of a predetermined area image are managed for each IP address of a communication terminal (destination terminal) to which data of two hemispherical images that are the basis of a celestial sphere panoramic image is transmitted. Specifically, the display restriction parameters indicate the contents of the restriction on each element of the radius (r), polar angle (θ), azimuth angle (φ), and viewing angle (α) shown in Fig. 7 and Fig. 8. For example, for a communication terminal with an IP address of "1.3.1.2", the contents are managed such that the viewing angle α must exceed 120° (it cannot be set to 120° or less).

[0081] (Display Restriction Management Table) 18 is a conceptual diagram showing a display restriction management table. This display restriction management table manages, for each IP address of a communication terminal (sender terminal) that transmits data of two hemispherical images that are the basis of a full celestial sphere panoramic image, a display restriction parameter indicating the content of the restriction on display of a predetermined area image that is imposed (received) on the own terminal (video conference terminal 3). This display restriction parameter is the same as the display restriction parameter managed in the display restriction management table.

[0082] (Functional configuration of video conferencing terminal) Next, each functional configuration of the video conference terminal 3 will be described in more detail with reference to FIG. 11 and FIG.

[0083] The transmission / reception unit 31 of the video conference terminal 3 is mainly realized by the processing of the network I / F 311 and CPU 301 shown in FIG. 11, and transmits and receives various data (or information) to and from the communication management system 5 via the communication network 100.

[0084] The reception unit 32 is realized mainly by the operation buttons 308 and processing by the CPU 301, and receives various selections or inputs from the user. In addition to the operation buttons 308, a touch panel or the like may be used as another input means.

[0085] 11, and performs image processing on image data obtained by capturing an image of a subject with camera 312. After the user's voice is converted into an audio signal by microphone 314, image / audio processing unit 33 performs audio processing on audio data related to this audio signal.

[0086] Furthermore, the image / sound processing unit 33 performs image processing on image data received from another communication terminal based on image type information such as a source name, so that the display control unit 34 displays an image on the display 4. Specifically, when the image type information indicates that the image is a special image, the image / sound processing unit 33 creates omnidirectional panoramic image data by converting the image data (for example, data of each hemispherical image as shown in Figs. 3(a) and (b)) into omnidirectional panoramic image data as shown in Fig. 4(b), and further creates a predetermined area image as shown in Fig. 6(b). In addition, the image / sound processing unit 33 outputs an audio signal related to audio data received from another communication terminal via the communication management system 5 to the speaker 315, and causes the speaker 315 to output audio.

[0087] The display control unit 34 is realized mainly by the processing of the display I / F 317 and the CPU 301, and performs control for displaying various images, characters, and the like on the display 4.

[0088] The determination unit 35 is realized mainly by the processing of the CPU 301, and determines, for example, the image type related to the image data received from the photographing device 1a.

[0089] The creation unit 36 ​​is mainly realized by the processing of the CPU 301, and creates a source name, which is an example of image type information, in accordance with the naming rules described above based on the result of the determination by the determination unit 35 of whether the image is a general image or a special image (here, a spherical panoramic image). For example, if the determination unit 35 determines that the image is a general image, the creation unit 36 ​​creates a source name "Video" indicating that it is a general image. On the other hand, if the determination unit 35 determines that the image is a special image, the creation unit 36 ​​creates a source name "Video_Theta" indicating that it is a special image.

[0090] The communication unit 38 is mainly realized by the processing of the short-range communication circuit 319, the antenna 318a, and the CPU 301, and can communicate with the communication unit 18a of the photographing device 1a by short-range wireless technology such as NFC, BlueTooth, Wi-Fi, etc. Although the communication unit 38 and the transmission / reception unit 31 have been described as having separate communication units, they may be configured to share a common communication unit.

[0091] The storage / reading unit 39 is mainly realized by the processing of the CPU 301 shown in FIG. 11, and stores various data (or information) in the storage unit 3000 and reads out various data (or information) from the storage unit 3000.

[0092] <Functional configuration of the communication management system> Next, each functional component of the communication management system 5 will be described in detail with reference to Fig. 12 and Fig. 14. The communication management system 5 has a transmitting / receiving unit 51, a determining unit 55, a generating unit 56, and a storing / reading unit 59. Each of these units is a function or means realized by any of the components shown in Fig. 12 operating in response to an instruction from the CPU 501 in accordance with the program for the communication management system 5 loaded from the HD 504 onto the RAM 503.

[0093] The communication management system 5 also has a storage unit 5000 constructed by the RAM 503 and HD 504 shown in Fig. 12. A session management DB 5001, an image type management DB 5002, and a total display restriction management DB 5003 are constructed in this storage unit 5000. Of these, the session management DB 5001 is constructed by the session management table shown in Fig. 19. The image type management DB 5002 is constructed by the image type management table shown in Fig. 20. The total display restriction management DB 5003 is constructed by the total display restriction management table shown in Fig. 25.

[0094] (Session management table) 19 is a conceptual diagram showing a session management table. In this session management table, the session ID and the IP addresses of the participating communication terminals are stored and managed in association with each other. Among them, the session ID is an example of session identification information for identifying a communication session that realizes a video call, and is generated for each virtual conference room. The session ID is also managed by each communication terminal such as the video conference terminal 3, and is used when selecting a communication session in each communication terminal. The IP addresses of the participating communication terminals indicate the IP addresses of the communication terminals that have participated in the virtual conference room indicated by the associated session ID.

[0095] (Image type management table) FIG. 20 is a conceptual diagram showing an image type management table. In the image type management table shown in FIG. 20, in addition to each piece of information managed in the image type management table shown in FIG. 15, a session ID that is the same as the session ID managed in the session management table is associated and managed. Here, it is shown that three communication terminals with IP addresses of "1.2.1.3", "1.2.2.3", and "1.3.1.3" are participating in the virtual conference room indicated by the same session ID "se101". In addition, the communication management system 5 manages the same image data ID, the IP address of the sender terminal, and the image type information managed by the communication terminal such as the video conference terminal 3 in order to transmit image type information, etc. to the communication terminal already in a video call and the newly participating communication terminal when a new communication terminal enters the virtual conference room. This eliminates the need to transmit and receive image type information, etc. between the communication terminal already in a video call and the newly participating communication terminal.

[0096] (All display restriction management table) Fig. 21 is a conceptual diagram showing a full display restriction management table. This full display restriction management table is a table that combines the display restriction table (see Fig. 17) and the display restricted table (see Fig. 18). That is, the full display restriction management table manages display restriction parameters that indicate the content of restrictions on the display of a predetermined area image for each IP address of a communication terminal (sender terminal) that transmits data of two hemispherical images that are the basis of a full celestial sphere panoramic image and each IP address of a communication terminal (destination terminal) that transmits the data. This allows the communication management system 5 to manage what restrictions the sender of image data applies to each destination.

[0097] (Functional configuration of communication management system) Next, each functional configuration of the communication management system 5 will be described in detail with reference to FIG. 12 and FIG.

[0098] The transmission / reception unit 51 of the communication management system 5 is mainly realized by the processing of the network I / F 509 and the CPU 501 shown in FIG. 12, and transmits and receives various data (or information) to / from the video conference terminal 3 or the PC 7 via the communication network 100.

[0099] The determination unit 55 is mainly realized by the processing of the CPU 501 and makes various determinations.

[0100] The generation unit 56 is mainly realized by the processing of the CPU 501 and generates an image data ID.

[0101] The storage / reading unit 59 is mainly realized by the HDD 505 shown in FIG. 12 and the processing of the CPU 501, and stores various data (or information) in the storage unit 5000 or reads out various data (or information) from the storage unit 5000.

[0102] <Functional Configuration of PC> Next, with reference to FIGS. 12 and 14, the functional configuration of the PC 7 will be described in detail. The PC 7 basically has the same functions as the video conference terminal 3. That is, as shown in FIG. 14, the PC 7 has a transmission / reception unit 71, a reception unit 72, an image / audio processing unit 73, a display control unit 74, a determination unit 75, a creation unit 76, a communication unit 78, and a storage / reading unit 79. Each of these units is a function or means realized by any of the components shown in FIG. 12 operating according to an instruction from the CPU 501 according to the program for the PC 7 expanded from the HD 504 onto the RAM 503.

[0103] In addition, the PC 7 has a storage unit 7000 constructed by the ROM 502, the RAM 503, and the HD 504 shown in FIG. 12. In this storage unit 7000, an image type management DB 7001 and a photographing device management DB 7002 are constructed. Note that since the image type management DB 7001 and the photographing device management DB 7002 have the same configuration as the image type management DB 3001 and the photographing device management DB 3002, respectively, the description thereof will be omitted.

[0104] (PC functional configuration) The transmission / reception unit 71 of the PC 7 is realized mainly by the processing of the network I / F 509 and the CPU 501 shown in FIG.

[0105] The reception unit 72 is mainly realized by the processing of the keyboard 511, the mouse 512 and the CPU 501, and realizes the same function as the reception unit 32. The image / sound processing unit 73 is mainly realized by an instruction from the CPU 501, and realizes the same function as the image / sound processing unit 33. The display control unit 74 is mainly realized by the processing of the CPU 501, and realizes the same function as the display control unit 34. The determination unit 75 is mainly realized by the processing of the CPU 501, and realizes the same function as the determination unit 35. The creation unit 76 is mainly realized by the processing of the CPU 501, and realizes the same function as the creation unit 36. The communication unit 78 is mainly realized by the processing of the CPU 501, and realizes the same function as the communication unit 38. The storage / readout unit 79 is realized by the processing of the CPU 501, and stores various data (or information) in the storage unit 7000 and reads out various data (or information) from the storage unit 7000.

[0106] <Smartphone functional configuration> Next, the functional configuration of the smartphone 9 will be described in detail with reference to Fig. 13 and Fig. 14. The smartphone 9 basically has the same functions as the video conference terminal 3. That is, as shown in Fig. 14, the smartphone 9 has a transmission / reception unit 91, a reception unit 92, an image / sound processing unit 93, a display control unit 94, a determination unit 95, a creation unit 96, a communication unit 98, and a storage / readout unit 99. Each of these units is a function or means realized by any of the components shown in Fig. 13 operating in response to an instruction from the CPU 901 in accordance with the program for the smartphone 9 expanded from the EEPROM 904 onto the RAM 903.

[0107] 13. An image type management DB 9001 and an imaging device management DB 9002 are configured in the storage unit 9000. Note that the image type management DB 9001 and the imaging device management DB 9002 have the same configurations as the image type management DB 3001 and the imaging device management DB 3002, respectively, and therefore a description thereof will be omitted.

[0108] (Smartphone functional configuration) The transmitter / receiver 91 of the smartphone 9 is mainly realized by the processing of the long-distance communication circuit 911 and the CPU 901 shown in FIG.

[0109] The reception unit 92 is realized mainly by the processing of the touch panel 921 and the CPU 901 , and realizes the same functions as the reception unit 32 .

[0110] The image / sound processor 93 is implemented mainly by instructions from the CPU 901 , and realizes the same functions as the image / sound processor 33 .

[0111] The display control unit 94 is realized mainly by the processing of the CPU 901, and realizes the same functions as the display control unit .

[0112] The determination unit 95 is realized mainly by the processing of the CPU 901, and realizes the same functions as the determination unit .

[0113] The creation unit 96 is realized mainly by the processing of the CPU 901, and realizes the same functions as the creation unit .

[0114] The communication unit 98 is mainly realized by the processing of the CPU 901, and realizes the same functions as the communication unit .

[0115] The storage / reading unit 99 is realized by the processing of the CPU 901 , and stores various data (or information) in the storage unit 9000 and reads various data (or information) from the storage unit 9000 .

[0116] <<Processing or Operation of the Embodiment>> <Participation process> Next, the processing or operation of this embodiment will be described with reference to Fig. 22 to Fig. 27. First, the process of joining a specific communication session will be described with reference to Fig. 22 and Fig. 23. Fig. 22 is a sequence diagram showing the process of joining a specific communication session. Fig. 23 is a diagram showing a selection screen for a communication session (virtual conference room).

[0117] First, when a user at site A (e.g., user A1) operates the videoconference terminal 3 to display a selection screen for a communication session (virtual conference room), the reception unit 32 receives the operation to display the selection screen, and the display control unit 34 displays the selection screen as shown in Fig. 23 on the display 4 (step S21). This selection screen displays selection buttons b1, b2, b3, etc., which indicate the virtual conference rooms R1, R2, R3, etc., to be selected. Each selection button b1, etc. is associated with a session ID.

[0118] When user A1 selects a desired selection button (selection button b1 in this case) of the virtual conference room, the reception unit 32 accepts the selection of the communication session (step S22). Then, the transmission / reception unit 31 transmits a participation request to the virtual conference room to the communication management system 5 (step S23). This participation request includes a session ID indicating the communication session whose selection was accepted in step S22, and the IP address of the video conference terminal 3 that is the request source terminal. As a result, the transmission / reception unit 51 of the communication management system 5 receives the participation request.

[0119] Next, the storage / reading unit 99 performs a process of participating in the communication session by adding the IP address received in step S23 to the participating terminal IP address field in the record of the session management DB 5001 (see FIG. 19) that has the same session ID as the session ID received in step S23 (step S24). Then, the transmitting / receiving unit 51 transmits a participation request response to the video conference terminal 3 (step S25). This participation request response includes the session ID received in step S23 and the result of the participation process. As a result, the transmitting / receiving unit 31 of the video conference terminal 3 receives the participation request response. Hereinafter, a case where the participation process is successful will be described.

[0120] <Image type information management process> Next, the management process of image type information will be described with reference to Fig. 24. Fig. 24 is a sequence diagram showing the management process of image type information.

[0121] First, when a user at site A (for example, user A1) connects the USB cable of the cradle 2a with the photographing device 1a attached thereto to the video conference terminal 3, the memory / read unit 19a of the photographing device 1a reads out the GUID of its own device (the photographing device 1a) stored in the memory unit 1000a, and the communication unit 18a transmits the GUID of its own device to the communication unit 38 of the video conference terminal 3 (step S51). As a result, the communication unit 38 of the video conference terminal 3 receives the GUID of the photographing device 1a.

[0122] Next, the determination unit 35 of the video conference terminal 3 determines the image type by determining whether the same vendor ID and product ID as the vendor ID and product ID in the GUID received in step S51 are managed in the photographing device management DB 3002 (see FIG. 16) (step S52). Specifically, if the same vendor ID and product ID are managed in the photographing device management DB 3002, the determination unit 35 determines that the photographing device 1a is a photographing device that captures special images (here, a celestial sphere panoramic image). On the other hand, if the same vendor ID and product ID are not managed in the photographing device management DB 3002, the determination unit 35 determines that the photographing device 1a is a photographing device that captures general images.

[0123] Next, the storage / reading unit 39 stores in the image type management DB 3001 (see FIG. 15) the IP address of the terminal (video conference terminal 3) which is the source terminal, in association with the image type information which is the result of the determination made in step S52 (step S53). In this state, the image data ID is not associated. The image type information is, for example, a source name determined according to a predetermined naming rule, and an image type (general image, special image).

[0124] Next, the transmitting / receiving unit 31 transmits a request to add image type information to the communication management system 5 (step S54). This request to add image type information includes the IP address of the own terminal, which is the transmission source terminal stored in step S53, and the image type information. As a result, the transmitting / receiving unit 51 of the communication management system 5 receives the request to add image type information.

[0125] Next, the storage / read unit 59 of the communication management system 5 searches the session management DB 5001 (see FIG. 19) using the IP address of the source terminal received in step S54 as a search key, and reads out the corresponding session ID (step S55).

[0126] Next, the generating unit 56 generates a unique image data ID (step S56). Then, the storing / reading unit 59 stores the session ID read out in step S55, the image data ID generated in step S56, and the IP address of the transmission source terminal and the image type information received in step S54 in association with each other as a new record in the image type management DB 5002 (see FIG. 20) (step S57). Then, the transmitting / receiving unit 51 transmits the image data ID generated in step S56 to the video conference terminal 3. As a result, the transmitting / receiving unit 31 of the video conference terminal 3 receives the image data ID (step S58).

[0127] Next, the storage / reading unit 39 of the videoconference terminal 3 stores the image data ID received in step S58 in the image type management DB 3001 (see FIG. 15) in association with the IP address and image type information of the terminal itself (videoconference terminal 3), which is the sending terminal, stored in step S53 above (step S59).

[0128] Meanwhile, the transmitting / receiving unit 51 of the communication management system 5 transmits an addition notification of the image type information to the smartphone 9, which is another communication terminal (step S60). This addition notification of the image type information includes the image data ID generated in step S56, and the IP address and image type information of the terminal (video conference terminal 3), which is the transmission source terminal, stored in step S53. As a result, the transmitting / receiving unit 91 of the smartphone 9 receives the addition notification of the image type information. Note that the transmission destination of the transmitting / receiving unit 51 is another IP address associated with the same session ID as the IP address of the video conference terminal 3 in the session management DB 5001 (see FIG. 19). In other words, the transmission destination is another communication terminal in the same virtual conference room as the video conference terminal 3.

[0129] Next, the storage / readout unit 99 of the smartphone 9 associates the image data ID, the IP address of the sending terminal, and the image type information received in step S60 and stores them as a new record in the image type management DB 9001 (see FIG. 15) (step S61). Similarly, an additional communication of the image type information is sent to another communication terminal, PC 7, which also stores the information in the image type management DB 7001 (see FIG. 15). As a result, the communication terminals can share the same information in the image type management DBs 3001, 7001, 9001.

[0130] <Image data communication processing> The communication process of image data in a video call will be described with reference to Fig. 25 to Fig. 28. Fig. 25 is a sequence diagram showing the communication process of image data in a video call.

[0131] First, image data obtained by photographing an object, scenery, or the like is transmitted from the communication unit 18a of the photographing device 1a to the communication unit 38 of the video conference terminal 3 (step S71). In this case, since the photographing device 1a is a device capable of obtaining two hemispherical images that are the basis of a celestial sphere panoramic image, the image data is composed of data of two hemispherical images as shown in Figures 3(a) and (b). As a result, the communication unit 38 of the video conference terminal 3 receives the image data.

[0132] Next, the transmitting / receiving unit 31 of the video conference terminal 3 transmits the image data sent from the image capturing device 1a to the communication management system 5 (step S72). This transmission includes an image data ID for identifying the image data to be transmitted. The IP address of the video conference terminal 3, which is the transmission source, is also transmitted. As a result, the transmitting / receiving unit 51 of the communication management system 5 receives the image data, the image data ID, and the IP address.

[0133] Next, the transmitting / receiving unit 51 of the communication management system 5 transmits the image data sent from the video conference terminal 3 to the smartphone 9 (step S73). This transmission includes an image data ID for identifying the image data to be transmitted. The IP address of the video conference terminal 3, which is the transmission source, is also transmitted. As a result, the transmitting / receiving unit 51 of the smartphone 9 receives the image data, the image data ID, and the IP address.

[0134] Next, in the smartphone 9, the storage / readout unit 99 searches the image type management DB 9001 (see FIG. 15) using the image data ID received in step S73 as a search key to read out corresponding image type information (source name) (step S74). Then, the smartphone 9 performs a process of creating a predetermined area image (step S75). Here, the process of step S75 will be described in detail with reference to FIG. 26. FIG. 26 is a flowchart showing the process of creating a predetermined area image.

[0135] First, the judgment unit 95 of the smartphone 9 judges whether the image type of the image data received in step S73 is a special image (here, a spherical panoramic image) based on the image type information read in step S74 (step S71-1). If it is a special image (YES), the storage / reading unit 99 searches for corresponding display restriction parameters using the IP address of the source communication terminal (here, the video conference terminal 3) as a search key (step S71-2).

[0136] Next, the determination unit 95 determines whether there is a display restriction parameter, that is, whether the display is restricted (step S71-3). If the display is restricted (YES), the determination unit 95 further determines whether the display restriction parameter restricts the display size more than the default display parameter (step S71-4). If the determination unit 95 determines that the display size is restricted (YES), the image / sound processing unit 93 creates a celestial sphere panoramic image from the image data received in step S73, and further creates a predetermined area image based on the display restriction parameter (step S71-5). In this case, the image / sound processing unit 93 combines an icon 191 indicating that the image is a celestial sphere panoramic image (described later) with the predetermined area image based on image type information indicating that the image is a special image (Video_Theta).

[0137] On the other hand, if the determination unit 95 determines in step S71-3 that the display is not restricted (NO), and if the determination unit 95 determines in step S71-4 that the display is not restricted (NO), the image / sound processing unit 93 creates a celestial sphere panoramic image from the image data received in step S73, and further creates a predetermined area image based on default display parameters (step S71-6). In this case, the image / sound processing unit 93 combines an icon 191 indicating that the image is a celestial sphere panoramic image (described later) with the predetermined area image based on image type information indicating that the image is a special image (Video_Theta).

[0138] Moreover, in the above step S71-1, if the determination unit 95 determines that the image type of the image data is not a special image (NO), the process of step S71 ends.

[0139] 25, the display control unit 94 displays the predetermined area image including the icon 191 on the display 917 of the smartphone 9 (step S76). When the image type information indicates that the image is a general image (Video), the image / sound processing unit 93 does not create a celestial sphere panoramic image from the image data received in step S73, and the display control unit 94 displays a general image not including the icon 191.

[0140] Next, the state of the video call will be described with reference to Fig. 27. Fig. 27 is an image diagram showing the state of the video call. Of these, Fig. 27(a) shows the case where the photographing device 1a is not used, and Fig. 27(b) shows the case where the photographing device 1a is used.

[0141] First, as shown in Fig. 27(a), when using the camera 312 (see Fig. 11) provided in advance in the video conference terminal 3 without using the image capturing device 1a, the angle of view is 125 degrees horizontally and 70 degrees vertically, so the video conference terminal 3 must be placed at the edge of the desk where each user A1, etc. can be seen. Therefore, each user A1, etc. must face the video conference terminal 3 when speaking. Also, since each user A1, etc. faces the video conference terminal 3, the display 4 must also be placed on the side of the video conference terminal 3. As a result, users A2 and A4 who are far from the video conference terminal 3 must speak relatively loudly because they are far from the microphone 314 (see Fig. 11), and the contents displayed on the display 4 are difficult to see.

[0142] 27(b), when the image capturing device 1a is used, two hemispherical images that are the basis of a celestial sphere panoramic image can be obtained, so the video conference terminal 3 and the display 4 can be placed relatively close to the center of the desk. This allows each user A1, etc., to speak relatively quietly since they are close to the microphone 314, and the contents displayed on the display 4 can be easily seen.

[0143] Next, a display example of the display 917 at the site B will be described with reference to FIG. 28. FIG. 28 shows an example of a screen during a video call at the site B. Among these, FIG. 28(a) shows a case where a celestial sphere panorama image and a predetermined area image are not created from the image data sent from the video conference terminal 3 (camera 1a) and the camera 1b, and the image data sent from the PC 7 (camera 8) is displayed as is. FIG. 28(b) shows a case where a celestial sphere panorama image and a predetermined area image are created from the image data sent from the video conference terminal 3 (camera 1a) and the camera 1b, and the image data sent from the PC 7 (camera 8) is displayed as is. Note that an image of the site A is displayed in the upper left display area of ​​the display 917, an image of the site B (own site) is displayed in the upper right display area, and an image of the site C is displayed in the lower left display area. In this example, since simultaneous video calls are being made at three sites, nothing is displayed in the lower right display area.

[0144] When image data sent from the image capturing devices 1a and 1b capable of capturing spherical panoramic images is displayed as is, it is displayed as shown in Fig. 28(a), where location A on the upper left side and location B (own location) on the upper right side are displayed as the front hemispherical image and the rear hemispherical image, respectively, as shown in Figs.

[0145] In response to this, when the image / sound processor 93 creates a celestial sphere panoramic image from image data sent from the photographing devices 1a and 1b capable of obtaining two hemispherical images that are the basis of the celestial sphere panoramic image, and further creates a predetermined area image, the predetermined area image, which is a planar image, is displayed as shown in Fig. 28(b). Note that, since the site C is equipped with the photographing device 8 that obtains a general image, a general image is displayed in both Figs. 28(a) and (b).

[0146] Furthermore, an icon 191 indicating that the image is a spherical panoramic image is displayed at the top left of each of the images of site A and site B (own site). The display position of the icon 191 may be anywhere, such as the top right, bottom left, or bottom right, instead of the top left. The type of icon 191 is not limited to that shown in FIG. 28(b). Instead of the icon 191, text such as "spherical image" may be displayed, or a combination of an icon and text may be displayed. As shown in FIG. 28(b), an icon 192 is also displayed in the display area of ​​site B.

[0147] Furthermore, the user can change the predetermined area related to the predetermined area image in the same spherical panoramic image. That is, the users B1 and B2 place their fingers on the touch panel 921 of the smartphone 9 and move them, so that the reception unit 92 receives the movement of the fingers, and the display control unit 94 can shift, rotate, reduce, or enlarge the predetermined area image. As a result, even if only users A1 and A2, who are part of the site A, are displayed in the initial setting (default), as shown in FIG. 28(b), the predetermined area image can be shifted to display users A4 and A3.

[0148] <Display restriction parameter setting process> Next, a process for setting a display restriction parameter will be described with reference to Fig. 29. Fig. 29 is a sequence diagram showing a process for setting a display restriction parameter.

[0149] First, in the source communication terminal (here, the video conference terminal 3), the user A1 or the like uses the operation button 308 to set the initial value of the display restriction parameter, and the reception unit 32 receives the initial value (step S101). For example, the initial value may be a state in which α (viewing angle)>120° and only the enlarged state display is permitted. Next, the storage / readout unit 99 stores the initial value of the display restriction parameter in the display restriction management DB 3003 (step S102). After that, in the video conference terminal 3, in order to notify the communication terminals of all the bases participating in the conference of the initial value of the display restriction parameter, the transmission / reception unit 31 transmits display restriction setting information indicating the display restriction setting to the communication management system 5 (step S103). This display restriction setting information includes the IP address of the source communication terminal (here, the video conference terminal 3), a list of IP addresses of the destination communication terminals, and the display restriction parameter set in step S101. The list of IP addresses includes the IP addresses of all communication terminals (here, the PC 7 and the smartphone 9) other than the own terminal (here, the video conference terminal) participating in the conference. As a result, the transmission / reception unit 51 of the communication management system 5 receives the display restriction setting information.

[0150] Next, in the communication management system 5, the storage / reading unit 99 stores display restriction parameters for each source communication terminal IP address and destination communication terminal IP address received in step S103 in the all display restriction management DB 5003 (step S105), as shown in Fig. 21. In this case, if a display restriction parameter has already been stored for the same source communication terminal IP address and destination communication terminal IP address pair, the storage / reading unit 99 overwrites and updates it.

[0151] Next, the transmitting / receiving unit 51 of the communication management system 5 transmits display restriction setting information indicating the setting of the display restriction to the smartphone 9 (step S106). This display restriction setting information includes the IP address of the transmission source communication terminal (here, the video conference terminal 3) and the display restriction parameters set in step S101. As a result, the transmitting / receiving unit 91 of the smartphone 9 receives the display restriction setting information. Then, in the smartphone 9, the storage / reading unit 99 associates the IP address of the transmission source communication terminal and the display restriction parameters received in step S105, and stores them in the display restriction management DB 9004 (step S106). As a result, the setting of the display restriction parameters requested by the video conference terminal 3 is completed in the smartphone 9.

[0152] Similarly, the transmitting / receiving unit 51 of the communication management system 5 transmits display restriction setting information indicating the setting of the display restriction to the PC 7 (step S107). This display restriction setting information includes the IP address of the sending communication terminal (here, the video conference terminal 3) and the display restriction parameters set in step S101. As a result, the transmitting / receiving unit 71 of the PC 7 receives the display restriction setting information. Then, in the PC 7, the storage / reading unit 79 associates the IP address of the sending communication terminal and the display restriction parameters received in step S107 and stores them in the display restriction management DB 7004 (step S108). As a result, the setting of the display restriction parameters requested by the video conference terminal 3 is completed in the PC 7.

[0153] This completes the display restriction parameter setting process.

[0154] <Display restriction parameter setting cancellation process (restricted side)> Next, a process of canceling the setting of the display restriction parameter at the request of the restricted side will be described with reference to Fig. 30. Fig. 30 is a sequence diagram showing a process of canceling the setting of the display restriction parameter at the request of the restricted side.

[0155] For example, when a user of the restricted smartphone 9 wishes to remove a display restriction parameter restricted by the video conference terminal 3, the user of the smartphone 9 performs an operation to remove the display restriction, and the reception unit 92 of the smartphone 9 accepts the removal of the display restriction (step 121). For example, on the screen shown in Fig. 28(b), user B1 of the smartphone 9 double-tap the display area of ​​the image for which the user wishes to remove the display restriction (here, the upper left display area showing user A1, etc.).

[0156] Next, the transmitting / receiving unit 91 transmits, to the communication management system 5, display restriction release request information indicating a request to release the display restriction (step S122). This display restriction release request information includes the IP address of the requesting communication terminal (here, the smartphone 9) and the IP address of the request destination communication terminal (here, the video conference terminal 3). The requesting communication terminal is the destination communication terminal that accepted the setting of the display restriction. The request destination communication terminal is the source communication terminal that set the display restriction. As a result, the transmitting / receiving unit 51 of the communication management system 5 receives the display restriction release request information. Then, the transmitting / receiving unit 51 transmits the display restriction release request information received in step S122 to the video conference terminal 3 (step S123). This display restriction release request information includes the IP address of the requesting communication terminal (here, the smartphone 9) received in step S122. As a result, the transmitting / receiving unit 31 of the video conference terminal 3 receives the display restriction release request information.

[0157] Next, the video conference terminal 3 executes a process of temporarily releasing the display restriction (step S124). For example, the storage / readout unit 39 searches the display restriction management DB 3003 using the IP address of the request source (destination) communication terminal (here, the smartphone 9) received in step S123 as a search key, thereby temporarily deleting the corresponding display restriction parameter. In this case, the image / sound processing unit 33 may output a notification sound or display a notification screen 351 as shown in FIG. 32 to notify the user A1 or the like that the release of the display restriction has been requested. FIG. 32 is an example of a screen of a video call including a notification screen. In addition, the video conference terminal 3 may automatically cause the storage / readout unit 39 to temporarily delete the display restriction parameter after a certain time (for example, 10 seconds) has elapsed since the output of the notification sound. In addition, the temporarily deleted display restriction parameter is restored after a predetermined time (for example, 10 minutes) from the deletion, or is restored by a restoration instruction from the communication terminal that is the source of the display restriction instruction.

[0158] Also, as the process of step S124, the display control unit 34 may display an approval request screen 352 for requesting approval for the release of the display restriction from the user A1 of the communication terminal of the request destination (here, the video conference terminal 3) as shown in FIG. 33. FIG. 33 is an example of a screen of a video call including an approval request screen. This approval request screen 352 includes a "Yes" button to be pressed when approving and a "No" button to be pressed when not approving. Then, when approving, the storage / reading unit 39 searches the display restriction management DB 3003 using the IP address of the communication terminal of the request source (destination) (here, the smartphone 9) received in step S123 as a search key, thereby temporarily deleting the corresponding display restriction parameter. On the other hand, when not approving, the transmission / reception unit 31 notifies the communication terminal of the request source (here, the smartphone 9) via the communication management system 5 that the approval is not approved (denied).

[0159] Next, the transmitting / receiving unit 31 of the video conference terminal 3 transmits display restriction release instruction information indicating that the display restriction release has been completed to the communication management system 5 (step S125). This display restriction release instruction information includes the IP address of the request destination (sender) communication terminal, which is the terminal itself (here, the video conference terminal 3), and the request source (sender) terminal IP address. As a result, the transmitting / receiving unit 51 of the communication management system 5 receives the display restriction release instruction information.

[0160] Next, in the communication management system 5, the storage / read unit 59 searches the full display restriction management DB 5003 using the pair of both IP addresses received in step S125 as a search key, and performs a restriction release process by temporarily deleting the corresponding display restriction parameters (step S126). Then, the transmission / reception unit 51 transmits, to the smartphone 9, display restriction release instruction information indicating that the display restriction release has been completed. This display restriction release instruction information includes the IP address of the communication terminal of the request destination (sender) received in step S125. As a result, the transmission / reception unit 91 of the smartphone 9 receives the display restriction release instruction information.

[0161] Next, the smartphone 9 performs a process of removing the display restriction (step S128). For example, the storage / readout unit 99 searches the display restriction management DB 3004 using the IP address of the request destination (source) communication terminal (here, the video conference terminal 3) received in step S127 as a search key, thereby temporarily deleting the corresponding display restriction parameter. As a result, in the smartphone 9, in step 71-3 shown in Fig. 26, it is determined that the display is not restricted, and the image / sound processing unit 93 can display the predetermined area image on the display 917 according to the operation of the users B1 and B2.

[0162] This completes the display restriction parameter setting release process (restricted side).

[0163] <Display restriction parameter setting cancellation process (restriction side)> Next, a process of individually canceling the display restriction parameter settings in response to an instruction from the limiting side will be described with reference to Fig. 35. Fig. 31 is a sequence diagram showing a process of canceling the display restriction parameter settings in response to a request from the limiting side.

[0164] For example, the display control unit 34 displays "Release display restriction" buttons 361 and 362 during a video call as shown in FIG. 34. When the user A1 or the like of the video conference terminal 3 on the restricting side wants to release the display restriction only for the site B among the sites B and C, the user A1 or the like presses the "Release display restriction" button 361, and the reception unit 32 of the video conference terminal 3 accepts the release of the display restriction for the site B (step 201). As a result, the storage / reading unit 39 searches the display restriction management DB 3003 using the IP address of the communication terminal of the selected site B as a search key, and temporarily deletes the corresponding display restriction parameter, thereby releasing the display restriction (step S202). Then, in the video conference terminal 3, in order to notify the communication terminal of the site B (here, the smartphone 9) participating in the conference of the release of the display restriction, the transmission / reception unit 31 transmits display restriction release information indicating the release of the display restriction to the communication management system 5 (step S203). This display restriction lifting information includes the IP address of the communication terminal that is the sender (here, the video conference terminal 3) and the IP address of the communication terminal that is the destination of the display lifting. As a result, the transmitting / receiving unit 51 of the communication management system 5 receives the display restriction lifting information.

[0165] Next, in the communication management system 5, the memory / reading unit 99 performs a restriction removal process by temporarily deleting from the all display restriction management DB 5003 the display restriction parameters corresponding to the pair of IP address of the source communication terminal and the IP address of the destination communication terminal received in step S203 (step S204).

[0166] Next, the transmitting / receiving unit 51 of the communication management system 5 transmits display restriction release information indicating the release of the display restriction to the smartphone 9 (step S205). This display restriction release information includes the IP address of the source communication terminal (here, the video conference terminal 3) that is the source of the display restriction release. As a result, the transmitting / receiving unit 91 of the smartphone 9 receives the display restriction release information. Then, in the smartphone 9, the storage / reading unit 99 searches the display restriction management DB 9004 using the IP address of the source communication terminal received in step S205 as a search key, thereby performing a restriction release process of temporarily deleting the corresponding display restriction parameter (step S206). As a result, in the smartphone 9, the display restriction requested by the video conference terminal 3 is released, and in the PC 7, the display restriction is not released, and the display restriction parameter setting release process (restriction side) is terminated.

[0167] <Display restriction parameter setting change process (restriction side)> Next, a process of canceling the setting of the display restriction parameter in response to an instruction from the limiting side will be described with reference to Fig. 35. Fig. 35 is a sequence diagram showing a process of changing the setting of the display restriction parameter in response to an instruction from the limiting side.

[0168] FIG. 36 is an example of a screen of a video call including a change screen. For example, during a video call, the user A1 operates the operation button 308, and the display control unit 34 displays a change screen 371 of the display restriction as shown in FIG. 36. Then, when the user A1 operates the operation button 308 to select whether to restrict or not restrict the display for each of the locations B and C, and presses the "OK" button, the reception unit 32 of the video conference terminal 3 receives the presence or absence of the display restriction for each location (step 221). Here, a case is shown in which the display is restricted for the location B, and the display is not restricted for the location C. As a result, the storage / reading unit 39 searches the display restriction management DB 3003 using the IP address of the communication terminal of the selected location C as a search key, and temporarily deletes the corresponding display restriction parameter, thereby collectively changing the presence or absence of the display restriction (step S222). Note that the display restriction for the location B is continued, so the storage / reading unit 39 maintains the display restriction parameter for the location B without temporarily deleting it.

[0169] Next, the transmitting / receiving unit 31 transmits display restriction change information indicating a change in the display restriction to the communication management system 5 (step S223). This display restriction change information includes the IP address of the source communication terminal (here, the video conference terminal 3), a list of IP addresses including the IP addresses of the destination communication terminals to which the display restriction change is to be made, display restriction release instruction information indicating an instruction to release the display restriction, and display restriction reset instruction information indicating an instruction to reset the display restriction. In this case, the display restriction release instruction information or the display restriction reset instruction information is associated with each IP address of the destination communication terminal. As a result, the transmitting / receiving unit 51 of the communication management system 5 receives the display restriction change information.

[0170] Next, in the communication management system 5, for the destination communication terminal associated with the display restriction release instruction information, the storage / reading unit 99 performs a restriction change process by temporarily deleting the display restriction parameters corresponding to the pair of the source communication terminal IP address and the destination communication terminal IP address received in step S223 from the all display restriction management DB 5003 (step S224). Note that for the destination communication terminal associated with the display restriction release instruction information, the storage / reading unit 99 maintains the display restriction parameters in the all display restriction management DB 5003 without temporarily deleting them.

[0171] Next, the transmitting / receiving unit 51 of the communication management system 5 transmits display restriction change information including an instruction to reset the display restriction to the smartphone 9 (step S225). This display restriction setting information includes the IP address of the communication terminal that transmitted it (here, the video conference terminal 3). As a result, the transmitting / receiving unit 91 of the smartphone 9 receives the display restriction change information. Then, in the smartphone 9, the storage / reading unit 99 performs a process of resetting the display restriction parameters in the display restriction management DB 9004 for the IP address of the communication terminal that transmitted it received in step S205 (step S226). In the case of resetting, the communication management system 5 does not need to transmit the display restriction change information to the communication terminal.

[0172] The transmitting / receiving unit 51 of the communication management system 5 transmits display restriction change information including an instruction to release the display restriction to the PC 7 (step S227). This display restriction change information includes the IP address of the source communication terminal (here, the video conference terminal 3) that releases the display restriction. As a result, the transmitting / receiving unit 71 of the PC 7 receives the display restriction change information. Then, in the PC 7, the storing / reading unit 79 searches the display restriction management DB 7004 using the IP address of the source communication terminal received in step S227 as a search key, thereby performing a restriction release process of temporarily deleting the corresponding display restriction parameter (step S228). As a result, in the PC 7, the display restriction instructed by the video conference terminal 3 is released, and the display restriction parameter setting change process (restriction side) is completed.

[0173] <<Main Effects of This Embodiment>> As described above, according to this embodiment, a communication terminal such as the video conference terminal 3 can create a celestial sphere panoramic image and further create a predetermined area image by corresponding image type information based on an image data ID transmitted together with image data. This has the effect of preventing the front hemispherical image and the rear hemispherical image from being displayed as shown in Fig. 28(a).

[0174] Furthermore, the communication terminal may release the display restriction in response to a request from another communication terminal (see steps S121 to S128), may proactively release the display restriction from its own terminal for a specific other communication terminal (steps S201 to S206), or may proactively reset (or maintain) or release the display restriction from its own terminal for each other communication terminal (steps S221 to S228). This allows for flexible setting and release of display restrictions, so that if a user of another communication terminal that has once been subject to a display restriction wants to roughly grasp the location of the communication terminal, they can do so by releasing the display restriction. [Explanation of symbols]

[0175] 1a Imaging device 1b Imaging device 3. Video conferencing terminal (an example of a communication terminal) 5. Communications Management System 7 PC (an example of a communication terminal) 8. Imaging Equipment 9 Smartphones (an example of a communication device) 31 Transmitting / receiving unit (an example of a transmitting means, an example of a receiving means) 32 Reception unit (an example of a reception means) 34 Display control unit (an example of a display control means) 71 Transmitting / receiving unit (an example of a transmitting means, an example of a receiving means) 72 Reception unit (an example of reception means) 74 Display control unit (an example of a display control means) 91 Transmitting / receiving unit (an example of a transmitting means, an example of a receiving means) 92 Reception unit (an example of reception means) 94 Display control unit (an example of a display control means) [Prior art documents] [Patent documents]

[0176] [Patent Document 1] JP 2012-178135 A [Patent Document 2] JP 2011-223076 A [Patent Document 3] JP 2002-034009 A [Patent Document 4] JP 2001-148850 A

Claims

1. A determination means for determining whether or not display restriction is imposed based on the type of image data; a display control means for displaying a predetermined area image of a special image from the image data based on a result of the judgment by the judgment means; A communication terminal having the above configuration.

2. A communication terminal as described in claim 1, wherein the judgment means judges whether or not the image data is subject to display restrictions when the image data is a special image.

3. A communication terminal as described in claim 1 or 2, wherein the judgment means does not judge whether or not the image data is subject to display restrictions if the image data is not a special image.

4. A communication terminal as described in any one of claims 1 to 3, wherein the display control means displays a general image if the image data is not a special image.

5. The determination means, when the display is restricted, determines whether the display restriction parameters limit the display size more than the default display parameters, The communication terminal according to any one of claims 1 to 4, wherein when it is determined that the display restriction parameters limit the display size more than the default display parameters, the display control means displays a predetermined area image of the special image from the image data based on the display restriction parameters.

6. A communication terminal as described in Claim 5, wherein the display control means, when it is determined that the display restriction parameters are less restrictive in display size than the default display parameters, displays a specified area image of the special image from the image data based on the default display parameters.

7. A communication terminal as described in any one of claims 1 to 6, wherein the display control means changes and displays a specified area in the specified area image in response to a user's operation.

8. A communication terminal described in any one of claims 1 to 7, wherein the special image is a spherical panoramic image.

9. A display method executed on a communication terminal, comprising: a determination step of determining whether or not the image data is subject to display restriction based on the type of the image data; a display control step of displaying a predetermined area image of a special image from the image data based on a result of the determination step; Display methods including.

10. A computer comprising: A determination means for determining whether or not display restriction is imposed based on the type of image data; a display control means for displaying a predetermined area image of a special image from the image data based on a result of the judgment by the judgment means; A program that functions as a

11. An image communication system having a communication terminal and a communication management system capable of communicating with the communication terminal, A determination means for determining whether or not display restriction is imposed based on the type of image data; a display control means for displaying a predetermined area image of a special image from the image data based on a result of the judgment by the judgment means; An image communication system having the above configuration.

Images