COMMUNICATION MANAGEMENT DEVICE, IMAGE COMMUNICATION SYSTEM, COMMUNICATION MANAGEMENT METHOD, AND PROGRAM

Abstract

To display a photographic image of an area including a predetermined position without individually operating a plurality of photographing devices.SOLUTION: A communication management device 50 receives coordinate values indicating the position of an attention point P in an area in which a plurality of photographing devices designated by a user are arranged, and division number information when the area is divided at the attention point P as the boundary (step S73). Subsequently, the communication management device 50 transmits, to a communication terminal 70, first predetermined area information indicating a predetermined area including a predetermined position corresponding to predetermined position information in a photographic image photographed by a reference photographing device, of the plurality of photographing devices, the reference photographing device arranged at the shortest distance to the predetermined position, and second predetermined area information indicating a predetermined area including the predetermined position in a photographic image photographed by a compensation photographing device that is arranged with respect to the reference photographing device at an angle closest to an angle obtained through division according to the division number designated by the user (step S77).SELECTED DRAWING: Figure 32

Description

[Technical field]

[0001] The present invention relates to a communication management device, an image communication system, a communication management method, and a program. [Background technology]

[0002] There is known a camera capable of capturing images in all directions using multiple wide-angle or fisheye lenses. There is also known a system that distributes image data captured by such a camera in real time and enables a user to view in real time the status of a distribution base where the camera is installed (placed) at another base.

[0003] For example, a system has been disclosed that allows a user to grasp the situation in a remote location by displaying images captured by a plurality of cameras placed in the remote location on a terminal (see Patent Document 1). Summary of the Invention [Problem to be solved by the invention]

[0004] However, with conventional methods, when a user wants to view a specific location in an area where multiple image capture devices are placed, a problem arises in that the user needs to perform individual operations on the multiple image capture devices placed in the area that includes the specific location. [Means for solving the problem]

[0005] In order to solve the above-mentioned problems, the invention according to claim 1 provides a communication management device that communicates with a communication terminal that displays images captured by a plurality of image capture devices, the communication management device comprising: a receiving means for receiving, from the communication terminal, predetermined position information indicating a predetermined position in an area in which the plurality of image capture devices are arranged, and division number information indicating a division number when the area is divided with the predetermined position as a boundary; a selection means for selecting a first photographing device and a second photographing device from the plurality of photographing devices based on arrangement position information indicating arrangement positions of the plurality of photographing devices, the predetermined position information and the division number information received by the receiving means; a generation means for generating first predetermined area information indicating a predetermined area including the predetermined position in a photographed image photographed by the first photographing device selected by the selection means and second predetermined area information indicating a predetermined area including the predetermined position in a photographed image photographed by the second photographing device selected by the selection means; First predetermined area information and the above Second predetermined area information and and a transmitting means for transmitting the above-mentioned to the communication terminal. Effect of the Invention

[0006] As described above, according to the present invention, it is possible to display a captured image of an area including a predetermined position without operating a plurality of image capturing devices individually. [Brief description of the drawings]

[0007] [Figure 1] 1A is a left side view of the imaging device, FIG. 1B is a front view of the imaging device, and FIG. 1C is a plan view of the imaging device. [Diagram 2] FIG. 13 is a conceptual diagram of the imaging device in use. [Diagram 3] (A) is a hemispherical image (front side) captured by the camera, (B) is a hemispherical image (rear side) captured by the camera, and (C) is an image represented by equirectangular projection. [Figure 4] (A) is a conceptual diagram showing how a sphere is covered with an equirectangular projection image, and (B) is a diagram showing a spherical image. [Diagram 5] FIG. 13 is a diagram showing the positions of a virtual camera and a predetermined area when a celestial sphere image is a three-dimensional solid sphere. [Figure 6] FIG. 6(A) is a three-dimensional perspective view of FIG. 5, and FIG. 6(B) is a diagram showing a state in which an image of a predetermined area is displayed on a display. [Figure 7] 13 is a diagram showing the relationship between predetermined region information and an image of a predetermined region T. FIG. [Figure 8] FIG. 1 illustrates points in three-dimensional Euclidean space in spherical coordinates. [Figure 9] FIG. 1 is a diagram showing an overall configuration of an image communication system. [Figure 10] FIG. 2 is a diagram for explaining an outline of a distribution point in the image communication system. [Figure 11] FIG. 2 is a diagram illustrating an example of a hardware configuration of the imaging device. [Figure 12] FIG. 2 is a diagram illustrating an example of a hardware configuration of a distribution terminal, a communication management device, and a communication terminal. [Figure 13] FIG. 1 illustrates an example of a functional configuration of an image communication system. [Figure 14] FIG. 1 illustrates an example of a functional configuration of an image communication system. [Figure 15] FIG. 1A is a schematic diagram showing an example of a photographing device management table, and FIG. 1B is a conceptual diagram showing an example of an image type management table. [Figure 16] FIG. 1A is a schematic diagram showing an example of a session management table, and FIG. 1B is a conceptual diagram showing an example of an image type management table. [Figure 17] FIG. 1A is a schematic diagram showing an example of a predetermined area management table, and FIG. 1B is a conceptual diagram showing an example of a placement information management table. [Figure 18] FIG. 13 is a schematic diagram illustrating an example of a distribution base management table. [Figure 19] FIG. 1A is a conceptual diagram showing an example of a weighting coefficient management table, and FIG. 1B is a conceptual diagram showing an example of a specific weighting coefficient management table. [Figure 20] FIG. 1A is a schematic diagram showing an example of an image type management table, and FIG. 1B is a conceptual diagram showing an example of a predetermined area management table. [Figure 21] FIG. 13 is a diagram for roughly explaining an example of the relationship between attention point P and the imaging device when the number of divisions is 2. [Figure 22] FIG. 13 is a diagram for explaining roughly an example of a selection process based on the angle between the image capturing devices when the number of divisions is 2. [Diagram 23] FIG. 13 is a diagram for roughly explaining an example of the relationship between attention point P and the imaging device when the number of divisions is 3. [Figure 24] FIG. 13 is a diagram for explaining roughly an example of a selection process based on the angle between the image capturing devices when the number of divisions is three. [Diagram 25] FIG. 13 is a diagram for explaining roughly an example of a selection process based on the angle between the image capturing devices when the number of divisions is three. [Figure 26] FIG. 13 is a diagram for roughly explaining an example of the relationship between attention point P and the imaging device when the number of divisions is four. [Figure 27] 1 is a diagram for explaining an example of a relationship between image capturing devices arranged at irregular positions and a point of interest P. FIG. [Figure 28] FIG. 11 is a sequence diagram showing an example of a process for participating in a specific communication session in the image communication system. [Figure 29] FIG. 13 is a diagram illustrating an example of a selection screen for a communication session. [Diagram 30] 11 is a sequence diagram showing an example of a management process of image type information in the image communication system. FIG. [Diagram 31] 1 is a sequence diagram showing an example of a communication process of captured image data and sound data in an image communication system. [Diagram 32] FIG. 11 is a sequence diagram showing an example of a display process of an attention point P in the image communication system. [Diagram 33] 10 is a flowchart illustrating an example of a selection process of an imaging device. [Diagram 34] FIG. 13 is a diagram illustrating an example of a location display screen displayed on the communication terminal. [Diagram 35] 10 is a flowchart illustrating an example of a selection process of a complementary photographing device. [Diagram 36] FIG. 13 is a diagram showing an example of a display screen on which a predetermined area image is displayed. [Figure 37] FIG. 11 is a sequence diagram showing a modified example of the display process of the attention point P in the image communication system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Note that if there is a duplicated portion in the description of the drawings, the description will be omitted.

[0009] [Embodiment] [Method of generating spherical images] A method for generating a spherical image will be described with reference to FIGS.

[0010] First, the appearance of the imaging device 10 will be described with reference to Fig. 1. The imaging device 10 is a digital camera for obtaining captured images that are the basis of a full celestial sphere (360°) image. Fig. 1(A) is a left side view of the imaging device, Fig. 1(B) is a front view of the imaging device, and Fig. 1(C) is a plan view of the imaging device.

[0011] As shown in Fig. 1(A), the photographing device 10 is sized so that a person can hold it in one hand. Also, as shown in Figs. 1(A), 1(B), and 1(C), an image sensor 103a is provided on the front side (front side) and an image sensor 103b is provided on the back side (rear side) of the upper part of the photographing device 10. These image sensors (image sensors) 103a and 103b are used in conjunction with optical members (for example, lenses 102a and 102b described later) capable of photographing a hemispherical image (angle of view of 180° or more). Also, as shown in Fig. 1(B), an operation unit 115 such as a shutter button is provided on the surface opposite to the front side of the photographing device 10.

[0012] Next, the usage of the photographing device 10 will be described with reference to Fig. 2. Fig. 2 is an image diagram of the photographing device in use. As shown in Fig. 2, the photographing device 10 is used, for example, by a user holding it in his / her hand to photograph subjects around the user. In this case, two hemispherical images can be obtained by capturing images of subjects around the user using the image sensor 103a and the image sensor 103b shown in Fig. 1, respectively.

[0013] Next, an outline of the process of creating a celestial sphere image from an image captured by the image capture device 10 will be described with reference to Fig. 3 and Fig. 4. Fig. 3(A) is a diagram showing a hemispherical image (front side) captured by the image capture device, Fig. 3(B) is a diagram showing a hemispherical image (rear side) captured by the image capture device, and Fig. 3(C) is a diagram showing an image expressed by equirectangular projection (hereinafter referred to as "equirectangular projection image"). Fig. 4(A) is a conceptual diagram showing a state in which a sphere is covered with an equirectangular projection image, and Fig. 4(B) is a diagram showing a celestial sphere image.

[0014] As shown in Fig. 3(A), the image obtained by the image sensor 103a becomes a hemispherical image (front side) curved by the lens 102a described below. Also, as shown in Fig. 3(B), the image obtained by the image sensor 103b becomes a hemispherical image (rear side) curved by the lens 102b described below. Then, the photographing device 10 synthesizes the hemispherical image (front side) and the hemispherical image (rear side) flipped 180 degrees to generate an equirectangular projection image EC as shown in Fig. 3(C).

[0015] Then, the photographing device 10 uses OpenGL ES (Open Graphics Library for Embedded Systems) to paste an equirectangular projection image EC so as to cover the spherical surface as shown in Fig. 4(A) and create a celestial sphere image (celestial sphere panoramic image) CE as shown in Fig. 4(B). In this way, the celestial sphere image CE is expressed as an image in which the equirectangular projection image EC faces the center of the sphere. Note that OpenGL ES is a graphics library used to visualize 2D (2-Dimensions) and 3D (3-Dimensions) data. Also, the celestial sphere image CE may be a still image or a video.

[0016] As described above, the spherical image CE is an image pasted to cover the spherical surface, which gives a sense of incongruity to people when they see it. Therefore, the imaging device 10 can display a predetermined region T (hereinafter, referred to as a "predetermined region image") that is a part of the spherical image CE as a planar image with little curvature, thereby enabling a display that does not give a sense of incongruity to people. This will be described with reference to Figs. 5 to 8.

[0017] FIG. 5 is a diagram showing the positions of the virtual camera and the predetermined area when the omnidirectional image is a three-dimensional sphere. The virtual camera IC corresponds to the position of the viewpoint of a user who views the omnidirectional image CE displayed as a three-dimensional sphere. FIG. 6(A) is a three-dimensional perspective view of FIG. 5, and FIG. 6(B) is a diagram showing a state in which an image of a predetermined area is displayed on a display. FIG. 6(A) shows the omnidirectional image CE shown in FIG. 5 as a three-dimensional sphere CS. If the omnidirectional image CE generated in this way is a sphere CS, as shown in FIG. 5, the virtual camera IC is located inside the omnidirectional image CE. The predetermined area T in the omnidirectional image CE is a shooting area of ​​the virtual camera IC, and is specified by the predetermined area information indicating the shooting direction and the angle of view of the virtual camera IC in the three-dimensional virtual space including the omnidirectional image CE. In addition, the zoom of the predetermined area T can also be expressed by moving the virtual camera IC closer to or farther away from the omnidirectional image CE. The predetermined area image Q is an image of a predetermined area T in the omnidirectional image CE. Therefore, the predetermined area T can be specified by the angle of view α and the distance f from the virtual camera IC to the omnidirectional image CE (see FIG. 7).

[0018] Then, the predetermined area image Q shown in Fig. 6(A) is displayed on a predetermined display as an image of the shooting area of ​​the virtual camera IC as shown in Fig. 6(B). The image shown in Fig. 6(B) is a predetermined area image represented by the initial setting (default) predetermined area information. The following description will be given using the shooting direction (ea, aa) and angle of view (α) of the virtual camera IC. Note that the predetermined area T may be represented by the shooting area (X, Y, Z) of the virtual camera IC, which is the predetermined area T, instead of the angle of view α and the distance f.

[0019] Next, the relationship between the predetermined area information and the image of the predetermined area T will be described with reference to FIG. 7. FIG. 7 is a diagram showing the relationship between the predetermined area information and the image of the predetermined area T. As shown in FIG. 7, "ea" indicates an elevation angle, "aa" indicates an azimuth angle, and "α" indicates an angle of view. That is, the attitude of the virtual camera IC is changed so that the gaze point of the virtual camera IC indicated by the shooting direction (ea, aa) becomes the center point CP(x, y) of the predetermined area T, which is the shooting area of ​​the virtual camera IC. As shown in FIG. 7, the center point CP(x, y) when the diagonal angle of view of the predetermined area T represented by the angle of view α of the virtual camera IC is α becomes the parameter ((x, y)) of the predetermined area information. The predetermined area image Q is an image of the predetermined area T in the omnidirectional image CE. f is the distance from the virtual camera IC to the center point CP(x, y). L is the distance between any vertex of the predetermined area T and the center point CP(x, y) (2L is the diagonal line). In FIG. 7, the trigonometric function shown in the following formula 1 generally holds.

[0020]

number

[0021] The image capturing device 10 described above is an example of an image capturing device capable of capturing a wide-angle image, and a celestial sphere image is an example of a wide-angle image. Here, a wide-angle image is generally an image captured using a wide-angle lens, and is captured with a lens that can capture a wider range than the human eye can sense. Also, it generally means an image captured with a lens with a focal length of 35 mm or less in 35 mm film equivalent.

[0022] FIG. 8 is a diagram showing points in a three-dimensional Euclidean space in spherical coordinates. Here, the position coordinates of the center point CP when expressed in a spherical polar coordinate system are (r, θ, φ). (r, θ, φ) are the radius, polar angle, and azimuth angle, respectively. The radius r is the distance from the origin of the three-dimensional virtual space including the omnidirectional image to the center point CP, and is therefore equal to f. FIG. 8 is a diagram showing these relationships. Hereafter, the explanation will be given using the position coordinates (r, θ, φ) of the virtual camera IC.

[0023] [Image communication system overview] Next, an outline of the configuration of the image communication system of this embodiment will be described with reference to Fig. 9. Fig. 9 is a diagram showing an example of the overall configuration of an image communication system. The image communication system 1 shown in Fig. 9 is a system that allows captured images such as videos distributed from multiple distribution bases to be displayed at multiple viewing bases, thereby allowing a wide range of images (for example, spherical images) captured at the distribution bases to be viewed in real time. Note that in this embodiment, the distribution bases are also referred to as areas.

[0024] As shown in FIG. 9, the image communication system 1 includes an image capture device 10 (10A, 10B; hereinafter referred to as image capture device 10 when no distinction is necessary) and a distribution terminal 30 (30A, 30B; hereinafter referred to as distribution terminal 30 when no distinction is necessary) located at each of a plurality of distribution bases (distribution base A (area A), distribution base B (area B)), a communication management device 50, and a communication terminal 70 (70C, 70D; hereinafter referred to as communication terminal 70 when no distinction is necessary) located at each of a plurality of viewing bases (viewing base A, viewing base B).

[0025] The distribution terminal 30, the communication management device 50, and the communication terminal 70 constituting the image communication system 1 can communicate via a communication network 100. The communication network 100 is constructed by the Internet, a mobile communication network, a LAN (Local Area Network), etc. Note that the communication network 100 may include not only wired communication but also wireless communication networks such as 3G (3rd Generation), 4G (4th Generation), 5G (5th Generation), Wi-Fi (Wireless Fidelity) (registered trademark), WiMAX (Worldwide Interoperability for Microwave Access), or LTE (Long Term Evolution).

[0026] Among these, the photographing device 10 is a special digital camera for photographing a subject, scenery, etc., and obtaining two hemispherical images that are the basis of a celestial sphere image, as described above. The photographed image obtained by the photographing device 10 may be a moving image or a still image, or may be both a moving image and a still image. The photographed image may also include sound together with the image. The distribution terminal 30 distributes the image acquired from the photographing device 10 via a wired cable such as a USB (Universal Serial Bus) cable to the communication terminal 70 via the communication management device 50. Here, the photographing device 10A and the distribution terminal 30A are located at the same base, that is, distribution base A. Meanwhile, the photographing device 10B and the distribution terminal 30B are located at the same base, that is, distribution base B. Note that the number of distribution bases is not limited to two bases, and may be only one base, or may be three or more bases. The connection relationship between the photographing device 10 and the distribution terminal 30 may be a wireless connection using short-range wireless communication or the like, instead of a wired connection using a wired cable.

[0027] The communication management device 50 manages and controls the communication between the distribution terminal 30 and the communication terminal 70, and manages the type of image data being transmitted and received (for example, types of general images and special images). Here, the special images are spherical images. The communication management device 50 is disposed in a service company or the like that provides image communication services. The communication management device 50 has a server function, and in response to a request from the communication terminal 70, provides identification information of the image data (image data ID), the IP address of the photographing device, image type information indicating the type of image data, and the like.

[0028] The communication management device 50 may be constructed by a single computer, or may be constructed by multiple computers in which each unit (function or means) is divided and arbitrarily assigned. In addition, all or part of the functions of the communication management device 50 may be a server computer existing in a cloud environment, or a server computer existing in an on-premise environment.

[0029] The communication terminal 70 is a computer such as a PC used by a user at each viewing location. The communication terminal 70 displays an image (still image or video) distributed from the distribution terminal 30. The communication terminal 70 acquires a spherical image, which is an image captured by the imaging device 10, via the communication network 100. In addition, the communication terminal 70 has OpenGL ES installed, and can create predetermined area information indicating a partial area of ​​the spherical image, or create a predetermined area image from the spherical image transmitted from the distribution terminal 30. Here, the communication terminal 70C is placed at the viewing location A where the user A1 is located, and the communication terminal 70D is placed at the viewing location B where the user B is located.

[0030] 9 are merely examples, and other examples may be used. The communication terminal 70 is not limited to a PC, and may be, for example, a tablet terminal, a smartphone, a wearable terminal, a PJ (Projector), an IWB (Interactive White Board: an electronic whiteboard with a blackboard function that allows intercommunication), an autonomous robot, or the like.

[0031] Here, the distribution base in the image communication system 1 will be briefly described with reference to Fig. 10. Fig. 10 is a diagram for explaining the outline of the distribution base in the image communication system. Fig. 10 shows an image capture device 10A and a distribution terminal 30A arranged at distribution base A. Note that Fig. 10 describes an example of distribution base A, but the same applies to other distribution bases such as distribution base B.

[0032] In the distribution base A shown in FIG. 10, a plurality of image capture devices 10A (image capture devices 10A-1 to 10A-8) and a distribution terminal 30A capable of communicating with each of the image capture devices 10A are arranged in a space such as an office floor. The distribution base is not limited to an office floor, but may be a space such as a school, a factory, a warehouse, a construction site, a server room, or a store, where a user (viewer) of a viewing base needs to know the situation of a remote base. The image capture devices 10A are arranged at a predetermined interval on a desk arranged on an office floor, for example. In the distribution base A, the entire distribution base A is captured using the plurality of image capture devices 10A. In addition, the distribution terminal 30A receives captured image data from the image capture devices 10A and distributes the received captured image data to a communication terminal 70 of the viewing base. The number of image capture devices 10A and distribution terminals 30A arranged in the distribution base A is not limited to those illustrated in FIG. 10. In other words, as shown in the distribution base A in FIG. 9, the relationship between the image capture device 10A and the distribution terminal 30A may be one-to-one. 10, the relationship between the image capture device 10A and the distribution terminal 30A may be many to one. The same relationship applies to the distribution base B.

[0033] Conventionally, in a system in which images captured by a plurality of remotely located image capturing devices can be viewed from a viewing base, when a user wants to see a specific location at a distribution base, the user does not know which image captured by which image capturing device to view. In addition, when users at different viewing bases want to view different locations, operations (e.g., PTZ (pan-tilt-zoom) operations) on the image capturing devices conflict, so that multiple users viewing the same distribution base cannot perform exclusive viewing such as viewing different locations. Furthermore, even when the above-mentioned omnidirectional image is captured by an image capturing device, a process must be performed to individually operate the multiple omnidirectional images to display the desired location, and the user cannot intuitively operate it. Therefore, the image communication system 1 enables multiple users to view different locations at a distribution base by focusing on images captured by the same image capturing device through intuitive operations on the user's display screen.

[0034] [Hardware configuration of image communication system] Next, the hardware configuration of each device or terminal constituting the image communication system according to the embodiment will be described with reference to Figures 11 and 12. Note that components may be added or deleted from the hardware configurations shown in Figures 11 and 12 as necessary.

[0035] <Hardware configuration of the imaging device> First, the hardware configuration of the imaging device 10 will be described with reference to Fig. 11. Fig. 11 is a diagram showing an example of the hardware configuration of the imaging device. In the following, the imaging device 10 is an omnidirectional (all-directional) imaging device using two imaging elements, but the number of imaging elements may be two or more. In addition, the imaging device does not necessarily have to be a device dedicated to omnidirectional imaging, and an omnidirectional imaging unit may be attached to a normal digital camera, smartphone, or the like to have substantially the same functions as the imaging device 10.

[0036] As shown in FIG. 11, the photographing device 10 is composed of an imaging unit 101, an image processing unit 104, an imaging control unit 105, a microphone 108, a sound processing unit 109, a CPU (Central Processing Unit) 111, a ROM (Read Only Memory) 112, an SRAM (Static Random Access Memory) 113, a DRAM (Dynamic Random Access Memory) 114, an operation unit 115, an input / output I / F 116, a short-range communication circuit 117, an antenna 117a of the short-range communication circuit 117, an electronic compass 118, a gyro sensor 119, an acceleration sensor 120, and a network I / F 121.

[0037] Of these, the imaging unit 101 includes wide-angle lenses (so-called fisheye lenses) 102a and 102b (hereinafter, when there is no need to distinguish between them, they will be referred to as lenses 102) each having an angle of view of 180° or more for forming a hemispherical image, and two imaging elements 103a and 103b provided corresponding to the lenses. The imaging elements 103a and 103b include an image sensor such as a CMOS (Complementary Metal Oxide Semiconductor) sensor or a CCD (Charge Coupled Device) sensor that converts an optical image by the lenses 102a and 102b into image data of an electrical signal and outputs it, a timing generation circuit that generates horizontal or vertical synchronization signals and pixel clocks for the image sensors, and a group of registers in which various commands or parameters required for the operation of the imaging elements are set.

[0038] The imaging elements 103a and 103b of the imaging unit 101 are each connected to the image processing unit 104 via a parallel I / F bus. Meanwhile, the imaging elements 103a and 103b of the imaging unit 101 are each connected to the imaging control unit 105 via a serial I / F bus (such as an I2C bus). The image processing unit 104, the imaging control unit 105, and the sound processing unit 109 are connected to the CPU 111 via a bus 110. In addition, the ROM 112, the SRAM 113, the DRAM 114, the operation unit 115, the input / output I / F 116, the short-range communication circuit 117, the electronic compass 118, the gyro sensor 119, the acceleration sensor 120, and the network I / F 121 are also connected to the bus 110.

[0039] The image processing unit 104 takes in the image data output from the image sensors 103a and 103b via a parallel I / F bus, performs a predetermined processing on each image data, and then synthesizes the image data to create equirectangular projection image data such as that shown in Figure 3(C).

[0040] The imaging control unit 105 generally sets commands and the like in the register groups of the imaging elements 103a and 103b using an I2C bus, with the imaging control unit 105 acting as a master device and the imaging elements 103a and 103b acting as slave devices. Necessary commands and the like are received from the CPU 111. The imaging control unit 105 also uses the I2C bus to retrieve status data and the like from the register groups of the imaging elements 103a and 103b and send it to the CPU 111.

[0041] Furthermore, the imaging control unit 105 instructs the imaging elements 103a and 103b to output image data when the shutter button of the operation unit 115 is pressed (or tapped or otherwise operated, hereinafter simply referred to as "operation"). Some imaging devices 10 have a preview display function or a function corresponding to video display on a display (for example, a display of an external terminal such as a smartphone that performs short-distance communication with the imaging device 10 using the short-distance communication circuit 117). In this case, the image data is output from the imaging elements 103a and 103b continuously at a predetermined frame rate (frames / minute).

[0042] As described later, the imaging control unit 105 also functions as a synchronization control means for synchronizing the output timing of image data from the imaging elements 103a and 103b in cooperation with the CPU 111. In this embodiment, the imaging device 10 is not provided with a display unit, but a display unit may be provided. The microphone 108 converts sound into sound (signal) data. The sound processing unit 109 takes in the sound data output from the microphone 108 through an I / F bus and performs a predetermined process on the sound data.

[0043] The CPU 111 controls the overall operation of the photographing device 10 and executes necessary processing. The ROM 112 stores various programs for the CPU 111. The SRAM 113 and the DRAM 114 are work memories, and store programs executed by the CPU 111, data in the middle of processing, etc. In particular, the DRAM 114 stores image data in the middle of processing by the image processing unit 104, and data of a processed equirectangular projection image.

[0044] The operation unit 115 is a collective term for various operation buttons, a power switch, a shutter button, a touch panel having both display and operation functions, etc. A user operates the operation unit 115 to input various shooting modes, shooting conditions, etc.

[0045] The input / output I / F 116 is a general term for an interface circuit (such as a USB I / F) with an external medium such as an SD card or a personal computer. The input / output I / F 116 may be wireless or wired. The data of the equirectangular projection image stored in the DRAM 114 is recorded on an external medium via the input / output I / F 116, or transmitted to an external terminal (device) via the input / output I / F 116 as necessary.

[0046] The short-range communication circuit 117 communicates with an external terminal (device) by short-range wireless communication technology such as NFC (Near Field Communication), Bluetooth (registered trademark), or Wi-Fi (registered trademark) via an antenna 117a provided in the photographing device 10. The short-range communication circuit 117 can transmit data of the equirectangular projection image to the external terminal (device).

[0047] The electronic compass 118 calculates the direction of the image capturing device 10 from the earth's magnetism and outputs the direction information. This direction information is an example of related information (metadata) according to Exif and is used for image processing such as image correction of the captured image. The related information also includes each data of the image capture date and time and the data capacity of the image data. The gyro sensor 119 is a sensor that detects angle changes (roll angle, pitch angle, yaw angle) accompanying the movement of the image capturing device 10. The angle changes are an example of related information (metadata) according to Exif and are used for image processing such as image correction of the captured image. The acceleration sensor 120 is a sensor that detects acceleration in three axial directions. The image capturing device 10 calculates the attitude (angle with respect to the direction of gravity) of the own device (image capturing device 10) based on the acceleration detected by the acceleration sensor 120. The accuracy of image correction is improved by providing the acceleration sensor 120 in the image capturing device 10. The network I / F 121 is an interface for performing data communication using a communication network 100 such as the Internet via a router or the like.

[0048] <Hardware configuration of distribution terminal> Fig. 12 is a diagram showing an example of the hardware configuration of a distribution terminal. Each hardware component of distribution terminal 30 is indicated by a reference number in the 300 series. Distribution terminal 30 is constructed by a computer, and includes a CPU 301, a ROM 302, a RAM 303, a HD (Hard Disk) 304, a HDD controller 305, a display 306, an external device connection I / F 308, a network I / F 309, a bus line 310, a keyboard 311, a pointing device 312, a DVD-RW (Digital Versatile Disk Rewritable) drive 314, a media I / F 316, an audio input / output I / F 317, a microphone 318, a speaker 319, and a short-distance communication circuit 320, as shown in Fig. 12.

[0049] Among these, the CPU 301 controls the operation of the entire distribution terminal 30. The ROM 302 stores programs such as an IPL used to drive the CPU 301. The RAM 303 is used as a work area for the CPU 301. The HD 304 stores various data such as programs. The HDD controller 305 controls reading or writing of various data from or to the HD 304 under the control of the CPU 301. The display 306 displays various information such as a cursor, a menu, a window, a character, or an image. The display 306 is an example of a display unit. The display 306 may be a touch panel display equipped with an input means. The external device connection I / F 308 is an interface for connecting various external devices. In this case, the external device is, for example, a USB memory or a printer. The network I / F 309 is an interface for data communication using the communication network 100. The bus line 310 is an address bus or a data bus for electrically connecting each component such as the CPU 301 shown in FIG. 12.

[0050] The keyboard 311 is a type of input means having a plurality of keys for inputting characters, numbers, various instructions, etc. The pointing device 312 is a type of input means for selecting or executing various instructions, selecting a processing target, moving a cursor, etc. The input means may be not only the keyboard 311 and the pointing device 312, but also a touch panel or a voice input device, etc. The DVD-RW drive 314 controls reading or writing of various data from a DVD-RW 313 as an example of a removable recording medium. The DVD-RW is not limited to a DVD-RW, and may be a DVD-R or a Blu-ray (registered trademark) Disc, etc. The media I / F 316 controls reading or writing (storing) of data from a recording medium 315 such as a flash memory. The microphone 318 is a type of built-in sound collecting means for inputting sound. The sound input / output I / F 317 is a circuit for processing input / output of sound signals between the microphone 318 and the speaker 319 under the control of the CPU 301. The short-range communication circuit 320 is a communication circuit for communicating with an external terminal (device) by short-range wireless communication technology such as NFC, Bluetooth, or Wi-Fi.

[0051] <Hardware configuration of communication management device> Fig. 12 is a diagram showing an example of the hardware configuration of the communication management device. Each piece of hardware configuration of the communication management device 50 is indicated by a reference number in the 500 range in parentheses. The communication management device 50 is constructed by a computer, and as shown in Fig. 12, has the same configuration as the distribution terminal 30, so a description of each piece of hardware configuration will be omitted.

[0052] <Hardware configuration of communication terminal> Fig. 12 is a diagram showing an example of the hardware configuration of a communication terminal. Each piece of hardware configuration of the communication terminal 70 is indicated by a reference number in the 700 range in parentheses. The communication terminal 70 is constructed by a computer, and as shown in Fig. 12, has the same configuration as the distribution terminal 30, so a description of each piece of hardware configuration will be omitted.

[0053] Each of the above programs may be recorded in a computer-readable recording medium as an installable or executable file and distributed. Examples of the recording medium include a CD-R (Compact Disc Recordable), a DVD (Digital Versatile Disk), a Blu-ray Disc, an SD card, and a USB memory. The recording medium may be provided domestically or internationally as a program product. For example, the communication terminal 70 realizes the communication management method according to the present invention by executing the program according to the present invention.

[0054] [Functional configuration of image communication system] Next, the functional configuration of the image communication system according to the embodiment will be described with reference to Fig. 13 to Fig. 20. Fig. 13 and Fig. 14 are diagrams showing an example of the functional configuration of the image communication system. Fig. 13 and Fig. 14 show devices or terminals shown in Fig. 9 that are related to the processing or operation described below.

[0055] <Functional configuration of the imaging device> First, the functional configuration of the photographing device 10 will be described with reference to Fig. 13. The photographing device 10 has a communication unit 11, a reception unit 12, an imaging unit 13, a sound collection unit 14, and a storage / readout unit 19. Each of these units is a function or means realized by operating any of the components shown in Fig. 11 by an instruction from the CPU 111 according to a program for the photographing device that is expanded from the SRAM 113 onto the DRAM 114. The photographing device 10 also has a storage unit 1000 constructed by the ROM 112, SRAM 113, and DRAM 114 shown in Fig. 11. The storage unit 1000 stores a GUID (Globally Unique Identifier) ​​of the own device.

[0056] The communication unit 11 is mainly realized by the processing of the CPU 111, and communicates various data or information with other devices or terminals. The communication unit 11 performs data communication with other devices or terminals by short-range wireless communication technology, for example, using a short-range communication circuit 117. The communication unit 11 also performs data communication with other devices or terminals via various cables, for example, using an input / output I / F 116. Furthermore, the communication unit 11 performs data communication with other devices or terminals via a communication network 100, using a network I / F 121.

[0057] The reception unit 12 is mainly realized by the processing of the CPU 111 on the operation unit 115, and receives various selections or inputs from the user. The imaging unit 13 is mainly realized by the processing of the CPU 111 on the imaging unit 101, the image processing unit 104, and the imaging control unit 105, and captures an image of a subject such as a landscape, and acquires captured image data. The sound collection unit 14 is mainly realized by the processing of the CPU 111 on the microphone 108 and the sound processing unit 109, and collects sounds around the imaging device 10.

[0058] The storage / reading unit 19 is realized mainly by the processing of the CPU 111 , and stores various data (or information) in the storage unit 1000 and reads various data (or information) from the storage unit 1000 .

[0059] <Functional configuration of distribution terminal> Next, the functional configuration of the distribution terminal 30 will be described with reference to Fig. 13. The distribution terminal 30 has a transmission / reception unit 31, a reception unit 32, an image / sound processing unit 33, a display control unit 34, a determination unit 35, a creation unit 36, a communication unit 37, and a storage / readout unit 39. Each of these units is a function or means realized by operating any of the components shown in Fig. 12 by an instruction from the CPU 301 according to the program for the distribution terminal loaded from the HD 304 onto the RAM 303. The distribution terminal 30 also has a storage unit 3000 constructed by the ROM 302, RAM 303, and HD 304 shown in Fig. 12.

[0060] The transmission / reception unit 31 is realized mainly by the processing of the CPU 301 for the network I / F 309, and transmits and receives various data or information to and from other devices or terminals via the communication network 100.

[0061] The reception unit 32 is realized mainly by the processing of the CPU 301 on the keyboard 311 or the pointing device 312, and receives various selections or inputs from the user.

[0062] The image / sound processing unit 33 is mainly realized by the processing of the CPU 301, and performs image processing on the captured image data acquired by the photographing device 10 by capturing an image of a subject. In addition, the image / sound processing unit 33 performs audio processing on the audio data related to the audio signal after the user's voice is converted into an audio signal by the microphone 318. For example, the image / sound processing unit 33 performs image processing on the captured image data received from the photographing device 10 based on image type information such as a source name so that the display control unit 34 displays an image on the display 306. Specifically, when the image type information indicates that the image is a special image, the image / sound processing unit 33 creates the celestial sphere image data by converting the captured image data (for example, data of each hemispherical image as shown in Fig. 3(A) and (B)) into celestial sphere image data as shown in Fig. 4(B). Furthermore, the image / sound processing unit 33 outputs an audio signal related to the audio data distributed from another terminal via the communication management device 50 to the speaker 319, and causes the speaker 319 to output the audio.

[0063] The display control unit 34 is mainly realized by the processing of the CPU 301, and displays various images, characters, etc. on the display 306. The determination unit 35 is realized by the processing of the CPU 301, and performs various determinations. The determination unit 35 determines, for example, the image type related to the captured image data received from the imaging device 10.

[0064] The creation unit 36 ​​is mainly realized by the processing of the CPU 301, and creates a source name, which is an example of image type information, in accordance with a naming rule based on the result of the determination by the determination unit 35 of whether the image is a general image or a special image (here, a spherical image). For example, if the determination unit 35 determines that the image is a general image, the creation unit 36 ​​creates a source name "Video" indicating that it is a general image. On the other hand, if the determination unit 35 determines that the image is a special image, the creation unit 36 ​​creates a source name "Video_Omni" indicating that it is a special image.

[0065] The communication unit 37 is mainly realized by processing of the CPU 301 for the short-range communication circuit 320, and communicates with the communication unit 11 of the photographing device 10 using a short-range wireless communication technology such as NFC, Bluetooth, or WiFi. Note that although the communication unit 37 and the transmission / reception unit 31 have been described as having separate communication units, they may be configured to share a common communication unit.

[0066] The storage / reading unit 39 is realized mainly by the processing of the CPU 301 , and stores various data (or information) in the storage unit 3000 and reads various data (or information) from the storage unit 3000 .

[0067] (Radiation device management table) Fig. 15(A) is a conceptual diagram showing an example of a photographing device management table. A photographing device management DB 3001 configured by the photographing device management table shown in Fig. 15(A) is constructed in the storage unit 3000. In this photographing device management table, the vendor ID and product ID of the GUID of the photographing device capable of obtaining two hemispherical images that are the basis of the omnidirectional image are stored and managed. As the GUID, for example, a vendor ID (VID) and a product ID (PID) used in a USB device can be used. The vendor ID and the product ID may be stored, for example, at the time of shipment from the factory of the distribution terminal 30, or may be additionally stored after shipment from the factory.

[0068] (Image type management table) FIG. 15B is a conceptual diagram showing an example of an image type management table. In the storage unit 3000, an image type management DB 3002 configured by the image type management table shown in FIG. 15B is constructed. In this image type management table, an IP (Internet Protocol) address of a photographing device, which is an example of a destination of the photographing device, and a source name (image type information) are managed in association with each image data ID. Among them, the image data ID is an example of image data identification information for identifying image data when performing image distribution. The IP address of the photographing device indicates the IP address of the photographing device 10 that photographed the image data indicated by the associated image data ID. The source name is a name for identifying the photographing device 10 that photographed the image data indicated by the associated image data ID, and is an example of image type information. This source name is a name created by the distribution terminal 30 according to a predetermined naming rule.

[0069] For example, four image capture devices, whose IP addresses are "2.1.2.3", "2.1.1.5", "2.1.5.4", and "2.1.5.6", respectively, are transmitting image data indicated by image data IDs "RS001", "RS002", "RS003", and "RS004". Furthermore, the types of images indicated by the source names of the image capture devices are "Video_Omni", "Video_Omni", "Video", and "Video", which indicate that the image types of the captured images are "special image", "special image", "general image", and "general image", respectively. In this embodiment, the special image is a spherical image. Note that the IP address is an example of destination information, and the destination information may be a MAC (Media Access Control) address or a terminal ID (Identification), etc. Furthermore, the IP address is represented by simplifying the IPv4 address, but the IP address may be an IPv6 address. Furthermore, data other than image data may also be managed in association with the image data ID. Data other than image data may be, for example, sound data, material data when sharing a screen between a distribution base and a viewing base, etc.

[0070] <Functional configuration of the communication management device> Next, the functional configuration of the communication management device 50 will be described with reference to Fig. 14. The communication management device 50 has a transmission / reception unit 51, a selection unit 53, a determination unit 55, a generation unit 56, a distribution base management unit 57, and a storage / readout unit 59. Each of these units is a function or means realized by operating any of the components shown in Fig. 12 by an instruction from the CPU 501 in accordance with the program for the communication management device expanded from the HD 504 onto the RAM 503. The communication management device 50 also has a storage unit 5000 constructed by the ROM 502, RAM 503, and HD 504 shown in Fig. 12.

[0071] The transmission / reception unit 51 is realized mainly by the processing of the CPU 501 for the network I / F 509, and transmits and receives various data or information to and from other devices via the communication network 100.

[0072] The selection unit 53 is mainly realized by the processing of the CPU 501, and selects a specific image capture device 10 from the multiple image capture devices 10. The selection unit 53 selects a specific image capture device 10 from the multiple image capture devices 10 based on, for example, coordinate information of the attention point P accepted by the communication terminal 70, arrangement position information of the multiple image capture devices 10, and division number information indicating the division number of the area accepted by the communication terminal 70. The image capture device 10 selected at this time is particularly called a complementary image capture device. Note that, in this embodiment, the complementary image capture device functions or has a means that functions as an example of a second image capture device. Also, in this embodiment, the selection unit 53 functions or has a means that functions as an example of a selection means.

[0073] The determination unit 55 performs various determinations and is mainly realized by the processing of the CPU 501. The selection unit 53 selects the image capturing device 10 in response to a request from the communication terminal .

[0074] The generating unit 56 is mainly realized by the processing of the CPU 501, and generates image data ID and predetermined area information. The generating unit 56 generates predetermined area information indicating, for example, a predetermined area (for example, predetermined area T shown in FIG. 5, etc.) in a captured image captured by the imaging device 10 selected by the selecting unit 53. Note that an image in which the entire captured image is displayed (for example, the omnidirectional image CE shown in FIG. 5, etc.) is also referred to as an "entire image." The distribution base management unit 57 is mainly realized by the processing of the CPU 501, and manages distribution base information indicating the status of the distribution base.

[0075] The storage / reading unit 59 is realized mainly by the processing of the CPU 501 , and stores various data (or information) in the storage unit 5000 and reads various data (or information) from the storage unit 5000 .

[0076] (Session management table) FIG. 16(A) is a conceptual diagram showing an example of a session management table. A session management DB 5001 configured by the session management table shown in FIG. 16(A) is constructed in the storage unit 5000. In this session management table, a base ID and an IP address of a participating communication terminal are associated and managed for each session ID. Among them, the session ID is an example of session identification information for identifying a communication session that realizes image communication, and is generated for each virtual floor. The session ID is also managed by each communication terminal, and is used when selecting a communication session in each communication terminal. The base ID is an example of base identification information for identifying a distribution base. The IP address of the participating communication terminal indicates the IP address of the communication terminal that has participated in the virtual floor indicated by the associated session ID.

[0077] (Image type management table) FIG. 16(B) is a conceptual diagram showing an example of an image type management table. In the storage unit 5000, an image type management DB 5002 configured by the image type management table shown in FIG. 16(B) is constructed. In this image type management table, in addition to each piece of information managed in the image type management table shown in FIG. 15(B), a session ID that is the same as the session ID managed in the session management table is managed in association with each other. Note that the communication management device 50 manages the same image data ID, the IP address of the photographing device, and the image type information managed in the distribution terminal 30 and the communication terminal 70 in order to transmit image type information, etc. to a communication terminal already in an image call and a newly participating communication terminal when a new communication terminal enters the virtual floor, etc. This eliminates the need to transmit and receive image type information, etc. between a communication terminal already in a call and a newly participating communication terminal.

[0078] (Prescribed area management table) Fig. 17(A) is a conceptual diagram showing an example of a predetermined area management table. A predetermined area management DB 5003 configured by the predetermined area management table shown in Fig. 17(A) is constructed in the storage unit 5000. In this predetermined area management table, the IP address of the communication terminal to which the captured image data is to be transmitted and predetermined area information indicating parameters of the predetermined area image being displayed on the communication terminal to which the captured image data is to be transmitted are managed in association with each other for each IP address of the photographing device (image transmission source). As shown in Figs. 6 and 7, the predetermined area information is a conversion parameter for converting the photographed image into an image of a predetermined area T in the photographed image (predetermined area image).

[0079] For example, the first row of the predetermined area management table in Fig. 17(A) manages that when the IP address of the photographing device 10 is "2.1.2.3", after the streaming terminal 30 receives the photographed image data photographed by the photographing device 10, the photographed image data transmitted from the streaming terminal 30 is transmitted to the communication terminal 70 with the IP address "1.2.1.3" via the communication management device 50. Furthermore, the streaming terminal 30 manages that it is the source of the four pieces of predetermined area information in the first to fourth rows of the predetermined area management table in Fig. 17(A).

[0080] In addition, when the transmission / reception unit 51 receives new specified area information that includes the same set of IP addresses as the IP address of the shooting device that is the source of the captured image data that is already managed and the IP address of the communication terminal to which the captured image data is to be sent, the storage / reading unit 59 overwrites the specified area information already managed with the newly received specified area information.

[0081] (Layout information management table) FIG. 17(B) is a conceptual diagram showing an example of the arrangement information management table. In the storage unit 5000, an arrangement information management DB 5004 configured by the arrangement information management table shown in FIG. 17(B) is constructed. In this arrangement information management table, for each base ID for identifying a distribution base, the IP address of the camera device 10 arranged at the distribution base, the coordinate value indicating the position where the camera device 10 is arranged, and the arrangement direction of the camera device 10 are associated and managed. Among these, the coordinate value indicates the coordinate value on the map of the distribution base where the camera device 10 is arranged. Also, the arrangement direction of the camera device 10 indicates the direction (angle) on the map to which the front of the camera device 10 faces. Note that this arrangement information is set to any direction (angle) by the administrator of the distribution base or the distribution terminal 30.

[0082] (Distribution location management table) FIG. 18 is a conceptual diagram showing an example of a distribution base management table. A distribution base management DB 5005 configured by the distribution base management table shown in FIG. 18 is constructed in the storage unit 5000. In this distribution base management table, distribution base information indicating a base name and a status of the distribution base is managed in association with each base ID for identifying a distribution base. Among them, the distribution base information includes a URL (Uniform Resource Locator) for accessing map image data indicating a map of the distribution base and base coordinate information indicating the coordinate value of the distribution base. The communication terminal 70 can obtain the map image data and base coordinate information of the distribution base by accessing the URL transmitted from the communication management device 50. The URL is an example of storage destination information. The storage destination information is not limited to a URL, and may be a URI (Uniform Resource Identifier) ​​or the like.

[0083] (Weighting coefficient management table) FIG. 19(A) is a conceptual diagram showing an example of a weighting coefficient management table. In the storage unit 5000, a weighting coefficient management DB 5006 configured by the weighting coefficient management table shown in FIG. 19(A) is constructed. In the weighting coefficient management table, a supplementary photographing device name and a coefficient are associated and managed for each reference photographing device name indicating the name of the reference photographing device. Here, the reference photographing device indicates a photographing device located at a position closest to the attention point P (predetermined position) specified by the user. In addition, the supplementary photographing device indicates a photographing device that can most multilaterally photograph a part that cannot be photographed by the reference photographing device with respect to the attention point P. Furthermore, the coefficient is a value indicating the arrangement relationship between the reference photographing device and the supplementary photographing device, and an arbitrary value is set in advance by an administrator who places the photographing device 10 at the distribution base A or the like or an administrator who manages the communication management device 50.

[0084] (Specific weighting factor management table) FIG. 19(B) is a conceptual diagram showing an example of a specific weighting coefficient management table. In the storage unit 5000, a specific weighting coefficient management DB 5007 configured by the specific weighting coefficient management table shown in FIG. 19(B) is constructed. In the specific weighting coefficient management table, a supplementary imaging device name indicating the name of a complementary imaging device and a specific coefficient are associated and managed for each reference imaging device name indicating the name of the reference imaging device. Among these, the specific coefficient is given as a value larger than other coefficients when at least one of the following conditions is satisfied, such as when an obstacle (wall, etc.) exists between the reference imaging device and the complementary imaging device, and when there is some abnormality in the complementary imaging device, with respect to the coefficients of the weighting coefficient management table shown in FIG. 19(A). The specific coefficient is also set to an arbitrary value in advance by an administrator who places the imaging device 10 at the distribution base A, etc., or an administrator who manages the communication management device 50.

[0085] <Functional configuration of communication terminal> Next, the functional configuration of the communication terminal 70 will be described with reference to Fig. 14. The communication terminal 70 has a transmission / reception unit (acquisition unit) 71, a reception unit 72, an image / sound processing unit 73, a display control unit 74, a judgment unit 75, a creation unit 76, a focus point identification unit 77, a selection unit 81, a generation unit 82, and a storage / readout unit 79. Each of these units is a function or means realized by operating any of the components shown in Fig. 12 by an instruction from the CPU 701 according to the program for the communication terminal expanded from the HD 704 onto the RAM 703. The communication terminal 70 also has a storage unit 7000 constructed by the ROM 702, RAM 703, and HD 704 shown in Fig. 12.

[0086] The transmitting / receiving unit (acquisition unit) 71 is mainly realized by processing of the CPU 701 on the network I / F 709, and transmits and receives various data or information to and from other devices or terminals via the communication network 100. The transmitting / receiving unit (acquisition unit) 71 receives, for example, captured image data distributed from the distribution terminal 30 via the communication management device 50. The transmitting / receiving unit (acquisition unit) 71 also functions as, for example, an acquisition unit, and acquires predetermined area information indicating a predetermined area including the attention point P in an image captured by a specific image capturing device 10 selected based on the coordinate information (predetermined position information) of the attention point P accepted by the acceptance unit 72 and the placement position (placement position information) of the image capturing device 10.

[0087] The reception unit 72 is mainly realized by processing of the CPU 701 on the keyboard 711 or the pointing device 712, and receives various selections or inputs from the user. The reception unit 72 receives, for example, an input of a point of interest P at a distribution base designated by the user, and a number of divisions for dividing a predetermined range (for example, a circle with a radius r) centered on the point of interest P, with the point of interest P as a boundary. In this embodiment, the reception unit 72 functions or has a means for functioning as an example of a reception means.

[0088] The image / sound processing unit 73 is mainly realized by the processing of the CPU 701, and performs image processing on the captured image data distributed from the distribution terminal 30. In addition, the image / sound processing unit 73 performs audio processing on the audio data distributed from the distribution terminal 30. For example, in order to display an image of a predetermined area corresponding to the predetermined area information received by the transmission / reception unit (acquisition unit) 71 on the display 706, the image / sound processing unit 73 performs perspective projection transformation using the predetermined area information on the captured image (omnidirectional image) to generate a predetermined area image corresponding to the predetermined area information. Furthermore, the image / sound processing unit 73 outputs an audio signal related to the audio data distributed from the distribution terminal 30 via the communication management device 50 to the speaker 719, and causes the speaker 719 to output audio.

[0089] The display control unit 74 is mainly realized by the processing of the CPU 701, and causes various images, characters, etc. to be displayed on the display 706. The display control unit 74 causes the display 706 to display, for example, a predetermined area image generated by the image / sound processing unit 73.

[0090] The determination unit 75 performs various determinations and is realized by processing of the CPU 701. In this embodiment, the display control unit 74 functions as an example of a display control unit or has a unit that functions as such.

[0091] The creation unit 76 is mainly realized by the processing of the CPU 701, and realizes the same function as the creation unit 36. The attention point identification unit 77 is mainly realized by the processing of the CPU 701, and identifies an attention point P at a distribution base where the image capturing device 10 is installed. The attention point identification unit 77 identifies, for example, coordinate information of the attention point P on a map image of the distribution base specified by the user.

[0092] The selection unit 81 is mainly realized by the processing of the CPU 701, and realizes the same function as the selection unit 53. The generation unit 82 is mainly realized by the processing of the CPU 701, and generates predetermined area information. For example, the generation unit 82 generates predetermined area information indicating a predetermined area in a captured image captured by the image capture device 10 selected by the selection unit 81. The generation unit 82 also functions as, for example, an acquisition unit, and acquires predetermined area information indicating a predetermined area including the attention point P in an image captured by a specific image capture device 10 selected based on the coordinate information of the attention point P accepted by the acceptance unit 72 and the arrangement position of the image capture device 10.

[0093] The storage / reading unit 79 is realized mainly by the processing of the CPU 701 , and stores various data (or information) in the storage unit 7000 and reads various data (or information) from the storage unit 7000 .

[0094] Image type management table Fig. 20(A) is a conceptual diagram showing an example of an image type management table. An image type management DB 7001 configured by the image type management table shown in Fig. 20(A) is constructed in the storage unit 7000. This image type management table has the same data configuration as the image type management table shown in Fig. 15(B), so a description thereof will be omitted.

[0095] Predetermined area management table Fig. 20(B) is a conceptual diagram showing an example of a predetermined area management table. A predetermined area management DB 7002 configured by the predetermined area management table shown in Fig. 20(B) is constructed in the storage unit 7000. In this predetermined area management table, predetermined area information indicating the predetermined area image being displayed is associated and managed for each IP address of the photographing device that photographed the photographed image data. As shown in Figs. 6 and 7, the predetermined area information is a conversion parameter for converting the photographed image into an image of a predetermined area T in the photographed image (predetermined area image).

[0096] Next, the relationship between the attention point P and the image capturing devices when a plurality of image capturing devices 10 are arranged at the distribution base A will be described with reference to FIGS.

[0097] <Relationship between attention point P and imaging device when the number of divisions is 2> FIG. 21 is a diagram for explaining an example of the relationship between the attention point P and the photographing devices when the number of divisions is 2. For example, in a distribution base A (area A) in the image communication system 1 shown in FIG. 10, a plurality of photographing devices 10A are regularly arranged in a lattice pattern as shown in FIG. 21. This arrangement state is generated as a two-dimensional arrangement image (map image) by the generating unit 56, for example, based on the coordinate values ​​of each photographing device 10. At this time, a user who uses the communication terminal 70 operates a "attention point" button 605 indicating a predetermined position and a "number of divisions" button 670 for dividing a predetermined range (for example, a circle with a radius r) centered on the attention point P with the attention point P as a boundary. At this time, the user can operate the "attention point" button 605 by referring to a schematic image 650 corresponding to the two-dimensional arrangement image displayed on a base display screen 600 of the communication terminal 70 described later. Furthermore, the selection unit 53 selects the image capture device 10A-1 that is located closest to the attention point P from among the image capture devices 10A-1, 10A-2, 10A-3, and 10A-4 that are located within the circle of radius r.

[0098] Next, the selection unit 53 extends a line segment virtually connecting the attention point P (predetermined position) and the image capturing device 10A-1, and divides the two-dimensional arrangement image generated by the generation unit 56 into two regions. This is an example of the two-dimensional arrangement image obtained in this manner when the division number is set to 2.

[0099] <Selection process of imaging device when the number of divisions is 2> FIG. 22 is a diagram for roughly explaining an example of a selection process based on the angle between the camera devices when the division number is 2. As shown in the example of FIG. 22, the angle (azimuth angle) between the camera device 10A-1 and the camera device 10A-2 is 101°. The angle between the camera device 10A-1 and the camera device 10A-3 is 121°. The angle between the camera device 10A-1 and the camera device 10A-4 is 163°. The distribution point A (area A) is divided into two by a thick dotted line that is an extension of a line segment virtually connecting the attention point P and the camera device 10A-1.

[0100] <Relationship between attention point P and imaging device when the number of divisions is 3> FIG. 23 is a diagram for roughly explaining an example of the relationship between the attention point P and the photographing devices when the number of divisions is 3. For the sake of simplicity, it is assumed that the arrangement of the photographing devices 10 at the distribution base A shown in FIG. 23 is the same as that in FIG. 21. In this state, when the user of the communication terminal 70 operates the "attention point" button 605 and the "number of divisions" button 670, the selection unit 53 extends a line segment virtually connecting the attention point P (predetermined position) and the photographing device 10A-1, and divides the two-dimensional arrangement image generated by the generation unit 56 into three regions separated by 120°. The two-dimensional arrangement image obtained in this manner is an example when the number of divisions is set to 3.

[0101] <Selection process of imaging device when the number of divisions is 3> Fig. 24 is a diagram for roughly explaining an example of a selection process based on the angle between the camera devices when the division number is 3. The arrangement of the camera devices shown in Fig. 24 is the same as that of the camera devices shown in Fig. 22. In Fig. 24, distribution base A (area A) is divided into three by thick dotted lines each forming an angle of 120° with respect to a line segment (thick dotted line) virtually connecting attention point P and camera device 10A-1.

[0102] <Selection process of the imaging device when the division number is 3 (when there is an obstacle)> Fig. 25 is a diagram for roughly explaining an example of the selection process based on the angle between the image capturing devices when the number of divisions is 3. In the example of Fig. 25, a case is shown in which a wall exists between the image capturing device 10A-3 and the attention point P.

[0103] <Relationship between attention point P and imaging device when the number of divisions is 4> FIG. 26 is a diagram for roughly explaining an example of the relationship between the attention point P and the photographing devices when the number of divisions is 4. The case of FIG. 26 is also assumed to be the same as the arrangement of the photographing devices 10 at the distribution base A shown in FIG. 21 and FIG. 23. In this state, when the user of the communication terminal 70 operates the "attention point" button 605 and the "number of divisions" button 670, the selection unit 53 extends the line virtually connecting the attention point P (predetermined position) and the photographing device 10A-1, and divides the two-dimensional arrangement image generated by the generation unit 56 into four areas divided by 90°. The two-dimensional arrangement image obtained in this way is an example when the number of divisions is set to 4.

[0104] On the other hand, Fig. 27 is a diagram for roughly explaining an example of the relationship between the camera devices arranged at irregular positions and the points of interest. As shown in Fig. 27, the camera devices 10 arranged at the distribution base A may be arranged at irregular positions relative to each other. Even in this case, the selection unit 53 can divide the area into a given number of divisions based on the positional relationship between the camera devices and the points of interest within the area of ​​radius r according to the number of divisions 2, 3, 4, etc. given by the user.

[0105] [Processing or Operation of the Embodiment] <Session Join Processing> Next, the process or operation of the image communication system according to the embodiment will be described with reference to Figs. 28 to 36. In the following description, an example will be described in which an image captured at distribution point A is distributed to the communication terminal 70, but similar processes are performed in image distribution from other distribution points such as distribution point B. First, a process of joining a specific communication session will be described with reference to Figs. 28 and 29. Fig. 28 is a sequence diagram showing an example of a process of joining a specific communication session in the image communication system. Fig. 29 is a diagram showing an example of a selection screen for a communication session.

[0106] First, a user at the viewing location A (e.g., user A1) performs an operation to display a selection screen for a communication session, and the reception unit 72 receives the operation to display the selection screen, whereby the display control unit 74 of the communication terminal 70C displays the selection screen 800 shown in Fig. 29 on the display 706 (step S11). The selection screen 800 shown in Fig. 29 displays selection buttons 810a, 810b, 810c, etc., which indicate each of the floors A1, B1, B2, etc., to be selected. Each selection button 810a, etc. is associated with a session ID.

[0107] Here, when user A1 selects the desired selection button (here, selection button 810a) of the virtual floor that is the distribution base, the reception unit 72 accepts the selection of the communication session (step S12). Then, the transmission / reception unit (acquisition unit) 71 transmits a participation request to the communication management device 50 for the communication session with the distribution base (step S13). As a result, the transmission / reception unit 51 of the communication management device 50 receives the participation request. This participation request includes a session ID indicating the communication session whose selection was accepted in step S12, and the IP address of the communication terminal 70C that is the request source terminal.

[0108] Next, the storage / reading unit 59 of the communication management device 50 performs a process of participating in the communication session by adding the IP address received in step S13 to the participation terminal IP address field of the record of the same session ID as the session ID received in step S13 in the session management table (session management DB 5001, see FIG. 16(A)) (step S14). The storage / reading unit 59 also reads out the base ID associated with the session ID of the communication session for which the participation process has been performed in the session management DB 5001. Next, the storage / reading unit 59 searches the distribution base management table (distribution base management DB 5005, see FIG. 18) using the base ID read out in step S14 as a search key, thereby reading out distribution base information associated with the same base ID as the read out base ID (step S15). Then, the transmission / reception unit 51 transmits a participation request response to the communication terminal 70C (step S16). As a result, the transmission / reception unit (acquisition unit) 71 of the communication terminal 70C receives the participation request response. This participation request response includes the session ID received in step S13, the distribution base information read in step S15, and the participation processing result. The participation processing result also includes the base ID read in step S14. The following describes a case where the participation processing is successful. Note that the communication terminal 70D at the viewing base B performs the communication session participation processing by the same processing as that shown in Fig. 28.

[0109] <Image type information management process> Next, the management processing of image type information will be described with reference to Fig. 30. Fig. 30 is a sequence diagram showing the management processing of image type information in the image communication system.

[0110] First, when a user at the distribution site A connects the photographing device 10A to the distribution terminal 30A, the storage / readout unit 19 of the photographing device 10A reads out the GUID of its own device (the photographing device 10A) stored in the storage unit 1000. Then, the communication unit 11 of the photographing device 10A transmits the GUID of its own device to the distribution terminal 30A (step S31). As a result, the communication unit 37 of the distribution terminal 30A receives the GUID of the photographing device 10A.

[0111] Next, the determination unit 35 of the distribution terminal 30A determines whether the same vendor ID and product ID as the vendor ID and product ID in the GUID received in step S31 are managed in the photographing device management table (photographing device management DB3001, see FIG. 15(A)), thereby determining the image type (step S32). Specifically, if the same vendor ID and product ID are managed in the photographing device management DB3001, the determination unit 35 determines that the photographing device 10A is a photographing device that captures special images (here, spherical images). On the other hand, if the same vendor ID and product ID are not managed in the photographing device management table (photographing device management DB3001, see FIG. 15(A)), the determination unit 35 determines that the photographing device 10A is a photographing device that captures general images.

[0112] Next, the storage / reading unit 39 stores the IP address of the photographing device 10A in an image type management table (image type management DB3002, see FIG. 15(B)) in association with the image type information determined in step S32 (step S33). In this state, no image data ID is associated. The image type information is, for example, a source name or image type (general image, special image) determined according to a predetermined naming rule.

[0113] Next, the transmitting / receiving unit 31 transmits a request to add image type information to the communication management device 50 (step S34). This request to add image type information includes the IP address of the imaging device 10A, the location ID of the distribution location A, and the image type information stored in step S33. As a result, the transmitting / receiving unit 51 of the communication management device 50 receives the request to add image type information.

[0114] Next, the memory / read unit 59 of the communication management device 50 searches the session management table (session management DB 5001, see FIG. 16(A)) using the location ID received in step S34 as a search key, and reads out the corresponding session ID (step S35).

[0115] Next, the generating unit 56 generates a unique image data ID (step S36). Then, the storing / reading unit 59 stores the session ID read out in step S35, the image data ID generated in step S36, and the IP address and image type information of the photographing device 10A received in step S34 in association with each other as a new record in the image type management table (image type management DB 5002, see FIG. 16(B)) (step S37). Then, the transmitting / receiving unit 51 transmits the image data ID generated in step S36 to the distribution terminal 30A (step S38). As a result, the transmitting / receiving unit 31 of the distribution terminal 30A receives the image data ID.

[0116] Next, the memory / read unit 39 of the distribution terminal 30A stores the image data ID received in step S38 in an image type management table (image type management DB3002, see FIG. 15(B)) in association with the IP address and image type information of the photographing device 10A stored in step S33 (step S39).

[0117] Meanwhile, the transmitting / receiving unit 51 of the communication management device 50 transmits an addition notification of the image type information to the communication terminal 70C (step S40). As a result, the transmitting / receiving unit (acquisition unit) 71 of the communication terminal 70C receives the addition notification of the image type information. This addition notification of the image type information includes the image data ID generated in step S36, and the IP address and image type information of the imaging device 10A stored in step S37.

[0118] Next, the storage / readout unit 79 of the communications terminal 70C associates and stores the image data ID, the IP address of the photographing device 10A, and the image type information received in step S40 as a new record in the image type management table (image type management DB 7001, see FIG. 20(A)) (step S41). As described above, the distribution terminal 30A and the communications terminal 70C can share the same information in the image type management DB 3002 and the image type management DB 7001. Similarly, an additional communication of the image type information is transmitted to another communications terminal, communications terminal 70D, and the image type information is also stored in the image type management DB 7001 of communications terminal 70D.

[0119] <Communication processing of captured image data> Next, a process in which photographed image data and sound data obtained at the distribution base A are transmitted to each communication terminal (communication terminals 70C, 70D) via the communication management device 50 will be described with reference to Fig. 31. Fig. 31 is a sequence diagram showing an example of communication processing of photographed image data and sound data in an image communication system. Note that Fig. 31 shows an example in which photographed image data obtained by one photographing device 10 is distributed to each communication terminal 70, but the same process is also performed in the case where multiple photographed image data obtained by other photographing devices 10 arranged at the distribution base are distributed.

[0120] First, the communication unit 11 of the photographing device 10A transmits photographed image data obtained by photographing a subject, scenery, or the like, and sound data obtained by collecting sound to the distribution terminal 30A (step S51). As a result, the communication unit 37 of the distribution terminal 30A receives the photographed image data and sound data. In this case, since the photographing device 10A is a device that can obtain two hemispherical images that are the basis of a celestial sphere image, the photographed image data is composed of data of two hemispherical images, as shown in Figs. 3(A) and (B).

[0121] Next, the transmitting / receiving unit 31 of the distribution terminal 30A transmits the photographed image data and sound data sent from the photographing device 10A to the communication management device 50 (step S52). As a result, the transmitting / receiving unit 51 of the communication management device 50 receives the photographed image data, sound data, and image data ID. This transmission and reception includes an image data ID for identifying the photographed image data to be transmitted and received.

[0122] Next, the transmitting / receiving unit 51 of the communication management device 50 transmits the captured image data and the sound data to the communication terminals (communication terminals 70C, 70D) participating in the same session as the distribution terminal 30A (steps S53, S54). As a result, the transmitting / receiving units (acquisition units) 71 of the communication terminals 70C and 70D receive the captured image data and the sound data, respectively. Each of these transmissions and receptions includes an image data ID for identifying the captured image data to be transmitted and received.

[0123] <Display process of attention point P> Next, a process for displaying an image of a point of interest P at a distribution base designated by a user at a viewing base will be described with reference to Fig. 32 to Fig. 36. Fig. 32 is a sequence diagram showing an example of a display process for a point of interest P in an image communication system. Fig. 32 describes an example in which a communication terminal 70C at a viewing base A displays a captured image distributed from a distribution terminal 30A. Note that a similar process is performed when a captured image is displayed on a communication terminal 70D at another viewing base, viewing base B.

[0124] First, the display control unit 74 of the communication terminal 70C uses the distribution base information received in step S16 to display a base display screen 600 showing the status of the distribution base A on the display 706 (step S71). Figure 34 is a diagram showing an example of the base display screen displayed on the communication terminal. The base display screen 600 shown in Figure 34 displays a map image showing the status of the distribution bases.

[0125] The base display screen 600 shown in FIG. 34 includes a tour image 610 for remotely viewing the distribution base by viewing a celestial sphere image of the distribution base, a viewpoint change icon 630 for changing the viewpoint to a predetermined shooting point of the tour image 610, a cross (or a star mark, etc.) 635 indicating a point of interest P, a schematic image 650 showing a schematic diagram of the distribution base, a "point of interest" button 605 operated to display the point of interest P in the distribution base, a "cancel" button 609 operated to end viewing of the distribution base, and a "number of divisions" button 670 for dividing the area of ​​the distribution base with the point of interest P at the distribution base as the center (the point of interest P as the boundary). Furthermore, by operating the "number of divisions" button 670, a number of divisions setting screen 675 is displayed near the "number of divisions" button 670. An input section for inputting the number of divisions is displayed next to "number of divisions=" on the number of divisions setting screen, and the user inputs a desired value (e.g., 2, 3, etc.) into this input section and operates the "OK" button. On the other hand, if the user wishes to change or cancel the input value, the user can operate the "cancel" button.

[0126] Among these, the tour image 610 and the schematic image 650 are displayed using map image data included in the distribution base information received in step S16. The tour image 610 is an image of the distribution base photographed in advance by the photographing device 10, and the user A1 can grasp the layout of the entire distribution bases by operating the viewpoint change icon 630. Note that, although the tour image 610 is described as being displayed using map image data included in the distribution base information received in step S16, the tour image 610 may be configured to display photographed image data of the distribution base received in real time.

[0127] Moreover, the schematic image 650 includes a viewpoint position icon 655 indicating the current display position of the tour image 610. The user A1 can view a desired location in the distribution base by, for example, operating the viewpoint change icon 630 using the pointer p1 while looking at the tour image 610 and the schematic image 650.

[0128] Next, the user A1 operates the viewpoint change icon 630 using the pointer p1, and operates the "attention point" button 605, whereby the reception unit 72 receives the designation (input) of the attention point P (step S72). Note that the reception unit 72 may receive the designation of the attention point P by the user A1 clicking or double-clicking the mouse, which is an example of an input means.

[0129] Furthermore, when the user A1 operates the "number of divisions" button 670 by operating the pointing device 712, the display control unit 74 displays a number of divisions setting screen 675 on the base display screen 600. As a result, when the user A1 inputs a desired number of divisions n (e.g., 2, 3, etc.) and operates the OK button, the reception unit 72 receives the input of the number of divisions (step S73).

[0130] Next, the attention point identification unit 77 identifies attention point coordinates, which are the coordinates of the attention point P whose designation was accepted in step S72 (step S74). Here, the attention point coordinates identified by the attention point identification unit 77 are the coordinates of the center point of the tour image 610 that is displayed when the designation of the attention point P was accepted in step S72. Note that the attention point coordinates may be coordinates indicating the position of the pointer p1 in the tour image 610.

[0131] The transmitting / receiving unit (acquiring unit) 71 transmits an imaging device selection request indicating a request to select an imaging device 10 to the communication management device 50 (step S75). As a result, the transmitting / receiving unit 51 of the communication management device 50 receives the imaging device selection request transmitted from the communication terminal 70. This imaging device selection request includes the point of interest coordinates identified in step S74 and the base ID received in step S16.

[0132] <Selection of imaging modality> Next, the communication management device 50 executes a selection process of the photographing device 10 in response to the photographing device selection request received in step S74 (step S76). Here, the process of step S76 will be described in detail with reference to Fig. 33 and Fig. 21 to Fig. 26. Fig. 33 is a flow chart showing an example of the photographing device selection process.

[0133] First, the storage / reading unit 59 searches the distribution base management table (distribution base management DB 5005, see FIG. 18) using the base ID received in step S74 as a search key, and reads out distribution base information associated with the same base ID as the received base ID (step S101).

[0134] Next, the selection unit 53 divides the distribution base into multiple areas (step S102) with the attention point P (predetermined position) as the center (at the attention point P as the boundary) based on the attention point coordinates received in step S74 and the division number n accepted in step S73. Specifically, the selection unit 53 divides the entire area of ​​the distribution base into two areas based on the base coordinate information included in the distribution base information read in step S101 and the division number information (division number n) accepted in step S73, starting from the received attention point coordinates. In the example shown in Fig. 21, the attention point P specified by the user A1 is indicated by a black circle (●).

[0135] Next, the storage / reading unit 59 searches the placement information management table (placement information management DB 5004, see FIG. 17(B)) using the location ID received in step S74 as a search key, and reads out placement information associated with the same location ID as the received location ID (step S103).

[0136] Next, the selection unit 53 selects the camera devices 10 arranged around the attention point coordinates indicating the coordinates of the attention point P based on the coordinate values ​​indicated by the arrangement information read out in step S103 (step S104). Specifically, the selection unit 53 extends a line segment virtually connecting the attention point P (predetermined position) and the camera device 10A-1 to divide the two-dimensional arrangement image generated by the generation unit 56 into two regions. Note that, when drawing a circle with a radius r, the selection unit 53 may set the value of the radius r so that the circle includes at least two or more camera devices 10 relatively close to the attention point P. In setting the value of the radius r, the selection unit 53 can calculate the distance to the attention point P from the coordinate values ​​of the attention point P and the camera devices 10 arranged around it. Next, the selection unit 53 selects the camera devices 10A-2, 10A-3, and 10A-4, including the camera device 10A-1, as the camera devices 10 within the range arranged around the attention point coordinates indicating the coordinates of the attention point P.

[0137] In this embodiment, the image capturing device 10A-1 arranged at a position closest to the attention point P selected in the processing of step S104 is called the reference image capturing device. Here, the reference image capturing device 10A-1 functions or has a means for functioning as an example of a first image capturing device.

[0138] Next, the selection unit 53 executes a process of selecting a complementary photographing device capable of photographing a portion of the subject at the attention point P that cannot be sufficiently photographed by the reference photographing device 10A-1 from among the in-range photographing devices 10 (step S105). Here, the complementary photographing device functions or has a means for functioning as an example of a second photographing device.

[0139] <Selection process of supplementary imaging device> FIG. 35 is a flowchart showing an example of the selection process of the complementary photographing device. First, the selection unit 53 draws a predetermined range (for example, a circle with a radius r) centered on the attention point P. Next, the selection unit 53 selects the photographing device 10A-1 arranged at the position closest to the attention point P among the photographing devices 10A-1, 10A-2, 10A-3, and 10A-4 arranged within the circle with a radius r (step S105-1). At this time, the selection unit 53 calculates the distance between each coordinate value of the photographing devices 10A-1, 10A-2, 10A-3, and 10A-4 and the attention point coordinate in the example of FIG. 21. Then, the selection unit 53 sets the photographing device 10A-1 arranged at the position closest to the coordinate of the attention point P (closest distance) as the closest reference photographing device 10A-1 within the predetermined range, and stores the identification information of the photographing device 10A-1 (for example, the IP address of the photographing device 10A-1) and the calculated distance in the storage unit 5000. When calculating the distance between each of the coordinate values ​​of the multiple image capture devices 10A and the coordinates of the point of interest, for example, a known method for finding the straight line distance connecting each of the coordinates is used.

[0140] Next, the selection unit 53 calculates the angle between the reference camera 10A-1 and other camera devices around the attention point P (step S105-2). Specifically, the selection unit 53 searches the arrangement information management table (arrangement information management DB 5004, see FIG. 17(B)) using the IP address of each camera device as a search key to read out the corresponding coordinate values, and calculates the angle between each camera device and the attention point P. As a result, in the example of FIG. 22 described above, the angle (azimuth angle) between the camera device 10A-1 and the camera device 10A-2 sandwiching the attention point P is 101°. Also, the angle between the camera device 10A-1 and the camera device 10A-3 sandwiching the attention point P is 121°. Furthermore, the angle between the camera device 10A-1 and the camera device 10A-4 sandwiching the attention point P is 163°. The selection unit 53 stores these calculated angles in the storage unit 5000 as well as the distances calculated in the process of step S105-2. The selection unit 53 can obtain these angles by, for example, using an inverse trigonometric function for each coordinate value. Also, the distribution base A (area A) is divided into two by a thick dotted line that is an extension of a line segment that virtually connects the attention point P and the image capture device 10A-1.

[0141] After calculating the angle between the respective photographing devices in step S105-2, the selection unit 53 selects the following photographing devices as candidates for complementary photographing devices (second photographing devices) that complement the portion that cannot be photographed by the reference photographing device (step S105-3). In particular, when the number of divisions is n (n is a natural number of 2 or more), the selection unit 53 selects, as a candidate for the complementary photographing device (second photographing device), a photographing device that is arranged such that the angle between the reference photographing device (first photographing device) and the point of interest P is closest to the angle obtained by dividing 360° by the number of divisions n.

[0142] ●When the number of divisions is 2● Specifically, as shown in FIG. 22, the selection unit 53 selects the camera closest to 360° / n, i.e., 360° / 2=180°, with respect to the reference camera 10A-1 as a candidate for a supplementary camera for the reference camera 10A-1. That is, the selection unit 53 selects the camera closest to 180° with respect to the reference camera 10A-1 as a candidate for a supplementary camera. In this case, the respective images captured by the reference camera 10A-1 and the supplementary camera 10A-4 can be captured to the maximum extent (multi-angled) of the front, rear, left and right of the image at the attention point P, for example, so that the selection unit 53 selects the camera 10A-4 as a candidate for a supplementary camera for the reference camera 10A-1. However, the selection unit 53 may add a slight margin of angle (θ) to 180° as a criterion for selecting a supplementary camera. In other words, the selection unit 53 may select, as a candidate for the complementary photographing device, a photographing device that is disposed at a position closest to 180°±θ with respect to the reference photographing device 10A-1.

[0143] A specific example of coefficient calculation using a data table and selection of a complementary imaging device will be described below. As described in a weighting coefficient management table (weighting coefficient management DB 5006, see FIG. 19(A)) and a specific weighting coefficient management table (specific weighting coefficient management DB 5007, see FIG. 19(B)), for convenience, the imaging device 10A-1 will be referred to as CAM1, the imaging device 10A-2 as CAM2, the imaging device 10A-3 as CAM3, and the imaging device 10A-4 as CAM4.

[0144] When four camera devices CAM1, CAM2, CAM3, and CAM4 are arranged within a circle of radius r centered on the attention point P shown in Fig. 22, the selection unit 53 selects the following combinations for selecting two from the four camera devices. At this time, the angles between the camera devices on either side of the attention point P are as follows: 1. When the selection unit 53 selects CAM1 and CAM2: Angle between CAM1 and CAM2 = 101° 2. When the selection unit 53 selects CAM1 and CAM3: Angle between CAM1 and CAM3 = 121° 3. When the selection unit 53 selects CAM1 and CAM4: Angle between CAM1 and CAM4 = 163° The optimal position for viewing (capturing) the attention point P from multiple angles with two camera devices is when the angle between the two camera devices sandwiching the attention point P is 180°. Therefore, the selection unit 53 calculates the absolute value of the difference between the angle between the two camera devices sandwiching the attention point P and 180°, and selects the combination with the smallest absolute value. In the case of the angles of the camera devices exemplified in FIG. 22, 1.|101 - 180|= 79 2.|121 - 180|= 59 3.|163 - 180|= 17 The combination of two image capturing devices that minimizes the absolute value of this difference is disposed at a position closest to 180°±θ as the angle between each image capturing device and the point of interest P. Therefore, the selection unit 53 selects CAM4 in case 3 as a candidate for a supplementary image capturing device for CAM1, which is the reference image capturing device.

[0145] In this state, the selection unit 53 performs coefficient calculations using at least one of the weighting coefficient management table (weighting coefficient management DB 5006, see FIG. 19(A)) and the specific weighting coefficient management table (specific weighting coefficient management DB 5006, see FIG. 19(B)) (step S105-4). That is, the selection unit 53 performs the following three calculations including the reference imaging device 10A-1 (CAM1) after selecting the imaging device 10A-4 (CAM4) as a candidate for a complementary imaging device. 1.|101 - 180|* 1.0= 79 2. |121 - 180|* 1.0 = 59 (However, in this case, since there is no obstacle such as a wall in front of CAM3, the coefficient in the data table is calculated as 1.0.) 3.|163 - 180|* 1.0= 17 Therefore, in the processing of step S105-4, the selection unit 53 determines the complementary photographing device for the photographing device 10A-1 (CAM1), which is the reference photographing device, to be the photographing device 10A-4 (CAM4), since this is the combination that produces the smallest calculation result in the case of case 3 where the angle between CAM1 and CAM4 is 163°, and then exits this flow.

[0146] ●When the number of divisions is 3● Next, a case where the division number=3 is given as shown in FIG. 23 will be described. The selection unit 53 selects the photographing device arranged at the position closest to 360° / n, i.e., 360° / 3=120°, with respect to the reference photographing device 10A-1 as a candidate for a complementary photographing device for the reference photographing device 10A-1 (step S105-3). However, the selection unit 53 may give a slight margin of accuracy (θ) to 120° as a criterion for selecting as a complementary photographing device. In other words, the selection unit 53 may select the photographing device arranged at the position closest to 120°±θ with respect to the reference photographing device 10A-1 as a candidate for a complementary photographing device. From the above, when the division number=3, the selection unit 53 selects two photographing devices as candidates for a complementary photographing device for the reference photographing device 10A-1.

[0147] Below, a specific example of coefficient calculation using a data table and selection of a complementary imaging device will be described. As in the case of the division number = 2, for convenience, the imaging device 10A-1 will be referred to as CAM1, the imaging device 10A-2 as CAM2, the imaging device 10A-3 as CAM3, and the imaging device 10A-4 as CAM4.

[0148] When four camera devices CAM1, CAM2, CAM3, and CAM4 are arranged within a circle of radius r centered on the attention point P shown in Fig. 23, the selection unit 53 selects the following combinations for selecting three from the four camera devices. At this time, the angles between the camera devices on either side of the attention point P are as follows: 1. When the selection unit 53 selects CAM1, CAM2, and CAM3: The angle between CAM1 and CAM2 = 101°, Angle between CAM2 and CAM3 = 138°, Angle between CAM3 and CAM1 = 121° 2. When the selection unit 53 selects CAM1, CAM2, and CAM4: The angle between CAM1 and CAM2 = 101°, Angle between CAM2 and CAM4 = 71°, Angle between CAM4 and CAM1 = 188° 3. When the selection unit 53 selects CAM1, CAM3, and CAM4: Angle between CAM1 and CAM3 = 121°, Angle between CAM3 and CAM4 = 67°, Angle between CAM4 and CAM1 = 172° It becomes.

[0149] The optimal position for viewing (capturing) the attention point P from multiple angles with the three camera devices is when the angle between the three camera devices sandwiching the attention point P is 120°. Therefore, the selection unit 53 calculates the absolute value of the difference between the angle between the three camera devices sandwiching the attention point P and 120°, and selects the combination with the smallest absolute value. In the case of the angles of the camera devices exemplified in FIG. 24, 1. |101 - 120| + |138 - 120| + |121 - 120| = 38 2. |101 - 120| + |71 - 120| + |188 - 120| = 136 3. |121 - 120| + |67 - 120| + |172 - 120| = 106 The combination of three imaging devices that minimizes the absolute value of this difference is disposed at a position closest to 120°±θ as the angle between each imaging device and the attention point P. Therefore, the selection unit 53 selects CAM2 and CAM3 in case 1 as candidates for the complementary imaging device for CAM1, which is the reference imaging device.

[0150] In this state, the selection unit 53 performs coefficient calculation using at least one of the data tables of the weighting coefficient management table (weighting coefficient management DB 5006, see FIG. 19(A)) and the specific weighting coefficient management table (specific weighting coefficient management DB 5006, see FIG. 19(B)) in the same manner as in the case where the number of divisions is 2 (step S105-4). That is, the selection unit 53 performs the following calculation after setting the imaging device 10A-2 (CAM2) and the imaging device 10A-3 (CAM3) as candidates for complementary imaging devices.

[0151] ●If there is an obstacle in front of the camera● Here, when selecting a complementary photographing device, if there is an obstacle between the attention point P and the photographing device, the selection unit 53 performs calculation using a predetermined coefficient so that a combination of these photographing devices is unlikely to be selected. As shown in Fig. 25, it is assumed that there is a wall between the photographing device 10A-3 and the attention point P, making it difficult for the photographing device 10A-3 to photograph the attention point P.

[0152] In this case, a weighting coefficient management table (weighting coefficient management DB 5006, see FIG. 19(A)) is used for each term of the above-mentioned calculation formula. The coefficients in this weighting coefficient management table can be set to any value in advance by the administrator or the like who manages the communication management device 50. Therefore, the administrator or the like checks the multiple imaging devices arranged at the distribution base, and when it is determined that a wall, an obstacle, or the like is present near the imaging device, the coefficient related to the corresponding imaging device is set to a value larger than the coefficients of the other imaging devices. As a result, the selection unit 53 performs a process of removing the imaging device hidden by the wall, the obstacle, or the like from the candidates for the complementary imaging device from the imaging devices within the range. In this way, the user who uses the communication terminal 70 can more easily observe the subject at the attention point P. Furthermore, even when the selection unit 53 determines that there is a problem such as an abnormal operation among the imaging devices within the range, it is possible to remove such an imaging device from the candidates for the complementary imaging device. This is because the complementary imaging device selected by the selection unit 53 is selected to be one with a small calculation result based on the above-mentioned calculation formula.

[0153] The calculation results for each imaging device shown in FIG. 25 are as follows. 1.|101 - 120|* 1.0 + |138 - 120|* 10.0 + |121 - 120|* 10.0 = 209 2. |101 - 120|* 1.0 + |71 - 120|* 1.0 + |188 - 120|* 1.0 = 136 3. |121 - 120|* 10.0 + |67 - 120|* 1.0 + |172 - 120|* 1.0 = 115 As a result of the calculation, the selection unit 53 selects the combination of CAM1, CAM3, and CAM4, which has the smallest value of the calculation result, as shown in FIG. 25. However, since there is a wall in front of CAM3, which is not practical, the selection unit 53 cooperates with the judgment unit 55 to perform the next judgment and selection. That is, in this case, the selection unit 53 selects CAM1 (camera 10A-1) as the reference camera device, and CAM2 (camera 10A-2) and CAM4 (camera 10A-4), which have the next smallest value of the calculation result, as candidates for the supplementary camera device. This prevents CAM3 (camera 10A-3), which is hidden by an obstacle, from being selected, making it easier to observe the subject at the attention point P. However, even if CAM4 (camera 10A-4) is selected as one of the candidates for the supplementary camera device as in the example of FIG. 25, it is still highly likely that it is difficult to capture the front, back, left and right of each image to be captured with respect to the attention point P to the maximum extent (multi-angled). In such a case, the selection unit 53 may cancel the selection of the complementary photographing device, or may send a message to the user of the communication terminal 70 urging him / her to cancel the selection process or to specify another point of interest.

[0154] ●When the number of divisions is 4● The example shown in Fig. 26 is an example in which the division number n is 4. In this case, the selection unit 53 selects the camera device 10 arranged at the position closest to a central angle of 90°±θ around the attention point P in a circle of radius r around the attention point P as a complementary camera device that complements the reference camera device 10A-1. In the example shown in Fig. 26, three complementary camera devices, namely, the camera device 10A-2, the camera device 10A-3, and the camera device 10A-4, are selected.

[0155] ●When the number of divisions is 5 or more● Similarly, when the division number n is 5 or more, the selection unit 53 selects, as a candidate for a complementary imaging device (second imaging device), an imaging device arranged such that the angle between a virtual line segment connecting the reference imaging device (first imaging device) and the attention point P is closest to the angle obtained by dividing 360° by the division number n (step S105-3). After that, the selection unit 53 performs a coefficient calculation using at least one of the data tables of the weighting coefficient management table (weighting coefficient management DB 5006, see FIG. 19(A)) and the specific weighting coefficient management table (specific weighting coefficient management DB 5006, see FIG. 19(B)) (step S105-4), and can select the reference imaging device and a complementary imaging device that complements the reference imaging device.

[0156] The communication management device 50 may perform the processes of steps S105-3 and S105-4 in the reverse order. That is, after performing the coefficient calculation using the data table described in step S105-4, the selection process of the photographing device (complementary photographing device) closest to 360° / n with respect to the reference photographing device for the selected photographing device described in step S105-3 may be performed. However, when an obstacle such as a wall is present around the selected complementary photographing device, or when there is a problem with the operation of the selected complementary photographing device, the selection unit 53 and the determination unit 55 may exclude the photographing device in such a state from candidates for complementary photographing devices and determine whether or not another photographing device can be used as a candidate for an alternative complementary photographing device.

[0157] In addition to the weighting coefficient management table (weighting coefficient management DB 5006, see FIG. 19(A)), the communication management device 50 may manage a specific weighting coefficient management table (specific weighting coefficient management DB 5007, see FIG. 19(B)) that excludes combinations between specific imaging devices, and may calculate a coefficient of 1.0 for combinations of a reference imaging device and a supplementary imaging device that are not managed in this table. With this method, the administrator who manages the communication management device 50 only needs to set and manage specific weighting coefficients for the imaging devices that require setting, which reduces the effort required for the administrator to set weighting coefficients in advance for the imaging devices at a distribution center where many imaging devices are located.

[0158] The process of selecting the reference imaging device and the supplementary imaging device is also applied to a state in which the imaging devices 10 arranged at the distribution bases are arranged in different positions as shown in FIG.

[0159] Returning to FIG. 33, the generating unit 56 generates, for each of the image capturing devices 10A selected in step S104, predetermined area information corresponding to the attention point coordinates received in step S74 (step S106). Specifically, the generating unit 56 generates predetermined area information for displaying a predetermined area image (perspective projection transformation image) in which the attention point coordinates are located at the center of the omnidirectional image captured by the selected image capturing device 10A. Here, a predetermined value set in advance is used for the radius vector (r) and the polar angle (θ). This indicates that the angle of view and the elevation angle of the perspective projection transformation image are processed as constant. The azimuth angle (φ) can be calculated as a relative value between the installation direction of the image capturing device 10A indicated in the installation information and the direction of the attention point coordinates from the image capturing device 10. Then, the generating unit 56 generates predetermined area information for displaying (perspective projection display) a predetermined area image centered on the attention point coordinates from the omnidirectional image acquired by the image capturing device 10A based on the calculation result. The generating unit 56 executes such a process for all the camera devices 10A selected in step S104. The placement position (coordinate value) and placement direction (the direction in which the front of the camera device 10A faces) of the camera device 10 are set in advance in the placement information management DB 5004 by an administrator or the like. In the process of step S106 described above, in order to obtain a combination of the IP address of the camera device and the predetermined area information, the storing / reading unit 59 and the selecting unit 53 generate the corresponding predetermined area information by using the predetermined area management table (predetermined area management DB 5003, see FIG. 17(A)) and the placement information management table (placement information management DB 5004, see FIG. 17(B)), respectively, which are managed by the communication management device 50.

[0160] Then, the storage / reading unit 59 stores the predetermined area information generated in step S105 in a predetermined area management table (predetermined area management DB 5003, see FIG. 17(A)) in association with the IP address of the corresponding image capturing device 10A (step S107).

[0161] In this way, the communication management device 50 can select a camera device 10 that will capture the attention point P based on the attention point coordinates transmitted from the communication terminal 70 and the placement information of the multiple camera devices 10, and can calculate the angle of view of the selected camera device 10A for displaying the attention point P.

[0162] In this embodiment, the generating unit 56 has a function or a means to function as an example of a generating means. Furthermore, the generating unit 56 that generates the predetermined area information may be configured to be included in another device provided between the communication management device 50 and the communication terminal 70, in addition to being included in the communication management device 50.

[0163] Returning to Fig. 32, the transmitting / receiving unit 51 of the communication management device 50 transmits a selection result notification, which is the processing result in step S75, to the communication terminal 70C (step S77). As a result, the transmitting / receiving unit (acquisition unit) 71 of the communication terminal 70C receives the selection result notification transmitted from the communication management device 50. This selection result notification includes a set of IP addresses of the imaging devices 10 corresponding to the number of pieces of predetermined area information generated in step S106. In this way, the transmitting / receiving unit (acquisition unit) 71 of the communication terminal 70C acquires the predetermined area information indicating the predetermined area including the attention point P accepted in step S72.

[0164] As described above, the transmitting / receiving unit 51 transmits to the communication terminal 70 the predetermined area information (first predetermined area information) indicating the predetermined area including the predetermined position in the photographed image of the reference photographing device 10A-1 (first photographing device) among the multiple photographing devices arranged in the area of ​​the distribution base, and transmits to the communication terminal 70 the predetermined area information (second predetermined area information) indicating the predetermined area including the predetermined position in the photographed image of the complementary photographing device, which is a complementary photographing device (second photographing device) arranged at an angle closest to the angle obtained by dividing according to the division number indicated by the division number information. At this time, the complementary photographing device (second photographing device) arranged at the angle closest to the angle obtained by dividing according to the division number is the photographing device 10A-4 in the example of FIG. 21 in the case of division into two. Furthermore, the complementary photographing device (second photographing device) arranged at the angle closest to the angle obtained by dividing according to the division number is the photographing device 10A-2 and the photographing device 10A-3 in the example of FIG. 23 in the case of division into three.

[0165] The transmitting / receiving unit 51 of the communication management device 50 can transmit the first and second predetermined area information simultaneously or at different times to the communication terminal 70. Furthermore, when transmitting at different times, the order of transmission of the first and second predetermined area information does not matter.

[0166] Next, the memory / read unit 79 of the communication terminal 70C stores the specified area information received in step S76 in the specified area management table (specified area management DB7002, see Figure 20 (B)) in association with the IP address of the imaging device 10 (step S78).

[0167] Next, the transmitting / receiving unit (acquiring unit) 71 transmits a captured image request to the communication management device 50 in order to acquire the captured image captured by the selected imaging device 10 (step S79). As a result, the transmitting / receiving unit 51 of the communication management device 50 receives the captured image request transmitted from the communication terminal 70. Note that the captured image request includes the image data ID acquired in step S40.

[0168] The storage / readout unit 59 of the communication management device 50 that has received the captured image request searches the image type management table (image type management DB 5002, see FIG. 16(B)) using the image data ID as a search key to read out the corresponding source name (image type information) (step S80). This identifies the image data representing the captured image that corresponds to the image data ID.

[0169] The transmitting / receiving unit 51 transmits image data representing the captured image that has been read out to the communication terminal 70 (step S81). As a result, the transmitting / receiving unit 71 of the communication terminal 70 receives the image data transmitted from the communication management device 50. This image data includes an image data ID and sound data relating to the captured image.

[0170] Next, the user A1 performs a predetermined input operation, and the reception unit 72 receives a selection of a display image to be displayed on the communication terminal 70C from the captured image data received in step S53 (step S82). Note that if the communication terminal 70 is capable of simultaneously displaying multiple captured images, or if the number of received captured images is less than the number that can be simultaneously displayed on the communication terminal 70, the process of step S78 may be omitted.

[0171] Next, the image / sound processing unit 73 generates a predetermined area image by performing perspective projection transformation using the predetermined area information received in step S76 in order to display an image of the predetermined area specified by the predetermined area information corresponding to the display image selected in step S76 (step S83). This allows the communication terminal 70C to generate a predetermined area image including the attention point P specified by the user from the celestial sphere image which is the captured image captured by the imaging device 10 selected in the communication management device 50.

[0172] Then, the display control unit 74 causes the display 706 to display the predetermined area image generated in step S79 (step S84). Fig. 36 is an example of a display screen displayed on the communication terminal 70C in step S84. The display screen 900 shown in Fig. 36 displays the predetermined area image X generated in step S83. The user A1 can check the current situation of the position of the attention point P at the distribution base in detail by checking the predetermined area image X corresponding to the position of the attention point P at the distribution base specified using the base display screen 600.

[0173] In this way, the image communication system 1 can select an appropriate photographing device 10 for viewing the attention point P specified by the user at the viewing location, using the communication management device 50. Then, the image communication system 1 can cause the communication terminal 70 to display an image showing the attention point P by transmitting, from the communication management device 50 to the communication terminal 70, information on a predetermined area capable of displaying an image of the attention point P photographed by the selected photographing device 10.

[0174] In this embodiment, an example has been described in which the selection unit 53 divides the image into two and three areas. However, the selection unit 53 may increase or decrease the number of divisions depending on the number of image capturing devices 10 or distribution terminals located at the distribution center, the network bandwidth of the communication network 100, or the number of images that can be simultaneously received and displayed on the communication terminal 70, etc.

[0175] In addition, in the image communication system 1 according to this embodiment, when the above-mentioned processes of steps S75 to S77, steps S79 to S81, etc. are executed, other devices may exist between the communication management device 50 (communication management device) and the communication terminal 70 (communication terminal). In other words, each piece of information transmitted and received between the communication management device 50 and the communication terminal 70 may be configured to be transmitted and received once via another device.

[0176] <Modification of display process of attention point P> Next, a modified example of the display process of the attention point P in the image communication system 1 will be described with reference to Fig. 37. Fig. 37 is a sequence diagram showing a modified example of the display process of the attention point P in the image communication system. Fig. 37 shows an example in which the selection process of the image capture device 10 described above is executed by the communication terminal 70 at the viewing location. Note that the processes of steps S201 to S204 are similar to the processes of steps S71 to S74 in Fig. 32, and therefore description thereof will be omitted.

[0177] 37, the transmitting / receiving unit (acquisition unit) 71 transmits a placement information acquisition request, which is a request to acquire placement information indicating the placement position of the image capturing device 10, to the communication management device 50 (step S205). As a result, the transmitting / receiving unit 51 of the communication management device 50 receives the placement information acquisition request transmitted from the communication terminal 70C. This placement information acquisition request includes the base ID received in steps S14 and S16.

[0178] Next, the communication management device 50 searches the placement information management table (placement information management DB 5004, see FIG. 17(B)) using the location ID received in step S205 as a search key to read out placement information associated with the same location ID as the received location ID (step S206). Then, the transmitting / receiving unit 51 transmits the placement information read out in step S205 to the communication terminal 70C (step S207). As a result, the transmitting / receiving unit (acquisition unit) 71 of the communication terminal 70C receives the placement information transmitted from the communication management device 50.

[0179] Next, the selection unit 81 executes a selection process of the photographing device based on the coordinate values ​​indicated by the arrangement information and the division number indicated by the division number information received in step S206 (step S208). The selection process of the photographing device executed in step S208 is the same as the process of step S76 shown in FIG. 32 (details are shown in the flowchart of FIG. 33), so a description thereof will be omitted. Among them, the processes of steps S101 and S103 may be omitted. Furthermore, the storage of the predetermined area information by the storage / readout unit 79 executed in the process of step S107 is performed in the predetermined area management table (predetermined area management DB 7002, see FIG. 20(B)) instead of the predetermined area management table (predetermined area management DB 5003, see FIG. 17(A)). Furthermore, the processes of steps S209 to S214 are the same as the processes of steps S79 to S84 in FIG. 32, so a description thereof will be omitted.

[0180] In this way, the image communication system 1 according to the modified example of the embodiment can display an image showing the attention point P on the communication terminal 70 by having the communication terminal 70 select the photographing device 10 that photographs an image showing the attention point P designated by the user. In this way, the image communication system 1 can allow a user at a viewing location to view the image of the designated attention point P, similar to the above-described embodiment.

[0181] [Main Effects of the Present Embodiment] As described above, the communication management device 50 according to an embodiment of the present invention receives coordinate values ​​indicating the position of the attention point P in the area where the multiple image capture devices designated by the user are arranged, and division number information when dividing the area with the attention point P as a boundary (step S73). After that, the communication management device 50 transmits to the communication terminal 70 first predetermined area information indicating a predetermined area including a predetermined position in an image captured by a reference image capture device arranged at a distance closest to a predetermined position corresponding to the predetermined position information among the multiple image capture devices, and second predetermined area information indicating a predetermined area including a predetermined position in an image captured by a complementary image capture device arranged at an angle closest to an angle obtained by dividing the reference image capture device according to the division number designated by the user (step S77). This allows the communication management device 50 to display an image of the attention point P at the distribution base captured by the image capture device selected by the division number designated by the user, without performing processing on the image capture device 10. Therefore, the communication management device 50 allows the communication terminal 70 to display an image of an area including a predetermined position, without performing operations on the multiple image capture devices individually.

[0182] In addition, the communication management device 50 uses a data table that manages coefficients that are weighted in advance by the administrator according to the number of divisions specified by the user, making it possible to exclude from among multiple image capture devices those image capture devices that are not suitable for use in viewing images.

[0183] [Supplementary description of the embodiment] In the above-described embodiment, a case has been described in which the captured image (whole image) is a three-dimensional panoramic image as an example of a celestial sphere image (celestial sphere panoramic image), but the captured image may be a two-dimensional panoramic image.

[0184] Also, in the above-described embodiment, the communication terminal 70 at the viewing location does not need to be a dedicated viewing terminal, and may be configured to distribute captured images from the terminal itself, and perform both the distribution function and the viewing function simultaneously. Similarly, the distribution terminal 30 at the distribution location does not need to be a dedicated distribution terminal, and may be configured to display captured images distributed from other locations, and perform both the distribution function and the viewing function simultaneously. In this way, the image communication system 1 may be configured to perform two-way communication of captured images between multiple locations.

[0185] In addition, each function of the above-mentioned embodiment can be realized by one or more processing circuits. Here, the "processing circuit" in this specification includes a device programmed to execute each function by software, such as a processor implemented by an electronic circuit. This device refers to, for example, a processor, an ASIC (Application Specific Integrated Circuit), a DSP (digital signal processor), an FPGA (field programmable gate array), a SOC (System on a chip), a GPU (Graphics Processing Unit), and a conventional circuit module designed to execute each function described above.

[0186] In addition, the various tables in the above-mentioned embodiments may be generated by the learning effect of machine learning, and the data of each associated item may be classified by machine learning, making it unnecessary to use tables. Here, machine learning is a technology for making a computer acquire human-like learning capabilities, and refers to a technology in which a computer autonomously generates an algorithm required for judgment such as data identification from learning data previously acquired, and applies this to new data to make predictions. The learning method for machine learning may be any of supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, and deep learning, or may be a combination of these learning methods, and any learning method for machine learning may be used.

[0187] So far, we have described a communication terminal, an image communication system, an image display method, and a program according to one embodiment of the present invention. However, the present invention is not limited to the above-described embodiment, and can be modified within the scope of what a person skilled in the art can conceive, such as adding, changing, or deleting other embodiments. As long as the functions and effects of the present invention are achieved in any aspect, it is included in the scope of the present invention. [Explanation of symbols]

[0188] 1. Image communication system 10. Imaging device (example of reference imaging device, example of supplementary imaging device) 30 Distribution terminal 50 Communication management device 51 Transmitting / receiving unit (an example of a transmitting means) 53 Selection unit (an example of a selection means) 55 Judgment unit (an example of a judgment means) 56 Generation unit (an example of a generation means) 70 Communication terminal 71 Transmitting / receiving unit (an example of an acquiring means, an example of a receiving means) 72 Reception unit (an example of reception means) 74 Display control unit (an example of a display control means) 77 Predetermined position identification part 81 Selection unit (an example of a selection means) 82 Generation unit (an example of a generation means) [Prior art documents] [Patent documents]

[0189] [Patent Document 1] JP 2017-22587 A

Claims

1. A communication management device that communicates with a communication terminal that displays images captured by a plurality of image capture devices, a receiving means for receiving from the communication terminal predetermined position information indicating a predetermined position in an area in which the plurality of image capturing devices are arranged and division number information indicating a number of divisions when the area is divided with the predetermined position as a boundary; a selection means for selecting a first photographing device and a second photographing device from the plurality of photographing devices based on arrangement position information indicating arrangement positions of the plurality of photographing devices, the predetermined position information and the division number information received by the receiving means; a generating means for generating first predetermined area information indicating a predetermined area including the predetermined position in an image captured by the first photographing device selected by the selecting means, and second predetermined area information indicating a predetermined area including the predetermined position in an image captured by the second photographing device selected by the selecting means; A transmitting means for transmitting the first predetermined area information and the second predetermined area information generated by the generating means to the communication terminal. A communication management device comprising:

2. The selection means selects, as a first image capture device, an image capture device that is closest to the predetermined position from among the plurality of image capture devices that are located within a predetermined range from the predetermined position.

2. The communication management device according to claim 1.

3. The selection means selects a candidate for the second image capturing device from among the image capturing devices within the range based on an angle between the image capturing devices sandwiching the predetermined position and the number of divisions.

3. The communication management device according to claim 2.

4. When the number of divisions is n (the number of divisions is a natural number equal to or greater than 2), the selection means selects, as a candidate for the second image capture device, an image capture device that is disposed at an angle between a virtual line segment connecting the first image capture device and the predetermined position that is closest to an angle obtained by dividing 360° by the number of divisions n.

4. The communication management device according to claim 3.

5. When it is determined that there is an obstacle between the predetermined position and the within-range photographing device, or when it is determined that there is a problem with any of the within-range photographing devices, the selection means removes the determined within-range photographing device from candidates for the second photographing device.

4. The communication management device according to claim 3.

6. The communication management device according to claim 1 , wherein the captured image is a spherical image.

7. An image communication system having a communication terminal that displays images captured by a plurality of image capture devices, and a communication management device that communicates with the communication terminal, The communication management device includes: a receiving means for receiving from the communication terminal predetermined position information indicating a predetermined position in an area in which the plurality of image capturing devices are arranged and division number information indicating a number of divisions when the area is divided with the predetermined position as a boundary; a selection means for selecting a first photographing device and a second photographing device from the plurality of photographing devices based on arrangement position information indicating arrangement positions of the plurality of photographing devices, the predetermined position information and the division number information received by the receiving means; a generating means for generating first predetermined area information indicating a predetermined area including the predetermined position in an image captured by the first photographing device selected by the selecting means, and second predetermined area information indicating a predetermined area including the predetermined position in an image captured by the second photographing device selected by the selecting means; A transmitting means for transmitting the first predetermined area information and the second predetermined area information generated by the generating means to the communication terminal.

1. An image communication system comprising:

8. 8. The image communication system according to claim 7, The communication terminal includes: a display control means for causing a display means to display a planar image showing the region; a reception means for receiving an input of the predetermined position and the number of divisions in the planar image displayed by the display control means; A transmission means for transmitting predetermined position information indicating the predetermined position and division number information indicating the division number received by the reception means to the communication management device, 1. An image communication system comprising:

9. A communication management method executed by a communication management device that communicates with a communication terminal that displays images captured by a plurality of image capture devices, comprising: a receiving step of receiving, from the communication terminal, predetermined position information indicating a predetermined position in an area in which the plurality of image capturing devices are arranged, and division number information indicating a division number when the area is divided with the predetermined position as a boundary; a selection step of selecting a first photographing device and a second photographing device from the plurality of photographing devices based on arrangement position information indicating arrangement positions of the plurality of photographing devices, the predetermined position information and the division number information received by the receiving step; a generating step of generating first predetermined area information indicating a predetermined area including the predetermined position in the photographed image photographed by the first photographing device selected by the selecting step, and second predetermined area information indicating a predetermined area including the predetermined position in the photographed image photographed by the second photographing device selected by the selecting step; a transmitting step of transmitting the first predetermined area information and the second predetermined area information generated by the generating step to the communication terminal. A communication management method comprising:

10. A program for causing a computer to function as each of the means according to any one of claims 1 to 6.

Images