Image communication system and image display method

Abstract

To display an image of a distribution site corresponding to an attention point designated on the distribution site side without performing processing on a photographing device.SOLUTION: In an image communication system, a communication terminal 70 can display an image of a predetermined area in the entire image (for example, an entire-celestial-sphere image), and comprises: an attention point management DB 7003 (an example of storage means) that stores attention point area information (an example of first predetermined area information) indicating a predetermined area (an example of a first predetermined area) including an attention point at a distribution site where the entire image is photographed; a transmitting and receiving unit 71 (an example of receiving means) that receives attention point designation information indicating the attention point from a distribution system installed at the distribution site; and a display control unit 74 (an example of display control means) that, based on the attention point area information corresponding to the attention point indicated in the received attention point designation information, displays a predetermined area image (an example of a first predetermined area image) including the attention point.SELECTED DRAWING: Figure 14

Description

[Technical field]

[0001] The present disclosure relates to a communication terminal, an image communication system, an image display method, and a program. [Background technology]

[0002] There is known a camera capable of capturing images in all directions using multiple wide-angle or fisheye lenses. There is also known a system that distributes image data captured by such a camera in real time and enables a user to view the status of a distribution base where the camera is installed in real time at another base.

[0003] For example, Patent Document 1 discloses a system that allows a user to grasp the situation in a remote location by displaying images captured by a plurality of cameras placed in the remote location on a terminal. Summary of the Invention [Problem to be solved by the invention]

[0004] However, with the conventional method, when a point of interest, which is a location at a distribution center that requires attention, is specified from outside, a problem occurred in that the viewer had to individually operate the imaging device installed at the distribution center in order to view the point of interest. [Means for solving the problem]

[0005] In order to solve the above-mentioned problem, the invention according to claim 1 provides a communication terminal capable of displaying an image of a predetermined area in an entire image. and a distribution system installed at a distribution base where the entire image is captured. And, The communication terminal includes: before Allocation A storage means for storing first predetermined area information indicating a first predetermined area including a point of interest in a reception center; The above a receiving means for receiving attention point designation information indicating the attention point from a distribution system; and a display control means for displaying a first predetermined area image based on the first predetermined area information corresponding to the attention point indicated in the attention point designation information received by the receiving means. The distribution system includes an acquisition means for acquiring the entire image of the distribution base, a second determination means for determining whether or not the attention point is designated based on the entire image acquired by the acquisition means, and a transmission means for transmitting the entire image acquired by the acquisition means to the communication terminal, and transmitting attention point designation information indicating the attention point to the communication terminal when it is determined that the attention point is designated. It is. Effect of the Invention

[0006] According to the present invention, it is possible to display an image of a distribution base corresponding to a point of interest designated on the distribution base side without performing any processing on the imaging device. [Brief description of the drawings]

[0007] [Figure 1] 1A is a left side view of the imaging device, (B) is a front view of the imaging device, and (C) is a plan view of the imaging device. [Diagram 2] FIG. 13 is a conceptual diagram of the imaging device in use. [Diagram 3] (A) is a hemispherical image (front) captured by the camera, (B) is a hemispherical image (back) captured by the camera, and (C) is an image represented by equirectangular projection. [Figure 4] (A) A conceptual diagram showing how a sphere is covered with an equirectangular projection image, and (B) a diagram showing a spherical image. [Diagram 5] FIG. 13 is a diagram showing the positions of a virtual camera and a predetermined area when a celestial sphere image is a three-dimensional sphere. [Figure 6] FIG. 6(A) is a three-dimensional perspective view of FIG. 5, and FIG. 6(B) is a diagram showing a state in which an image of a predetermined area is displayed on a display. [Figure 7] 13 is a diagram showing the relationship between predetermined region information and an image of a predetermined region T. FIG. [Figure 8] FIG. 1 illustrates points in three-dimensional Euclidean space in spherical coordinates. [Figure 9] FIG. 1 is a diagram illustrating an example of an overall configuration of an image communication system. [Figure 10] FIG. 1 is a diagram for explaining an outline of an image communication system. [Figure 11] FIG. 2 is a diagram illustrating an example of a hardware configuration of the imaging device. [Figure 12] FIG. 2 is a diagram illustrating an example of a hardware configuration of a distribution terminal, a communication management system, a communication terminal, and a point of interest designation device. [Figure 13]FIG. 1 illustrates an example of a functional configuration of an image communication system. [Figure 14] FIG. 1 illustrates an example of a functional configuration of an image communication system. [Figure 15] FIG. 1A is a schematic diagram showing an example of a photographing device management table, and FIG. 1B is a conceptual diagram showing an example of an image type management table. [Figure 16] FIG. 13 is a conceptual diagram illustrating an example of a focus point management table. [Figure 17] FIG. 1A is a schematic diagram showing an example of a session management table, and FIG. 1B is a conceptual diagram showing an example of an image type management table. [Figure 18] FIG. 2A is a schematic diagram showing an example of a predetermined area management table, and FIG. 2B is a conceptual diagram showing an example of an installation information management table. [Figure 19] FIG. 13 is a schematic diagram illustrating an example of a distribution base management table. [Figure 20] FIG. 1A is a schematic diagram showing an example of an image type management table, and FIG. 1B is a conceptual diagram showing an example of a predetermined area management table. [Figure 21] FIG. 13 is a schematic diagram illustrating an example of a focus point management table. [Figure 22] FIG. 11 is a sequence diagram showing an example of a process for participating in a specific communication session in the image communication system. [Figure 23] FIG. 13 is a diagram illustrating an example of a selection screen for a communication session. [Figure 24] 11 is a sequence diagram showing an example of a management process of image type information in the image communication system. FIG. [Diagram 25] 10 is a sequence diagram showing an example of a communication process of captured image data and sound data in the image communication system. FIG. [Figure 26] FIG. 11 is a sequence diagram showing an example of a display process of a predetermined area image. [Figure 27] FIG. 11 is a sequence diagram showing an example of a display process of a point of interest in the image communication system. [Figure 28] 11 is a flowchart illustrating an example of a display process of a point of interest in the communication terminal. [Figure 29] FIG. 11 is a sequence diagram showing a modified example of the display process of the point of interest in the image communication system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the description of the drawings, the same elements are given the same reference numerals, and duplicated description will be omitted.

[0009] ●Embodiment● How to generate spherical images A method for generating a spherical image will be described with reference to FIGS.

[0010] First, the appearance of the imaging device 10 will be described with reference to Fig. 1. The imaging device 10 is a digital camera for obtaining captured images that are the basis of a full celestial sphere (360°) panoramic image. Fig. 1(A) is a left side view of the imaging device, Fig. 1(B) is a front view of the imaging device, and Fig. 1(C) is a plan view of the imaging device.

[0011] As shown in FIG. 1(A), the photographing device 10 is sized so that a person can hold it in one hand. Also, as shown in FIG. 1(A), FIG. 1(B), and FIG. 1(C), an image sensor 103a is provided on the front side (front side) and an image sensor 103b is provided on the back side (rear side) of the upper part of the photographing device 10. These image sensors (image sensors) 103a and 103b are used in conjunction with optical members (for example, lenses 102a and 102b described later) capable of photographing a hemispherical image (angle of view of 180° or more). Also, as shown in FIG. 1(B), an operation unit 115 such as a shutter button is provided on the surface opposite to the front side of the photographing device 10.

[0012] Next, the usage of the photographing device 10 will be described with reference to Fig. 2. Fig. 2 is an image diagram of the photographing device in use. As shown in Fig. 2, the photographing device 10 is used, for example, by a user holding it in his / her hand to photograph subjects around the user. In this case, two hemispherical images can be obtained by capturing images of subjects around the user using the image sensor 103a and the image sensor 103b shown in Fig. 1, respectively.

[0013] Next, an outline of the process of creating a celestial sphere image from an image captured by the image capture device 10 will be described with reference to Fig. 3 and Fig. 4. Fig. 3(A) is a diagram showing a hemispherical image (front side) captured by the image capture device, Fig. 3(B) is a diagram showing a hemispherical image (rear side) captured by the image capture device, and Fig. 3(C) is a diagram showing an image expressed by equirectangular projection (hereinafter referred to as "equirectangular projection image"). Fig. 4(A) is a conceptual diagram showing a state in which a sphere is covered with an equirectangular projection image, and Fig. 4(B) is a diagram showing a celestial sphere image.

[0014] As shown in Fig. 3(A), the image obtained by the image sensor 103a becomes a hemispherical image (front side) curved by the lens 102a described below. Also, as shown in Fig. 3(B), the image obtained by the image sensor 103b becomes a hemispherical image (rear side) curved by the lens 102b described below. Then, the photographing device 10 synthesizes the hemispherical image (front side) and the hemispherical image (rear side) flipped 180 degrees to generate an equirectangular projection image EC as shown in Fig. 3(C).

[0015] Then, the image capturing device 10 uses OpenGL ES (Open Graphics Library for Embedded Systems) to paste an equirectangular projection image EC so as to cover the spherical surface as shown in FIG. 4A, and creates a celestial sphere image CE as shown in FIG. 4B. In this way, the celestial sphere image CE is expressed as an image in which the equirectangular projection image EC faces the center of the sphere. Note that OpenGL ES is a graphics library used to visualize 2D (2-Dimensions) and 3D (3-Dimensions) data. Also, the celestial sphere image CE may be a still image or a video.

[0016] As described above, the spherical image CE is an image pasted to cover the spherical surface, which gives a sense of incongruity to people when they see it. Therefore, the imaging device 10 can display a predetermined region T (hereinafter, referred to as a "predetermined region image") that is a part of the spherical image CE as a planar image with little curvature, thereby enabling a display that does not give a sense of incongruity to people. This will be described with reference to Figs. 5 to 8.

[0017] FIG. 5 is a diagram showing the positions of the virtual camera and the predetermined area when the omnidirectional panoramic image is a three-dimensional sphere. The virtual camera IC corresponds to the position of the viewpoint of a user who views the omnidirectional image CE displayed as a three-dimensional sphere. FIG. 6(A) is a three-dimensional perspective view of FIG. 5, and FIG. 6(B) is a diagram showing the predetermined area image when displayed on a display. FIG. 6(A) shows the omnidirectional image CE shown in FIG. 5 as a three-dimensional sphere CS. If the omnidirectional image CE generated in this way is a sphere CS, as shown in FIG. 5, the virtual camera IC is located inside the omnidirectional image CE. The predetermined area T in the omnidirectional image CE is the shooting area of ​​the virtual camera IC, and is specified by the predetermined area information indicating the shooting direction and the angle of view of the virtual camera IC in the three-dimensional virtual space including the omnidirectional image CE. In addition, the zoom of the predetermined area T can also be expressed by moving the virtual camera IC closer to or farther away from the omnidirectional image CE. The predetermined area image Q is an image of a predetermined area T in the omnidirectional image CE. Therefore, the predetermined area T can be specified by the angle of view α and the distance f from the virtual camera IC to the omnidirectional image CE (see FIG. 7).

[0018] Then, the predetermined area image Q shown in Fig. 6(A) is displayed on a predetermined display as an image of the shooting area of ​​the virtual camera IC as shown in Fig. 6(B). The image shown in Fig. 6(B) is a predetermined area image represented by the initial setting (default) predetermined area information. The following description will be given using the shooting direction (ea, aa) and angle of view (α) of the virtual camera IC. Note that the predetermined area T may be represented by the shooting area (X, Y, Z) of the virtual camera IC, which is the predetermined area T, instead of the angle of view α and the distance f.

[0019] Next, the relationship between the predetermined area information and the image of the predetermined area T will be described with reference to FIG. 7. FIG. 7 is a diagram showing the relationship between the predetermined area information and the image of the predetermined area T. As shown in FIG. 7, "ea" indicates an elevation angle, "aa" indicates an azimuth angle, and "α" indicates an angle of view. That is, the attitude of the virtual camera IC is changed so that the gaze point of the virtual camera IC indicated by the shooting direction (ea, aa) becomes the center point CP(x, y) of the predetermined area T, which is the shooting area of ​​the virtual camera IC. As shown in FIG. 7, the center point CP(x, y) when the diagonal angle of view of the predetermined area T represented by the angle of view α of the virtual camera IC is α becomes the parameter ((x, y)) of the predetermined area information. The predetermined area image Q is an image of the predetermined area T in the omnidirectional image CE. f is the distance from the virtual camera IC to the center point CP(x, y). L is the distance between any vertex of the predetermined area T and the center point CP(x, y) (2L is the diagonal line). In FIG. 7, the trigonometric function shown in the following formula 1 generally holds.

[0020]

number

[0021] The image capturing device 10 described above is an example of an image capturing device capable of capturing a wide-angle image, and a celestial sphere image is an example of a wide-angle image. Here, a wide-angle image is generally an image captured using a wide-angle lens, and is captured with a lens that can capture a wider range than the human eye can sense. Also, it generally means an image captured with a lens with a focal length of 35 mm or less in 35 mm film equivalent.

[0022] FIG. 8 is a diagram showing points in a three-dimensional Euclidean space in spherical coordinates. Here, the position coordinates of the center point CP when expressed in a spherical polar coordinate system are (r, θ, φ). (r, θ, φ) are the radius, polar angle, and azimuth angle, respectively. The radius r is the distance from the origin of the three-dimensional virtual space including the omnidirectional image to the center point CP, and is therefore equal to f. FIG. 8 is a diagram showing these relationships. Hereafter, the explanation will be given using the position coordinates (r, θ, φ) of the virtual camera IC.

[0023] ●Outline of image communication system Next, an outline of the configuration of the image communication system of the first embodiment will be described with reference to Fig. 9. Fig. 9 is a diagram showing an example of the overall configuration of the image communication system. The image communication system 1 shown in Fig. 9 is a system in which captured images such as videos distributed from a distribution base are displayed at a plurality of viewing bases, thereby allowing a wide range of images (e.g., spherical images) captured at the distribution base to be viewed in real time.

[0024] As shown in FIG. 9, the image communication system 1 includes a photographing device 10, a point of interest designation device 20 and a distribution terminal 30 located at a distribution center, a communication management system 50, and communication terminals 70 (70A, 70B; hereinafter, referred to as communication terminal 70 when there is no need to distinguish between them) located at each of multiple viewing centers (viewing center A, viewing center B).

[0025] The distribution terminal 30, the communication management system 50, and the communication terminal 70 constituting the image communication system 1 can communicate with each other via a communication network 100. The communication network 100 is constructed by the Internet, a mobile communication network, a LAN (Local Area Network), and the like. Note that the communication network 100 may include not only wired communication but also wireless communication networks such as 3G (3rd Generation), 4G (4th Generation), 5G (5th Generation), Wi-Fi (Wireless Fidelity) (registered trademark), WiMAX (Worldwide Interoperability for Microwave Access), or LTE (Long Term Evolution).

[0026] The image capturing device 10, the point of interest designation device 20, and the distribution terminal 30 located at the distribution base constitute the distribution system 3. Of these, the image capturing device 10 is a special digital camera for capturing images of subjects, scenery, etc., and obtaining two hemispherical images that are the basis of a celestial sphere image, as described above. The captured images acquired by the image capturing device 10 may be moving images or still images, or may be both moving images and still images. The captured images may also include audio along with the images.

[0027] The attention point designation device 20 is a dedicated device for designating an attention point, which is a specific location in a distribution base. The attention point designation device 20 is fixedly installed in advance at a specific position in the distribution base in order to detect trouble such as an abnormality occurring in the specific location in the distribution base.

[0028] The distribution terminal 30 distributes images acquired from the photographing device 10 via a wired cable such as a USB (Universal Serial Bus) cable to the communication terminal 70 via the communication management system 50. The distribution terminal 30 also distributes point of interest designation information acquired from the point of interest designation device 20 via a wired cable such as a USB cable to the communication terminal 70 via the communication management system 50.

[0029] The number of distribution bases included in the image communication system 1 is not limited to one, and the image communication system 1 may include multiple distribution bases. The connection between the photographing device 10 and the point of interest designation device 20 and the distribution terminal 30 may be a wireless connection using short-range wireless communication or the like, instead of a wired connection using a wired cable.

[0030] The communication management system 50 manages and controls the communication between the distribution terminal 30 and the communication terminal 70, and manages the types of image data (classification of general image and special image) transmitted and received. In this example, the special image is a spherical image. The communication management system 50 is installed in a service company that provides image communication services.

[0031] The communication management system 50 may be constructed by a single computer, or may be constructed by multiple computers to which each unit (function or means) is divided and arbitrarily assigned. In addition, all or part of the functions of the communication management system 50 may be a server computer existing in a cloud environment, or a server computer existing in an on-premise environment.

[0032] The communication terminal 70 is a computer such as a PC used by a user at each viewing location. The communication terminal 70 displays an image (still image or video) distributed from the distribution terminal 30. The communication terminal 70 acquires a spherical image, which is an image captured by the imaging device 10, via the communication network 100. In addition, the communication terminal 70 has OpenGL ES installed, and can create predetermined area information indicating a partial area of ​​the spherical image, and create a predetermined area image from the spherical image transmitted from the distribution terminal 30. Here, the communication terminal 70A is placed at the viewing location A where the user A1 is located, and the communication terminal 70B is placed at the viewing location B where the user B is located.

[0033] 9 are merely examples, and other examples may be used. The communication terminal 70 is not limited to a PC, and may be, for example, a tablet terminal, a smartphone, a wearable terminal, a PJ (Projector), an IWB (Interactive White Board: an electronic whiteboard with a blackboard function that allows intercommunication), or an autonomous robot.

[0034] Here, the image communication system 1 will be described in outline with reference to FIG. 10. FIG. 10 is a diagram for explaining the outline of the image communication system. The distribution base shown in FIG. 10 is, for example, a factory or a server room. The distribution base is not limited to a factory or a server room, and may be an office floor, a school, a warehouse, a construction site, a store, or any other space where a user (viewer) of a viewing base needs to know the situation of a remote base. FIG. 10 shows a state in which a plurality of image capturing devices 10 (image capturing devices 10a, 10b) and a plurality of attention point designation devices 20 (attention point designation devices 20a, 20b) are installed at a distribution base. The image capturing devices 10 are arranged at predetermined intervals in a work area such as a factory line. At the distribution base, the entire distribution base is captured using the plurality of image capturing devices 10. The image capturing device 10 is an image capturing device capable of capturing the above-mentioned omnidirectional image, and captures a wide range of images while being physically fixed. The communication terminal 70 at the viewing base displays images (photographed image 1, photographed image 2) captured by a plurality of image capturing devices 10. Note that the number of image capturing devices 10 installed at the distribution base is not limited to this.

[0035] The attention point designation devices 20 are placed at specific positions in the distribution bases and designate attention points corresponding to their respective installation positions. The attention point designation device 20a, for example, designates attention point 1, which is a candidate for the attention point at the distribution base, as the attention point. On the other hand, the attention point designation device 20b, for example, designates attention point 2, which is a candidate for the attention point at the distribution base, as the attention point.

[0036] The example shown in FIG. 10 is an example of a case where, when a problem or the like occurs at attention point 2, a user at a viewing base is directed to attention point 2 at a distribution base. When a problem or the like occurs at attention point 2, attention point designation device 20b transmits attention point designation information to communication terminal 70. Based on the received attention point designation information, communication terminal 70 refers to attention point area information registered in advance and changes the display so that attention point 2 is displayed on the screen as a predetermined area displayed in each captured image. In this way, communication terminal 70 can change the display of all displayed captured images to images that include attention point 2.

[0037] Conventionally, in a system in which images captured by a plurality of remotely located image capturing devices can be viewed from a viewing base, when a user wants to see a specific location at a distribution base, the user does not know which image captured by which image capturing device to view. In addition, when users at different viewing bases want to view different locations, operations (e.g., PTZ (pan-tilt-zoom) operations) on the image capturing devices conflict, so that multiple users viewing the same distribution base cannot perform exclusive viewing such as viewing different locations. Furthermore, even when the above-mentioned omnidirectional image is captured by an image capturing device, a process must be performed to individually operate the multiple omnidirectional images to display a desired location, and the user cannot intuitively operate the image. Therefore, the image communication system 1 enables multiple users to view different locations at a distribution base by focusing on the images captured by the same image capturing device through intuitive operations on the user's display screen.

[0038] ●Hardware configuration Next, the hardware configuration of each device or terminal constituting the image communication system according to the embodiment will be described with reference to Figures 10 and 11. Note that components may be added or deleted from the hardware configurations shown in Figures 10 and 11 as necessary.

[0039] ○Hardware configuration of the imaging device○ First, the hardware configuration of the imaging device 10 will be described with reference to Fig. 10. Fig. 10 is a diagram showing an example of the hardware configuration of the imaging device. In the following, the imaging device 10 is an omnidirectional (all-directional) imaging device using two imaging elements, but the number of imaging elements may be two or more. In addition, the imaging device does not necessarily have to be a device dedicated to omnidirectional imaging, and an omnidirectional imaging unit may be attached to a normal digital camera, smartphone, or the like to have substantially the same functions as the imaging device 10.

[0040] As shown in FIG. 10, the photographing device 10 is composed of an imaging unit 101, an image processing unit 104, an imaging control unit 105, a microphone 108, a sound processing unit 109, a CPU (Central Processing Unit) 111, a ROM (Read Only Memory) 112, an SRAM (Static Random Access Memory) 113, a DRAM (Dynamic Random Access Memory) 114, an operation unit 115, an input / output I / F 116, a short-range communication circuit 117, an antenna 117a of the short-range communication circuit 117, an electronic compass 118, a gyro sensor 119, an acceleration sensor 120 and a network I / F 121.

[0041] Of these, the imaging unit 101 includes wide-angle lenses (so-called fisheye lenses) 102a and 102b (hereinafter, when there is no need to distinguish between them, they will be referred to as lenses 102) each having an angle of view of 180° or more for forming a hemispherical image, and two imaging elements 103a and 103b provided corresponding to the lenses. The imaging elements 103a and 103b include an image sensor such as a CMOS (Complementary Metal Oxide Semiconductor) sensor or a CCD (Charge Coupled Device) sensor that converts an optical image by the lenses 102a and 102b into image data of an electrical signal and outputs it, a timing generation circuit that generates horizontal or vertical synchronization signals and pixel clocks for the image sensors, and a group of registers in which various commands or parameters required for the operation of the imaging elements are set.

[0042] The imaging elements 103a and 103b of the imaging unit 101 are each connected to the image processing unit 104 via a parallel I / F bus. Meanwhile, the imaging elements 103a and 103b of the imaging unit 101 are each connected to the imaging control unit 105 via a serial I / F bus (such as an I2C bus). The image processing unit 104, the imaging control unit 105, and the sound processing unit 109 are connected to the CPU 111 via a bus 110. In addition, the ROM 112, the SRAM 113, the DRAM 114, the operation unit 115, the input / output I / F 116, the short-range communication circuit 117, the electronic compass 118, the gyro sensor 119, the acceleration sensor 120, and the network I / F 121 are also connected to the bus 110.

[0043] The image processing unit 104 takes in the image data output from the image sensors 103a and 103b via a parallel I / F bus, performs a predetermined processing on each image data, and then synthesizes the image data to create equirectangular projection image data such as that shown in Figure 3(C).

[0044] The imaging control unit 105 generally sets commands and the like in the register groups of the imaging elements 103a and 103b using an I2C bus, with the imaging control unit 105 acting as a master device and the imaging elements 103a and 103b acting as slave devices. Necessary commands and the like are received from the CPU 111. The imaging control unit 105 also uses the I2C bus to retrieve status data and the like from the register groups of the imaging elements 103a and 103b and send it to the CPU 111.

[0045] Furthermore, the imaging control unit 105 instructs the imaging elements 103a and 103b to output image data when the shutter button of the operation unit 115 is pressed. Some imaging devices 10 have a preview display function or a function corresponding to video display on a display (for example, a display of an external terminal such as a smartphone that performs short-distance communication with the imaging device 10 using the short-distance communication circuit 117). In this case, the image data is output from the imaging elements 103a and 103b continuously at a predetermined frame rate (frames / min).

[0046] As described later, the imaging control unit 105 also functions as a synchronization control means for synchronizing the output timing of image data from the imaging elements 103a and 103b in cooperation with the CPU 111. In this embodiment, the imaging device 10 is not provided with a display unit, but a display unit may be provided. The microphone 108 converts sound into sound (signal) data. The sound processing unit 109 takes in the sound data output from the microphone 108 through an I / F bus and performs a predetermined process on the sound data.

[0047] The CPU 111 controls the overall operation of the photographing device 10 and executes necessary processing. The ROM 112 stores various programs for the CPU 111. The SRAM 113 and the DRAM 114 are work memories, and store programs executed by the CPU 111, data in the middle of processing, etc. In particular, the DRAM 114 stores image data in the middle of processing by the image processing unit 104, and data of a processed equirectangular projection image.

[0048] The operation unit 115 is a collective term for various operation buttons, a power switch, a shutter button, a touch panel having both display and operation functions, etc. A user operates the operation unit 115 to input various shooting modes, shooting conditions, etc.

[0049] The input / output I / F 116 is a general term for an interface circuit (such as a USB I / F) with an external medium such as an SD card or a personal computer. The input / output I / F 116 may be wireless or wired. The data of the equirectangular projection image stored in the DRAM 114 is recorded on an external medium via the input / output I / F 116, or transmitted to an external terminal (device) via the input / output I / F 116 as necessary.

[0050] The short-range communication circuit 117 communicates with an external terminal (device) by short-range wireless communication technology such as NFC (Near Field Communication), Bluetooth (registered trademark), or Wi-Fi (registered trademark) via an antenna 117a provided in the image capturing device 10. The short-range communication circuit 117 can transmit data of the equirectangular projection image to the external terminal (device).

[0051] The electronic compass 118 calculates the direction of the image capturing device 10 from the earth's magnetism and outputs the direction information. This direction information is an example of related information (metadata) according to Exif and is used for image processing such as image correction of the captured image. The related information also includes each data of the image capture date and time and the data capacity of the image data. The gyro sensor 119 is a sensor that detects angle changes (roll angle, pitch angle, yaw angle) accompanying the movement of the image capturing device 10. The angle changes are an example of related information (metadata) according to Exif and are used for image processing such as image correction of the captured image. The acceleration sensor 120 is a sensor that detects acceleration in three axial directions. The image capturing device 10 calculates the attitude (angle with respect to the direction of gravity) of the own device (image capturing device 10) based on the acceleration detected by the acceleration sensor 120. The accuracy of image correction is improved by providing the acceleration sensor 120 in the image capturing device 10. The network I / F 121 is an interface for performing data communication using a communication network 100 such as the Internet via a router or the like.

[0052] ○Hardware configuration of distribution terminal○ Fig. 12 is a diagram showing an example of the hardware configuration of a distribution terminal. Each hardware component of distribution terminal 30 is indicated by a reference number in the 300 series. Distribution terminal 30 is constructed by a computer, and includes a CPU 301, a ROM 302, a RAM 303, a HD (Hard Disk) 304, a HDD controller 305, a display 306, an external device connection I / F 308, a network I / F 309, a bus line 310, a keyboard 311, a pointing device 312, a DVD-RW (Digital Versatile Disk Rewritable) drive 314, a media I / F 316, an audio input / output I / F 317, a microphone 318, a speaker 319, and a short-distance communication circuit 320, as shown in Fig. 12.

[0053] Among these, the CPU 301 controls the operation of the entire distribution terminal 30. The ROM 302 stores programs such as an IPL used to drive the CPU 301. The RAM 303 is used as a work area for the CPU 301. The HD 304 stores various data such as programs. The HDD controller 305 controls reading or writing of various data from or to the HD 304 under the control of the CPU 301. The display 306 displays various information such as a cursor, a menu, a window, a character, or an image. The display 306 is an example of a display unit. The display 306 may be a touch panel display equipped with an input means. The external device connection I / F 308 is an interface for connecting various external devices. In this case, the external device is, for example, a USB memory or a printer. The network I / F 309 is an interface for data communication using the communication network 100. The bus line 310 is an address bus or a data bus for electrically connecting each component such as the CPU 301 shown in FIG. 12.

[0054] The keyboard 311 is a type of input means having a plurality of keys for inputting characters, numbers, various instructions, and the like. The pointing device 312 is a type of input means for selecting or executing various instructions, selecting a processing target, or moving a cursor. The input means may be not only the keyboard 311 and the pointing device 312, but also a touch panel or a voice input device, and the like. The DVD-RW drive 314 controls reading or writing of various data from the DVD-RW 313 as an example of a removable recording medium. The DVD-RW may be a DVD-R or a Blu-ray (registered trademark) Disc, and the like. The media I / F 316 controls reading or writing (storing) of data from the recording medium 315 such as a flash memory. The microphone 318 is a type of built-in sound collecting means for inputting sound. The sound input / output I / F 317 is a circuit for processing input / output of sound signals between the microphone 318 and the speaker 319 under the control of the CPU 301. The short-range communication circuit 320 is a communication circuit for communicating with an external terminal (device) by short-range wireless communication technology such as NFC, Bluetooth, or Wi-Fi.

[0055] ○Hardware configuration of communication management system○ Fig. 12 is a diagram showing an example of the hardware configuration of the communication management system. Each piece of hardware configuration of the communication management system 50 is indicated by a reference number in parentheses in the 500 range. The communication management system 50 is constructed by a computer, and as shown in Fig. 12, has a similar configuration to the distribution terminal 30, so a description of each piece of hardware configuration will be omitted.

[0056] ○Hardware configuration of communication terminal○ Fig. 12 is a diagram showing an example of the hardware configuration of a communication terminal. Each piece of hardware configuration of the communication terminal 70 is indicated by a reference number in the 700 range in parentheses. The communication terminal 70 is constructed by a computer, and as shown in Fig. 12, has the same configuration as the distribution terminal 30, so a description of each piece of hardware configuration will be omitted.

[0057] ○Hardware configuration of the point of interest designation device○ Fig. 12 is a diagram showing an example of the hardware configuration of the point of interest designation device 20. Each piece of hardware configuration of the point of interest designation device 20 is indicated by a reference number in parentheses in the 200 range. The point of interest designation device 20 is constructed by a computer, and as shown in Fig. 12, has the same configuration as the distribution terminal 30, so a description of each piece of hardware configuration will be omitted.

[0058] Each of the above programs may be recorded in a computer-readable recording medium as an installable or executable file and distributed. Examples of the recording medium include a CD-R (Compact Disc Recordable), a DVD (Digital Versatile Disk), a Blu-ray Disc, an SD card, and a USB memory. The recording medium may be provided domestically or internationally as a program product. For example, the communication terminal 70 realizes the image display method according to the present invention by executing the program according to the present invention.

[0059] ●Function configuration Next, the functional configuration of the image communication system according to the embodiment will be described with reference to Fig. 13 to Fig. 21. Fig. 13 and Fig. 14 are diagrams showing an example of the functional configuration of the image communication system. Fig. 13 and Fig. 14 show devices or terminals shown in Fig. 9 that are related to the processing or operation described below.

[0060] ○Functional configuration of the imaging device○ First, the functional configuration of the photographing device 10 will be described with reference to Fig. 13. The photographing device 10 has a communication unit 11, a reception unit 12, an imaging unit 13, a sound collection unit 14, a determination unit 15, and a storage / readout unit 19. Each of these units is a function or means realized by operating any of the components shown in Fig. 11 by an instruction from the CPU 111 according to a program for the photographing device expanded from the SRAM 113 onto the DRAM 114. The photographing device 10 also has a storage unit 1000 constructed by the ROM 112, SRAM 113, and DRAM 114 shown in Fig. 11. The storage unit 1000 stores a GUID (Globally Unique Identifier) ​​of the own device.

[0061] The communication unit 11 is mainly realized by the processing of the CPU 111, and communicates various data or information with other devices or terminals. The communication unit 11 performs data communication with other devices or terminals by short-range wireless communication technology, for example, using a short-range communication circuit 117. The communication unit 11 also performs data communication with other devices or terminals via various cables, for example, using an input / output I / F 116. Furthermore, the communication unit 11 performs data communication with other devices or terminals via a communication network 100, using a network I / F 121.

[0062] The reception unit 12 is mainly realized by the processing of the CPU 111 on the operation unit 115, and receives various selections or inputs from the user. The imaging unit 13 is mainly realized by the processing of the CPU 111 on the imaging unit 101, the image processing unit 104, and the imaging control unit 105, and captures an image of a subject such as a landscape, and acquires captured image data. The sound collection unit 14 is mainly realized by the processing of the CPU 111 on the microphone 108 and the sound processing unit 109, and collects sounds around the imaging device 10. The judgment unit 15 is mainly realized by the processing of the CPU 111, and makes various judgments.

[0063] The storage / reading unit 19 is realized mainly by the processing of the CPU 111 , and stores various data (or information) in the storage unit 1000 and reads various data (or information) from the storage unit 1000 .

[0064] ○Functional configuration of the attention point designation device○ Next, the functional configuration of the point of interest designation device 20 will be described with reference to Fig. 13. The point of interest designation device 20 has a communication unit 21 and a detection unit 22. These units are functions or means realized by operating any of the components shown in Fig. 12 in response to an instruction from the CPU 201 in accordance with the program for the point of interest designation device loaded onto the RAM 203 from the HD 204.

[0065] The communication unit 21 is mainly realized by the processing of the CPU 201, and communicates various data or information with other devices or terminals. The communication unit 21 performs data communication with other devices or terminals by short-distance wireless communication technology, for example, using the short-distance communication circuit 220. The communication unit 21 also performs data communication with other devices or terminals via various cables, for example, using the external device connection I / F 208. Furthermore, the communication unit 21 performs data communication with other devices or terminals via the communication network 100, using the network I / F 209.

[0066] The detection unit 22 is mainly realized by the processing of the CPU 201, and detects the designation of a point of interest corresponding to a specific position where the point of interest designation device 20 is installed. For example, when an abnormality occurs at a specific position (point of interest) where the point of interest designation device 20 is installed, the detection unit 22 detects the designation of a point of interest indicating a location to which a user at a viewing base should pay attention. The detection method in the detection unit 22 may be a method in which the distribution base detects the occurrence of an abnormality or the like by image recognition of an image captured, or the occurrence of an abnormality or the like may be detected using various sensors provided in the point of interest designation device 20. In addition, the detection unit 22 may be configured to detect the designation of a point of interest when an operation of a dedicated button or the like provided in the point of interest designation device 20 is received by a worker or the like at the distribution base.

[0067] ○ Functional configuration of distribution terminal ○ Next, the functional configuration of the distribution terminal 30 will be described with reference to Fig. 13. The distribution terminal 30 has a transmission / reception unit 31, a reception unit 32, an image / sound processing unit 33, a display control unit 34, a determination unit 35, a creation unit 36, a communication unit 37, and a storage / readout unit 39. Each of these units is a function or means realized by operating any of the components shown in Fig. 12 by an instruction from the CPU 301 according to the program for the distribution terminal loaded from the HD 304 onto the RAM 303. The distribution terminal 30 also has a storage unit 3000 constructed by the ROM 302, RAM 303, and HD 304 shown in Fig. 12.

[0068] The transmission / reception unit 31 is realized mainly by the processing of the CPU 301 for the network I / F 309, and transmits and receives various data or information to and from other devices or terminals via the communication network 100.

[0069] The reception unit 32 is mainly realized by the processing of the CPU 301 on the keyboard 311 or the pointing device 312, and receives various selections or inputs from the user. The image / sound processing unit 33 is mainly realized by the processing of the CPU 301, and performs image processing on the captured image data acquired by the image capture device 10 capturing an image of a subject. In addition, the image / sound processing unit 33 performs audio processing on audio data related to the audio signal after the user's voice is converted into an audio signal by the microphone 318. Furthermore, the image / sound processing unit 33 performs image processing on the captured image data received from the image capture device 10 based on image type information such as a source name so that the display control unit 34 displays an image on the display 306. Specifically, when the image type information indicates that the image is a special image, the image / sound processing unit 33 creates omnidirectional panoramic image data by converting the captured image data (for example, data of each hemispherical image as shown in FIGS. 3(A) and (B)) into omnidirectional image data as shown in FIG. 4(B), and further creates a predetermined area image as shown in FIG. 6(B).

[0070] The display control unit 34 is mainly realized by the processing of the CPU 301, and displays various images, characters, etc. on the display 306. The determination unit 35 is realized by the processing of the CPU 301, and performs various determinations. The determination unit 35 determines, for example, the image type related to the captured image data received from the imaging device 10.

[0071] The creation unit 36 ​​is mainly realized by the processing of the CPU 301, and creates a source name, which is an example of image type information, in accordance with a naming rule based on the result of the determination by the determination unit 35 of whether the image is a general image or a special image (here, a spherical image). For example, if the determination unit 35 determines that the image is a general image, the creation unit 36 ​​creates a source name "Video" indicating that it is a general image. On the other hand, if the determination unit 35 determines that the image is a special image, the creation unit 36 ​​creates a source name "Video_Theta" indicating that it is a special image.

[0072] The communication unit 37 is mainly realized by processing of the CPU 301 for the short-range communication circuit 320, and communicates with the communication unit 11 of the photographing device 10 and the communication unit 21 of the attention point designation device 20 by a short-range wireless communication technology such as NFC, Bluetooth, or WiFi. Note that although the communication unit 37 and the transmission / reception unit 31 have been described as having separate communication units, they may be configured to share a common communication unit.

[0073] The storage / reading unit 39 is realized mainly by the processing of the CPU 301 , and stores various data (or information) in the storage unit 3000 and reads various data (or information) from the storage unit 3000 .

[0074] ○ Shooting device management table Fig. 15(A) is a conceptual diagram showing an example of a photographing device management table. A photographing device management DB 3001 configured by the photographing device management table shown in Fig. 15(A) is constructed in the storage unit 3000. This photographing device management table stores and manages the vendor ID and product ID of the GUID of the photographing device that can obtain two hemispherical images that are the basis of the omnidirectional image. As the GUID, for example, a vendor ID (VID) and a product ID (PID) used in a USB device can be used. For example, the vendor ID and the product ID may be stored at the time of shipment from the factory of the distribution terminal 30, or may be additionally stored after shipment from the factory.

[0075] Image type management table FIG. 15B is a conceptual diagram showing an example of an image type management table. In the storage unit 3000, an image type management DB 3002 configured by the image type management table shown in FIG. 15B is constructed. This image type management table manages image data IDs, IP (Internet Protocol) addresses which are an example of destinations of the photographing devices, and source names in association with each other. Among these, the image data ID is an example of image data identification information for identifying image data when performing image distribution. The IP address of the photographing device indicates the IP address of the photographing device 10 which photographed the image data indicated by the associated image data ID. The source name is a name for identifying the photographing device 10 which photographed the image data indicated by the associated image data ID, and is an example of image type information. This source name is a name created by the distribution terminal 30 according to a predetermined naming rule.

[0076] For example, it is shown that four image capture devices with IP addresses "2.1.2.3", "2.1.1.5", "2.1.5.4", and "2.1.5.6" transmit image data indicated by image data IDs "RS001", "RS002", "RS003", and "RS004", respectively. Furthermore, the types of images indicated by the source names of the image capture devices are "Video_Theta", "Video_Theta", "Video", and "Video", which indicate that the image types are "special image", "special image", "general image", and "general image", respectively. In this embodiment, the special image is a spherical image. Note that the IP address is an example of destination information, and the destination information may be a MAC (Media Access Control) address or a terminal ID (Identification), etc. Furthermore, the IP address is represented by simplifying the IPv4 address, but the IP address may be an IPv6 address. Furthermore, data other than image data may also be managed in association with the image data ID. Data other than image data may be, for example, sound data, material data when sharing a screen between a distribution base and a viewing base, etc.

[0077] ○Attention point management table Fig. 16 is a conceptual diagram showing an example of a focus point management table. A focus point management DB 3003 configured by the focus point management table shown in Fig. 16 is constructed in the storage unit 3000. This focus point management table manages focus point IDs that identify focus points at distribution bases and coordinate values ​​that indicate the positions of the focus points in association with each other.

[0078] ○Functional configuration of communication management system○ Next, the functional configuration of the communication management system 50 will be described with reference to Fig. 14. The communication management system 50 has a transmission / reception unit 51, a judgment unit 52, a generation unit 53, a distribution base management unit 54, and a storage / readout unit 59. Each of these units is a function or means realized by any of the components shown in Fig. 12 operating in response to an instruction from the CPU 501 in accordance with the program for the communication management system loaded from the HD 504 onto the RAM 503. The communication management system 50 also has a storage unit 5000 constructed by the ROM 502, RAM 503, and HD 504 shown in Fig. 12.

[0079] The transmission / reception unit 51 is realized mainly by the processing of the CPU 501 for the network I / F 509, and transmits and receives various data or information to and from other devices via the communication network 100.

[0080] The determination unit 52 is realized by the processing of the CPU 501, and performs various determinations. The generation unit 53 is mainly realized by the processing of the CPU 501, and generates image data ID and predetermined area information. The generation unit 53 generates, for example, predetermined area information indicating a predetermined area in an image captured by the imaging device 10. The distribution base management unit 54 is mainly realized by the processing of the CPU 501, and manages distribution base information indicating the status of the distribution base.

[0081] The storage / reading unit 59 is realized mainly by the processing of the CPU 501 , and stores various data (or information) in the storage unit 5000 and reads various data (or information) from the storage unit 5000 .

[0082] ○Session management table FIG. 17(A) is a conceptual diagram showing an example of a session management table. A session management DB 5001 configured by the session management table shown in FIG. 17(A) is constructed in the storage unit 5000. This session management table manages a session ID, a base ID, and an IP address of a participating communication terminal in association with each other. Among them, the session ID is an example of session identification information for identifying a communication session that realizes image communication, and is generated for each virtual conference room. The session ID is also managed by each communication terminal, and is used when selecting a communication session in each communication terminal. The base ID is an example of base identification information for identifying a distribution base. The IP address of a participating communication terminal indicates the IP address of a communication terminal that has participated in a virtual conference room indicated by the associated session ID.

[0083] Image type management table FIG. 17(B) is a conceptual diagram showing an example of an image type management table. In the storage unit 5000, an image type management DB 5002 configured by the image type management table shown in FIG. 17(B) is constructed. In addition to each piece of information managed in the image type management table shown in FIG. 15(B), this image type management table manages a session ID that is the same as the session ID managed in the session management table in association with each other. In addition, the communication management system 5 manages the same image data ID, the IP address of the photographing device, and the image type information managed in the distribution terminal 30 and the communication terminal 70 in order to transmit image type information, etc. to a communication terminal already in an image call and a newly participating communication terminal when a new communication terminal enters a virtual conference room, etc. This eliminates the need to transmit and receive image type information, etc. between a communication terminal already in a call and a newly participating communication terminal.

[0084] Prescribed area management table Fig. 18(A) is a conceptual diagram showing an example of a predetermined area management table. A predetermined area management DB 5003 configured by the predetermined area management table shown in Fig. 18(A) is constructed in the storage unit 5000. This predetermined area management table manages the IP address of the distribution terminal that is the transmission source of the photographed image data, the IP address of the communication terminal that is the transmission destination of the photographed image data, and predetermined area information indicating the predetermined area image being displayed on the communication terminal that is the transmission destination of the photographed image data in association with each other. Note that the communication terminal that is the transmission destination of the photographed image data is also the communication terminal that is the transmission source of the predetermined area information. The predetermined area information is a conversion parameter for converting the photographed image into an image of the predetermined area T in the photographed image (predetermined area image), as shown in Figs. 6 and 7.

[0085] For example, the first to fourth lines of the predetermined area management table in Fig. 18(A) manage that when the IP address of the streaming terminal 30 is "1.3.2.3", the captured image data transmitted from the streaming terminal 30 is transmitted to the communication terminal 70 having the IP address "1.2.1.3" via the communication management system 5. Furthermore, the first to fourth lines of the predetermined area management table in Fig. 18(A) manage that the streaming terminal 30 is the source of the four pieces of predetermined area information.

[0086] In addition, when the transmission / reception unit 51 receives new specified area information that includes the same set of IP addresses as the IP address of the distribution terminal that is the sender of the captured image data that is already managed and the IP address of the communication terminal that is the destination of the captured image data, the storage / reading unit 59 rewrites the specified area information already managed with the newly received specified area information.

[0087] ○Installation information management table FIG. 18(B) is a conceptual diagram showing an example of the installation information management table. In the storage unit 5000, an installation information management DB 5004 configured by the installation information management table shown in FIG. 18(B) is constructed. This installation information management table manages installation information in which a base ID for identifying a distribution base, an IP address of the camera device 10 installed at the distribution base, a coordinate value indicating the position where the camera device 10 is installed, and an installation direction of the camera device 10 are associated. Among these, the coordinate value indicates the coordinate value on a map of the distribution base where the camera device 10 is installed. Also, the installation direction of the camera device 10 indicates the direction on the map in which the front of the camera device 10 faces. Note that this installation information is set by an administrator of the distribution base or the distribution terminal 30.

[0088] ○Distribution base management table FIG. 19 is a conceptual diagram showing an example of a distribution base management table. A distribution base management DB 5005 configured by the distribution base management table shown in FIG. 19 is constructed in the storage unit 5000. This distribution base management table manages a base ID and a base name for identifying a distribution base, and distribution base information indicating the status of the distribution base in association with each other. Among them, the distribution base information includes a URL (Uniform Resource Locator) for accessing map image data indicating a map of the distribution base and base coordinate information indicating the coordinate value of the distribution base. The communication terminal 70 can obtain the map image data and base coordinate information of the distribution base by accessing the URL transmitted from the communication management system 50. The URL is an example of storage destination information. The storage destination information is not limited to a URL, and may be a URI (Uniform Resource Identifier) ​​or the like.

[0089] ○Functional configuration of communication terminal○ Next, the functional configuration of the communication terminal 70 will be described with reference to Fig. 14. The communication terminal 70 has a transmission / reception unit 71, a reception unit 72, an image / sound processing unit 73, a display control unit 74, a judgment unit 75, a creation unit 76, a focus point identification unit 77, a generation unit 78, and a storage / readout unit 79. Each of these units is a function or means realized by any of the components shown in Fig. 12 operating according to an instruction from the CPU 701 in accordance with the program for the communication terminal loaded from the HD 704 onto the RAM 703. The communication terminal 70 also has a storage unit 7000 constructed by the ROM 702, the RAM 703, and the HD 704 shown in Fig. 12.

[0090] The transmission / reception unit 71 is realized mainly by the processing of the CPU 701 for the network I / F 709, and transmits and receives various data or information to and from other devices or terminals via the communication network 100.

[0091] The reception unit 72 is mainly realized by the processing of the CPU 701 on the keyboard 711 or the pointing device 712, and receives various selections or inputs from the user. The image / sound processing unit 73 is mainly realized by the processing of the CPU 701, and realizes the same function as the image / sound processing unit 33. The display control unit 74 is mainly realized by the processing of the CPU 701, and causes various images or characters, etc. to be displayed on the display 706. The judgment unit 75 is mainly realized by the processing of the CPU 701, and makes various judgments.

[0092] The creation unit 76 is mainly realized by the processing of the CPU 701, and realizes the same function as the creation unit 36. The attention point identification unit 77 is mainly realized by the processing of the CPU 701, and identifies an attention point at the distribution base where the image capturing device 10 is installed. The attention point identification unit 77 identifies an attention point to be displayed on the display screen based on attention point designation information distributed from the distribution system 3 and attention point information stored in the attention point management DB 7003.

[0093] The generating unit 78 generates predetermined area information, and is mainly realized by processing of the CPU 701. The generating unit 78 generates predetermined area information indicating a predetermined area in a captured image captured by the image capturing device 10, for example.

[0094] The storage / reading unit 79 is realized mainly by the processing of the CPU 701 , and stores various data (or information) in the storage unit 7000 and reads various data (or information) from the storage unit 7000 .

[0095] Image type management table Fig. 20(A) is a conceptual diagram showing an example of an image type management table. An image type management DB 7001 configured by the image type management table shown in Fig. 20(A) is constructed in the storage unit 7000. This image type management table has the same data configuration as the image type management table shown in Fig. 15(B), so a description thereof will be omitted.

[0096] Prescribed area management table Fig. 20(B) is a conceptual diagram showing an example of a predetermined area management table. A predetermined area management DB 7002 configured by the predetermined area management table shown in Fig. 20(B) is constructed in the storage unit 7000. This predetermined area management table manages the IP address of the distribution terminal that is the transmission source of the captured image data, the IP address of the photographing device that captured the captured image data, and predetermined area information indicating the displayed predetermined area image in association with each other. As shown in Figs. 6 and 7, the predetermined area information is a conversion parameter for converting the captured image into an image of the predetermined area T in the captured image (predetermined area image).

[0097] ○Attention point management table FIG. 21 is a conceptual diagram showing an example of a focus point management table. In the storage unit 7000, a focus point management DB 7003 configured by the focus point management table shown in FIG. 21 is constructed. This focus point management table manages the IP address of the distribution terminal that is the transmission source of the captured image data, the IP address of the imaging device that captured the captured image data, and focus point area information indicating the predetermined area information for displaying each focus point in association with each other. Among them, the focus point area information includes, for each focus point, predetermined area information (radius r, polar angle θ, azimuth angle φ) indicating the direction and angle of view for displaying each focus point in the omnidirectional image. The focus point management table shown in FIG. 21 shows an example in which there are two focus points set in advance. For example, when a focus point is specified, the communication terminal 70 changes the image to be displayed to a predetermined area image in which the focus point appears by performing perspective projection transformation on the omnidirectional image distributed from the distribution terminal 30 that is the image transmission source based on the focus point area information stored in the focus point management table. The focus point area information is an example of first predetermined area information.

[0098] Processing or operation of the embodiment ○Session participation processing○ Next, the process or operation of the image communication system according to the embodiment will be described with reference to Fig. 22 to Fig. 28. First, the process of joining a specific communication session will be described with reference to Fig. 22 and Fig. 23. Fig. 22 is a sequence diagram showing an example of the process of joining a specific communication session in the image communication system. Fig. 23 is a diagram showing an example of a selection screen for a communication session.

[0099] First, a user at the viewing location A (e.g., user A1) performs an operation to display a selection screen for a communication session, and the reception unit 72 receives the operation to display the selection screen, whereby the display control unit 74 of the communication terminal 70A displays the selection screen 800 shown in Fig. 21 on the display 706 (step S11). The selection screen 800 shown in Fig. 21 displays selection buttons 810a, 810b, 810c, etc., which indicate each of the floors A1, B1, B2, etc., to be selected. Each selection button 810a, etc. is associated with a session ID.

[0100] Here, when user A1 selects the desired selection button (here, selection button 810a) of the virtual floor that is the distribution base, the reception unit 72 accepts the selection of the communication session (step S12). Then, the transmission / reception unit 71 transmits a participation request to the communication management system 50 for the communication session with the distribution base (step S13). This participation request includes a session ID indicating the communication session whose selection was accepted in step S12, and the IP address of the communication terminal 70A that is the request source terminal. As a result, the transmission / reception unit 51 of the communication management system 50 receives the participation request.

[0101] Next, the storage / read unit 59 of the communication management system 50 performs a process of participating in the communication session by adding the IP address received in step S13 to the participation terminal IP address field of the record of the same session ID as the session ID received in step S13 in the session management DB 5001 (see FIG. 17(A)), (step S14). The storage / read unit 59 also reads out the base ID associated with the session ID of the communication session to which the participation process has been performed in the session management DB 5001. Next, the storage / read unit 59 searches the distribution base management DB 5005 (see FIG. 19) using the base ID read out in step S14 as a search key, thereby reading out distribution base information associated with the same base ID as the read out base ID (step S15). Then, the transmission / reception unit 51 transmits a participation request response to the communication terminal 70A (step S16). This participation request response includes the session ID received in step S13, the distribution base information read out in step S15, and the participation process result. The participation process result includes the location ID read out in step S14. As a result, the transmitter / receiver 71 of the communication terminal 70A receives the participation request response. Hereinafter, a case where the participation process is successful will be described.

[0102] ○Image type information management process○ Next, the management process of image type information will be described with reference to Fig. 24. Fig. 24 is a sequence diagram showing the management process of image type information.

[0103] First, when a user at a distribution base connects the photographing device 10 to the distribution terminal 30, the storage / readout unit 19 of the photographing device 10 reads out the GUID of its own device (the photographing device 10) stored in the storage unit 1000. Then, the communication unit 11 of the photographing device 10 transmits the GUID of its own device to the distribution terminal 30 (step S31). As a result, the communication unit 37 of the distribution terminal 30 receives the GUID of the photographing device 10.

[0104] Next, the determination unit 35 of the distribution terminal 30 determines whether the same vendor ID and product ID as the vendor ID and product ID in the GUID received in step S31 are managed in the photographing device management DB 3001 (see FIG. 15(A)), thereby determining the image type (step S32). Specifically, if the same vendor ID and product ID are managed in the photographing device management DB 3001, the determination unit 35 determines that the photographing device 10 is a photographing device that captures special images (here, spherical images). On the other hand, if the same vendor ID and product ID are not managed in the photographing device management DB 3001, the determination unit 35 determines that the photographing device 10 is a photographing device that captures general images.

[0105] Next, the storage / reading unit 39 stores the IP address of the photographing device 10 in association with the image type information determined in step S32 in the image type management DB 3002 (see FIG. 15(B)) (step S33). In this state, no image data ID is associated. The image type information is, for example, a source name or image type (general image, special image) determined according to a predetermined naming rule.

[0106] Next, the transmitting / receiving unit 31 transmits a request to add image type information to the communication management system 50 (step S34). This request to add image type information includes the IP address of the imaging device 10, the location ID of the distribution location, and the image type information stored in step S33. As a result, the transmitting / receiving unit 51 of the communication management system 50 receives the request to add image type information.

[0107] Next, the storage / read unit 59 of the communication management system 50 searches the session management DB 5001 (see FIG. 17(A)) using the location ID received in step S34 as a search key, and reads out the corresponding session ID (step S35).

[0108] Next, the generating unit 53 generates a unique image data ID (step S36). Then, the storing / reading unit 59 stores the session ID read out in step S35, the image data ID generated in step S36, and the IP address and image type information of the photographing device 10 received in step S34 in association with each other as a new record in the image type management DB 5002 (see FIG. 17(B)) (step S37). Then, the transmitting / receiving unit 51 transmits the image data ID generated in step S36 to the distribution terminal 30 (step S38). As a result, the transmitting / receiving unit 31 of the distribution terminal 30 receives the image data ID.

[0109] Next, the storage / reading unit 39 of the distribution terminal 30 stores the image data ID received in step S38 in the image type management DB 3002 (see Figure 15 (B)) in association with the IP address of the photographing device 10 and the image type information stored in step S33 (step S39).

[0110] Meanwhile, the transmitting / receiving unit 51 of the communication management system 50 transmits an addition notification of the image type information to the communication terminal 70A (step S40). This addition notification of the image type information includes the image data ID generated in step S36, and the IP address and image type information of the photographing device 10 stored in step S37. As a result, the transmitting / receiving unit 71 of the communication terminal 70A receives the addition notification of the image type information.

[0111] Next, the storage / readout unit 79 of the communications terminal 70A associates the image data ID, the IP address of the photographing device 10, and the image type information received in step S40 and stores them as a new record in the image type management DB 7001 (see FIG. 20(A)) (step S41). As described above, the distribution terminal 30 and the communications terminal 70A can share the same information in the image type management DBs 3002 and 7001. Similarly, an additional communication of image type information is transmitted to another communications terminal, communications terminal 70B, and the image type information is also stored in the image type management DB 7001 of communications terminal 70B.

[0112] ○Communication processing of captured image data○ Next, a process in which photographed image data and sound data obtained at a distribution base are transmitted to each communication terminal (communication terminals 70A, 70B) via the communication management system 50 will be described with reference to Figs. 25 and 26. Fig. 25 is a sequence diagram showing an example of communication processing of photographed image data and sound data in an image communication system. Note that Fig. 25 shows an example in which photographed images obtained by one photographing device 10 are displayed on each communication terminal 70, but similar processing is also performed when multiple photographed images obtained by other photographing devices 10 installed at a distribution base are displayed.

[0113] First, the communication unit 11 of the photographing device 10 transmits photographed image data obtained by photographing a subject, scenery, etc., and sound data obtained by collecting sound to the distribution terminal 30 (step S51). In this case, since the photographing device 10 is a device capable of obtaining two hemispherical images that are the basis of a celestial sphere image, the photographed image data is composed of data of the two hemispherical images, as shown in Figures 3(A) and (B). As a result, the communication unit 37 of the distribution terminal 30 receives the photographed image data and sound data.

[0114] Next, the transmitting / receiving unit 31 of the distribution terminal 30 transmits the captured image data and sound data sent from the photographing device 10 to the communication management system 50 (step S52). This transmission includes an image data ID for identifying the captured image data to be transmitted. As a result, the transmitting / receiving unit 51 of the communication management system 50 receives the captured image data and sound data, along with the image data ID.

[0115] Next, the transmitting / receiving unit 51 of the communication management system 50 transmits the captured image data and sound data to the communication terminals (communication terminals 70A and 70B) participating in the same session as the distribution terminal 30 (steps S53 and S55). Each of these transmissions includes an image data ID for identifying the captured image data to be transmitted. As a result, the transmitting / receiving units 71 of the communication terminals 70A and 70B each receive the captured image data and sound data, as well as the image data ID. Then, the display control units 74 of the communication terminals 70A and 70B each display the captured image on the display 706 based on the various received data (steps S54 and S56).

[0116] Here, a process of displaying a predetermined area image on the communication terminal 70 will be described with reference to Fig. 26. Fig. 26 is a sequence diagram showing an example of a display process of a predetermined area image. Note that Fig. 26 describes a process of displaying a predetermined area image on the communication terminal 70A, but the same process is also performed when displaying a predetermined area image on another communication terminal 70 (for example, communication terminal 70B).

[0117] First, the user A1 performs a predetermined input operation on the spherical image, which is a captured image displayed on the communication terminal 70A, and the reception unit 72 of the communication terminal 70A receives the selection of a predetermined display position (step S71). Next, the transmission / reception unit 71 transmits a predetermined area information acquisition request indicating a request to acquire predetermined area information to the communication management system 50 (step S72). This predetermined area information acquisition request includes the position coordinates of the selected position received in step S71. As a result, the transmission / reception unit 51 of the communication management system 50 receives the predetermined area information acquisition request transmitted from the communication terminal 70A.

[0118] Next, the memory / read unit 59 of the communication management system 50 searches the installation information management DB 5004 (see FIG. 18) using the IP address of the imaging device 10 received in step S52 as a search key, and reads out the installation information associated with the same IP address as the received IP address (step S73).

[0119] Then, the generating unit 53 generates predetermined area information corresponding to the position coordinates received in step S72 (step S74). Specifically, the generating unit 53 generates predetermined area information for displaying a predetermined area image (perspective projection transformation image) with the received position coordinates at the center of the omnidirectional image. Here, a predetermined value set in advance is used for the radius (r) and the polar angle (θ). This indicates that the angle of view and the elevation angle of the perspective projection transformation image are processed as constant. The azimuth angle (φ) can be calculated as a relative value between the installation direction of the image capturing device 10 indicated in the installation information read out in step S73 and the direction of the position coordinates from the image capturing device 10. Then, the generating unit 53 generates predetermined area information for displaying (perspective projection display) a predetermined area image centered on the position coordinates from the omnidirectional image acquired by the image capturing device 10 based on the calculation result. The installation position (coordinate values) and installation direction (the direction in which the front of the camera device 10 faces) of the camera device 10 are set in advance in the installation information management DB 5004 by an administrator or the like.

[0120] Next, the storage / reading unit 59 stores the predetermined area information generated in step S74 in the predetermined area management DB 5003 (see FIG. 18(A)) in association with the IP address of the imaging device 10 (step S75). Then, the transmitting / receiving unit 51 transmits the predetermined area information generated in step S74 to the communication terminal 70A (step S76). As a result, the transmitting / receiving unit 71 of the communication terminal 70A receives the predetermined area information transmitted from the communication management system 50.

[0121] Next, the storage / reading unit 79 of the communications terminal 70A stores the predetermined area information received in step S76 in the predetermined area management DB 7002 (see FIG. 20(B)) in association with the IP address of the imaging device 10 (step S77).

[0122] Next, the user A1 performs a predetermined input operation, and the reception unit 72 receives a selection of a display image to be displayed on the communication terminal 70A from the captured image data received in step S53 (step S78). Note that if the communication terminal 70A is capable of simultaneously displaying multiple captured images, or if the number of received captured images is less than the number that can be simultaneously displayed on the communication terminal 70A, the process of step S78 may be omitted.

[0123] Next, the image / sound processing unit 73 generates a predetermined area image by performing perspective projection transformation using the received predetermined area information in order to display an image of the predetermined area specified by the predetermined area information received in step S76 (step S78). Then, the display control unit 74 causes the display 706 to display the predetermined area image generated in step S78 (step S79). This allows the communication terminal 70A to display the predetermined area image including the display position selected by the user A1 in the displayed spherical image.

[0124] ○ Display processing of points of interest ○ Next, a process of displaying an image of a point of interest at a distribution base on the communication terminal 70 will be described with reference to Fig. 27 and Fig. 28. Fig. 27 is a sequence diagram showing an example of a display process of a point of interest in an image communication system. Fig. 27 shows a process in the case where a captured image is displayed on the communication terminal 70 (70A, 70B) by the process shown in Fig. 25.

[0125] First, the detection unit 22 of the attention point designation device 20 detects the designation of the attention point (step S103). In this case, the detection unit 22 detects the designation of the attention point when, for example, an abnormality or an event that should be noted has occurred at the distribution base based on the detection results of image recognition or various sensors. In addition to the above-mentioned automatic detection process, the detection unit 22 may be configured to detect the designation of the attention point by accepting an operation of a dedicated button or the like provided on the attention point designation device 20 by an operator or the like at the distribution base.

[0126] Next, the communication unit 21 transmits attention point designation information to the distribution terminal 30 (step S102). This attention point designation information includes a attention point ID that identifies the attention point designation device 20. As a result, the communication unit 37 of the distribution terminal 30 receives the attention point designation information transmitted from the attention point designation device 20.

[0127] Next, the transmitting / receiving unit 31 of the distribution terminal 30 transmits the point of interest designation information transmitted from the point of interest designation device 20 to the communication management system 50 (step S103). As a result, the transmitting / receiving unit 51 of the communication management system 50 receives the point of interest designation information.

[0128] Next, the transmitting / receiving unit 51 of the communication management system 50 transmits the attention point designation information to the communication terminals (communication terminals 70A and 70B) participating in the same session as the distribution terminal 30 (steps S104 and S106). As a result, the transmitting / receiving units 71 of the communication terminals 70A and 70B each receive the attention point designation information. Then, the communication terminals 70A and 70B use the received attention point designation information to execute a display process of the attention point (steps S105 and S107).

[0129] Here, the display processing of the attention point in the communication terminals 70A and 70B will be described in detail with reference to Fig. 28. Fig. 28 is a flowchart showing an example of the display processing of the attention point in the communication terminal. Note that, since the processing in the communication terminals 70A and 70B is similar, Fig. 28 will be described as the processing in the communication terminal 70.

[0130] First, when the transmitting / receiving unit 71 receives attention point designation information, the determining unit 75 determines whether or not to change the display on the display screen (step S201). Specifically, the determining unit 75 determines whether or not to accept the display change by receiving a predetermined input operation by the user through the receiving unit 72. In this way, the image communication system 1 can have a communication terminal that changes the display for the attention point and a communication terminal that maintains the current display state without changing the display coexist. Therefore, when the user does not want to change the display, such as when the user is in the middle of viewing another location in the distribution center, the user can continue viewing another area of ​​the omnidirectional image without changing the display on the communication terminal 70, even when attention point designation information is received.

[0131] When the judgment unit 75 judges that the display change of the display screen is to be accepted (YES in step S201), the process proceeds to step S202. On the other hand, when the judgment unit 75 judges that the display change of the display screen is not to be accepted (NO in step S201), the process ends without making the display change.

[0132] Next, the attention point identification unit 77 searches the attention point management DB 7003 (see FIG. 21) using the IP address of the photographing device 10 that photographed the displayed photographed image as a search key, and reads and acquires the attention point area information associated with the same IP address as the IP address of the photographing device 10 via the storage / readout unit 79 (step S202). In this case, the attention point identification unit 77 acquires, from the attention point area information, attention point area information corresponding to the attention point ID included in the received attention point designation information.

[0133] Next, the reception unit 72 receives a selection of a display image to be displayed from the attention area information acquired in step S76 by a predetermined input operation by the user A1 (step S203). Note that if the communication terminal 70 is capable of simultaneously displaying a plurality of captured images, or if the number of received captured image data is less than the number that can be simultaneously displayed by the communication terminal 70, the process of step S203 may be omitted.

[0134] Next, the image / sound processing unit 73 generates a predetermined area image by performing perspective projection transformation using the attention point area information corresponding to the display image selected in step S203 (step S204). Then, the display control unit 74 changes the display of the display 706 to the predetermined area image generated in step S204 (step S205). For example, the display control unit 74 switches the display from the predetermined area image displayed in step S79 (e.g., the second predetermined area image) to the predetermined area image generated in step S204 (e.g., the first predetermined area image).

[0135] The communication terminal 70 executes such a process for all the displayed captured images. In this case, the communication terminal 70 acquires attention point area information for each of all the captured images currently being displayed (step S202), thereby changing the state so that the attention point is displayed in all the captured images.

[0136] In this way, the communication terminal 70 acquires the position of the attention point designated by the attention point designation device 20 based on preset attention point area information, and displays the predetermined area image generated so that the attention point designated based on the acquired attention point area information is displayed. In this way, the communication terminal 70 can display an image including the attention point without performing an operation on the imaging device 10 by changing the display position of the omnidirectional image so as to display the designated attention point.

[0137] Conventionally, when using a camera that captures a normal two-dimensional image, it was necessary to physically change the orientation of the camera to display the point of interest on the communication terminal. In this case, the image displayed on the other communication terminal also displays the point of interest. Therefore, the image communication system 1 uses a celestial sphere image captured by the camera 10, so that the communication terminal 70 can display only an area of ​​the celestial sphere image that captures the point of interest, and the other communication terminals 70 can display a different area.

[0138] ○Modification of the display process of the point of interest○ Next, a modified example of the display process of a point of interest in the image communication system 1 will be described with reference to Fig. 29. Fig. 29 is a sequence diagram showing a modified example of the display process of a point of interest in the image communication system. Fig. 29 shows an example of the case where a point of interest designation device 20 is not provided at a distribution base and a communication terminal 70 is caused to display the point of interest at the distribution base.

[0139] First, the communication unit 11 of the photographing device 10 transmits photographed image data and sound data to the distribution terminal 30 (step S301). As a result, the transmission / reception unit 31 of the distribution terminal 30 receives the photographed image data and sound data transmitted from the photographing device 10.

[0140] The determination unit 35 of the distribution terminal 30 determines whether or not a point of interest is designated based on the acquired captured image data (step S302). Specifically, the determination unit 35 determines to designate a point of interest when, for example, a parameter such as a degree of coincidence or similarity between a reference image of a preset distribution base and the acquired captured image is outside a predetermined range. In addition, the determination unit 35 determines, for example, a position of a celestial sphere image whose parameter is outside a predetermined range as a point of interest to be designated. Note that, hereinafter, a case where the determination unit 35 determines to designate a point of interest will be described.

[0141] Next, when it is determined in step S302 that a point of interest is to be designated, the storage / reading unit 39 reads out point of interest designation information stored in the point of interest management DB 3003 based on the coordinate value determined as the point of interest to be designated (step S303). Specifically, the storage / reading unit 39 reads out, as the point of interest designation information, the point of interest ID associated with the coordinate value closest to the coordinate value of the point of interest to be designated among the coordinate values ​​stored in the point of interest management DB 3003.

[0142] Then, the transmitting / receiving unit 31 transmits the captured image data and sound data received in step S301, and the point of interest designation information read out in step S303, to the communication management system 50 (step S304). As a result, the transmitting / receiving unit 51 of the communication management system 50 receives the captured image data, sound data, and point of interest designation information distributed from the distribution terminal 30. Since the subsequent processes of steps S305 to S308 are similar to the processes of steps S104 to S107 in Fig. 27, their explanations will be omitted.

[0143] In this way, even if the distribution base does not have attention point designation device 20 installed, the image communication system according to the modified embodiment has the distribution device determine whether or not a point of interest has been designated and the location of the point of interest based on the content of the captured image, thereby displaying a noteworthy area on the communication terminal 70. As a result, even if the image communication system 1 does not have attention point designation device 20, it is possible to specify a point of interest using the distribution terminal 30, and therefore the user at the viewing base can view the specified point of interest.

[0144] Effect of the embodiment As described above, when a point of interest is specified by the distribution system 3 for the communication terminal 70 at the viewing point, the image communication system 1 can display an image of the specified point of interest on the communication terminal 70 without performing any operation on the imaging device 10 at the distribution point.

[0145] Furthermore, the image communication system 1 cuts out and displays only an area of ​​the omnidirectional image that represents a point of interest on the communication terminal 70. Therefore, even if another area is displayed on another communication terminal 70, the other communication terminal 70 can continue to display an arbitrary area of ​​the omnidirectional image without changing the display.

[0146] In the above embodiment, the predetermined area T is specified by the predetermined area information indicating the shooting direction and the angle of view of the virtual camera IC in a three-dimensional virtual space including the omnidirectional image CE, but is not limited thereto. For example, in a case where the angle of view is kept constant, the predetermined area T may be specified by the predetermined point information indicating the center point CP or any point at each of the four corners of the rectangular predetermined area T in FIG. 7. In the above embodiment, the case where the captured image (whole image) is a three-dimensional panoramic image has been described as an example of the omnidirectional image (omnidirectional panoramic image), but the captured image may be a two-dimensional panoramic image.

[0147] In the above embodiment, the communication terminal 70 at the viewing location does not need to be a dedicated viewing terminal, and may be configured to distribute captured images from the terminal itself, and perform both the distribution function and the viewing function simultaneously. Similarly, the distribution terminal 30 at the distribution location does not need to be a dedicated distribution terminal, and may be configured to display captured images distributed from other locations, and perform both the distribution function and the viewing function simultaneously. In this way, the image communication system 1 may be configured to perform two-way communication of captured images between multiple locations.

[0148] ●Summary● As described above, the communication terminal according to an embodiment of the present invention is a communication terminal 70 capable of displaying an image of a predetermined area in an entire image (for example, a spherical image), and includes a focus point management DB 7003 (an example of a storage means) that stores focus point area information (an example of first predetermined area information) indicating a predetermined area (an example of a first predetermined area) including a focus point at a distribution base where the entire image was captured, a transmission / reception unit 71 (an example of a receiving means) that receives focus point designation information indicating the focus point from a distribution system 3 installed at the distribution base, and a display control unit 74 (an example of a display control means) that displays a predetermined area image including the focus point (an example of a first predetermined area image) based on the focus point area information corresponding to the focus point indicated in the received focus point designation information. This allows the communication terminal 70 to display an image of the distribution base corresponding to the focus point designated by the distribution system 3 without performing processing on the imaging device 10.

[0149] Furthermore, the communication terminal according to an embodiment of the present invention displays a second predetermined area image that is an image of a second predetermined area in an entire image (for example, a spherical image), and when attention point designation information is received, changes the display from the second predetermined area image to a predetermined area image including the attention point (an example of a first predetermined area image). This allows the communication terminal 70 to change the area in the entire image to be displayed to an area including the attention point designated by the distribution system 3.

[0150] Furthermore, the communication terminal according to an embodiment of the present invention includes a determination unit 75 (an example of a first determination means) that determines whether or not to change the display when attention point designation information is received, and when it is determined that the display should be changed, changes the display from the second predetermined area image to a predetermined area image (an example of a first predetermined area image) including the attention point. This allows the communication terminal 70 to switch between making a display change to the attention point and continuing the current display state as is without making a display change, even when attention point designation information is received.

[0151] Moreover, the image communication system according to an embodiment of the present invention is an image communication system 1 including a communication terminal 70 and a delivery system 3. The delivery system 3 includes a photographing device 10 (an example of an acquisition means) that acquires an entire image (for example, a celestial sphere image) of a delivery base, and a transmitting / receiving unit 31 (an example of a transmission means) that transmits the acquired entire image to the communication terminal 70. The delivery system 3 also includes a focus point designation device 20 that designates a focus point, and transmits focus point designation information indicating the focus point designated by the focus point designation device 20 to the communication terminal 70. As a result, the image communication system 1 allows a user at a viewing base to view an image of the delivery base corresponding to the focus point by displaying a predetermined area image including the focus point designated by the focus point designation device 20 on the communication terminal 70. Moreover, the image communication system 1 allows a plurality of communication terminals 70 to display images of different areas by performing a process for the communication terminal 70 to display the focus point using the focus point designation information transmitted from the delivery system 3.

[0152] Furthermore, in the image communication system according to one embodiment of the present invention, the delivery system 3 includes a determination unit 35 (an example of a second determination means) that determines whether or not a point of interest is designated based on the acquired whole image (for example, a spherical image), and when it is determined that a point of interest is to be designated, transmits point of interest designation information indicating the point of interest to the communication terminal 70. As a result, the image communication system 1 can designate a point of interest using the delivery system 3, and therefore allows a user at a viewing location to view the designated point of interest.

[0153] ●Additional Information● Each function of the above-described embodiment can be realized by one or more processing circuits. Here, the "processing circuit" in the present embodiment includes a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, as well as devices such as an ASIC (Application Specific Integrated Circuit), a DSP (digital signal processor), an FPGA (field programmable gate array), a SOC (System on a chip), a GPU (Graphics Processing Unit), and a conventional circuit module designed to execute each function described above.

[0154] In addition, the various tables in the above-described embodiment may be generated by the learning effect of machine learning, and the data of each associated item may be classified by machine learning, making it unnecessary to use tables. Here, machine learning is a technology for making a computer acquire human-like learning capabilities, and refers to a technology in which a computer autonomously generates an algorithm required for judgment such as data identification from learning data previously acquired, and applies this to new data to make predictions. The learning method for machine learning may be any of supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, and deep learning, or may be a combination of these learning methods, and any learning method for machine learning may be used.

[0155] So far, we have described a communication terminal, an image communication system, an image display method, and a program according to one embodiment of the present invention. However, the present invention is not limited to the above-described embodiment, and can be modified within the scope of what a person skilled in the art can conceive, such as adding, modifying, or deleting other embodiments. In any aspect, as long as the functions and effects of the present invention are achieved, it is included in the scope of the present invention. [Explanation of symbols]

[0156] 1. Image communication system 3. Distribution System 10. Photographing device (an example of acquisition means) 20 Point of interest designation device 30 Distribution terminal 31 Transmitting / receiving unit (an example of a transmitting means) 35 Determination unit (an example of a second determination means) 50 Communication Management System 70 Communication terminal 71 Transmitting / receiving unit (an example of a receiving means) 74 Display control unit (an example of a display control means) 75 Judgment unit (an example of a first judgment means) 7003 Point of interest management DB (an example of storage means) [Prior art documents] [Patent documents]

[0157] [Patent Document 1] JP 2017-22587 A

Claims

1. An image communication system including a communication terminal capable of displaying an image of a predetermined region in an entire image, and a distribution system installed at a distribution base where the entire image is captured, The communication terminal includes: A storage means for storing first predetermined area information indicating a first predetermined area including a point of interest in the distribution base; a receiving means for receiving, from the distribution system, attention point designation information indicating the attention point; a display control means for displaying a first predetermined area image based on the first predetermined area information corresponding to the attention point indicated in the attention point designation information received by the receiving means; Equipped with The distribution system includes: An acquisition means for acquiring the entire image of the distribution center; a second determination means for determining whether or not the attention point has been designated based on the entire image acquired by the acquisition means; a transmitting means for transmitting the whole image acquired by the acquiring means to the communication terminal, and when it is determined that the attention point is to be designated, transmitting attention point designation information indicating the attention point to the communication terminal; Equipped Image communication system.

2. The image communication system according to claim 1, wherein the display control means displays a second predetermined area image which is an image of a second predetermined area in the overall image, and when the attention point designation information is received, changes the display from the second predetermined area image to the first predetermined area image.

3. 3. The image communication system according to claim 2, further comprising: a first determination means for determining whether or not a display change has occurred when the attention point designation information is received, The display control means changes the display from the second predetermined area image to the first predetermined area image when it is determined that the display change should be performed.

4. 4. The image communication system according to claim 1, wherein the display control means displays a plurality of first predetermined area images which are images of the first predetermined area in an entire image captured by different image capturing devices.

5. The image communication system according to claim 1 , wherein the whole image is a spherical image.

6. 6. The image communication system according to claim 1, further comprising: a point of interest designation device for designating the point of interest, The transmitting means transmits, to the communication terminal, the attention point designation information indicating the attention point designated by the attention point designation device.

7. An image display method executed by an image communication system including a communication terminal capable of displaying an image of a predetermined region in an entire image and a distribution system installed at a distribution base where the entire image is captured, the method comprising: the communication terminal includes a storage means for storing first predetermined area information indicating a first predetermined area including a point of interest in a distribution base where the entire image was captured; The distribution system includes an acquisition means for acquiring the entire image of the distribution base, a second determination means for determining whether or not the attention point is designated based on the entire image acquired by the acquisition means, and a transmission means for transmitting the entire image acquired by the acquisition means to the communication terminal, and transmitting attention point designation information indicating the attention point to the communication terminal when it is determined that the attention point is designated, a receiving step in which the communication terminal receives the attention point designation information from the distribution system; a display control step of displaying a first predetermined area image on the communication terminal based on the first predetermined area corresponding to the attention point indicated in the attention point designation information received in the receiving step; Perform the image display method.

Images