Communication terminal, image communication system, image display method, and program

Abstract

To display an image in an appropriate display direction according to the reproduction time of video data shot at another base.SOLUTION: A communication terminal 30 capable of displaying a predetermined region of part of a subject appearing in a captured image receives, from a communication management system 50 that manages a captured image distributed from another communication terminal 30, video data including the captured image, and predetermined region information indicating a predetermined region to be displayed by the communication terminal 30 for each reproduction time of the video data, and when there is predetermined region information corresponding to the reproduction time of the received video data, displays an image indicating the predetermined region indicated in the received predetermined region information.SELECTED DRAWING: Figure 30

Description

Technical Field

[0001] The present disclosure relates to a communication terminal, an image communication system, an image display method, and a program.

Background Art

[0002] There is known a photographing device capable of photographing the entire range of directions using a plurality of wide-angle lenses or fisheye lenses. Further, there is known a system in which image data photographed using such a photographing device is distributed in real time, and the situation of a distribution base where the photographing device is installed can be viewed in real time at another base.

[0003] In addition, a photographing device capable of acquiring a full-sphere panoramic image in real time is connected to a communication terminal, and the full-sphere panoramic image acquired from this photographing device is transmitted to each communication terminal of the other party. At each communication terminal of the other party, a predetermined region image indicating a predetermined region that is a part of the full-sphere panoramic image is displayed on a display or the like. Thereby, the users at each base can independently determine and display a predetermined region image indicating a predetermined region that they are interested in among the entire image of the full-sphere panoramic image.

[0004] Furthermore, in the distribution of a full-sphere panoramic image, since the viewer can freely change the display direction, the viewer can view the image in different display directions. For example, Patent Document 1 discloses a configuration in which, for the purpose of reproducing image data reflecting various production intentions of a user from omnidirectional image data, the development method and the angle of view of the omnidirectional image data are changed, and the recording time data at the time of the change is stored and reproduced.

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in the conventional method, in the distribution of video data, there is a problem that it is impossible to display an appropriate predetermined area on the communication terminal according to the requests of the distributor or the viewer, such that in a certain time period, the viewer can freely operate the display direction to be shown on the communication terminal, and in a certain time period, the display direction to be shown on the communication terminal is synchronized.

Means for Solving the Problem

[0006] In order to solve the above-described problem, the invention according to claim 1 is a communication terminal capable of displaying a predetermined area of a part of a subject shown in a photographed image, and includes a receiving means for receiving video data including the photographed image and predetermined area information indicating a predetermined area to be displayed on the communication terminal for each reproduction time of the video data from a communication management system that manages the photographed image distributed from another communication terminal, and a display control means for displaying an image indicating the predetermined area indicated by the received predetermined area information when there is predetermined area information corresponding to the reproduction time of the received video data. When there is no predetermined area information at the playback time of the received video data, reception means for receiving a selection of a predetermined area where the received video data is to be displayed is provided, and the display control means displays a predetermined area image indicating the predetermined area for which the selection has been received. It is a communication terminal.

Effect of the Invention

[0007] According to the present invention, there is an effect that an image with an appropriate display direction can be displayed according to the reproduction time of video data photographed at another site.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Figure 29

Figure 30

Figure 31

Figure 32

Figure 33

Embodiments for Carrying Out the Invention

[0009] Hereinafter, embodiments for carrying out the invention will be described with reference to the drawings. In the description of the drawings, the same reference numerals are assigned to the same elements, and redundant descriptions are omitted.

[0010] ●Embodiment● ●Method for Generating an Omnidirectional Image The method for generating an omnidirectional image will be described with reference to FIGS. 1 to 8.

[0011] First, the appearance of the imaging device 10 will be described with reference to FIG. 1. The imaging device 10 is a digital camera for obtaining a captured image that serves as the basis for a panoramic (360°) image. FIG. 1(A) is a left side view of the imaging device, FIG. 1(B) is a front view of the imaging device, and FIG. 1(C) is a plan view of the imaging device.

[0012] As shown in FIG. 1(A), the imaging device 10 is sized such that a person can hold it with one hand. Also, as shown in FIGS. 1(A), 1(B), and 1(C), on the upper part of the imaging device 10, an imaging element 103a is provided on the front side (front) and an imaging element 103b is provided on the back side (rear). These imaging elements (image sensors) 103a and 103b are used in combination with an optical member (for example, lenses 102a and 102b described later) capable of capturing a hemispherical image (angle of view of 180° or more). Further, as shown in FIG. 1(B), an operation unit 115 such as a shutter button is provided on the surface of the imaging device 10 opposite to the front side.

[0013] Next, the usage situation of the imaging device 10 will be described with reference to FIG. 2. FIG. 2 is an image diagram showing the usage of the imaging device. As shown in FIG. 2, the imaging device 10 is used, for example, when a user holds it in their hand to capture a subject around the user. In this case, by the imaging element 103a and the imaging element 103b shown in FIG. 1, subjects around the user are respectively imaged, and two hemispherical images can be obtained.

[0014] Next, with reference to FIGS. 3 and 4, an overview of the process from the image captured by the imaging device 10 to the creation of a panoramic image will be described. FIG. 3(A) is a hemispherical image (front side) captured by the imaging device, FIG. 3(B) is a hemispherical image (rear side) captured by the imaging device, and FIG. 3(C) is a diagram showing an image represented by the orthographic cylindrical projection method (hereinafter referred to as "orthographic cylindrical projection image"). FIG. 4(A) is a conceptual diagram showing a state where a sphere is covered with the orthographic cylindrical projection image, and FIG. 4(B) is a diagram showing a panoramic image.

[0015] As shown in FIG. 3(A), the image obtained by the imaging device 103a becomes a hemispherical image (front side) curved by the lens 102a described later. Also, as shown in FIG. 3(B), the image obtained by the imaging device 103b becomes a hemispherical image (rear side) curved by the lens 102b described later. Then, the imaging device 10 synthesizes the hemispherical image (front side) and the hemispherical image (rear side) inverted by 180 degrees to create an orthographic cylindrical projection image EC as shown in FIG. 3(C).

[0016] Then, by using OpenGL ES (Open Graphics Library for Embedded Systems), the imaging device 10 pastes the orthographic cylindrical projection image EC so as to cover the spherical surface as shown in FIG. 4(A) to create an omnidirectional image (omnidirectional panoramic image) CE as shown in FIG. 4(B). Thus, the omnidirectional image CE is represented as an image in which the orthographic cylindrical projection image EC faces the center of the sphere. Note that OpenGL ES is a graphics library used to visualize 2D (2-Dimensions) and 3D (3-Dimensions) data. Also, the omnidirectional image CE may be a still image or a moving image.

[0017] As described above, since the omnidirectional image CE is an image pasted so as to cover the spherical surface, it will feel uncomfortable to humans. Therefore, the imaging device 10 can display a part of a predetermined region T of the omnidirectional image CE (hereinafter referred to as a "predetermined region image") as a planar image with less curvature, so as not to give an uncomfortable feeling to humans. This will be described with reference to FIGS. 5 to 8.

[0018] FIG. 5 is a diagram showing the position of a virtual camera and a predetermined area when the omnidirectional image is a three-dimensional spherical object. The virtual camera IC corresponds to the position of the viewpoint of a user who views the omnidirectional image CE displayed as a three-dimensional spherical object. FIG. 6(A) is a perspective view of FIG. 5, and FIG. 6(B) is a diagram showing a predetermined area image when displayed on a display. FIG. 6(A) represents the omnidirectional image CE shown in FIG. 5 as a three-dimensional spherical object CS. When the thus-generated omnidirectional image CE is the spherical object CS, as shown in FIG. 5, the virtual camera IC is located inside the omnidirectional image CE. The predetermined area T in the omnidirectional image CE is the imaging area of the virtual camera IC and is specified by predetermined area information indicating the imaging direction and the angle of view of the virtual camera IC in a three-dimensional virtual space including the omnidirectional image CE. Also, the zoom of the predetermined area T can be expressed by moving the virtual camera IC closer to or farther from the omnidirectional image CE. The predetermined area image Q is an image of the predetermined area T in the omnidirectional image CE. Therefore, the predetermined area T can be specified by the angle of view α and the distance f from the virtual camera IC to the omnidirectional image CE (see FIG. 7).

[0019] Then, as shown in FIG. 6(B), the predetermined area image Q shown in FIG. 6(A) is displayed on a predetermined display as an image of the imaging area of the virtual camera IC. The image shown in FIG. 6(B) is a predetermined area image represented by predetermined area information set by default (initial setting). Hereinafter, the description will be made using the imaging direction (ea, aa) and the angle of view (α) of the virtual camera IC. Note that the predetermined area T may be indicated by the imaging area (X, Y, Z) of the virtual camera IC that is the predetermined area T instead of the angle of view α and the distance f.

[0020] Next, with reference to FIG. 7, the relationship between the predetermined region information and the image of the predetermined region T will be described. FIG. 7 is a diagram showing the relationship between the predetermined region information and the image of the predetermined region T. As shown in FIG. 7, "ea" represents the elevation angle, "aa" represents the azimuth angle, and "α" represents the angle of view (Angle). That is, the posture of the virtual camera IC is changed so that the fixation point of the virtual camera IC indicated by the shooting direction (ea, aa) becomes the center point CP(x, y) of the predetermined region T which is the shooting region of the virtual camera IC. As shown in FIG. 7, when the diagonal angle of view of the predetermined region T represented by the angle of view α of the virtual camera IC is α, the center point CP(x, y) becomes the parameter ((x, y)) of the predetermined region information. The predetermined region image Q is an image of the predetermined region T in the omnidirectional image CE. f is the distance from the virtual camera IC to the center point CP(x, y). L is the distance between an arbitrary vertex of the predetermined region T and the center point CP(x, y) (2L is the diagonal). And in FIG. 7, generally, the trigonometric function represented by the following (Equation 1) holds.

[0021]

Number

[0022] Note that the imaging device 10 described above is an example of an imaging device capable of acquiring a wide-angle image, and the omnidirectional image is an example of a wide-angle image. Here, a wide-angle image is generally an image taken using a wide-angle lens, and is taken with a lens that can capture a wider range than that felt by the human eye. Also, generally, it means an image taken with a lens having a focal length of 35 mm or less in terms of 35 mm film conversion.

[0023] FIG. 8 is a diagram showing points in a three-dimensional Euclidean space in spherical coordinates. Here, let the position coordinates when the center point CP is expressed in the spherical polar coordinate system be (r, θ, φ). (r, θ, φ) are the radial distance, polar angle, and azimuth angle, respectively. Since the radial distance r is the distance from the origin of the three-dimensional virtual space including the entire sky image to the center point CP, it is equal to f. FIG. 8 is a diagram showing these relationships. Hereinafter, the position coordinates (r, θ, φ) of the virtual camera IC will be used for explanation.

[0024] ● Outline of the image communication system Subsequently, with reference to FIG. 9, an outline of the configuration of the image communication system according to the embodiment will be described. FIG. 9 is a diagram showing an example of the overall configuration of the image communication system. The image communication system 1 shown in FIG. 9 is a system that bidirectionally transmits and receives captured images such as videos between a plurality of sites, and by displaying the captured images such as videos distributed from a certain site at other sites, a wide range of images (for example, the entire sky image) captured at other sites can be browsed. The image communication system 1 can, for example, allow an image captured at a certain site such as an office floor to be browsed at another site located remotely. Note that the site where the image is captured is not limited to the office floor, and any space where there is a need for the users (browsers) at the browsing side to grasp the situation of the remote site, such as schools, factories, warehouses, construction sites, server rooms, or stores, etc., is acceptable.

[0025] As shown in FIG. 9, the image communication system 1 includes imaging devices 10 (10A, 10B; hereinafter referred to as the imaging device 10 when no distinction is necessary) located at each of a plurality of sites (site A, site B), a communication management system 50, and communication terminals 30 (30A, 30C, 30D; hereinafter referred to as the communication terminal 30 when no distinction is necessary) located at each of a plurality of sites (site A, site C, site D).

[0026] The communication terminal 30 and the communication management system 50 that constitute the image communication system 1 can communicate via the communication network 100. The communication network 100 is constructed by the Internet, a mobile communication network, a LAN (Local Area Network), etc. Note that the communication network 100 may include not only wired communication but also networks by wireless communication such as 3G (3rd Generation), 4G (4th Generation), 5G (5th Generation), Wi-Fi (Wireless Fidelity) (registered trademark), WiMAX (Worldwide Interoperability for Microwave Access), or LTE (Long Term Evolution).

[0027] Among these, the imaging device 10 is a special digital camera for capturing a subject, scenery, etc. and obtaining two hemispherical images that are the basis of the omnidirectional image as described above. The captured image obtained by the imaging device 10 may be a moving image, a still image, or both a moving image and a still image. Also, the captured image may be a video including audio together with the image. The communication terminal 30 distributes the image acquired from the imaging device 10 via a wired cable such as a USB (Universal Serial Bus) cable to the communication terminal 30 at another site via the communication management system 50. Here, the imaging device 10A and the communication terminal 30A are placed at site A where users A1 and A2 are located. On the other hand, the imaging device 10B is placed at site B where user B1 is located and can directly communicate with the communication management system 50 via the communication network 100. Note that the site where the imaging device 10 is installed is not limited to two sites and may be only one site or three or more sites. Also, the connection relationship between the imaging device 10 and the communication terminal 30 may be a wireless connection using short-range wireless communication or the like instead of a wired connection using a wired cable.

[0028] The communication terminal 30 is a computer such as a PC used by users at each site. The communication terminal 30 displays images (still images or moving images) distributed from other sites. The communication terminal 30 acquires, for example, a panoramic image, which is a captured image taken by the imaging device 10, via the communication network 100. Also, the communication terminal 30 has OpenGL ES installed and can generate predetermined area information indicating a partial area of the panoramic image, or generate a predetermined area image from the panoramic image sent from another communication terminal 30. That is, the communication terminal 30 can display a predetermined area of a subject captured in the panoramic image, which is a captured image.

[0029] Here, the communication terminal 30A is placed at the viewing site A where the users A1 and A2 are located, and the communication terminal 30C is placed at the viewing site C where the user C1 is located. Also, the communication terminal 30D is placed at the viewing site D where the users D1, D2, and D3 are located. The communication terminals 30C and 30D placed at the sites C and D where the imaging device 10 does not exist distribute images of their own sites captured by the cameras of their own terminals to other sites. Note that at the site A where both the imaging device 10A and the communication terminal 30A exist, the communication terminal 30A may distribute, to other sites, the image (panoramic image) taken by the imaging device 10A and the image taken by the camera of its own terminal.

[0030] Note that the arrangement of each terminal and device (communication terminal and imaging device) shown in FIG. 9, as well as the arrangement of the users, is an example, and other examples are also possible. Also, the communication terminal 30 is not limited to a PC, and may be, for example, a tablet terminal, a smartphone, a wearable terminal, a PJ (Projector), an IWB (Interactive White Board: a whiteboard having an electronic blackboard function enabling mutual communication), or an autonomous mobile robot. Furthermore, when the imaging device 10 is provided with a display unit, it may be configured to display images distributed from other sites.

[0031] The communication management system 50 manages and controls the communication of the communication terminals 30 between each site, and manages the types of image data (general image and special image types) transmitted and received. Here, the special image is an omnidirectional image. Also, the communication management system 50 is installed in a service company or the like that provides an image communication service.

[0032] Note that the communication management system 50 may be constructed by a single computer, or may be constructed by a plurality of computers arbitrarily assigned by dividing each part (function or means). Also, all or part of the functions of the communication management system 50 may be a server computer existing in a cloud environment, or may be a server computer existing in an on-premises environment.

[0033] In a conventional system for distributing video data including omnidirectional images, since viewers can freely change the display direction of the video data, whether it is a real-time live distribution or a recorded distribution, viewers at each site can view images in different display directions. On the other hand, in a system for performing such video distribution of omnidirectional images, there may be cases where it is desired to synchronize the display direction with the viewers during a time period when there is a target that the distributor wants the viewers to view, or during a time period when there is a target that the distributor does not want the viewers to view. In such cases, with the conventional method, there is a problem that it is not possible to display an appropriate predetermined area on the communication terminal according to the requests of the distributor or the viewers, such as allowing the viewers to freely operate the display direction during a certain time period and synchronizing the display direction during a certain time period for the video distribution of the omnidirectional image.

[0034] Therefore, the image communication system 1 stores the predetermined area information corresponding to the playback time of the video data including the omnidirectional image transmitted from the communication terminal 30 by the communication management system 50, and when performing the recording and distribution of the video data, together with the video data, transmits the predetermined area information corresponding to the playback time to the communication terminal 30. Then, when playing back the received video data, the communication terminal 30 synchronizes the display direction of the video data if the predetermined area information corresponding to the playback time exists, and allows the user to freely operate the display direction of the video data if the predetermined area information corresponding to the playback time does not exist. Thereby, when the image communication system 1 distributes the video data including the recorded omnidirectional image, the synchronization of the display direction is reflected in the recorded video data, and the viewer of the image can freely operate the display direction of the video data in a certain time period and synchronize the display direction in a certain time period.

[0035] ●Hardware Configuration Subsequently, with reference to FIGS. 10 and 11, the hardware configuration of each device or terminal constituting the image communication system according to the embodiment will be described. Note that the hardware configuration shown in FIGS. 10 and 11 may have components added or deleted as necessary.

[0036] ○Hardware Configuration of the Imaging Device○ First, with reference to FIG. 10, the hardware configuration of the imaging device 10 will be described. FIG. 10 is a diagram showing an example of the hardware configuration of the imaging device. Hereinafter, the imaging device 10 is an omnidirectional (360-degree) imaging device using two imaging elements, but the number of imaging elements may be two or more. Also, it is not necessarily a device dedicated to omnidirectional imaging, and a substantially same function as the imaging device 10 may be achieved by attaching a retrofittable omnidirectional imaging unit to an ordinary digital camera, smartphone, or the like.

[0037] As shown in FIG. 10, the imaging device 10 includes an imaging unit 101, an image processing unit 104, an imaging control unit 105, a microphone 108, a sound processing unit 109, a CPU (Central Processing Unit) 111, a ROM (Read Only Memory) 112, an SRAM (Static Random Access Memory) 113, a DRAM (Dynamic Random Access Memory) 114, an operation unit 115, an input / output I / F 116, a short-range communication circuit 117, an antenna 117a of the short-range communication circuit 117, an electronic compass 118, a gyro sensor 119, an acceleration sensor 120, and a network I / F 121.

[0038] Among these, the imaging unit 101 includes wide-angle lenses 102a and 102b (hereinafter referred to as lens 102 when there is no need to distinguish) each having an angle of view of 180° or more for forming a hemispherical image, and two imaging elements 103a and 103b provided corresponding to each lens. The imaging elements 103a and 103b are image sensors such as CMOS (Complementary Metal Oxide Semiconductor) sensors or CCD (Charge Coupled Device) sensors that convert the optical image formed by the lenses 102a and 102b into image data of an electrical signal and output it, a timing generation circuit that generates a horizontal or vertical synchronization signal, a pixel clock, etc. of this image sensor, and a register group in which various commands or parameters necessary for the operation of this imaging element are set, etc.

[0039] The imaging elements 103a and 103b of the imaging unit 101 are each connected to the image processing unit 104 via a parallel I / F bus. On the other hand, the imaging elements 103a and 103b of the imaging unit 101 are each connected to the imaging control unit 105 via a serial I / F bus (such as an I2C bus). The image processing unit 104, the imaging control unit 105, and the audio processing unit 109 are connected to the CPU 111 via the bus 110. Further, the bus 110 is also connected to a ROM 112, an SRAM 113, a DRAM 114, an operation unit 115, an input / output I / F 116, a short-range communication circuit 117, an electronic compass 118, a gyro sensor 119, an acceleration sensor 120, a network I / F 121, etc.

[0040] The image processing unit 104 captures the image data output from the imaging elements 103a and 103b through the parallel I / F bus, performs predetermined processing on each piece of image data, and then synthesizes these pieces of image data to create data of an orthographic cylindrical projection image as shown in Fig. 3(C).

[0041] The imaging control unit 105 generally uses the I2C bus with the imaging control unit 105 as the master device and the imaging elements 103a and 103b as slave devices to set commands, etc. in the register groups of the imaging elements 103a and 103b. Necessary commands, etc. are received from the CPU 111. Also, the imaging control unit 105 similarly uses the I2C bus to capture status data, etc. of the register groups of the imaging elements 103a and 103b and send them to the CPU 111.

[0042] Also, the imaging control unit 105 instructs the imaging elements 103a and 103b to output image data at the timing when the shutter button of the operation unit 115 is pressed. Depending on the imaging device 10, it may have a preview display function using a display (for example, the display of an external terminal such as a smartphone that performs short-range communication with the imaging device 10 using the short-range communication circuit 117) or a function corresponding to video display. In this case, the output of the image data from the imaging elements 103a and 103b is continuously performed at a predetermined frame rate (frames / minute).

[0043] In addition, as will be described later, the imaging control unit 105 also functions as synchronization control means for synchronizing the output timings of the image data of the imaging elements 103a and 103b in cooperation with the CPU 111. Note that in this embodiment, the imaging device 10 is not provided with a display unit (display), but a display unit may be provided. The microphone 108 converts sound into sound (signal) data. The sound processing unit 109 takes in the sound data output from the microphone 108 through the I / F bus and performs predetermined processing on the sound data.

[0044] The CPU 111 controls the overall operation of the imaging device 10 and executes necessary processing. The ROM 112 stores various programs for the CPU 111. The SRAM 113 and the DRAM 114 are work memories that store programs executed by the CPU 111, data during processing, and the like. In particular, the DRAM 114 stores image data during processing in the image processing unit 104 and data of the processed orthographic cylindrical projection images.

[0045] The operation unit 115 is a general term for various operation buttons, a power switch, a shutter button, and a touch panel having both display and operation functions. The user inputs various shooting modes, shooting conditions, and the like by operating the operation unit 115.

[0046] The input / output I / F 116 is a general term for an interface circuit (such as a USB I / F) with an external medium such as an SD card or a personal computer. The input / output I / F 116 may be wireless or wired. The data of the orthographic cylindrical projection images stored in the DRAM 114 is recorded on an external medium via the input / output I / F 116 or transmitted to an external terminal (device) via the input / output I / F 116 as necessary.

[0047] The short-range communication circuit 117 communicates with an external terminal (device) via an antenna 117a provided in the imaging device 10 by means of short-range wireless communication technologies such as NFC (Near Field Communication), Bluetooth (registered trademark), or Wi-Fi. The short-range communication circuit 117 can transmit the data of the orthographic cylindrical projection image to the external terminal (device).

[0048] The electronic compass 118 calculates the orientation of the imaging device 10 from the earth's magnetism and outputs orientation information. This orientation information is an example of related information (metadata) conforming to Exif and is used for image processing such as image correction of the captured image. Note that the related information also includes each data such as the shooting date and time of the image and the data capacity of the image data. Further, the gyro sensor 119 is a sensor that detects changes in angles (Roll angle, Pitch angle, Yaw angle) accompanying the movement of the imaging device 10. The change in angle is an example of related information (metadata) conforming to Exif and is used for image processing such as image correction of the captured image. Furthermore, the acceleration sensor 120 is a sensor that detects accelerations in three axial directions. The imaging device 10 calculates the posture (angle with respect to the gravity direction) of its own device (imaging device 10) based on the accelerations detected by the acceleration sensor 120. By providing the acceleration sensor 120, the imaging device 10 improves the accuracy of image correction. The network I / F 121 is an interface for performing data communication using a communication network 100 such as the Internet via a router or the like.

[0049] ○Hardware Configuration of Communication Terminal○ FIG. 11 is a diagram showing an example of the hardware configuration of a communication terminal. Each hardware component of the communication terminal 30 is indicated by a reference numeral in the 300s. The communication terminal 30 is constructed by a computer and, as shown in FIG. 11, includes a CPU 301, a ROM 302, a RAM 303, an HD (Hard Disk) 304, an HDD controller 305, a display 306, an external device connection I / F 308, a network I / F 309, a bus line 310, a keyboard 311, a pointing device 312, a DVD-RW (Digital Versatile Disk Rewritable) drive 314, a media I / F 316, an audio input / output I / F 317, a microphone 318, a speaker 319, a short-range communication circuit 320, and a camera 321.

[0050] Among these, the CPU 301 controls the operation of the entire communication terminal 30. The ROM 302 stores programs used for driving the CPU 301 such as the IPL. The RAM 303 is used as a work area for the CPU 301. The HD 304 stores various data such as programs. The HDD controller 305 controls the reading or writing of various data to and from the HD 304 according to the control of the CPU 301. The display 306 displays various information such as a cursor, a menu, a window, characters, or an image. The display 306 is an example of a display unit. Note that the display 306 may be a touch panel display having an input means. The external device connection I / F 308 is an interface for connecting various external devices. The external devices in this case are, for example, a USB memory or a printer. The network I / F 309 is an interface for performing data communication using the communication network 100. The bus line 310 is an address bus, a data bus, or the like for electrically connecting each component such as the CPU 301 shown in FIG. 12.

[0051] Also, the keyboard 311 is a type of input means having a plurality of keys for inputting characters, numerical values, various instructions, etc. The pointing device 312 is a type of input means for selecting or executing various instructions, selecting a processing target, or moving a cursor, etc. Note that the input means may be not only the keyboard 311 and the pointing device 312, but also a touch panel or a voice input device, etc. The DVD-RW drive 314 controls reading or writing of various data with respect to the DVD-RW 313 as an example of a removable recording medium. Note that it is not limited to DVD-RW, and it may be DVD-R or Blu-ray (registered trademark) Disc, etc. The media I / F 316 controls reading or writing (storage) of data with respect to the recording medium 315 such as a flash memory. The microphone 318 is a type of built-in sound collection means for inputting sound. The audio input / output I / F 317 is a circuit that processes input / output of sound signals between the microphone 318 and the speaker 319 according to the control of the CPU 301. The short-range communication circuit 320 is a communication circuit for communicating with an external terminal (device) by a short-range wireless communication technology such as NFC, Bluetooth, or Wi-Fi. The camera 321 is a type of built-in imaging means for imaging a subject to obtain image data. Note that the microphone 318, the speaker 319, and the camera 321 may be external devices instead of being built-in in the communication terminal 30.

[0052] ○ Hardware Configuration of Communication Management System ○ FIG. 11 is a diagram showing an example of the hardware configuration of the communication management system. Each hardware configuration of the communication management system 50 is indicated by a reference numeral in the 500s in parentheses. The communication management system 50 is constructed by a computer and has the same configuration as the communication terminal 30 as shown in FIG. 12, so the description of each hardware configuration is omitted.

[0053] Incidentally, each of the above programs may be in an installable or executable file format and recorded on a computer-readable recording medium for distribution. Examples of the recording medium include CD-R (Compact Disc Recordable), DVD (Digital Versatile Disk), Blu-ray Disc, SD card, USB memory, etc. Further, the recording medium can be provided as a program product, either domestically or abroad. For example, the communication terminal 30 realizes the image display method according to the present invention by executing the program according to the present invention.

[0054] ●Functional configuration Subsequently, with reference to FIGS. 12 to 20, the functional configuration of the image communication system according to the embodiment will be described. FIGS. 12 and 13 are diagrams showing an example of the functional configuration of the image communication system. Note that FIGS. 12 and 13 show those of the devices or terminals shown in FIG. 9 that are related to the processes or operations described later.

[0055] ○Functional configuration of the imaging device○ First, with reference to FIG. 12, the functional configuration of the imaging device 10 will be described. The imaging device 10 includes a communication unit 11, a reception unit 12, an imaging unit 13, a sound collection unit 14, and a storage / reading unit 19. Each of these units is a function or means realized by any of the components shown in FIG. 10 operating according to an instruction from the CPU 111 according to the program for the imaging device developed from the SRAM 113 onto the DRAM 114. Further, the imaging device 10 has a storage unit 1000 constructed by the ROM 112, SRAM 113, and DRAM 114 shown in FIG. 11. The storage unit 1000 stores the GUID (Globally Unique Identifier) of the own device.

[0056] The communication unit 11 is mainly realized by the processing of the CPU 111 and conducts communication of various data or information with other devices or terminals. The communication unit 11 conducts data communication with other devices or terminals using, for example, the short-range communication circuit 117 based on short-range wireless communication technology. Also, the communication unit 11 conducts data communication with other devices or terminals using, for example, the input / output I / F 116 via various cables or the like. Furthermore, the communication unit 11 conducts data communication with other devices or terminals using the network I / F 121 via the communication network 100.

[0057] The reception unit 12 is mainly realized by the processing of the CPU 111 with respect to the operation unit 115 and receives various selections or inputs from the user. The imaging unit 13 is mainly realized by the processing of the CPU 111 with respect to the imaging unit 101, the image processing unit 104, and the imaging control unit 105, images a subject such as a landscape, and acquires photographed image data. The sound collection unit 14 is mainly realized by the processing of the CPU 111 with respect to the microphone 108 and the sound processing unit 109, and collects the sounds around the photographing device 10.

[0058] The storage / reading unit 19 is mainly realized by the processing of the CPU 111, stores various data (or information) in the storage unit 1000, or reads out various data (or information) from the storage unit 1000.

[0059] ○Functional Configuration of Communication Terminal○ Next, the functional configuration of the communication terminal 30 will be described with reference to FIG. 12. The communication terminal 30 includes a transmission / reception unit 31, a reception unit 32, an image / sound processing unit 33, a display control unit 34, a determination unit 35, a creation unit 36, a generation unit 37, a communication unit 38, an imaging unit 41, a sound collection unit 42, and a storage / reading unit 39. Each of these units is a function or means realized by any of the components shown in FIG. 11 operating according to an instruction from the CPU 301 in accordance with the communication terminal program expanded from the HD 304 onto the RAM 303. Also, the communication terminal 30 has a storage unit 3000 constructed by the ROM 302, the RAM 303, and the HD 304 shown in FIG. 11.

[0060] The transmission / reception unit 31 is mainly realized by the processing of the CPU 301 for the network I / F 309, and transmits and receives various data or information to and from other devices or terminals via the communication network 100.

[0061] The reception unit 32 is mainly realized by the processing of the CPU 301 for the keyboard 311 or the pointing device 312, and receives various selections or inputs from the user.

[0062] The image / audio processing unit 33 is mainly realized by the processing of the CPU 301, and performs image processing on the captured image data obtained by the imaging device 10 or the camera 321 capturing a subject. Also, after the audio of the user is converted into an audio signal by the microphone 318, the image / audio processing unit 33 performs audio processing on the audio data related to this audio signal. For example, since the display control unit 34 displays an image on the display 306, the image / audio processing unit 33 performs image processing on the captured image data received from the imaging device 10 or captured by the camera 321 based on image type information such as the source name. Specifically, when the image type information indicates that it is a special image, the image / audio processing unit 33 creates omnidirectional image data by converting the captured image data (for example, the data of each hemispherical image as shown in FIGS. 3(A) and (B)) into omnidirectional image data as shown in FIG. 4(B).

[0063] Also, the image / audio processing unit 33 performs image processing on the captured image data distributed from another communication terminal 30. Further, the image / audio processing unit 33 outputs the audio signal related to the audio data distributed from another communication terminal 30 via the communication management system 50 to the speaker 319, and causes the speaker 319 to output audio.

[0064] The display control unit 34 is mainly realized by the processing of the CPU 301, and causes the display 306 to display various images, characters, etc. The determination unit 35 is realized by the processing of the CPU 301 and makes various determinations. The determination unit 35 determines, for example, the image type of the captured image data received from the imaging device 10 or captured by the camera 321.

[0065] The creation unit 36 is mainly realized by the processing of the CPU 301, and creates a source name, which is an example of image type information, according to the naming rule based on the result determined by the determination unit 35 as a general image or a special image (here, an omnidirectional image). For example, when the determination unit 35 determines that it is a general image, the creation unit 36 creates a source name "Video" indicating that it is a general image. On the other hand, when the determination unit 35 determines that it is a special image, the creation unit 36 creates a source name "Video_Theta" indicating that it is a special image.

[0066] The generation unit 37 is mainly realized by the processing of the CPU 301, and generates a predetermined area image to be displayed on the display 306. The generation unit 37 performs a perspective projection transformation using the predetermined area information for the captured image (omnidirectional image) to generate a predetermined area image corresponding to the predetermined area information, for example, in order to display on the display 306 an image of a predetermined area indicated by the predetermined area information in the captured image (omnidirectional image) captured by the imaging device 10 or the predetermined area information received by the transmission / reception unit 31.

[0067] The communication unit 38 is mainly realized by the processing of the CPU 301 with respect to the short-range communication circuit 320, and communicates with the communication unit 11 of the imaging device 10 by short-range wireless communication technologies such as NFC, Bluetooth, or WiFi. Although the communication unit 38 and the transmission / reception unit 31 have been described as having separate communication units, they may have a shared configuration.

[0068] The imaging unit 41 is mainly realized by the processing of the CPU 301 on the camera 321, captures a subject such as a landscape, and acquires captured image data. The sound collection unit 42 is mainly realized by the processing of the CPU 301 on the microphone 318 and the sound input / output I / F 317, and collects the sounds around the communication terminal 30.

[0069] The storage / reading unit 39 is mainly realized by the processing of the CPU 301, stores various data (or information) in the storage unit 3000, or reads out various data (or information) from the storage unit 3000.

[0070] ○ Image Type Management Table FIG. 14 is a conceptual diagram showing an example of the image type management table. In the storage unit 3000, an image type management DB 3001 configured by the image type management table shown in FIG. 14 is constructed. This image type management table manages by associating an image data ID, an example of the destination IP (Internet Protocol) address of the source terminal, and a source name. Among these, the image data ID is an example of image data identification information for identifying the image data when performing image distribution. The same image data ID is added to the image data transmitted from the same source terminal. Thereby, the destination terminal (the receiving communication terminal) can specify the source terminal of the received image data. The IP address of the source terminal indicates the IP address of the imaging device 10 or the communication terminal 30 that transmits the image data indicated by the associated image data ID. The source name is a name for specifying the imaging device 10 or the camera 321 that outputs the image data indicated by the associated image data ID, and is an example of image type information. This source name is a name created by the communication terminal 30 according to a naming rule of a predetermined name.

[0071] For example, it is shown that five source terminals with IP addresses "1.2.1.3", "1.2.2.3", "1.3.1.3", "1.3.2.3", and "1.2.2.5" are respectively transmitting image data indicated by image data IDs "RS001", "RS002", "RS003", "RS004", and "RS005". Further, the types of images indicated by the source names of each source terminal are "Video_Theta", "Video_Theta", "Video", "Video", "Video_Theta", which in order indicate that the image types are "special image", "special image", "general image", "general image", "special image". In this embodiment, the special image is a panoramic image. Note that the IP address is an example of destination information, and the destination information may be a MAC (Media Access Control) address or a terminal ID (Identification), etc. Also, although the IP address represents a simplified IPv4 address, the IP address may be IPv6. Furthermore, data other than image data may also be managed in association with an image data ID. Data other than image data is, for example, audio data, material data during screen sharing, etc.

[0072] ○Shooting device management table FIG. 15 is a conceptual diagram showing an example of a shooting device management table. In the storage unit 3000, a shooting device management DB3002 configured by the shooting device management table shown in FIG. 15 is constructed. This shooting device management table stores and manages the vendor ID and product ID among the GUIDs of the shooting device 10 that can obtain two hemispherical images that are the basis of the panoramic image. As the GUID, for example, the vendor ID (VID) and product ID (PID) used for USB devices can be used. This vendor ID and product ID may be stored, for example, since the factory shipment of the communication terminal 30, or may be additionally stored after the factory shipment.

[0073] ○Predetermined area management table FIG. 16 is a conceptual diagram showing an example of a predetermined area management table. In the storage unit 3000, a predetermined area management DB 3003 configured by the predetermined area management table shown in FIG. 16 is constructed. This predetermined area management table manages by associating a time stamp indicating the reproduction time of video data, which is photographed image data, and predetermined area information indicating a predetermined area image to be displayed on the communication terminal 30. As shown in FIGS. 6 and 7, the predetermined area information is conversion parameters for converting a photographed image into an image (predetermined area image) of a predetermined area T in this photographed image.

[0074] For example, the first row in the predetermined area management table of FIG. 16 manages that video data with a reproduction time corresponding to the time stamp "10001" is to be displayed in the predetermined area (r = 10, θ = 20, φ = 30). Similarly, the second row in the predetermined area management table of FIG. 16 manages that video data with a reproduction time corresponding to the time stamp "10002" is to be displayed in the predetermined area (r = 20, θ = 30, φ = 40), and the second row in the predetermined area management table of FIG. 16 manages that video data with a reproduction time corresponding to the time stamp "10004" is to be displayed in the predetermined area (r = 20, θ = 20, φ = 30). On the other hand, in the third row of the predetermined area management table of FIG. 16, no predetermined area information is stored, and it manages that the display direction of the predetermined area image is not synchronized with respect to the video data with a reproduction time corresponding to the time stamp "10003". Also, when the transmission / reception unit 31 newly receives predetermined area information corresponding to the time stamp of video data that has already been managed, the storage / readout unit 39 rewrites the already managed predetermined area information with the newly received predetermined area information.

[0075] ○Functional Configuration of Communication Management System○ Next, with reference to FIG. 13, the functional configuration of the communication management system 50 will be described. The communication management system 50 includes a transmission / reception unit 51, a determination unit 52, a recording processing unit 53, a generation unit 54, a distribution condition setting unit 55, and a storage / reading unit 59. Each of these units is a function or means realized by any of the components shown in FIG. 11 operating according to instructions from the CPU 501 in accordance with the communication management system program expanded from the HD 504 onto the RAM 503. Further, the communication management system 50 has a storage unit 5000 constructed by the ROM 502, the RAM 503, and the HD 504 shown in FIG. 11.

[0076] The transmission / reception unit 51 is mainly realized by the processing of the CPU 501 for the network I / F 509, and transmits and receives various data or information to and from other devices via the communication network 100.

[0077] The determination unit 52 is realized by the processing of the CPU 501 and makes various determinations. The recording processing unit 53 is mainly realized by the processing of the CPU 501 and records video data composed of captured image data and audio data transmitted from each communication terminal 30. For example, when the data transmitted from each communication terminal 30 is video data, the recording processing unit 53 stores the received video data as an individual recording file and stores metadata (bibliographic information) in the video data management DB 5003.

[0078] The generation unit 54 is mainly realized by the processing of the CPU 501 and generates an image data ID and predetermined area information. For example, the generation unit 54 generates predetermined area information indicating a predetermined area (for example, the predetermined area T shown in FIG. 5 etc.) in the captured image captured by the imaging device 10. Note that an image when the entire captured image is displayed (for example, the full-sphere image CE shown in FIG. 5 etc.) is also referred to as an "entire image".

[0079] The distribution condition setting unit 55 is mainly realized by the processing of the CPU 501, and sets the distribution conditions when distributing the video data recorded by the recording processing unit 53 to the communication terminals 30 at each site. For example, the distribution condition setting unit 55 sets predetermined area information corresponding to the playback time in association with a time stamp indicating the playback time of the recorded video data.

[0080] The storage / reading unit 59 is mainly realized by the processing of the CPU 501, and stores various data (or information) in the storage unit 5000 and reads out various data (or information) from the storage unit 5000.

[0081] ○ Session management table FIG. 17 is a conceptual diagram showing an example of the session management table. In the storage unit 5000, a session management DB 5001 configured by the session management table shown in FIG. 17 is constructed. This session management table manages by associating the session ID and the IP address of the participating communication terminal. Among these, the session ID is an example of session identification information for identifying a communication session that realizes image communication, and is generated for each virtual room. The session ID is also managed by each communication terminal and is used when selecting a communication session at each communication terminal. The IP address of the participating communication terminal indicates the IP address of the communication terminal that has participated in the virtual room indicated by the associated session ID.

[0082] ○ Image type management table FIG. 18 is a conceptual diagram showing an example of an image type management table. In the storage unit 5000, an image type management DB 5002 configured by the image type management table shown in FIG. 18 is constructed. This image type management table manages by associating the same session ID as the session ID managed in the session management table, in addition to each piece of information managed in the image type management table shown in FIG. 14. In the communication management system 50, when a new communication terminal enters a virtual room or the like, the same things as the image data ID, the IP address of the transmission source terminal, and the image type information managed by the communication terminal 30 are managed in order to transmit image type information and the like to the communication terminal that is already in an image call and the newly joined communication terminal. Thereby, it is not necessary to perform transmission and reception of image type information and the like between the communication terminal that is already in a call and the newly joined communication terminal.

[0083] ○ Video data management table FIG. 19 is a conceptual diagram showing an example of a video data management table. In the storage unit 5000, a video data management DB 5003 configured by the video data management table shown in FIG. 19 is constructed. This video data management table manages by associating the IP address of the communication terminal 30 or the imaging device 10 that is the transmission source of the captured image data and the recording data file of the video data that is the captured image data.

[0084] ○ Predetermined area management table FIG. 20 is a conceptual diagram showing an example of a predetermined area management table. In the storage unit 5000, a predetermined area management DB 5004 configured by the predetermined area management table shown in FIG. 20 is constructed. This predetermined area management table associates and manages, for each recording data file of video data, the IP address of the transmission source terminal of the captured image data, the IP address of the communication terminal 30 of the transmission destination of the captured image data, and a time stamp indicating the playback time of the video data that is the captured image data and predetermined area information indicating a predetermined area image to be displayed on the communication terminal 30. As shown in FIGS. 6 and 7, the predetermined area information is conversion parameters for converting a captured image into an image (predetermined area image) of a predetermined area T in this captured image.

[0085] For example, the first row of the predetermined area management table in FIG. 20 manages that video data with a playback time corresponding to the time stamp "10001" is transmitted from the communication terminal 30 with the IP address "1.2.1.3" to the communication terminal 30 with the IP address "1.3.1.3" via the communication management system 50, and the predetermined area (r = 10, θ = 20, φ = 30) is to be displayed on the transmission destination communication terminal 30. Also, as shown in the second and third rows of the predetermined area management table in FIG. 20, when the IP address of the image transmission destination is "all", the communication management system 50 transmits the predetermined area information corresponding to the associated time stamp to all the communication terminals 30 serving as the distribution destination, and causes the same predetermined area to be displayed on all the communication terminals 30.

[0086] Furthermore, as shown in the third and sixth rows of the predetermined area management table in FIG. 20, when the predetermined area information is not stored, the communication management system 50 does not synchronize the display direction of the predetermined area image to be displayed on the communication terminal 30, which is the distribution destination (image transmission destination) of the video data, without transmitting the predetermined area information corresponding to the associated timestamp. Also, as shown in the fourth and fifth rows of the predetermined area management table in FIG. 20, when different image transmission destinations or predetermined area information is stored for the same image transmission source and timestamp, the communication management system 50 transmits the predetermined area information corresponding to the associated timestamp to the communication terminal 30 that is the corresponding image transmission destination, and causes different predetermined areas to be displayed for each communication terminal 30.

[0087] Also, when the transmission / reception unit 51 newly receives the predetermined area information corresponding to the set of the IP addresses of the communication terminals that are the transmission source and destination of the already managed captured image data, and the timestamp of the video data, the storage / readout unit 59 rewrites the already managed predetermined area information with the newly received predetermined area information. Here, the timestamp is an example of time information indicating the playback time of the video data. The timestamp indicates the time indicating the playback location within the total playback time of the video data in the recorded data file. Note that the timestamp may be in a format representing the playback time of the video data by storing the time when the captured image data related to the video data was received from the image transmission source.

[0088] ● Processing or operations of the embodiment ○ Session participation processing ○ Next, with reference to FIGS. 21 to 33, the processing or operation of the image communication system according to the embodiment will be described. In the following description, an example in which a captured image taken at Site A is distributed to each site will be described, but the same processing is performed in image distribution from other sites. First, with reference to FIGS. 21 and 22, the participation processing for a specific communication session will be described. FIG. 21 is a sequence diagram showing an example of the participation processing for a specific communication session in the image communication system. FIG. 22 is a diagram showing an example of the selection screen for the communication session.

[0089] First, a user at Site A (for example, User A1) performs an operation to display the selection screen for the communication session, and the reception unit 32 of the communication terminal 30A receives the operation to display the selection screen. As a result, the display control unit 34 of the communication terminal 30A causes the display 306 to display the selection screen 800 shown in FIG. 22 (step S11). The selection screen 800 shown in FIG. 22 displays selection buttons 810a, 810b, 810c, etc. indicating each virtual room A1, B1, B2, etc. that is a selection target. Also, each selection button 810a, etc. is associated with each session ID.

[0090] Here, when User A1 selects a desired selection button (here, selection button 810a) for the virtual room that is the site, the reception unit 32 of the communication terminal 30A receives the selection of the communication session (step S12). Then, the transmission / reception unit 31 of the communication terminal 30A transmits a participation request for the communication session with other sites to the communication management system 50 (step S13). This participation request includes the session ID indicating the communication session accepted in step S12 and the IP address of the communication terminal 30A, which is the request source terminal. Thereby, the transmission / reception unit 51 of the communication management system 50 receives the participation request.

[0091] Next, the storage / reading unit 59 of the communication management system 50 adds the IP address received in step S13 to the field of the participating terminal IP address in the record with the same session ID as the session ID received in step S13 in the session management DB 5001 (see FIG. 17), thereby performing the process of participating in the communication session (step S14). Then, the transmission / reception unit 51 transmits a participation request response to the communication terminal 30A (step S15). This participation request response includes the session ID received in step S13 and the participation process result in step S14. As a result, the transmission / reception unit 31 of the communication terminal 30A receives the participation request response. Hereinafter, the case where the participation process is successful will be described. By the same process as the process shown in FIG. 21, the communication terminals 30 at each site perform the process of participating in the communication session.

[0092] ○ Management Process of Image Type Information ○ Next, the management process of image type information will be described with reference to FIG. 23. FIG. 23 is a sequence diagram showing the management process of image type information.

[0093] First, when a user at Site A connects the imaging device 10A to the communication terminal 30A, the storage / reading unit 19 of the imaging device 10A reads out the GUID of its own device (imaging device 10A) stored in the storage unit 1000. Then, the communication unit 11 of the imaging device 10A transmits the GUID of its own device to the communication terminal 30A (step S31). As a result, the communication unit 38 of the communication terminal 30A receives the GUID of the imaging device 10A.

[0094] Next, the determination unit 35 of the communication terminal 30A determines the image type by determining whether the vendor ID and product ID in the GUID received in step S31 are managed in the imaging device management DB 3002 (see FIG. 15) (step S32). Specifically, in the imaging device management DB 3002, if the same vendor ID and product ID are managed, the determination unit 35 of the communication terminal 30A determines that the imaging device 10A is an imaging device that captures special images (here, omnidirectional images). On the other hand, if the same vendor ID and product ID are not managed in the imaging device management DB 3002, the determination unit 35 of the communication terminal 30A determines that the imaging device 10A is an imaging device that captures general images.

[0095] Next, the storage / reading unit 39 of the communication terminal 30A stores, in association with each other, the IP address of its own terminal (communication terminal 30A), which is the source terminal, and the image type information, which is the determination result determined in step S32, in the image type management DB 3001 (see FIG. 14) (step S33). In this state, the image data ID is not associated. The image type information is, for example, a source name or an image type (general image, special image) determined according to a predetermined naming rule.

[0096] Next, the transmission / reception unit 31 of the communication terminal 30A transmits a request to add image type information to the communication management system 50 (step S34). This request to add image type information includes the IP address of its own terminal (communication terminal 30A), which is the source terminal, and the image type information stored in step S33. As a result, the transmission / reception unit 51 of the communication management system 50 receives the request to add image type information.

[0097] Next, the storage / reading unit 59 of the communication management system 50 reads out the corresponding session ID by searching the session management DB 5001 (see FIG. 17) using the IP address of the source terminal received in step S34 as a search key (step S35).

[0098] Next, the generation unit 54 generates a unique image data ID (step S36). Then, the storage / reading unit 59 stores, as a new record in the image type management DB5002 (see FIG. 18), the session ID read in step S35, the image data ID generated in step S36, and the IP address and image type information of the source terminal received in step S34 in an associated manner (step S37). Then, the transmission / reception unit 51 transmits the image data ID generated in step S36 to the communication terminal 30A (step S38). As a result, the transmission / reception unit 31 of the communication terminal 30A receives the image data ID.

[0099] Next, the storage / reading unit 39 of the communication terminal 30A stores the image data ID received in step S38 in the image type management DB3001 (see FIG. 14) in an associated manner with the IP address and image type information of its own terminal (communication terminal 30A), which is the source terminal, stored in step S33 (step S39).

[0100] On the other hand, the transmission / reception unit 51 of the communication management system 50 transmits an additional notification of image type information to another communication terminal (here, the communication terminal 30D) (step S40). This additional notification of image type information includes the image data ID generated in step S36, and the IP address and image type information of the communication terminal 30A, which is the source terminal, stored in step S37. As a result, the transmission / reception unit 31 of the communication terminal 30D receives the additional notification of image type information. Note that the transmission destination of the transmission / reception unit 51 is another IP address associated with the same session ID as the IP address of the communication terminal 30A in the session management DB5001 (see FIG. 17). That is, the transmission destination is another communication terminal that has entered the same virtual room as the communication terminal 30A.

[0101] Next, the storage / reading unit 39 of the communication terminal 30D stores, as a new record in the image type management DB 3001D (see FIG. 18), the image data ID received in step S40, the IP address of the communication terminal 30A which is the source terminal, and the image type information in association with each other (step S41). Similarly, additional communication of the image type information is also transmitted to the communication terminal 30C which is another communication terminal, and is stored in the image type management DB 3001C. As described above, each communication terminal can share the same information in each of the image type management DBs 3001A, 3001C, and 3001D.

[0102] ○Communication processing of photographed image data○ Next, with reference to FIGS. 24 and 25, the process in which the photographed image data and audio data obtained at base A are transmitted to each communication terminal (communication terminals 30C and 30D) via the communication management system 50 will be described. FIG. 24 is a sequence diagram showing an example of the communication processing of the photographed image data and audio data in the image communication system. Note that FIG. 24 shows an example in which the photographed image data obtained by one photographing device 10 is distributed to each communication terminal 30, but the same process is performed even when a plurality of photographed image data obtained by other photographing devices 10 installed at the same base are distributed.

[0103] First, the communication unit 11 of the photographing device 10A transmits the photographed image data obtained by photographing a subject, a landscape, etc. and the audio data obtained by sound collection to the communication terminal 30A (step S51). In this case, since the photographing device 10A is a device that can obtain two hemispherical images that are the basis of the omnidirectional image, as shown in FIGS. 3(A) and (B), the photographed image data is composed of the data of the two hemispherical images. Thereby, the communication unit 38 of the communication terminal 30A receives the photographed image data and the audio data.

[0104] Next, the transmitting and receiving unit 31 of the communication terminal 30A transmits the captured image data and audio data sent from the imaging device 10A to the communication management system 50 (step S52). This transmission includes an image data ID for identifying the captured image data that is the transmission target. As a result, the transmitting and receiving unit 51 of the communication management system 50 receives the captured image data, audio data, and the image data ID.

[0105] Next, the transmitting and receiving unit 51 of the communication management system 50 transmits the captured image data and audio data to the communication terminals (communication terminals 30C and 30D) participating in the same session as the communication terminal 30A (steps S53 and S54). Each of these transmissions includes an image data ID for identifying the captured image data that is the transmission target. As a result, the transmitting and receiving units 31 of the communication terminals 30C and 30D receive the captured image data, audio data, and the image data ID, respectively.

[0106] Here, with reference to FIG. 25, a display example of the communication terminal 30 at each site will be described. FIG. 25 is a diagram showing an example of a display screen displayed on the communication terminal. The captured image data captured at each site is displayed on the display screen 200. For example, when the captured image captured at a predetermined site is an omnidirectional image, the display screen 200 shows an image generated by generating an omnidirectional image and a predetermined region image from the captured image data sent from other sites.

[0107] Here, if the captured image data sent from the imaging devices 10A and 10B capable of capturing all-sky images is displayed as it is, the images of base A and base B are displayed as the front hemisphere image and the rear hemisphere image, respectively, as shown in FIGS. 3(A) and (B). On the other hand, when the generation unit 37 generates an all-sky image and further generates a predetermined area image from the captured image data output by the imaging devices 10A and 10B that can obtain two hemisphere images that are the basis of the all-sky image, a predetermined area image that is a planar image is displayed as shown in FIG. 25. Note that at base C, the communication terminal 30C that obtains a general image captures an image, and at base D as well, the communication terminal 30D that obtains a general image captures an image, so a general image (here, a planar image) is displayed.

[0108] Also, the users at each base can change the predetermined area related to the predetermined area image in the same all-sky image. For example, the users at each base can, by operating an input means such as a pointing device 312, etc., cause the reception unit 32 to receive the movement of the predetermined area image, and the display control unit 35 can shift, rotate, reduce, or enlarge the predetermined area image.

[0109] Note that the image of base A is displayed in the display area on the left side (layout number "1") of the display screen 200, and the image of base B is displayed in the upper right display area (layout number "2"). Further, the image of base C is displayed in the middle right display area (layout number "3") of the display screen 200, and the image of base D is displayed in the lower right display area (layout number "4") of the display screen 200. The display area with layout number "1" is the main display area, and the display areas with layout numbers "2", "3", and "4" are sub-display areas. The image in the main display area and the images in the sub-display areas can be changed by each communication terminal. Usually, at each base, the image of the base where the central person in the video call is located is displayed in the main display area.

[0110] In addition, the all-sky icons 210a and 210b shown in FIG. 25 are an example of a special image identification icon for indicating that they are predetermined region images showing a predetermined region T of a part of the all-sky image. Note that the display positions of the all-sky icons 210a and 210b may be anywhere, such as the upper left, lower left, and lower right, instead of the upper right. Also, the types of the all-sky icons 210a and 210b are not limited to those shown in FIG. 25. Instead of the all-sky icons 210a and 210b, characters such as "all-sky image" may be used, or a combination of an icon and characters may be used.

[0111] ○ Recording process of video data Next, with reference to FIG. 26, the process of recording video data transmitted from the communication terminal 30 at each site in the communication management system 50 will be described. FIG. 26 is a flowchart showing an example of the recording process of video data. The recording process of video data shown in FIG. 26 is executed when the communication management system 50 receives the captured image data, as in step 52 of FIG. 25.

[0112] First, the determination unit 52 of the communication management system 50 determines whether the data transmitted from the communication terminal 30 (for example, the communication terminal 30A in FIG. 24) that transmits the captured image data and the like is video data. Then, when the determination unit 52 determines that the video data has been received by the transmission / reception unit 51 (YES in step S71), the process proceeds to step S72.

[0113] Next, the recording processing unit 53 stores the data file of the video data received by the transmission / reception unit 51 in the video data management DB5003 (see FIG. 19) (step S72). In this case, the recording processing unit 53 associates the IP address of the transmission source of the received video data and the recording data file of the received video data (for example, the file name) and stores them in the video data management DB5003. The recording processing unit 53 stores them in the video data management DB5003, for example, by synthesizing the captured image data and audio data transmitted from the communication terminal 30 at each site and recording them as one recording data file for each site.

[0114] On the other hand, when the determination unit 52 determines that the data received by the transmission / reception unit 51 is not video data (NO in step S71), the process proceeds to step S73.

[0115] Next, the determination unit 52 determines whether it has received predetermined area information transmitted from a communication terminal 30 (for example, communication terminal 30C or communication terminal 30D in FIG. 24) that views the captured image data distributed from another base station. In the live distribution of video data, the communication terminal 30 always notifies the communication management system 50 of which omnidirectional image the own base station is viewing when synchronizing the display direction of the omnidirectional image with other communication terminals 30, and thus transmits the predetermined area information to the communication management system 50. When the determination unit 52 determines that it has received the predetermined area information transmitted from the communication terminal 30 by the transmission / reception unit 51 (YES in step S73), the process proceeds to step S74. Then, the storage / reading unit 59 stores the predetermined area information received by the transmission / reception unit 51 in the predetermined area management DB 5004 (see FIG. 20) (step S74). In this case, the storage / reading unit 59 associates the transmission source and destination IP addresses of the video data corresponding to the received predetermined area information, the time stamp indicating the reproduction time of the video data corresponding to the received predetermined area information, and the received predetermined area information, and stores them in the predetermined area management DB 5004.

[0116] On the other hand, in step S73, when the determination unit 52 determines that the predetermined area information has not been received by the transmission / reception unit 51 (NO in step S73), the process ends. Note that the communication terminal 30 performs control such that it does not transmit the predetermined area information or does not store the predetermined area information in the communication management system 50 when the display direction of the omnidirectional image is not synchronized with other communication terminals 30 in the live distribution of video data.

[0117] ○Distribution condition setting process○ Next, with reference to FIGS. 27 and 28, the setting process for the distribution conditions of the recorded video data will be described. FIG. 27 is a sequence diagram showing an example of the setting process for the distribution conditions. FIG. 27 describes the process of setting the distribution conditions for video data consisting of a panoramic image captured by the imaging device 10A by the user A1 located at the base A using the communication terminal 30A.

[0118] First, in response to a predetermined input operation by the user A1, the transceiver 31 of the communication terminal 30A transmits a distribution condition setting request to the communication management system 50 (step S91). As a result, the transceiver 51 of the communication management system 50 receives the distribution condition setting request transmitted from the communication terminal 30A.

[0119] Next, the storage / reading unit 59 of the communication management system 50 searches the video data management DB5003 (see FIG. 19) using the IP address of the communication terminal 30A, which is the transmission source of the distribution condition setting request received in step S91, as a search key, and reads out the data file of the video data associated with the IP address of the communication terminal 30A (step S92). Then, the transceiver 51 transmits the video data read in step S92 to the requesting communication terminal 30A (step S93). As a result, the transceiver 31 of the communication terminal 30A receives the video data transmitted from the communication management system 50.

[0120] Next, the display control unit 34 of the communication terminal 30A causes the distribution condition setting screen 700 to be displayed on the display 306 (step S94). FIG. 28 is a diagram showing an example of the distribution condition setting screen. The distribution condition setting screen 700 shown in FIG. 28 is a screen for setting a predetermined area to be displayed on the communication terminal 30 when distributing video data to each site while playing back the recorded video data. The distribution condition setting screen 700 includes an image display area 710 for displaying an image related to the video data being played back, a playback location display area 730 for displaying the playback location of the video data, a predetermined area information display area 740 indicating a predetermined area corresponding to the display direction displayed in the image display area 710, a setting button 750 to be pressed when setting the distribution conditions, and an end button 755 to be pressed when ending the setting of the distribution conditions.

[0121] Among these, the image display area 710 includes display direction change buttons 720 (720a, 720b, 720c, 720d) for changing the display direction of the video data. For example, the user A1 can change the display direction of the image to be displayed in the image display area 710 by selecting a desired display direction change button 720. Further, the playback location display area 730 includes a playback button 731 to be pressed when playing back the video data, a pause button 732 to be pressed when pausing the playback of the video data, and a slider 735 indicating the playback history (playback location) and the like at the total playback time. The slider 735 constitutes a seek bar. The seek bar is for displaying the playback location of the video data and is also an operation area for specifying the playback location of the video data. For example, the user A1 can visually grasp where in the video data is being played from the start to the end by the position of the slider 735. Also, for example, the user A1 can move the slider 735 using an input means such as the pointing device 312 and thereby play back the video data from an arbitrary playback location.

[0122] The predetermined area information display area 740 shows the predetermined area information corresponding to the predetermined area image displayed in the image display area 710. For example, when user A1 selects and moves the display direction change button 720 to change the display direction of the image to be displayed in the image display area 710, the value of the predetermined area corresponding to the displayed predetermined area image is changed as needed. Note that user A1 may manually input the numerical value of the predetermined area to be displayed in the predetermined area information display area 740 to change the display direction of the image to be displayed in the image display area 710.

[0123] The reception unit 32 of the communication terminal 30A receives the input of the distribution conditions (step S95) when user A1 selects the display direction change button 720 or inputs a numerical value to the predetermined area information display area 740 and presses the setting button 750. In this case, the reception unit 32 receives the predetermined area information selected or input by user A1 as the distribution conditions. Then, the transmission / reception unit 31 of the communication terminal 30A transmits the predetermined area information input in step S95 and the time information indicating the reproduction time of the video data corresponding to the input predetermined area information to the communication management system 50 (step S96). As a result, the transmission / reception unit 51 of the communication management system 50 receives the predetermined area information and the time information transmitted from the communication terminal 30A.

[0124] Then, the distribution condition setting unit 55 of the communication management system 50 associates the time stamp corresponding to the predetermined area information and the time information received in step S96 with the IP address of the communication terminal 30A, which is the transmission source of the distribution condition setting request received in step S91, and stores it in the predetermined area management DB5004 (see FIG. 20) (step S97). In this case, the distribution condition setting unit 55 sets, for example, the IP address of the image transmission destination to be stored in association with the received predetermined area information to "all". When the distribution condition setting unit 55 stores the predetermined area information corresponding to the combination of the transmission source of the photographed image data and the time stamp of the video data already managed in the predetermined area management DB5004, the already managed predetermined area information is rewritten with the newly received predetermined area information.

[0125] In addition, when the IP address of a specific image transmission destination corresponding to a set of predetermined area information to be stored in the predetermined area management DB5004 already exists, the distribution condition setting unit 55 may directly use the already existing IP address of the specific image transmission destination without setting the IP address of the image transmission source to "all". Further, the distribution condition setting unit 55 may, for example, have a configuration in which the IP address of the image transmission destination is input by the user A1 using the distribution condition setting screen 700 shown in FIG. 28, and the IP address of the specific image transmission destination is stored in association with the received predetermined area information.

[0126] ○ Distribution process of recorded data ○ Next, with reference to FIGS. 29 to 33, the distribution process of the video data recorded in the communication management system 50 will be described. FIG. 29 is a sequence diagram showing an example of the distribution process of the recorded video data. The example of FIG. 29 will describe the process of distributing the video data recorded by the communication management system 50 to the communication terminal 30C at the base C.

[0127] First, in response to a predetermined input operation by the user C1, the transceiver 31 of the communication terminal 30C transmits a video distribution request to the communication management system 50 (step S111). This video distribution request includes information for specifying the video data for which distribution is requested (for example, the IP address of the video data transmission source or the file name of the video data). Thereby, the transceiver 51 of the communication management system 50 receives the video distribution request transmitted from the communication terminal 30C.

[0128] Next, the storage / reading unit 59 of the communication management system 50 reads out the predetermined area information corresponding to the video data targeted by the video distribution request received in step S92 from the predetermined area management DB5004 (see FIG. 20) (step S112). In this case, the storage / reading unit 59 reads out the time stamp associated with the predetermined area information together with the target predetermined area information. Then, the transmission / reception unit 51 transmits a set of time information indicating the playback time specified by the predetermined area information read out in step S112 and the time stamp associated with the predetermined area information to the communication terminal 30C that is the request source (step S113). As a result, the transmission / reception unit 31 of the communication terminal 30C receives the set of the predetermined area information and the time information transmitted from the communication management system 50. Note that the communication management system 50 may perform the processes of step S113 and step S114 for all of the target predetermined area information stored in the predetermined area management DB5004 at once, or may perform them sequentially at regular intervals (for example, 30 seconds) according to the playback time of the video data.

[0129] Next, the storage / reading unit 39 of the communication terminal 30C stores, in the predetermined area management DB3003C (see FIG. 16), the time stamp associated with the predetermined area information and the time information received in step S113 (step S114).

[0130] Next, the storage / reading unit 59 of the communication management system 50 reads out the recorded data file of the video data targeted by the video distribution request received in step S92 from the video data management DB5003 (see FIG. 19) (step S115). Then, based on the recorded data file read out in step S115, the transmission / reception unit 51 sequentially transmits the photographed image data and the audio data related to the video data to the communication terminal 30C that is the request source (step S116). Each of these transmissions includes an image data ID for identifying the photographed image data that is the transmission target. As a result, the transmission / reception unit 31 of the communication terminal 30C receives the photographed image data and the audio data transmitted from the communication management system 50.

[0131] Then, the display control unit 34 of the communication terminal 30C plays the video data received in step S116 to display the captured image (step S117). Note that if there are a plurality of recorded data files recorded simultaneously in the same session, the communication management system 50 may distribute them simultaneously and display them in combination on the communication terminal 30C on the viewing side. Further, the communication management system 50 may perform data distribution to individual communication terminals 30 in response to the video distribution request in step S111, or may synchronize and distribute data to a plurality of communication terminals 30.

[0132] Here, with reference to FIGS. 30 to 33, a display example when the recorded video data is displayed on the communication terminal 30C will be described. First, with reference to FIGS. 30 and 31, the reproduction of the video data when the display directions of the video data are synchronized will be described. FIG. 30 is a flowchart showing an example of the reproduction process of video data in a communication terminal.

[0133] First, when the communication terminal 30C receives the predetermined area information transmitted from the communication management system 50 as shown in step S113 of FIG. 29 (YES in step S131), it proceeds to step S132. On the other hand, when the communication terminal 30C has not received the predetermined area (NO in step S131), the process ends.

[0134] Next, the storage / reading unit 39 of the communication terminal 30C associates the predetermined area information received in step S131 with a time stamp indicating the reproduction time of the video data corresponding to the predetermined area information and stores it in the predetermined area management DB 3003C (step S132) as shown in step S114 of FIG. 29.

[0135] Next, the determination unit 35 of the communication terminal 30C refers to the predetermined area management DB 3003C in which the predetermined area information was stored in step S132. If the predetermined area information corresponding to the time stamp of the video data transmitted from the communication management system 50 exists (YES in step S133), the process proceeds to step S134. On the other hand, if the determination unit 35 of the communication terminal 30C refers to the predetermined area management DB 3003C in which the predetermined area information was stored in step S132 and the predetermined area information corresponding to the time stamp of the video data transmitted from the communication management system 50 does not exist (NO in step S133), the process ends without synchronizing the display direction of the video data.

[0136] Next, the generation unit 37 of the communication terminal 30C synchronizes the display direction of the video data and generates a predetermined area image by performing a perspective projection transformation using the predetermined area information received in step S132 in order to display an image of the predetermined area specified by the predetermined area information stored in step S132 (step S134). Thereby, the communication terminal 30C can generate a predetermined area image corresponding to the distribution conditions set by the communication management system 50.

[0137] Then, the display control unit 35 causes the display 306 to display the predetermined area image generated in step S134 (step S135).

[0138] FIG. 31 is a diagram showing an example of a display screen displayed on a communication terminal when the display directions are synchronized. The display screen 400 shown in FIG. 31 shows an example when video data with synchronized display directions is displayed at a predetermined playback time. The display screen 400 includes a predetermined area image 410 generated from the video data in step S135, a synchronized icon 415 indicating that the display directions are synchronized, and a playback position display area 430 for displaying the playback position of the video data. Among these, the playback position display area 430 includes a play button 431 pressed when playing the video data, a pause button 432 pressed when pausing the playback of the video data, and a slider 435 indicating the playback history (playback position) over the entire playback time. The configurations of the play button 431, the pause button 432, and the slider 435 are the same as the configurations of the play button 731, the pause button 732, and the slider 735 shown in FIG. 28.

[0139] In this way, by causing the communication terminal 30C to display a predetermined area image corresponding to the distribution conditions set by the communication management system 50, the communication management system 50 can synchronize the display directions of the video data to be displayed on the communication terminal 30. In this case, the user C1 cannot change the display direction while viewing the predetermined area image 410 displayed on the display screen 400, and by checking the synchronized icon 415 displayed on the display screen 400, the user C1 can grasp that the display direction of the video data cannot be changed. Also, the predetermined area information transmitted from the communication management system 50 to the communication terminal 30C may be information transmitted from the communication terminal 30 on the viewing side during the live distribution of the video data and stored in the communication management system 50 as shown in step S74 of FIG. 26, or may be information stored in the communication management system 50 by the distribution condition setting process as shown in step S97 of FIG. 27.

[0140] Subsequently, with reference to FIGS. 32 and 33, the playback of video data when the display directions are not synchronized will be described. FIG. 32 is a flowchart showing an example of a display direction change process in a communication terminal.

[0141] First, when the display direction of the video data transmitted from the communication management system 50 is synchronized (YES in step S151), the communication terminal 30C ends the process. In this case, the communication terminal 30C displays a predetermined area image corresponding to the synchronized display direction as shown in steps S134 and S135 of FIG. 30. On the other hand, when the display direction of the video data is not synchronized (NO in step S151), the communication terminal 30C shifts the process to step S152.

[0142] Next, the communication terminal 30C determines whether it has received a change in the display direction of the video data (step S152). FIG. 33 is a diagram showing an example of a display screen displayed on the communication terminal when the display direction is not synchronized. The display screen 400 shown in FIG. 33 includes a predetermined area image 410 that is an image of an arbitrary predetermined area of the video data received in step S116, display direction change buttons 420 (420a, 420b, 420c, 420d) for changing the display direction of the video data, and a playback location display area 430 for displaying the playback location of the video data. The user C1 can change the display direction of the video data corresponding to the predetermined area image 410 to be displayed by selecting a desired display direction change button 420, for example. Note that the playback location display area 430 has the same configuration as that shown in FIG. 31.

[0143] When the reception unit 32 of the communication terminal 30C receives a change in the display direction by the user C1 selecting the display direction change button 420 (YES in step S152), the process is shifted to step S153. On the other hand, when the reception unit 32 of the communication terminal 30C has not received a change in the display direction (NO in step S152), the process ends.

[0144] Next, the storage / reading unit 39 of the communication terminal 30C stores, in the predetermined area management DB 3003C, predetermined area information indicating a predetermined area corresponding to the display direction received in step S152, in association with a time stamp indicating the playback time of the video data corresponding to the predetermined area information (step S153).

[0145] Next, the generation unit 37 of the communication terminal 30C generates a predetermined area image by performing perspective projection transformation using the predetermined area information stored in step S153 in order to display an image of the predetermined area specified by the predetermined area information stored in step S153 (step S154). Then, the display control unit 35 causes the display 306 to display the predetermined area image generated in step S154 (step S155).

[0146] In this way, when the display direction of the video data is not synchronized by the communication management system 50, the communication terminal 30C can display a predetermined area image corresponding to a desired display direction by the input operation of the user C1.

[0147] ● Effects of the Embodiment As described above, when the image communication system 1 distributes video data including a recorded omnidirectional image, the image communication system 1 reflects the synchronization of the display direction in the recorded video data, allows the viewer of the image to freely operate the display direction of the video data in a certain time period, and synchronizes the display direction in a certain time period. Thereby, the image communication system 1 can cause the communication terminal 30 to display an image in an appropriate display direction according to the reproduction time of the video data captured at another site.

[0148] In addition, the image communication system 1 can synchronize the display direction for displaying the recorded video data on the communication terminal 30 by the communication management system 50 setting predetermined area information corresponding to the reproduction time of the video data when performing recording distribution on the recording data file of the video data transmitted from the imaging device 10 or the communication terminal 30 at each site.

[0149] In the above embodiment, the predetermined area T is specified by predetermined area information indicating the shooting direction and the angle of view of the virtual camera IC in the three-dimensional virtual space including the omnidirectional image CE, but it is not limited thereto. For example, when the angle of view is used as a constant, the predetermined area T may be specified by predetermined point information indicating the center point CP or any point at the four corners of the rectangular predetermined area T in FIG. 7. Further, in the above embodiment, as an example of the omnidirectional image (omnidirectional panorama image), the case where the captured image (entire image) is a three-dimensional panorama image has been described, but it may also be a two-dimensional panorama image.

[0150] ●Supplementary Note● Each function of the embodiment described above can be realized by one or more processing circuits. Here, the "processing circuit" in the present embodiment refers to a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, and an ASIC (Application Specific Integrated Circuit), DSP (digital signal processor), FPGA (field programmable gate array), SOC (System on a chip), GPU (Graphics Processing Unit), and devices such as conventional circuit modules designed to execute each function described above.

[0151] In addition, the various tables of the embodiments described above may be generated by the learning effect of machine learning, and the data of each related item may be classified by machine learning, so that the tables may not be used. Here, machine learning is a technology for enabling a computer to acquire learning ability like a human. That is, the computer autonomously generates an algorithm necessary for judgment such as data identification from learning data taken in advance, and applies this to new data to make a prediction. The learning method for machine learning may be any one of supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, and deep learning, and further, a learning method combining these learning methods may be used, and the learning method for machine learning is not limited.

[0152] So far, the communication terminal, image communication system, image display method, and program according to an embodiment of the present invention have been described. However, the present invention is not limited to the above-described embodiment, and can be changed within the scope that those skilled in the art can conceive, such as addition, change, or deletion of other embodiments. As long as the effects of the present invention can be achieved in any aspect, it is included in the scope of the present invention.

Explanation of Reference Numerals

[0153] 1 Image communication system 10 Photographing device 30 Communication terminal 31 Transmission / reception unit (an example of reception means) 32 Reception unit (an example of reception means) 34 Display control unit (an example of display control means) 37 Generation unit (an example of generation means) 50 Communication management system 51 Transmission / reception unit (an example of photographing image data reception means, an example of video data transmission means) 53 Recording processing unit (an example of recording processing means) 55 Distribution condition setting unit (an example of setting means)

Prior Art Documents

Patent Documents

[0154] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-25640

Claims

1. A communication terminal capable of displaying a predetermined area of a part of a subject shown in a photographed image, receiving means for receiving video data including the photographed image and predetermined area information indicating a predetermined area to be displayed on the communication terminal for each reproduction time of the video data from a communication management system that manages the photographed image distributed from another communication terminal; display control means for displaying an image indicating the predetermined area indicated by the received predetermined area information when there is predetermined area information corresponding to the reproduction time of the received video data; reception means for receiving a selection of a predetermined area in which the received video data is to be displayed when there is no predetermined area information at the reproduction time of the received video data; the display control means displays a predetermined area image indicating the predetermined area for which the selection has been received; Communication terminal.

2. The communication terminal according to claim 1, further comprising: generation means for generating a predetermined area image indicating the predetermined area indicated by the received predetermined area information when there is predetermined area information at the reproduction time corresponding to the received predetermined area information; the display control means displays the generated predetermined area image. Communication terminal.

3. The communication terminal according to claim 1, further comprising: generation means for generating a predetermined area image indicating the predetermined area for which the selection has been received; the display control means displays the generated predetermined area image. Communication terminal.

4. The communication terminal according to any one of claims 1 to 3, wherein the video data is recorded data distributed by another communication terminal.

5. The communication terminal according to any one of claims 1 to 4, wherein the photographed image is an omnidirectional image.

6. An image communication system comprising the communication terminal according to any one of claims 1 to 5 and the communication management system, wherein the communication management system, has a photographed image data receiving means for receiving photographed image data transmitted from another communication terminal, when the received photographed image data is video data, has a recording processing means for recording the video data, and has a video data transmitting means for transmitting the recorded video data to the communication terminal. An image communication system comprising the above.

7. The image communication system according to claim 6, wherein the communication management system further has, a setting means for setting the predetermined area for each reproduction time in the recorded video data, and the video data transmitting means transmits the recorded video data and predetermined area information indicating the set predetermined area to the communication terminal. An image communication system.

8. The image communication system according to claim 7, wherein the setting means sets a different predetermined area for each communication terminal that is the transmission destination of the recorded video data.

9. The image communication system according to claim 7 or 8, further having, a storage means for storing in association the recorded video data and the predetermined area information indicating the set predetermined area. An image communication system.

10. An image display method executed by a communication terminal capable of displaying a predetermined area of a part of a subject shown in a photographed image, a receiving step of receiving, from a communication management system that manages the photographed image distributed from another communication terminal, video data including the photographed image and predetermined area information indicating a predetermined area to be displayed on the communication terminal for each reproduction time of the video data, When there is predetermined region information corresponding to the playback time of the received video data, a display control step of displaying an image indicating the predetermined region indicated by the received predetermined region information; When there is no predetermined region information at the playback time of the received video data, an acceptance step of accepting selection of a predetermined region in which the received video data is to be displayed is executed, Displaying a predetermined region image indicating the predetermined region for which the selection has been accepted; An image display method.

11. In a communication terminal capable of displaying a predetermined region of a part of a subject captured in a captured image, A receiving step of receiving video data including the captured image and predetermined region information indicating a predetermined region to be displayed on the communication terminal for each playback time of the video data from a communication management system that manages the captured image distributed from another communication terminal; When there is predetermined region information corresponding to the playback time of the received video data, a display control step of displaying an image indicating the predetermined region indicated by the received predetermined region information; When there is no predetermined region information at the playback time of the received video data, an acceptance step of accepting selection of a predetermined region in which the received video data is to be displayed is executed, Displaying a predetermined region image indicating the predetermined region for which the selection has been accepted; A program.

Images