Image delivery device, control method, and program

Abstract

To notify, when a plurality of images obtained by capturing a subject from mutually different directions by a plurality of image capture devices are displayed and played back on the plurality of display devices, the image capture devices that are capturing the images in accordance with an attention level based on a viewing status of the images.SOLUTION: An image delivery device for delivering images to a plurality of display devices, includes: acquiring means for acquiring a plurality of images generated by capturing a predetermined subject by a plurality of image capture devices from mutually different directions; image transmitting means for transmitting the plurality of images to the plurality of display devices; receiving means for receiving from the display devices viewing image information regarding the images being displayed and played back on the display devices; calculating means for calculating, on the basis of the viewing image information, an attention level for each of the image capture devices capturing the images; and information transmitting means for transmitting, to the image capture devices, notification control information for causing a notification unit, disposed in the same direction as a lens unit of the image capture devices, to make a notification based on the attention level.SELECTED DRAWING: Figure 12

Description

[Technical Field]

[0001] The present invention relates to a system for displaying and reproducing a plurality of images of a subject photographed from different directions by a plurality of imaging devices on a plurality of display devices. [Background technology]

[0002] Volumetric capture technology is known, which generates 3D spatial data from images captured by multiple imaging devices (hereinafter referred to as cameras) and generates images (free viewpoint video) captured from desired viewpoints by virtually moving the cameras in 3D space. Images generated by such technology have a problem in that the data volume is huge, which means that data processing and transmission take a long time when distributing them in real time. However, this problem is expected to be resolved with the future spread of 5G. Patent Document 1 describes a technology that generates images of virtual participants called avatars in viewer-participation programs to give viewers the feeling that they are actually participating in the program, and generates images captured from viewpoints corresponding to the avatars' positions.

[0003] Furthermore, when a performer is being photographed from different directions by multiple cameras on a filming location, the performer is notified of the camera that is capturing the image that is being broadcast. Patent Document 2 describes a technique for notifying the subject of the camera that is capturing the image that is being broadcast by operating a controller to control the lighting of a tally lamp. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2006-067405 [Patent Document 2] Japanese Patent Application Laid-Open No. 2013-162501 Summary of the Invention [Problem to be solved by the invention]

[0005] While Patent Document 1 allows viewers to view images from a desired viewpoint, it has the problem of making it difficult for subjects to know which camera to focus on. In response to this problem, Patent Document 2 addresses this problem by showing that, in the past, the image to be aired was uniquely determined by the distribution source, and the subject could identify the camera capturing the image to be aired by a tally lamp. However, assuming that free viewpoint video is distributed using the volumetric capture technology described above, each of multiple viewers will be able to view from a desired viewpoint, and therefore, it is thought that Patent Document 2 would make it difficult for the subject to identify the camera they should focus on.

[0006] The present invention has been made in consideration of the above-mentioned problems, and its purpose is to realize a technology that, when multiple images of a subject taken from different directions by multiple imaging devices are displayed and played back on multiple display devices, notifies the imaging device taking the images in accordance with the level of attention based on the viewing state of the images. [Means for solving the problem]

[0007] In order to solve the above problems and achieve the object, the present invention provides an image distribution device that distributes images to multiple display devices, comprising: an acquisition means that acquires multiple images generated by photographing a specified subject from different directions using multiple imaging devices; an image transmission means that transmits the multiple images to the multiple display devices; a receiving means that receives from the display devices viewed image information regarding the images being displayed and played back on the display devices; a calculation means that calculates an attention level for each imaging device that is capturing the images based on the viewed image information; and an information transmission means that transmits notification control information to the imaging devices to cause a notification based on the attention level to be executed by a notification unit arranged in the same orientation as the lens unit of the imaging devices. [Effects of the Invention]

[0008] According to the present invention, when multiple images of a subject taken from different directions using multiple imaging devices are displayed and played back on multiple display devices, a notification can be sent to the imaging device taking the images depending on the level of attention based on the viewing state of the images. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a configuration diagram of a system for distributing images according to a first embodiment. [Figure 2] FIG. 1 is a diagram illustrating the configuration of an imaging system according to a first embodiment. [Figure 3] FIG. 1 is a diagram illustrating a shooting environment according to a first embodiment. [Figure 4] FIG. 2 is a diagram illustrating an example of multi-stream data according to the first embodiment. [Figure 5] FIG. 2 is a diagram illustrating an example of an image selection screen according to the first embodiment. [Figure 6] FIG. 10 is a diagram illustrating an example of an image that has been subjected to stitching processing in the first embodiment. [Figure 7] 5A to 5C are diagrams for explaining a method for selecting stitched images according to the first embodiment. [Figure 8] FIG. 2 is a diagram illustrating an example of 3D space data according to the first embodiment. [Figure 9] 5A to 5C are diagrams for explaining a method for selecting an image from 3D spatial data according to the first embodiment. [Figure 10] 5A to 5C are diagrams for explaining a notification operation of a specific image capture device according to the first embodiment. [Figure 11] 5A to 5C are diagrams for explaining a method for linking notification operations of a plurality of image capture devices according to the first embodiment. [Figure 12] 5 is a flowchart illustrating an example of the operations of the imaging device, image delivery device, and display device according to the first embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention claimed. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0011] [First embodiment] First, the first embodiment will be described.

[0012] Below, we will describe an embodiment in which the present invention is applied to a system that notifies a subject so that the imaging device that the subject should focus on can be identified based on the viewing state of images captured by multiple imaging devices and distributed to multiple display devices.

[0013] <System Configuration> First, the system configuration of the first embodiment will be described with reference to Figures 1 and 2. Figure 1 is a diagram illustrating the configuration of a system for distributing images according to the first embodiment.

[0014] The system of this embodiment includes an imaging system 100 that distributes images captured by a plurality of imaging devices 110, and a display device 300 that displays the images distributed from the imaging system 100 so that they can be viewed by a viewer.

[0015] The photographing system 100 has multiple imaging devices 110 and an image delivery device 200. The photographing system 100 delivers images generated by the imaging devices 110 to a viewer's display device 300 using the image delivery device 200, and also acquires viewed image information related to the image being viewed on the display device 300, and controls each component. The imaging device 110 is a digital camera that photographs a subject, generates an image, and outputs it externally.

[0016] The display device 300 includes an image receiving unit 301, an image generating unit 302, a display control unit 303, a selection unit 304, a display unit 305, and an information transmitting / receiving unit 306. The display device 300 selects a desired image from images delivered from the image capturing system 100, displays and plays the selected desired image, and transmits viewing image information related to the selected desired image to the image capturing system 100. The image receiving unit 301 is a communication interface capable of receiving images via a predetermined communication method. The image generating unit 302 is an image processor that generates multiple images from a composite image. The display control unit 303 is a controller equipped with an arithmetic processor, ROM, RAM, etc. that controls the operation of the entire display device. The selection unit 304 is an operation unit such as a push button switch, rotary dial, or touch panel that accepts user operations. The display unit 305 is a display device such as a liquid crystal panel or an organic EL panel. The information transmitting / receiving unit 306 is a communication interface capable of transmitting and receiving data via a predetermined communication method.

[0017] Fig. 2 is a diagram illustrating an example of the configuration of the imaging system 100 of Fig. 1. The imaging system 100 includes a plurality of imaging devices 110 and an image delivery device 200. Fig. 12(a) is a flowchart illustrating an example of the operation of the imaging device 110.

[0018] The imaging device 110 includes an imaging unit 111, an imaging control unit 112, an image output unit 113, an information input / output unit 114, and a notification unit 115. The imaging unit 111 is an image sensor equipped with a photoelectric conversion element such as a CCD or CMOS. The imaging control unit 112 is a controller equipped with a calculation processor, a ROM, a RAM, etc. that controls the imaging operation of the imaging unit 111. The image output unit 113 is an output interface capable of transferring images. The information input / output unit 114 is an input / output interface capable of sending and receiving data. The notification unit 115 is a tally lamp equipped with a light-emitting element such as an LED that is visible to a subject and has the function of providing notification by emitting light. The notification unit 115 may be disposed in a position visible to a subject imaged through a lens unit provided in the imaging device 110. That is, the notification unit 115 is disposed facing the shooting direction, similar to the lens unit.

[0019] The imaging device 110 uses the imaging unit 111 and imaging control unit 112 to convert an optical image of a subject formed by an optical system into an electrical signal to generate an image (S1201). Various image processing operations are performed on the image, and metadata and audio data are added to the image, which is then converted into a file or streamed. The imaging device 110 uses the image output unit 113 to transmit the image to the image delivery device 200 (S1202).

[0020] The image delivery device 200 has an image input unit 201, an image synthesis unit 202, an image transmission unit 203, an information transmission / reception unit 204, a system control unit 205, and a recording unit 206. The image delivery device 200 performs predetermined image processing on images acquired from multiple imaging devices 110 and delivers the images to the viewer's display device 300. The image input unit 201 is an input interface capable of receiving images. The image synthesis unit 202 is an image processing processor that synthesizes multiple images. The image transmission unit 203 is a communication interface capable of delivering images using a predetermined communication method. The information transmission / reception unit 204 is a communication interface capable of sending and receiving data using a predetermined communication method. The system control unit 205 is a controller equipped with an arithmetic processor, ROM, RAM, etc. that controls the operation of the entire system. The recording unit 206 is a storage device such as a flash memory or a hard disk.

[0021] Next, the functions of the components of the imaging system 100 and the display device 300 will be described with reference to Fig. 2. Fig. 12(b) is a flowchart showing an example of the operation of the image delivery device 200. Fig. 12(c) is a flowchart showing an example of the operation of the display device 300.

[0022] In the image delivery device 200, the image input unit 201 receives image data generated by the multiple imaging devices 110 (S1211), and the image synthesis unit 202 generates synthetic image data by synthesizing the multiple images (S1212). The synthetic image data is multi-stream data in which multiple images are combined into one, image data obtained by stitching multiple image data, or 3D space data generated from multiple image data. The synthetic image data may be data in which multiple images are associated as a set, and may or may not be integrated into a single file or stream. The synthetic image data generated by the image synthesis unit 202 is delivered from the image transmission unit 203 to multiple display devices 300 via communication lines such as 4G, 5G, the Internet, or broadcast waves (S1213), and is displayed and played on each display device 300 of multiple viewers, allowing the viewers to view each image.

[0023] The display device 300 receives the composite image data delivered from the image delivery device 200 via the image receiving unit 301 (S1221). The display control unit 303 selects one of the multiple images included in the composite image data using the selection unit 304 using a selection method described later (S1222). An example of the selection method will be described later with reference to FIGS. 4 to 9. The composite image data is output to the image generation unit 302, which, based on data indicating the selected image, performs decoding and image superimposition processing on the composite image data under the control of the display control unit 303 to generate image data for display, and outputs the image data to the display unit 305. The image generation unit 302 generates display image data that displays only the selected image from the multiple images included in the composite image data on a full screen. Alternatively, the display image data is generated so that the selected image is positioned in the center of the screen and the unselected images are positioned at the edges of the screen with a smaller area than the selected image. The display unit 305 displays and plays back the image data for display generated by the image generation unit 302 (S1223).

[0024] Next, a method for transmitting viewing image information from the display device 300 to the image delivery device 200 will be described.

[0025] In the display device 300, the selection unit 304 can select an image to be viewed from multiple images received by the display device 300, and can execute various actions on the selected image. The various actions include, for example, instructions to evaluate the selected image being displayed and played back, to pay a fee, or to provide support. Viewed image information including identification information for identifying the image selected by the selection unit 304 and information indicating the content of the action is output to the display control unit 303. The display control unit 303 controls the processing performed by the image generation unit 302 in accordance with the data selected by the selection unit 304, and transmits the viewed image information of the selection unit 304 to the image delivery device 200 via the information transmission / reception unit 306 (S1224).

[0026] The image delivery device 200 receives the viewed image information transmitted from the plurality of display devices 300 via the information transmitting / receiving unit 204 (S1214) and outputs it to the system control unit 205. The system control unit 205 analyzes the viewed image information and updates the analysis results recorded in the recording unit 206 (S1215). The analysis results include information indicating the number of times the plurality of images generated by the plurality of image capture devices 110 have been viewed (the number of times the images have been selected on each display device) and the level of attention calculated from the actions performed. That is, information indicating the level of attention is generated for each image capture device 110 according to the analysis results. The analysis results may be updated at predetermined time intervals, when the amount of change in a parameter included in the viewed image information exceeds a predetermined threshold, or in response to a user's desired timing request.

[0027] When the analysis results recorded in the recording unit 206 are updated, the system control unit 205 generates notification control information for the multiple imaging devices 110 based on the analysis results (S1216) and outputs the information (S1217). The notification control information is received by the information transmitting / receiving unit 204 of the image delivery device 200, transmitted to the information input / output units 114 of the multiple imaging devices 110, and received by the imaging control units 112 of the multiple imaging devices 110 (S1203). The imaging control unit 112 controls the notification unit 115 based on the notification control information to perform notification processing (S1204). An example of the notification processing will be described later with reference to FIGS. 10 and 11. The system control unit 205 may generate notification control information for the multiple imaging devices 110 and transmit it to each imaging device 110. Alternatively, the system control unit 205 may generate notification control information that can be commonly recognized by the multiple imaging devices 110 and transmit it to each imaging device 110, and each imaging device 110 may individually analyze the notification control information and control the execution of notification processing based on the analysis results.

[0028] Next, a method for generating composite image data in the image delivery device 200 of the image capture system 100, and a method for selecting an image and transmitting viewing image information in the display device 300 will be described with reference to FIGS.

[0029] Fig. 3 is a diagram illustrating an example of an imaging environment in the imaging system 100 of this embodiment. In the example of Fig. 3, multiple imaging devices 110A-110H are arranged to surround a subject 500, and each imaging device 110A-110H captures an image of the subject 500 from a different direction. Multiple images 600A-600H captured by the multiple imaging devices 110A-110H are transmitted to the image delivery device 200, combined by the image combination unit 202, and delivered to multiple display devices 300 as combined image data.

[0030] Below, examples of image selection methods will be described when the composite image data is multi-stream data, a stitched image, or 3D space data. Also, while Fig. 3 illustrates an example of a state in which subject 500 is photographed by eight imaging devices 110A-110H, for the sake of simplicity, the following description will use an example of first to third images 600A-600C photographed by first to third imaging devices 110A-110C.

[0031] First, an image selection method when the composite image data is multi-stream data will be described with reference to FIGS.

[0032] 4 is a diagram illustrating an example of multi-stream data generated by the image synthesis unit 202. In FIG. 4, images A to C are first to third images 600A to 600C captured by the first to third imaging devices 110A to 110C. First to third metadata 610A to 610C including identification information for identifying one of the first to third imaging devices 110A to 110H that captured the image 600A to 600C are added to the first to third images 600A to 600C. The image synthesis unit 202 encodes the input image data of the first to third images 600A to 600C and the first to third metadata to generate one multi-stream data 611, and the image transmission unit 203 distributes the multi-stream data to the display device 300.

[0033] In the display device 300, the image generation unit 302 decodes the multi-stream data 611 to generate image data corresponding to the first to third images 600A to 600C, and displays the generated image data on the display unit 305. The selection unit 304 can accept a user operation to select an image to be displayed full-screen on the display unit 305 from the first to third images 600A to 600C. When the selection unit 304 accepts an instruction to select an image, an image selection screen 612 is displayed on the display unit 305, as shown in FIG. 5. The image selection screen 612 displays an image selection sub-screen 613, a previous image display sub-screen 614, and a next image display sub-screen 615. The image selection sub-screen 613 displays identification information of the first to third metadata 610A to 610C included in the multi-stream data 611, and the user can select an image with desired identification information by operating the selection unit 304. The previous image display subscreen 614 displays identification information and an image that precedes the identification information selected on the image selection subscreen 613. The next image display subscreen 615 displays identification information and an image that follow the identification information selected on the image selection subscreen 613. When the user changes the identification information selected on the image selection subscreen 613 by operating the selection unit 304, the display unit 305 displays an image corresponding to the identification information selected on the image selection subscreen 613 (the first image 600A in the example of FIG. 5), and updates and displays the identification information and images on the previous image display subscreen 614 and the next image display subscreen 615. When the user finally determines the identification information by operating the selection unit 304, the image selection subscreen 613, the previous image display subscreen 614, and the next image display subscreen 615 are hidden, and the image of the finally determined identification information is displayed on the full screen. Furthermore, the finally determined identification information is transmitted as viewing image information to the image distribution device 200 of the imaging system 100 by the information transmitting / receiving unit 306. In the image delivery device 200, the system control unit 205 counts the number of views (or the number of viewers) of the imaging device 110 that captured the image of the identification information received from the display device 300, and the count value is recorded in the recording unit 206.

[0034] Next, an image selection method when the composite image data is an image that has undergone stitching processing will be described with reference to FIGS.

[0035] FIG. 6 is a diagram illustrating an example of an image that has been subjected to stitching processing and that is generated by the image synthesis unit 202. In FIG. 6, images A to C are first to third images 600A to 600C captured by the first to third imaging devices 110A to 110C. A stitched image 621 is a synthesized image that has been generated by stitching the first to third images 600A to 600C. Stitching processing is a process that performs processing such as deformation, movement, and synthesis on multiple images to synthesize them into a single image. The image synthesis unit 202 performs stitching processing on the input image data of the first to third images 600A to 600C to generate a single stitched image 621. The image transmission unit 203 transmits stitched image data that has been encoded from the stitched image 621 to the display device 300.

[0036] In the display device 300, the image generation unit 302 decodes the stitching image data to generate image data corresponding to the stitched image 621, and displays the generated image data on the display unit 305. The selection unit 304 can accept a user operation to select a specific image region from the stitched image 621. When the selection unit 304 accepts an instruction to select an image, an image selection screen 622 is displayed on the display unit 305, as shown in FIG. 7. On the image selection screen 622, a region selection indicator 623 and a display confirmation screen 624 are superimposed on the stitched image 621. The user can select a desired region from the stitched image 621 as the image region that the user wishes to view by performing an operation such as moving, enlarging, reducing, or rotating the region selection indicator 623 using the selection unit 304. The image region selected by the region selection indicator 623 is displayed on the display confirmation screen 624, and the user can confirm the image region that the user wishes to view by looking at the image displayed on the display confirmation screen 624. Then, when the user operates the selection unit 304 to finally determine the image area that they want to view, the image selection screen 622 is hidden and the image of the finally determined image area is displayed full-screen. Furthermore, information about the finally determined image area is transmitted as viewed image information to the image delivery device 200 of the imaging system 100 by the information transmission / reception unit 306. In the image delivery device 200, the system control unit 205 counts the number of views of the imaging device 110 that captured the image of the image area received from the display device 300, and the count value is recorded in the recording unit 206. In this case, the count value of the number of views may be weighted according to the proportion of the area that the image area occupies.

[0037] Next, an image selection method when the composite image data is 3D space data will be described with reference to FIGS.

[0038] Fig. 8 is a diagram illustrating an example of 3D space data generated by the image synthesis unit 202. In Fig. 8, the image synthesis unit 202 of the image delivery device 200 generates a 3D space image 631 by acquiring three-dimensional information from image data generated by the first to third imaging devices 110A to 110C. The image transmission unit 203 encodes the 3D space data corresponding to the 3D space image 631 and delivers it to the display device 300.

[0039] In the display device 300, the image generation unit 302 decodes the 3D space data, and a 3D space image 631 is displayed on the display unit 305. The selection unit 304 is capable of accepting user operations to place a virtual camera 633 at a desired position in the 3D space image 631 and to pan, tilt, and zoom the virtual camera 633. When the selection unit 304 accepts an instruction to select an image, an image selection screen 632 is displayed on the display unit 305, as shown in FIG. 9 . The image selection screen 632 displays the virtual camera 633, a shooting assist display 634, and a display confirmation screen 635 superimposed on the 3D space image 631. The user can set the virtual camera 633 to a desired position in the 3D space image 631 and perform pan, tilt, and zoom operations by operating the selection unit 304. The shooting assist display 634 is auxiliary information that changes in response to pan, tilt, and zoom of the virtual camera 633, and a line indicating the shooting range of the virtual camera 633 is displayed in a discernible manner. In this embodiment, the shooting assist display 634 is displayed using a dashed line. However, information such as the position coordinates and focal length of the virtual camera 633 in the 3D space image 631 may also be displayed. The image generation unit 302 of the display device 300 generates image data captured by the virtual camera 633 based on the 3D space image 631 and the information about the virtual camera 633, and the image captured by the virtual camera 633 is displayed on the display confirmation screen 635. The user can adjust the shooting parameters of the virtual camera 633 while viewing the image displayed on the display confirmation screen 635. When the user operates the selection unit 304 to finally determine the shooting parameters of the virtual camera 633, the virtual camera 633, the shooting assist display 634, and the display confirmation screen 635 are hidden, and the image captured by the finally determined virtual camera 633 is displayed full-screen. Furthermore, information about the finally determined virtual camera 633 is transmitted as viewing image information by the information transmission / reception unit 306 to the image distribution device 200 of the imaging system 100. In the image delivery device 200, the system control unit 205 counts the number of views of the imaging device 110 that captured the image of the most selected imaging range on the multiple display devices 300 based on the shooting parameters of the virtual camera 633 received from the multiple display devices 300, and the count value is recorded in the recording unit 206.

[0040] In this embodiment, an example has been shown in which the photographing system 100 counts the number of times an image is viewed on each photographing device based on the viewed image information regarding the image displayed on the display unit 305. However, for example, information regarding actions such as rating the image being viewed or paying a fee may be sent to the photographing system 100, and the number of times each action is performed may be counted.

[0041] Furthermore, in this embodiment, the photographing system 100 transmits composite image data to the display device 300, and the display device 300 performs processing in accordance with the selection operation of the selection unit 304. However, the photographing system 100 may receive the selection operation of the selection unit 304 from the display device 300, perform processing in accordance with the selection operation, and transmit the processing results to the display device 300; for example, the data exchanged between the devices may be changed as appropriate.

[0042] 10 and 11, an example of a notification method using the viewed image information in the image capture system 100 will be described. The viewed image information transmitted from the multiple display devices 300 to the image capture system 100 is used for various purposes. In this embodiment, an example will be described in which notification is made so that the subject 500 can identify a specific image capture device 110 that the subject 500 should focus on from the multiple image capture devices 110A to 110H.

[0043] The viewed image information transmitted from the display devices 300 to the imaging system 100 is received by the information transmitting / receiving unit 204 of the image delivery device 200 and output to the system control unit 205. The system control unit 205 analyzes the viewed image information received from the multiple display devices 300 and updates the analysis results recorded in the recording unit 206. Table 1 shows an example of the analysis result data recorded in the recording unit 206. (Table 1) As shown in Table 1, the viewed image information includes information on the number of times an action was performed and the number of times an image captured by the multiple image capture devices 110A-110H selected by a viewer user was viewed. It should be noted that the information is not limited to the number of times, and may include the viewing time, the amount of payment for the action, and the rating. The system control unit 205 calculates the attention level of the multiple image capture devices 110A-110H using the viewed image information. As an example of calculating the attention level A, an attention weight Wi is set for actions such as the number of times viewed and a high rating, and the attention level A can be calculated using Equation 1 using the number of times an action was performed Ni by the multiple image capture devices 110A-110H. (Formula 1) A x =ΣW i ×N i A x is the attention level of the imaging device X, W i is the relative importance of the attention given to action i, N i is the number of times action i is executed.

[0044] In this embodiment, the degree of attention is calculated using an arithmetic formula, but the calculation method can be changed as appropriate, such as a calculation method using machine learning that uses a learning model such as a neural network.

[0045] The system control unit 205 generates notification control information for each of the imaging devices 110A-110H based on the attention level A of each of the imaging devices 110A-110H. The notification control information is control information for controlling the notification unit 115 of the second imaging device 110B, which has a higher attention level than a predetermined reference value (predetermined value) or the highest attention level (hereinafter abbreviated as "high attention level"). The notification control information is transmitted to the second imaging device 110B via the information transmission / reception unit 204. The second imaging device 110B receives the notification control information from the image delivery device 200 via the information input / output unit 114 and outputs it to the imaging control unit 112. The imaging control unit 112 generates a control signal based on the notification control information and drives the notification unit 115. Note that when the imaging device with the highest attention level is to perform notification processing, the notification control information only needs to be transmitted to the second imaging device 110B and does not need to be transmitted to the imaging devices 110A, 110C-110H. Alternatively, notification control information indicating that notification processing does not need to be executed, or notification control information for executing notification processing indicating that the degree of attention is not high, may be transmitted to the imaging devices 110A, 110C to 110H.

[0046] FIG. 10 illustrates an example of a notification form of the notification unit 115 when the second image capture device 110B has a high level of attention. When the system control unit 205 determines that the second image capture device 110B has a high level of attention among the multiple image capture devices 110A to 110H, it transmits notification control information to the second image capture device 110B. Upon receiving the notification control information, the second image capture device 110B turns on the notification unit 115B to notify the subject 500 that the image captured by the second image capture device 110B is attracting attention from the viewer. The notification unit 115 may be any device that allows the subject to be visually recognized, such as a tally lamp. Furthermore, although the present embodiment has described an example in which the notification unit 115 of the second image capture device 110B, which has a high level of attention, is driven, the notification units 115 of the multiple image capture devices 110A to 110H may be driven to change the brightness or color of their light emission depending on the level of attention. For example, notification unit 115 may be configured to emit light when the level of attention is higher than a predetermined reference value, when the level of attention is the lowest, or when the level is lower than a predetermined reference value. Alternatively, notification unit 115 may emit light in any of these cases, and the color, lighting, or blinking of the light may be different depending on the case so that the subject can identify the level of attention.

[0047] In addition, in this embodiment, the system control unit 205 of the image delivery device 200 generates notification control information and transmits it to a specific imaging device, but each of the multiple imaging devices 110A to 110H may acquire the analysis results recorded in the recording unit 206 of the image delivery device 200 and generate notification control information using the imaging control unit 112.

[0048] Furthermore, the photography system 100 may perform notification by coordinating the notification units 115 of the multiple image capture devices 110 as shown in Fig. 11. In Fig. 11, if the specific image capture device that has a high level of attention and that the subject 500 should gaze at is the sixth image capture device 110F, the sixth image capture device 110F is located behind the subject 500 and out of the field of view of the subject 500. In such a case, the system control unit 205 generates and transmits notification control information to the multiple image capture devices 110A-110H based on the levels of attention of the multiple image capture devices 110A-110H and their positional relationships with the subject 500, so as to notify the subject 500 that the first to fifth, seventh, and eighth image capture devices 110A-110E, 110G, and 110H other than the sixth image capture device 110F should gaze at the sixth image capture device 110F in a coordinated manner as shown in Fig. 11. Of the notification units 115A to 115F of the first to sixth image capture devices 110A to 110F, the notification unit 115F of the sixth image capture device 110F that the subject 500 should gaze at is illuminated most brightly, and the notification unit 115B of the third image capture device 110B that is farthest from the sixth image capture device 110F is illuminated most dimly, and the notification units 115A, 115C to 115E, 115G, and 115H of the first, third to fifth, seventh, and eighth image capture devices 110A, 110C to 110E, 110G, and 110H are illuminated more brightly as they get closer to the sixth image capture device 110F. Furthermore, notification units 116A to 116F different from the notification units 115A to 115F may be provided to display symbols such as arrows to notify the subject 500 that they should gaze at and guide him or her to the sixth image capture device 110F. In addition, the notification method of the notification unit 115 can be changed as appropriate, such as by changing the color or by displaying an arbitrary image.

[0049] In addition, in the photography system 100, the system control unit 205 distributes composite image data and controls the notification unit 115 based on the attention levels of the multiple image capture devices 110A to 110H. In addition, when an image of an image capture device designated in advance by a distribution source is distributed, the control method may be switched to drive the notification unit of the image capture device of the image being distributed. This makes it possible to switch between control for notifying a pre-designated image capture device regardless of the number of views and control for notifying an image capture device with an image that is attracting a lot of attention.

[0050] Furthermore, if the notification unit changes frequently in response to changes in highly popular images (number of views), there is a concern that the subject's gaze may fluctuate as they follow the notification unit. Therefore, the notification unit may be controlled to change the content of its notification processing after a predetermined time (fixed or variable) has elapsed when the level of attention changes, or to change the content of its notification processing when the number of views has changed by more than a predetermined value.

[0051] According to the first embodiment, it is possible to provide a notification that enables a subject to identify the imaging device that the subject should focus on based on the viewing state of images captured by multiple imaging devices and distributed to multiple display devices.

[0052] [Second embodiment] Next, a second embodiment will be described.

[0053] In the above-described first embodiment, the viewed image information collected from the multiple display devices 300 is used by the imaging system 100 to provide a notification that enables the subject 500 to identify the specific imaging device 110 that the subject 500 should focus on. In contrast, in the second embodiment, the viewed image information is used to notify viewers of other display devices 300 of the image that a viewer of one display device 300 is paying attention to.

[0054] It is expected that the attention levels of images captured by multiple image capture devices 110 will change over time. Therefore, if a user starts viewing distributed images partway through, the user may be confused about which image capture device's image to view. Therefore, in the second embodiment, the viewed image information of a specific display device 300 is used to notify viewers of other display devices 300 of images that a viewer of a specific display device 300 is paying attention to.

[0055] The configurations and functions of the system and device of the second embodiment are the same as those of the first embodiment.

[0056] In the second embodiment, the imaging system 100 embeds the viewing image information acquired from a specific display device 300 and the attention level calculated by the system control unit 205 as metadata in composite image data and transmits the composite image data to another display device 300. The composite image data with the embedded metadata is transmitted to the other display device 300 by the image transmission unit 203, as in the first embodiment. The other display device 300 receives the composite image data with the embedded metadata by the image receiving unit 301 and outputs it to the image generation unit 302. The image generation unit 302 extracts the metadata from the composite image data. Then, when the other display device 300 receives the composite image data from the image delivery device 200 and initially displays it on the display unit 305, it references the metadata and controls the display so as to select and display an image with a high attention level on the specific display device 300. Furthermore, information such as an image attracting attention from a viewer or a shooting range on the specific display device 300 may be presented on the image selection screens of FIGS. 5, 7, and 9, or a message may be sent to notify the viewer of the image with a high attention level.

[0057] According to the second embodiment, when a user starts viewing distributed images halfway through, the user can start viewing without wondering which image capture device's image to view.

[0058] [Other embodiments] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0059] The invention is not limited to the present embodiment, and various changes and modifications can be made without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0060] 100... photography system, 110... imaging device, 115... notification section, 200... image delivery device, 202... image synthesis section, 205... system control section, 300... display device, 303... display control section, 304... selection section, 305... display section

Claims

1. An image distribution device that distributes images to a plurality of display devices, an acquisition means for acquiring a plurality of images generated by photographing a predetermined subject from different directions using a plurality of imaging devices; an image transmitting means for transmitting the plurality of images to the plurality of display devices; a receiving means for receiving, from the plurality of display devices, viewing image information relating to images being displayed and played back on the plurality of display devices; a calculation means for calculating an attention level for each imaging device capturing the image based on the viewing image information; An image distribution device characterized by having an information sending means for sending notification control information to the imaging device to cause a notification based on the attention level to be executed by a notification unit arranged in the same orientation as the lens unit of the imaging device.

2. 2. The image delivery device according to claim 1, wherein the information transmission means transmits the notification control information to an image capture device capturing an image with a degree of attention higher than a predetermined value.

3. The image distribution device described in claim 1, characterized in that the notification control information is information for executing a notification process that can identify whether the attention level is higher than a predetermined value, or the attention level is high, or the attention level is lower than a predetermined value, or the attention level is lowest.

4. the notification unit has a function of providing notification by emitting light or displaying a symbol, 4. The image delivery device according to claim 1, wherein the notification control information includes information for changing the brightness of light emitted by the notification unit or the display of symbols.

5. the viewed image information is viewed image information relating to an image selected by a viewer and displayed and played back on each of the plurality of display devices, further comprising an analysis means for analyzing the number of times a predetermined action is performed or the number of times each image generated by the imaging device is viewed based on a plurality of pieces of viewing image information received from each of the plurality of display devices; 5. The image delivery device according to claim 1, wherein the calculation means calculates the attention level in accordance with the analysis result of the analysis means.

6. The image distribution device described in claim 5, characterized in that the specified action is the receipt of an instruction from a viewer to give a high rating or an instruction to pay a fee for an image being displayed and played on the multiple display devices.

7. the plurality of images are combined; The viewing image information relates to a shooting range of the synthesized image set in each of the plurality of display devices, 7. The image delivery device according to claim 1, wherein the calculation means calculates the attention level for each imaging device capturing an image in a capturing range of the composite image.

8. A control method for an image delivery device that delivers images to a plurality of display devices, comprising: acquiring a plurality of images generated by photographing a predetermined subject from different directions using a plurality of imaging devices; transmitting the plurality of images to the plurality of display devices; receiving, from the display device, viewing image information relating to an image being displayed and played back on the display device; calculating an attention level for each imaging device capturing the image based on the viewed image information; a step of transmitting notification control information to the imaging device to cause a notification based on the attention level to be executed by a notification unit arranged in the same orientation as a lens unit of the imaging device.

9. A computer-readable program for causing a computer to execute the control method according to claim 8.

Images