Image distribution apparatus, display apparatus, camera apparatus, control method, and medium

Abstract

An image distribution device for distributing images to a plurality of display devices, the image distribution device including: an obtaining section for obtaining a plurality of images generated by a plurality of imaging devices capturing images of a predetermined subject from different directions; an image transmitting section for transmitting the plurality of images to the plurality of display devices; a receiving section for receiving viewing image information from the plurality of display devices regarding images being displayed / reproduced in the plurality of display devices; a calculating section for calculating a degree of attention of each imaging device that captured an image based on the viewing image information; and an information transmitting section for transmitting notification control information for causing a notification unit of each imaging device to give a notification based on the degree of attention, the notification unit being disposed in the same orientation as a lens unit of the imaging device.

Description

Technical Field

[0001] The present invention relates to a system for displaying / reproducing multiple images of a subject captured from different directions by multiple camera devices on multiple display devices. Background Technology

[0002] For example, volumetric capture technology is known, which uses images captured by multiple camera devices (hereinafter referred to as "cameras") to generate 3D spatial data and generates images taken from a desired viewpoint (free viewpoint images) by virtually moving the cameras in 3D space. Images generated by these technologies have a huge amount of data, thus presenting a problem of time-consuming data processing and transmission when allocating images in real time. However, with the future widespread adoption of 5G, this problem is expected to be solved. Patent Document 1 describes a technology that generates images of virtual participants called avatars in viewer-participatory programs to provide viewers with the atmosphere of the viewer's participation in the program, and generates images taken from a viewpoint corresponding to the avatar's position.

[0003] When multiple cameras are capturing images of a performer from different directions at a shooting location, the performer is notified of the camera capturing the live image. Patent Document 2 describes a technique that notifies the subject of the camera capturing the live image by operating a controller to control the turning on and off of a tally lamp.

[0004] Reference List

[0005] Patent documents

[0006] Patent Document 1: Japanese Patent Application Publication No. 2006-067405

[0007] Patent Document 2: Japanese Patent Application Publication No. 2013-162501 Summary of the Invention

[0008] Technical issues

[0009] In Patent Document 1, viewers can view the image from their desired viewpoint, but the subject cannot easily discern which camera they should be looking at. Regarding this issue, according to Patent Document 2, traditionally, live images are uniquely determined by a distributor, allowing the subject to identify the camera capturing the live image based on indicator lights. However, when assuming the distribution of free-viewpoint images using techniques such as the aforementioned volumetric imaging method, multiple viewers can view the subject's image from their respective desired viewpoints, potentially making it difficult for the subject to identify the camera they should be looking at using the technology in Patent Document 2.

[0010] The present invention was made in view of the above-mentioned problems and implements the following technology: when multiple display devices are displaying / reproducing multiple images of a subject taken from different directions by multiple camera devices, a notification corresponding to the attention level based on the viewing state of each image is given to the camera devices that took the images.

[0011] Technical means to solve the problem

[0012] To address the aforementioned problems, the present invention provides an image allocation device for allocating images to multiple display devices. The image allocation device includes: an obtaining component for obtaining multiple images generated by multiple camera devices capturing images of a predetermined subject from different directions; an image transmitting component for transmitting the multiple images to the multiple display devices; a receiving component for receiving viewing image information related to the images being displayed / reproduced on the multiple display devices; a calculating component for calculating the attention level of each camera device capturing the images based on the viewing image information; and an information transmitting component for transmitting notification control information to each camera device to cause a notification unit of that camera device to issue a notification based on the attention level, wherein the notification unit is arranged in the same orientation as the lens unit of the camera device.

[0013] Advantages of the present invention

[0014] According to the present invention, when multiple display devices are displaying / reproducing multiple images of a subject taken from different directions by multiple camera devices, a notification corresponding to the attention level based on the viewing state of each image can be given to the camera devices that took the images.

[0015] Other features and advantages of the invention will become clear from the following description taken in conjunction with the accompanying drawings. Note that throughout the drawings, the same reference numerals denote the same or similar components. Attached Figure Description

[0016] The accompanying drawings, which are included in and form part of this specification, illustrate embodiments of the invention and, together with the textual description, serve to explain the principles of the invention.

[0017] Figure 1 The configuration of the system for allocating images according to Embodiment 1 is shown.

[0018] Figure 2 An example of the configuration of the camera system according to Embodiment 1 is shown.

[0019] Figure 3 The camera environment according to Embodiment 1 is shown.

[0020] Figure 4 An example of multi-stream data according to Embodiment 1 is shown.

[0021] Figure 5 An example of selecting a screen according to Embodiment 1 is shown.

[0022] Figure 6 An example of a stitched image according to Embodiment 1 is shown.

[0023] Figure 7 A method for selecting and stitching images according to Embodiment 1 is shown.

[0024] Figure 8 An example of 3D spatial data according to Embodiment 1 is shown.

[0025] Figure 9 A method for selecting an image from 3D spatial data according to Embodiment 1 is shown.

[0026] Figure 10 The notification operation performed by a specific camera device according to Embodiment 1 is shown.

[0027] Figure 11 A method for linking notification operations performed by multiple camera devices according to Embodiment 1 is shown.

[0028] Figure 12A This is a flowchart illustrating an example of operation performed by a camera device according to Embodiment 1.

[0029] Figure 12B This is a flowchart illustrating an example of the operation performed by the image allocation device according to Embodiment 1.

[0030] Figure 12C This is a flowchart illustrating an example of operation performed by a display device according to Embodiment 1. Detailed Implementation

[0031] The embodiments will be described in detail below with reference to the accompanying drawings. Note that the following embodiments are not intended to limit the scope of the invention. Several features are described in the embodiments, but the invention is not limited to requiring all of these features; rather, these features can be appropriately combined. Furthermore, in the drawings, the same or similar configurations are given the same reference numerals, and redundant descriptions thereof are omitted.

[0032] Example 1

[0033] Example 1 will be described first.

[0034] The following describes an embodiment of applying the present invention to a system in which, based on the viewing status of images captured by multiple camera devices and distributed to multiple display devices, a notification is given to the subject that enables the subject to identify the camera device it is looking at.

[0035] System Configuration

[0036] First, refer to Figure 1 and Figure 2 The system configuration of Example 1 is described. Figure 1 An example of the configuration of the system for allocating images according to Embodiment 1 is shown.

[0037] The system of this embodiment includes a camera system 100 for distributing images captured by a plurality of camera devices 110, and a display device 300 for displaying the images distributed by the camera system 100 so that a viewer can view the images.

[0038] The camera system 100 includes multiple camera devices 110 and an image distribution device 200. The camera system 100 uses the image distribution device 200 to distribute images generated by the camera devices 110 to a viewer's display device 300, obtains viewing image information about the image being viewed on the display device 300, and controls the various components. Each camera device 110 is a digital camera that captures and generates images of the subject and outputs the generated images to an external device.

[0039] The display device 300 includes an image receiving unit 301, an image generating unit 302, a display control unit 303, a selection unit 304, a display unit 305, and an information sending / receiving unit 306. The display device 300 selects a desired image from images allocated by the camera system 100, displays / reproduces the selected image, and sends viewing image information about the selected desired image to the camera system 100. The image receiving unit 301 is a communication interface capable of receiving images using a predetermined communication method. The image generating unit 302 is an image processor that generates multiple images using composite images. The display control unit 303 is a controller having an arithmetic processor, ROM, RAM, etc., for controlling the operation of the entire display device. The selection unit 304 is an operation unit for accepting user operations; it may be a push-button switch, rotary dial, touchscreen, etc. The display unit 305 is a display device such as a liquid crystal panel or an organic EL panel. The information sending / receiving unit 306 is a communication interface capable of sending and receiving data using a predetermined communication method.

[0040] Figure 2 It shows Figure 1 An example of the configuration of the camera system 100. The camera system 100 includes multiple camera devices 110 and image distribution devices 200. Figure 12A This is a flowchart illustrating examples of the operations performed by each camera device 110.

[0041] Each camera device 110 includes a camera unit 111, a camera control unit 112, an image output unit 113, an information input / output unit 114, and a notification unit 115. The camera unit 111 is an image sensor including photoelectric conversion elements (such as CCD or CMOS). The camera control unit 112 is a controller having a processing unit, ROM, RAM, etc., for controlling the camera operation of the camera unit 111. The image output unit 113 is an output interface capable of transmitting images. The information input / output unit 114 is an input / output interface capable of sending and receiving data. The notification unit 115 is an indicator light that includes a light-emitting element (such as an LED) visible to the subject and has the function of giving a notification by emitting light. The notification unit 115 only needs to be located in a position visible to the subject, where an image of the subject is captured by a lens unit included in the camera device 110. That is, similar to the lens unit, the notification unit 115 faces the camera direction.

[0042] The camera device 110 uses the camera unit 111 and the camera control unit 112 to convert the optical image of the subject formed by the optical system into an electrical signal and generate an image (S1201). Various types of image processing are performed on the image, metadata and audio data are added to the processed image, and the data is converted into file or stream data. The camera device 110 uses the image output unit 113 to send the image to the image distribution device 200 (S1202).

[0043] Image distribution device 200 includes an image input unit 201, an image compositing unit 202, an image sending unit 203, an information sending / receiving unit 204, a system control unit 205, and a recording unit 206. The image distribution device 200 performs predetermined image processing on images obtained from multiple imaging devices 110 and distributes the processed images to a viewer's display device 300. The image input unit 201 is an input interface capable of receiving images. The image compositing unit 202 is an image processor that composites multiple images. The image sending unit 203 is a communication interface capable of distributing images using a predetermined communication method. The information sending / receiving unit 204 is a communication interface capable of sending and receiving data using a predetermined communication method. The system control unit 205 is a controller having an arithmetic processor, ROM, RAM, etc., for controlling the operation of the entire system. The recording unit 206 is a storage device such as a flash memory or a hard disk.

[0044] Next, we will refer to Figure 2 Describe the functions of the components of the camera system 100 and the display device 300. Figure 12B This is a flowchart illustrating an example of the operation performed by the image distribution device 200. Figure 12C This is a flowchart illustrating an example of the operation performed by the display device 300.

[0045] In the image distribution device 200, the image input unit 201 receives image data generated by multiple camera devices 110 (S1211), and the image synthesis unit 202 generates synthesized image data by compositing multiple images (S1212). The synthesized image data is multi-stream data integrating multiple images, image data obtained by stitching together data from multiple images, or 3D spatial data generated using data from multiple images. The synthesized image data simply needs to be data from multiple images that are correlated into a set, and can but does not necessarily need to be integrated into a single file or stream. The synthesized image data generated by the image synthesis unit 202 is distributed from the image transmission unit 203 to multiple display devices 300 via communication lines (such as 4G, 5G, or the Internet) or broadcast radio waves (S1213), and displayed / reproduced on the display devices 300 of multiple viewers, thereby enabling viewers to view the images.

[0046] In the display device 300, the image receiving unit 301 receives composite image data allocated from the image distributing device 200 (S1221). The display control unit 303 causes the selection unit 304 to select one of the multiple images included in the composite image data using the selection method described later (S1222). (Refer to later...) Figures 4 to 9 An example of the selection method is described. The synthesized image data is output to the image generation unit 302, and the image generation unit 302, under the control of the display control unit 303, decodes the synthesized image data based on data indicating the selected image, performs image overlay processing to generate display image data, and outputs the generated display image data to the display unit 305. For example, the image generation unit 302 generates display image data to be displayed in full screen only for the selected image among the multiple images included in the synthesized image data. Alternatively, the image generation unit 302 generates display image data such that the selected image is placed in the center of the screen, while the unselected images are placed at the ends of the screen with an area smaller than the selected image. The display unit 305 displays / reproduces the display image data generated by the image generation unit 302 (S1223).

[0047] Next, a method for sending viewing image information from display device 300 to image distribution device 200 will be described.

[0048] In the display device 300, the selection unit 304 can select an image to view from a plurality of images received by the display device 300, and can also perform various actions on the selected image. For example, various actions may include instructions to evaluate, pay, support, etc., on the selected image being displayed / reproduced. Viewing image information containing identification information is output to the display control unit 303, wherein the identification information specifies the content of the image and action selected by the selection unit 304. The display control unit 303 controls the processing performed by the image generation unit 302 according to the data selected by the selection unit 304, and causes the information sending / receiving unit 306 to send the viewing image information about the image selected by the selection unit 304 to the image distribution device 200 (S1224).

[0049] In the image distribution device 200, the information sending / receiving unit 204 receives viewing image information sent from multiple display devices 300 (S1214) and outputs the received information to the system control unit 205. The system control unit 205 analyzes the viewing image information and updates the analysis results recorded in the recording unit 206 (S1215). The analysis results include information representing the attention level of each image, which is calculated based on the number of times each image generated by multiple camera devices 110 is viewed (the number of times an image is selected in each display device) and the actions performed on the image. That is, based on the analysis results, information representing the attention level is generated for each camera device 110. The analysis results can be updated at predetermined intervals, updated when the change in parameters included in the viewing image information exceeds a predetermined threshold, or updated in response to a request made by a user at a desired time.

[0050] If the analysis results recorded in the recording unit 206 are updated, the system control unit 205 generates notification control information for multiple camera devices 110 based on the analysis results (S1216) and outputs the generated information to the camera devices 110 (S1217). The notification control information is received by the information sending / receiving unit 204 of the image distribution device 200, sent to the information input / output unit 114 of each camera device 110, and received by the camera control unit 112 of the camera device 110 (S1203). The camera control unit 112 controls the notification unit 115 to perform notification processing based on the notification control information (S1204). See below for further details. Figure 10 and Figure 11 An example of notification processing is described below. Notification control information can be generated by the system control unit 205 for multiple camera devices 110 and sent to each camera device 110. Furthermore, the system control unit 205 can generate notification control information that can be commonly recognized by the multiple camera devices 110, send the generated information to each camera device 110, and each camera device 110 can analyze the notification control information separately and control the execution of notification processing based on the analysis results.

[0051] Next, we will refer to Figures 3 to 9 The method for generating composite image data in the image distribution device 200 of the camera system 100, and the method for selecting an image and sending viewing image information in the display device 300 are described.

[0052] Figure 3 An example of the camera environment of the camera system 100 according to this embodiment is shown. Figure 3 In the example, multiple camera devices 110A to 110H are arranged around the subject 500 and capture images of the subject 500 from different directions. The multiple images 600A to 600H captured by the multiple camera devices 110A to 110H are sent to the image distribution device 200, combined by the image synthesis unit 202, and distributed as composite image data to multiple display devices 300.

[0053] The following describes examples of image selection methods for various cases where the synthesized image data is multi-stream data, stitched image data, and 3D spatial data. Figure 3 Examples of images of subject 500 captured by eight camera devices 110A to 110H are shown; however, for the sake of simplicity, the following description uses examples of first images 600A to third images 600C captured by three (i.e., first to third) camera devices 110A to 110C respectively.

[0054] First, refer to Figure 4 and Figure 5 Describe an image selection method when the synthesized image data is multi-stream data.

[0055] Figure 4 An example of multi-stream data generated by the image synthesis unit 202 is shown. Figure 4 In the image, images A to C correspond to first images 600A to third images 600C captured by first camera device 110A to third camera device 110C. First metadata 610A to third metadata 610C are added to the first images 600A to third images 600C, and each metadata includes identification information for identifying the corresponding camera device among the first camera device 110A to third camera device 110C that captured the images 600A to 600C. The image synthesis unit 202 encodes the input image data of the first images 600A to third images 600C and the first to third metadata to generate a single multi-stream data 611, and the image transmission unit 203 distributes the multi-stream data to the display device 300.

[0056] In the display device 300, the image generation unit 302 decodes the multi-data stream data 611 to generate image data corresponding to the first image 600A to the third image 600C, and displays the generated image data in the display unit 305. The selection unit 304 can accept user operation to select the image to be displayed in full screen in the display unit 305 from the first image 600A to the third image 600C. When the selection unit 304 receives an instruction for selecting an image, such as... Figure 5 As shown, an image selection screen 612 is displayed in the display unit 305. The image selection screen 612 displays an image selection sub-screen 613, a previous image display sub-screen 614, and a next image display sub-screen 615. The image selection sub-screen 613 displays identification information about the first metadata 610A to the third metadata 610C included in the multi-stream data 611, and the user can select an image with the desired identification information by operating the selection unit 304. The previous image display sub-screen 614 displays the identification information and image preceding the identification information selected in the image selection sub-screen 613 in sequence. The next image display sub-screen 615 displays the identification information and image following the identification information selected in the image selection sub-screen 613 in sequence. If the user changes the identification information selected in the image selection sub-screen 613 by operating the selection unit 304, the display unit 305 displays the image corresponding to the identification information selected in the image selection sub-screen 613. Figure 5 In the example of the first image 600A, the system updates the identification information and image displayed in the previous image display sub-screen 614 and the next image display sub-screen 615. Then, if the user finally determines the identification information by operating the selection unit 304, the image selection sub-screen 613, the previous image display sub-screen 614, and the next image display sub-screen 615 are hidden, and the image corresponding to the finally determined identification information is displayed in full screen. In addition, the information sending / receiving unit 306 sends the finally determined identification information as viewing image information to the image distribution device 200 in the camera system 100. In the image distribution device 200, the system control unit 205 counts the number of times (or the number of viewers) the camera device 110 captures the image corresponding to the identification information received from the display device 300, and records the count value in the recording unit 206.

[0057] Next, we will refer to Figure 6 and Figure 7 Describe an image selection method when the synthesized image data is stitched image data.

[0058] Figure 6 An example of a stitched image generated by the image synthesis unit 202 is shown. Figure 6In the image, images A to C are first images 600A to third images 600C captured by first camera device 110A to third camera device 110C, respectively. The stitched image 621 is a composite image generated by stitching the first images 600A to third images 600C. Stitching refers to the process of transforming, moving, and compositing multiple images to generate a single composite image. The image compositing unit 202 generates a stitched image 621 by stitching the input image data of the first images 600A to third images 600C. The image sending unit 203 sends the stitched image data obtained by encoding the stitched image 621 to the display device 300.

[0059] In the display device 300, the image generation unit 302 decodes the stitched image data and generates image data corresponding to the stitched image 621, and the display unit 305 displays the generated image. The selection unit 304 is capable of accepting user operation to select a specific image area from the stitched image 621. When the selection unit 304 receives an instruction for selecting an image, such as... Figure 7 As shown, an image selection screen 622 is displayed in the display unit 305. The image selection screen 622 displays a region selection indicator 623 and a display check screen 624 superimposed on the stitched image 621. The user can use the selection unit 304 to perform operations such as moving, zooming in, zooming out, and rotating the region selection indicator 623, and select a desired region from the stitched image 621 as the image region to be viewed. The image region selected by the region selection indicator 623 is displayed in the display check screen 624, and the user can check the image region to be viewed by viewing the image displayed in the display check screen 624. If the user finally determines the image region to be viewed by operating the selection unit 304, the image selection screen 622 is hidden, and the image of the finally determined image region is displayed in full screen. Furthermore, the information sending / receiving unit 306 sends the finally determined image region as viewing image information to the image distribution device 200 in the camera system 100. In the image distribution device 200, the system control unit 205 counts the number of times the camera device 110 captures the image of the image area received from the display device 300, and records the count in the recording unit 206. In this case, the count of the number of views can be weighted according to the proportion of the area occupied by the image area.

[0060] Next, we will refer to Figure 8 and Figure 9 Describe an image selection method when the synthesized image data is 3D spatial data.

[0061] Figure 8 An example of 3D spatial data generated by the image synthesis unit 202 is shown. Figure 8In the image distribution device 200, the image synthesis unit 202 obtains 3D information from image data generated by multiple (i.e., first to third) camera devices 110A to 110C to generate a 3D spatial image 631. The image transmission unit 203 encodes the 3D spatial data corresponding to the 3D spatial image 631 and sends the encoded data to the display device 300.

[0062] In the display device 300, the image generation unit 302 decodes the 3D spatial data, and the 3D spatial image 631 is displayed in the display unit 305. The selection unit 304 can accept user operation to position a virtual camera 633 at a desired location in the 3D spatial image 631, and to pan, tilt, and zoom the virtual camera 633. When the selection unit 304 receives an instruction for selecting an image, such as... Figure 9 As shown, an image selection screen 632 is displayed in the display unit 305. The image selection screen 632 displays a virtual camera 633 superimposed on the 3D spatial image 631, a camera assistance display 634, and a display check screen 635. The user can use the selection unit 304 to set the virtual camera 633 to a desired position in the 3D spatial image 631 and can perform pan, tilt, and zoom operations. The camera assistance display 634 presents auxiliary information that changes according to the pan, tilt, and zoom of the virtual camera 633, and displays lines indicating the camera range of the virtual camera 633 in a recognizable manner. Although the camera assistance display 634 in this embodiment is presented with dashed lines, alternatively, information such as the position coordinates of the virtual camera 633 in the 3D spatial image 631 and its focal length can also be displayed. The image generation unit 302 of the display device 300 generates data of the image captured by the virtual camera 633 based on information about the 3D spatial image 631 and the virtual camera 633, and displays the image captured by the virtual camera 633 on the display inspection screen 635. The user can adjust the camera parameters of the virtual camera 633 while viewing the image displayed on the display inspection screen 635. Then, if the user finally determines the camera parameters of the virtual camera 633 by operating the selection unit 304, the virtual camera 633, the camera auxiliary display 634, and the display inspection screen 635 are hidden, and the finally determined image captured by the virtual camera 633 is displayed in full screen. Furthermore, the information sending / receiving unit 306 sends information about the finally determined virtual camera 633 as viewing image information to the image distribution device 200 in the camera system 100. In the image distribution device 200, the system control unit 205 counts the number of times the camera device 110, which captures the image with the most selected camera range by the multiple display devices 300, is viewed based on the camera parameters of the virtual camera 633 received from the multiple display devices 300, and records the count value in the recording unit 206.

[0063] Note that in this embodiment, an example is described in which the camera system 100 counts the number of times an image captured by each camera device is viewed based on viewing image information about the image being displayed in the display device 305. However, alternatively, for example, information about actions (such as ratings and payments for the viewed image) may also be sent to the camera system 100, and the camera system 100 may count the number of times each action is performed.

[0064] In this embodiment, composite image data is sent from the camera system 100 to the display device 300, and the display device 300 performs processing corresponding to the selection operation of the selection unit 304. However, the data exchanged between the devices can be appropriately modified; for example, the camera system 100 can receive the selection operation from the selection unit 304 from the display device 300, perform processing corresponding to the selection operation, and send the processing result to the display device 300.

[0065] Next, we will refer to Figure 10 and Figure 11 An example of a notification method using viewed image information in a camera system 100 is described. Viewed image information sent from multiple display devices 300 to the camera system 100 is used in various applications. In this embodiment, an example will be described whereby a notification is given that enables a subject 500 to identify the specific camera device 110 that the subject 500 is looking at from among multiple camera devices 110A to 110H.

[0066] Viewing image information sent from display device 300 to camera system 100 is received by information sending / receiving unit 204 of image allocation device 200 and output to system control unit 205. System control unit 205 analyzes the viewing image information received from multiple display devices 300 and updates the analysis results recorded in recording unit 206. Table 1 shows an example of the analysis result data recorded in recording unit 206.

[0067] Table 1

[0068]

[0069] As shown in Table 1, the image viewing information includes information selected by the user (i.e., the viewer), such as the number of times each image captured by the multiple camera devices 110A to 110H has been viewed, and the number of times each action has been performed. Note that the image viewing information is not limited to the number of times; it can also include information such as viewing time, the amount paid for each action, and high ratings. The system control unit 205 uses the image viewing information to calculate the attention level of each of the multiple camera devices 110A to 110H. As an example of calculating attention level A, a weight Wi for attention level is set for the number of views and actions (such as giving a high rating), and attention level A is obtained from Equation 1 using the number of times the action was performed on the multiple camera devices 110A to 110H.

[0070] Equation 1

[0071] A x =ΣW i ×N i

[0072] A x W represents the level of attention given to camera device X. i The weight representing the level of attention given to action i, and N i This indicates the number of times action i has been performed.

[0073] Note that in this embodiment, an arithmetic expression is used to calculate attention, but the calculation method can be appropriately changed to, for example, a machine learning calculation method using a learning model (such as a neural network).

[0074] The system control unit 205 generates notification control information for each of the camera devices 110A to 110H based on the attention level A of each camera device 110A to 110H. For example, the notification control information is control information for controlling the notification unit 115 of the second camera device 110B, where the attention level of the second camera device 110B is higher than a predetermined reference value (predetermined value) or has the highest attention level (hereinafter collectively referred to as "high attention level"). The notification control information is sent to the second camera device 110B via the information sending / receiving unit 204. The second camera device 110B receives the notification control information from the image distribution device 200 using the information input / output unit 114 and outputs the received information to the camera control unit 112. The camera control unit 112 generates a control signal based on the notification control information and drives the notification unit 115. When the camera device with the highest attention level is to perform notification processing, the notification control information only needs to be sent to the second camera device 110B, and does not need to be sent to the cameras 110A, 110C to 110H. Alternatively, notification control information indicating that no notification processing is required or notification control information used to perform notification processing to indicate that low-attention notification control information may be sent to camera devices 110A and 110C to 110H.

[0075] Figure 10 An example of the notification mode of the notification unit 115 when the second camera device 110B has high attention is shown. If the system control unit 205 determines that the second camera device 110B has high attention among the attention of the multiple camera devices 110A to 110H, the system control unit 205 sends notification control information to the second camera device 110B. Upon receiving the notification control information, the second camera device 110B activates the notification unit 115B to notify the subject 500 that the image captured by the second camera device 110B is attracting the viewer's attention. The notification unit 115 only needs to be visually recognizable by the subject and may be, for example, an indicator light. In this embodiment, an example of driving the notification unit 115 of the second camera device 110B with high attention has been described; however, alternatively, a configuration may be adopted in which the notification unit 115 is driven according to each level of attention to change the brightness, color, etc. of the notification units 115 of the multiple camera devices 110A to 110H when emitting light. For example, each notification unit 115 may emit light when the attention level is higher than a predetermined reference value, when the attention level is lowest, or when the attention level is lower than a predetermined reference value. Alternatively, each notification unit 115 may emit light in any of the above situations and adopt different colors, lighting modes, or flashing modes as appropriate, so that the subject can recognize the attention level.

[0076] In this embodiment, the system control unit 205 of the image distribution device 200 generates notification control information and sends the generated information to a specific camera device; however, multiple camera devices 110A to 110H can each obtain the analysis results recorded in the recording unit 206 of the image distribution device 200 and cause the camera control unit 112 to generate notification control information.

[0077] like Figure 11 As shown, the camera system 100 can also interconnect the notification units 115 of multiple camera devices 110 to provide notifications. Figure 11 If the sixth camera device 110F is a specific camera device with high attention that the subject 500 wants to look at, then the sixth camera device 110F is located behind the subject 500 and outside the subject 500's line of sight. In this case, based on the attention of each camera device 110A to 110H and the positional relationship between the camera devices 110A to 110H and the subject 500, the system control unit 205 generates notification control information and sends the generated information to the camera devices 110A to 110H, so that the camera devices other than the sixth camera device 110F (i.e., the first camera devices 110A to the fifth camera device 110E, the seventh camera device 110G, and the eighth camera device 110H) notify the subject 500 in a mutually related manner that the subject 500 should look at the sixth camera device 110F. Figure 11 As shown. Then, the notification unit 115 is turned on, such that among the notification units 115A to 115F of the first camera device 110A to the sixth camera device 110F, the notification unit 115F of the sixth camera device 110F, which the subject 500 is looking at, emits the brightest light, and the notification unit 115B of the second camera device 110B, which is furthest from the sixth camera device 110F, emits the dimmest light, and the brightness of the light emitted by the notification units 115A, 115C to 115E, 115G, and 115H of the first camera device 110A, the third camera device 110C to the fifth camera device 110E, the seventh camera device 110G, and the eighth camera device 110H increases towards the sixth camera device 110F. Alternatively, a configuration can be adopted where notification units 116A to 116F, different from notification units 115A to 115F, are provided, and notification units 116A to 116F display symbols such as arrows to provide notification, thereby guiding the subject 500 to the sixth camera device 110F to which the subject 500 is to look. The notification method of notification unit 115 can also be appropriately changed; for example, notification can be provided by changing colors or displaying any image.

[0078] Note that, in addition to the configuration of the camera system 100 (in which the system control unit 205 allocates composite image data and controls the notification unit 115 based on the attention given to multiple camera devices 110A to 110H), if images captured by camera devices pre-specified by the allocator are being allocated, the control method can be switched to drive the notification unit of the camera device capturing the allocated image. This allows switching between control in the case of giving notifications to pre-specified camera devices (regardless of the number of views) and control in the case of giving notifications to camera devices with high attention.

[0079] Furthermore, if the notification unit is frequently switched in response to changes in images with high attention (view count), there is a concern that the subject's gaze may waver as they follow the notification unit. Therefore, controls can be implemented to change the content of the notification processing performed by the notification unit after a predetermined (fixed or variable) time has elapsed when attention changes, or to change the content of the notification processing when the number of views exceeds a predetermined value.

[0080] According to Embodiment 1, a notification can be given to the subject based on the viewing status of images captured by multiple camera devices and distributed to multiple display devices, so that the subject can identify the camera device that the subject is looking at.

[0081] Example 2

[0082] Example 2 will be described next.

[0083] In Embodiment 1 described above, the camera system 100 uses viewing image information collected from multiple display devices 300 to provide a notification, enabling the subject 500 to identify the specific camera device 110 that the subject 500 is looking at. Conversely, in Embodiment 2, viewing image information is used to provide a notification to viewers of other display devices 300 indicating that the viewers of those display devices 300 are focusing on an image.

[0084] It is anticipated that the level of attention given to images captured by multiple camera devices 110 will change over time. Therefore, if a user begins viewing an assigned image midway through the viewing process, the user may be confused about which image captured by the camera device should be viewed. In Embodiment 2, viewing image information about a specific display device 300 is used to notify viewers on other display devices 300 of the image that the viewer on the specific display device 300 is currently focusing on.

[0085] Note that the configuration and functions of the system and device in Embodiment 2 are the same as those in Embodiment 1.

[0086] In Embodiment 2, the camera system 100 embeds viewing image information obtained from a specific display device 300 and attention calculated by the system control unit 205 as metadata into the composite image data, and sends this data to other display devices 300. Similar to Embodiment 1, the image sending unit 203 sends the composite image data with embedded metadata to other display devices 300. Each of the other display devices 300 receives the composite image data with embedded metadata using the image receiving unit 301, and then the composite image data is output to the image generation unit 302. The image generation unit 302 extracts metadata from the composite image data. When each of the other display devices 300 receives the composite image data from the image distribution device 200 and displays the received data on the display device 305 for the first time, the display device 300 refers to the metadata and performs control to select and display images with high attention from the specific display device 300. Alternatively, the following configuration can be used: an image selection screen (e.g., Figure 5 , Figure 7 or Figure 9 (As shown) presents information such as the image and camera range that a viewer of a specific display device 300 is focusing on, or provides a notification of a highly relevant image in the form of a message.

[0087] According to Example 2, if a user starts watching the assigned image midway through the viewing process, the user can start watching the image without considering which image captured by the camera device they should be watching.

[0088] Other embodiments

[0089] The present invention can also be implemented by providing a program for implementing one or more functions in the above embodiments to a system or device via a network or storage medium, and causing one or more processors in the computer of the system or device to read and execute the program. The present invention can also be implemented by a circuit (e.g., an ASIC) that implements one or more functions.

[0090] This invention is not limited to the embodiments described above, and various changes and modifications can be made within the spirit and scope of this invention. Therefore, the appended claims are made to disclose the scope of this invention.

[0091] This application claims priority to Japanese Patent Application 2021-68531, filed April 14, 2021, and Japanese Patent Application 2022-9525, filed January 25, 2022, the entire contents of which are incorporated herein by reference.

[0092] List of reference numerals

[0093] 100…Camera system, 110…Camera equipment, 115…Notification unit, 200…Image distribution device, 202…Image composition unit, 205…System control unit, 300…Display device, 303…Display control unit, 304…Selection unit, 305…Display unit

Claims

1. An image distribution device for distributing images to multiple display devices, the image distribution device being characterized by comprising: A component for acquiring multiple images generated by capturing images of a predetermined subject from different directions using multiple camera devices; An image transmitting component is used to transmit the plurality of images to the plurality of display devices; A receiving component is configured to receive viewing image information related to the image being displayed / reproduced on the plurality of display devices from the plurality of display devices; A computing component is used to calculate the attention level of each camera device capturing the image based on the viewed image information; as well as An information sending component is used to send notification control information to each camera device to cause the notification unit of that camera device to give a notification based on the attention level, the notification unit being arranged in the same orientation as the lens unit of that camera device.

2. The image distribution device according to claim 1, characterized in that, If at least one camera device is capturing an image with a higher-than-predetermined level of attention, the information sending component sends the notification control information to the at least one camera device.

3. The image distribution device according to claim 1, characterized in that, The notification control information is information used to perform notification processing, which enables the identification of whether the attention level is higher than a predetermined value, the attention level is the highest, the attention level is lower than a predetermined value, or the attention level is the lowest.

4. The image allocation device according to claim 1, characterized in that, The notification unit has the function of giving notifications by illuminating or displaying symbols, and The notification control information includes information for changing the brightness of the notification unit when it emits light or changing the symbol displayed by the notification unit.

5. The image distribution device according to claim 1, characterized in that, The viewed image information is related to the images displayed / reproduced on each of the plurality of display devices selected by the viewer. The image allocation device further includes an analysis component, which analyzes the number of views or the number of actions performed for each image generated by the camera device based on multiple viewing image information received from each display device. The calculation component calculates the level of attention based on the analysis results performed by the analysis component.

6. The image distribution device according to claim 5, characterized in that, The action is an instruction from the viewer to give a high rating to the image being displayed / reproduced on the plurality of display devices, or an instruction to pay for the image being displayed / reproduced on the plurality of display devices.

7. The image distribution apparatus according to any one of claims 1 to 6, characterized in that, The multiple images are then combined. The viewed image information is related to the camera range of the composite image set in each of the plurality of display devices, and The computing unit calculates the attention level of each camera device capturing images within the camera range of the synthesized image.

8. A camera device, characterized in that... include: A generating component is used to generate an image by capturing an image of the predetermined subject from a direction different from the direction in which images of the predetermined subject are captured by other imaging devices; A transmitting component is used to transmit the generated image to an image distribution device, wherein the generated image is also transmitted from the image distribution device to a display device; A receiving unit is configured to receive notification control information from the image distribution device for performing attention-based notification processing, wherein the attention level is calculated based on viewing image information related to the image being displayed / reproduced in the display device; as well as A notification component is used to perform the notification processing based on the notification control information.

9. A control method for controlling an image distribution device, the image distribution device being used to distribute images to multiple display devices, the control method being characterized by comprising the following steps: To obtain multiple images generated by capturing images of a predetermined subject from different directions using multiple camera devices; Send the plurality of images to the plurality of display devices; Receive viewing image information related to the image being displayed / reproduced on the display device; Based on the viewed image information, the attention level of each camera device among the plurality of camera devices that captured the image is calculated; as well as Notification control information is sent to each camera device to enable the notification unit of that camera device to issue a notification based on the attention level, the notification unit being arranged in the same orientation as the lens unit of that camera device.

10. A control method for controlling a camera device, the control method being characterized by comprising the following steps: An image is generated by capturing an image of the predetermined subject from a direction different from the direction in which images of the predetermined subject are captured by other camera devices; The generated image is sent to an image distribution device, and the generated image is also transmitted from the image distribution device to a display device; The image distribution device receives notification control information for performing attention-based notification processing, wherein the attention level is calculated based on viewing image information related to the image being displayed / reproduced in the display device; as well as The notification processing is performed based on the notification control information.

11. A computer-readable storage medium storing a program that, when executed by a computer, causes the computer to perform the control method according to claim 9 or 10.

12. A computer program product comprising a program that, when executed by a computer, causes the computer to perform the control method according to claim 9 or 10.

Images