Imaging apparatus, control method for imaging apparatus, and program
The imaging device controls mode transitions based on distribution status to stabilize image quality by avoiding mode switches during image distribution, addressing abrupt color and brightness changes.
Patent Information
- Application Number
- JP2023082793
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-19
- Publication Date
- 2024-11-29
AI Technical Summary
Conventional imaging devices experience abrupt changes in color and brightness during mode transitions due to automatic adjustments based on environmental light changes, causing discomfort to viewers during image distribution.
An imaging device with a control system that determines whether an image is being distributed and only switches imaging modes when distribution is not occurring, using an optical filter to minimize color and brightness changes.
Minimizes color and brightness changes during mode transitions by ensuring imaging mode switches occur outside of distribution periods, maintaining consistent image quality for viewers.
Smart Images

Figure 2024166597000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to an imaging device, a control method for an imaging device, and a program. [Background technology]
[0002] In recent years, imaging devices that not only record imaging data on a recording medium but also have the function of transferring images being captured or already captured to a server, etc., via a network have come into widespread use. In addition, image distribution systems are being used in which a plurality of such imaging devices are connected via a network, images are collected in a server, etc., and one or a plurality of images are selected from the plurality of images and distributed to a large number of viewers via the network.
[0003] Furthermore, in a conventional imaging device, a method has been proposed in which an infrared cut filter installed on the optical path to the imaging element is inserted or removed depending on the amount of light incident from the subject to the imaging element, thereby enabling shooting suited to the imaging environment. For example, Patent Document 1 discloses a method of switching between a day mode, in which imaging is performed through a detachable infrared cut filter for an image sensor, and a night mode, in which imaging is performed without using an infrared cut filter, depending on the amount of light incident from a subject on the image sensor. [Prior art documents] [Patent documents]
[0004] [Patent Document 1] JP 2004-120202 A Summary of the Invention [Problem to be solved by the invention]
[0005] In an image capture device having a function of transferring images as described above, if switching between day mode and night mode is automatically performed in response to fluctuations in brightness in the image capture environment (i.e., the amount of light incident on the image capture device), the color and brightness of the image will change before and after the switching. For example, consider a case in which the amount of light of the lighting for capturing an object is instantly reduced to create a scene. If the conventional technology is applied to such a case, switching to night mode will be performed automatically in response to fluctuations in brightness in the image capture environment, and therefore, if switching to night mode is performed during image distribution, the color and brightness of the image will change before and after the switching. Images with such changes in color and brightness may cause discomfort to viewers who view the images.
[0006] The present invention has been made in consideration of the above-mentioned problems, and aims to provide a technology for minimizing changes in the color and brightness of an image even when the amount of light incident on an imaging device during image distribution fluctuates. [Means for solving the problem]
[0007] As one means for achieving the above object, the imaging device of the present invention has the following configuration: That is, the imaging device includes an imaging section including an imaging optical system, an optical filter inserted into and removed from an optical path of the imaging optical system, a first determination means for determining whether a luminance value of an image captured by the imaging section is equal to or less than a threshold value or exceeds the threshold value, a transmission means for transmitting the image captured by the imaging section to an external device, a second determination means for determining whether the image is being distributed or not when the first determination means determines that the luminance value of the image is equal to or less than the threshold value or exceeds the threshold value, and a control means for controlling the insertion and removal of the optical filter into and from the optical path when the second determination means determines that the image is not being distributed. Effect of the Invention
[0008] According to the present invention, even if the amount of light incident on an imaging device during image delivery fluctuates, it is possible to minimize changes in the color and brightness of an image. [Brief description of the drawings]
[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of an image delivery system according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of the configuration of an imaging device according to an embodiment. [Diagram 3] FIG. 3 shows an imaging mode switching sequence according to the first embodiment. [Figure 4] FIG. 4 shows a flowchart of the process executed by the imaging device according to the first embodiment. [Diagram 5] FIG. 5 shows an imaging mode switching sequence according to the second embodiment. [Figure 6] FIG. 6 shows a flowchart of the process executed by the imaging device according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. Note that the embodiment described below is one example of a means for realizing the present invention, and should be appropriately modified or changed depending on the configuration of the device to which the present invention is applied and various conditions, and the present invention is not limited to the embodiment described below.
[0011] <First embodiment> First, a first embodiment of the present invention will be described. In this embodiment, the term "image" is understood to include at least one of a still image and a video (moving image). In this embodiment, the term "image" may be understood to be synonymous with "image data."
[0012] [Image distribution system configuration] 1 is a diagram showing an example of the configuration of an image delivery system 100 according to this embodiment. It is assumed that the image delivery system 100 will be used in situations such as video production in a studio, live events, and image broadcasting at sports stadiums. The image delivery system 100 includes multiple imaging devices, and is configured to select one or more images of the same scene captured by the multiple imaging devices at the same time period, and deliver them to a viewer (user) in real time.
[0013] In FIG. 1, the image delivery system 100 includes imaging devices 101a, 101b, and 101c, an image selection device 102, and a server device 104. The image delivery system 100 may also include one or more lighting devices that irradiate light onto the shooting range of the imaging devices 101a, 101b, and 101c, and a lighting control device (none of which are shown) that controls the lighting devices. The imaging devices 101a, 101b, and 101c are imaging devices that are arranged at intervals in places such as studios, live events, and sports stadiums. For example, the imaging devices 101a, 101b, and 101c are arranged at positions that allow them to capture the same scene from different angles, and capture the image. The configurations of the imaging devices 101a, 101b, and 101c will be described later with reference to FIG. 2.
[0014] Images captured by the imaging devices 101a, 101b, and 101c are transmitted to the image selection device 102 via a wired LAN (Local Area Network) such as Ethernet (registered trademark) or a wireless LAN. Although three imaging devices 101a, 101b, and 101c are illustrated in Fig. 1, this is not limiting, and the image distribution system 100 may be configured to include one or more imaging devices.
[0015] The image selection device 102 functions as an information processing device, and is, for example, a hardware device such as a video switcher. Alternatively, the image selection device 102 may have its functions implemented in an arbitrary hardware device and be operated by a hardware device such as a personal computer (PC) or a smartphone. The operation is performed, for example, via a GUI (Graphical User Interface).
[0016] 1, the image selection device 102 selects one or more images from among images received from the imaging devices 101a, 101b, and 101c in response to an operation (e.g., an input operation) by an operator 105, and transmits the images to a server device 104 via a network 103a for distribution to viewers. Note that the selection of the one or more images is not limited to being based on the operation by the operator 105. The image selection device 102 can set the timing of image selection based on the operation by the operator 105, or arbitrarily.
[0017] Furthermore, the image selection device 102 transmits, via the server device 104, to the imaging devices 101a, 101b, and 101c, distribution information regarding whether or not the images received from the imaging devices 101a, 101b, and 101c are being distributed to viewers. For example, when the image received from the imaging device 101a is being distributed, the image selection device 102 transmits distribution information indicating that the image is being distributed to the imaging device 101a, and transmits distribution information indicating that the image is not being distributed to the imaging devices 101b and 101c. The imaging devices 101a, 101b, and 101c can determine whether or not the transmitted image is being distributed by receiving the distribution information as feedback for the image transmitted to the image selection device 102. Furthermore, the imaging devices 101a, 101b, and 101c may output information indicating whether or not the image captured by the imaging device is being distributed based on the received distribution information so that it can be recognized from the outside. For example, when the imaging devices 101a, 101b, and 101c each have a tally lamp, they may be configured to change the lighting color or lighting state according to the distribution information. The imaging device 101a may be configured to turn on the tally lamp when it receives distribution information indicating that an image is being distributed, and the imaging devices 101b and 101c may be configured not to turn on the tally lamp when it receives distribution information indicating that an image is not yet distributed. In this way, for example, when the subject is a performer, the performer can determine which imaging device is currently distributing an image captured by the imaging device based on the tally lamps provided in the imaging devices 101a, 101b, and 101c.
[0018] The image selection device 102 does not necessarily need to transmit the images received and selected from the imaging devices 101a, 101b, and 101c to the server device 104. For example, the image selection device 102 may output (i.e., record) the images received from the imaging devices 101a, 101b, and 101c to a storage medium (not shown) for distribution, or may output the images to an arbitrary external device (not shown) for use in a local area without going through the network 103b. Even in such a case, the image selection device 102 may transmit the above-mentioned distribution information to each of the imaging devices 101a, 101b, and 101c. For example, when the image received from the imaging device 101a has been output, but the images received from the imaging devices 101b and 101c have not been output, the image selection device 102 may transmit distribution information indicating that an image is being distributed to the imaging device 101a, and may transmit distribution information indicating that the image is not being distributed to the imaging devices 101b and 101c.
[0019] Moreover, the image selection device 102 can perform various settings by transmitting control signals to each of the image capture devices 101a, 101b, and 101c. For example, the image selection device 102 can change image capture settings including image capture parameters including exposure sensitivity and exposure time, the orientation of the image capture device, and control of a lens unit (corresponding to the lens unit 201 in FIG. 2) or an optical filter (corresponding to the optical filter 202 in FIG. 2).
[0020] The network 103a is a network that mediates between the image selection device 102 and the server device 104. The network 103b is a network that mediates between the server device 104 and the display device 106. An example of the networks 103a and 103b is the Internet.
[0021] Server device 104 temporarily holds the image acquired from image selection device 102 via network 103a. Furthermore, server device 104 outputs the image to display device 106 used by the viewer via network 103b. In other words, server device 104 distributes the image to the viewer.
[0022] The display device 106 is a display device used by a viewer. The display device 106 displays an image received from the server device 104 via the network 103b. The display device 106 is, for example, a display device such as a display connected to a PC, a smartphone, or a television, and may be realized by any device as long as it can be connected to the network 103b and can output images.
[0023] [Configuration of imaging device] Next, a description will be given of the configurations of the imaging devices 101a, 101b, and 101c in the image delivery system 100 shown in Fig. 1. In this embodiment, the imaging devices 101a, 101b, and 101c have the same configuration, and may be collectively referred to as the imaging device 101 in the following description.
[0024] Fig. 2 is a diagram showing an example of the configuration of the imaging device 101. In Fig. 2, the imaging device 101 includes a lens unit 201, an optical filter 202, an imaging section 203, a lens unit driving section 204, an optical filter driving section 205, an image processing section 206, a control section 208, a distribution determination section 207, and a communication interface section (I / F section) 209. The imaging device 101 may also include a tally lamp (illumination section) (not shown).
[0025] The lens unit 201 includes a plurality of optical lenses that constitute an imaging optical system. The lens unit 201 can be an interchangeable lens that can be attached to the imaging device 101, or a lens unit that is integrated into the imaging device 101. The lens unit driving section 204 drives the lens unit 201 based on a control signal from the control section 208 .
[0026] The optical filter 202 is a filter for selecting light to the imaging unit 203 described later. In this embodiment, the optical filter 202 includes an infrared cut filter (IRCF). The infrared cut filter is a filter that blocks only infrared light and transmits visible light. However, the optical filter 202 is not limited to this, and may include an ND (Neutral Density) filter or the like. The optical filter driving unit 205 drives the optical filter 202 upon receiving a control signal from the control unit 208. In this embodiment, the optical filter driving unit 205 controls whether to block infrared light and transmit light including visible light, or to transmit light including infrared light and visible light. As described above, in this embodiment, the optical filter 202 includes an infrared cut filter. The optical filter driving unit 205 performs insertion / removal control of an infrared cut filter as the optical filter 202 based on a control signal from the control unit 208. In this way, the optical filter driving unit 205 can control whether to insert an infrared cut filter into the optical path reaching the imaging unit 203 from the lens unit 201.
[0027] The imaging unit 203 converts light from a subject, which is focused on an imaging plane by the lens unit 201 and the optical filter 202, into an analog electrical signal for each pixel and outputs the signal to the image processing unit 206. The imaging unit 203 can set and change an exposure time or an analog gain for each pixel or pixel block on the imaging plane, based on a control signal from the control unit 208. Furthermore, the imaging unit 203 includes a solid-state imaging element such as a CCD (Charge-Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The solid-state imaging element has high sensitivity mainly to visible light, and also has sensitivity to infrared light. Therefore, a subject illuminated with visible light and infrared light can be clearly imaged.
[0028] The image processing unit 206 generates a digital electrical signal by performing predetermined analog processing and analog-to-digital conversion on the analog electrical signal output from the imaging unit 203. Furthermore, the image processing unit performs various corrections on the generated digital electrical signal, and then performs compression encoding processing and the like to generate an image (image data). The image generated by the image processing unit 206 (in this disclosure, also referred to as an image captured by the imaging unit 203) is transmitted by the control unit 208 to the image selection device 102 via the communication I / F unit 209.
[0029] The distribution determination unit 207 determines whether an image captured by the imaging unit 203 has been selected by the image selection device 102 and distributed to a viewer, or whether it has been output to a storage medium or an external device. For example, the distribution determination unit 207 acquires the above-mentioned distribution information via the communication I / F unit 209, and when the distribution information indicates that an image is being distributed, determines that an image generated by the image processing unit 206 is being distributed or has been output to a storage medium or an external device. On the other hand, when the distribution information indicates that the image is not being distributed, the distribution determination unit 207 determines that an image captured by the imaging unit 203 is not being distributed or has not been output to a storage medium or an external device.
[0030] The control unit 208 may be configured with an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field-Programmable Gate Array). The control unit 208 is connected to the imaging unit 203, the lens unit driving unit 204, the optical filter driving unit 205, the image processing unit 206, the distribution determination unit 207, and the communication I / F unit 209, and controls various functions related to the imaging device 200. For example, the control unit 208 transmits a control signal received from the image selection device 102 via the communication I / F unit 209 to the imaging unit 203, the lens unit driving unit 204, and the optical filter driving unit 205 according to the content of the control signal. The control unit 208 may also generate a control signal according to a predetermined program and transmit the control signal to the imaging unit 203, the lens unit driving unit 204, the optical filter driving unit 205, the image processing unit 206, and the distribution determination unit 207.
[0031] Furthermore, in this embodiment, the control unit 208 has a role of detecting a change in brightness in the imaging environment of the imaging device 200 based on the brightness values of the image acquired from the image processing unit 206. Specifically, the control unit 208 determines whether the average brightness value of the image acquired from the image processing unit 206 is equal to or less than a threshold value or exceeds a threshold value. For example, when the control unit 208 determines that the average brightness value of the image is equal to or less than the threshold value, it determines that a change in brightness has occurred (from bright to dark). A change in brightness in the imaging environment can occur due to control of one or more lighting devices by a lighting control device.
[0032] Furthermore, if the imaging device 101 is equipped with a tally lamp (not shown), the control unit 208 may control the lighting color and lighting state of the tally lamp in accordance with the distribution information received from the image selecting device 102. For example, when the distribution information indicates that an image is being distributed, the control unit 208 may control the tally lamp to be turned on, and when the distribution information indicates that an image is not yet being distributed, the control unit 208 may control the tally lamp not to be turned on.
[0033] The communication I / F unit 209 is an interface for connecting to the image selection device 102 in the image distribution system 100. The communication I / F unit 209 is connected to the image selection device 102 via a wired LAN such as Ethernet or a wireless LAN. The imaging device 101 can transmit images to the image selection device 102 and receive distribution information and control signals from the image selection device 102 via the communication I / F unit 209.
[0034] [Optical filter drive control] Next, the drive control of the optical filter 202 of the imaging device 101 according to this embodiment will be described. In this embodiment, in the imaging device 101, the optical filter drive unit 205 controls insertion and removal of the infrared cut filter constituting the optical filter 202 according to the control by the control unit 208. In this embodiment, "insertion and removal of the infrared cut filter" corresponds to "switching of imaging modes". Specifically, the "imaging modes" include "day mode" and "night mode". The "day mode" corresponds to a case in which an infrared cut filter is inserted in the optical path reaching the imaging unit 203 from the lens unit 201 in the imaging device 101 (i.e., a case in which infrared light is blocked and light including visible light is transmitted). In addition, the "night mode" corresponds to a case in which an infrared cut filter is not inserted in the optical path reaching the imaging unit 203 from the lens unit 201 in the imaging device 101 (i.e., a case in which light including infrared light and visible light is transmitted). The optical filter drive unit 205 can switch the imaging mode by controlling the insertion and removal of the infrared cut filter.
[0035] In this embodiment, focusing on one of the multiple imaging devices constituting the image delivery system 100, switching of imaging modes in the imaging device when the brightness of the imaging environment changes instantaneously will be described with reference to Fig. 3 and Fig. 4. In this example, the imaging device will be described as imaging device 101a.
[0036] Fig. 3 shows an image capture mode switching sequence according to this embodiment. Specifically, Fig. 3 shows the image capture mode and image delivery state of the image capture device 101a when the brightness of the image capture environment changes instantaneously. In Fig. 3, the image delivery state of the image capture device 101a is shown by a solid line when an image is being delivered, and by a dotted line when an image is not being delivered. Fig. 4 shows a flowchart of the process executed by the image capture device 101a.
[0037] 3, it is assumed that the imaging environment becomes dark at the moment of timing 3a, for example, for the purpose of producing a scene. The control unit 208 of the imaging device 101a detects that there has been a change in brightness in the imaging environment (i.e., that the average brightness value of the image has become equal to or less than a threshold value or has exceeded a threshold value) based on the image acquired from the image processing unit 206. For example, the control unit 208 determines that there has been a change in brightness when the average brightness value included in the image acquired from the image processing unit 206 has become equal to or less than a threshold value or has exceeded a threshold value.
[0038] In order to capture a subject clearly without being affected by variations in light and darkness, it is preferable that the imaging device 101a quickly switches the imaging mode from day mode to night mode after timing 3a. However, if the imaging device 101a switches the imaging mode from day mode to night mode at any timing during image distribution, the image being distributed will be one in which the imaging mode is being switched. If the color or brightness of an image changes before and after the switching of the imaging mode, the appearance may change, which may cause a viewer who views the image to feel uncomfortable. Therefore, the imaging device 101a needs to avoid switching the imaging mode during image distribution.
[0039] In this embodiment, when the control unit 208 of the imaging device 101a detects a change in brightness in the imaging environment (determines that the average brightness value of the image has become equal to or less than a threshold value, or determines that it has exceeded the threshold value) (Yes in S41), it requests the distribution determination unit 207 to confirm whether or not the image captured by the imaging unit 203 is being distributed (including output for distribution). That is, the control unit 208 requests confirmation of whether the distribution information received from the image selection device 102 indicates that an image is being distributed, or indicates that an image has not yet been distributed. In response to this, the distribution determination unit 207 confirms whether the distribution information received from the image selection device 102 indicates that an image is being distributed, or indicates that an image has not yet been distributed (S42).
[0040] When the distribution information indicates that the distribution information is not yet distributed (Yes in S43), the distribution determination unit 207 notifies the control unit 208 that the distribution information indicates that the distribution information is not yet distributed. In response to this, the control unit 208 switches the imaging mode (S44). Specifically, the control unit 208 controls the optical filter driving unit 205 to perform insertion / removal control of the infrared cut filter included in the optical filter 202, and in response to this, the optical filter driving unit 205 controls insertion / removal of the infrared cut filter. On the other hand, when the distribution information does not indicate that the distribution information is not yet distributed (i.e., when the image distribution is being performed), the distribution determination unit 207 notifies the control unit 208 that the distribution information does not indicate that the distribution information is not yet distributed. In this case, the control unit 208 does not switch the imaging mode, and the process returns to S42, and the distribution determination unit 207 checks the distribution information received from the image selection device 102, for example, periodically. Here, if the control unit 208 does not receive distribution information indicating an undistributed state for a certain period of time from the distribution determination unit 207, the control unit 208 may request the image selection device 102 to switch the imaging device that distributes the image from the imaging device 101a to another imaging device.
[0041] 3 indicates the time when the image selection device 102 switches the imaging device that captures the image to be distributed from the imaging device 101a to another imaging device. In response to the switching, the control unit 208 of the imaging device 101a recognizes that the distribution information received from the image selection device 102 is in a non-distribution state (Yes in S43), and transitions the image distribution state from image distribution in progress (solid line) to image not yet distributed (dotted line). In addition, the control unit 208 switches the imaging mode (S44).
[0042] Timing 3c in FIG. 3 indicates the time when the control unit 208 of the imaging device 101a starts to transition the imaging mode from day mode to night mode, and timing 3d indicates the time when the control unit 208 of the imaging device 101a finishes transitioning the imaging mode to night mode.
[0043] In this way, even if the imaging device 101a detects a change in brightness in the imaging environment during the distribution of an image captured by the imaging unit 203, the imaging device 101a does not switch the imaging mode until it recognizes that the imaging device capturing the image to be distributed is switched to another imaging device. Then, the imaging device 101a switches the imaging mode after recognizing that the imaging device capturing the image to be distributed has been switched. This makes it possible to continue the distribution of images that do not give a sense of incongruity to the viewer.
[0044] In this embodiment, an example focusing on the imaging device 101a has been described with reference to Fig. 3 and Fig. 4, but the same description can be applied to the imaging devices 101b and 101c. Also, in this embodiment, a case where the imaging environment transitions from a bright state to a dark state has been described, but the same description can be applied to a case where the imaging environment transitions from a dark state to a bright state.
[0045] Second Embodiment In the first embodiment, attention is focused on one imaging device (imaging device 101a in the examples of Figs. 3 and 4) in the image delivery system 100. In the present embodiment, attention is focused on multiple imaging devices in the image delivery system 100. With reference to Figs. 5 and 6, switching of imaging modes in each imaging device when the brightness of the imaging environment changes instantaneously will be described. In this example, the multiple imaging devices will be described as imaging device 101a and imaging device 101b. Note that descriptions of configurations and features similar to those of the first embodiment will be omitted.
[0046] Fig. 5 shows an imaging mode switching sequence according to this embodiment. Specifically, Fig. 5 shows imaging modes and image delivery states in the imaging devices 101a and 101b when the brightness of the imaging environment changes instantaneously. In Fig. 5, the image delivery states of the imaging devices 101a and 101b are indicated by solid lines when images are being delivered and by dotted lines when images are not being delivered. Fig. 5 also shows a flowchart of the process executed by the imaging devices 101a and 101b. In the following description, the imaging devices 101a and 101b will be collectively referred to as the imaging device 101.
[0047] In Fig. 5, the imaging device 101a is distributing an image, and the imaging device 101b is not distributing an image. In this state, assume that the imaging environment becomes dark at the moment of timing 5a, for example, for the purpose of producing a scene. The control unit 208 of the imaging device 101 detects that the imaging environment has become dark based on the average value of the luminance values of the images acquired from the image processing unit 206. For example, when the average value of the luminance values of the images acquired from the image processing unit 206 becomes equal to or less than a threshold value, the control unit 208 determines that the imaging environment has become dark and that a change in brightness has occurred.
[0048] In order to capture a clear image of a subject without being affected by variations in light and darkness, it is preferable that the imaging device 101 quickly switches the imaging mode from day mode to night mode after timing 5a. However, if the imaging device 101a switches the imaging mode from day mode to night mode at any timing during image distribution, an image in the middle of switching the imaging mode will be distributed. If the color or brightness of an image changes before and after switching the imaging mode, the appearance may change, which may cause a viewer who views the image to feel uncomfortable. Therefore, the imaging device 101a needs to avoid switching the imaging mode during image distribution.
[0049] In this embodiment, when the control unit 208 of the imaging device 101 detects a change in brightness in the imaging environment (determines that the average brightness value of the image has become equal to or lower than the threshold value) (Yes in S61), it requests the distribution determination unit 207 to confirm whether or not the image captured by the imaging unit 203 is being distributed (including output for distribution). That is, it requests the distribution determination unit 207 to confirm whether the distribution information received from the image selection device 102 indicates that an image is being distributed or indicates that an image has not yet been distributed. In response to this, the distribution determination unit 207 confirms whether the distribution information received from the image selection device 102 indicates that an image is being distributed or indicates that an image has not yet been distributed (S62).
[0050] When the distribution information indicates that the distribution information is not yet distributed (Yes in S63), the distribution determination unit 207 of the imaging device 101 notifies the control unit 208 that the distribution information is not yet distributed. In response to this, the control unit 208 switches the imaging mode (S64). The process of S64 is similar to the process of S44 in FIG. 4, and therefore a description thereof will be omitted. With reference to FIG. 5, timing 5b indicates the time when the control unit 208 of the imaging device 101b, which is in the not yet distributed state, starts to shift the imaging mode from day mode to night mode. Timing 5c indicates the time when the control unit 208 of the imaging device 101b finishes shifting the imaging mode to night mode.
[0051] In this example, the imaging device 101b does not distribute the image captured by the imaging unit 203 at the timing of light-dark switching (timing 5a). Therefore, when the imaging mode switching is set in the imaging device 101b according to the brightness of the imaging environment, it is preferable to immediately control the insertion / removal of the infrared cut filter and transition to the night mode. That is, it is preferable that the time from timing 5a to timing 5b (the time from S61 to S64 in FIG. 6) is as short as possible. For example, when the imaging device 101 is notified of a time-series control pattern (i.e., a light-dark fluctuation pattern) for one or more lighting devices by the lighting control device, the control unit 208 of the imaging device 101b can quickly perform detection in S61 according to the control pattern. This can shorten the time between timings 5a and 5b in FIG. 5.
[0052] When the control unit 208 of the imaging device 101 has completed the transition of the imaging mode to the night mode, it notifies the image selection device 102 that the imaging mode switching has ended (S65). That is, referring to Fig. 5, the control unit 208 of the imaging device 101b promptly notifies the image selection device 102 that the imaging mode switching has ended after timing 5c. Note that, if timing 5c can be predicted, the control unit 208 of the imaging device 101b may notify the image selection device 102 of information on the predicted end time of the imaging mode switching.
[0053] In FIG. 6, when the distribution information does not indicate that distribution is not in progress (i.e., when image distribution is in progress) (No in S63), the distribution determination unit 207 of the imaging device 101 notifies the control unit 208 that the distribution information is not in a state of not yet being distributed. Next, the control unit 208 requests the image selection device 102 to switch the imaging device that distributes the image after receiving the notification in S66 (S65). Here, when the control unit 208 recognizes that the image selection device 102 may receive the predicted end time, it may request that the imaging device that distributes the image be switched after the predicted end time. Next, the control unit 208 performs the processes of S67 to S69. The processes of S67 to S69 are similar to those of S62 to S64, and therefore will not be described.
[0054] 5, the control unit 208 of the imaging device 101a that is distributing an image requests the image selection device 102 to switch the imaging device that distributes the image from the imaging device 101a to the imaging device 101b after receiving the notification of S66 from the imaging device 101b. The timing of this request may be immediately after timing 5a.
[0055] Timing 5d in Fig. 5 indicates the timing at which the image selection device 102 switches the imaging device that delivers images from the imaging device 101a to the imaging device 101b. In order to prevent the imaging mode of the imaging device 101b from being switched during image delivery from the imaging device 101b, timing 5d needs to come after timing 5c on the time axis. In S65 in Fig. 6, the delivery determination unit 207 of the imaging device 101a requests the image selection device 102 to switch from the imaging device 101a to the imaging device 101b after the notification in S66 from the imaging device 101b, so that timing 5d is guaranteed to come after timing 5c on the time axis.
[0056] 5 indicates the timing when the control unit 208 of the imaging device 101a starts to transition the imaging mode from day mode to night mode. Also, timing 5f indicates the timing when the control unit 208 of the imaging device 101a finishes transitioning the imaging mode to night mode. When the distribution information received from the image selection device 102 indicates an undistributed state (Yes in S68), the control unit 208 of the imaging device 101a switches the imaging mode (S69). Therefore, timing 5e is later on the time axis than timing 5d.
[0057] In this way, even if the imaging device 101a detects a change in brightness in the imaging environment during the distribution of an image captured by the imaging unit 203, the imaging device 101a does not switch the imaging mode until the imaging device distributing the image is switched to the imaging device 101b. Also, when the imaging device 101b detects a change in brightness in the imaging environment while not distributing an image captured by the imaging unit 203, the imaging device 101b promptly switches the imaging mode. Then, when the imaging device 101b has finished the switching, it notifies the image selection device 102 of the completion. In response to receiving the notification, the image selection device 102 switches the imaging device distributing the image to the imaging device 101b. Then, the imaging device 101a switches the imaging mode after recognizing that the imaging device distributing the image has been switched to the imaging device 101b. This makes it possible to continue distributing images that do not give a sense of incongruity to the viewer.
[0058] In this embodiment, an example focusing on the imaging device 101a and the imaging device 101b has been described with reference to Fig. 5 and Fig. 6, but the same description can be applied to any two or more imaging devices, namely, the imaging devices 101a, 101b, and 101c. In addition, in this embodiment, a case where the imaging environment transitions from a bright state to a dark state has been described, but the same description can be applied to a case where the imaging environment transitions from a dark state to a bright state. In addition, the processing of this embodiment can be applied even when there are three or more imaging devices.
[0059] <Modification> In the above embodiment, the image selection device 102 selected one or more images captured by the imaging devices 101a, 101b, and 101c. In addition to this, or instead of this, the image selection device 102 may generate a new image (for example, a virtual viewpoint image) from the multiple images captured by the imaging devices 101a, 101b, and 101c, and distribute or output the new image. In this case, too, the imaging device that captures the image for generating the new image may switch the imaging mode in response to a change in brightness in the imaging environment of the imaging device.
[0060] <Other embodiments> The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.
[0061] The disclosure of this embodiment includes the following configurations and methods. (Configuration 1) an imaging section including an imaging optical system; an optical filter that is inserted into and removed from an optical path of the imaging optical system; a first determination means for determining whether a luminance value of an image captured by the imaging unit is equal to or less than a threshold value or exceeds the threshold value; a transmitting means for transmitting the image captured by the imaging unit to an external device; a second determination means for determining whether or not the image is being distributed when the first determination means determines that the luminance value of the image is equal to or less than the threshold value or exceeds the threshold value; a control means for controlling insertion / removal of the optical filter into / from an optical path when the second determination means determines that the image has not been distributed; An imaging device comprising:
[0062] (Configuration 2) The imaging device according to configuration 1, wherein the control means switches between imaging modes including a first mode that blocks infrared light and transmits light including visible light, and a second mode that transmits light including infrared light and visible light.
[0063] (Configuration 3) the optical filter is an infrared cut filter that blocks infrared light and transmits visible light, 3. The imaging device according to claim 2, wherein the control means switches the imaging mode by controlling insertion and removal of the infrared cut filter.
[0064] (Configuration 4) 3. The imaging device according to configuration 2, wherein the control means, after switching the imaging mode, notifies the external device that the switching of the imaging mode has been completed.
[0065] (Configuration 5) When the second determination means determines that the image is being distributed, the control means requests the external device not to distribute the image after receiving a notification that the switching of the imaging mode has been completed; When the determination means determines that the image has not been delivered after the request, the control means switches the imaging mode. 5. The imaging device according to configuration 4.
[0066] (Configuration 6) The imaging device according to any one of configurations 1 to 5, wherein the second determination means determines that the image is not being distributed when information indicating that the image is not being distributed via the external device is notified from the external device.
[0067] (Configuration 7) The imaging device according to any one of configurations 1 to 5, wherein the second determination means determines that the image is not being distributed when information indicating that the image is not being output from the external device is notified from the external device.
[0068] (Configuration 8) Further comprising a lighting unit, 2. The imaging device according to configuration 1, wherein the control means controls lighting of the illumination unit depending on whether the second determination means determines whether the image is being distributed.
[0069] (Method 1) an imaging section including an imaging optical system; an optical filter that is inserted into or removed from an optical path of the imaging optical system, a first determination step of determining whether a luminance value of an image captured by the imaging unit is equal to or less than a threshold value or exceeds the threshold value; a transmitting step of transmitting the image captured by the imaging unit to an external device; a second determination step of determining whether or not the image is being distributed when the first determination means determines that the luminance value of the image is equal to or less than the threshold value or exceeds the threshold value; a control step of controlling insertion / removal of the optical filter onto / from an optical path when it is determined that the image is not being distributed; A control method comprising: [Explanation of symbols]
[0070] 100: image video distribution system, 101a; 101b; 101c: imaging device, 102: image selection device, 103a; 103b: network, 104: server device, 105: operator, 106: display device, 201: lens unit, 202: optical filter, 203: imaging section, 204: lens unit drive section, 205: optical filter drive section, 206: image processing section, 207: distribution determination section, 208: control section, 209: communication I / F section
Claims
1. an imaging section including an imaging optical system; an optical filter that is inserted into and removed from an optical path of the imaging optical system; a first determination means for determining whether a luminance value of an image captured by the imaging unit is equal to or less than a threshold value or exceeds the threshold value; a transmitting means for transmitting the image captured by the imaging unit to an external device; a second determination means for determining whether or not the image is being distributed when the first determination means determines that the luminance value of the image is equal to or less than the threshold value or exceeds the threshold value; a control means for controlling insertion / removal of the optical filter onto / from an optical path when the second determination means determines that the image has not been distributed; An imaging device comprising:
2. 2. The imaging device according to claim 1, wherein the control means switches between imaging modes including a first mode in which infrared light is blocked and light including visible light is transmitted, and a second mode in which light including infrared light and visible light is transmitted.
3. the optical filter is an infrared cut filter that blocks infrared light and transmits visible light, 3. The imaging apparatus according to claim 2, wherein the control means switches the imaging mode by controlling insertion and removal of the infrared cut filter.
4. 3. The imaging device according to claim 2, wherein the control means, after switching the imaging mode, notifies the external device that the switching of the imaging mode has been completed.
5. When the second determination means determines that the image is being distributed, the control means requests the external device not to distribute the image after receiving a notification that the switching of the imaging mode has been completed; When the determination means determines that the image has not been delivered after the request, the control means switches the imaging mode.
5. The imaging device according to claim 4.
6. The imaging device according to any one of claims 1 to 5, characterized in that the second determination means determines that the image is not being distributed when information indicating that the image is not being distributed via the external device is notified from the external device.
7. The imaging device according to any one of claims 1 to 5, characterized in that the second determination means determines that the image is not being distributed when information indicating that the image is not being output from the external device is notified from the external device.
8. Further comprising a lighting unit, 2. The imaging device according to claim 1, wherein the control means controls lighting of the illumination unit depending on whether the second determination means determines whether the image is being distributed.
9. an imaging section including an imaging optical system; an optical filter that is inserted into or removed from an optical path of the imaging optical system, a first determination step of determining whether a luminance value of an image captured by the imaging unit is equal to or less than a threshold value or exceeds the threshold value; a transmitting step of transmitting the image captured by the imaging unit to an external device; a second determination step of determining whether or not the image is being distributed when the first determination means determines that the luminance value of the image is equal to or less than the threshold value or exceeds the threshold value; a control step of controlling insertion / removal of the optical filter onto / from an optical path when it is determined that the image is not being distributed; A control method comprising:
10. Computer An imaging device including an imaging section including an imaging optical system and an optical filter inserted into or removed from an optical path of the imaging optical system, a first determination means for determining whether a luminance value of an image captured by the imaging unit is equal to or less than a threshold value or exceeds the threshold value; a transmitting means for transmitting the image captured by the imaging unit to an external device; a second determination means for determining whether or not the image is being distributed when the first determination means determines that the luminance value of the image is equal to or less than the threshold value or exceeds the threshold value; A program characterized by functioning as a control means for controlling insertion and removal of the optical filter into and from the optical path when the second determination means determines that the image is not being distributed.
Citation Information
Patent Citations
Imaging apparatus, and imaging mode switching method
JP2004120202A
