Camera control device, camera device, and control method of camera control device

Abstract

The present application discloses a camera control device, a camera device, and a control method for the camera control device. In order to reduce the power consumption of a camera device that captures multiple image data. A camera device includes a signal processing unit and a control unit. The signal processing unit performs compound eye processing for synthesizing multiple image data by performing signal processing on each of the multiple image data according to a predetermined control signal, or performs monocular processing for performing signal processing on any one of the multiple image data. The control unit provides a predetermined control signal to the signal processing unit, and switches one of the compound eye processing and the monocular processing to the other of the compound eye processing and the monocular processing based on a comparison result between a measured predetermined physical quantity and a predetermined threshold.

Description

[0001] This application is a divisional application of the PCT international application with the application date of August 7, 2018 and the PCT international application number of PCT / JP2018 / 029570, which entered the Chinese national phase on March 18, 2020 with the application number 201880060598.X and the invention name of "Camera control device, camera device, control method of camera control device and non-temporary computer-readable medium". Technical Field

[0002] The present technology relates to a camera control device, a camera device, a control method for a camera control device, and a non-transitory computer-readable medium; more specifically, it relates to a camera control device including multiple solid-state camera devices, a camera device, a control method for a camera control device, and a non-transitory computer-readable medium. Background Art

[0003] In the past, in order to capture multiple pieces of image data at the same time, a compound eye imaging device including multiple solid-state imaging devices was used. For example, an imaging device is proposed that uses a solid-state imaging device including a color filter to capture color image data, and uses a solid-state imaging device not including a color filter to capture monochrome image data (for example, see Patent Document 1). Here, because the incident light is not attenuated by the color filter, the monochrome image data becomes much brighter than the color image data. The imaging device captures and synthesizes those color image data and monochrome imaging data at the same time.

[0004] Citation List

[0005] Patent Literature

[0006] PTL 1: JP2011-239260 Summary of the invention

[0007] Technical issues

[0008] According to the above-mentioned past technology, by synthesizing relatively bright monochrome image data with color image data, it is possible to generate color image data that is brighter than when the image data is not synthesized. However, since the processing amount of the synthesis process of synthesizing a plurality of image data is greater than the processing amount in the case of processing one image data, there is a problem that the power consumption of the camera device increases compared to the case where the image data is not synthesized. As the power consumption increases, there is a concern that the speed at which the remaining power of the camera device decreases and the heat generation of the camera device will accelerate, so it is desired to reduce the power consumption.

[0009] The present technology has been made in consideration of the circumstances as described above, and is intended to reduce power consumption of an image pickup apparatus that captures a plurality of pieces of image data.

[0010] Problem Solution

[0011] The present technology has been made to solve the above-mentioned problems, and according to a first aspect of the present technology, there is provided an imaging control device, including: a signal processing unit, which performs compound eye processing for synthesizing the plurality of image data by performing signal processing on each of a plurality of image data or performs monocular processing for performing signal processing on any one of the plurality of image data according to a predetermined control signal; and a control unit, which provides a predetermined control signal to the signal processing unit based on a comparison result between a measured predetermined physical quantity and a predetermined threshold value and a control method thereof, and switches one of the compound eye processing and the monocular processing to the other of the compound eye processing and the monocular processing. Therefore, an operation of switching the processing of the signal processing unit from one of the compound eye processing and the monocular processing to the other of the compound eye processing and the monocular processing according to the physical quantity can be obtained.

[0012] In addition, in the first aspect, the predetermined physical quantity may be a physical quantity that fluctuates according to the imaging environment. Therefore, an operation of switching the processing of the signal processing unit from one of the compound eye processing and the monocular processing to the other of the compound eye processing and the monocular processing according to the imaging environment can be obtained.

[0013] In addition, in the first aspect, the camera control device may further include a temperature sensor that measures temperature as a predetermined physical quantity, and when the temperature is higher than a predetermined threshold, the control unit may switch the compound eye processing to the monocular processing. Therefore, an operation of switching the processing of the signal processing unit from the compound eye processing to the monocular processing when the temperature is higher than the threshold can be obtained.

[0014] In addition, in the first aspect, the signal processing unit may include a process of measuring the amount of incident light as a predetermined physical quantity and acquiring the amount of light as a photometric quantity, and the control unit may switch the compound eye process to the monocular process when the photometric quantity is greater than a predetermined threshold value. Therefore, an operation of switching the process of the signal processing unit from the compound eye process to the monocular process when the photometric quantity is greater than the threshold value can be obtained.

[0015] In addition, in the first aspect, the signal processing unit may include a process of measuring the amount of movement of the object as a predetermined physical quantity, and the control unit may switch the monocular process to the compound eye process when the amount of movement is greater than a predetermined threshold. Therefore, an operation of switching the signal processing unit from the monocular process to the compound eye process when the amount of movement of the object is greater than the threshold can be obtained.

[0016] Furthermore, in this first aspect, the plurality of pieces of image data may be images captured in synchronization with a predetermined synchronization signal, and after performing a transition process for performing synthesis while changing the synthesis ratio every time a predetermined period elapses, the signal processing unit may switch one of the compound eye process and the monocular process to the other of the compound eye process and the monocular process. Thus, an operation of switching the process after gradually changing the synthesis ratio can be obtained.

[0017] In addition, according to a second aspect of the present technology, there is provided an image capture control device, including: a synthesis processing unit that synthesizes a plurality of image data and outputs the data as synthesized image data; a transmission unit that performs a single-channel transmission process of transmitting the synthesized image data together with a single image data of any one of the plurality of image data via a single communication channel, or performs a multi-channel transmission process of transmitting the single image data and the synthesized image data respectively via mutually different communication channels, according to a predetermined control signal; and a control unit that provides a predetermined control signal to the signal processing unit based on a comparison result between a measured predetermined physical quantity and a predetermined threshold value, and switches one of the single-channel transmission process and the multi-channel transmission process to the other of the single-channel transmission process and the multi-channel transmission process. Therefore, an operation of switching one of the single-channel transmission process and the multi-channel transmission process to the other of the single-channel transmission process and the multi-channel transmission process according to the physical quantity can be obtained.

[0018] In addition, in the second aspect, the camera apparatus may further include a scaling processing unit that reduces any one of the plurality of image data, and the control unit may control the scaling processing unit to change the reduction rate based on a comparison result between a predetermined physical quantity and a predetermined threshold value. Thus, an operation of changing the reduction rate according to the physical quantity can be obtained.

[0019] In addition, in the second aspect, the camera apparatus may further include a synchronization control unit that provides a predetermined synchronization signal, each of the plurality of image data may be image data captured in synchronization with the predetermined synchronization signal, and the control unit may control the synchronization control unit to change the frequency of the predetermined synchronization signal based on a comparison result between a predetermined physical quantity and a predetermined threshold value. Thus, an operation of changing the frequency of the synchronization signal according to the physical quantity can be obtained.

[0020] In addition, according to a third aspect of the present technology, there is provided an imaging device, including: a camera module that captures a plurality of image data; a signal processing unit that performs compound eye processing for synthesizing a plurality of image data by performing signal processing on each of the plurality of image data, or performs monocular processing for performing signal processing on any one of the plurality of image data, according to a predetermined control signal; and a control unit that provides a predetermined control signal to the signal processing unit based on a comparison result between a predetermined physical quantity measured and a predetermined threshold value, and switches one of the compound eye processing and the monocular processing to the other of the compound eye processing and the monocular processing. Therefore, an operation of switching the processing of the image data from the camera module from one of the compound eye processing and the monocular processing to the other of the compound eye processing and the monocular processing according to the physical quantity can be obtained.

[0021] Advantageous Effects of the Invention

[0022] According to the present technology, it is possible to achieve an excellent effect of reducing power consumption in an image pickup device that captures a plurality of pieces of image data. It should be noted that the effects described herein are not necessarily restrictive, and any effect described in the present disclosure can be obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A block diagram showing a configuration example of an image pickup apparatus according to a first embodiment of the present technology;

[0024] Figure 2 A block diagram showing a configuration example of a camera module according to a first embodiment of the present technology;

[0025] Figure 3 An example of a cross-sectional view of a solid-state imaging device according to a first embodiment of the present technology is shown;

[0026] Figure 4 A block diagram showing a configuration example of a signal processing unit according to a first embodiment of the present technology;

[0027] Figure 5 A block diagram showing a configuration example of a color image signal processing unit according to a first embodiment of the present technology;

[0028] Figure 6 A block diagram showing a configuration example of a synthesis processing unit according to a first embodiment of the present technology;

[0029] Figure 7 A timing chart showing an operation example of the image pickup apparatus according to the first embodiment of the present technology;

[0030] Figure 8 A block diagram showing an example of a state of an image pickup apparatus in a case where a synthesis process is not performed in the first embodiment of the present technology;

[0031] Fig. 9 A flowchart showing an operation example of the image pickup apparatus according to the first embodiment of the present technology;

[0032] Fig.10 A block diagram showing a configuration example of an image pickup apparatus according to a first modified example of the first embodiment of the present technology;

[0033] Fig.11 A block diagram showing a configuration example of a signal processing unit according to a first modified example of the first embodiment of the present technology;

[0034] Fig.12 A block diagram showing a configuration example of a color image signal processing unit according to a second modified example of the first embodiment of the present technology;

[0035] Fig.13 A block diagram showing a configuration example of a signal processing unit according to a second embodiment of the present technology;

[0036] Fig.14 A block diagram showing a configuration example of an application processor according to a second embodiment of the present technology;

[0037] Fig.15 A block diagram showing a configuration example of an image pickup apparatus according to a modified example of the second embodiment of the present technology;

[0038] Fig.16 A block diagram showing a schematic configuration example of a vehicle control system;

[0039] Fig.17 An explanatory diagram showing an example of the setting position of the imaging unit. DETAILED DESCRIPTION

[0040] Hereinafter, a mode for embodying the present technology (hereinafter, referred to as an embodiment) will be described. The description will be made in the following order.

[0041] 1. First embodiment (example of switching from compound eye processing to single eye processing when temperature is higher than a threshold)

[0042] 2. Second Implementation (Example of Switching from 2 Channels to 1 Channel When Temperature is Higher than a Threshold)

[0043] 3. Application examples of moving objects

[0044] <1. First Embodiment>

[0045] "Configuration example of camera device"

[0046] Figure 11 is a block diagram showing a configuration example of an image pickup apparatus 100 according to a first embodiment of the present technology. The image pickup apparatus 100 captures image data and includes camera modules 110 and 130, an image pickup control apparatus 200, and an application processor 150. As the image pickup apparatus 100, a smartphone, a sports camera, an IoT (Internet of Things) camera, a car camera, etc. are assumed.

[0047] The camera modules 110 and 130 capture image data in synchronization with a vertical synchronization signal VSYNC having a predetermined frequency (30 Hz, etc.). These camera modules 110 and 130 simultaneously capture image data in synchronization with the same vertical synchronization signal VSYNC. Then, the camera module 110 provides the image data to the camera control device 200 via the signal line 119, and the camera module 130 provides the image data to the camera control device 200 via the signal line 139.

[0048] Furthermore, the camera module 110 operates or stops according to an enable signal ENc1 from the imaging control device 200, and the camera module 130 operates or stops according to an enable signal ENc2 from the imaging control device 200. The enable signal ENc1 is transmitted via a signal line 217, and the enable signal ENc2 is transmitted via a signal line 218.

[0049] The imaging control apparatus 200 controls an imaging operation of the imaging apparatus 100 , and includes a control unit 210 , a temperature sensor 220 , and a signal processing unit 240 .

[0050] The temperature sensor 220 measures the temperature of the imaging apparatus 100. The temperature sensor 220 supplies temperature data indicating a measured value to the control unit 210 via a signal line 229.

[0051] The signal processing unit 240 performs predetermined signal processing on the two pieces of image data from the camera modules 110 and 130. Under the control of the control unit 210, the signal processing unit 240 performs compound-eye processing or monocular processing. Here, compound-eye processing is a process of performing signal processing on each of the two pieces of image data and synthesizing them. In addition, monocular processing is a process of performing signal processing on only one of the two pieces of image data. The signal processing unit 240 provides the processed image data to the application processor 150 via the signal line 209.

[0052] In addition, the signal processing unit 240 controls the exposure amount of the camera modules 110 and 130 and the position of the focus lens.

[0053] The application processor 150 performs various types of image processing, such as object recognition processing and filtering processing, on the image data from the signal processing unit 240. For example, the application processor 150 records the processed image data in a nonvolatile memory (not shown) or the like. In addition, for example, the application processor 150 provides the processed image data to a display device (not shown) for display.

[0054] The control unit 210 controls the camera modules 110 and 130 and the signal processing unit 240 based on the comparison result between the measured predetermined physical quantity and the predetermined threshold value. It is expected that the physical quantity to be measured fluctuates according to the imaging environment. For example, temperature is used as such a physical quantity. When a predetermined application for performing imaging is executed, the control unit 210 provides enable signals ENc1 and ENc2 to the camera modules 110 and 130 to operate the camera modules 110 and 130. In addition, at this time, the control unit 210 provides enable signals ENs1, ENs2 and ENs3 to the signal processing unit 240 so that the signal processing unit 240 performs compound eye processing. The details of the enable signals ENs1 to ENs3 will be described later. These enable signals ENs1 to ENs3 are transmitted via the signal line 219. It should be noted that the enable signals ENs1, ENs2 and ENs3 are examples of control signals described in the scope of the claims.

[0055] Then, the control unit 210 monitors the temperature and determines whether the temperature is higher than a predetermined upper threshold. When the temperature becomes higher than the upper threshold, the control unit 210 provides the enable signals ENs1, ENs2, and ENs3 to the signal processing unit 240 to control the signal processing unit 240 to start switching from compound eye processing to single eye processing. Then, when the switching is completed, the control unit 210 provides the enable signal ENc1 or ENc2 to the camera modules 110 and 130 to stop one of the camera modules 110 and 130.

[0056] In this way, when the temperature becomes higher than the upper threshold, one of the camera modules 110 and 130 is stopped and switched to the single-eye process, resulting in reduced power consumption and suppressed temperature rise.

[0057] In addition, after switching to the monocular processing, the control unit 210 determines whether the temperature is lower than a predetermined lower threshold. When the temperature becomes lower than the lower threshold, the control unit 210 controls the signal processing unit 240 through the enable signals ENs1, ENs2, and ENs3 to start switching from the monocular processing to the compound eye processing. Then, when the switching is completed, the control unit 210 enables both the camera modules 110 and 130 to operate through the enable signals ENc1 and ENc2.

[0058] "Configuration example of camera module"

[0059] Figure 2 1 is a block diagram showing a configuration example of camera modules 110 and 130 according to a first embodiment of the present technology. 'a' in the figure is a block diagram showing a configuration example of the camera module 110, and 'b' in the figure is a block diagram showing a configuration example of the camera module 130.

[0060] The camera module 110 includes an optical system 111 and a solid-state imaging device 120. The optical system 111 is composed of a plurality of optical components including a diaphragm and a focus lens, and collects incident light and guides the incident light to the solid-state imaging device 120. The solid-state imaging device 120 captures color image data including color information and brightness information in synchronization with a vertical synchronization signal VSYNC.

[0061] The signal processing unit 240 controls the aperture value of the optical system 111 and the exposure time of the solid-state imaging device 120 to set the exposure amount to an appropriate value. In addition, the signal processing unit 240 controls the position of the focus lens of the optical system 111 for focusing.

[0062] In addition, the camera module 130 includes an optical system 131 and a solid-state imaging device 140. The configuration of the optical system 131 is similar to that of the optical system 111. The solid-state imaging device 140 captures monochrome image data including only brightness information in synchronization with the vertical synchronization signal VSYNC. The signal processing unit 240 also controls the exposure amount and focus of the camera module 130.

[0063] It should be noted that although the optical system 111 and the solid-state imaging device 120 and the optical system 131 and the solid-state imaging device 140 are provided in separate camera modules, they may all be provided in a single camera module.

[0064] Figure 3 ”a” in the figure is an example of a cross-sectional view of the solid-state imaging device 120 , and ”b” in the figure is an example of a cross-sectional view of the solid-state imaging device 140 .

[0065] On the light receiving surface of the solid-state image pickup device 120, a microlens 121 is provided for each pixel. A color filter 122 is provided below each microlens 121, and the direction from the microlens 121 to the focus is the downward direction. A light receiving device 123 is provided below each color filter 122. The color filter 122 transmits light having any one of a red wavelength, a green wavelength, and a blue wavelength. By performing photoelectric conversion on these transmitted lights, color image data is generated, wherein the individual pixel signals of R (red), G (green), and B (blue) are arranged in a two-dimensional grid. A wiring layer 124 is provided below the light receiving device 123. Since the wiring layer 124 is the front side, the solid-state image pickup device 120 in which light is irradiated to the back side relative to the front side as described above is referred to as a back-illuminated solid-state image pickup device.

[0066] On the light receiving surface of the solid-state image pickup device 140, a microlens 141 is provided for each pixel. A light receiving device 143 is provided below each microlens 141. A wiring layer 144 is provided below the light receiving device 143. Since the solid-state image pickup device 140 does not include a color filter, the solid-state image pickup device 140 captures monochrome image data. In addition, since the incident light is not attenuated by the color filter in the solid-state image pickup device 140, monochrome image data brighter than color image data can be obtained under a light source of the same light amount.

[0067] It should be noted that although a back-illuminated solid-state imaging device is used as the solid-state imaging devices 120 and 140 , a front-illuminated solid-state imaging device may also be used.

[0068] "Configuration example of signal processing unit"

[0069] Figure 4 2 is a block diagram showing a configuration example of the signal processing unit 240 according to the first embodiment of the present technology. The signal processing unit 240 includes a color image signal processing unit 241 , a monochrome image signal processing unit 247 , a synthesis processing unit 250 , and a transmission unit 248 .

[0070] The color image signal processing unit 241 performs predetermined signal processing on the color image data from the camera module 110. The color image signal processing unit 241 supplies the processed color image data to the synthesis processing unit 250. In addition, the color image signal processing unit 241 operates or stops according to the enable signal ENs1 from the control unit 210.

[0071] The monochrome image signal processing unit 247 performs predetermined signal processing on the monochrome image data from the camera module 130. The monochrome image signal processing unit 247 provides the processed monochrome image data to the synthesis processing unit 250. In addition, the monochrome image signal processing unit 247 operates or stops according to the enable signal ENs2 from the control unit 210.

[0072] The synthesis processing unit 250 synthesizes the color image data and the monochrome image data that have been signal processed. The synthesis processing unit 250 synthesizes the color image data and the monochrome image data in synchronization with the vertical synchronization signal VSYNC to generate synthesized image data. Then, the synthesis processing unit 250 changes the synthesis ratio according to the enable signal ENs3 from the control unit 210. Here, the enable signal ENs3 is a signal for indicating the start or end of synthesis. The method of changing the synthesis ratio will be described later. The synthesis processing unit 250 provides the processed image data as output image data to the transmission unit 248.

[0073] The transmission unit 248 transmits the output image data. The transmission unit 248 transmits the output image data to the application processor 150 according to the MIPI (Mobile Industry Processor Interface) standard, for example.

[0074] Figure 5 241 is a block diagram showing a configuration example of a color image signal processing unit 241 according to the first embodiment of the present technology. The color image signal processing unit 241 includes a demosaic processing unit 242, a detection unit 243, an exposure control unit 244, and a focus control unit 245.

[0075] The demosaic processing unit 242 performs a demosaic processing on the color image data, in which insufficient color information is interpolated for each pixel based on surrounding pixels. Through the demosaic processing, color image data including R, G, and B signals of each pixel is generated. The demosaic processing unit 242 provides the demosaiced color image data to the synthesis processing unit 250. It should be noted that in addition to the demosaic processing, the color image signal processing unit 241 can also perform other processing, such as filtering processing and defective pixel interpolation processing.

[0076] The detection unit 243 measures the amount of incident light from the color image data and obtains the amount of incident light as a photometric amount. For example, the photometric amount can be obtained as a detection value by center-weighted photometry or spot photometry. The detection unit 243 provides the detection value (photometric amount) to the exposure control unit 244.

[0077] The exposure control unit 244 controls the exposure amount to an appropriate value based on the detection value. The exposure control unit 244 controls the aperture value and exposure time of the camera module 110 to set the exposure amount to an appropriate value.

[0078] The focus control unit 245 focuses the camera module 110. The focus control unit 245 detects a focus position using, for example, an image plane phase difference detection method or a contrast detection method, and moves the focus lens of the camera module 110 to the position.

[0079] The configuration of the monochrome image signal processing unit 247 is similar to that of the color image signal processing unit 241 except that the demosaicing process is not performed.

[0080] "Configuration example of the synthesis processing unit"

[0081] Figure 6 2 is a block diagram showing a configuration example of the synthesis processing unit 250 according to the first embodiment of the present technology. The synthesis processing unit 250 includes a positioning processing unit 251, a YC conversion unit 252, a luminance signal synthesis unit 253, an RGB conversion unit 254, and a switch 255.

[0082] The positioning processing unit 251 positions the color image data and the monochrome image data. The positioning processing unit 251 corrects the positional deviation between the two image data due to the attachment error or difference of the camera modules 110 and 130 by parallel movement, rotation, etc. The positioning processing unit 251 provides the positioned color image data to the YC conversion unit 252, and provides the monochrome image data to the brightness signal synthesis unit 253.

[0083] For each pixel in the color image data, the YC conversion unit 252 converts the R, G, and B signals into a luminance signal Y1 and color difference signals Cb and Cr by the following expressions.

[0084] Y1=0.257R+0.504G+0.098B+16

[0085] Cb=-0.148R-0.291G+0.439B+128

[0086] Cr=0.439R-0.368G-0.071B+128

[0087] The YC conversion unit 252 supplies the luminance signal Y1 to the luminance signal synthesis unit 253 , and supplies the color difference signals Cb and Cr to the RGB conversion unit 254 .

[0088] The luminance signal synthesis unit 253 synthesizes the luminance signal Y1 of the color image data and the luminance signal Y2 of the monochrome image data for each pixel according to the following expression.

[0089] Y3=Y1*(A / 100)+Y2*(1-A) / 100

[0090] In the above expression, A represents the synthesis ratio expressed as a percentage, and Y3 represents the brightness signal obtained after synthesis.

[0091] The luminance signal synthesis unit 253 supplies the luminance signal Y3 obtained after synthesis to the RGB conversion unit 254 .

[0092] The RGB conversion unit 254 converts the luminance signal Y3 and the color difference signals Cr and Cb of each pixel into R, G, and B signals by the following expression: The RGB conversion unit 254 supplies the converted image data to the switch 255 .

[0093] R=1.164(Y3-16)-1.596(Cr-128)

[0094] G=1.164(Y3-16)-0.391(Cb-128)-0.813(Cr-128)

[0095] B=1.164(Y3-16)-2.018(Cb-128)

[0096] The switch 255 selects the image data from the RGB conversion unit 254 or the color image data from the color image signal processing unit 241 under the control of the brightness signal synthesis unit 253. In the initial state, the image data from the RGB conversion unit 254 is selected. The switch 255 supplies the selected image data to the transmission unit 248 as output image data.

[0097] When the enable signal ENs3 indicates the start of synthesis, the brightness signal synthesis unit 253 sets the initial value (for example, "0") of the synthesis ratio A and starts synthesis. Then, when the end of synthesis is indicated by the enable signal ENs3 after the start of synthesis, the brightness signal synthesis unit 253 performs a transition process for increasing the synthesis ratio A each time a frame period, which is a period of the vertical synchronization signal VSYNC, passes. For example, in the case where the synthesis ratio A increases by "10" percentage points (%) in each frame period, the synthesis ratio A becomes "100" percentage points (%) in 10 frame periods, and the transition process ends. At the end of the transition process, the brightness signal synthesis unit 253 controls the switch 255 to select the color image data from the color image signal processing unit 241. By performing this transition process, a sudden change in brightness in the output image data can be suppressed.

[0098] Then, when synthesis start is instructed by the enable signal ENs3 after synthesis ends, the luminance signal synthesis unit 253 performs transition processing for reducing the synthesis ratio A every time a frame period elapses. Furthermore, the luminance signal synthesis unit 253 controls the switch 255 to select image data from the RGB conversion unit 254 .

[0099] It should be noted that although the synthesis processing unit 250 synthesizes the monochrome image data and the color image data, the combination of the image data to be synthesized is not limited to the monochrome image data and the color image data. For example, the synthesis processing unit 250 may synthesize two pieces of color image data. In addition, although the synthesis processing unit 250 performs synthesis while separating the brightness signal and the color difference signal, it is also possible to perform synthesis without separation.

[0100] Figure 7 1 is a timing chart showing an operation example of the image pickup apparatus 100 according to the first embodiment of the present technology. Before image pickup starts, all of the enable signals ENc1 , ENc2 , ENs1 , ENs2 , and ENs3 are set to, for example, disabled.

[0101] When an application program or the like for performing imaging is executed at time T0, the control unit 210 sets all enable signals ENc1, ENc2, ENs1, ENs2, and ENs3 to enable. As a result, the camera modules 110 and 130 start operating. In addition, the signal processing unit 240 performs compound eye processing and outputs composite image data.

[0102] Due to the increase in the operation of the camera modules 110 and 130 and the processing amount of the signal processing unit 240, the power consumption and heat generation of the imaging device 100 increase, and the temperature of the imaging device 100 gradually increases. Then, at time T1, the temperature becomes higher than the upper limit threshold Th1. Here, in the electronic device, in order to prevent the circuit from being disconnected due to heat, in many cases, a safety circuit for stopping the circuit operation when the temperature of the device becomes higher than a certain stop threshold is incorporated. As the upper limit threshold Th1, a value lower than such a stop threshold is set.

[0103] At time T1 , the control unit 210 sets the enable signal ENs3 to be disabled based on the comparison result between the temperature and the upper limit threshold Th1 , thereby starting the switching from the compound eye process to the single eye process.

[0104] According to the enable signal ENs3, the synthesis processing unit 250 performs a transition process for gradually changing the synthesis ratio. After a period of time, the transition process ends. For example, in the case where the synthesis ratio A increases by "10" percentage points (%) in each frame period, the synthesis ratio A becomes "100" percentage points (%) at 10 frame periods, and then the transition process ends.

[0105] At time T2 when the transition process ends, the control unit 210 stops the camera module 130 and the monochrome image signal processing unit 247 according to the enable signal ENc2 and the enable signal ENs2. By stopping the monochrome image signal processing unit 247, the signal processing unit 240 starts monocular processing and outputs color image data.

[0106] By stopping the camera module 130 and switching from the compound-eye process to the single-eye process, the power consumption and heat generation of the image pickup apparatus 100 are reduced, and the temperature of the image pickup apparatus 100 is gradually reduced.

[0107] Then, when the temperature becomes lower than the lower threshold Th2 at time T3 , the control unit 210 sets the enable signals ENs2 and ENs3 to enable to start switching from the single-eye process to the compound-eye process.

[0108] The monochrome image signal processing unit 247 starts operating according to the enable signal ENs2 , and the synthesis processing unit 250 performs a transition process for gradually changing the synthesis ratio according to the enable signal ENs3 .

[0109] It should be noted that although the signal processing unit 240 first performs transition processing and then performs the switched processing (single-eye processing or compound-eye processing) when switching is instructed by the enable signal, those processes may be performed without performing transition processing.

[0110] Figure 8 1 is a block diagram showing an example of the state of the image pickup apparatus when the synthesis processing according to the first embodiment of the present technology is not performed. When the temperature becomes higher than the upper limit threshold value Th1, the control unit 210 disables the enable signals ENc2, ENs2, and ENs3. As a result, the camera module 130 and the monochrome image signal processing unit 247 stop operating, and the synthesis processing unit 250 stops the synthesis processing. Therefore, the power consumption and heat generation of the image pickup apparatus 100 are reduced.

[0111] For example, when the temperature of the imaging device 100 rises to exceed the stop threshold value during recording, there is a possibility that a part of the circuit will automatically stop in order to protect the circuit of the imaging device 100 and the safety of the user, and the recording will be interrupted as a result. However, when the temperature becomes higher than the upper limit threshold value Th1 which is lower than the stop threshold value, the imaging device 100 switches from the compound eye processing to the single eye processing to reduce power consumption and heat generation. Therefore, the temperature rise is suppressed, and recording can be continued.

[0112] "Examples of camera operation"

[0113] Fig. 9 1 is a flowchart showing an operation example of the image pickup apparatus 100 according to the first embodiment of the present technology. For example, when an application for executing image pickup processing is executed, the operation is started.

[0114] The imaging apparatus 100 starts capturing color image data and monochrome image data (step S901). Then, the imaging apparatus 100 performs a synthesis process for synthesizing the color image data and the monochrome image data (step S902).

[0115] The imaging apparatus 100 determines whether the temperature is higher than the upper limit threshold Th1 (step S903 ). In a case where the temperature is equal to or lower than the upper limit threshold Th1 (step S903 : No), the imaging apparatus 100 executes step S902 and subsequent steps again.

[0116] On the other hand, in a case where the temperature is higher than the upper limit threshold value Th1 (step S903 : Yes), the imaging apparatus 100 performs the transition process (step S904 ) and performs the single-eye process (step S905 ).

[0117] Then, the imaging apparatus 100 determines whether the temperature is lower than the lower threshold Th2 (step S906 ). If the temperature is equal to or higher than the lower threshold Th2 (step S906 : No), the imaging apparatus 100 executes step S905 and subsequent steps again.

[0118] On the other hand, when the temperature is lower than the lower limit threshold Th2 (step S906 : Yes), the imaging apparatus 100 performs the transition process (step S907 ), and repeatedly executes step S902 and subsequent steps.

[0119] In this way, in the first embodiment of the present technology, when the temperature becomes higher than the upper limit threshold value Th1, the control unit 210 controls the signal processing unit 240 to switch from compound eye processing to single eye processing, so that the processing amount of the signal processing unit 240 can be reduced and its power consumption can be reduced. For various different examples of physical quantities, the physical quantity can be determined according to a voltage value that changes corresponding to the physical quantity. For example, a temperature sensor can output a voltage value that changes according to the sensed temperature, and the voltage value can change according to changes in the remaining amount of battery capacity, etc.

[0120] "First Modification Example"

[0121] In the above-mentioned first embodiment, when the temperature becomes higher than the upper limit threshold value Th1, the control unit 210 switches the compound eye processing to the monocular processing, thereby reducing power consumption. However, by the method of switching according to the temperature rise, the power consumption may not be sufficiently reduced. For example, in a case where the external temperature of the imaging device 100 is very low, the imaging device 100 is cooled by the external air so that the temperature becomes equal to or lower than the threshold value, and thus the switch to the monocular processing may not be performed. The control unit 210 according to the first modified example of the first embodiment differs from the first embodiment in that the switch from the compound eye processing to the monocular processing is performed according to a physical quantity other than the temperature.

[0122] Fig.101 is a block diagram showing a configuration example of an imaging apparatus 100 according to a first modification example of the first embodiment of the present technology. The imaging apparatus 100 according to the first modification example of the first embodiment is different from the first embodiment in that a statistical processing unit 230 is provided instead of the temperature sensor 220.

[0123] The statistical processing unit 230 obtains statistics (average value, total value, etc.) of the detection values ​​from the signal processing unit 240. The signal processing unit 240 supplies the detection values ​​of the color image data and the detection values ​​of the monochrome image data to the statistical processing unit 230 via the signal line 238. Then, the statistical processing unit 230 obtains the statistics of these detection values ​​and supplies them to the control unit 210 via the signal line 239.

[0124] The control unit 210 compares the statistic from the statistical processing unit 230 with the threshold value, and switches between monocular processing and compound eye processing based on the comparison result. The statistic represents the amount of light obtained from the two image data. For example, when the statistic (the amount of light) becomes greater than the upper threshold value, the control unit 210 switches the compound eye processing to the monocular processing. Therefore, in the case where the camera environment is bright, it is not necessary to increase the brightness by synthesis, so the monocular processing is performed and the power consumption is reduced. On the other hand, in the case where the amount of light becomes less than the lower threshold value, the control unit 210 switches the monocular processing to the compound eye processing.

[0125] Fig.11 1 is a block diagram showing a configuration example of the signal processing unit 240 according to the first modification example of the first embodiment of the present technology. In this first modification example of the first embodiment, the color image signal processing unit 241 and the monochrome image signal processing unit 247 provide the statistical processing unit 230 with detection values.

[0126] As described above, in the first modified example of the first embodiment of the present technology, when the amount of light becomes greater than the threshold, the control unit 210 switches the compound eye processing to the monocular processing, thereby reducing the processing amount of the signal processing unit 240 and reducing its power consumption according to the amount of light.

[0127] "Second modified example"

[0128] In the first modified example of the above-mentioned first embodiment, when the amount of light becomes greater than a threshold value, the control unit 210 switches from compound eye processing to monocular processing, thereby reducing power consumption. However, by the method of switching according to the amount of light, there is a possibility that power consumption cannot be sufficiently reduced. For example, in a case where the imaging environment of the imaging device 100 is dark, the amount of light may become less than the threshold value, and switching to monocular processing may not be performed. The control unit 210 according to this second modified example of the first embodiment differs from the control unit 210 of the first modified example of the first embodiment in that switching from compound eye processing to monocular processing is performed based on a physical quantity other than the light intensity.

[0129] Fig.12 1 is a block diagram showing a configuration example of a color image signal processing unit 241 according to a second modified example of the first embodiment of the present technology. The color image signal processing unit 241 according to the second modified example of the first embodiment is different from the first modified example of the first embodiment in that a movement amount measuring unit 246 is further included.

[0130] The movement amount measuring unit 246 measures the movement amount of the object based on a plurality of color image data captured in a time series. The movement amount measuring unit 246 temporarily stores a predetermined number of color image data, for example, and detects the moving object using an inter-frame difference method, a background difference method, or the like. Then, the movement amount measuring unit 246 calculates, as the movement amount, the vector length from the representative position (center of gravity, etc.) of the moving object detected in one of the two color image data captured at different times to the representative position of the moving object detected in the other of the two color image data. The movement amount measuring unit 246 provides the measured movement amount to the statistical processing unit 230.

[0131] In addition, the monochrome image signal processing unit 247 measures the amount of movement and provides the amount of movement to the statistical processing unit 230. The statistical processing unit 230 obtains statistics (average value, etc.) of the amount of movement and provides it to the control unit 210. For example, when the statistic is equal to or greater than the threshold, the control unit 210 switches the monocular processing to the compound eye processing. On the other hand, when the statistic becomes less than the threshold, the control unit 210 switches the compound eye processing to the monocular processing. Therefore, in the case where the object is not moving, the synthesis processing for increasing the brightness can be stopped, so that power consumption can be reduced.

[0132] It should be noted that although the imaging device 100 switches between the compound eye processing and the monocular processing based on the comparison result between the movement amount and the threshold value, it is also possible to switch based on the comparison result between the remaining battery amount and the threshold value. In this case, for example, it is only necessary to add a remaining battery amount measurement unit that measures the remaining battery amount so that when the remaining battery amount becomes less than the threshold value, the control unit 210 controls the signal processing unit 240 to switch from the compound eye processing to the monocular processing.

[0133] In this way, in the second modified example of the first embodiment of the present technology, when the movement amount becomes smaller than the threshold, the control unit 210 switches the compound eye processing to the monocular processing, so that the processing amount of the signal processing unit 240 can be reduced according to the movement amount, and thus its power consumption can be reduced.

[0134] <2. Second Embodiment>

[0135] In the first embodiment described above, when the temperature becomes higher than the upper limit threshold value Th1, the control unit 210 switches from the compound eye processing to the single eye processing to reduce power consumption. However, since the synthesis processing for improving the brightness is stopped when switching to the single eye processing, there is a possibility that the image quality of the image data is reduced due to insufficient brightness. The camera apparatus 100 according to the second embodiment is different from the first embodiment in that, when the temperature becomes higher than the upper limit threshold value Th1, the power consumption of the interface is reduced while the synthesis processing is performed.

[0136] Fig.13 2 is a block diagram showing a configuration example of a signal processing unit 240 according to a second embodiment of the present technology. The signal processing unit 240 according to the second embodiment is different from the first embodiment in that a scaling processing unit 249 is further included.

[0137] The scaling processing unit 249 reduces the image size of the signal-processed monochrome image data at a predetermined reduction rate. Here, for example, the reduction rate R is represented by the following expression.

[0138] R = (image size after reduction) / (image size before reduction)

[0139] The scaling processing unit 249 supplies the image data obtained after the reduction as the reduced image data to the transmission unit 248. In addition, the switching signal SW1 from the control unit 210 is input to the scaling processing unit 249. The switching signal SW1 is a signal indicating switching from one of the initial value and the target value regarding the reduction rate to the other. In addition, a value smaller than the initial value is set as the target value, and the reduced image data can be superimposed on the synthetic image data to be transmitted by 1 channel using the target value. It should be noted that the initial value may be "1", that is, a value in which reduction is not performed.

[0140] In addition, the switching signal SW2 from the control unit 210 is input to the transmission unit 248 of the second embodiment. The switching signal SW2 is a signal that instructs switching from one of the 1 channel and the 2 channels with respect to the number of communication channels for transmitting image data to the other of the 1 channel and the 2 channels. Here, as the communication channel, for example, a virtual channel compliant with the MIPI standard is used. In the case of transmission through 1 channel, the transmission unit 248 superimposes the reduced image data on the synthesized image data and transmits it.

[0141] The control unit 210 of the second embodiment sets the enable signals ENs1 to ENs3 to be enabled when starting image capture, and causes the zoom processing unit 249 to perform reduction by the initial value reduction rate through the switching signal SW1. In addition, the control unit 210 sets the communication channel to 2 channels through the switching signal SW2, and transmits the synthesized image data and the reduced image data. These image data are transmitted via mutually different communication channels.

[0142] Then, when the temperature becomes higher than the upper threshold value Th1, the control unit 210 instructs the scaling processing unit 249 to switch from the initial value of the reduction rate to the target value through the switching signal SW1. The scaling processing unit 249 performs a transition process for gradually reducing the reduction rate to the target value for each frame period. In addition, the control unit 210 switches the number of communication channels from 2 channels to 1 channel through the switching signal SW2. At the same time, in the second embodiment, even if the temperature rises, the synthesis process is continued.

[0143] Then, when the temperature becomes lower than the lower threshold value Th2, the control unit 210 instructs the scaling processing unit 249 to switch the reduction rate to the initial value through the switching signal SW1. The scaling processing unit 249 performs a transition process for gradually increasing the reduction rate to the initial value for each frame period. In addition, the control unit 210 switches the number of communication channels from 1 channel to 2 channels through the switching signal SW2.

[0144] Fig.141 is a block diagram showing a configuration example of an application processor 150 according to a second embodiment of the present technology. The application processor 150 includes a receiving unit 151 , an image recognition unit 152 , and a memory 153 .

[0145] The receiving unit 151 receives image data from the camera control device 200. The receiving unit 151 receives the reduced image data and the synthesized image data via a communication channel of 1 channel or 2 channels. Then, the receiving unit 151 provides the reduced image data to the image recognition unit 152 and causes the memory 153 to store the synthesized image data.

[0146] The image recognition unit 152 performs image recognition processing, such as face recognition and object recognition, on the reduced image data. For example, the recognition result is used to determine whether the synthesized image data is stored (ie, recorded) in the nonvolatile memory.

[0147] The memory 153 stores the synthesized image data. For example, a DRAM (Dynamic RAM) is used as the memory 153. The synthesized image data stored in the memory 153 is supplied to, for example, a display device or a nonvolatile memory.

[0148] As described above, by reducing the number of communication channels, the power consumption of the interface including the transmission unit 248 and the reception unit 151 can be reduced.

[0149] It should be noted that although the imaging device 100 switches the number of communication channels based on the comparison result between the temperature and the threshold value, the switching may be performed based on the comparison result between a physical quantity other than the temperature and the threshold value. For example, the imaging device 100 may use a physical quantity (e.g., the amount of light, the amount of movement of the object, and the remaining battery amount) to determine the switching.

[0150] As described above, in the second embodiment of the present technology, when the temperature becomes higher than the upper limit threshold Th1, the control unit 210 reduces the number of communication channels, and thus the power consumption of the interface can be reduced.

[0151] "Modification Example"

[0152] In the above-mentioned second embodiment, when the temperature becomes higher than the upper limit threshold value Th1, the control unit 210 changes the reduction rate to a small value, and even when the number of communication channels is reduced, the image data can be transmitted using the small value. However, if the reduction rate becomes smaller, the image size of the reduced image data becomes smaller, and the recognition accuracy in image recognition decreases. The camera apparatus 100 according to the modified example of the second embodiment is different from the first embodiment in that when the temperature becomes higher than the upper limit threshold value Th1, the frame rate is reduced without changing the reduction rate.

[0153] Fig.151 is a block diagram showing a configuration example of an imaging apparatus 100 according to a modification example of the second embodiment of the present technology. This imaging apparatus 100 according to the modification example of the second embodiment is different from the second embodiment in that a synchronization control unit 260 is additionally provided in the imaging control apparatus 200.

[0154] The synchronization control unit 260 provides the vertical synchronization signal VSYNC to the camera modules 110 and 130 and the signal processing unit 240. The switching signal SW3 from the control unit 210 is input to the synchronization control unit 260 via the signal line 269. The switching signal SW3 is a signal that instructs switching from one of the initial value and the target value regarding the frequency (i.e., the frame rate) of the vertical synchronization signal VSYNC to the other of the values. A value lower than the initial value is set as the target value, and with this target value, the reduced image data can be superimposed on the synthesized image data to be transmitted by 1 channel.

[0155] Furthermore, according to the modified example of the second embodiment, the switching signal SW1 for switching the reduction rate is not input to the signal processing unit 240 .

[0156] When the temperature becomes higher than the upper threshold value Th1, the control unit 210 instructs the frame rate to be switched from the initial value to the target value through the switching signal SW3. Then, when the temperature becomes lower than the lower threshold value Th2, the control unit 210 instructs the frame rate to be switched to the initial value through the switching signal SW3. Meanwhile, in the modified example of the second embodiment, the reduction rate is unchanged.

[0157] In this way, in the modified example of the second embodiment of the present technology, when the temperature becomes higher than the upper limit threshold Th1, the control unit 210 reduces the number of communication channels to reduce the frame rate, and thus the power consumption of the interface can be reduced without changing the reduction rate.

[0158] <3. Application examples for mobile objects>

[0159] The technology according to the present disclosure (the present technology) is applicable to various products. For example, the technology according to the present disclosure can be implemented as a device installed on any type of mobile body (e.g., a car, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, a personal mobile device, an airplane, a drone, a ship, and a robot).

[0160] Fig.16 : is a block diagram showing a schematic configuration example of a vehicle control system as an example of a moving body control system to which the technology according to the present disclosure is applicable.

[0161] The vehicle control system 12000 includes a plurality of electronic control units connected via a communication network 12001. Fig.16In the example shown, the vehicle control system 12000 includes a driving system control unit 12010, a body system control unit 12020, a vehicle exterior information detection unit 12030, a vehicle interior information detection unit 12040, and an integrated control unit 12050. In addition, as a functional configuration of the integrated control unit 12050, a microcomputer 12051, an audio image output unit 12052, and an in-vehicle network I / F (interface) 12053 are shown.

[0162] The driving system control unit 12010 controls the operation of devices related to the vehicle driving system according to various programs. For example, the driving system control unit 12010 serves as a control device for a driving force generating device (e.g., an internal combustion engine and a driving motor) that generates a driving force for the vehicle, a driving force transmission mechanism for transmitting the driving force to the wheels, a steering mechanism for adjusting the direction angle of the vehicle, a braking device for generating a braking force for the vehicle, and the like.

[0163] The body system control unit 12020 controls the operation of various devices installed in the body according to various programs. For example, the body system control unit 12020 is used as a control device for a keyless entry system, a smart key system, a power window device, and various lights (e.g., a headlight, a reverse light, a brake light, a turn signal lamp, and a fog light). In this case, radio waves transmitted from a mobile device that replaces a key or a signal of various switches may be input to the body system control unit 12020. The body system control unit 12020 receives input of these radio waves or signals and controls the door lock device, the power window device, the lights, etc. of the vehicle.

[0164] The vehicle exterior information detection unit 12030 detects exterior information of the vehicle on which the vehicle control system 12000 is installed. For example, the camera unit 12031 is connected to the vehicle exterior information detection unit 12030. The vehicle exterior information detection unit 12030 causes the camera unit 12031 to capture an image of the exterior of the vehicle and receives the captured image. The vehicle exterior information detection unit 12030 may perform object detection processing or distance detection processing regarding a person, a car, an obstacle, a sign, characters on a road surface, etc. based on the received image.

[0165] The imaging unit 12031 is a light sensor that receives light and outputs an electrical signal corresponding to the amount of light received by the light. The imaging unit 12031 can output the electrical signal as an image, or output the electrical signal as distance measurement information. In addition, the light received by the imaging unit 12031 can be visible light or invisible light, for example, infrared light.

[0166] The vehicle interior information detection unit 12040 detects information of the interior of the vehicle. For example, a driver state detection unit 12041 that detects the driver's state is connected to the vehicle interior information detection unit 12040. The driver state detection unit 12041 includes, for example, a camera that captures the driver, and the vehicle interior information detection unit 12040 can calculate the driver's fatigue level or concentration level based on the detection information input from the driver state detection unit 12041, or determine whether the driver is dozing off.

[0167] The microcomputer 12051 can calculate a control target value of a driving force generating device, a steering mechanism, or a braking device based on external information or vehicle internal information acquired by the vehicle external information detection unit 12030 or the vehicle internal information detection unit 12040, and output a control command to the driving system control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing functions of an ADAS (Advanced Driver Assistance System), including collision avoidance or collision mitigation of vehicles, follow-up driving based on a distance between vehicles, driving while maintaining a vehicle speed, vehicle collision warning, vehicle lane departure warning, and the like.

[0168] In addition, the microcomputer 12051 controls the driving force generating device, steering mechanism, braking device, etc. based on the peripheral information of the vehicle acquired by the vehicle external information detection unit 12030 or the vehicle internal information detection unit 12040, so as to be able to perform collaborative control aimed at achieving autonomous driving, which allows the vehicle to drive autonomously without relying on the driver's operation, etc.

[0169] Furthermore, the microcomputer 12051 can output a control command to the body system control unit 12020 based on the vehicle exterior information acquired by the vehicle exterior information detection unit 12030. For example, the microcomputer 12051 can control the headlights according to the position of the preceding vehicle or the oncoming vehicle that has been detected by the vehicle exterior information detection unit 12030, and perform cooperative control aimed at achieving anti-glare by switching the high beam to the low beam, and the like.

[0170] The audio image output unit 12052 transmits an output signal of at least one of audio and image to an output device, which can visually or auditorily notify information to the passengers of the vehicle or the outside of the vehicle. Fig.16 In the example shown, an audio speaker 12061, a display unit 12062, and a dashboard 12063 are illustrated as output devices. For example, the display unit 12062 may include at least one of an in-vehicle display and a head-up display.

[0171] Fig.17 12031 is a diagram showing an example of the setting position of the camera unit 12031.

[0172] exist Fig.17 In the figure, the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104 and 12105.

[0173] For example, the camera units 12101, 12102, 12103, 12104, and 12105 are arranged at positions corresponding to the front nose, side mirrors, rear bumper, rear door, upper part of the front window of the cab, etc. of the vehicle 12100. The camera unit 12101 arranged at the front nose and the camera unit 12105 arranged at the upper part of the front window of the cab mainly acquire images in front of the vehicle 12100. The camera units 12102 and 12103 arranged at the side mirrors mainly acquire images of the sides of the vehicle 12100. The camera unit 12104 arranged at the rear bumper or rear door mainly acquires images behind the vehicle 12100. The camera unit 12105 arranged at the upper part of the front window of the cab is mainly used to detect the front vehicle, pedestrians, obstacles, traffic signals, traffic signs, lanes, etc.

[0174] It should be noted that Fig.17 An example of the imaging ranges of the imaging units 12101 to 12104 is shown. The imaging range 12111 indicates the imaging range of the imaging unit 12101 disposed at the front nose, the imaging ranges 12112 and 12113 respectively indicate the imaging ranges of the imaging units 12102 and 12103 disposed at the side mirrors, and the imaging range 12114 indicates the imaging range of the imaging unit 12104 disposed at the rear bumper or the rear door. For example, by overlapping the image data captured by the imaging units 12101 to 12104 with each other, a top-view image of the vehicle 12100 viewed from above can be obtained.

[0175] At least one of the imaging units 12101 to 12104 may include a function of acquiring distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera including a plurality of imaging devices, or may be an imaging device including pixels for phase difference detection.

[0176] For example, the microcomputer 12051 can calculate the distance to each three-dimensional object within the imaging ranges 12111 to 12114 and the time change of the distance (relative speed relative to the vehicle 12100) based on the distance information obtained from the imaging units 12101 to 12104, thereby extracting a three-dimensional object that is closest to the vehicle 12100 on the driving path and is particularly traveling at a predetermined speed (e.g., 0 km / h or higher) in the same direction as the vehicle 12100 as the leading vehicle. In addition, the microcomputer 12051 can pre-set the inter-vehicle distance ensured behind the leading vehicle in order to perform automatic braking control (including follow-up stop control), automatic acceleration control (including follow-up start control), etc. In this way, cooperative control aimed at realizing autonomous driving in which the vehicle travels autonomously without relying on the operation of the driver, etc. can be performed.

[0177] For example, based on the distance information obtained from the camera units 12101 to 12104, the microcomputer 12051 can extract three-dimensional object data related to the three-dimensional object, while classifying it as a two-wheeled vehicle, a standard-sized vehicle, a large vehicle, a pedestrian, or other three-dimensional objects such as a utility pole, and use it to automatically avoid obstacles. For example, the microcomputer 12051 identifies obstacles around the vehicle 12100 as obstacles visible to the driver of the vehicle 12100 or obstacles that are difficult to see. Then, the microcomputer 12051 determines a collision risk indicating the degree of risk of collision with each obstacle, and when the collision risk is equal to or greater than a set value and therefore there is a possibility of collision, a warning is output to the driver via the audio speaker 12061 or the display unit 12062, or forced deceleration or evasive steering is performed via the driving system control unit 12010, so that driving assistance for avoiding collision can be performed.

[0178] At least one of the camera units 12101 to 12104 may be an infrared camera that detects infrared rays. For example, the microcomputer 12051 may identify a pedestrian by determining whether there is a pedestrian in an image captured by the camera units 12101 to 12104. This identification of pedestrians is performed by, for example, a program that extracts feature points in an image captured by the camera units 12101 to 12104 as infrared cameras, and a program that performs pattern matching processing on a series of feature points indicating the outline of an object to determine whether there is a pedestrian. When the microcomputer 12051 determines that there is a pedestrian in the captured image of the camera units 12101 to 12104 and identifies the pedestrian, the audio image output unit 12052 controls the display unit 12062 to superimpose a rectangular outline during display to emphasize the identified pedestrian. Alternatively, the audio image output unit 12052 may control the display unit 12062 to display an icon indicating a pedestrian at a desired position, etc.

[0179] So far, an example of a vehicle control system to which the technology according to the present disclosure is applicable has been described. The technology according to the present disclosure is applicable to the camera unit 12031 in the above configuration. Specifically, Figure 1 The camera device 100 shown is suitable for Fig.16 The camera unit 12031 shown. By applying the technology according to the present disclosure to the camera unit 12031, the power consumption of the camera unit 12031 can be reduced, thereby suppressing the consumption of the vehicle-mounted battery.

[0180] It should be noted that the above-mentioned embodiments are examples that embody the present technology, and the matters in the embodiments and the specific matters of the present invention within the scope of the claims have a corresponding relationship, respectively. Similarly, the specific matters of the present invention within the scope of the claims and the matters with the same names as the specific matters of the present invention within the scope of the claims in the embodiments of the present technology have a corresponding relationship, respectively. However, the present technology is not limited to the above-mentioned embodiments, and can be implemented by making various modifications without departing from the gist of the present technology.

[0181] In addition, the processing procedures described in the above embodiments may be regarded as a method including the series of procedures, or may be regarded as a program for causing a computer to execute the series of procedures or a recording medium storing such a program. Examples of such recording media include CDs (Compact Discs), MDs (Mini Discs), DVDs (Digital Versatile Discs), memory cards, Blu-ray Discs (registered trademarks), and the like.

[0182] It should be noted that the effects described in the specification are merely examples and should not be limited thereto, and other effects may be obtained.

[0183] It should be noted that the present technology can also take the following configurations.

[0184] (1) A camera control device comprising:

[0185] a signal processing unit that performs compound eye processing of performing signal processing on each of the plurality of image data to synthesize the plurality of image data or single eye processing of performing signal processing on any one of the plurality of image data according to a predetermined control signal; and

[0186] The control unit supplies a predetermined control signal to the signal processing unit based on a comparison result between the measured predetermined physical quantity and a predetermined threshold value, and switches one of the compound eye processing and the monocular processing to the other of the compound eye processing and the monocular processing.

[0187] (2) The imaging control device according to (1), wherein:

[0188] The predetermined physical quantity is a physical quantity that fluctuates according to the imaging environment.

[0189] (3) The imaging control device according to (2), further comprising

[0190] Temperature sensor, measuring temperature as a predetermined physical quantity,

[0191] in,

[0192] When the temperature is higher than a predetermined threshold, the control unit switches the compound eye process to the single eye process.

[0193] (4) The imaging control device according to (2), wherein:

[0194] The signal processing unit includes a process of measuring the light amount of incident light as a predetermined physical quantity and acquiring the light amount as a photometric quantity, and

[0195] When the amount of light is greater than a predetermined threshold, the control unit switches the compound eye processing to the single eye processing.

[0196] (5) The imaging control device according to (1), wherein:

[0197] The signal processing unit includes a process of measuring the movement amount of the object as a predetermined physical amount, and

[0198] When the movement amount is greater than a predetermined threshold, the control unit switches the monocular processing to the compound eye processing.

[0199] (6) The imaging control device according to any one of (1) to (5), wherein:

[0200] The plurality of pieces of image data are images captured in synchronization with a predetermined synchronization signal, and

[0201] After a transition process for performing synthesis is performed while changing the synthesis ratio every time a predetermined cycle elapses, the signal processing unit switches one of the compound eye process and the monocular process to the other of the compound eye process and the monocular process.

[0202] (7) A camera control device comprising:

[0203] a synthesis processing unit that synthesizes a plurality of pieces of image data and outputs the data as synthesized image data;

[0204] a transmission unit that performs, according to a predetermined control signal, a single-channel transmission process for transmitting the synthesized image data together with a single piece of image data of any one of the plurality of pieces of image data via a single communication channel, or performs a multi-channel transmission process for transmitting the single piece of image data and the synthesized image data, respectively, via mutually different communication channels; and

[0205] The control unit provides a predetermined control signal to the signal processing unit based on a comparison result between the measured predetermined physical quantity and a predetermined threshold value, and switches one of the single-channel transmission processing and the multi-channel transmission processing to the other of the single-channel transmission processing and the multi-channel transmission processing.

[0206] (8) The imaging device according to (7), further comprising

[0207] The scaling processing unit reduces any one of the multiple image data.

[0208] in,

[0209] The control unit controls the scaling processing unit to change the reduction rate based on a comparison result between the predetermined physical quantity and a predetermined threshold value.

[0210] (9) The imaging device according to (7), further comprising

[0211] A synchronization control unit provides a predetermined synchronization signal,

[0212] in,

[0213] Each of the plurality of pieces of image data is image data captured in synchronization with a predetermined synchronization signal, and

[0214] The control unit controls the synchronization control unit to change the frequency of the predetermined synchronization signal based on a comparison result between the predetermined physical quantity and the predetermined threshold value.

[0215] (10) A camera device comprising:

[0216] A camera module, capturing multiple image data;

[0217] a signal processing unit that performs compound eye processing of performing signal processing on each of the plurality of pieces of image data to synthesize the plurality of pieces of image data, or performs single eye processing of performing signal processing on any one of the plurality of pieces of image data, according to a predetermined control signal; and

[0218] The control unit supplies a predetermined control signal to the signal processing unit based on a comparison result between the measured predetermined physical quantity and a predetermined threshold value, and switches one of the compound eye processing and the monocular processing to the other of the compound eye processing and the monocular processing.

[0219] (11) A control method for a camera control device, comprising:

[0220] a comparing step for comparing the measured predetermined physical quantity with a predetermined threshold value, and providing a predetermined control signal to the signal processing unit based on the comparison result; and

[0221] A switching step is used to switch from one of compound eye processing that performs signal processing on each of multiple image data to synthesize the multiple image data and single eye processing that performs signal processing on any one of the multiple image data to the other of compound eye processing and single eye processing according to a predetermined control signal.

[0222] (1A) A camera control device comprising:

[0223] a signal processor configured to perform compound eye processing of synthesizing a plurality of pieces of image data and single eye processing of processing any one of the plurality of pieces of image data; and

[0224] The controller is configured to perform a comparison between the physical quantity and a threshold value and cause the signal processor to perform compound eye processing or single eye processing based on the comparison.

[0225] (2A) The imaging control device according to (1A), wherein:

[0226] Physical quantities are measured physical conditions that fluctuate in the imaging environment.

[0227] (3A) The imaging control device according to (2A), further comprising:

[0228] Temperature sensors measure temperature as a physical quantity,

[0229] in,

[0230] When the temperature is higher than the threshold, the controller causes the signal processor to switch from compound eye processing to single eye processing.

[0231] (4A) The imaging control device according to (2A), wherein:

[0232] The controller determines the amount of incident light as a physical quantity, and

[0233] When the incident light amount is greater than the threshold, the signal processor switches the compound eye processing to the single eye processing.

[0234] (5A) The imaging control device according to (1A), wherein:

[0235] The controller determines the amount of movement of the object as a physical quantity, and

[0236] When the movement amount is greater than the threshold, the signal processor switches the monocular processing to the compound eye processing.

[0237] (6A) The imaging control device according to (1A), wherein:

[0238] The plurality of pieces of image data are images captured in synchronization with a predetermined synchronization signal, and

[0239] After a transition process for performing synthesis is performed while changing the synthesis ratio every time a predetermined cycle elapses, the signal processor switches one of the compound eye process and the monocular process to the other of the compound eye process and the monocular process.

[0240] (7A) The imaging control device according to (2A), wherein:

[0241] Once it is determined that the physical quantity is higher than the threshold, the controller causes the signal processor to gradually switch from compound eye processing to monocular processing.

[0242] (8A) The imaging control device according to (2A), wherein:

[0243] After the controller causes the signal processor to switch to the monocular processing when determining that the physical amount is higher than the threshold, the controller causes the monocular processing to continue the monocular processing until the physical amount decreases to a second threshold lower than the threshold.

[0244] (9A) The imaging control device according to (2A), wherein:

[0245] When the physical amount increases to a third threshold value higher than the threshold value, the controller causes the signal processor to terminate the compound eye process and the single eye process.

[0246] (10A) The imaging control device according to (1A), wherein:

[0247] The plurality of pieces of image data include a first piece of image data captured by the first camera module and a second piece of image data captured by the second camera module.

[0248] (11A) The imaging control device according to (1A), wherein:

[0249] The physical quantity is the remaining amount of a battery that supplies power to the camera control device.

[0250] (12A) The imaging control device according to (1A), wherein:

[0251] The physical quantity is determined based on the voltage value that changes corresponding to the physical quantity.

[0252] (13A) A camera device comprising:

[0253] at least one lens configured to capture a plurality of pieces of image data; and

[0254] A signal processor configured to perform compound eye processing of synthesizing a plurality of pieces of image data or monocular processing of performing processing on any one of the plurality of pieces of image data; and

[0255] The controller is configured to cause the signal processor to perform compound eye processing or single eye processing based on a comparison result between the physical quantity and the threshold value.

[0256] (14A) A camera control device comprising:

[0257] a signal processor configured to synthesize the plurality of pieces of image data to generate synthesized image data;

[0258] a transmission unit configured to perform a single-channel transmission process of transmitting the synthesized image data together with any one of the plurality of image data via a single communication channel, or a multi-channel transmission process of transmitting the any one of the image data and the synthesized image data separately via mutually different communication channels; and

[0259] The controller is configured to cause the signal processor to perform single-channel transmission processing or multi-channel transmission processing based on a comparison result between the physical quantity and the threshold value.

[0260] (15A) The imaging device according to (14A), further comprising:

[0261] The scaling processing circuit reduces any one of the multiple image data.

[0262] in,

[0263] The controller controls the scaling processing circuit to change the reduction rate based on a comparison result between the physical quantity and the threshold value.

[0264] (16A) The imaging device according to (14A), further comprising:

[0265] The synchronization control circuit provides a predetermined synchronization signal.

[0266] in,

[0267] Each of the plurality of pieces of image data is image data captured in synchronization with a predetermined synchronization signal, and

[0268] The controller controls the synchronization control circuit to change the frequency of the predetermined synchronization signal based on a comparison result between the physical quantity and the threshold value.

[0269] (17A) A control method for a camera control device, comprising:

[0270] controlling a signal processor configured to perform a compound eye process of synthesizing a plurality of pieces of image data and a single eye process of processing any one of the plurality of pieces of image data; and

[0271] A comparison of the physical quantity with a threshold value is performed, and based on the comparison, a signal processor is caused to perform compound eye processing or single eye processing.

[0272] (18A) A non-transitory computer readable medium storing program code for processing image data, the program code being executable by a processor to perform operations comprising:

[0273] controlling a signal processor configured to perform a compound eye process of synthesizing a plurality of pieces of image data and a single eye process of processing any one of the plurality of pieces of image data; and

[0274] A comparison of the physical quantity with a threshold value is performed, and based on the comparison, a signal processor is caused to perform compound eye processing or single eye processing.

[0275] Reference numerals list

[0276] 100 Camera Equipment

[0277] 110, 130 camera module

[0278] 111, 131 Optical System

[0279] 120, 140 Solid-state imaging device

[0280] 121, 141 Microlens

[0281] 122 Color Filter

[0282] 123, 143 Optical receiving device

[0283] 124, 144 wiring layer

[0284] 150 Application Processor

[0285] 151 Receiving Unit

[0286] 152 Image Recognition Unit

[0287] 153 Memory

[0288] 200 Camera control equipment

[0289] 210 Control unit

[0290] 220 Temperature sensor

[0291] 230 Statistical Processing Unit

[0292] 240 Signal Processing Unit

[0293] 241 Color Image Signal Processing Unit

[0294] 242 De-mosaic processing unit

[0295] 243 Detection Unit

[0296] 244 Exposure control unit

[0297] 245 Focus control unit

[0298] 246 Movement measurement unit

[0299] 247 Monochrome Image Signal Processing Unit

[0300] 248 transmission units

[0301] 249 Scaling Processing Unit

[0302] 250 Composition Processing Units

[0303] 251 Positioning Processing Unit

[0304] 252 YC conversion unit

[0305] 253 Luminance signal synthesis unit

[0306] 254 RGB conversion units

[0307] 255 Switch

[0308] 260 Synchronous Control Unit

[0309] 12031 Camera unit.

Claims

1. A camera control device, comprising: a signal processor configured to perform compound eye processing for synthesizing a plurality of pieces of image data and single eye processing for processing any one of the plurality of pieces of image data; as well as a controller configured to perform a comparison between the physical quantity and a threshold value and, based on the comparison, cause the signal processor to perform the compound eye processing or the monocular processing, wherein the controller is further configured to perform a comparison of the physical quantity with a first threshold value and a second threshold value smaller than the first threshold value among the threshold values, cause the signal processor to perform the compound eye processing until the physical quantity is higher than the first threshold value, and cause the signal processor to return from the monocular processing to the compound eye processing in response to the physical quantity becoming lower than the second threshold value, The physical quantity is determined according to a voltage value that changes corresponding to the physical quantity.

2. The imaging control device according to claim 1, wherein: The physical quantity is a measured physical condition that fluctuates in the imaging environment.

3. The imaging control device according to claim 2, further comprising: Temperature sensor, measuring temperature as the physical quantity, in, When the temperature is higher than the first threshold, the controller causes the signal processor to switch the compound-eye processing to the single-eye processing.

4. The imaging control device according to claim 2, wherein: The controller determines the amount of incident light as the physical quantity, and The controller causes the signal processor to switch the compound eye process to the single eye process when the amount of the incident light is greater than the first threshold.

5. The imaging control device according to claim 1, wherein: The plurality of pieces of image data are images captured in synchronization with a predetermined synchronization signal, and The signal processor switches one of the compound eye processing and the monocular processing to the other of the compound eye processing and the monocular processing after a transition process for performing synthesis is performed every time a synthesis ratio is changed over a predetermined period.

6. The imaging control device according to claim 2, wherein: Upon determining that the physical quantity is higher than the first threshold, the controller causes the signal processor to gradually switch the compound eye processing to the monocular processing.

7. A camera device, comprising: at least one lens configured to capture a plurality of image data; as well as A signal processor configured to perform compound eye processing for synthesizing a plurality of pieces of image data or monocular processing for performing processing on any one of the plurality of pieces of image data; as well as a controller configured to cause the signal processor to perform the compound eye processing or the monocular processing based on a comparison result between the physical quantity and a threshold value, wherein the controller is further configured to perform a comparison of the physical quantity with a first threshold value and a second threshold value smaller than the first threshold value among the threshold values, cause the signal processor to perform the compound eye processing until the physical quantity is higher than the first threshold value, and cause the signal processor to return from the monocular processing to the compound eye processing in response to the physical quantity becoming lower than the second threshold value, The physical quantity is determined according to a voltage value that changes corresponding to the physical quantity.

8. A control method for a camera control device, the control method comprising: controlling a signal processor configured to perform a compound eye process of synthesizing a plurality of pieces of image data and a single eye process of processing any one of the plurality of pieces of image data; and performing a comparison between a physical quantity and a threshold value, and causing the signal processor to perform the compound eye processing or the single eye processing based on the comparison, Wherein, performing the comparison between the physical quantity and the threshold value comprises performing the comparison between the physical quantity and a first threshold value and a second threshold value smaller than the first threshold value among the threshold values, wherein causing the signal processor to perform the compound eye processing or the monocular processing based on the comparison includes causing the signal processor to perform the compound eye processing until the physical quantity is higher than the first threshold, and causing the signal processor to return from the monocular processing to the compound eye processing in response to the physical quantity becoming lower than the second threshold, The physical quantity is determined according to a voltage value that changes corresponding to the physical quantity.

Images