Control device, imaging apparatus, control method, and program

Abstract

To provide a control device capable of acquiring highly accurate distance information while suppressing electric power consumption.SOLUTION: A control device controls a first optical system used for acquiring image information and a second optical system different from the first optical system. The control device includes: an acquisition section that acquires at least one of first distance information corresponding to the image information obtained by using the first optical system and second distance information corresponding to the image information obtained by using the second optical system; and a control section that controls the first optical system to acquire the first distance information by the acquisition section in a case where reliability of the first distance information is higher than a predetermined value and controls the second optical system to acquire the second distance information by the acquisition section in a case where the reliability of the first distance information is lower than the predetermined value.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a control device, an imaging device, a control method, and a program. [Background technology]

[0002] Conventionally, as a method for measuring the distance to an object, a passive ranging method that uses image information to measure the distance and an active ranging method that measures the distance by irradiating auxiliary light are known. With the passive ranging method, it is difficult to obtain the distance to a distant object with high accuracy. As an active ranging method, LiDAR (Light Detection and Ranging) is known, which measures the distance to an object from the time it takes to receive reflected light from an object irradiated with infrared laser light. LiDAR can obtain distance information with high accuracy regardless of the distance to the object, but consumes more power than the passive ranging method. Each ranging method has its advantages and disadvantages, and Patent Document 1 discloses a configuration in which either the active ranging method or the passive ranging method is selected based on the average brightness of the captured image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2015 / 083539 Summary of the Invention [Problem to be solved by the invention]

[0004] In the configuration of Patent Document 1, when obtaining the distance to an object with low contrast or when the distance to an object is in a blur where the entire screen is out of focus, the system does not switch from the passive ranging method to the active ranging method, which results in a deterioration in the accuracy of the distance information.

[0005] An object of the present invention is to provide a control device that can acquire distance information with high accuracy while suppressing power consumption. [Means for solving the problem]

[0006] A control device as one aspect of the present invention is a control device for controlling a first optical system used to acquire image information and a second optical system different from the first optical system, and is characterized in having an acquisition unit that acquires at least one of first distance information corresponding to the image information obtained using the first optical system and second distance information corresponding to the image information obtained using the second optical system, and a control unit that controls the first optical system so that the acquisition unit acquires the first distance information when the reliability of the first distance information is higher than a predetermined value, and controls the second optical system so that the acquisition unit acquires the second distance information when the reliability of the first distance information is lower than the predetermined value. Effect of the Invention

[0007] According to the present invention, it is possible to provide a control device capable of acquiring distance information with high accuracy while suppressing power consumption. [Brief description of the drawings]

[0008] [Figure 1] 1 is a block diagram illustrating a configuration of an imaging apparatus according to a first embodiment. [Diagram 2] FIG. 2 is an explanatory diagram of an image sensor according to the first embodiment. [Diagram 3] 2 is a cross-sectional view showing the imaging relationship of an optical image on the image sensor of the first embodiment. FIG. [Figure 4] FIG. 2 is an explanatory diagram of a LiDAR distance measuring unit according to the first embodiment. [Diagram 5] 4 is a flowchart showing the operation of the imaging apparatus according to the first embodiment. [Figure 6] 10 is a flowchart showing a second distance information acquisition process according to the first embodiment. [Figure 7] 10 is a timing chart showing timings for acquiring distance information according to the second embodiment. [Figure 8] FIG. 13 is a block diagram illustrating a configuration of an imaging device according to a third embodiment. [Figure 9]13 is a flowchart showing a second distance information acquisition process according to the third embodiment. [Figure 10] FIG. 13 is a block diagram illustrating a configuration of an imaging device according to a fourth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same reference numerals are used to refer to the same components, and duplicated descriptions will be omitted.

[0010] 2. Description of the Related Art Conventionally, known methods for measuring the distance to an object include a passive distance measuring method that measures the distance using image information and an active distance measuring method that measures the distance by irradiating an auxiliary light.

[0011] In the phase difference detection method as a passive distance measurement method, phase difference detection pixels that detect signals with different phases are arranged in an image sensor, and distance information of a subject is obtained by performing a correlation calculation between the signals with different phases.

[0012] In the active ranging method, the distance to the object is measured from the time difference between the emission timing of the infrared laser and the detection timing of the reflected light from the object (the round trip time of the infrared laser light). For example, a single photon avalanche diode (SPAD) sensor capable of detecting a single photon can be used as a sensor to detect the reflected light. The SPAD sensor detects the incident single photon as an extremely short detection pulse by avalanche multiplication. A technology that uses a time to digital converter (TDC) to measure the time from the emission timing of the infrared laser to the detection pulse timing has been put into practical use. In reality, since there is a large fluctuation in the arrival time of a single photon, the emission of the infrared laser and the detection of the single photon are periodically repeated, and the time measurement results are histogrammed and statistically processed. This can improve the accuracy of the time difference, that is, the distance measurement (hereinafter referred to as LiDAR ranging). In addition, by forming the infrared laser and the SPAD sensor into a two-dimensional array, the results of the LiDAR ranging can be two-dimensionalized, and a so-called distance map can be generated. Recent photographic equipment can use distance maps and captured images to generate three-dimensional computer graphics models and two-dimensional maps (hereinafter collectively referred to as spatial models or 3D models) and to achieve AF (Auto Focus). [First embodiment] In this embodiment, an example will be described in which the reliability of distance information obtained by the passive ranging method is determined, and if the reliability is low, the method is switched to the active ranging method. Specifically, if the reliability of distance information obtained by using the first optical system is low, distance information is obtained by using the second optical system.

[0013] FIG. 1 is a block diagram showing the configuration of an imaging device 100 according to the present embodiment. The imaging device 100 is, for example, a digital camera, a smartphone, or a drone. The imaging device 100 includes a control unit 101, an imaging lens 102, an imaging sensor (imaging unit) 103, a sensor correction unit 104, an image shift amount calculation unit 106, a reliability determination unit 107, a defocus conversion unit 108, and a lens driving unit 109. The imaging device 100 also includes a LiDAR distance measurement unit 110, a LiDAR correction unit 113, a histogram calculation unit 114, and a viewpoint position correction unit 115. The imaging lens 102, the imaging sensor 103, and the image shift amount calculation unit 106 function as a first optical system and acquire image information and first distance information corresponding to the image information. The LiDAR distance measurement unit 110 functions as a second optical system and acquires second distance information corresponding to the image information acquired using the first optical system.

[0014] In addition, s101 is incident light related to imaging, s102 is visible light RAW data, s103 is variously corrected image signals, s105 is the image shift amount, s106 is the reliability judgment result, and s107 is the defocus amount. In addition, s108 is laser light, s109 is reflected light from an object irradiated with the laser light s108, s110 is LiDAR distance measurement information, s111 is variously corrected distance information, s112 is a distance map, and s113 is distance information with the viewpoint position corrected. Furthermore, s114 is the lens drive amount.

[0015] The control unit 101 is a control device that controls the entire imaging device 100, and receives a reliability determination result s106 from the reliability determination means 107 to control the LiDAR distance measurement unit 110. The control unit 101 executes arithmetic processing and control processing according to various programs stored in a storage unit (not shown).

[0016] The control unit 101 includes an acquisition unit 101a and an optical system control unit (control unit) 101b. The acquisition unit 101a acquires at least one of first distance information corresponding to image information obtained using a first optical system and second distance information corresponding to image information obtained using a second optical system. When the reliability of the first distance information is higher than a predetermined value, the optical system control unit 101b controls the first optical system so that the acquisition unit 101a acquires the first distance information. When the reliability of the first distance information is lower than the predetermined value, the optical system control unit 101b controls the second optical system so that the acquisition unit 101a acquires the second distance information. Note that the optical system to be controlled when the reliability of the first distance information is equal to the predetermined value can be set arbitrarily.

[0017] In other words, in this embodiment, at least one processor executes a program stored in at least one memory to function as the acquisition unit 101a and the optical system control unit 101b. Specifically, at least one processor executes a process of acquiring at least one of the first and second distance information and a process of controlling either the first or second optical system depending on a period during which one of the first and second distance information cannot be acquired.

[0018] The imaging lens 102 focuses incident light s101 onto the imaging sensor 103. The imaging lens 102 performs autofocus control by moving based on a lens driving amount s114 from a lens driving unit 109. A first optical system including the imaging lens 102 and the imaging sensor 103 shares at least a part of the angle of view with a second optical system including a LiDAR distance measuring unit 110. Specifically, the first optical system and the second optical system capture at least one or more of the same objects.

[0019] The imaging sensor 103 includes a plurality of pixels each having a microlens and a photoelectric conversion unit, and performs photoelectric conversion on the image formed by the imaging lens 102 to generate visible light RAW data s102.

[0020] FIG. 2 is an explanatory diagram of the image sensor 103 of this embodiment. FIG. 2(a) shows the configuration of the image sensor 103. The image sensor 103 includes a pixel array 201, a vertical scanning circuit 202, a horizontal scanning circuit 203, and a timing generator TG204. The pixel array 201 includes a plurality of unit pixel cells arranged in a two-dimensional matrix. The timing generator TG204 generates timings for an image capturing period, a transfer period, and the like, and sends timing signals to the vertical scanning circuit 202 and the horizontal scanning circuit 203. When the image capturing period ends, the vertical scanning circuit 202 transmits signals output from the unit pixel cells to a vertical transmission path. The horizontal scanning circuit 203 sequentially outputs the accumulation signals to the outside via an output transmission path.

[0021] 2(b) shows one unit pixel cell 205 in the pixel array 201. The unit pixel cell 205 includes one microlens 206 and a pair of photoelectric conversion units 207a and 207b. The photoelectric conversion units 207a and 207b perform pupil division by receiving light beams that have passed through different pupil regions in the exit pupil of the imaging lens 102 through the common microlens 206.

[0022] Fig. 2(c) shows a pixel array 201. In order for the image sensor 103 to provide a two-dimensional image signal, the pixel array 201 has a plurality of unit pixel cells two-dimensionally arranged in row and column directions. 208, 209, 210, and 211 correspond to the unit pixel cell 205 in Fig. 2(b). 208L, 209L, 210L, and 211L correspond to the photoelectric conversion unit 207a in Fig. 2(b). 208R, 209R, 210R, and 211R correspond to the photoelectric conversion unit 207b in Fig. 2(b).

[0023] Here, the imaging relationship of an optical image (subject image) on the imaging sensor 103 will be described with reference to Fig. 3. Fig. 3 is a cross-sectional view showing the imaging relationship of an optical image on the imaging sensor 103, and conceptually shows how a light beam emitted from the exit pupil of the imaging lens 102 enters the imaging sensor 103. Reference numeral 301 denotes a microlens, and 302 denotes a color filter. Reference numeral 303 denotes the exit pupil of the imaging lens 102.

[0024] The light beam emitted from the exit pupil 303 is incident on the image sensor 103 with the optical axis 306 as the center. Reference numerals 304 and 305 denote partial regions of the exit pupil 303. Reference numerals 307 and 308 denote the outermost rays of light passing through the partial region 304 of the exit pupil 303, and reference numerals 309 and 310 denote the outermost rays of light passing through the partial region 305 of the exit pupil 303.

[0025] 3, of the light beams emitted from exit pupil 303, the upper light beam is incident on photoelectric conversion unit 207b and the lower light beam is incident on photoelectric conversion unit 207a with optical axis 306 as the boundary. That is, photoelectric conversion units 207a and 207b each receive light from a different region of exit pupil 303. Phase difference detection is performed using such characteristics.

[0026] The phase difference detection method will be described below with reference to Fig. 2(c). The photoelectric conversion unit 207a in the unit pixel cell 205 is used as an A image pixel group that photoelectrically converts the A image of a pair of subject images for focus detection by the phase difference detection method. The photoelectric conversion unit 207b is used as a B image pixel group that photoelectrically converts the B image of the pair of subject images.

[0027] In the pixel array 201 of FIG. 2(c), the row 212 that references the photoelectric conversion units 208L-211L... is the A image pixel group, and the row 213 that references the photoelectric conversion units 208R-211R... is the B image pixel group. A phase difference signal can be obtained by performing correlation calculation on the signal obtained from the A image pixel group and the signal obtained from the B image pixel group. Rows that output phase difference signals such as the rows 212 and 213 to the image shift amount calculation means 106 are called phase difference detection pixel rows. AF that performs focus detection by the phase difference detection method using the A image pixel group and the B image pixel group provided in the imaging sensor 103 in this way is called imaging surface phase difference AF. In addition, in the row 214, an image signal can be read out by adding up signals from two photoelectric conversion units of a unit pixel cell. Rows that output image signals such as the row 214 to the sensor correction means 104 are called normal pixel rows. A unit pixel cell in a normal pixel row does not have to have two separate photoelectric conversion units, and may have only one photoelectric conversion unit.

[0028] Note that a phase difference detection method other than the method described in this embodiment may be used. For example, a light shielding portion and focus detection pixels may be disposed under a microlens that performs pupil division, and image signals of a pair of subject images may be formed by combining outputs of two types of focus detection pixels with different opening positions of the light shielding portion.

[0029] The sensor correction unit 104 performs various correction processes such as shading correction and blackout correction on the signal output from the image sensor 103 .

[0030] The image shift amount calculation means 106 performs correlation calculation of each image signal that receives light beams with different incident directions, and acquires information related to the distance of the subject (first distance information). That is, the image shift amount calculation means 106 functions as a distance measurement means that acquires the first distance information based on the output of the phase difference detection pixel. The image shift amount calculation means 106 also calculates the image shift amount s105 from the correlation calculation result.

[0031] The reliability determination means 107 determines the reliability of the image shift amount s105 output from the image shift amount calculation means 106, thereby determining the reliability of the first distance information acquired by the image shift amount calculation means 106. The reliability determination means 107 outputs the reliability determination result s106 to the control unit 101. In this embodiment, the reliability determination means 107 determines the reliability of the image shift amount s105 by using the contrast value of the captured image. Specifically, when the contrast value of the captured image is higher than a predetermined value, the reliability determination means 107 determines that the reliability of the image shift amount s105 is higher than the predetermined value, and when the contrast value of the captured image is lower than the predetermined value, the reliability determination means 107 determines that the reliability of the image shift amount s105 is lower than the predetermined value.

[0032] The defocus conversion means 108 multiplies the image shift amount s105 output from the image shift amount calculation means 106 by a predetermined conversion coefficient to calculate a defocus amount s107.

[0033] The lens driving unit 109 uses the defocus amount s107 from the defocus conversion means 108 or the distance information s113 from the viewpoint position correction means 115 to calculate the lens driving amount s114 by which the imaging lens 102 is moved.

[0034] The LiDAR distance measurement unit 110 includes a laser emitter 112 and a laser receiver 111.

[0035] Here, the LiDAR distance measuring unit 110 will be described with reference to Fig. 4. Fig. 4 is an explanatory diagram of the LiDAR distance measuring unit 110. Fig. 4(a) shows the laser light receiving unit 111, and Fig. 4(b) shows the laser light emitting unit 112.

[0036] The laser emission unit 112 includes a plurality of laser emission elements 404 arranged two-dimensionally along the horizontal and vertical directions, and irradiates infrared laser light to the outside in accordance with a laser pulse control signal from the control unit 101.

[0037] The laser light receiving unit 111 includes a plurality of SPAD elements 402 arranged two-dimensionally, each corresponding to a laser light emitting element 404, receives the infrared laser light from the laser emitting unit 112 reflected by an object, and generates LiDAR distance measurement information s110. Ideally, distance information can be obtained by arranging one SPAD element 402 for one laser light emitting element 404, but in reality, the reflected light may deviate from the intended point. Therefore, in this embodiment, a SPAD element group 403 consisting of four SPAD elements 402 is made to function as one SPAD element for one laser light emitting element 404. Highly accurate distance information can be obtained by averaging the output results of the four SPAD elements 402.

[0038] The LiDAR correction means 113 performs various correction processes on the LiDAR distance measurement information s110, such as correction of positional deviation between the laser receiving unit 111 and the laser emitting unit 112 and correction related to temperature characteristics. The LiDAR correction means 113 outputs distance information s111 after correcting the LiDAR distance measurement information s110 to the histogram calculation means 114.

[0039] The histogram calculation means 114 performs a histogramming process on the distance information s111 to improve the distance measurement accuracy, and outputs the result as a two-dimensional distance map s112 having the same number as the number of laser emission units 112.

[0040] The viewpoint position correction means 115 corrects the deviation of the viewpoint positions between the LiDAR distance measurement unit 110 and the imaging lens 102 for the distance map s112, and generates distance information s113.

[0041] Hereinafter, the operation of the imaging device 100 of this embodiment will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the operation of the imaging device 100 of this embodiment. The process of the flowchart in Fig. 5 is started when a user presses a shutter button included in an operation unit (not shown).

[0042] In step S501, the control unit 101 drives the image sensor 103 to obtain visible light RAW data s102.

[0043] In step S502, the control unit 101 drives the sensor correction means 104 to perform various correction processes and obtains an image signal s103.

[0044] In step S503, the control unit 101 drives the image shift amount calculation unit 106 to obtain the image shift amount s105.

[0045] In step S504, the control unit 101 drives the reliability determination means 107 to obtain a determination result indicating whether the reliability of the image shift amount s105 is high. In this embodiment, the reliability determination means 107 determines that the reliability of the image shift amount s105 is higher than a predetermined value when the contrast value of the captured image is higher than a predetermined value, and determines that the reliability of the image shift amount s105 is lower than the predetermined value when the contrast value of the captured image is lower than the predetermined value. Note that, when the contrast value of the captured image is equal to the predetermined value, it is possible to arbitrarily set whether the reliability of the image shift amount s105 is determined to be high or low. When the reliability of the image shift amount s105 is higher than the predetermined value, the control unit 101 executes the process of step S505, and when the reliability of the image shift amount s105 is lower than the predetermined value, the control unit 101 executes the process of step S506.

[0046] In step S505, the control unit 101 drives the defocus conversion means 108 to perform defocus conversion on the image shift amount s105, and acquires a defocus amount s107.

[0047] In step S506, the control unit 101 controls the second optical system to acquire distance information (second distance information) because the reliability of the distance information is low even if the control unit 101 controls the first optical system to acquire distance information (first distance information). Specifically, the control unit 101 drives the LiDAR distance measurement unit 110 and executes a process of acquiring the second distance information (second distance information acquisition process).

[0048] In step S508, the control unit 101 drives the lens driving unit 109 to obtain a lens driving amount s114 based on the defocus amount s107 obtained in step S505 or the distance information s113 obtained in step S506.

[0049] The second distance information acquisition process in step S507 in Fig. 5 will be described below with reference to Fig. 6. Fig. 6 is a flowchart showing the second distance information acquisition process.

[0050] In step S601, the control unit 101 drives the laser emission unit 112 to irradiate infrared laser light to the outside at specific intervals.

[0051] In step S602, the control unit 101 drives the laser light receiving unit 111 to receive the reflected light from the object irradiated with the infrared laser light in step S601.

[0052] In step S603, the control unit 101 extracts time-of-flight (TOF) information of the infrared laser light reflected by the target after being irradiated at a specific interval, and acquires LiDAR distance measurement information s110.

[0053] In step S604, the control unit 101 drives the LiDAR correction means 113 to perform various correction processes on the LiDAR distance measurement information s110 and obtain corrected distance information s111.

[0054] In step S605, the control unit 101 drives the histogram calculation means 114 to perform a histogramming process on the distance information s111 to improve the distance measurement accuracy, and obtains a distance map s112.

[0055] In step S606, the control unit 101 drives the viewpoint position correction means 115 to correct the deviation of the viewpoint position between the LiDAR distance measurement unit 110 and the imaging lens 102, and acquires distance information s113.

[0056] As described above, according to the configuration of this embodiment, when the reliability of the distance information acquired using the first optical system is low, the LiDAR distance measuring unit 110 is driven to acquire distance information with high accuracy. In addition, it is possible to reduce power consumption compared to the case where the LiDAR distance measuring unit 110 is constantly driven.

[0057] In this embodiment, the reliability determination means 107 determines the reliability of the image blur amount s105 using the contrast value of the captured image, but the present invention is not limited to this. For example, the reliability determination means 107 may determine the reliability of the image blur amount s105 based on whether the first optical system is in a blurred state where the focus is not correct. In the phase difference detection method, when the first optical system is in a blurred state, distance information cannot be obtained correctly. On the other hand, since the LiDAR uses a second optical system different from the first optical system, it is possible to obtain distance information with high accuracy without being influenced by whether the first optical system is in a blurred state. It should be noted that whether the first optical system is in a blurred state may be determined based on whether the focus position of the first optical system is outside a predetermined range. Specifically, when the focus position of the first optical system is within a predetermined range, it is determined that the first optical system is not in a blurred state, and when the focus position of the first optical system is outside the predetermined range, it is determined that the first optical system is in a blurred state. [Second embodiment] In this embodiment, an example will be described in which the second optical system is used to acquire distance information while the first optical system is acquiring image information (still image) for recording. The configuration of the imaging device 100 of this embodiment is the same as that of the imaging device 100 of the first embodiment, and in this embodiment, only the configuration different from the first embodiment will be described, and the description of the common configuration will be omitted.

[0058] The operation of the imaging device 100 of this embodiment will be described with reference to Fig. 7. Fig. 7 is a timing chart showing the timing of acquiring distance information of this embodiment. In this embodiment, as an example, a case will be described in which a live view image is acquired at 120 fps (frames per sec) and a still image for recording is acquired at 30 fps. A live view image is an image to be displayed on an electronic viewfinder (EVF) (not shown) before actual shooting.

[0059] A normal digital camera does not require as many pixels as a still image, so it uses a live view image to obtain distance information used for autofocus. In other words, since distance information is not obtained in frames exposed for still images, continuous distance information may not be obtained. In this case, for example, if distance information is used for moving object prediction to track a subject, there is a concern that defective frames will occur and the accuracy of moving object prediction will decrease.

[0060] Therefore, in this embodiment, while the first optical system is acquiring still images, the LiDAR distance measurement unit 110 included in the second optical system is driven to acquire distance information. This makes it possible to continuously acquire distance information even during exposure for still images, thereby improving the accuracy of moving object prediction.

[0061] Furthermore, the frame rate at which distance information is acquired using the second optical system may be set lower than the frame rate at which distance information is acquired using the first optical system. As described in this embodiment, by acquiring distance information using the first optical system at 120 fps and acquiring distance information using the second optical system at 30 fps, it is possible to reduce power consumption compared to driving multiple optical systems at the same frame rate all the time. [Third embodiment] In this embodiment, an example will be described in which the remaining battery level is detected, and when the detected remaining battery level is less than a predetermined level, LiDAR distance measurement is not performed. In smartphones and digital cameras, reducing battery consumption is one of the most important issues. In this embodiment, only the configuration different from the first and second embodiments will be described, and the description of the common configuration will be omitted.

[0062] 8 is a block diagram showing the configuration of an image capturing apparatus 100 of this embodiment. The configuration of the image capturing apparatus 100 of this embodiment is basically the same as the configuration of the image capturing apparatus 100 of the first embodiment. Unlike the image capturing apparatus 100 of the first embodiment, the image capturing apparatus 100 of this embodiment has a battery remaining amount detection unit 801. The battery remaining amount detection unit 801 detects the remaining amount of a battery (not shown) that supplies power to the entire image capturing apparatus 100. Note that different batteries may be used for the first optical system and the second optical system.

[0063] FIG. 9 is a flowchart showing the second distance information acquisition process of the present embodiment.

[0064] In step S901, the control unit 101 drives the battery remaining amount detection unit 801 to acquire the battery remaining amount and determine whether the battery remaining amount is greater than a predetermined amount. When the control unit 101 determines that the battery remaining amount is greater than the predetermined amount, it executes the process of step S902. The processes of steps S902 to S907 are similar to the processes of steps S601 to S606 in FIG. 6, respectively, and therefore will not be described. When the control unit 101 determines that the battery remaining amount is less than the predetermined amount, it ends this flow and then executes the process of step S505 in FIG. 5. That is, the control unit 101 does not perform LiDAR ranging, but controls the first optical system to acquire the first distance information. Note that when the battery remaining amount is equal to a predetermined value, it is possible to arbitrarily set which step to proceed to. In addition, the process of step S901 is performed before performing LiDAR ranging in this embodiment, but may be performed before or during the process of FIG. 5, for example.

[0065] As described above, according to the configuration of this embodiment, it is possible to reduce power consumption. [Fourth embodiment] In this embodiment, an example will be described in which the amount of ambient light is detected, and when the amount of detected ambient light is less than a predetermined value, LiDAR distance measurement is not performed. In this embodiment, only configurations different from the first to third embodiments will be described, and descriptions of common configurations will be omitted.

[0066] FIG. 10 is a block diagram showing the configuration of the imaging device 100 of this embodiment. The configuration of the imaging device 100 of this embodiment is basically the same as the configuration of the imaging device 100 of the first embodiment. In this embodiment, unlike the first embodiment, the LiDAR distance measuring unit 110 has an ambient light detection unit 1001. The ambient light detection unit 1001 detects the amount of ambient light. Ambient light is light other than the infrared laser light emitted by the laser emission unit 112. The laser receiving unit 111 is provided with an IR filter (not shown) that passes only infrared light. However, when the amount of ambient light is large, visible light other than infrared light leaks into the photoelectric conversion unit, causing noise, making it difficult to perform high-precision distance measurement. In addition, when there are multiple LiDAR-equipped devices in the vicinity, infrared laser light from the other devices may erroneously enter the imaging device 100. In this way, when there is a lot of ambient light, whether visible light or non-visible light, it is difficult to drive the LiDAR distance measuring unit 110 to obtain high-precision distance information.

[0067] Therefore, in this embodiment, when the amount of ambient light detected by the ambient light detection unit 1001 is greater than a predetermined amount, the control unit 101 does not drive the LiDAR distance measurement unit 110 and acquires distance information using the first optical system. This makes it possible to reduce power consumption while suppressing deterioration in accuracy of the distance information. [Other Examples] The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-mentioned embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0068] The disclosure of this embodiment includes the following configurations and methods. (Configuration 1) A control device for controlling a first optical system used to acquire image information and a second optical system different from the first optical system, an acquisition unit that acquires at least one of first distance information corresponding to the image information obtained using the first optical system and second distance information corresponding to the image information obtained using the second optical system; A control device characterized by having a control unit that controls the first optical system so that the acquisition unit acquires the first distance information when the reliability of the first distance information is higher than a predetermined value, and controls the second optical system so that the acquisition unit acquires the second distance information when the reliability is lower than the predetermined value. (Configuration 2) 2. The control device according to configuration 1, wherein when the contrast of the image information is higher than a predetermined value, the reliability is higher than the predetermined value, and when the contrast is lower than the predetermined value, the reliability is lower than the predetermined value. (Configuration 3) The control device described in configuration 1 or 2, characterized in that when the focus position of the first optical system is within a predetermined range, the reliability is higher than the predetermined value, and when the focus position is outside the predetermined range, the reliability is lower than the predetermined value. (Configuration 4) A control device described in any one of configurations 1 to 3, characterized in that the control unit controls the second optical system so that the acquisition unit acquires the second distance information when the reliability is higher than the specified value and an image for recording is acquired using the first optical system. (Configuration 5) 5. The control device according to any one of configurations 1 to 4, wherein a frame rate for acquiring the second distance information is lower than a frame rate for acquiring the first distance information. (Configuration 6) A control device described in one of configurations 1 to 5, characterized in that the control unit controls the first optical system so that the acquisition unit acquires the first distance information when the reliability is lower than the predetermined value and the remaining charge of the battery for driving the first and second optical systems is less than a predetermined amount. (Configuration 7) A control device described in any one of configurations 1 to 6, characterized in that the control unit controls the first optical system so that the acquisition unit acquires the first distance information when the reliability is lower than the predetermined value and the amount of ambient light other than light reflected by an object after being emitted from the second optical system is greater than a predetermined amount. (Configuration 8) The control device according to any one of configurations 1 to 7, characterized in that the first optical system includes a normal pixel that generates an image signal of an image and a phase difference detection pixel that detects a phase difference of the image, and is equipped with an imaging unit that captures an image based on an output of the normal pixel, and a ranging unit that acquires the first distance information based on an output of the phase difference detection pixel. (Configuration 9) The control device described in any one of configurations 1 to 8, characterized in that the second optical system includes an emitter that emits light and a light receiver that acquires the second distance information using light emitted from the emitter and reflected by an object. (Configuration 10) A control device according to any one of configurations 1 to 9; A first optical system; and a second optical system different from the first optical system. (Method 1) 1. A control method for controlling a first optical system used to acquire image information and a second optical system different from the first optical system, comprising: acquiring at least one of first distance information corresponding to the image information obtained using the first optical system and second distance information corresponding to the image information obtained using the second optical system; A control method characterized by comprising the steps of: when the reliability of the first distance information is higher than a predetermined value, the acquisition unit controls the first optical system to acquire the first distance information; and when the reliability of the first distance information is lower than the predetermined value, the acquisition unit controls the second optical system to acquire the second distance information. (Configuration 11) A program for causing a computer to execute the control method according to method 1.

[0069] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various combinations, modifications, and alterations are possible within the scope of the gist of the present invention. [Explanation of symbols]

[0070] 101 Control unit (control device) 101a Acquisition Department 101b Optical system control unit (control unit)

Claims

1. A control device for controlling a first optical system used to acquire image information and a second optical system different from the first optical system, An acquisition unit that acquires at least one of first distance information corresponding to the image information obtained using the first optical system and second distance information corresponding to the image information obtained using the second optical system, A control device comprising: a control unit that controls the first optical system so that the acquisition unit acquires the first distance information when the reliability of the first distance information is higher than a predetermined value; and a control unit that controls the second optical system so that the acquisition unit acquires the second distance information when the reliability is lower than the predetermined value.

2. The control device according to claim 1, characterized in that when the contrast of the image information is higher than a predetermined value, the reliability is higher than the predetermined value, and when the contrast is lower than the predetermined value, the reliability is lower than the predetermined value.

3. The control device according to claim 1, characterized in that when the focal position of the first optical system is within a predetermined range, the reliability is higher than the predetermined value, and when the focal position is outside the predetermined range, the reliability is lower than the predetermined value.

4. The control device according to any one of claims 1 to 3, characterized in that the control unit controls the second optical system so that the acquisition unit acquires the second distance information when the reliability is higher than the predetermined value and an image for recording is acquired using the first optical system.

5. The control device according to any one of claims 1 to 3, characterized in that the frame rate for acquiring the second distance information is smaller than the frame rate for acquiring the first distance information.

6. The control device according to any one of claims 1 to 3, characterized in that the control unit controls the first optical system so that the acquisition unit acquires the first distance information when the reliability is lower than the predetermined value and the remaining battery charge for driving the first and second optical systems is less than a predetermined amount.

7. The control device according to any one of claims 1 to 3, characterized in that the control unit controls the first optical system so that the acquisition unit acquires the first distance information when the reliability is lower than the predetermined value and the amount of ambient light, which is different from the light reflected by the object after being emitted from the second optical system, is greater than a predetermined amount.

8. The control device according to any one of claims 1 to 3, wherein the first optical system includes normal pixels that generate an image signal of an image and phase difference detection pixels that detect the phase difference of the image, and comprises an imaging unit that performs imaging based on the output of the normal pixels and a distance measuring unit that acquires first distance information based on the output of the phase difference detection pixels.

9. The control device according to any one of claims 1 to 3, characterized in that the second optical system comprises a light-emitting unit that emits light and a light-receiving unit that acquires the second distance information using light emitted from the light-emitting unit and reflected by an object.

10. A control device according to any one of claims 1 to 3, The first optical system and An imaging apparatus characterized by having a second optical system different from the first optical system described above.

11. A control method for controlling a first optical system used to acquire image information and a second optical system different from the first optical system, An acquisition step of acquiring at least one of first distance information corresponding to the image information obtained using the first optical system and second distance information corresponding to the image information obtained using the second optical system, A control method characterized by comprising: a control step of controlling the first optical system to acquire the first distance information in the acquisition step when the reliability of the first distance information is higher than a predetermined value; and a control step of controlling the second optical system to acquire the second distance information in the acquisition step when the reliability of the first distance information is lower than the predetermined value.

12. A program characterized by causing a computer to execute the control method described in claim 11.

Images