Shooting device and focusing control program

Abstract

A shooting device is provided comprising: a first camera unit having a first optical system; a second camera unit installed with a second optical system facing the same direction as the first camera unit; and a focusing control unit that performs focusing control of the first camera unit and the second camera unit. The first camera unit and the second camera unit each have a shooting element formed by a two-dimensional arrangement of a general pixel that outputs an image signal for forming an image and a phase pixel that is discretely installed surrounded by the general pixel and outputs a phase difference signal for detecting focus. When the focusing control unit forms an image using an image signal output from the shooting element of the first camera unit, it performs focusing control of the first optical system by referring to second defocus information obtained from the phase difference signal output by the shooting element of the second camera unit.

Description

Technology Field

[0001] The present invention relates to a shooting device and a focusing control program. Background Technology

[0002] A conventional imaging device is known to perform automatic focusing control using a phase difference signal obtained from the upper surface of an imaging element. An example of an imaging element that outputs such a phase difference signal is an imaging element formed by a two-dimensional arrangement of a general pixel that outputs an image signal dedicated to image generation and a phase pixel that outputs a phase difference signal dedicated to automatic focusing (e.g., refer to Patent Document 1). Such an imaging element has the advantage of being able to be manufactured at a lower cost compared to an imaging element that is equipped with two photoelectric converters in which every pixel is divided into pupils for each microlens and is capable of switching between the output of an image signal and the output of a phase difference signal.

[0003] Prior art literature

[0004] Patent documents

[0005] Patent Document 1: Patent Publication No. 2016-90785. The problem to be solved

[0006] Phase pixels, which output only phase difference signals, do not output image signals for generating an image. Therefore, when an image is generated, the pixel values ​​installed within the phase pixel's address are created through the interpolation of the surrounding pixel values. Consequently, if the proportion of phase pixels among the pixels forming the imaging element increases, the quality of the generated image deteriorates. In particular, if multiple phase pixels are arranged adjacently and continuously, they may appear as noticeable noise in the generated image. Therefore, it is desirable to install phase pixels discretely, surrounded by general pixels, and to keep their proportion small. However, when phase pixels are installed in this manner, it becomes difficult to detect phase difference signals, which reflect the subject as relatively small compared to the overall image captured by the imaging element, making it impossible to focus on the subject. For example, in a scene where a long, slender main subject is placed in front of a background with many high-frequency components, a failure is likely to occur where the focus shifts to the background instead of the main subject.

[0007] The present invention has been completed to solve these problems and provides a shooting device, etc., capable of accurately autofocusing on a main subject even when generating a wide-angle image in which the main subject is reflected relatively small. means of solving the problem

[0008] A shooting device of the first embodiment of the present invention comprises: a first camera unit having a first optical system; a second camera unit installed with a second optical system facing the same direction as the first camera unit; and a focusing control unit that performs focusing control of the first camera unit and the second camera unit; wherein the first camera unit and the second camera unit each have a shooting element formed by a two-dimensional arrangement of a general pixel that outputs an image signal for forming an image and a phase pixel that is discretely installed surrounded by the general pixel and outputs a phase difference signal for detecting focus; and when the focusing control unit forms the image using the image signal output from the shooting element of the first camera unit, it performs focusing control of the first optical system by referring to second defocus information obtained from the phase difference signal output by the shooting element of the second camera unit.

[0009] A focusing control program of a second embodiment of the present invention performs focusing control of a first camera unit and a second camera unit of a shooting device, wherein the shooting device comprises: a first camera unit having a first optical system; and a second camera unit installed with a second optical system facing the same direction as the first camera unit; wherein the first camera unit and the second camera unit each have a shooting element formed by a two-dimensional arrangement of a general pixel that outputs an image signal for forming an image and a phase pixel that is discretely installed surrounded by the general pixel and outputs a phase difference signal for detecting focus, and when the image is formed using the image signal output from the shooting element of the first camera unit, the computer performs an acquisition step of acquiring second defocus information based on the phase difference signal output from the shooting element of the second camera unit; and a driving step of driving a focusing lens of the first optical system by referring to the second defocus information. Effects of the invention

[0010] According to the present invention, a shooting device capable of accurately autofocusing on a main subject is provided even when generating a wide-angle image in which the main subject is reflected relatively small. Brief explanation of the drawing

[0011] FIG. 1 is a drawing illustrating the external appearance of the imaging device of the present embodiment. Figure 2 is a diagram illustrating the installation of the main hardware of the imaging device. Figure 3 is a diagram illustrating the pixel array of a shooting element. Figure 4 is a drawing illustrating an example of a scene to be filmed. FIG. 5 is a diagram illustrating an example of an image obtained from a first camera unit when automatic focusing control is performed by a phase difference signal from a first camera unit. FIG. 6 is a diagram illustrating an example of an image obtained from a second camera unit when automatic focusing control for the same scene is performed by a phase difference signal from a second camera unit. FIG. 7 is a diagram illustrating an example of an image obtained from a first camera unit when automatic focusing control of a first optical system is performed by referring to second defocus information. Figure 8 is a diagram illustrating the correspondence relationship between two focusing areas. Figure 9 is a diagram illustrating the processing process up to the generation of a wide-angle image. Specific details for implementing the invention

[0012] The present invention will be described below through embodiments thereof, but the invention defined by the claims is not limited to the following embodiments. Furthermore, not all installations described in the embodiments are essential means for solving technical problems. Additionally, in cases where there are multiple identical or identical installations in each drawing, reference numerals may sometimes be indicated only on some parts to avoid complex structures, while identical reference numeral notations may be omitted for other parts.

[0013] FIG. 1 is a drawing illustrating the external appearance of a shooting device (100) of the present embodiment. In particular, FIG. 1 (a) is a drawing illustrating the first surface of the shooting device (100), and FIG. 1 (b) is a drawing illustrating the second surface, which is the opposite side of the first surface. The shooting device (100) of the present embodiment is a so-called smartphone, that is, a smartphone that functions as a shooting device. Below, the shooting function related to the present invention among the smartphone functions will be explained, and other functions as a smartphone using image data generated by shooting will be omitted. Furthermore, in the present embodiment, the shooting device (100) is described as an example of a smartphone, but it is obvious that it may be a shooting device used as an independent camera or a device equipped with a shooting function, such as one embedded in a tablet computer terminal.

[0014] The shooting device (100) is equipped with a first camera unit (110) and a second camera unit (120) installed facing the same direction on a first surface. The first camera unit (110) is a camera unit for generating a wide-angle image. The second camera unit (120) is a camera unit for generating a telephoto image. When a user wants to acquire a wide-angle image, they specify the first camera unit to take a picture, and when they want to acquire a telephoto image, they specify the second camera unit to take a picture. The first camera unit (110) and the second camera unit (120) are installed parallel to the long side of the shooting device (100) in the drawing, but the installation of the two camera units is not limited to this; for example, they may be installed along a straight line that intersects the long side at an angle. In addition, the positions of the first camera unit (110) and the second camera unit (120) in the drawing may be opposite to each other.

[0015] The shooting device (100) is equipped with a display device (130) on the second surface. The display device (130) is, for example, a display device using an organic electroluminescence (EL) panel, and displays a real-time image of the subject before shooting (real-time background display) or displays an image after shooting. Additionally, a selfie camera unit independent of the first camera unit (110) and the second camera unit (120) may be installed on the second surface.

[0016] A shutter button (161) is installed on the side of the shooting device (100). The user can provide shooting instructions to the shooting device (100) by pressing the shutter button (161). Additionally, a touch panel (162) is installed overlapping with the display device (130). Instead of pressing the shutter button (161), the user may provide shooting instructions to the shooting device (100) by tapping the shutter button displayed on the display device (130). Additionally, the user may tap any part of the subject displayed on the real-time background to designate a specific area including that part as a focusing area. Furthermore, the user may perform a switch between the first camera unit (110) and the second camera unit (120) or select a displayed menu item through a contact action such as tapping.

[0017] FIG. 2 is a drawing illustrating the main hardware installation of a shooting device (100). The shooting device (100) is installed in addition to the first camera unit (110), the second camera unit (120), and the display device (130), as well as a system control unit (150) that controls them, and peripheral elements that cooperate with the system control unit (150).

[0018] As described above, the first camera unit (110) is a camera unit for generating a wide-angle image and mainly comprises a first optical system (111), a first driving mechanism (112), a first imaging element (113), and a first analog front end (AFE) (114). The first optical system (111) is an optical system for forming an image of an incident subject's light beam on the imaging surface of the first imaging element (113). Although it is depicted as a single lens in the drawing, it is generally installed as a plurality of lenses, and at least some of them are focusing lenses capable of advancing and retracting along the optical axis direction. The first driving mechanism (112) is a driving mechanism for moving the focusing lens of the first optical system (111) along the optical axis direction and includes an actuator that operates according to the instructions of the system control unit (150).

[0019] The first imaging element (113) is, for example, a CMOS image sensor. Detailed information regarding the first imaging element (113) will be described later. The first imaging element (113) transmits a pixel signal (an image signal and a phase difference signal described later), which is an output signal, to the first analog front end (114) in accordance with the instructions of the system control unit (150). The first analog front end (114) adjusts the level of the pixel signal according to the gain instructed by the system control unit (150), converts the A / D into digital data, and transmits it to the working memory (151).

[0020] As described above, the second camera unit (120) is a camera unit for generating a telephoto image and mainly comprises a second optical system (121), a second driving mechanism (122), a second imaging element (123), and a second analog front end (AFE) (124). The second optical system (121) is an optical system for forming an image of an incident subject's light beam on the imaging surface of the second imaging element (123). Although it is depicted as a single lens in the drawing, the second optical system (121) is generally installed with multiple lenses, similar to the first optical system (111), and at least some of them are focusing lenses capable of advancing and retracting along the optical axis direction. The second driving mechanism (122) is a driving mechanism for moving the focusing lens of the second optical system (121) along the optical axis direction and includes an actuator that operates according to the instructions of the system control unit (150).

[0021] The second imaging element (123) is, for example, a CMOS image sensor. Detailed information regarding the second imaging element (123) will be described later together with the first imaging element (113). The second imaging element (123) transmits a pixel signal, which is an output signal, to the second analog front end (124) according to the instructions of the system control unit (150). The second analog front end (124) adjusts the level of the pixel signal according to the gain instructed by the system control unit (150), converts the A / D into digital data, and transmits it to the working memory (151).

[0022] In this embodiment, it is assumed that both the first optical system (111) and the second optical system (121) are single-focus optical systems having fixed focal lengths, but at least one may be a variable-focus optical system (zoom lens) capable of changing the focal length. Even when using a variable-focus optical system, the focal length of the first optical system (111) may be set shorter than the focal length of the second optical system (121). That is, the angle of view of the second optical system (121) is set to a telephoto angle of view relative to the angle of view of the first optical system (111).

[0023] The system control unit (150) is a processor (CPU: Central Processor Unit) that directly or indirectly controls each element of the shooting device (100). The system control unit (150) acts as a various function control unit according to the control program executed. For example, when performing focusing control of the first camera unit (110) and the second camera unit (120), it functions as a focusing control unit, and when displaying the captured image on the display device (130), it functions as a display control unit.

[0024] The shooting device (100) is equipped with a working memory (151), an image processing unit (152), an operating unit (160), a storage unit (170), and a communication interface (180) as peripheral elements that cooperate with the system control unit (150). The working memory (151) is a volatile high-speed memory and is installed, for example, as a static random access memory (SRAM). The working memory (151) receives pixel data converted sequentially from the first analog front end (114) and the second analog front end (124), respectively, and if the pixel data is data converted from an image signal, it stores it collectively in one frame of frame data. In addition, if the data is data converted from a phase difference signal, it stores it collectively in two waveform data. The working memory (151) transmits the frame data to the image processing unit (152) and transmits the waveform data to the system control unit (150). Additionally, the working memory (151) is also appropriately used as a temporary storage area in a processing step where the image processing unit (152) performs image processing or in a processing step where the system control unit (150) performs focusing processing.

[0025] The image processing unit (152) is installed, for example, as an Application Specific Integrated Circuit (ASIC) that specializes in image processing, and performs various image processing, such as interpolation processing, on the received frame data to generate image data that conforms to a preset format. If the generated image data is to be stored, it is stored in the storage unit (170), and if it is to be displayed, it is displayed on the display device (130).

[0026] The operating unit (160) is an input device including a shutter button (161) or a touch panel (162) and is a component that is operated when a user provides instructions to the shooting device (100). When the shooting device (100) receives voice input, the operating unit (160) may further include a microphone. The storage unit (170) is a non-volatile memory and is installed, for example, as a Solid State Drive (SSD). The storage unit (170) not only stores image data generated by shooting but also preserves constants, variables, settings, and control programs necessary for the operation of the shooting device (100). The communication interface (180) may include a 5G circuit or a wireless LAN communication unit. The communication interface (180) is intended to transmit the generated image data to an external device.

[0027] FIG. 3 is a diagram illustrating the pixel arrangement of the first imaging element (113). In this embodiment, since the second imaging element (123) is identical to the first imaging element (113), the first imaging element (113) will be described here.

[0028] The first imaging element (113) is an imaging element formed by a two-dimensional arrangement of a general pixel (210) that outputs an image signal dedicated to image generation and a phase pixel (220) that outputs a phase difference signal dedicated to focus detection. The general pixel (210) is a pixel in which a single photoelectric converter, generally square in shape, is installed without displacement relative to a microlens. In the general pixel (210), a color filter of RGB is installed between the microlens and the photoelectric converter.

[0029] A phase pixel (220) is a pixel in which a photoelectric converter, which is generally rectangular and similar in shape to one of the two parts of a general pixel's photoelectric converter divided by a single microlens, is displaced with respect to the optical axis of the microlens. The phase pixel (220) does not have a color filter installed between the microlens and the photoelectric converter. All pixels adjacent to the phase pixel (220) are general pixels (210), that is, each phase pixel (220) is surrounded by general pixels (210) and installed discretely from one another. Additionally, in the installation of the phase pixel (220), the installation of the photoelectric converter may be the same as that of the general pixel (210), and a light-blocking mask having a displacement opening is installed between the microlens and the photoelectric converter, and said displacement opening produces the same effect as when the photoelectric converter is displaced as described above.

[0030] In the phase pixel (220), there are two types: a first phase pixel (221) in which the photoelectric conversion part is displaced toward the first direction (downward in the drawing), and a second phase pixel (222) in which the photoelectric conversion part is displaced toward the opposite direction (upward in the drawing). The first phase pixel (221) and the second phase pixel (222) are each arranged in a defined pattern. Specifically, a plurality of detection lines are set along the displacement direction (upward and downward direction in the drawing) of the displacement pixel, and the first phase pixel (221) is periodically arranged on one side (right in the drawing) of each detection line, and the second phase pixel (222) is arranged on the other side (left in the drawing) with the same period and in a different phase.

[0031] A first phase waveform is formed according to a phase difference signal, which is the output signal of the first phase pixel (221), and a second phase waveform is formed according to a phase difference signal, which is the output signal of the second phase pixel (222). Additionally, in focusing control, the system control unit (150) obtains defocus information by calculating a defocus amount, which is the relative deviation amount of the first phase waveform and the second phase waveform, a defocus direction, which is the deviation direction, and a focusing evaluation value obtained according to the degree of overlap between the two waveforms. The system control unit (150) performs focusing processing based on the obtained defocus information to focus on a defined subject. A detailed explanation of the focusing processing will be provided later.

[0032] Additionally, if the first imaging element (113) and the second imaging element (123) are both imaging elements formed by two-dimensionally arranging a phase pixel that is discretely installed and surrounded by a general pixel, they may not be identical imaging elements. Each imaging element may differ from one another, for example, in terms of the total number of pixels or the arrangement pattern of the phase pixel.

[0033] In addition, as shown in the example of the attached drawing, a detection line is set along a single axis direction in the up-down direction, but a detection line can also be set in an orthogonal direction (left-right direction in the example of the attached drawing) and phase pixels suitable for the detection line (phase pixels shifted to the right and phase pixels shifted to the left in the example of the attached drawing) can be arranged. In this case, it is preferable that each phase pixel be surrounded by a normal pixel. When the detection line is set along two orthogonal axes in this way, focusing precision can be improved.

[0034] In addition, since phase pixels that output only phase difference signals do not output image signals for generating an image, when an image is generated based on an image signal output by a general pixel, the pixel value of the pixel where the phase pixel address is installed is generally generated by interpolation processing of the pixel values ​​of surrounding pixels. Therefore, if the proportion of phase pixels among the pixels forming the imaging element increases, the quality of the generated image deteriorates. In particular, if multiple phase pixels are arranged adjacently and continuously, they may appear as noticeable noise in the generated image. Therefore, as in the first imaging element (113) and the second imaging element (123) of the present embodiment, it is preferable that phase pixels be installed discretely in a manner surrounded by general pixels, and it is also preferable that their proportion be small. In the first imaging element (113) and the second imaging element (123) of the present embodiment, the total number of phase pixels (220) is less than 5% of the total number of pixels.

[0035] However, as described above, if phase pixels are installed in small quantities and discretely, the focusing control unit cannot obtain accurate defocus information in which the subject is reflected relatively small relative to the entire image captured by the imaging element, and as a result, focusing on the subject may not be possible. For example, in a scene where a long, slender main subject is placed in front of a background with many high-frequency components, a failure is likely to occur where the focus is on the background side rather than the main subject. Such a failure occurs more easily when using an optical system with a wide angle of view. Therefore, when the imaging device (100) of the present embodiment forms a wide-angle image using an image signal output from the first imaging element (113) of the first camera unit (110), the focusing control of the first optical system (111) is performed by referring to the second defocus information obtained from the phase difference signal output by the second imaging element (123) of the second camera unit (120). Below, the focusing control is described in order with specific scenes.

[0036] FIG. 4 is a drawing illustrating an example of a scene to be photographed. Specifically, it shows a case where a user intends to photograph a scene in which a forest (920) unfolds behind a person (910) as the main subject through a shooting device (100). Here, it is assumed that the user intends to focus the attention on the person (910). The user can determine the composition while checking the real-time background image that is continuously acquired through the first camera unit (110) or the second camera unit (120) and displayed on the display device (130).

[0037] FIG. 5 is a diagram illustrating an example of an image obtained from the first camera unit (110) when automatic focusing control of the first optical system (111) is performed by a phase difference signal from the first camera unit (110). Since the forest (920), which is the background of the scene, is a collection of many trees, the upper area (background area) of the forest (920) within the field of view captured by the first camera unit (110) is an area with a high spatial frequency. On the other hand, the upper area (main area) of the main subject, the person (910), occupies only a very small part within the field of view captured by the first camera unit (110) and is an area with a relatively low spatial frequency.

[0038] When the detection line intersects a background area with a high spatial frequency and a main area with a low spatial frequency, so-called long-distance and short-distance collisions occur, and the focusing control unit calculates defocus information for the background area with a high spatial frequency. When the focusing lens of the first optical system (111) is driven based on the defocus information calculated in this way, the first optical system (111) is focused on the forest (920) which is the background, and the wide-angle image (301) generated after focusing is a blurry image of the main subject, a person (910), as shown in the drawing. In particular, if the phase pixels (220) installed along the detection line are discrete, the number of phase pixels (220) included in the small main area becomes smaller, and it becomes more difficult for the focusing control unit to calculate defocus information based on the waveform formed by the phase difference signal output from the phase pixels (220) included in the main area.

[0039] That is, if automatic focusing control is performed using only the first defocus information obtained from the phase difference signal output by the first shooting element (113) in the scene of FIG. 4, it can be said that it is difficult to focus on the person (910) that the user wants to focus on.

[0040] FIG. 6 is a diagram illustrating an example of an image obtained from a second camera unit (120) when automatic focusing control of a second optical system (121) for the same scene is performed by a phase difference signal from a second camera unit (120). Since the second optical system (121) of the second camera unit (120) has a telephoto angle of view compared to the first optical system (111), the proportion of the image area (main area) of the person (910) that occupies the entire area is greater than in the case of FIG. 5. Next, the number of phase pixels (220) included in the main area increases, and detailed parts of the person (910) are more distinguishable, so the spatial frequency is also higher.

[0041] Even when the detection line intersects the main area and the background area, if the proportion of the main area is large and the waveform formed by the phase difference signal output by the phase pixel (220) within the main area occupies a dominant position, it is not easy for a long-distance or short-distance collision to occur. Therefore, the focusing control unit can calculate defocus information for the main area. When the focusing lens of the second optical system (121) is driven based on the defocus information calculated in this way, the second optical system (121) is focused on the main subject, the person (910), and as shown in the drawing, the telephoto image (302) generated after focusing is an image in which the focus is concentrated on the person (910).

[0042] That is, when automatic focusing control is performed based on the second defocus information obtained from the phase difference signal output by the second imaging element (123) in the scene of FIG. 4, it can be said that it is easy for the user to focus on the person (910) they wish to focus on. Therefore, even when the user selects the first camera unit (110) to acquire a wide-angle image, if the second defocus information obtained from the phase difference signal output by the second imaging element (123) is referenced, the possibility of the first optical system (111) focusing on the person (910) can be increased.

[0043] For such focusing control, even when the user selects the first camera unit (110), the focusing control unit drives the second imaging element (123) so that the phase pixel (220) outputs a phase difference signal to obtain second defocus information. The focusing control unit refers to the second defocus information to move the focusing lens of the first optical system (111) so that it focuses on the person (910). Specifically, the second defocus information includes area information of the second focusing area (320), which is an area where a focusing evaluation is performed in the second imaging element (123), and the amount of defocus, the direction of defocus, and the focusing evaluation value within the second focusing area.

[0044] FIG. 7 is a diagram illustrating an example of an image obtained from the first camera unit (110) when automatic focusing control of the first optical system (111) is performed by referring to the second defocus information. The focusing control unit determines the first focusing area (310), which is the focusing area of ​​the first camera unit (110), according to the area information of the second focusing area (320) included in the second defocus information. The first camera unit (110) and the second camera unit (120) are installed close to each other, and since the optical axis of the first optical system (111) and the optical axis of the second optical system (121) are parallel, the depth of the subject may not be considered for simplification, and a conversion formula or reference table that can correspond one-to-one based on each angle of view is prepared in advance. The focusing control unit determines the first focusing area (310) according to the area information of the second focusing area (320) using this conversion formula or reference table.

[0045] When the first focusing area (310) is determined, the focusing control unit performs focusing control of the first optical system (111) on the premise that the main subject exists in the said area. Specifically, a phase waveform is generated only for the phase difference signal included in the first focusing area (310) from the phase difference signal output by the first imaging element (113). In addition, by limiting the focusing range (i.e., the depth range where the main subject is assumed to exist) by referring to the defocus amount and defocus direction of the second defocus information, the background area is not affected. The defocus amount and defocus direction of the first defocus information are determined according to these limited conditions. The focusing control unit determines the direction and amount of movement of the focusing lens of the first optical system (111) according to the determined defocus amount and defocus direction, and can be focused on the main subject, the person (910), by moving in this manner.

[0046] After the focusing lens moves, the focusing control unit causes the first imaging element (113) to output a phase difference signal again and evaluates the phase waveform thereof to determine whether it is in a focusing state. If it is determined that it is in a focusing state, it performs shooting processing to generate a wide-angle image (301). If it is determined that it has not reached a focusing state, it can acquire the first defocus information again to correct the position of the focusing lens. Alternatively, it can correct the position of the focusing lens while wobbling the focusing lens so that the contrast of some images generated according to the image signal output from the general pixels included in the first focusing area (310) is maximized. The latter is so-called contrast AF. Furthermore, more simply, the focusing control unit can execute contrast AF targeting the first focusing area (310) after determining the first focusing area (310) based on the second focusing area (320) without acquiring the first defocus information from the first imaging element (113).

[0047] Additionally, if the focusing control unit determines that focusing control of the first optical system (111) is difficult to perform based on the first defocus information, it may perform focusing control of the first optical system by referring to the second defocus information. That is, if it determines that focusing control of the first optical system (111) is performed based on the first defocus information, it may perform focusing control of the first optical system without referring to the second defocus information. For example, it may be determined whether it is difficult to perform focusing control of the first optical system (111) based on the first defocus information depending on whether the focusing evaluation value among the first defocus information is smaller than or greater than the threshold value. Alternatively, the second defocus information may be obtained in parallel with the acquisition of the first defocus information, and the determination may be made by comparing the respective defocus amount and defocus direction. Specifically, if the depth of the subject calculated according to the defocus amount and defocus direction included in the first defocus information, and the depth of the subject calculated according to the defocus amount and defocus direction included in the second defocus information are within a fixed range, it is assumed that the same subject has been captured, so it is determined that focusing control of the first optical system (111) can be performed based on the first defocus information. If they are not within the fixed range, it is assumed that different subjects have been captured, so it is determined that focusing control of the first optical system (111) based on the first defocus information is difficult to perform.

[0048] In the above, when the first focusing area (310) is determined according to the second focusing area (320), an example using a conversion formula for brevity has been explained, but in reality, the first focusing area (310) corresponding to the second focusing area (320) can change according to the change in the depth of the subject (distance to the subject). FIG. 8 is a diagram illustrating the correspondence between two focusing areas. Specifically, the above diagram illustrates the case where the first camera unit (110) and the second camera unit (120) each capture a person located at a distance d1 and a person located at a distance d2 from the shooting device (100), respectively, and the telephoto image (302) and wide-angle image (301) captured in this manner.

[0049] As illustrated in the drawing, it is assumed that both a person at a close distance (d1) and a person at a far distance (d2) are close to the second camera unit (120). In this case, when comparing the telephoto image (302) generated from the second camera unit (120) and the wide-angle image (301) generated from the first camera unit, the gap between the second focusing area (320) that captures the person at the close distance (d1) and the first focusing area (310) is greater than the gap between the second focusing area (320) that captures the person at the far distance (d2) and the first focusing area (310) that captures the person at the far distance (d2). Additionally, when comparing with the same wide-angle image (301), it can be seen that the first focusing area (310) is displaced to the right when the person is at the d1 distance compared to when the person is at the d2 distance.

[0050] This correspondence can be calculated through a triangulation method. Specifically, when calculating the distance between the second focusing area (320) and the main subject, a person, in a telephoto image (302), the first focusing area (310) in the wide-angle image (301) can be determined using the reference line length, which is the distance between the optical axes of two optical systems, and the ratio of the angles of view of two optical systems. Additionally, the distance to the person can be calculated based on the amount of defocus and the direction of defocus in the second defocus information, and the position of the focusing lens at that time.

[0051] If the focusing control unit determines the first focusing area (310) corresponding to the second focusing area (320) more accurately in this way, the accuracy of the focusing control of the first optical system (111) can be further improved.

[0052] Additionally, if the user does not specifically designate a focusing area, the focusing control unit may determine the main subject to be focused according to an algorithm of general near point priority (priority for subjects close to the shooting device) or center priority (priority for subjects near the center of the angle of view). In this case, if the subject calculated according to the defocus amount and defocus direction included in the first defocus information is at a greater distance than the subject calculated according to the defocus amount and defocus direction included in the second defocus information, or is located far from the center, it may be determined that focusing control of the first optical system (111) is difficult to perform based on the first defocus information.

[0053] Additionally, if the user specifies a defined focusing area, the area can be used as the first focusing area (310). Additionally, if a defined area is specified through a face area recognition program, the area can be used as the first focusing area (310). Even in such cases, if it is determined that focusing control of the first optical system (111) is difficult to perform based on the first defocus information, a second focusing area (320) may be determined, and focusing control of the first optical system (111) may be performed by referring to the second defocus information within the area.

[0054] Next, an example of a series of processes in which the system control unit (150) functions as a main focusing control unit when the user selects the first camera unit (110) to take a wide-angle image is described. FIG. 9 is a diagram illustrating the main processing steps until the system control unit (150) generates a wide-angle image. For example, the flow begins from the moment the user presses the shutter button (161).

[0055] In step (S101), the focusing control unit acquires first defocus information. Specifically, as described above, the first imaging element (113) is driven to output a phase difference signal from the phase pixel (220), and various operations are performed to acquire the first defocus information. In step (S102), the focusing control unit acquires second defocus information in the same manner as acquiring the first defocus information. The processing of step (S102) may be performed in parallel with step (S101).

[0056] In step (S103), the focusing control unit determines whether focusing control of the first optical system (111) can be performed based on the first defocus information. If it is determined that it can be done, step (S104) is skipped and the process proceeds to step (S105); if it is determined that it cannot be done, the process proceeds to step (S104).

[0057] When entering step (S104), the focusing control unit determines the first focusing area (310) among the first imaging element (113) by referring to the second defocus information obtained in step (S102) and enters step (S105).

[0058] When entering step (S105), if step (S104) is omitted, the focusing control unit moves the focusing lens of the first optical system (111) based on the first defocus information to focus on the main subject. After passing through step (S104), in the determined first focusing area (310), as described above, for example, by applying limited conditions, the first defocus information is obtained again, and the focusing lens of the first optical system (111) is moved based on the first defocus information to focus on the main subject.

[0059] The focusing control unit enters step (S106) to cause the first imaging element (113) to output a phase difference signal again, and evaluates the phase waveform thereof to determine whether it is in a focusing state. If it is determined that it is in a focusing state, step (S107) is skipped and it enters step (S108), and if it is determined that it is not in a focusing state, it enters step (S107).

[0060] When entering step (S107), as described above, the focusing control unit performs contrast AF and corrects the position of the focusing lens so that it focuses on the main subject. Then, it enters step (S108).

[0061] When entering step (S108), the system control unit (150) drives the first capturing element (113) to output an image signal from a general pixel and generates image data in the image processing unit (152). The system control unit (150) completes a series of processing by storing the generated image data in the storage unit (170) or displaying it on the display device (130) according to preset instructions, or by transmitting it to an external device through the communication interface (180).

[0062] In the embodiment described above, it was assumed that a still image was captured by the first camera unit (110), but the same focusing control can be performed even when capturing a moving image. For example, by continuously generating frame images according to the image signal among the pixel signals output from the first capturing element (113) and generating first defocus information according to the phase difference signal, the same focusing control as above can be performed even when capturing a moving image by referring to the second defocus information generated in parallel.

[0063] In addition, although the above-described embodiment describes a case where the shooting device (100) is equipped with two camera units, the same focusing control can be performed even if the shooting device is equipped with three or more camera units. For example, if the three camera units each have optical systems for a telephoto angle, a standard angle, and a wide angle, the focusing control of the wide angle camera unit can refer to defocus information obtained from the standard angle camera unit and the telephoto angle camera unit, respectively. In addition, the focusing control of the standard angle camera unit can refer to defocus information obtained from the telephoto angle camera unit.

[0064] In addition, although the above-described embodiment describes an example in which focusing control is performed based on one defocus information obtained by each focusing lens of the first optical system (111) and the second optical system (121) in any state, focusing control may also be performed by obtaining multiple defocus information while changing the position of the focusing lens. For example, if an optical system with a small aperture F-value or an optical system with a long focal length is used, the position of the focusing lens may be changed multiple times according to its characteristics to obtain defocus information each time. In this case, if a camera unit with a telephoto angle of view uses a liquid lens capable of changing the focal length according to the applied voltage, the range of movement of the focusing lens can be determined based on the applied voltage. That is, since such a liquid lens may be a high-magnification zoom lens, the time to obtain defocus information can be shortened by allowing the focusing lens to move only within that range, on the premise that it is focused on a subject located 1.5m away in the telephoto area.

[0065] In other possible embodiments, the liquid lens may be a wide-angle lens or a telephoto lens, and since the liquid lens must change the thickness of the liquid unit via a motor during focusing, the stroke required during focusing may be relatively long or the movement speed of the motor may be relatively slow, which may affect the focusing speed. In this case, focusing control can be performed by assisting the optical system including the liquid lens based on defocus information of another optical system at the same time. For example, by determining a plurality of focusing distance intervals, the liquid lens is focused only within a relatively small range, thereby improving the focusing speed. Explanation of the symbols

[0066] 100: Imaging device, 110: First camera unit, 111: First optical system, 112: First driving mechanism, 113: First imaging element, 114: First analog front end (AFE), 120: Second camera unit, 121: Second optical system, 122: Second driving mechanism, 123: Second imaging element, 124: Second analog front end (AFE), 130: Display device, 150: System control unit, 151: Working memory, 152: Image processing unit, 160: Actuator unit, 161: Shutter button, 162: Touch panel, 170: Storage unit, 180: Communication interface, 210: Normal pixel, 220: Phase pixel, 221: First phase pixel, 222: Second phase pixel, 301: Wide-angle image, 302: Telephoto image, 310: 1st focusing area, 320: 2nd focusing area, 910: Person, 920: Forest.

Claims

Claim 1 In a shooting device, a first camera unit having a first optical system; a second camera unit installed with a second optical system facing the same direction as the first camera unit; and, a focusing control unit that performs focusing control of the first camera unit and the second camera unit; wherein the first camera unit and the second camera unit each have a general pixel that outputs an image signal for forming an image and a two-dimensionally arranged imaging element formed by a phase pixel that is discretely installed surrounded by the general pixel and outputs a phase difference signal for detecting focus; and when the focusing control unit forms the image using the image signal output from the imaging element of the first camera unit, it performs focusing control of the first optical system by referring to second defocus information obtained from the phase difference signal output by the imaging element of the second camera unit; the focusing control unit obtains the first defocus information and the second defocus information in parallel, and the focusing control unit compares the first defocus information and the second defocus information to determine whether it is difficult to perform focusing control of the first optical system; and when the focusing control unit determines that the second defocus information is necessary to assist focusing control of the first optical system, the A shooting device characterized by performing focusing control of the first optical system based on second defocus information, wherein the second defocus information includes area information of a second focusing area, which is an area in which a focusing evaluation is performed at the imaging element of the second camera unit, and a defocus amount, a defocus direction, and a focusing evaluation value within the second focusing area. Claim 2 A shooting device according to claim 1, characterized in that the focusing control unit determines a focusing area during the focusing control of the first optical system based on a reference line length determined by the installation of the first camera unit and the second camera unit. Claim 3 A shooting device according to claim 1, wherein the focusing control unit drives the focusing lens of the first optical system by referring to the second defocus information, and then corrects the position of the focusing lens based on contrast information calculated using the image signal output from the imaging element of the first camera unit. Claim 4 A shooting device according to any one of claims 1 to 3, wherein the first optical system includes a liquid lens capable of focusing according to an electrical signal, and the focusing control unit controls the first optical system to perform focusing based on the electrical signal determined by at least one of first defocus information and second defocus information. Claim 5 A focusing control program stored on a computer-readable storage medium performs focusing control of a first camera unit and a second camera unit of a shooting device, wherein the shooting device comprises: a first camera unit having a first optical system; and a second camera unit installed with a second optical system facing the same direction as the first camera unit; wherein the first camera unit and the second camera unit each comprise a shooting element formed by a two-dimensional arrangement of a general pixel that outputs an image signal for forming an image and a phase pixel that is discretely installed surrounded by the general pixel and outputs a phase difference signal for detecting focus, and wherein when the image is formed using the image signal output from the shooting element of the first camera unit, a computer executes the focusing control program: a step of acquiring second defocus information based on the phase difference signal output from the shooting element of the second camera unit; A focusing control program characterized by performing a driving step of driving a focusing lens of the first optical system by referring to the second defocus information; acquiring the first defocus information and the second defocus information in parallel, comparing the first defocus information and the second defocus information to determine whether it is difficult to perform focusing control of the first optical system, and if it is determined that the second defocus information is needed to assist focusing control of the first optical system, performing focusing control of the first optical system based on the second defocus information, wherein the second defocus information includes area information of a second focusing area, which is an area where focusing evaluation is performed in the imaging element of the second camera unit, and a defocus amount, a defocus direction, and a focusing evaluation value within the second focusing area. Claim 6 delete Claim 7 delete Claim 8 delete

Images