Image capture device, image capture element, image capture device control method, and image capture element control method

Abstract

To enable focus adjustment at high speed with high accuracy.SOLUTION: An imaging apparatus 100 having an imaging element 130 and an image processing engine 120 includes a control signal switching section 124 that switches control information output for use in driving a focus lens 102 to one of first control information generated by the imaging element 130 for controlling driving of the focus lens 102 and second control information generated by the image processing engine 120 for controlling driving of the focus lens 102.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an autofocus function. [Background technology]

[0002] Imaging devices have an autofocus function (hereinafter, AF) that automatically adjusts the focus on a subject. It is desired to complete the AF operation quickly so as not to miss a photographing opportunity even for a fast-moving subject. The AF operation is performed by moving the focus lens in the optical axis direction of the imaging optical system. When performing AF, first, all pixels of the image captured by the imaging element are read, and then the phase difference and contrast difference in the image are recognized in the image engine of the camera to obtain the defocus amount of the lens to the in-focus position. Then, the drive amount is calculated based on the obtained defocus amount and the current position information of the lens. After that, a drive command for moving the position of the focus lens is generated and transmitted to the lens side. The lens notifies the lens drive motor of the control information through the MPU. In this series of operations, the imaging element, the image engine, and the lens are controlled in sequence, so the AF operation takes a certain amount of time. In order to complete the AF operation quickly, Patent Document 1 discloses a technology in which the imaging element completes the process from reading all pixels of the captured image to generating a lens drive command and transmitting it to the lens. By generating the lens drive commands without going through the image engine, AF operations can be completed faster than with conventional technology.

[0003] Incidentally, sensors equipped with AI have been developed in recent years. Patent Document 2 discloses an imaging element that receives light with a photodiode and has an image recognition function within the imaging element that reads out an image signal. The way in which the image signal is read can also be changed; for example, recognition can be performed with an image output by thinning out the image to 1 / 4 pixels, so that if recognition is possible with an image of 1 / 4 pixels, the image recognition result can be output before the completion of reading all pixels. For example, if recognition is not possible with 1 / 4 pixels, 2 / 4 pixels, or 3 / 4 pixels, the recognition result is output after reading all pixels (4 / 4 pixels). There is also a technology that incorporates a scene recognition unit with a learning function within the imaging element. By controlling the driving of the imaging unit based on the scene, it is possible to detect a subject without waiting for image processing on the image engine side. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2017-98903 A [Patent Document 2] Patent No. 6635221 Summary of the Invention [Problem to be solved by the invention]

[0005] In Patent Document 1, for example, when a subject targeted by a user moves at high speed into the frame of a fixed imaging device, the operation of setting an AF area for the subject may not be completed in time. In Patent Document 2, the current shooting scene is recognized in advance by a learning function installed in the imaging element, and the target subject itself can be detected at high speed, but there is no mention of a technology for performing AF operation within the sensor. In addition, in both Patent Documents 1 and 2, image correction such as distortion correction, which has been performed on the camera side until now, cannot be performed, so that it may be difficult to detect the subject when it is in the peripheral area.

[0006] An object of the present invention is to enable high-speed and high-precision focus adjustment. [Means for solving the problem]

[0007] In order to solve the above problems, an imaging device of the present invention is an imaging device having an imaging element and an image processing device, and has a switching means for switching control information to be output for use in driving a focus lens between first control information for controlling the driving of the focus lens generated by the imaging element and second control information for controlling the driving of the focus lens generated by the image processing device. the control information includes information indicating a drive amount of the focus lens, the image sensor has a processor different from the image processing device, and generates the first control information by using a result of subject detection inside the image sensor, without using a result of subject detection in the image processing device. . Effect of the Invention

[0008] According to the present invention, high speed and high precision focus adjustment is possible. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating a configuration of an imaging device according to a first embodiment. [Diagram 2] FIG. 1 is a diagram illustrating image recognition. [Diagram 3] FIG. 13 is a diagram illustrating thinning readout. [Figure 4] FIG. 1 is a diagram illustrating the timing of conventional AF processing. [Diagram 5] 4A to 4C are diagrams illustrating the timing of AF processing in the present embodiment. [Figure 6] 5 is a flowchart showing a process relating to AF in the image sensor. [Figure 7] 13 is a flowchart showing a process of a control signal switching unit. [Figure 8] FIG. 13 is a diagram illustrating a configuration of an imaging device according to a second embodiment. [Figure 9] FIG. 13 is a diagram illustrating a configuration of an imaging device according to a third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] (First embodiment) 1 is a block diagram showing an example of the configuration of an imaging device 100. The imaging device 100 is, for example, an imaging device such as a digital single-lens reflex camera, a mirrorless camera, a video camera, or a compact digital camera. The imaging device 100 has a lens unit 101 and a camera unit 110. The imaging device 100 may be an imaging device in which the lens unit 101 and the camera unit 110 are integrated, or may be an interchangeable lens imaging device in which the lens unit 101 is detachable from the camera unit 110.

[0011] First, the configuration of the lens unit 101 will be described. The lens unit 101 forms an optical image of a subject on the image sensor 130. The lens unit 101 has an imaging optical system including a plurality of lenses such as a focus lens 102. The lens unit 101 further has a position detection unit 103, a drive circuit 104, a motor 105, a lens control unit 106, and an IF (interface) 107. The focus lens 102 is a lens whose focal length can be changed, and moves in the direction of the optical axis to adjust the focus.

[0012] The position detection unit 103 detects the position of the focus lens 102 and outputs it to the lens control unit 106. The drive circuit 104 and the motor 105 are drive units that drive the focus lens 102. The drive circuit 104 drives the motor 105 based on control information for driving the focus lens 102 to a focus position received from the lens control unit 106. The motor 105 moves the focus lens 102 in the optical axis direction of the imaging optical system. The motor 105 is, for example, a stepping motor. Note that the motor 105 is not limited to a stepping motor, and may be an ultrasonic motor or the like.

[0013] The lens control unit 106 controls the overall operation of the lens unit 101. The lens control unit 106 is, for example, an MPU (Micro Processing Unit), and is connected to the camera unit 110 via an IF 107 that is a lens mount unit. The lens control unit 106 outputs control information received from the camera unit 110 via the IF 107 to the drive circuit 104 for driving the focus lens to a focused position. Based on the control information from the lens control unit 106, the drive circuit 104 and the motor 105 drive the focus lens 102, thereby performing focus control.

[0014] Next, the configuration of camera unit 110 will be described. Camera unit 110 has image processing engine 120 and image sensor 130. In camera unit 110 of this embodiment, image processing engine 120 and image sensor 130 can each generate control information such as a relative drive amount and an absolute drive amount related to driving for driving focus lens 102 to a focus position. In the following, the drive information generated by image sensor 130 is referred to as first control information, and the drive information generated by image processing engine 120 is referred to as second drive information.

[0015] First, the image processing engine 120 will be described. The image processing engine 120 performs various image processes and a process of generating control information (second control information) for driving the focus lens 102 to a focus position. Furthermore, the image processing engine 120 performs a process of switching between the control information (first control information) generated by the image sensor 130 and the control information (second control information) generated by the image processing engine 120 as the control information used for driving the focus lens 102. The image processing engine 120 has an image processing unit 121, a recognition unit 122, a drive amount calculation unit 123, a control signal switching unit 124, and a control unit 125. The image processing engine 120 includes, for example, a CPU (Central Processing Unit) and a memory for storing a program. Each process by the image processing unit 121 to the control unit 125 of the image processing engine 120 is realized by the CPU executing a program stored in the memory.

[0016] The image processing unit 121 performs image processing such as image format conversion, resizing, color conversion, noise removal, distortion correction, chromatic aberration correction, and white balance adjustment on the image received from the imaging unit 131 of the imaging element 130. The image acquired by the image processing unit 121 from the imaging element 130 is an image acquired by reading out all pixels. The recognition unit 122 calculates the defocus amount of the subject from the image processed by the image processing unit 121. The recognition unit 122 calculates the defocus amount of the subject by, for example, detecting the subject and performing phase difference AF or contrast AF processing. The drive amount calculation unit 123 calculates the drive amount (relative drive amount) of the focus lens 102 required for focusing based on the defocus amount calculated by the recognition unit 122. Note that the calculation method is not limited as long as the drive amount of the focus lens 102 required for focusing can be calculated. For example, the recognition unit 122 does not need to be used. The drive amount calculation unit 123 of the present embodiment sends the calculated drive amount of the focus lens 102 to the control signal switching unit 124 .

[0017] Control signal switching unit 124 acquires control information and priority information of focus lens 102 calculated by image sensor 130, and drive information of focus lens 102 calculated by image processing engine 120. Then, control signal switching unit 124 sends only the drive amount used to drive focus lens 102, out of the two drive amounts, to control unit 125. Which drive amount is to be used for drive control of focus lens 102 is determined by drive amount calculation unit 133 in image sensor 130, which will be described later, and control signal switching unit 124 switches the drive amount to be output in accordance with the priority information determined by drive amount calculation unit 133.

[0018] The control unit 125 transmits the control information acquired from the control signal switching unit 124 to the lens unit 101 via the driver 112 and the IF 111. When the control information acquired from the control signal switching unit 124 is a relative drive amount generated by the image processing engine 120, the control unit 125 updates the control information based on the position information of the focus lens 102. The control unit 125 calculates an absolute drive amount based on the relative drive amount and the position information of the focus lens 102 acquired from the lens unit 101, updates the control information, and outputs it to the lens unit 101. The control unit 125 calculates an absolute drive amount of the focus lens 102 based on the relative drive amount of the focus lens 102 and the position information of the focus lens 102 acquired from the lens unit 101. The control information of the focus lens 102 input to the lens unit 101 is information that defines the drive of the motor 105 required to move the focus lens 102 to a focus position. For example, when the motor 105 is a stepping motor, the control unit 125 generates information that defines a drive target pulse number as the control information. Furthermore, when the control information is fixed byte length data, the control unit 125 transmits the control information to the driver 112 after the previous value transmission process is completed. In this manner, the image processing engine 120 acquires an image of all pixels from the imaging element 130, performs image processing in the image processing unit 121, and then calculates the drive amount for AF. Note that, although an example in which the image processing engine 120 acquires all pixels and performs processing for AF control has been described in this embodiment, it is also possible to acquire and process image signals of more pixels than the pixels used by the imaging element 130 for AF control, even if not all pixels.

[0019] Next, the imaging element 130 will be described. The imaging element 130 is an image sensor that has a photoelectric conversion unit and photoelectrically converts an optical image formed through the imaging optical system of the lens unit 101 to output a signal related to the captured image. Furthermore, the imaging element 130 of this embodiment is equipped with AI and performs image recognition processing such as subject detection and scene recognition, and calculation processing of the drive amount of the focus lens 102 for focusing. The imaging element 130 has an imaging unit 131, a recognition unit 132, and a drive amount calculation unit 133. The imaging element 130 includes, for example, a CPU (Central Processing Unit) and a memory that stores a program. Each process by the imaging unit 131 to the determination unit 134 of the imaging element 130 is realized by the CPU executing a program stored in the memory.

[0020] The imaging unit 131 captures an image of a subject (optical image) formed via an imaging optical system by photoelectric conversion, and outputs an image signal. The recognition unit 132 is equipped with a learning function, and performs image recognition such as subject detection and scene recognition on the output of the imaging unit 131. The recognition unit 132 then calculates a defocus amount of the subject based on the results of the subject detection and scene recognition by image recognition. The recognition unit 132 sends the calculated defocus amount to the drive amount calculation unit 133. The recognition unit 132 also sends the results of the image recognition such as subject detection and scene recognition to the determination unit 134.

[0021] The drive amount calculation unit 133 generates first control information for driving the focus lens 102 to a focus position. Specifically, the drive amount calculation unit 133 calculates a relative drive amount of the focus lens 102 required for focusing according to the defocus amount of the subject. Then, the drive amount calculation unit 133 calculates an absolute drive amount of the focus lens 102 based on the position information of the focus lens 102 acquired from the lens unit 101 and the calculated relative drive amount, and generates first drive information. The drive amount calculation unit 133 transmits the generated first drive information to the control signal switching unit 124 of the image processing engine 120 via the IF 135 and the IF 126.

[0022] The determination unit 134 determines whether the drive amount calculated by the image sensor 130 or the drive amount calculated by the image processing engine 120 should be prioritized, and generates priority information indicating the drive information to be prioritized. The determination unit 134 transmits the generated priority information to the control signal switching unit 124 of the image processing engine 120 via the IF 135 and the IF 126. The calculation of the defocus amount by the recognition unit 132 and the generation of the first control information by the drive amount calculation unit 133, and the generation of the priority information by the determination unit 134 are performed in parallel, and the first control information and the priority information are simultaneously sent from the image sensor 130 to the image processing engine 120. In this way, while the drive amount is calculated after image processing in the image processing engine 120, the image sensor 130 can calculate the control information without waiting for all pixels to be read out and without image processing, making it possible to calculate the control information at a higher speed.

[0023] Here, the timing of generating control information for focus lens 102 in image sensor 130 will be described with reference to Figs. 2 to 5. Image sensor 130 calculates a defocus amount in response to scene recognition and subject detection in recognition unit 132, and calculates control information for focus lens 102 based on the defocus amount. Fig. 2 is a diagram for explaining image recognition. Fig. 2 shows a case where a baseball scene 140 is about to be photographed. When recognition unit 132 of image sensor 130 recognizes a baseball scene, for example, recognition unit 132 calculates the defocus amount when it detects white ball 141.

[0024] Furthermore, by performing scene recognition, it is possible to detect a specific object (for example, a white ball in baseball) according to the recognized scene before reading all pixels, and the calculation of control information for focusing on a specific object can be accelerated. FIG. 3 is a diagram for explaining thinning readout. FIG. 3(A) shows an example of thinning out on a line-by-line basis. FIG. 3(B) shows an example of thinning out on a pixel-by-pixel basis. In FIGS. 3(A) and 3(B), the hatched portions indicate the pixels to be thinned out. Whether thinning out on a line-by-line basis as shown in FIG. 3(A) or thinning out the pixels to 1 / 4 as shown in FIG. 3(B), it is possible to detect a white ball 141, which is an object to be detected in a baseball scene 140.

[0025] FIG. 4 is a diagram for explaining the timing of the conventional AF process. In the case where all pixels are read as shown in FIG. 2, as shown in FIG. 4, after the start of accumulation, it is necessary to wait for the completion of the readout of all pixels, and then start AF control to obtain the position of the lens, complement the amount of lens movement, and determine the final lens position. FIG. 5 is a diagram for explaining the timing of the AF process of this embodiment. In the case where only some pixels are read out as shown in FIG. 3, if the subject is recognized at the stage where, for example, 1 / 4 pixels are read out as shown in FIG. 5, AF control can be started without waiting for the remaining 3 / 4 pixels to be read out. In this way, control information for AF control can be generated at the timing when the image signals of some pixels are read out, without synchronizing with the readout of the image signals. This makes it possible to complete the calculation of the final focus lens position earlier than in the past, regardless of whether the AF is phase difference AF or contrast AF. In this embodiment, the image processing engine 120 performs lens position interpolation (generation of first control information for the focus lens 102) after the readout of all pixels shown in FIG. 4. Meanwhile, the image sensor 130 performs lens position interpolation (generation of second control information for the focus lens 102) by reading out some pixels without waiting for the readout of all pixels shown in FIG. 5. Therefore, the generation of the first control information by the image capture element 130 is faster than the generation of the second control information by the image processing engine 120.

[0026] Furthermore, when the location where the subject will appear can be predicted by image recognition such as recognition of the shooting scene (hereinafter referred to as scene recognition), the generation of the first control information in the image sensor 130 can be further accelerated by narrowing down the area for subject detection and focus detection in advance. For example, in FIG. 2 and FIG. 3, the timing to focus is the moment of impact by the batter, and it can be predicted that the white ball 141, which is the subject, will appear at about waist height of the batter. When image plane phase difference AF is employed, the line for performing image plane phase difference detection is set only at the position where the subject is expected to appear, thereby speeding up the detection of the subject and the calculation of the defocus amount and the control information for the focus lens 102. Note that the detection of the subject and the calculation of the control information using the prediction of the subject position by image recognition may also be performed by the image processing engine 120.

[0027] Here, the determination by the determination unit 134 as to which of the first control information calculated in the image sensor 130 and the second control information calculated in the image processing engine 120 should be prioritized will be described. The calculation of the first control information of the focus lens 102 in the image sensor 130 can be performed at high speed because it can be performed without waiting for all pixels to be read out by image recognition. On the other hand, since image processing such as distortion aberration correction is not performed in the image sensor 130, the accuracy of scene recognition and subject detection may be lower than that of the image processing engine 120, and the accuracy of the control information and AF based thereon may also be lower. The influence of image processing such as distortion aberration correction not being performed in the image sensor 130 is particularly noticeable in the periphery of the screen where the image height is high, but on the other hand, the influence of not performing image processing is less noticeable near the center of the image height (center of the screen). The calculation of the second control information in the image processing engine 120 is performed after image processing by the image processing unit 121, so the accuracy of scene recognition and subject detection is high, and the accuracy of calculation of the lens driving amount and AF based thereon are also high. On the other hand, since image processing is performed after all pixels are read out, and then subject detection and defocus amount calculation are performed, calculation of the drive amount in the image processing engine 120 is slower than calculation of the drive amount in the image sensor 130.

[0028] Therefore, the determination unit 134 of the image sensor 130 determines which control information is to be prioritized based on the scene recognition detected by the recognition unit 132 and the position and movement (movement speed and movement direction) of the subject. When the reliability of the drive amount calculated in the image sensor 130 is high, the determination unit 134 notifies the control signal switching unit 124 of priority information indicating that the calculated value in the image sensor 130 is to be prioritized. For example, when the subject is in the center of the screen, the captured image is less susceptible to the influence of lens distortion and a good detection result can be obtained without going through the distortion correction calculation by the image processing unit 121, so the first control information calculated in the image sensor 130 is prioritized. When the subject moves randomly or when a stationary subject suddenly moves, and it is difficult to predict the focus position even by the predictive calculation process of the image processing engine 120 (when the reliability of the predictive calculation process is low), the first control information calculated in the image sensor 130 is also prioritized. On the other hand, when the subject is in the periphery of the image, when the moving speed of the subject is less than a predetermined value and moving in a fixed direction, etc., the control signal switching unit 124 is notified of priority information indicating that the second control information calculated in the image processing engine 120 should be prioritized. In addition, in the case of a scene that requires high accuracy in subject detection, such as a dark scene, the second control information is prioritized. Note that the judgment criteria are not limited to these, and are accumulated by the learning function of the AI ​​of the image sensor 130. In addition, it is possible to allow the user to set which drive amount is to be prioritized, and the prioritized drive amount may be determined based on the user's setting. When the user sets the AF speed to be prioritized, the control information calculated in the image sensor 130 is prioritized.

[0029] The image sensor 130 transmits the first control information and priority information indicating which control information of the image sensor 130 or the image processing engine 120 should be prioritized to the control signal switching unit 124 in the image processing engine 120. The control signal switching unit 124 switches the control information to be sent to the control unit 125 based on the priority information acquired from the image sensor 130. When the control signal switching unit 124 acquires information giving priority to the control information calculated by the image processing engine 120, the control signal switching unit 124 transmits the second control information calculated by the image processing engine 120 to the control unit 125. On the other hand, when the control signal switching unit 124 acquires information giving priority to the drive amount calculated by the image sensor 130, the control signal switching unit 124 transmits the first control information calculated by the image sensor 130 to the control unit 125. The switching of the control information by the control signal switching unit 124 is performed without synchronizing with the image update period, i.e., the period of reading all pixels by the image sensor 130.

[0030] The control unit 125, which has acquired the second control information calculated by the image processing engine 120, calculates an absolute drive amount based on the lens position information and the relative drive amount, updates the second control information, and sends it to the driver 112. On the other hand, the control unit, which has acquired the first control information calculated by the image sensor 130, sends the control information as is to the driver 112. The control information sent to the driver 112 is sent to the lens control unit 106 via the IF 111 on the camera unit 110 side and the IF 107 on the lens unit 101 side. The control signal switching unit 124 prevents the control information on the image sensor 130 side and the control information on the image processing engine 120 side from being sent to the lens unit 101 in a mixed state.

[0031] The flow of processing related to AF in the image sensor 130 will be described. FIG. 6 is a flowchart showing the flow of processing in the image sensor 130 related to AF. In step S601, the image capture unit 131 reads out image signals. In the image sensor 130, the processing of the next step S602 is performed when the reading of the thinned pixels is completed without waiting for the completion of the reading of all pixels. In steps S602 and S603, the recognition unit 132 performs image recognition. Specifically, in step S602, the recognition unit 132 performs scene recognition according to a learning model. If the scene is recognized, the recognition unit 132 predicts the subject position and limits the area in which the subject detection is performed in step S603. In step S603, the recognition unit 132 performs subject detection. The result of the scene recognition, which is the result of the image recognition, and the detection result of the subject are output to the drive amount calculation unit 133 and the determination unit 134.

[0032] In steps S604 to S606, the drive amount calculation unit 133 generates first control information. Specifically, in step S604, the drive amount calculation unit 133 acquires position information of the focus lens 102 from the lens unit 101. In step S605, the recognition unit 132 and the drive amount calculation unit 133 calculate a relative drive amount for driving the focus lens 102 to a focus position. First, the recognition unit 132 calculates a defocus amount, and the drive amount calculation unit 133 calculates a relative drive amount based on the defocus amount. Note that the calculation of the defocus amount by the recognition unit 132 may be performed in step S602 where the image recognition process is performed. In step S606, an absolute drive amount is calculated based on the position information of the focus lens 102 acquired in step S604 and step S603, and the first control information is generated.

[0033] In parallel with the processes of steps S604 to S606 for generating the first control information, the imaging element 130 performs a process for determining which control information is to be prioritized. In step S607, the determination unit 134 determines which of the first control information generated by the imaging element 130 and the second control information generated by the image processing engine 120 is to be prioritized, and generates priority information. The imaging element 130 then outputs the first control information generated in step S606 and the priority information generated in step S607 to the control signal switching unit 124 of the image processing engine 120.

[0034] Next, the flow of switching processing by the control signal switching unit 124 of the image processing engine 120 will be described. Fig. 7 is a flowchart showing processing by the control signal switching unit 124. In step S701, the control signal switching unit 124 determines whether to prioritize the first control information or the second control information based on the priority information acquired from the image sensor 130. If it is determined that the first control information generated by the image sensor 130 is to be prioritized, the process proceeds to step S702. On the other hand, if it is determined that the second control information generated by the image processing engine 120 is to be prioritized, the process proceeds to step S703.

[0035] In step S702, the control signal switching unit 124 sets the value for controlling the focus lens 102 to the in-focus position, which is output to the lens unit 101, as the first control information generated by the image sensor 130. In step S703, the control signal switching unit 124 sets the value for controlling the focus lens 102 to the in-focus position, which is output to the lens unit 101, as the second control information generated by the image processing engine 120. In step S704, the control signal switching unit 124 determines whether the output of the previous control information is complete. If it is determined that the output of the previous control information is complete, the process proceeds to step S705. On the other hand, if it is determined that the output of the previous control information is not complete, the process of step S704 is repeated until the output is complete. By the process of step S704, the control information can be switched after the transmission of the previous control information is completed. In step S705, the control signal switching unit 124 outputs the control information determined in step S702 or step S703 to the driver 112. The control information output from the control signal switching unit 124 is input to the lens control unit 106 via the driver 112, the IF 111 on the camera unit 110 side, and the IF 102 on the lens unit 101 side.

[0036] As described above, according to this embodiment, it is possible to switch the control information for driving the focus lens to a focus position depending on the scene and subject, thereby achieving high-speed, highly accurate AF operation.

[0037] Second embodiment In the first embodiment, an example has been described in which the absolute drive amount based on position information of the focus lens 102 is calculated by both the image sensor 130 and the image processing engine 120. That is, the absolute drive amount corresponding to the first control information is calculated within the image sensor 130. In the second embodiment, an example will be described in which the image sensor 130 calculates even the relative drive amount for the first control information, and the image processing engine 120 calculates the absolute drive amount. Note that the calculation of the relative drive amount and absolute drive amount related to the second control information is similar to the first embodiment.

[0038] Only the differences from the first embodiment will be described below. FIG. 8 is a diagram showing the configuration of the imaging device 100 of the second embodiment. The same components as those in the first embodiment are given the same reference numerals and their description will be omitted. In the first embodiment, the position information of the focus lens 102 acquired from the lens unit 101 is input to both the imaging element 130 and the image processing engine 120, but in the second embodiment, it is input only to the image processing engine 120. The camera-side IF 800 outputs the position information of the focus lens 102 acquired from the lens control unit 106 via the lens-side IF 107 only to the control unit 803 of the image processing engine 120, and does not output it to the imaging element 130. This makes it possible to reduce the physical wiring between the camera-side IF 800 and the imaging element 130.

[0039] Drive amount calculation unit 801 of image sensor 130 does not acquire position information of focus lens 102. Therefore, drive amount calculation unit 801 calculates up to a relative drive amount for driving focus lens 102 to a focus position based on the defocus amount calculated by recognition unit 132, and generates first control information. The first control information, which is the relative drive amount generated by drive amount calculation unit 801, is output to control signal switching unit 802 of image processing engine 120 together with priority information determined by determination unit 134.

[0040] The control signal switching unit 802 of the image processing engine 120 switches the control information to be output to the lens unit 101 based on the priority information. At this time, the control information output by the control signal switching unit 802 to the control unit 803 is a relative drive amount whether it is the first control information or the second control information. The control unit 803 calculates an absolute drive amount of the focus lens 102 based on the relative drive amount acquired from the control signal switching unit 802 and the position information of the focus lens 102 acquired from the lens unit 101. Then, the control unit 803 outputs the calculated absolute drive amount to the lens unit 101 via the driver 112 and the IF 800.

[0041] In this manner, in this embodiment, it is possible to switch control information for driving the focus lens to a focus position according to the scene or subject, and to realize high-speed and highly accurate AF operation. Furthermore, since the image sensor 130 does not receive position information of the focus lens 102 from the lens unit 101, it is possible to reduce the amount of physical wiring between the lens unit 101 and the image sensor 130.

[0042] Third embodiment In the first and second embodiments, an example has been described in which the process of switching the control information of focus lens 102 to be output to lens unit 101 is performed within image processing engine 120. In the third embodiment, an example will be described in which the process of switching the control information of focus lens 102 to be output to lens unit 101 is performed within image sensor 130.

[0043] Only the differences from the first embodiment will be described below. FIG. 9 is a diagram showing the configuration of the imaging device 100 of the third embodiment. The same components as those in the first embodiment are given the same reference numerals and the description thereof will be omitted. The imaging element 930 has an imaging unit 131, a recognition unit 132, a drive amount calculation unit 901, a control signal switching unit 902, a control unit 903, an IF 135 with the image processing engine 120, and an IF 904 with the driver. The image processing engine 120 has an image processing unit 121, a recognition unit 122, and a drive amount calculation unit 905. Thus, in this embodiment, the imaging element 930 has a control signal switching unit 902 that switches the control information to be output to the lens unit 101.

[0044] In the first embodiment, the position information of the focus lens 102 acquired from the lens unit 101 is input to both the image sensor 130 and the image processing engine 120, but in the third embodiment, the position information is input only to the image sensor 930. The drive amount calculation unit 901 of the image sensor 930 calculates an absolute drive amount based on the relative drive amount calculated from the defocus amount and the position information of the focus lens 102 acquired from the lens unit 101, and generates first control information for driving the focus lens 102. The drive amount calculation unit 901 then outputs the generated first control information to the control signal switching unit 902 in the image sensor 930. The drive amount calculation unit 905 of the image processing engine 920 calculates a relative drive amount from the defocus amount, and generates second control information for driving the focus lens 102. The drive amount calculation unit 905 then outputs the generated second control information to the control signal switching unit 902 in the image sensor 930.

[0045] The control signal switching unit 902 acquires first control information generated by the image sensor 930 from the drive amount calculation unit 901, and second control information generated by the image processing engine 920 from the drive amount calculation unit 905 of the image processing engine 920. The control signal switching unit 902 decides which of the first control information generated by the image sensor 930 and the second control information generated by the image processing engine 920 is to be prioritized, and switches the control information to be output in accordance with the decision. In other words, the control signal switching unit 902 has the functions of the determination unit 134 and the control signal switching unit 124 in the first embodiment.

[0046] The control unit 903 outputs the control information input from the control signal switching unit 902 to the lens unit 101 via the IF 904, the driver 112, and the IF 900. When the control information input from the control signal switching unit 902 is the first control information, the control unit 903 outputs the first control information to the driver 112 as it is because the absolute drive amount has already been calculated. On the other hand, when the control information input from the control signal switching unit 902 is the second control information, the control unit 903 updates the second control information based on the position information of the focus lens 102 acquired from the lens unit 101. The control unit 903 calculates the absolute drive amount of the focus lens 102 based on the relative drive amount of the second control information and the position information of the focus lens 102, updates the second control information, and outputs it to the driver 112. In the present embodiment, an example has been described in which the drive amount calculation unit 901 calculates the absolute drive amount related to the first control information. However, the drive amount calculation unit 901 may be limited to calculating a relative drive amount, and the control unit 903 may calculate the absolute drive amount related to the first control information.

[0047] In this way, in this embodiment, compared to the first embodiment, it is possible to reduce the amount of physical wiring between the IF 900 and the image processing engine 920. On the other hand, an IF 904 for connecting to the driver 112 is required in the image sensor 930.

[0048] In this manner, in this embodiment, it is possible to switch control information for driving the focus lens to a focus position according to the scene or subject, thereby achieving high-speed and highly accurate AF operation. Furthermore, by not receiving position information of the focus lens 102 from the lens unit 101 in the image processing engine 920, it is possible to reduce the amount of physical wiring between the lens unit 101 and the image processing engine 920.

[0049] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0050] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. [Explanation of symbols]

[0051] 100 Imaging device 110 Camera section 120 Image Processing Engine 121 Image processing section 122 Recognition part 123 Drive amount calculation unit 124 Control signal switching unit 125 Control Unit 130 Image sensor 131 Imaging unit 132 Recognition part 133 Drive amount calculation unit

Claims

1. An imaging device having an imaging element and an image processing device, a switching means for switching control information to be output for use in driving a focus lens between first control information for controlling the driving of the focus lens generated by the image sensor and second control information for controlling the driving of the focus lens generated by the image processing device; the control information includes information indicating a drive amount of the focus lens, The imaging element has a processor different from the image processing device, and generates the first control information by using a result of subject detection inside the imaging device, without using a result of subject detection in the image processing device.

1. An imaging device comprising:

2. 2 . The imaging device according to claim 1 , wherein the imaging element generates the first control information earlier than the image processing device generates the second control information.

3. 3. The imaging apparatus according to claim 1, wherein the imaging element generates the first control information asynchronously with readout of an image signal and outputs the first control information to the switching means.

4. 4. The imaging apparatus according to claim 1, wherein the imaging element generates the first control information and outputs the first control information to the switching means before image signals from all pixels are read out.

5. The imaging device according to any one of claims 1 to 4, characterized in that the imaging element generates the first control information based on image signals of some pixels, and the image processing device generates the second control information based on image signals of all pixels that have been subjected to image processing.

6. The imaging element includes: an imaging means for capturing an image of a subject and outputting an image signal; A recognition means for performing image recognition based on the image signal; A generating unit that generates first control information for controlling driving of the focus lens; a determination means for determining priority control information based on a result of the image recognition and generating priority information indicating the priority control information; 4. The imaging apparatus according to claim 1, wherein the switching unit switches the control information to be output based on the priority information.

7. 7. The image pickup apparatus according to claim 6, wherein the determining means determines the control information to be prioritized based on either the position and movement of a subject, or the photographed scene.

8. 8. The imaging apparatus according to claim 6, wherein the recognition means predicts a position of the subject by the image recognition, and detects the subject and calculates a defocus amount based on the predicted position of the subject.

9. 9. The imaging apparatus according to claim 1, wherein the switching means switches the control information asynchronously with a period in which the imaging element reads out an image signal.

10. 10. The imaging apparatus according to claim 1, wherein the switching unit switches the control information to be output next after output of the control information is completed.

11. 11. The imaging apparatus according to claim 1, wherein the image processing device comprises the switching means.

12. 12. The imaging apparatus according to claim 11, wherein the imaging element and the image processing device each acquire position information of a focus lens, and generate control information for the focus lens based on the position information.

13. The imaging device described in claim 11, characterized in that the imaging element does not acquire position information of a focus lens, and the image processing device updates the first control information based on position information of a focus lens and the first control information acquired from the imaging element.

14. the imaging element includes the switching means and an interface capable of outputting the control information to the lens device, An imaging device as described in any one of claims 1 to 10, characterized in that the image processing device does not acquire position information of a focus lens, and in the imaging element, the second control information is updated based on position information of a focus lens and the second control information acquired from the image processing device.

15. A focus lens for adjusting the focus; a driving means for driving the focus lens; A control means for controlling the driving means, 15. The imaging apparatus according to claim 1, wherein the control means controls the drive means to drive the focus lens in accordance with the control information for the focus lens output from the switching means.

16. A method for controlling an imaging device having an imaging element and an image processing device, comprising: generating first control information for controlling driving of a focus lens in the imaging element; generating second control information for controlling driving of a focus lens by the image processing device; switching control information to be output for use in driving a focus lens between the first control information generated by the image sensor and the second control information generated by the image processing device; the control information includes information indicating a drive amount of the focus lens, A method for controlling an imaging device, characterized in that the imaging element has a processor different from the image processing device, and the first control information is generated using a result of subject detection within the imaging device, without using a result of subject detection in the image processing device.

Images