Information processing device, information processing method, and program

Abstract

To allow auto-focusing in an imaging apparatus using an event-based sensor, according to the present invention.SOLUTION: An information processing apparatus according to the present invention that solves the problem has: control means that controls the position of a focus lens of an imaging apparatus; and determination means that acquires an evaluation value based on an address event signal for every position of the focus lens to determine a focus position of the focus lens.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to autofocus for event-based sensors. [Background technology]

[0002] An event-based sensor is known that outputs a change in luminance of each pixel as an address event signal in real time (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-134271 Summary of the Invention [Problem to be solved by the invention]

[0004] The problem to be solved by the present invention is to enable autofocus in an imaging device using an event-based sensor. [Means for solving the problem]

[0005] The information processing device according to the present invention for solving the above problem includes a control means for controlling the position of a focus lens of an imaging device having a photoelectric conversion element, an acquisition means for acquiring an address event signal output by detecting a change in luminance of each pixel of the photoelectric conversion element that is equal to or greater than a second threshold value, and a At each of the multiple positions an evaluation means for acquiring the second threshold value at which the number of pixels to which the address event signal is output is less than a third threshold value as an evaluation value at the position of the focus lens; and a determination means for determining a focus position of the focus lens based on the evaluation value acquired by the evaluation means. wherein the control means executes control to wobble the focus lens at a constant amplitude at each of a plurality of positions of the focus lens, the evaluation means increases the second threshold at a predetermined rate of change during the wobbling control operation, and acquires, as the evaluation value, the second threshold at which the number of pixels to which the address event signal is output at each of the plurality of positions of the focus lens is less than the third threshold, and the determination means determines a focus position of the focus lens based on the evaluation value acquired at each of the plurality of positions of the focus lens. It is characterized by: [Effects of the Invention]

[0006] According to the present invention, autofocusing becomes possible in an imaging device that uses an event-based sensor. [Brief explanation of the drawings]

[0007] [Figure 1] A block diagram showing an example of the hardware configuration of an information processing device. [Figure 2] A diagram showing an example of the configuration of an event-based sensor [Figure 3] A block diagram showing an example of the functional configuration of an information processing device. [Figure 4] A diagram showing an example of a subject [Figure 5] A diagram showing an example of a focus method [Figure 6] 10 is a flowchart illustrating a process executed by an information processing device. [Figure 7] A diagram showing an example of focus lens and threshold control. [Figure 8] A diagram showing an example of a display DETAILED DESCRIPTION OF THE INVENTION

[0008] There are high expectations for the application of event-based sensors to cameras for security applications. Security cameras sometimes shoot in low-light environments, such as during nighttime surveillance. In such scenes, the aperture value is often set to a small value to increase the amount of light received by the sensor. Setting a small aperture value results in a shallow depth of field, so it is often necessary to adjust the focus on the subject. For this reason, an autofocus (AF) function that allows users to easily adjust the focus is desirable.

[0009] Contrast AF is a well-known autofocus technology, but it is difficult to apply to cameras that use event-based sensors for the following reasons. Contrast AF determines the in-focus position based on a contrast evaluation value that represents the contrast level within a specific area within the image (hereafter referred to as the AF evaluation frame). In other words, the in-focus position is determined to be when the contrast evaluation value reaches its peak. Typical methods for calculating the contrast evaluation value include using the ratio of the maximum and minimum brightness within the AF evaluation frame, or using the difference between the maximum and minimum brightness.

[0010] However, with event-based sensors, changes in pixel brightness can only be expressed as three values (for example, +1 for a positive change in brightness, -1 for a negative change in brightness, and 0 for no change in brightness). With event-based sensors, both positive and negative changes in brightness often occur, except when the light incident on the sensor changes uniformly, such as when the brightness of the lighting or the diameter of the aperture is changed. For this reason, even if a known formula for calculating contrast evaluation values is applied, the evaluation value will be the same regardless of whether the image is in focus, making it impossible to determine the in-focus position. Therefore, the problem that the present invention aims to solve is to enable autofocus in an imaging device that uses an event-based sensor.

[0011] Hereinafter, an imaging device according to an embodiment of the present invention will be described with reference to the drawings. In this regard, components having the same functions in all the drawings will be assigned the same numbers, and repeated description will be omitted.

[0012] <Embodiment 1> <Hardware configuration of the imaging device 100: Figure 1> FIG. 1 is a schematic diagram showing an example of the hardware configuration of an imaging device (information processing device) 100. While the imaging device 100 is specifically an imaging device having an event-based sensor, the image processing function and the image analysis function (motion detection) may be performed by separate devices. The imaging device 100 includes an imaging unit 101 consisting of an imaging optical system 1010 and a photoelectric conversion element 1011, a CPU 102, a memory 103, a display unit 104, and an operation unit 105. The photoelectric conversion element 1011 is an event-based sensor that outputs an address event signal in response to received incident light. The event-based sensor detects a change in luminance for each pixel as an event, and the address event signal indicates the position and time of the pixel where the luminance change occurred. The imaging optical system 1010 is specifically a light-receiving lens that receives incident light and forms an image on the photoelectric conversion element 1011. The CPU 102 reads and executes the OS and other programs stored in the memory 103, controls each connected component, and performs calculations and logical judgments for various processes. The processes executed by the CPU 102 include the information processing according to this embodiment. The CPU 102 also controls the focus and aperture of the imaging optical system 1010, the photoelectric conversion element 1011, and other functions. The memory 103 is, for example, a hard disk drive or an external storage device, and stores programs and various data related to the information processing according to this embodiment. The display unit 104 is, for example, a display device that displays the results of calculations performed by the information processing device 100 in accordance with instructions from the CPU 102. The display device may be any type, such as a liquid crystal display device, a projector, or an LED indicator. The operation unit 105 is, for example, a touch panel, keyboard, mouse, or robot controller, and serves as a user interface that accepts input instructions from a user. The information processing device 100 may also have mechanisms other than the hardware configurations listed here.

[0013] <Photoelectric conversion element: Figure 2> An example of an event-based sensor according to this embodiment will be described. The event-based sensor counts the number of incident photons and determines the timing when the counted number of photons exceeds a predetermined threshold. The event-based sensor also measures the time (clock count) required for the number of photons to reach or exceed a first threshold, and detects a change in luminance by comparing the required times. Specifically, when the previously measured required time is T0 and the latest required time is T, if the difference T-T0 is greater than or equal to a second threshold, a negative change in luminance is detected. If the difference T0-T is greater than or equal to the second threshold, a positive change in luminance is detected. If the difference between T and T0 is less than the second threshold, no change in luminance is detected. The second threshold is a value greater than or equal to zero, and is set in accordance with a preset value or other parameters.

[0014] The detailed configuration will be described below. Fig. 2(a) is a diagram showing an example configuration of a photoelectric conversion element 1011. The photoelectric conversion element 1011 is composed of a pixel unit 110 and a peripheral circuit 120. The peripheral circuit 120 includes a vertical arbitration circuit 121 and a horizontal readout circuit 122.

[0015] FIG. 2(b) illustrates an example of the configuration of each pixel unit constituting an event-based sensor. The pixel unit 110 includes a photoelectric conversion unit 111, a pixel counter 112, a time counter 113, a first decision circuit 114, a memory 115, a comparator 116, a second decision circuit 117, a response circuit 118, and a selection circuit 119. The photoelectric conversion unit 111 includes an avalanche photodiode (SPAD) operating in Geiger mode, and the pixel counter 112 counts the number of photons incident on the photoelectric conversion unit 111. The time counter 113 counts the time during which a photon is incident on the photoelectric conversion unit 111. Using a SPAD to configure an event-based sensor enables detection of luminance changes at the level of a single photon. By detecting luminance changes at the level of a single photon, an address event signal can be acquired even in night vision conditions, such as at night.

[0016] When the number of photons counted by pixel counter 112 reaches a first threshold, first decision circuit 114 stops counting time by time counter 113. Past count values of time counter 113 are stored in memory 115, and comparator 116 is used to determine the difference between the current count value of time counter 113 and the past count value of time counter 113.

[0017] If the difference count value is equal to or greater than the second threshold, the second determination circuit 117 sends a request signal to the vertical arbitration circuit 121 via the response circuit 118. The response circuit 118 receives a response from the vertical arbitration circuit 121 indicating whether or not the output of address event data is permitted. If the difference count value is less than the second threshold, the response circuit 118 does not send a request signal.

[0018] When the response circuit 118 receives a response indicating permission to output, the selection circuit 119 outputs the count value of the time counter circuit 113 to the horizontal output circuit 122. The horizontal output circuit 122 outputs the received count value as an output signal from the photoelectric conversion element 1011 to the detection unit 103.

[0019] Because the differential count value calculated by the comparator 116 corresponds to the reciprocal of the incident frequency of photons, the photoelectric conversion element 1011 according to this embodiment has the function of measuring "changes in the incident frequency of photons," i.e., changes in luminance. Furthermore, using the second determination circuit 117, an address event is output only when the difference in the intervals at which the number of incident photons reaches the first threshold is equal to or greater than the second threshold. That is, the photoelectric conversion element outputs the incident frequency when the difference in the incident frequency is equal to or greater than the second threshold, and does not output the incident frequency when the difference is less than the threshold. This configuration realizes an asynchronous photoelectric conversion element that detects changes in luminance as address events in real time for each pixel address.

[0020] <Variations of photoelectric conversion elements> The above describes a case where a photoelectric conversion element is used that uses a SPAD as the photoelectric conversion unit and measures the time at which a photon is incident to detect changes in the frequency of photon incidence. However, the configuration shown in Figure 2 is not necessary as long as the photoelectric conversion element detects changes in luminance as address events in real time. For example, as described in Patent Document 1, a photoelectric conversion element that detects changes in luminance as voltage changes may be used.

[0021] <Image capture device 100: FIG. 3> An example of the functional configuration of an imaging device (information processing device) 100 according to this embodiment is shown in Fig. 3. First, an overall overview will be described, followed by details of each function. In Fig. 3, the imaging device 100 has an imaging unit 301, an evaluation unit 302, a control unit 303, an output unit 304, and an operation reception unit 305. The imaging unit 301 has an imaging optical system 1011 equipped with a focus control mechanism 1012 capable of changing the focus position, and a photoelectric conversion element 1011 which is an event-based sensor.

[0022] Specifically, the imaging optical system 1011 is composed of a light receiving lens, and the focus control mechanism 1012 is composed of a focus lens and an actuator for driving the lens. In this embodiment, it is composed of a stepping motor that can control the position of the focus lens using drive pulses. The imaging optical system 1011 and focus control mechanism 1012 form an object image on the image plane of the photoelectric conversion element 1011. The photoelectric conversion element 1011 outputs an address event signal corresponding to the incident light it receives. More specifically, it outputs an address event signal indicating the address and time of a pixel where a change in luminance occurred. The evaluation unit 302 acquires the address event signal output from the photoelectric conversion element 1011 and compares the integrated value of the total number of pixels where a change in luminance occurred over a predetermined period with a predetermined threshold (third threshold). The control unit 303 causes the focus lens to wobble (vibrate back and forth with a small amplitude along the optical axis) via the focus control mechanism 1012. Furthermore, simultaneously with the wobbling operation, the control unit 303 causes the photoelectric conversion element 1011 to change (sweep) the second threshold at a predetermined rate. The second threshold is a threshold used by the second determination circuit 117 to determine whether there is a sufficient difference in the time intervals at which the number of incident photons reaches the first threshold. If there is a time difference equal to or greater than the second threshold, an address event signal is output. At this time, the evaluation unit 302 changes the number of pixels at which a luminance change occurs as the second threshold changes. The value of the second threshold when the number of pixels at which a luminance change occurs falls below the third threshold is set as the evaluation value for that focus lens position. Once the evaluation unit 302 determines the evaluation value, the control unit 303 moves the focus lens a predetermined distance in the optical axis direction and then performs a wobbling operation again. Subsequently, the control unit 303 repeats the above process to determine the evaluation value for that focus lens position. Finally, an evaluation value for each focus lens position is determined, and the lens position with the highest evaluation value is determined as the in-focus position. The output unit 304 displays the address of the pixel where a luminance change occurred, the direction of the luminance change, the AF evaluation frame, etc. The operation reception unit 305 receives, through user operation, the setting of the AF evaluation frame, input of a threshold value for an address event (described later), etc. Note that the output unit 304 and the operation reception unit 305 may be realized by devices external to the imaging device.

[0023] Each function will be explained in detail below.

[0024] <Evaluation unit 302> The details of the evaluation unit 302 in FIG. 3 will be described. First, the control unit 303 wobbles the focus lens while changing the second threshold of the photoelectric conversion element 1011 at a predetermined rate. During this time, the evaluation unit 302 compares the total number of pixels in which a luminance change occurs over a specific time range from the output signal of the photoelectric conversion element 1011 with the third threshold. The second threshold value when this total number of pixels falls below the third threshold is set as the evaluation value for that focus lens position. Here, the third threshold is a value greater than 0, and is set based on a preset value or other parameters. The specific time range may be the minimum time width determined by the time resolution of the vertical arbitration circuit 121 of the photoelectric conversion element 1011, or the integration may be performed over a longer time range. A narrower time width improves focusing speed, but a wider time width reduces the influence of errors due to random noise and the like, thereby improving focusing accuracy.

[0025] The target area (AF evaluation frame) in which the evaluation unit 302 counts the total number of pixels in which a luminance change has occurred may be the entire shooting range, a preset range, or a range specified by the user via the operation reception unit 305.

[0026] <Control unit 303> While the evaluation unit 302 is calculating the evaluation value, the control unit 303 wobbles the focus lens and changes (sweeps) the second threshold of the photoelectric conversion element 1011 at a predetermined rate of change. If the focus lens is stationary, the subject image does not change and nothing appears in the shooting data of the photoelectric conversion element 1011. However, by performing a wobbling operation, the focus of the subject image constantly changes, so the subject is always displayed in the shot image.

[0027] Next, we will explain the luminance change that occurs when the second threshold is swept. FIG. 4 is a diagram showing a subject and a background. The circle 400 in the figure represents the subject, which has a higher luminance than the background. Next, FIG. 5 is a diagram showing the subject image formed on the image plane of the photoelectric conversion element 1011 when the subject 400 in FIG. 4 is photographed by the imaging device 100, and the image captured by the photoelectric conversion element 1011, arranged for each focus lens position. The image captured by the photoelectric conversion element 1011 is shown in two cases: when the second threshold is high and when it is low. As shown in FIG. 5, the subject image is blurred except at position C (the in-focus position) of the focus lens. Therefore, the farther away from the in-focus range, the lower the luminance value of the subject image is, and the wider the range is than the actual subject. In this case, the luminance difference with the background becomes smaller. When these subject images are photographed by the photoelectric conversion element 1011, which is an event-based sensor, the contour of the subject is detected at every position if the second threshold is sufficiently low. On the other hand, if the second threshold is set too high, the subject will not be detected at positions other than position C, which is the in-focus position. In actual autofocus, the second threshold is swept while performing a wobbling operation at each lens position, and the value of the second threshold is used as the evaluation value when the number of pixels with detected brightness changes becomes equal to or less than the third threshold. This process is repeated for each lens position, and the position with the highest evaluation value is determined to be the in-focus position. Alternatively, a predetermined value (fourth threshold) may be prepared, and a position with an evaluation value exceeding the fourth threshold may be determined to be in the in-focus range. Here, the fourth threshold is a value greater than 0, and may be a preset value or a value set according to other parameters.

[0028] Next, a method for determining the lens movement direction in autofocus will be described.

[0029] Now, let's say autofocus is started at position B. After obtaining an evaluation value at position B, if you move to position A and obtain an evaluation value, the evaluation value at position A will be lower than the evaluation value at position B, so it can be inferred that the in-focus position is on the opposite side of position A. In this case, you can move the focus lens in the far direction.

[0030] Conversely, if autofocus is started at position E, and an evaluation value is obtained at position E, and then the lens is moved to position D and an evaluation value is obtained, the evaluation value will be higher. Therefore, it can be seen that the in-focus position exists in the NEAR direction. In this way, the increase or decrease in the evaluation value during the initial lens movement can be used to infer the direction in which the in-focus position exists, making it possible to speed up autofocus.

[0031] <Output unit 304> The output unit 304, specifically, is composed of an LCD display, projector, LED indicator, or the like, and receives the output signal from the event-based sensor 103 to display the address of the pixel where the luminance change occurred and the direction of the luminance change. The image captured by the event-based sensor shown in FIG. 5 is a display image of the output unit 304. The coordinates of the pixel where the luminance change occurred are displayed in white if the luminance change of the pixel was in a positive direction, black if the luminance change was in a negative direction, and gray if there was no luminance change. An AF evaluation frame or other display may be superimposed on the captured image to improve user convenience. Furthermore, during autofocus, a message indicating that autofocus is in progress may be superimposed on the captured image or displayed as a pop-up to alert the user. Furthermore, the values of the previous frame may be displayed outside the AF frame to help the user recognize the position of the AF frame relative to the background. Furthermore, the luminance change may be displayed in color other than black and white only within the AF frame (e.g., blue for positive luminance change and red for negative luminance change) to help the user recognize the luminance change within the AF frame.

[0032] <Operation Reception Unit 305> The operation reception unit 305 is a part where the user controls the imaging device 100. Specifically, it receives operations such as an instruction to start autofocus, setting the AF evaluation frame, and changing the second and third threshold values described above. The operation reception unit 305 is configured with, for example, a touch panel, a keyboard, a mouse, a cross key, an operation dial, etc.

[0033] The output unit 304 and operation reception unit 305 may be provided in an external device connected to the imaging device 100 via a network cable or wireless transmission.

[0034] <Flowchart> The flow of processing when performing the above-described autofocus will now be described. Fig. 6 is a flowchart illustrating the processing performed by the image capture device 100. In the following description, each process (step) will be denoted with an S at the beginning, and the process (step) will not be described in detail. However, the image capture device 100 does not necessarily have to perform all of the steps described in this flowchart.

[0035] In S601, the operation reception unit 305 sets the AF evaluation area (position and size) based on a user input. If there is no setting from the user, a preset AF evaluation frame is set.

[0036] In S602, the control unit 303 causes the evaluation unit 302 to acquire an evaluation value while sweeping the second threshold value of the photoelectric conversion element 1011 and performing a wobbling operation on the focus lens.

[0037] In S603, the control unit 303 moves the focus lens in a predetermined direction by a predetermined distance.

[0038] In S604, the evaluation unit 302 acquires an evaluation value in the same manner as in S602, and compares it with the evaluation value in S602. If the evaluation value in S604 is lower, the process proceeds to S605, and if the evaluation value in S604 is higher, the process proceeds to S606.

[0039] In S605, since the S604 evaluation value is lower than the S602 evaluation value, it is determined that the in-focus position is in the opposite direction, and the focus lens is moved in the opposite direction to S603.

[0040] In S606, since the S604 evaluation value is higher than the S602 evaluation value, it is determined that the in-focus position is in the same direction, and the focus lens is moved in the same direction as in S603.

[0041] In S607, an evaluation value is obtained. In both cases where the process goes from S605 to S607 and where the process goes from S606 to S607, the evaluation value in S607 should be higher than the evaluation value in S604.

[0042] In S608, the focus lens is moved in the same direction as in S605 and S606.

[0043] In S609, the evaluation unit 302 acquires an evaluation value. If the evaluation value in S609 is higher than the evaluation value in S607, the process returns to S608 and the focus lens is further moved in the same direction. If the evaluation value in S609 is lower than the evaluation value in S607, it can be determined that the previous lens position was the in-focus position, and so in S610 the lens position is returned to the previous position, completing autofocus.

[0044] <Embodiment 2: Wobbling Variations> In the second embodiment, the method of focus lens control by the control unit 303 when acquiring an evaluation value differs from that in the first embodiment. The method of the present embodiment has the advantage that the control can be simplified compared to the first embodiment because a wobbling operation is not performed. The hardware configuration and functional configuration of the imaging device 100 are the same as those in FIGS. 1 and 3 in the first embodiment.

[0045] This will be explained using FIG. 7. FIG. 7 shows the control method of the control unit 303 when acquiring an evaluation value. The horizontal axis represents the position of the focus lens. The vertical axis represents the value of the second threshold of the photoelectric conversion element 1011. In this embodiment, while the evaluation unit 302 calculates the evaluation value, the focus lens is not wobbled, and the lens is continuously moved in a fixed direction (from near to far in the figure). At this time, the second threshold is continuously swept at a speed sufficiently fast relative to the lens movement speed. In other words, the second threshold is swept from a low value to a high value within a time period in which the position of the focus lens can be considered to be approximately the same, and acquisition of the evaluation value is completed. By controlling in this manner, an evaluation value can be acquired for each position of the focus lens, and the position with the highest evaluation value can be determined to be the in-focus position. Furthermore, as in the first embodiment, the acquired evaluation value may be compared with a fourth threshold, and a position with an evaluation value exceeding the fourth threshold may be considered to be within the in-focus range.

[0046] By using the method of this embodiment, there is no need to perform complicated wobbling control, which has the advantage of making system design easier.

[0047] <Embodiment 3: GUI Variations> In the third embodiment, the operations of the control unit 303 and the output unit 304 when the user adjusts the focus by manual focus are different. The method of this embodiment allows the user to easily perform manual focus. The hardware configuration and functional configuration of the imaging device 100 are the same as those in FIGS. 1 and 3 of the first embodiment.

[0048] According to the techniques of the first and second embodiments, the user can focus on a subject by performing autofocus. However, in special circumstances, manual focus may be additionally performed. In FIG. 8(a), subjects 700 and 701 are present within the AF evaluation frame. Assume that subject 701 is located farther away than subject 700 and is the subject on which the user wishes to focus. If autofocus is performed for such a subject arrangement, the imaging device 300 cannot determine whether the focus should be on subject 700 or 701, and in some cases the focus may be pulled toward subject 701. In such cases, the user must perform manual focus after autofocus to focus on the desired subject.

[0049] Here, during autofocus, the user may be informed of what kinds of subjects (their brightness changes) were distributed within the AF frame and which subjects were in focus. For example, among the evaluation values obtained for each position of the focus lens, pixels where brightness changes occurred at the position with the second highest evaluation value are marked in a different color and displayed superimposed on the image captured at the in-focus position. In this way, the user can grasp the brightness changes at the in-focus position and the brightness changes at the position with the second highest evaluation value from the difference in display color, making it easier to determine whether the camera is in focus on a different subject than intended.

[0050] In this embodiment, when a user performs manual focusing, the user is prompted to select manual focus mode via the operation reception unit 305. At this time, as shown in FIG. 8(b), the AF evaluation frame used during autofocus is divided into multiple smaller AF evaluation frames. This division may be performed automatically so that the area is uniform, or it may be unevenly divided to match the shape of the subject. Alternatively, the user may set individual AF evaluation frames. When the camera enters manual focus mode, the control unit 303 continuously wobbles the focus lens and simultaneously sweeps the second threshold value of the photoelectric conversion element 1011, causing the evaluation unit 302 to acquire an evaluation value. The evaluation unit 302 acquires an evaluation value for each divided AF evaluation frame and, as shown in the figure, superimposes the evaluation values on the display screen. While FIG. 8(b) shows the evaluation value as a numerical superimposition, an indicator indicating the focus state may also be superimposed. As can be seen from FIG. 8(b), the evaluation value of the AF evaluation frame that broadly encompasses the subject 700 is 99, which is higher than the other areas, indicating that the subject 700 is in focus. If the evaluation value of the target subject is lower than that of other areas, simply move the focus lens manually while looking at the value displayed in the target AF evaluation frame. This method makes it easier for the user to recognize the focus state and the target, making manual focusing easier.

[0051] FIG. 8(c) shows an example of a display on the display unit 107 with further improvements to user visibility. In this figure, AF evaluation frames below a certain evaluation value are masked to prevent the user from seeing the subject. In this case, it is desirable to display only a numerical value superimposed on the subject, so that the user can recognize that a subject exists behind the masked area. Also, AF evaluation frames with evaluation values greater than a certain value may be displayed in a different color from the other AF evaluation frames, or the address event of the subject may be displayed in a different color from the other address events. Displaying the subject in this manner allows the user to better focus on the target subject, making manual focusing even easier.

[0052] As described above, the method of this embodiment enables autofocus in an image capture device using an event-based sensor, improving user convenience. In particular, by wobbling the lens or oscillating the parameters at high frequency, the quality of the evaluation value is improved, enabling highly accurate focusing.

[0053] The present invention can also be realized by executing the following process. That is, software (programs) that realize the functions of the above-described embodiments are supplied to a system or device via a data communication network or various storage media. Then, a computer (or CPU, MPU, etc.) of the system or device reads and executes the programs. The programs may also be provided by recording them on a computer-readable storage medium. [Explanation of symbols]

[0054] 100 Imaging device 101 Imaging optical system 102 Focus Lens 103 Photoelectric conversion element 104 Evaluation Department 105 Control Unit 106 Display section

Claims

1. a control means for controlling the position of a focus lens of an imaging device having a photoelectric conversion element; an acquisition means for acquiring an address event signal outputted by detecting a change in luminance equal to or greater than a second threshold value in each pixel of the photoelectric conversion element; an evaluation means for acquiring, at each of a plurality of positions of the focus lens, the second threshold at which the number of pixels to which the address event signal is output is less than a third threshold as an evaluation value at the position of the focus lens; a determination unit that determines a focus position of the focus lens based on the evaluation value acquired by the evaluation unit, the control means controls the focus lens to wobble at a constant amplitude at each of a plurality of positions of the focus lens, The evaluation means increasing the second threshold value at a predetermined rate of change during the operation of the wobbling control; acquiring, as the evaluation value, the second threshold value at which the number of pixels to which the address event signal is output at each of a plurality of positions of the focus lens is less than the third threshold value; the determining means determines a focus position of the focus lens based on the evaluation values acquired at each of a plurality of positions of the focus lens.

1. An information processing device comprising:

2. 2. The information processing device according to claim 1, wherein the determining unit determines, as the in-focus position, the position of the focus lens at which the second threshold value is maximum among the evaluation values acquired at each of a plurality of positions of the focus lens.

3. The information processing device according to claim 2, characterized in that the evaluation means acquires, as the evaluation value, the second threshold value at which the number of pixels to which the address event signal is output in a partial area included in the angle of view of the imaging device becomes the third threshold value.

4. 4. The information processing apparatus according to claim 3, wherein the evaluation means acquires the evaluation value for each of a plurality of partial regions included in the angle of view of the imaging device.

5. A control means for controlling the position of a focus lens of an imaging device equipped with a photoelectric conversion element; an acquisition means for acquiring an address event signal outputted by detecting a change in luminance equal to or greater than a second threshold value in each pixel of the photoelectric conversion element; an evaluation means for acquiring, as an evaluation value for a position of the focus lens, the second threshold at which the number of pixels to which the address event signal is output is less than a third threshold at each of a plurality of positions of the focus lens; a determination unit that determines a focus position of the focus lens based on the evaluation value acquired by the evaluation unit, the control means executes control to move the focus lens in a predetermined direction, The evaluation means vibrating the second threshold value with a constant amplitude at each of a plurality of positions during movement of the focus lens; acquiring, as the evaluation value, the second threshold value at which the number of pixels to which the address event signal is output becomes the third threshold value; The information processing apparatus is characterized in that the determination means determines a focus position of the focus lens based on the evaluation values obtained at a plurality of positions of the focus lens.

6. The information processing device described in Claim 5, characterized in that the determination means determines the position of the focus lens at which the second threshold value is maximum among the evaluation values obtained at each of multiple positions of the focus lens as the in-focus position.

7. 7. The information processing device according to claim 1, further comprising a display control means for causing a display device to display an image indicating the position of a pixel among the pixels of the photoelectric conversion element in which a change in luminance of equal to or greater than the second threshold has occurred based on the address event signal, and the evaluation value.

8. 8. The information processing apparatus according to claim 7, wherein the display control means displays an image generated based on the address event signal of the previous frame for the area for which the evaluation value is not acquired.

9. 9. The information processing device according to claim 7, wherein the display control means displays the positions of pixels from which the address event signal is output in different colors in an area from which the evaluation value is acquired and in other areas.

10. The information processing device according to any one of claims 7 to 9, characterized in that the display control means displays the position of the pixel to which the address event signal is output in a different color based on the evaluation value for each partial area obtained by dividing the angle of view of the imaging device.

11. The information processing device according to any one of claims 1 to 10, further comprising a switching means for switching between a first mode in which an autofocus operation is performed to move the focus lens to a position of the focus lens where the evaluation value has not been acquired, and a second mode in which the position of the focus lens is controlled to a position according to user input.

12. In the second mode, the control means causes the focus lens to perform a wobbling operation, The evaluation means varying the second threshold during the wobbling operation; 12. The information processing apparatus according to claim 11, wherein when the number of pixels in which a change in luminance occurs becomes equal to or smaller than the third threshold, the second threshold is acquired as the evaluation value.

13. 13. The information processing apparatus according to claim 1, wherein the photoelectric conversion element detects the change in luminance in each pixel as a change in voltage in the pixel.

14. The photoelectric conversion element is measuring a time until the number of incident photons at each pixel becomes equal to or greater than a first threshold; 14. The information processing apparatus according to claim 1, wherein a change in time until the brightness reaches or exceeds the first threshold is detected as the change in brightness.

15. 15. The information processing apparatus according to claim 14, wherein the second threshold value is a threshold value for the change in time.

16. A program for causing a computer to function as each of the means included in the information processing device according to any one of claims 1 to 15.

17. a control step of controlling the position of a focus lens of an imaging device including a photoelectric conversion element; an acquisition step of detecting a change in luminance equal to or greater than a second threshold value in each pixel of the photoelectric conversion element and acquiring an address event signal outputted from the detected change in luminance; an evaluation step of acquiring, as an evaluation value for a position of the focus lens, the second threshold at which the number of pixels to which the address event signal is output is less than a third threshold at each of a plurality of positions of the focus lens; a determination step of determining a focus position of the focus lens based on the evaluation value acquired in the evaluation step, In the control step, a control is performed to wobble the focus lens with a constant amplitude at each of a plurality of positions of the focus lens, In the evaluation step, increasing the second threshold value at a predetermined rate of change during the operation of the wobbling control; acquiring, as the evaluation value, the second threshold value at which the number of pixels to which the address event signal is output at each of a plurality of positions of the focus lens is less than the third threshold value; In the determination step, a focus position of the focus lens is determined based on the evaluation values acquired at each of a plurality of positions of the focus lens.

1. An information processing method comprising:

18. The information processing method described in Claim 17, characterized in that in the determination process, the position of the focus lens at which the second threshold value is maximum among the evaluation values obtained at each of multiple positions of the focus lens is determined as the in-focus position.

Images