Information processing device, information processing method, and program

Abstract

To suppress a situation in which a subject is not detected, when an event-based sensor is used.SOLUTION: An information processing device acquires an address event signal indicating the position and time of a pixel at which a change in luminance occurs. The information processing device includes detection means for detecting a subject on the basis of the address event signal, and generation means for generating a stimulus that can be sensed by the subject when the time period that the subject is not detected is longer than a predetermined time period.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to object detection using an event-based sensor. [Background technology]

[0002] An event-based sensor is known that outputs a change in luminance of each pixel as an address event signal in real time (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-134271 Summary of the Invention [Problem to be solved by the invention]

[0004] The problem to be solved by the present invention is to suppress a situation in which a subject is not detected when an event-based sensor is used. [Means for solving the problem]

[0005] The information processing device according to the present invention that solves the above problem is an information processing device that acquires an address event signal indicating the position and time of a pixel where a change in luminance has occurred, and includes: a detection means that detects an object based on the address event signal; a generation means that generates a stimulus that can be sensed by the object when the time during which the object is not detected is longer than a predetermined threshold; and a display control means that, when the stimulus is generated by the generation means, causes a display unit to display information indicating that the stimulus has been generated. wherein the generating means changes a direction of the stimulus when the subject is not detected by the detecting means even after the stimulus is given to the subject, and reduces the predetermined threshold when the subject is detected by the detecting means after the stimulus whose direction has been changed is given to the subject. It is characterized by: [Effects of the Invention]

[0006] According to the present invention, when an event-based sensor is used, it is possible to reduce situations in which a subject is not detected. [Brief explanation of the drawings]

[0007] [Figure 1] A block diagram showing an example of the hardware configuration of an information processing device. [Figure 2] A block diagram showing an example of the functional configuration of an information processing device. [Figure 3] A diagram showing an example of the configuration of an event-based sensor [Figure 4] 10 is a flowchart illustrating a process executed by an information processing device. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, an information processing apparatus according to an embodiment of the present invention will be described with reference to the drawings. In this case, components having the same functions in all the drawings will be assigned the same numbers, and repeated description thereof will be omitted.

[0009] The event-based sensor disclosed in Patent Document 1 controls the detection sensitivity for address events by controlling the time width of the dead zone for luminance changes according to the detection frequency of address events. However, when the event-based sensor disclosed in Patent Document 1 is used for high-speed detection of specific objects, the following problems arise: If the detection sensitivity for address events is reduced (the width of the dead zone is narrowed), luminance changes will not be detected, and the detection accuracy of the object will decrease. On the other hand, if the detection sensitivity for address events is increased (the width of the dead zone is widened), luminance changes caused by random noise such as photon shot noise will also be detected in addition to luminance changes of the object, and the detection accuracy of the object will decrease.

[0010] <Embodiment 1> <Information processing device: Figure 1> FIG. 1 shows an information processing device according to this embodiment. In FIG. 1, the information processing device 100 includes an imaging optical system 1010, an imaging unit 101 (event-based camera) including a photoelectric conversion element 1011, a CPU 102, a memory 103, a display unit 104, and an operation unit 105. The imaging unit 101 is a sensor using the photoelectric conversion element 1011, which outputs an address event signal in response to received incident light. The imaging unit 101 detects a change in luminance for each pixel as an event, and the address event signal indicates the position and time of the pixel where the luminance change occurred. The imaging optical system 1010 is specifically a light-receiving lens that receives incident light and forms an image on the photoelectric conversion element 1011. The CPU 102 reads and executes an OS and other programs stored in the memory 103, controls each connected component, and performs various processing operations and logical judgments. The processing executed by the CPU 102 includes the information processing according to this embodiment. The CPU 102 also controls the focus and aperture driving of the imaging optical system 1010, the photoelectric conversion element 1011, and the like. The memory 103 is, for example, a hard disk drive or an external storage device, and stores programs and various data related to the information processing of the embodiment. The display unit 104 outputs the calculation results of the information processing device 100 to a display device in accordance with instructions from the CPU 102. The display device may be of any type, such as a liquid crystal display device, a projector, or an LED indicator. The operation unit 105 is, for example, a touch panel, a keyboard, a mouse, or a robot controller, and is a user interface that accepts input instructions from a user. The information processing device 100 may have mechanisms other than the hardware configurations listed here.

[0011] <Example of functional configuration of information processing device: Figure 2> Next, an example of the functional configuration of an information processing device according to this embodiment will be described with reference to FIG. 2. The information processing device 100 includes an imaging unit 101, a detection unit 201, a stimulus control unit 202 that controls stimuli that can be sensed by a subject, and an image processing unit 203. Furthermore, the information processing device 100 is connected to an output device 200 that can output stimuli generated by the stimulus control unit 202. The information processing device 100 may also have the functions of the output device 200. Note that the information processing device according to this embodiment is only required to include at least the detection unit 201 and the stimulus control unit 202, but may also include other functions. Here, an overview of each function will be described. The information processing device 100 performs various information processing based on the output of a photoelectric conversion element 1011 that outputs an address event signal in response to received incident light. The detection unit 201 detects a subject within a photographic field of view based on the address event signal. As an example, assuming that the event-based sensor photographs a fixed field of view, a moving object within the photographic field of view is detected as the subject. Conversely, when the shooting parameters are fixed and there is no moving object, the address event signals decrease and the detection unit 202 detects nothing. A detailed description of the detection unit 201 will be given later. The stimulus control unit 202 generates a stimulus that can be sensed by the subject and controls the output device 200 to output the stimulus. Details of the stimulus and its control will be given later. Based on the address event signals, the image processing unit 203 generates an image in which a predetermined pixel value according to the direction of the luminance change is assigned to the position of the pixel where a luminance change has occurred.

[0012] <Event-based sensor: Figure 3> An example of an event-based sensor according to this embodiment will be described. The event-based sensor counts the number of photons incident on each pixel and determines the timing when the counted number of photons exceeds a predetermined threshold. The event-based sensor also measures the time (clock count) required for the number of photons to reach or exceed a first threshold, and detects a change in luminance by comparing the required times. Specifically, when the previously measured required time is T0 and the latest required time is T, if the difference T-T0 is greater than or equal to a second threshold, a negative change in luminance is detected. If the difference T0-T is greater than or equal to the second threshold, a positive change in luminance is detected. If the difference between T and T0 is less than the second threshold, no change in luminance is detected. The second threshold is a value greater than or equal to zero, and is set in accordance with a preset value or other parameters.

[0013] 3a is a diagram showing an example of the configuration of a photoelectric conversion element 1011. The photoelectric conversion element 1011 is made up of a pixel unit 110 and a peripheral circuit 120. The peripheral circuit 120 includes a vertical arbitration circuit 121 and a horizontal readout circuit 122.

[0014] FIG. 3b shows an example of the configuration of each pixel unit constituting an event-based sensor. The pixel unit 110 includes a photoelectric conversion unit 111, a pixel counter 112, a time counter 113, a first decision circuit 114, a memory 115, a comparator 116, a second decision circuit 117, a response circuit 118, and a selection circuit 119. The photoelectric conversion unit 111 includes an avalanche photodiode (SPAD) operating in Geiger mode, and the pixel counter 112 counts the number of photons incident on the photoelectric conversion unit 111. The time counter 113 counts the time during which a photon is incident on the photoelectric conversion unit 111. Using a SPAD to configure an event-based sensor enables detection of luminance changes at the level of a single photon. By detecting luminance changes at the level of a single photon, an address event signal can be acquired even in night vision conditions, such as at night.

[0015] When the number of photons counted by pixel counter 112 reaches a first threshold, first decision circuit 114 stops counting time by time counter 113. Past count values ​​of time counter 113 are stored in memory 115, and comparator 116 is used to determine the difference between the current count value of time counter 113 and the past count value of time counter 113.

[0016] If the difference count value is equal to or greater than the second threshold, the second determination circuit 117 sends a request signal to the vertical arbitration circuit 121 via the response circuit 118. The response circuit 118 receives a response from the vertical arbitration circuit 121 indicating whether or not the output of address event data is permitted. If the difference count value is less than the second threshold, the response circuit 118 does not send a request signal.

[0017] When the response circuit 118 receives a response indicating permission to output, the selection circuit 119 outputs the count value of the time counter circuit 113 to the horizontal output circuit 122. The horizontal output circuit 122 outputs the received count value as an output signal from the photoelectric conversion element 1011 to the detection unit 201.

[0018] Because the differential count value calculated by the comparator 116 corresponds to the reciprocal of the incident frequency of photons, the photoelectric conversion element 1011 according to this embodiment has the function of measuring "changes in the incident frequency of photons," i.e., changes in luminance. Furthermore, using the second determination circuit 117, an address event is output only when the difference in the intervals at which the number of incident photons reaches the first threshold is equal to or greater than the second threshold. That is, the photoelectric conversion element outputs the incident frequency when the difference in the incident frequency is equal to or greater than the second threshold, and does not output the incident frequency when the difference is less than the threshold. This configuration realizes an asynchronous photoelectric conversion element that detects changes in luminance as address events in real time for each pixel address.

[0019] <Variations of photoelectric conversion elements> The above describes a case where a photoelectric conversion element is used that uses a SPAD as the photoelectric conversion unit and measures the time at which a photon is incident to detect changes in the frequency of photon incidence. However, the configuration shown in Figure 2 is not necessary as long as the photoelectric conversion element is an asynchronous type that detects changes in luminance as address events in real time. For example, as described in Patent Document 1, a photoelectric conversion element that detects changes in luminance as voltage changes may be used.

[0020] <Detection unit 201> The detection unit 201 detects objects based on address event signals. That is, it continuously detects specific objects (e.g., human eyes) that have been set as detection targets based on the brightness change information output by the event-based sensor. In one specific process, the image processing unit 203 first generates frame images by integrating the address event signals output by the event-based sensor for a certain period of time. The detection unit 203 detects feature points from the generated frame images and detects the location of the object (and the presence or absence of the specific object) by pattern matching with feature points detected in a template image of the specific object. The location of the object can be detected by a location with a predetermined similarity to the template. However, the object detection method may be other than pattern matching. For example, the object may be detected by machine learning the shape of the object in the generated frame images using a convolutional neural network (CNN). Alternatively, the address event signals (brightness change information) output by the event-based sensor may be directly machine-learned by a CNN without being converted into frame images. Alternatively, the object may be detected based on the number and range of responsive pixels from the address event signals without generating an image.

[0021] As described above, the event-based sensor outputs an address event only when the difference in the time intervals at which the number of incident photons reaches the first threshold is equal to or greater than a second threshold (predetermined threshold). For example, if the predetermined threshold is significantly large, an address event signal may not be output if the subject's movement is small. Conversely, if the predetermined threshold is small, any movement of the subject or objects included in the scene will be output as an address event signal, making it more difficult to detect the subject. On the other hand, the detection unit 201 cannot detect the subject unless an address event signal is output. Therefore, the predetermined threshold should be set appropriately depending on the situation, and setting the predetermined threshold appropriately may improve the detection accuracy of the detection unit 201.

[0022] Location information (e.g., coordinates) of the detected subject is output to an external device via an output interface such as a network cable (not shown). Furthermore, imaging device 100 may be provided with a display unit, which displays the subject's location by superimposing the brightness change output by the event-based sensor on a frame image obtained by integrating the brightness change over a certain period of time.

[0023] <Stimulus control unit 202> The stimulus control unit 202 generates a stimulus that can be sensed by the subject and controls the output device 200 to output the stimulus. The stimulus given to the subject may be a stimulus that changes the position of the subject within the angle of view of the imaging device 100. For example, if the subject is a human eye, illumination light may be irradiated. Irradiating light onto the human eye causes the eye to move by a reflex reaction.

[0024] <Effects of the present invention> In the imaging device of the present invention, when the detection unit 201 fails to detect the subject for a period of time that exceeds a specific threshold, the stimulus control unit 202 controls the stimulus to be given to the subject. By adopting such a configuration, it is possible to suppress a decrease in the accuracy of subject detection. This is explained below.

[0025] Consider a use case in which an information processing device using an event-based sensor continuously detects a specific object. The event-based sensor outputs a luminance change as an address event signal only when there is a change in luminance equal to or greater than a predetermined threshold. Therefore, if the movement of the object is small and only a change in luminance below the predetermined threshold occurs, an address event signal is not output, and the object cannot be detected. Therefore, in the information processing device of this embodiment, if the object cannot be detected, the stimulus is controlled to move the object and cause a change in luminance equal to or greater than the predetermined threshold. If a change in luminance equal to or greater than the predetermined threshold occurs, an address event signal is output from the event-based sensor, making it possible to detect the object.

[0026] If the subject cannot be detected, it is possible to detect small changes in brightness by increasing the detection sensitivity for address events, as disclosed in Patent Document 1. However, if the detection sensitivity for address events is increased (i.e., the predetermined threshold is lowered), brightness changes caused by random noise such as photon shot noise will also be detected. As a result, brightness changes caused by minute movements of the subject will be buried in brightness changes caused by random noise, reducing the accuracy of subject detection.

[0027] On the other hand, in the information processing device of this embodiment, the detection sensitivity for address events is not increased, and by moving the subject, the luminance change due to the subject's movement is increased, and the luminance change due to the subject's movement is output as an address event signal. Therefore, the luminance change due to the subject's movement is not buried in the luminance change due to random noise, and it is possible to suppress a decrease in the subject detection accuracy.

[0028] <Stimulus variation 1> Effective stimuli include illumination light, sound, the display of images or text, and vibration of the base. Other stimuli generated by the stimulus control unit 202 may be any stimulus that changes the position of the subject within the angle of view of the information processing device 100. When the subject is a living organism such as a human, the following methods can be used, for example: The information processing device 100 includes a lighting unit that emits light to move the subject through a reflex reaction; the information processing device 100 includes an audio output unit that emits sound and guides the subject's gaze in the direction of the sound, thereby moving the subject; or the information processing device 100 includes a display unit that displays images or text on the display to guide the subject's gaze and move the subject. Furthermore, when the subject is not a living organism such as a human but an object, the information processing device 100 can be linked to a base on which the object is mounted, and the vibration of the base can be controlled by the stimulus control unit 202.

[0029] <Stimulus variation 2> The stimulus may be illumination light in a wavelength range to which the event-based sensor that outputs the address event is not sensitive. For example, if the information processing device 100 is equipped with an illumination unit and the subject is moved by irradiating the subject with illumination, it is preferable to use illumination with a wavelength to which the photoelectric conversion element 1011 is not sensitive. This is because, if the photoelectric conversion element 1011 is sensitive to the wavelength of the illumination, the change in luminance due to the illumination will also be detected as an address event, and the change in luminance due to the illumination will become noise, reducing the accuracy of subject detection.

[0030] Specifically, the event-based sensor has an infrared cut filter on the light incident side, and an infrared LED is used as the illumination. Because the light emitted by the infrared LED is absorbed by the infrared cut filter, it does not enter the photoelectric conversion element 102, and changes due to illumination are not detected as address events. On the other hand, because the human eye also exhibits a reflective response to infrared light, it is possible to detect only brightness changes due to the movement of a subject as address events, without detecting brightness changes due to illumination, thereby preventing a decrease in subject detection accuracy.

[0031] <Variation 1> If the subject is not detected even after applying a stimulus to the subject, the following processing can be considered. One is to change the intensity of the stimulus. That is, it is preferable to change at least one of the intensity, direction, and type of the stimulus generated by the stimulus control unit 202. For example, let us consider the case of moving the subject by lighting. If the subject cannot be detected even when illuminated, it is possible that the subject's movement is small due to weak lighting intensity, and no change in luminance above a predetermined threshold occurs. Therefore, if the subject cannot be detected, the lighting intensity is increased to move the subject more significantly. That is, if the intensity of the stimulus controlled by the stimulus control unit can be changed, and if the subject cannot be detected even after applying a stimulus to the subject, the intensity of the stimulus applied by the stimulus control unit can be changed. This causes a change in luminance above a predetermined threshold, making it possible to detect the subject.

[0032] The second is to change the type of stimulus. It is possible that the environment is such that the subject is unable to perceive the stimulus from the lighting. For example, if the ambient light is too bright, the lighting emitted by the information processing device 100 may be obscured by the ambient light, causing the subject to perceive the stimulus from the lighting and resulting in no change in brightness above a predetermined threshold. Therefore, a different type of stimulus can be provided to the subject, such as sound, to guide the subject's gaze in the direction of the sound, causing a change in brightness above a predetermined threshold, thereby enabling the subject to be detected. In other words, the stimulus control unit may have multiple types of stimuli controlled, and if the subject is unable to be detected even after receiving the stimulus, the stimulus control unit may change the type of stimulus provided. This may result in a change in brightness above a predetermined threshold, making it possible to detect the subject.

[0033] The third method is to change the direction of the stimulus. If the subject moves with a change in luminance below a predetermined threshold during a period in which the subject cannot be detected, there is a possibility that the subject's movement will not be detected. Therefore, if the subject cannot be detected, the direction of the illumination can be changed, such as by changing the illumination range or widening the illumination angle, so that the subject is illuminated. In other words, if the direction of the stimulus controlled by the stimulus control unit can be changed, and if the subject cannot be detected even after receiving a stimulus, the direction of the stimulus applied by the stimulus control unit can be changed. This causes a change in luminance above the threshold, making it possible to detect the subject.

[0034] <Variation 2> The detection unit 201 may perform tracking processing by storing the position of the detected subject using the memory 103. In this case, if the subject is detected after the stimulus control unit 202 generates a stimulus for the subject and the subject has not moved from the original location, it is preferable to reduce the frequency of generating stimuli in the stimulus control unit 202. Specifically, if the period during which the subject cannot be detected is a predetermined period, Threshold In a configuration that controls the stimulus to be given to the subject when the subject has not moved from its original location, it is preferable to increase the predetermined threshold value. Increasing the predetermined threshold value reduces the frequency with which the stimulus is given by the stimulus control unit 202, which leads to a reduction in power consumption for lighting control, audio control, etc., to give the stimulus to the subject.

[0035] On the other hand, if the subject has moved from its original location, it is understood that the subject has moved with a change in brightness less than the threshold value. In such a case, it is preferable to increase the frequency of applying stimuli in the stimulus control unit 202. In other words, if the subject is detected at a position different from the stored position of the subject, the detection unit 201 makes the predetermined threshold value smaller. Specifically, if there is a period of time during which the subject cannot be detected, specifiedIn a configuration that controls the stimulus to be given to the subject when the subject has moved from its original location after a certain threshold has elapsed, it is preferable to set a smaller predetermined threshold if the subject has moved. By setting a smaller predetermined threshold, the stimulus control unit 202 generates stimuli more frequently, thereby reducing the possibility of overlooking subject movement due to a change in brightness below the predetermined threshold. Whether the subject is moving can be determined, for example, by whether the subject's center of gravity has moved by more than a specific number of pixels. The specific number of pixels is preferably one pixel or more and 1 / 10 or less of the angle of view.

[0036] Furthermore, if the stimulus control unit 202 applies a stimulus to the subject but fails to detect the subject, and the subject becomes detectable only after changing the stimulus direction, such as the direction of illumination, it can be determined that the subject has moved significantly due to a change in luminance below a predetermined threshold. In such cases, it is preferable to further reduce the predetermined threshold. By reducing the predetermined threshold, the stimulus control unit 202 generates stimuli more frequently, thereby reducing the possibility of overlooking subject movement due to a change in luminance below the threshold.

[0037] <Variation 3> The display control unit 204 outputs the address event signal or the subject detection result to the display unit 104. The display control unit 204 may display a frame image that has been image processed based on the address event signal. Furthermore, if a subject is not detected, the display control unit 204 may perform control so as to notify the user of an error.

[0038] Furthermore, it is preferable that the information processing device 100 be configured so that when a stimulus is being applied to the subject, the subject is aware that the stimulus is being applied. For example, the information processing device 100 may have a display unit 104, and when a stimulus is being applied, text information or the like may be displayed to indicate that the stimulus is being applied. With this configuration, it is possible to distinguish whether the subject is being moved by the stimulus from the imaging device or whether the subject itself is moving.

[0039] Furthermore, it is even more preferable that the information processing device 100 has a processing unit that processes the output of the photoelectric conversion element 1011 to generate a frame image, and is configured to display the frame image on the display unit, with the location where the stimulus is being applied being superimposed on the image.

[0040] <Overall flowchart> FIG. 4 is a flowchart illustrating the operation of the information processing device 100 according to this embodiment. The processing shown in the flowchart in FIG. 4 is executed by the CPU 102 of FIG. 1, which is a computer, in accordance with a computer program stored in the memory 103. In the following description, each process (step) is denoted by prefixing it with an S, and the process (step) is notated accordingly. In S11, the detection unit 201 determines whether a period during which the subject cannot be detected continues for a predetermined time or longer. If the result of the determination is equal to or greater than a specific threshold, the stimulus control unit 202 generates and controls to output a stimulus to be given to the subject (S12). If the predetermined time is less than the predetermined time, the detection unit 201 continues detecting the subject without changing the stimulus to be given to the subject (continues S11).

[0041] In S13, the detection unit 201 determines whether or not the subject has been detected as a result of applying a stimulus to the subject. If the subject has been detected, the detection unit 201 further determines whether or not the subject has moved (S19). On the other hand, if the subject has not been detected, the stimulus control unit 202 changes the intensity of the stimulus applied to the subject (S14).

[0042] In S15 following S14, it is determined whether the object has been detected as a result of changing the intensity of the stimulus applied to the object. If the object has been detected, the process returns to S11, the stimulus applied to the object is restored to its original state, and object detection continues. On the other hand, if the object has not been detected, the stimulus control unit 202 changes the type of stimulus applied to the object (S16).

[0043] In S17 following S16, the detection unit 201 determines whether the object has been detected as a result of changing the type of stimulus to be applied to the object. If the object has been detected, the process returns to S11, the output of the stimulus is terminated, and object detection continues. On the other hand, if the object has not been detected, the stimulus control unit 202 changes the direction of the stimulus to be applied to the object (S18).

[0044] If it is determined in S19 that the subject position has not moved, the predetermined threshold is increased (S20). On the other hand, if it is determined that the subject position has moved, the predetermined threshold is decreased (S21). Furthermore, in S22 following S18, the predetermined threshold is increased further than in S21. Lower .

[0045] The present invention can also be realized by executing the following process. That is, software (programs) that realize the functions of the above-described embodiments are supplied to a system or device via a data communication network or various storage media. Then, a computer (or CPU, MPU, etc.) of the system or device reads and executes the programs. The programs may also be provided by recording them on a computer-readable storage medium. [Explanation of symbols]

[0046] 100 Information processing device 200 Output Device 101 Imaging unit 201 Detection unit 202 Stimulation control section 203 Image Processing Unit

Claims

1. An information processing device that acquires an address event signal indicating a position and time of a pixel where a change in luminance has occurred, a detection means for detecting a subject based on the address event signal; a generating means for generating a stimulus that can be sensed by the subject when a time during which the subject is not detected is longer than a predetermined threshold; a display control means for, when the stimulus is generated by the generation means, displaying information indicating that the stimulus is being generated on a display unit; the generating means changes the direction of the stimulus when the subject is not detected by the detecting means even after the stimulus is given to the subject; when the subject is detected by the detection means after the stimulus whose direction has been changed is given to the subject, the predetermined threshold is made smaller.

1. An information processing device comprising:

2. 2. The information processing device according to claim 1, wherein the stimulus is any one of illumination light, sound, display of an image or text, and vibration of the base.

3. 3. The imaging device according to claim 2, wherein the stimulus is illumination light in a wavelength range to which the event-based sensor that outputs the address event signal has no sensitivity.

4. 4. The information processing device according to claim 1, wherein the generating means changes the intensity of the stimulus when the subject is not detected by the detecting means even after the stimulus is applied to the subject.

5. 5. The information processing device according to claim 1, wherein the generating means changes the type of the stimulus when the subject is not detected by the detecting means even after the stimulus is applied to the subject.

6. An information processing device that acquires an address event signal indicating the position and time of a pixel where a change in luminance has occurred, a detection means for detecting a subject based on the address event signal; a generating means for generating a stimulus that can be sensed by the subject when a time during which the subject is not detected is longer than a predetermined threshold; a display control means for, when the stimulus is generated by the generation means, displaying information indicating that the stimulus is being generated on a display unit; a storage means for storing the position of the subject detected by the detection means, an information processing device that reduces the predetermined threshold value when the subject is detected at a position different from the position of the subject stored by the storage means after the stimulus is given to the subject;

7. 7. The information processing device according to claim 6, wherein the detection means increases the predetermined threshold value when the subject is detected at the same position as the position of the subject stored by the storage means after the stimulus is applied to the subject.

8. The information processing apparatus according to claim 1 , further comprising a notification means for notifying that the stimulus has been generated.

9. 9. The information processing apparatus according to claim 8, wherein the notification means notifies an error when the subject is not detected by the detection means even after the stimulus is given to the subject.

10. 10. The information processing device according to claim 1, further comprising control means for controlling a predetermined mechanism to represent the stimulus generated by the generation means.

11. 11. The information processing apparatus according to claim 1, wherein the detection means detects the subject having a predetermined shape based on the position of a pixel having a specific signal included in the address event signal.

12. 12. The information processing device according to claim 1, wherein the address event signal is output by a photoelectric conversion element having a pixel that outputs a signal in response to an incidence of a photon.

13. 13. The information processing apparatus according to claim 1, further comprising a display unit that displays the address event signal and the subject detected by the detection unit.

14. A program for causing a computer to function as each of the means included in the information processing device according to any one of claims 1 to 13.

15. An information processing method for acquiring an address event signal indicating a position and time of a pixel where a change in luminance has occurred, comprising: a detecting step of detecting a subject based on the address event signal; a generating step of generating a stimulus that can be sensed by the subject when a time period during which the subject is not detected is longer than a predetermined threshold; a display control step of, when the stimulus is generated by the generation step, displaying information indicating that the stimulus has been generated on a display unit; the generating step changes a direction of the stimulus when the subject is not detected in the detecting step even after the stimulus is given to the subject; an information processing method, characterized in that, when the subject is detected in the detecting step after the stimulus whose direction has been changed is given to the subject, the predetermined threshold is made smaller.

16. An information processing method for acquiring an address event signal indicating a position and time of a pixel where a change in luminance has occurred, comprising: a detecting step of detecting a subject based on the address event signal; a generating step of generating a stimulus that can be sensed by the subject when a time period during which the subject is not detected is longer than a predetermined threshold; a display control step of displaying, on a display unit, information indicating that the stimulus has been generated, when the stimulus has been generated by the generation step; a storage step of storing the position of the subject detected in the detection step, An information processing method characterized in that, if the subject is detected at a position different from the position of the subject stored in the storage step after the stimulus is given to the subject, the predetermined threshold is made smaller.

Images