Information processing device, information processing method, and program

Abstract

To detect the outline of a stopped subject, when an event-based sensor is used.SOLUTION: An information processing device includes motion detection means for detecting a moving object in an imaging range on the basis of an address event signal indicating the position and time of a pixel at which a change in luminance occurs, a shape detection means for detecting the shape of the moving object from a frame image generated on the basis of the address event signal, and control means for changing the imaging range. The shape detection means detects the shape of the moving object by changing the imaging range by a predetermined value by the control means when the moving object detected by the motion detection means is no longer detected.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to object detection using an event-based sensor. [Background technology]

[0002] An event-based sensor is known that outputs a change in luminance of each pixel as an address event signal in real time (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-72317 Summary of the Invention [Problem to be solved by the invention]

[0004] The problem to be solved by the present invention is to detect the outer shape of a stationary subject when an event-based sensor is used. [Means for solving the problem]

[0005] The information processing device according to the present invention for solving the above-mentioned problems comprises: a motion detection unit that detects a moving object in a shooting range based on an address event signal that indicates the position and time of a pixel where a change in luminance has occurred; a shape detection unit that detects the shape of the moving object from frame data generated based on the address event signal; direction and a control means for changing the shape of the image when the moving object detected by the motion detection means is no longer detected. direction of Based on the size of the moving object detected by the motion detection means, The shape of the moving object is detected by changing the value by a predetermined value. [Effects of the Invention]

[0006] According to the present invention, when an event-based sensor is used, the outer shape of a stationary subject can be detected. [Brief explanation of the drawings]

[0007] [Figure 1] A block diagram showing an example of the hardware configuration of an information processing device. [Figure 2] A diagram showing an example of the configuration of an event-based sensor [Figure 3] A block diagram showing an example of the functional configuration of an information processing device. [Figure 4] Schematic diagram explaining the subject detection method [Figure 5] 10 is a flowchart illustrating a process executed by an information processing device. [Figure 6] 10 is a flowchart illustrating a process executed by an information processing device. DETAILED DESCRIPTION OF THE INVENTION

[0008] Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Note that the configurations shown in the following embodiments are merely examples, and the present invention is not limited to the illustrated configurations.

[0009] <Embodiment 1> Conventionally, synchronous photoelectric conversion elements that capture image data (frames) in synchronization with a synchronization signal such as a vertical synchronization signal have been used in imaging devices. This general synchronous photoelectric conversion element can only acquire image data at synchronization signal intervals (e.g., 1 / 60 seconds), making it difficult to respond to requests for high-speed processing. Therefore, asynchronous photoelectric conversion elements that detect luminance changes as address events for each pixel address in real time have been proposed (see, for example, Patent Document 1). Such photoelectric conversion elements that detect events for each pixel address are called dynamic vision sensors (DVSs, hereinafter referred to as event-based sensors). Because imaging devices using such event-based sensors excel at detecting the movement of subjects, potential use cases include road surveillance, where various moving objects such as vehicles and people are monitored.

[0010] <Hardware configuration of the imaging device 100: Figure 1> FIG. 1 is a schematic diagram showing an example of the hardware configuration of an imaging device (information processing device) 100. While the imaging device 100 is specifically an imaging device having an event-based sensor, the image processing function and the image analysis function (motion detection) may be performed by separate devices. The imaging device 100 includes an imaging unit 101 consisting of an imaging optical system 1010 and a photoelectric conversion element 1011, a CPU 102, a memory 103, a display unit 104, and an operation unit 105. The photoelectric conversion element 1011 is an event-based sensor that outputs an address event signal in response to received incident light. The event-based sensor detects a change in luminance for each pixel as an event, and the address event signal indicates the position and time of the pixel where the luminance change occurred. The imaging optical system 1010 is specifically a light-receiving lens that receives incident light and forms an image on the photoelectric conversion element 1011. The CPU 102 reads and executes the OS and other programs stored in the memory 103, controls each connected component, and performs calculations and logical judgments for various processes. The processes executed by the CPU 102 include the information processing according to this embodiment. The CPU 102 also controls the focus and aperture of the imaging optical system 1010, the photoelectric conversion element 1011, and other functions. The memory 103 is, for example, a hard disk drive or an external storage device, and stores programs and various data related to the information processing according to this embodiment. The display unit 104 is, for example, a display device that displays the results of calculations performed by the information processing device 100 in accordance with instructions from the CPU 102. The display device may be any type, such as a liquid crystal display device, a projector, or an LED indicator. The operation unit 105 is, for example, a touch panel, keyboard, mouse, or robot controller, and serves as a user interface that accepts input instructions from a user. The information processing device 100 may also have mechanisms other than the hardware configurations listed here.

[0011] <Photoelectric conversion element 1011: Figure 2> An example of a photoelectric conversion element (event-based sensor) according to this embodiment will be described. The event-based sensor counts the number of incident photons and determines the timing when the counted number of photons exceeds a predetermined threshold. The event-based sensor also measures the time (clock count) required for each pixel until the number of photons reaches or exceeds a first threshold, and detects a change in luminance by comparing the required times. Specifically, when the previously measured required time is T0 and the latest required time is T, if the difference T-T0 is equal to or greater than a second threshold, a negative change in luminance is detected. If the difference T0-T is equal to or greater than the second threshold, a positive change in luminance is detected. If the difference between T and T0 is less than the second threshold, no change in luminance is detected. The second threshold is a value greater than or equal to zero, and is set in accordance with a preset value or other parameters.

[0012] The detailed configuration will be described below. Fig. 2(A) is a diagram showing an example of the configuration of a photoelectric conversion element 1011. The photoelectric conversion element 1011 is composed of a pixel unit 110 and a peripheral circuit 120. The peripheral circuit 120 includes a vertical arbitration circuit 121 and a horizontal readout circuit 122.

[0013] FIG. 2B illustrates an example configuration of each pixel unit constituting an event-based sensor. The pixel unit 110 includes a photoelectric conversion unit 111, a pixel counter 112, a time counter 113, a first decision circuit 114, a memory 115, a comparator 116, a second decision circuit 117, a response circuit 118, and a selection circuit 119. The photoelectric conversion unit 111 includes an avalanche photodiode (SPAD) operating in Geiger mode, and the pixel counter 112 counts the number of photons incident on the photoelectric conversion unit 111. The time counter 113 counts the time during which a photon is incident on the photoelectric conversion unit 111. Using a SPAD to configure an event-based sensor enables detection of luminance changes at the level of a single photon. By detecting luminance changes at the level of a single photon, an address event signal can be acquired even in night vision conditions, such as at night.

[0014] When the number of photons counted by pixel counter 112 reaches a first threshold, first decision circuit 114 stops counting time by time counter 113. Past count values ​​of time counter 113 are stored in memory 115, and comparator 116 is used to determine the difference between the current count value of time counter 113 and the past count value of time counter 113.

[0015] If the difference count value is equal to or greater than the second threshold, the second determination circuit 117 sends a request signal to the vertical arbitration circuit 121 via the response circuit 118. The response circuit 118 receives a response from the vertical arbitration circuit 121 indicating whether or not the output of address event data is permitted. If the difference count value is less than the second threshold, the response circuit 118 does not send a request signal.

[0016] When the response circuit 118 receives a response indicating permission to output, the selection circuit 119 outputs the count value of the time counter circuit 113 to the horizontal output circuit 122. The horizontal output circuit 122 outputs the received count value as an output signal from the photoelectric conversion element 1011 to the detection unit 201.

[0017] Because the differential count value calculated by the comparator 116 corresponds to the reciprocal of the incident frequency of photons, the photoelectric conversion element 1011 according to this embodiment has the function of measuring "changes in the incident frequency of photons," i.e., changes in luminance. Furthermore, using the second determination circuit 117, an address event is output only when the difference in the intervals at which the number of incident photons reaches the first threshold is equal to or greater than the second threshold. That is, the photoelectric conversion element outputs the incident frequency when the difference in the incident frequency is equal to or greater than the second threshold, and does not output the incident frequency when the difference is less than the threshold. This configuration realizes an asynchronous photoelectric conversion element that detects changes in luminance as address events in real time for each pixel address.

[0018] <Variations of photoelectric conversion elements> The above describes a case where a photoelectric conversion element is used that uses a SPAD as the photoelectric conversion unit and measures the time at which a photon is incident to detect changes in the frequency of photon incidence. However, the configuration shown in Figure 2 is not necessary as long as the photoelectric conversion element is an asynchronous type that detects changes in luminance as address events in real time. For example, as described in Patent Document 1, a photoelectric conversion element that detects changes in luminance as voltage changes may be used.

[0019] <Example of functional configuration of imaging device 100: FIG. 3> 3 is a diagram showing an example of the functional configuration of the imaging device 100. The imaging device (information processing device) 100 includes an imaging unit 101, an image acquisition unit 201, a control unit 202, a motion detection unit 203, a shape detection unit 204, a communication unit 205, a drive unit 206, a network 207, and an external device 208.

[0020] The imaging unit 101 includes an imaging optical system 1010 and a photoelectric conversion element 1011. Light passing through the imaging optical system 1010 is focused on the solid-state imaging element 1011, converted into an electrical signal, and output as an image signal by the image acquisition unit 201. The imaging unit 101 particularly refers to a lens housing. The imaging optical system 1010 is composed of multiple lenses and holding members, has a mechanical structure for aperture, zoom, and focus control, and may be configured to be controlled by an imaging optical system control unit (not shown). The photoelectric conversion element 1011 detects changes in the luminance value of the subject for each pixel and outputs an address event signal for each pixel where a change has occurred. In other words, the address event signal is output by a sensor that detects the luminance change per unit time for each pixel. If no luminance change is detected, the address event signal is not output, or indicates that there is no luminance change.

[0021] Based on the address event signal, the image acquisition unit 201 acquires frame data indicating pixels in which a change in luminance occurred during a predetermined period. That is, the image acquisition unit 201 performs correction and development processes on the electrical signals converted by the photoelectric conversion element 1011, converting the signals into frame data, and outputs the converted image signals. The motion detection unit 203 uses this image signal to detect when the subject has stopped moving. When the subject moves, a change in luminance occurs in a group of areas. Therefore, a threshold is set for the area of ​​the area where the luminance value has changed, and if the area of ​​the area is equal to or greater than the threshold, it is determined that the subject is moving, and if it subsequently drops below the threshold, it is determined that the subject has stopped.

[0022] When the control unit 202 detects that the subject has stopped, it sends a control signal to the drive unit 206 and drives the image capture device 100 to perform a constant periodic movement in response to the control signal. The drive unit 206 has at least one-axis drive mechanism capable of changing the shooting direction of the image capture device 100. The drive unit 206 is not limited to driving the image capture device 100; it may also drive the image capture unit 101 if the shooting direction can be changed. The outer shape detection unit 108 extracts the outer shape of the subject from the image signal acquired by the image acquisition unit 201 using existing edge analysis processing that detects changes in luminance values, and recognizes the subject using image recognition processing such as machine learning. The image used for analysis is an integral image or additive image converted into frame data by the image acquisition unit 201. The image capture device 100 may further include a recognition unit that recognizes the class of the subject based on the detected shape.

[0023] The communication unit 205 transmits the image signal output from the image acquisition unit 201 to an external device 208 via a network 207. It is also possible to receive various parameters for the imaging device 100 from the external device 205 and output them to the control unit 202, etc. The various parameters include a control signal for the drive unit 206 and a signal indicating that shape detection has been completed.

[0024] The network 207 is configured with routers and cables that comply with communication standards such as Ethernet (registered trademark), and enables connection between the imaging device 100 and the external device 207 .

[0025] The external device 208 refers to a device with a display unit, such as a PC or a mobile terminal, and is capable of communicating with the imaging device 500 via the network 207. The external device 208 can display images output by the image acquisition unit 201. The external device 208 also has a user interface and can send commands to the imaging device 100 in response to user operations to control the imaging device 500. In particular, by operating the user interface on the display screen, it is possible to control the start and stop of driving of the driving unit 206 for objects whose outer shape cannot be detected. The external device 208 may also have a user interface that can adjust the amplitude and driving speed of the driving unit 206.

[0026] <Motion detection unit 203> A detailed description will be given of a method for detecting whether or not a subject is moving using the motion detection unit 203. The motion detection unit 203 detects a moving object in the frame data by detecting the number of connections between pixels where a change in luminance has occurred based on the frame data.

[0027] First, the image acquisition unit compares the pixel positions contained in the frame data acquired by the image acquisition unit to see if the connectivity count is equal to or greater than a first threshold N1 related to motion information. If the set of pixels (connectivity count) where a change in brightness has occurred, detected based on the address event signal, is equal to or greater than the first threshold N1 related to motion information, it is determined to be not noise (the frame data contains a moving object), and the process proceeds to matching with feature point data. Here, the connectivity count refers to the number of pixels that have pixel outputs at the same timestamp (which can be any period) that are connected (adjacent) on a two-dimensional plane.

[0028] <Example: Figure 4> FIG. 4 is a schematic diagram illustrating a subject detection method according to the first embodiment. First, referring to FIG. 4(a), output when a subject 401 stops will be described. The shooting direction of the image capture device 100 capturing the subject 401 is indicated by a dashed line 402. Frame data 403 indicates frame data of an image signal acquired by the image acquisition unit 201 when the motion detection unit 203 does not detect the subject. Because the photoelectric conversion element 1011 outputs a signal only when there is a change in the luminance value of the subject, if the subject stops moving, the subject will not be detected as shown in frame data 403. Therefore, as shown in FIG. 4(b), the driving unit 206 repeatedly drives the shooting direction of the image capture device 100 from 402-1 to 402-3. In other words, the shooting range is changed as shown in frame data 403-1, 403-2, and 403-3, forcibly changing the luminance so that the stopped subject is captured. This control makes it possible to detect a stationary subject.

[0029] If the drive amplitude is too small relative to the subject, the brightness value change will be small, while if it is too large, the time required for the brightness to change monotonically in one direction will be long, making it difficult to detect the subject's shape. Therefore, it is desirable to drive the drive unit 206 so that the amplitude is approximately equal to the size of the subject. Here, the size of the subject is defined as the length of the subject in the axial direction of the drive unit, and is calculated by the number of pixels in the axial direction of the area where a brightness value change occurs after the drive unit 206 is driven before or after the subject stops. The size of the subject may also be defined as an area. It is preferable to control the drive amplitude so that it is within the difference in the number of pixels previously stored relative to the size of the subject, or within a ratio to the size of the subject. Furthermore, to simplify the calculation process in the outer shape detection unit 204, it is desirable to keep the drive cycle of the drive unit 206 constant.

[0030] Variations in the driving method of the driving unit 206 will be described below.

[0031] The amplitude of the driving unit 206 may be relatively larger when the size of the subject is large compared to when it is small. The driving unit 206 may be driven so that the amplitude gradually increases from the start of driving. In this way, even when the subject is large, the increased amplitude can cause a sufficient change in luminance, making shape detection easier.

[0032] The image capture device 100 may have an electronic zoom or optical zoom mechanism, and when the zoom magnification is high, the amplitude may be relatively smaller or the drive speed may be slower compared to when the zoom magnification is low. By performing the above processing when the zoom magnification is high, it is possible to control the subject so that it does not go out of frame.

[0033] 4(c), the driving unit 206 may perform control so as to repeatedly drive in directions from 402-2 to 402-4 around the driving shaft 402-3 so that the subject 401 is at the center of the angle of view. By driving around the subject, it is possible to generate sufficient brightness changes while reducing the possibility of the subject going out of frame, and since the driving is performed periodically around the subject, calculations for shape detection become easier.

[0034] It is preferable that the drive unit 206 has a two-axis drive mechanism. Furthermore, while a single axis makes it difficult to detect subject edges perpendicular to the axis, moving in multiple directions makes it easier to detect all subject edges, so it is preferable to move the drive unit horizontally and vertically. Furthermore, for example, if the drive time in the horizontal direction is long, the frequency of edge detection by drive in the vertical direction decreases, so it is even more preferable to move the drive unit alternately horizontally and vertically. The drive direction is not limited to horizontal and vertical, and it may be diagonal, or it may be rotated.

[0035] <Flowchart> Fig. 5 shows an example of a flowchart for realizing embodiment 1. The processing shown in the flowchart in Fig. 5 is executed by CPU 101 in Fig. 1, which is a computer, in accordance with a computer program stored in memory 103. In the following description, each process (step) is represented by adding an S to the beginning, and the notation of the process (step) is omitted.

[0036] In S501, the image acquisition unit 201 acquires frame data (image) based on an address event signal. In S502, the motion detection unit 203 detects an object based on the frame data. The object is detected using the above-described motion detection processing method, and the object is detected when the number of connected pixels whose luminance has changed is equal to or greater than a predetermined threshold. If an object is detected, the process proceeds to S503. If an object is not detected, the process returns to S501. In S503, the motion detection unit 203 detects whether the detected object has stopped. If the number of connected pixels whose luminance has changed is less than a predetermined threshold based on frame data generated after the frame data in which the moving object was detected, the process determines that the object has stopped, and proceeds to S504. If the number of connected pixels whose luminance has changed is equal to or greater than a predetermined threshold, the process determines that there is movement (the object has not stopped), and proceeds to S505. In S504, the drive unit 206 starts driving. That is, the control unit 202 controls the drive unit 206 to perform a predetermined movement. In S505, the outer shape detection unit 204 detects the outer shape of the subject based on the frame data. If the outer shape of the subject is detected, the process proceeds to S506. If the outer shape of the subject is not detected, the process returns to S503 to detect the subject again. In S506, the drive unit 206 stops driving under the control of the control unit 202, and the flow ends.

[0037] As described above, an image capturing device using an event-based sensor can detect the outer shape of a stationary subject.

[0038] <Embodiment 2> In the second embodiment, a method will be described in which shape detection is started when the subject is moving, and the driving unit 206 is driven if the outer shape of the subject cannot be detected when the subject stops.

[0039] FIG. 6 shows a flowchart for controlling the driving unit 206 to be driven when a subject whose outline cannot be detected during operation stops. In S601, the image acquisition unit 201 acquires frame data (image) based on an address event signal. In S602, the motion detection unit 203 detects the subject based on the frame data. In S603, the outline detection unit 204 detects the outline of the subject based on the frame data. In S604, if the motion detection unit 203 detects that the subject has stopped, the process proceeds to S605. S603 is repeated until the stop of the subject is detected. In S605, the outline detection unit 204 determines whether the outline of the subject was detected before the subject stopped. If the outline was detected, the process proceeds to S405; if not, the process proceeds to S404. If the number of connections of pixels whose luminance has changed is equal to or greater than a predetermined threshold, it is highly likely that the outline of the subject has been detected, and it is therefore determined that the outline of the subject has been detected. Furthermore, the outer shape detection unit 204 may determine whether outer shape detection has been successful, for example, based on the likelihood obtained during the outer shape detection process. In S606, the control unit 202 drives the drive unit 206. Once driven, the process returns to S603, where it is determined whether the outer shape has been detected. If the outer shape has been detected in S605, the drive unit is stopped and the flow ends.

[0040] By limiting the condition for driving the drive unit to when the outer shape of the subject cannot be detected before the subject stops, rather than only when the subject stops, it is possible to reduce the frequency of driving.

[0041] <Embodiment 3> In the third embodiment, a case will be described in which the user can determine the external shape and give instructions to drive the drive unit 206.

[0042] The imaging device 100 has a receiving unit that receives inputs such as control instructions from the user. The user visually checks the display of frame data to input a determination result as to whether or not the outer shape has been detected. Furthermore, the user can input drive instructions for the driving unit 206. The user can perform various inputs through a user interface provided in the imaging device 100 or the external device 208. For example, the user can control the start and stop of driving the driving unit 206 with respect to the subject. Rather than mechanically detecting the outer shape of the subject, the user can visually confirm the outer shape of the subject by instructing the driving unit 206 to drive via the user interface if the user is unable to confirm it with the naked eye. The user may also specify the movement of the driving unit 206.

[0043] Regarding the display method, the display during the period when the driving unit 206 is being driven may be cut out and displayed so that the target subject is displayed in the center of the display screen. Also, if the outline of the subject can be detected, a display frame or a classification pop-up may be displayed.

[0044] The flow is basically the same as the flow described in Fig. 5, but variations are possible due to the addition of the external device 208. For example, when the user wants to visually check the outline of the subject while it is stopped, it is possible to start driving the driving unit 206. Similarly, the user can also decide the timing to stop the driving unit 206.

[0045] As described above, in an imaging device using an event-based sensor, the outer shape of a stationary subject can be detected when required by the user.

[0046] The following describes a case where there are multiple objects whose outer shapes are to be detected. Even when there are multiple objects, this problem can be solved by driving the driving unit 206 when the object stops. As a variation, when the motion detection unit 203 detects that the object is moving, the outer shape detection unit 204 detects the outer shape, and when an object that could not be detected stops, the driving unit 206 starts to be driven. Alternatively, the user may select an object whose outer shape is to be detected and drive the driving unit 206. Furthermore, the driving unit 206 may first be driven with an amplitude equivalent to the size of the object for an object whose shape is detected to be stopped, and after the outer shape is detected, the amplitude may be increased to detect other stopped objects.

[0047] As described above, in an imaging device using an event-based sensor, even if there are multiple subjects, it is possible to detect the outer shape of a stationary subject.

[0048] The present invention can also be realized by executing the following process. That is, software (programs) that realize the functions of the above-described embodiments are supplied to a system or device via a data communication network or various storage media. Then, a computer (or CPU, MPU, etc.) of the system or device reads and executes the programs. The programs may also be provided by recording them on a computer-readable storage medium. [Explanation of symbols]

[0049] 100 Information processing device 101 Imaging unit 1010 Imaging optical system 1011 Photoelectric conversion element 201 Image acquisition unit 202 Control section 206 Drive Unit 203 Motion detection unit 204 Shape detection unit 205 Reception 206 Drive Unit 207 Network 208 External device

Claims

1. a motion detection means for detecting a moving object in a photographing range based on an address event signal indicating the position and time of a pixel where a change in luminance has occurred; a shape detection means for detecting the shape of the moving object from frame data generated based on the address event signal; and a control means for changing the photographing direction, The information processing device is characterized in that, when a moving object detected by the motion detection means is no longer detected, the shape detection means detects the shape of the moving object by having the control means change the shooting direction by a predetermined value based on the size of the moving object that was detected by the motion detection means.

2. 2. The information processing device according to claim 1, wherein the address event signal is output by a sensor that detects a change in brightness for each pixel, and when no change in brightness is detected, the address event signal is not output or indicates that there is no change in brightness.

3. further comprising image acquisition means for acquiring the frame data indicating pixels in which a luminance change occurs during a predetermined period based on the address event signal; 3. The information processing apparatus according to claim 1, wherein the shape detection means detects the shape of the moving object based on the acquired frame data.

4. 3. The information processing apparatus according to claim 2, wherein said motion detection means detects a moving object in said frame data by detecting the number of connections of pixels in which a change in luminance occurs based on said frame data.

5. The information processing device described in claim 4, characterized in that the motion detection means detects a moving object when the number of connections in the frame data is greater than or equal to a threshold, and detects that the moving object has stopped when the number of connections in the frame data generated after detecting the moving object transitions to less than the threshold.

6. 6. The information processing apparatus according to claim 1, wherein the shape detection means detects the shape of the moving object from the frame data by edge analysis processing.

7. 7. The information processing apparatus according to claim 1, further comprising recognition means for recognizing a class of the moving object based on the shape detected by the shape detection means.

8. Further, the control means controls at least one axis of the drive unit, 8. The information processing device according to claim 1, wherein the control means controls the drive unit to make a constant periodic amplitude when the moving object detected by the motion detection means is no longer detected.

9. 9. The information processing apparatus according to claim 8, wherein the control means adjusts the amplitude in accordance with the size of the moving object when the moving object detected by the motion detection means is no longer detected.

10. 10. The information processing device according to claim 8, wherein the control means adjusts the amplitude of the drive unit to be relatively smaller when the zoom magnification is high than when the zoom magnification is low, for a mechanism capable of electronic zoom or optical zoom.

11. 11. The information processing apparatus according to claim 8, wherein the control means adjusts a central axis of the drive unit so that the moving object is located at the center of the image capturing range.

12. the drive unit further includes a two-axis drive mechanism, 12. The information processing apparatus according to claim 8, wherein the control means drives the driving unit so that the direction of movement of the driving unit changes over time.

13. 13. The information processing apparatus according to claim 1, wherein the control means stops the change of the image capturing direction by the control means when the shape of the moving object is detected by the shape detection means.

14. 14. The information processing apparatus according to claim 1, further comprising output means for outputting the detected shape to a display device.

15. 15. The information processing apparatus according to claim 14, wherein said output means outputs the detected shape so as to be superimposed on the frame data and displayed.

16. 16. The information processing apparatus according to claim 1, wherein the control means controls the change in the photographing direction based on a movement set by a user.

17. 17. The information processing apparatus according to claim 1, wherein said motion detection means detects said moving object based on the position of a pixel having a specific signal included in said address event signal.

18. 18. The information processing device according to claim 1, wherein the address event signal is output by a photoelectric conversion element having a pixel that outputs a signal in response to an incidence of a photon.

19. A program for causing a computer to function as each of the means included in the information processing device according to any one of claims 1 to 18.

20. a motion detection step of detecting a moving object in a photographing range based on an address event signal indicating the position and time of a pixel where a change in luminance has occurred; a shape detection step of detecting a shape of the moving object from frame data generated based on the address event signal; a control step of changing the photographing direction, The information processing method is characterized in that the shape detection process detects the shape of the moving object by changing the shooting direction by a predetermined value based on the size of the moving object detected in the motion detection process when the detected moving object is no longer detected.

Images