Information processing device, information processing method, and program

Abstract

To improve the estimation accuracy of the edge position of a target object when the target object is displaced within the sensing range of a sensor.SOLUTION: An information processing device comprises a movement trajectory estimation unit which estimates the movement trajectory of an edge position of a target object on the basis of the position information of event occurrence pixels detected by an event sensor during a past period prior to a reference time point under the condition that the positional relation between the event sensor and the target object changes as the target object displaces within the sensing range of the event sensor, and an edge position estimation unit which estimates the edge position of the target object at the reference time point as a reference edge position on the basis of the movement trajectory of the edge position estimated by the movement trajectory estimation unit.SELECTED DRAWING: Figure 7

Description

Technical Field

[0001] The present technology relates to an information processing apparatus, a method thereof, and a program, and particularly relates to a technology for improving the accuracy of estimating the edge position of an object when the object is displaced within the sensing range of a sensor.

Background Art

[0002] There are various technologies related to image sensing. For example, there is a technology for performing image analysis processing on a captured image by an image sensor such as an RGB sensor to estimate the edge position or shape of an object.

[0003] As an application of such image analysis technology, for example, in the field of logistics, for a so-called hand-eye type robot arm, based on the result of performing image analysis on a captured image of an object such as a cardboard box, the pickup timing of the object by the robot arm is determined. Specifically, in this case, the robot arm is movable horizontally toward the position where an object such as a cardboard box is placed, and a camera for imaging downward is attached to the hand portion at the tip of the robot arm. When the robot arm moves horizontally toward the object, the object is captured within the camera's field of view (sensing range), and then the timing when the object is positioned at a specific position within the camera's field of view is determined as the pickup timing of the object. In this case, as specific image analysis, a process of estimating the edge position of the object shown in the captured image is performed. Then, based on the edge position, the position of the object (position within the captured image) is specified, and the timing when the specified position of the object is located at a specific position defined in the coordinate system of the image is determined as the pickup timing of the object.

[0004] Regarding related prior arts, Patent Document 1 below can be cited. Patent Document 1 discloses a technique for detecting the edge position of an object from two angles using a device that moves along the object.

Prior Art Document

Patent Document

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] Here, for example, in the case of image analysis in the above-mentioned logistics field, when performing image analysis on an image captured while the positional relationship between the target object and the camera changes, in other words, an image captured while the target object is displaced within the sensing range of the camera (image sensor), it is difficult to accurately estimate the edge position of the target object. When the target object is moving within the sensing range, due to factors such as the change in the way light hits the target object during the movement process, an image in which the edge portion of the target object is not accurately captured may be captured. For such a captured image, it is difficult to accurately estimate the edge position of the target object. And when the captured image corresponding to the pickup timing becomes such a captured image, it becomes impossible to accurately determine the pickup timing.

[0007] This technology has been made in view of the above circumstances, and aims to improve the estimation accuracy of the edge position of a target object when the target object is displaced within the sensing range of the sensor.

Means for Solving the Problems

[0008] The information processing apparatus according to the present technology includes a movement trajectory estimation unit that estimates a movement trajectory of an edge position of a target object based on position information of event occurrence pixels detected by the event sensor within a past period before a reference time point while the positional relationship between the event sensor and the target object changes such that the target object is displaced within the sensing range of the event sensor, and an edge position estimation unit that estimates the edge position of the target object at the reference time point as a reference time edge position based on the movement trajectory of the edge position estimated by the movement trajectory estimation unit. In this specification, the event sensor corresponds to a so-called EVS (Event-based Vision Sensor), and specifically means a sensor defined as follows. That is, it means a sensor including a plurality of pixels each having a light receiving element, configured such that each pixel can detect a change in the light reception amount equal to or greater than a predetermined amount as an event, and capable of outputting information indicating the position of the pixel where the event is detected and information indicating the detection time of the event. According to the above configuration, even when an event misdetection occurs, the movement trajectory of the edge position of the target object is estimated based on the position information of the event occurrence pixels detected within the past period before the reference time point, so that the edge position at the reference time point can be accurately estimated based on the movement trajectory. Further, by using the event sensor, the sampling rate of the detection information used for estimating the movement trajectory of the edge position of the target object can be improved, and thus the estimation accuracy of the movement trajectory of the edge position can be improved.

[0009] Further, the information processing method according to the present technology is an information processing method in which an information processing apparatus estimates a movement trajectory of an edge position of a target object based on position information of event occurrence pixels detected by the event sensor within a past period before a reference time point while the positional relationship between the event sensor and the target object changes such that the target object is displaced within the sensing range of the event sensor, and estimates the edge position of the target object at the reference time point as a reference time edge position based on the estimated movement trajectory of the edge position.

[0010] Furthermore, the program according to the present technology is a program readable by a computer device, and based on the position information of event occurrence pixels detected by the event sensor within a past period before a reference time point while the positional relationship between the event sensor and the target object changes such that the target object is displaced within the sensing range of the event sensor, a moving trajectory estimation function for estimating a moving trajectory of an edge position of the target object, and an edge position estimation function for estimating, as a reference time edge position, the edge position of the target object at the reference time point based on the moving trajectory of the edge position estimated by the moving trajectory estimation function, and is a program for causing the computer device to realize the functions.

[0011] These information processing methods and programs implement the information processing apparatus according to the present technology described above.

Brief Description of Drawings

[0012]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Modes for Carrying Out the Invention

[0013] Hereinafter, with reference to the accompanying drawings, embodiments according to the present technology will be described in the following order. <1. Pickup System as an Embodiment> <2. Configuration of Information Processing Device and Event Sensor> <3. Pickup Timing Arrival Determination Method as an Embodiment> <4. Processing Procedure> <5. Modification Example> <6. Summary of Embodiment> <7. The Present Technology>

[0014] <1. Pickup System as an Embodiment> FIG. 1 is a block diagram showing a configuration example of a pickup system as an embodiment configured to include an information processing device 1 according to the present technology. As shown in the figure, the pickup system as an embodiment includes an information processing device 1, a robot arm 10, and an arm control device 11.

[0015] The robot arm 10 has a hand portion 10a that functions as a suction hand at the tip of the arm. The robot arm 10 can adsorb the object to be targeted ob by pressing the hand portion 10a against the object to be targeted ob, and can pick up the object to be targeted ob adsorbed to the hand portion 10a by driving the arm.

[0016] The arm control device 11 is configured as a control device that controls the movement of the robot arm 10.

[0017] The pickup system in this example is assumed to be applied to the field of logistics, and as the target object ob, for example, a load box such as a cardboard box is assumed. Since the load box has an outer shape of a cube or a rectangular parallelepiped, it can be expressed that the shape in top view is rectangular. Also, as another expression, it can be expressed that the shape has side edges in top view.

[0018] In the pickup system of this example, the robot arm is moved to the place where the target object ob as a load box is arranged, and the target object ob is adsorbed by the hand part 10a to pick up the target object ob. Specifically, in this example, the hand part 10a is directed downward, and in that state, the robot arm 10 is moved in the horizontal direction. When picking up the target object ob, the hand part 10a is lowered and pressed against the upper surface of the target object ob, so that the target object ob is adsorbed by the hand part 10a.

[0019] At this time, in order to appropriately pick up the target object ob, it is necessary to determine whether the hand part 10a has reached a position where it can adsorb the target object ob. For this determination, in this example, similar to the conventional hand-eye method, a device having a sensing function is attached to the hand part 10a. Specifically, the information processing device 1 is attached to the hand part 10a.

[0020] As will be described later, the information processing device 1 is provided with an EVS (Event-based Vision Sensor) 2 as an event sensor, and is capable of sensing the target object ob. Here, in this specification, an event sensor means a sensor including a plurality of pixels each having a light receiving element, each pixel being configured to be able to detect a change in the light reception amount equal to or more than a predetermined amount as an event, and being able to output information indicating the position of the pixel where the event is detected and information indicating the detection time of the event.

[0021] In the information processing apparatus 1, the sensing direction of the EVS 2 is the same direction as the direction in which the hand portion 10a faces, that is, downward in this example.

[0022] In addition to the EVS 2, the information processing apparatus 1 is also provided with a signal processing unit (control unit 4 described later) and the like for performing processing for estimating the edge position of the target object ob captured within the sensing range of the EVS 2 based on the detection information of the EVS 2, and processing for determining the arrival of the pickup timing of the target object ob based on the estimated edge position. However, the configuration of the information processing apparatus 1 will be described later again.

[0023] Referring to FIG. 2, an outline of the method for determining the arrival of the pickup timing of the target object ob in the embodiment will be described. In FIG. 2, in the upper parts of FIGS. 2A, 2B, and 2C, a state where the robot arm 10 moves horizontally and gradually approaches the target object ob is schematically shown. Here, it is assumed that as the target object ob, two target objects ob, namely, the front target object ob and the rear target object ob as viewed from the robot arm 10, are arranged side by side. The "front side" and "rear side" mentioned here are based on the moving direction of the robot arm 10. Specifically, the "front side" means the side opposite to the moving direction of the robot arm 10, and the "rear side" means the side in the same direction as the moving direction of the robot arm 10.

[0024] Also, in FIGS. 2A, 2B, and 2C, in the lower part of each figure, a state of the target object ob captured within the sensing range As of the EVS 2 corresponding to the state in the upper part is schematically shown. As can be understood by referring to this lower part, FIG. 2A shows a state where only the front edge portion of the front target object ob within the sensing range As is captured, FIG. 2B shows a state where the entire front target object ob within the sensing range As and only the front edge portion of the rear target object ob are captured, and FIG. 2C shows a state where the entire front target object ob and the entire rear target object ob within the sensing range As are captured.

[0025] In the determination of the arrival of the pickup timing in this example, for the sensing range As of EVS2, a target position Pt corresponding to the pickup timing is determined in advance. This target position Pt is the projection of the central position of the hand part 10a within the sensing range As. Since the separation distance from the optical axis of EVS2 to the central position of the hand part 10a is known, based on this separation distance, the central position of the hand part 10a in the sensing range As can be obtained. The central position of the hand part 10a in the sensing range As obtained in this way is determined in advance as the target position Pt.

[0026] Here, the sensing range of EVS2 is determined based on the effective pixel range of EVS2. Therefore, as the target position Pt, it is determined as the position in the pixel coordinate system (coordinate system indicating pixel positions) of EVS2.

[0027] In the determination of the arrival of the pickup timing in this example, based on the positional relationship between the central position of the target object ob captured within the sensing range As and the target position Pt, the pickup timing of the target object ob is determined. Specifically, as shown in the lower part of FIG. 2C, the timing when the central position of the target object ob captured within the sensing range As coincides with the target position Pt is determined as the pickup timing.

[0028] Note that determining the target position Pt based on the central position of the hand part 10a is merely an example. For example, in the hand part 10a, when the central position of the suction part is offset from the central position of the hand part 10a, it is conceivable to determine the target position Pt based on the central position of the suction part. Regarding which position of the hand part 10a the target position Pt is determined based on, it can be appropriately changed according to the actual embodiment and is not limited to being based on a specific position.

[0029] <2. Configuration of Information Processing Apparatus and Event Sensor> FIG. 3 is a diagram for explaining an internal configuration example of the information processing apparatus 1, and shows the internal configuration example of the information processing apparatus 1 together with the arm control apparatus 11 shown in FIG. 1. As shown in the drawing, the information processing apparatus 1 includes an EVS 2, a memory 3, and a control unit 4.

[0030] Referring to FIGS. 4 to 6, the configuration of the EVS 2 will be described. FIG. 4 is a diagram showing an internal configuration example of the EVS 2. As shown in the drawing, the EVS 2 includes a pixel array unit 21 in which a plurality of pixels 20 are two-dimensionally arranged, an X arbiter 22 and a Y arbiter 23, an event processing circuit 24, and an output I / F (interface).

[0031] In the pixel array unit 21, a plurality of pixels 20 are arranged in the row direction (horizontal direction: X direction in the drawing) and the column direction (vertical direction: Y direction in the drawing), respectively. Each pixel 20 has a light receiving element and is configured to be able to detect an event that is a change in the light reception amount equal to or greater than a predetermined amount. In this example, each pixel 20 is configured to be able to detect, as an event, a "positive polarity event" that is an increase-side change in the light reception amount equal to or greater than a predetermined amount, and a "negative polarity event" that is a decrease-side change in the light reception amount equal to or greater than a predetermined amount.

[0032] Regardless of whether it is a positive polarity event or a negative polarity event, when each pixel 20 detects an event, it outputs a request signal to the X arbiter 22 and the Y arbiter 23 to request reading (output) of an event signal from itself. As shown in the drawing, the request signal for the X arbiter 22 is referred to as "request signal Xrq", and the request signal for the Y arbiter 23 is referred to as "request signal Yrq". Then, each pixel 20 outputs an event signal in accordance with arbitration by the X arbiter 22 and Y arbiter 23 according to the above-mentioned request signals Xrq and Yrq. Specifically, each pixel 20, in response to receiving a response signal Xac output by the X arbiter 22 as a response (ACK) to the request signal Xrq and a response signal Yac output by the Y arbiter 23 as a response to the request signal Yrq, outputs a positive-side event signal Ip, which is a signal indicating the detection result of a positive-polarity event, and a negative-side event signal Im, which is a signal indicating the detection result of a negative-polarity event, to the event processing circuit 24.

[0033] The X arbiter 22 and Y arbiter 23 arbitrate the request signals (the above-mentioned request signals Xrq and Yrq) from each pixel 20, and transmit a response based on the arbitration result (permission / non-permission of output of the event signal) to the pixel 20, which is the output source of the request signal, as the above-mentioned response signals (Xac, Yac).

[0034] Based on the event signals (in this example, the positive-side event signal Ip and the negative-side event signal Im) input from the pixel 20, the event processing circuit 24 generates event data for the pixel 20. As this event data, data including at least position information of the pixel where the event is detected (address information in the pixel coordinate system: hereinafter referred to as "position information of the event-occurring pixel") and "detection time information" indicating the detection time of the event is generated. In this example, since positive-polarity events and negative-polarity events can be detected as events, corresponding thereto, as event data, data including the above-mentioned position information of the event-occurring pixel, detection time information, and "event type information", which is information indicating the type of the detected event (distinction between positive-polarity event / negative-polarity event), is generated.

[0035] The output I / F 25 sequentially outputs the event data output from the event processing circuit 24 in row units to the outside of the EVS2, specifically, to the memory 3 shown in FIG. 3 in this example.

[0036] FIG. 5 is a diagram showing a circuit configuration example of the pixel 20. As shown in the figure, pixel 20 includes a photodiode PD as a light receiving element, a logarithmic conversion unit 31, a buffer 32, an event detection circuit 33, and an output control / reset circuit 36.

[0037] In pixel 20, the charge accumulated in photodiode PD is transferred to logarithmic conversion unit 31. Logarithmic conversion unit 31 converts the photocurrent (current corresponding to the amount of received light) obtained by photodiode PD into a voltage signal of its logarithm. Buffer 32 corrects the voltage signal input from logarithmic conversion unit 31 and outputs it to event detection circuit 33.

[0038] As shown in the figure, logarithmic conversion unit 31 includes transistor Q1, transistor Q2, and transistor Q3. Here, in this example, MOSFET (metal-oxide-semiconductor field-effect transistor) is used for each type of transistor Q included in pixel 20.

[0039] In logarithmic conversion unit 31, transistor Q1 and transistor Q3 are N-type transistors, and transistor Q2 is a P-type transistor. The source of transistor Q1 is connected to the cathode of photodiode PD, and the drain is connected to the power supply terminal (reference potential VDD). Transistors Q2 and Q3 are connected in series between the power supply terminal and the ground terminal. Also, the connection point between transistor Q2 and transistor Q3 is connected to the gate of transistor Q1 and the input terminal of buffer 32 (the gate of transistor Q5 to be described later). Also, a predetermined bias voltage Vbias is applied to the gate of transistor Q2.

[0040] The drains of transistors Q1 and Q3 are connected to the power supply side (reference potential VDD), forming a source follower circuit. The photocurrent from the photodiode PD is converted into a logarithmic voltage signal by these two source followers connected in a loop. Transistor Q2 supplies a constant current to transistor Q3.

[0041] Buffer 32 includes transistors Q4 and Q5, each of which is a P-type transistor. Transistors Q4 and Q5 are connected in series between the power supply terminal and the ground terminal. The connection point of transistors Q4 and Q5 serves as the output terminal of buffer 32, and the corrected voltage signal is output from this output terminal as a received light signal to event detection circuit 33.

[0042] Event detection circuit 33 detects a change in the received light amount as an event by obtaining the difference between the level of the currently received light signal and the reference level Lref, which is the level of the received light signal in the past. Specifically, event detection circuit 33 detects the presence or absence of an event based on whether the level (absolute value) of the difference signal representing the difference between the reference level Lref and the level of the currently received light signal is equal to or greater than a predetermined threshold. The event detection circuit 33 in this example is configured to be able to detect and distinguish between a positive-polarity event, that is, an event where the difference from the reference level Lref is positive, and a negative-polarity event, that is, an event where the difference from the reference level Lref is negative. Event detection circuit 33 generates a positive-side output voltage Vop as the output voltage indicating the detection result of a positive-polarity event and a negative-side output voltage Vom as the output voltage indicating the detection result of a negative-polarity event, respectively.

[0043] Here, event detection circuit 33 resets the reference level Lref to the level of the currently received light signal based on the reset signal RST described later. By resetting such a reference level Lref, it becomes possible to perform new event detection based on the change in the received light signal level from the time when the reset is performed. That is, the reset of the reference level Lref is equivalent to returning the event detection circuit 33 to a state where new event detection is possible.

[0044] The event detection circuit 33 includes a subtractor 34 and a quantizer 35. The subtractor 34 reduces the level of the received light signal (voltage signal) from the buffer 32 according to the reset signal RST, and outputs the received light signal after the reduction to the quantizer 35. The quantizer 35 quantizes the received light signal from the subtractor 34 to obtain an output voltage indicating the quantization result. Specifically, in this example, a positive electrode side output voltage Vop and a negative electrode side output voltage Vom are obtained.

[0045] The subtractor 34 includes a capacitor C1 and a capacitor C2, a transistor Q7 and a transistor Q8, and a reset switch SWr. The transistor Q7 is a P-type transistor, and the transistor Q8 is an N-type transistor. The transistor Q7 and the transistor Q8 are connected in series between the power supply terminal and the ground terminal to form an inverter. Specifically, the source of the transistor Q7 is connected to the power supply terminal, the drain is connected to the drain of the transistor Q8, and the source of the transistor Q8 is connected to the ground terminal. A voltage Vbdif is applied to the gate of the transistor Q8. One end of the capacitor C1 is connected to the output terminal of the buffer 32, and the other end is connected to the gate of the transistor Q7 (the input terminal of the inverter). One end of the capacitor C2 is connected to the other end of the capacitor C1, and the other end is connected to the connection point between the transistor Q7 and the transistor Q8. The reset switch SWr has one end connected to the connection point between the capacitor C1 and the capacitor C2, and the other end connected to the connection point between the transistors Q7 and Q8 and the connection point between the capacitor C2, and is connected in parallel to the capacitor C2. The reset switch SWr is a switch that is turned ON / OFF according to the reset signal RST. The inverter formed by the transistors Q7 and Q8 inverts the received light signal input through the capacitor C1 and outputs it to the quantizer 35.

[0046] Here, in the subtractor 34, let the potential generated on the buffer 32 side of the capacitor C1 at a certain point in time be the potential Vinit. And assume that at this time, the reset switch SWr is turned ON. When the reset switch SWr is ON, the side opposite to the buffer 32 of the capacitor C1 becomes a virtual ground terminal. For the sake of convenience, the potential of this virtual ground terminal is set to zero. At this time, the charge CHinit stored in the capacitor C1, when the capacitance of the capacitor C1 is Cp1, is expressed by the following [Equation 1]. CHinit = Cp1 × Vinit ··· [Equation 1] Also, when the reset switch SWr is ON, both ends of the capacitor C2 are short-circuited, so the stored charge in it becomes zero.

[0047] Next, assume that the reset switch SWr is turned OFF. If there is a change in the amount of received light, the potential on the buffer 32 side of the capacitor C1 has changed from the above-mentioned Vinit. Let the potential after the change be Vafter, then the charge CHafter stored in the capacitor C1 is expressed by the following [Equation 2]. CHafter = Cp1 × Vafter ··· [Equation 2]

[0048] On the other hand, the charge CH2 stored in the capacitor C2, when the capacitance of the capacitor C2 is Cp2 and the output voltage of the subtractor 34 is Vout, is expressed by the following [Equation 3]. CH2 = -Cp2 × Vout ··· [Equation 3]

[0049] At this time, since the total charge amounts of capacitors C1 and C2 do not change, the following [Equation 4] holds. CHinit = CHafter + CH2 ··· [Equation 4]

[0050] When substituting [Equations 1] to [3] into [Equation 4] and transforming, the following [Equation 5] is obtained. Vout = -(Cp1 / Cp2) × (Vafter - Vinit) ··· [Equation 5] [Equation 5] represents the subtraction operation of the voltage signal, and the gain of the subtraction result is Cp1 / Cp2.

[0051] From this [Equation 5], it can be seen that the subtractor 34 outputs a signal representing the difference between the level (Vinit) of the received light signal in the past and the level (Vafter) of the current received light signal. Here, the potential Vinit corresponds to the reference level Lref described above. From the above description, this potential Vinit, that is, the reference level Lref, is reset to the level of the current received light signal, in other words, the level of the received light signal at the time when the reset switch SWr is turned on, when the reset switch SWr is turned on.

[0052] The quantizer 35 includes transistors Q9, Q10, Q11, and Q12 and is configured as a 1.5-bit quantizer. Transistors Q9 and Q11 are P-type transistors, and transistors Q10 and Q12 are N-type transistors. As shown in the figure, transistor Q9 and transistor Q10, and transistor Q11 and transistor Q12 are each connected in series between a power supply terminal and a ground terminal, and the output voltage (Vout) of the subtractor 34 is input to the gates of transistors Q9 and Q11. Also, voltage Vhigh is applied to the gate of transistor Q10, and voltage Vlow is applied to the gate of transistor Q12, respectively.

[0053] At the connection point of transistor Q9 and transistor Q10, a positive-side output voltage Vop indicating the detection result of a positive-polarity event is obtained, and at the connection point of transistor Q11 and transistor Q12, a negative-side output voltage Vom indicating the detection result of a negative-polarity event is obtained. Specifically, on the side of transistors Q9 and Q10, when the level of the output voltage (Vafter - Vinit) of the subtractor 34 is equal to or higher than the positive-side threshold corresponding to voltage Vhigh, a positive-side output voltage Vop at the H level is obtained at the connection point of transistor Q9 and transistor Q10. Also, when the level of the output voltage of the subtractor 34 is lower than the positive-side threshold, a positive-side output voltage Vop at the L level is obtained. That is, at the connection point of transistor Q9 and transistor Q10, a signal indicating whether the received light amount has changed by a predetermined threshold or more in the increasing direction, that is, a positive-side output voltage Vop indicating the detection result of a positive-polarity event is obtained. Also, on the side of transistors Q11 and Q12, when the level of the output voltage of the subtractor 34 is equal to or lower than the negative-side threshold corresponding to voltage Vlow, a negative-side output voltage Vom at the H level is obtained at the connection point of transistor Q11 and transistor Q12. Also, when the level of the output voltage of the subtractor 34 is higher than the negative-side threshold, a negative-side output voltage Vom at the L level is obtained. In this way, at the connection point of transistor Q11 and transistor Q12, a signal indicating whether the received light amount has changed by a predetermined threshold or more in the decreasing direction, that is, a negative-side output voltage Vom indicating the detection result of a negative-polarity event is obtained.

[0054] The output control and reset circuit 36 outputs the request signals Xrq and Yrq for the aforementioned X arbiter 22 and Y arbiter 23, the positive-polarity event signal Ip and the negative-polarity event signal Im according to the response signals Xac and Yac, and the reset signal RST for the reset switch SWr.

[0055] FIG. 6 is a diagram for explaining an internal configuration example of the output control and reset circuit 36, and shows the event detection circuit 33 together with the internal configuration example of the output control and reset circuit 36. The output control and reset circuit 36 includes a positive-polarity memory 37p, a negative-polarity memory 37m, an output circuit 38, an OR circuit 39, and a delay element 40.

[0056] The positive-polarity output voltage Vop from the event detection circuit 33 is held at the positive-polarity output voltage Vop and input to the OR circuit 39. Also, the negative-polarity output voltage Vom from the event detection circuit 33 is held in the negative-polarity memory 37m and input to the OR circuit 39. Here, the positive-polarity memory 37p and the negative-polarity memory 37m hold values (digital values) according to the voltage levels of the input positive-polarity output voltage Vop and negative-polarity output voltage Vom. Specifically, in this example, if the voltage levels of the positive-polarity output voltage Vop and the negative-polarity output voltage Vom are at the H level, "1" is held, and if they are at the L level, "0" is held.

[0057] The OR circuit 39 outputs a signal at the H level when at least either of the input positive-polarity output voltage Vop and negative-polarity output voltage Vom is at the H level, and outputs a signal at the L level when both are at the L level. That is, the OR circuit 39 sets the output signal to the H level in response to the detection of either a positive-polarity event or a negative-polarity event, and sets the output signal to the L level when no event has been detected. The output signal from such an OR circuit 39 is transmitted to the X arbiter 22 and Y arbiter 23 as the aforementioned request signals Xrq and Yrq, respectively.

[0058] When both the response signal Xac from the X arbiter 22 and the response signal Yac from the Y arbiter 23 are input, the output circuit 38 outputs the value held in the positive electrode side memory 37p and the value held in the negative electrode side memory 37m to the event processing circuit 24 as a positive electrode side event signal Ip and a negative electrode side event signal Im, respectively. When a positive polarity event is detected, "1" is output as the positive electrode side event signal Ip and "0" is output as the negative electrode side event signal Im, and the event processing circuit 24 can identify that a positive polarity event has been detected in the corresponding pixel 20. On the other hand, when a negative polarity event is detected, "0" is output as the positive electrode side event signal Ip and "1" is output as the negative electrode side event signal Im, and the event processing circuit 24 can identify that a negative polarity event has been detected in the corresponding pixel 20.

[0059] The delay circuit 40 delays the output signal from the OR circuit 39 and outputs it as a reset signal RST to the event detection circuit 33 (reset switch SWr). Thereby, in response to the detection of either a positive polarity event or a negative polarity event, the above-mentioned reference level Lref is reset, and the event detection circuit 33 is reset to a state where new event detection is possible. Note that as the delay time of the delay circuit 40, it is sufficient to set at least a time longer than the time required for the positive electrode side output voltage Vop and the negative electrode side output voltage Vom to be held in the positive electrode side memory 37p and the negative electrode side memory 37m when the positive electrode side output voltage Vop and the negative electrode side output voltage Vom change to the H level or the L level. Thereby, it is possible to prevent detection omission of events.

[0060] As described above, the EVS2 is configured to output data including event data for the pixel 20 that has detected an event as a positive polarity event or a negative polarity event, that is, position information of the event occurrence pixel, detection time information indicating the detection time of the event, and event type information indicating whether it is a positive polarity event / negative polarity event.

[0061] In FIG. 3, the event data output by EVS2 is stored in the memory 3. The event data stored in the memory 3 in this way can be referred to by the control unit 4.

[0062] The control unit 4 is configured to have a microcomputer including, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory). The CPU executes processing according to a program stored in the ROM or a program loaded into the RAM, thereby realizing various functions of the information processing apparatus 1. In particular, the control unit 4 performs processing for estimating the edge position of the target object ob captured within the sensing range of EVS2 based on the event data stored in the memory 3, and processing for determining the arrival of the pickup timing of the target object ob based on the estimated edge position. Further, the control unit 4 gives an instruction to pick up the target object ob to the arm control device 11 in response to determining that the pickup timing has arrived. That is, an instruction to start control for causing the robot arm 10 to execute a pickup operation of the target object ob is given to the arm control device 11.

[0063] <3. Pickup Timing Arrival Determination Method as an Embodiment> As described above, in the case of using an image sensor such as an RGB sensor for sensing the target object ob in a robot arm of the hand-eye type, since imaging is performed while the target object is displaced within the sensing range, an image in which it is difficult to detect the edge position of the target object ob may be captured, and it becomes difficult to accurately determine the pickup timing of the target object ob. Therefore, in the present embodiment, a method is adopted in which the edge position of the target object ob is estimated based on the event data detected by EVS2 as an event sensor, and the arrival of the pickup timing is determined based on the edge position.

[0064] Hereinafter, specific methods will be described with reference to FIGS. 7 to 10. FIG. 7 is a functional block diagram showing the functions of the control unit 4 in the information processing apparatus 1 as an embodiment. As shown in the figure, the control unit 4 includes a movement trajectory estimation unit F1, an edge position estimation unit F2, and a timing determination unit F3.

[0065] The movement trajectory estimation unit F1 estimates the movement trajectory of the edge position of the target object ob based on the position information of the event generation pixels detected by the EVS2 within a past period before the reference time point. Specifically, based on the position information of the event generation pixels detected by the EVS2 within a past period before the reference time point while the positional relationship between the EVS2 and the target object ob changes so that the target object ob is displaced within the sensing range As of the EVS2, the movement trajectory of the edge position of the target object ob is estimated.

[0066] FIG. 8 schematically shows the event data detected by the EVS2 within a certain period while the positional relationship between the EVS2 and the target object ob changes so that the target object ob is displaced within the sensing range As. Specifically, in FIG. 8, while the positional relationship between the EVS2 and the target object ob changes so that the target object ob is displaced within the sensing range As, the event data detected by the EVS2 within a certain period is arranged in the time direction (the axial direction of time t in the figure) and shown. Here, for the sake of illustration, only the event data detected for the first target object ob is shown. Here, as shown in the figure, the sensing range As can be expressed as a range in the X - Y plane when the row direction in the EVS2 (pixel array unit 21) is X and the column direction is Y.

[0067] Since the EVS2 detects a change in the received light amount equal to or greater than a predetermined amount as an event, it will detect an event in response to the edge position of the target object ob. And here, on the premise that the target object ob is displaced within the sensing range As, as shown in the figure, the detection position of the event changes in the same direction as the displacement direction of the target object ob over time. Specifically, in the figure, the event data of the positive-polarity event detected in response to the front edge of the target object ob by the black circles and the event data of the negative-polarity event detected in response to the back edge of the target object ob by the white circles are shown. However, for any of these events, the detection position changes in the same direction as the displacement direction of the target object ob over time.

[0068] Here, the front edge can be said to be the edge that is captured prior to the sensing range As among the two edges in the displacement direction of the target object ob, and the back edge can be said to be the edge that is captured by the sensing range As after the front edge. For this reason, the event of the back edge (negative-polarity event) starts to be detected with a delay relative to the event of the front edge (positive-polarity event).

[0069] As described above, the movement trajectory estimation unit F1 estimates the movement trajectory of the edge position of the target object ob based on the position information of the event occurrence pixels detected by the EVS2 within the past period before the reference time point. Specifically, the movement trajectory estimation unit F1 in this example estimates the movement trajectory of the edge position of the target object ob based on the position information of the event occurrence pixels detected within the past period at each processing time point in the processing cycle. This can be paraphrased as estimating the movement trajectory based on the position information of the event occurrence pixels detected within the past period for each processing time point (that is, for each current time) in the processing cycle.

[0070] FIG. 9 is a diagram for explaining the method of estimating the edge position movement trajectory in the embodiment. Here, for the sake of explanation, the event data detected by the EVS2 within the past period is plotted and shown on the two-dimensional plane defined by the Y-axis and the t-axis. As shown in the figure, for both the movement trajectory of the front edge and the movement trajectory of the back edge, the movement trajectory can be estimated by fitting a linear function based on a plurality of event data detected within the past period. This fitting can be considered to be performed as a fitting using the least squares method for a plurality of event data detected within the past period.

[0071] In this example, the estimation of such an edge position movement trajectory is performed at each processing time point in the processing cycle. By estimating the edge position movement trajectory at each processing time point in this way, at each processing time point, the edge position at that processing time point can be estimated by obtaining the intersection point with the current time on the estimated edge position movement trajectory (refer to the double-circle mark in the figure).

[0072] Based on the edge position movement trajectory estimated for each processing time point as described above, the edge position estimation unit F2 (Fig. 7) obtains the intersection point with the current time on the edge position movement trajectory for each processing time point, and estimates the edge position at that processing time point. This can be expressed as estimating the reference edge as the edge position at the reference time point.

[0073] Based on the positional relationship between the reference edge position estimated by the edge position estimation unit F2 and the target position Pt defined in the pixel coordinate system of the EVS2, the timing determination unit F3 determines the arrival of the pickup timing of the target object ob. Specifically, as shown in Fig. 9, the timing determination unit F3 in this example obtains the central position between the edge position of the estimated front-side edge and the edge position of the rear-side edge at the current time (the current processing time point) (refer to the black square mark in the figure). Then, it determines whether or not this central position has reached the target position Pt as the determination of the arrival of the pickup timing. Fig. 9 illustrates a state where the central position has not yet reached the target position Pt at the current time. However, in the future, a determination result that the pickup timing has arrived can be obtained at the timing when the extension line of the target position Pt and the central position shown in the figure intersects.

[0074] Here, a specific example of the method for estimating the edge position movement trajectory in this example will be described with reference to Fig. 10. In this example, when estimating the edge position movement trajectory, a line segment Ls corresponding to the edge of the target object ob is estimated. That is, the movement trajectory estimation unit F1 in this example estimates the line segment Ls based on the latest event data (event data newly stored since the previous processing time point) stored in the memory 3 at each processing time point. While the target object ob is displaced within the sensing range As, events corresponding to the edges of the target object ob are sequentially detected at new positions. Based on the event data of the edges newly detected in this way, the line segment Ls corresponding to the edge is estimated.

[0075] Specifically, in this case, for the line segment Ls, both the front-side edge and the back-side edge of the target object ob are estimated. For the line segment Ls corresponding to the front-side edge, it can be estimated based on the event data of the newly detected positive-polarity events. Also, for the line segment Ls corresponding to the back-side edge, it can be estimated based on the event data of the newly detected negative-polarity events. Hereinafter, the line segment Ls corresponding to the front-side edge is referred to as the "front-side edge line segment", and the line segment Ls corresponding to the back-side edge is referred to as the "back-side edge line segment".

[0076] The movement trajectory estimation unit F1 in this example estimates the front-side edge line segment and the back-side edge line segment as described above at each processing time point, and stores the information of the estimated front-side edge line segment and back-side edge line segment in the memory 3. And at each processing time point, based on the information of a plurality of front-side edge line segments and a plurality of back-side edge line segments estimated within the past period, the movement trajectory of the front-side edge line segment and the movement trajectory of the back-side edge line segment are respectively estimated. Note that in FIG. 10, only for the first target object ob, the movement trajectory of the front-side edge line segment and the movement trajectory of the back-side edge line segment are shown.

[0077] Here, as can be understood by referring to FIG. 10, since the movement locus of the edge line segment is represented in a band shape, not only the inclination in the Y-t plane (refer to the inclination shown in FIG. 9), but also the inclination in the X-Y plane should be estimated. Therefore, in this case, the movement locus estimation unit F1 performs fitting of a linear function indicating the inclination in the X-Y plane as well as a linear function indicating the inclination in the Y-t plane based on the edge line segments estimated within the past period, and estimates the movement locus of the edge line segment.

[0078] In this example, the edge position estimation unit F2 estimates the front edge position and the rear edge position of the target object ob based on the movement locus of the front edge line segment and the movement locus of the rear edge line segment estimated by the movement locus estimation unit F1 as described above. Specifically, the intersection line of the movement locus of the front edge line segment and the current time is estimated as the front edge position, and the intersection line of the movement locus of the rear edge line segment and the current time is estimated as the rear edge position. Then, the timing determination unit F3 obtains the central position between the estimated edge position of the front edge and the edge position of the rear edge, and determines whether or not this central position has reached the target position Pt as the arrival determination of the pickup timing.

[0079] <4. Processing Procedure> FIG. 11 is a flowchart showing a specific example of a processing procedure for realizing the pickup timing arrival determination method as the embodiment described above. The processing shown in this FIG. 11 is executed by the control unit 4 of the information processing apparatus 1 based on a program stored in a predetermined storage device such as the aforementioned ROM.

[0080] In step S101, the control unit 4 estimates the edge line segments based on the latest event data stored in the memory 3. That is, it estimates the front-side edge line segments based on the event data of the latest positive-polarity event stored in the memory 3, and estimates the back-side edge line segments based on the event data of the latest negative-polarity event stored in the memory 3. Here, when the target object ob is not captured within the sensing range As, no event regarding the target object ob is detected. Also, there is a situation where only the front-side edge of the target object ob is captured within the sensing range As, and in that case, a positive-polarity event regarding the front-side edge is detected, but a negative-polarity event regarding the back-side edge is not detected. In the process of step S101, the control unit 4 estimates the edge line segments for the edge where the corresponding event is detected, among the front-side edge and the back-side edge.

[0081] In step S102 following step S101, the control unit 4 performs the process of storing the estimated edge line segments in the memory 3. Then, in step S103 following step S102, the control unit 4 estimates the movement trajectory of the edge line segments based on the edge line segments within the past period stored in the memory 3. Here, as understood from the previous explanation, estimating the movement trajectory of the edge line segments requires a plurality of predetermined edge line segments. In the process of step S103, if a plurality of or more front-side edge line segments are stored in the memory 3, the control unit 4 estimates the movement trajectory of the front-side edge line segments based on the plurality of front-side edge line segments, and if a plurality of or more back-side edge line segments are stored in the memory 3, it estimates the movement trajectory of the back-side edge line segments based on the plurality of back-side edge line segments.

[0082] In step S104 following step S103, the control unit 4 determines whether the movement trajectories of the edge line segments of both edges are estimated. That is, it determines whether the movement trajectories of both the front-side edge line segments and the back-side edge line segments are estimated. In step S104, if the movement trajectory of the edge line segment of any of the end sides has not been estimated, or only the movement trajectory of the front side edge line segment has been estimated and a negative result is obtained as the determination result, the control unit 4 returns to step S101. As a result, the processes of steps S101 to S103 are repeated until the movement trajectories of the edge line segments of both end sides are estimated.

[0083] On the other hand, in step S104, if the movement trajectories of both the front side edge line segment and the back side edge line segment are estimated and an affirmative result that the movement trajectories of the edge line segments of both end sides are estimated is obtained, the control unit 4 proceeds to step S105, and for each end side, the intersection line of the edge line segment movement trajectory and the current time is estimated as the current edge position. That is, the intersection line of the movement trajectory of the front side edge line segment and the current time is estimated as the current edge position of the front side edge, and the intersection line of the movement trajectory of the back side edge line segment and the current time is estimated as the current edge position of the back side edge, respectively.

[0084] In step S106 following step S105, the control unit 4 calculates the central position between the current edge positions of both end sides. That is, the central position between the current edge positions of the front side edge and the back side edge estimated in step S105 is calculated.

[0085] In step S107 following step S106, the control unit 4 determines whether the central position has reached the target position Pt. This corresponds to determining the arrival of the pickup timing of the target object ob.

[0086] In step S107, if it is determined that the central position has not reached the target position Pt, the control unit 4 returns to step S101. As a result, the estimation of the central position of the target object ob is repeated until the arrival of the pickup timing.

[0087] In one step S107, when it is determined that the central position has reached the target position Pt, the control unit 4 proceeds to step S108 and gives a pickup instruction to the arm control device 11. That is, an instruction to start control for causing the robot arm 10 to perform a pickup operation on the target object ob is given to the arm control device 11.

[0088] In response to executing the process of step S108, the control unit 4 finishes a series of processes shown in FIG. 11.

[0089] Here, in the above, an example in which the estimation of the edge position movement trajectory is performed based only on event data has been given. However, the estimation of the edge position movement trajectory can also be performed using not only event data but also information indicating the relative speed between the EVS2 and the target object ob. Specifically, when picking up the target object ob by moving only the robot arm 10 side as in this example, information indicating the moving speed of the robot arm 10 is used as the information indicating the relative speed. When estimating the inclination of the edge position movement trajectory, it becomes possible to consider the relative speed between the EVS2 and the target object ob, and it becomes possible to improve the estimation accuracy of the movement trajectory against misdetection of events. As a specific example, when fitting the inclination of the edge position movement trajectory by the least squares method or the like, information on the relative speed is used. For example, when the difference between the inclination of the edge position movement trajectory estimated from only event data and the inclination of the edge position movement trajectory obtained from the relative speed is large, correction is performed so that the inclination of the movement trajectory estimated from only event data approaches the inclination of the movement trajectory obtained from the relative speed. Or, a limit may be imposed on the inclination of the movement trajectory so that the inclination of the movement trajectory estimated from only event data does not exceed a predetermined inclination range based on the inclination of the movement trajectory obtained from the relative speed.

[0090] Also, in the above, in correspondence with the case where the EVS2 is configured to be able to detect and separate positive-polarity events and negative-polarity events, an example in which the estimation of the back-side edge line segment is performed based on the event data of the negative-polarity events has been given. Here, as the event sensor, there is also one that detects a change in the amount of received light equal to or greater than a predetermined amount as an event regardless of the polarity of the change in the amount of received light. In the following, such an event sensor will be referred to as a "non-polarity event sensor" for convenience.

[0091] Even when a non-polarity event sensor is used, the following method can be considered as a method for enabling the estimation of the back edge line segment. That is, the estimation of the back edge line segment is performed based on the event data (position information of the event-occurring pixel) detected within a pixel region where the separation distance in the back direction from the estimated position of the front edge line segment is within a predetermined distance range. In a state where the front edge line segment can be estimated, when the size of the target object ob in a top view is known and the distance from the front edge to the back edge can be specified, the back edge line segment can be predicted to be captured within a pixel region where the separation distance in the back direction from the estimated position of the front edge line segment is within a predetermined distance range. Therefore, as described above, the estimation of the back edge line segment is performed based on the position information of the event-occurring pixels detected within a pixel region where the separation distance in the back direction from the estimated position of the front edge line segment is within a predetermined distance range. Thereby, the estimation of the back edge line segment can be performed based on the event data detected within a pixel region where an event corresponding to the back edge is predicted to be detected, and the estimation accuracy of the back edge line segment can be improved.

[0092] Note that the method for estimating the back edge line segment based on the separation distance from the estimated position of the front edge line segment can also be applied when an event sensor that can detect and distinguish between positive-polarity events and negative-polarity events, such as EVS2, is used. That is, in that case, based on the event data of the detected negative-polarity events, the estimation of the back edge line segment is performed within a pixel region where the separation distance in the back direction from the estimated position of the front edge line segment is within a predetermined distance range.

[0093] <5. Modification Example> Here, the specific examples described so far are merely examples, and the present technology can adopt configurations as various variations. For example, in the above, as an example of the case where the target object ob is displaced within the sensing range As, an example where the target object ob is displaced within the sensing range As as the robot arm 10 moves has been given. However, as illustrated in FIG. 12 for example, the target object ob may be displaced within the sensing range As by being transported by a transporting means such as a conveyor.

[0094] Also, it may be possible to move the robot arm 10 side while transporting the target object ob in a predetermined direction by a conveyor or the like in this way. As a specific case, there may be a case where the moving direction of the target object ob by a conveyor or the like and the moving direction of the robot arm 10 are in the same direction. In other words, there may be a case where the robot arm 10 chases and picks up the target object ob being transported by a conveyor or the like, and a case where the moving direction of the target object ob by a conveyor or the like and the moving direction of the robot arm 10 are in opposite directions. Therefore, in this case, it is also conceivable to use information on the moving direction of the target object ob by a conveyor or the like and the moving direction of the robot arm 10 in the estimation of the edge position movement locus. Specifically, it is conceivable to perform the estimation of the edge position movement locus based on the moving speed described above, taking into account the moving direction of the target object by a conveyor or the like and the moving direction of the robot arm 10.

[0095] Note that using information on the moving direction in the estimation of the edge position movement locus can also be performed when the target object ob is not transported by a conveyor or the like. For example, when the horizontal moving direction of the robot arm 10 is not fixed, it is effective to use information on the moving direction of the robot arm 10 in the estimation of the edge position movement locus. Specifically, when the robot arm 10 can move not only in a direction orthogonal to the front side edge or the back side edge of the target object ob but also in a direction oblique to the orthogonal direction, it is effective to use information on the moving direction of the robot arm 10 in the estimation of the edge position movement locus.

[0096] Although not particularly mentioned above, it is also conceivable to feed back information indicating the positional error between the central position of the target object ob estimated based on the event data and the target position Pt to the movement control of the robot arm 10. For example, when an error is recognized between the central position of the target object ob and the target position Pt, it is conceivable to issue an instruction to the arm control device 11 to continue the movement of the robot arm 10. Alternatively, it is also conceivable to control the movement amount and movement direction of the robot arm 10 so that the positional error is canceled.

[0097] In addition, above, when estimating the central position of the target object ob, both the front edge position and the back edge position of the target object ob are estimated based on the event data. However, when estimating the central position of the target object ob, at least either the front edge position or the back edge position may be estimated based on the event data. If the size of the target object ob in a top view is known, the distance from the front edge position or the back edge position to the central position is also known. Therefore, it is possible to estimate the central position from either the front edge position or the back edge position using the information on the distance.

[0098] In addition, above, an example of applying the estimation of the edge position movement trajectory according to the present technology and the estimation of the edge position based on the edge position movement trajectory to the determination of the arrival timing of the pickup timing of the target object ob was given. However, the application targets of these estimations of the edge position movement trajectory and the edge position are not limited to the determination of the arrival timing of the pickup timing of the target object ob. For example, it is also possible to apply the estimation of the edge position movement trajectory according to the present technology and the estimation of the edge position to the determination of the timing for performing labeling, wrapping, printing, etc. on the target object ob, or to the determination of the timing for pushing out the target object ob outside the conveyor as in the example of FIG. 12.

[0099] Also, in the above, the case where the central position of the target object ob is estimated based on the edge position of the target object ob estimated from the event data was exemplified. However, the position of the target object ob estimated based on the edge position of the target object ob estimated from the event data is not limited to the central position. For example, in the above-described applications such as labeling and printing, it is only necessary to estimate the position where labeling and printing should be performed on the target object ob, and the position to be estimated is not limited to the central position.

[0100] Also, in the above, as an example of the displacement of the target object ob within the sensing range As, the example where the target object ob is displaced in the horizontal direction, that is, in the direction parallel to the sensor surface of the EVS2, was given. However, in the present technology, the "displacement" of the target object ob includes not only the displacement in the direction parallel to such a sensor surface but also the displacement in the direction orthogonal to the sensor surface (the direction of approaching and separating from the EVS2). When the target object ob is displaced in the direction orthogonal to the sensor surface, when viewed from the EVS2 side, the target object ob expands and contracts. By applying the present technology, even in a situation where the target object ob expands and contracts in this way, the edge position of the target object ob can be accurately estimated.

[0101] Also, in the above, as the information processing apparatus 1, an apparatus in which the control unit 4 that performs processing for estimating the edge position based on the event data and the EVS2 are integrally configured was exemplified. However, a configuration in which the control unit 4 and the EVS2 are separated (a configuration in which each is provided in a separate apparatus) can also be adopted.

[0102] <Summary of Embodiment> As described above, the information processing apparatus (the same 1) as an embodiment includes a movement trajectory estimation unit (the same F1) that estimates the movement trajectory of the edge position of the target object based on the position information of the event occurrence pixels detected by the event sensor within a past period before the reference time while the positional relationship between the event sensor and the target object changes so that the target object is displaced within the sensing range of the event sensor (EVS2), and an edge position estimation unit (the same F2) that estimates the edge position of the target object at the reference time as the reference time edge position based on the movement trajectory of the edge position estimated by the movement trajectory estimation unit. According to the above configuration, even when an event is misdetected, the movement trajectory of the edge position of the target object is estimated based on the position information of the event occurrence pixels detected within the past period before the reference time, so that the edge position at the reference time can be accurately estimated based on the movement trajectory. In addition, by using the event sensor, the sampling rate of the detection information used for estimating the movement trajectory of the edge position of the target object can be improved, and thus the estimation accuracy of the movement trajectory of the edge position can be improved. According to this embodiment, in these two aspects, it is possible to improve the estimation accuracy of the edge position of the target object when the target object is displaced within the sensing range of the sensor.

[0103] Further, in the information processing apparatus as an embodiment, the target object has a shape with side edges in a top view, the movement trajectory estimation unit estimates the movement trajectory of a line segment corresponding to the side edge of the target object, and the edge position estimation unit estimates the edge position as the side edge as the reference time edge position based on the movement trajectory of the line segment. Thereby, when the target object has a shape with side edges in a top view, the edge position as the side edge of the target object can be accurately estimated.

[0104] Furthermore, in the information processing apparatus according to an embodiment, the target object has a rectangular shape in a top view, the movement trajectory estimation unit estimates the movement trajectories of line segments for two opposite sides among the four side edges in the top view of the target object, and the edge position estimation unit estimates the positions of the two sides as reference-time edge positions for each of the two sides based on the movement trajectories of the line segments estimated for the two sides. By estimating the edge positions of the opposite sides of the target object as described above, it becomes possible to estimate the central position of the target object from these edge positions. For example, in the application to the hand-eye method, since the central position of the target object can be estimated, the suction hand can be appropriately pressed against the center of the target object to stably pick up the target object. If the central position of the target object can be estimated in this way, it is possible to ensure that the target object is properly handled.

[0105] Furthermore, in the information processing apparatus according to an embodiment, when, among the two sides, the side that is captured in advance within the sensing range as the target object is displaced is defined as the front-side edge and the side opposite to the front-side edge is defined as the back-side edge, the movement trajectory estimation unit performs the estimation of the line segment corresponding to the back-side edge based on the position information of the event-occurrence pixels detected within a pixel region where the separation distance in the back-side direction from the estimated position of the line segment corresponding to the front-side edge is within a predetermined distance range. In a state where the line segment of the front-side edge has been estimated, when the size of the target object in the top view is known and the distance from the front-side edge to the back-side edge can be specified, the line segment of the back-side edge can be predicted to be captured within a pixel region where the separation distance in the back-side direction from the line segment position of the front-side edge is within a predetermined distance range. Therefore, as in the above configuration, the estimation of the line segment corresponding to the back-side edge is performed based on the position information of the event-occurrence pixels detected within a pixel region where the separation distance in the back-side direction from the estimated position of the line segment corresponding to the front-side edge is within a predetermined distance range. As a result, the estimation of the line segment of the back side edge can be performed based on the event information detected within the pixel region where an event corresponding to the back side edge is predicted to be detected, and the estimation accuracy of the line segment of the back side edge can be improved.

[0106] Also, in the information processing apparatus according to the embodiment, the event sensor is configured to be able to detect both a positive-polarity event, which is an increase-side change in the received light amount by a predetermined amount or more, and a negative-polarity event, which is a decrease-side change in the received light amount by a predetermined amount or more. The movement trajectory estimation unit performs the estimation of the line segment corresponding to the back side edge based on the position information of the event generation pixels for the negative-polarity events. When the front side edge is captured within the sensing range of the event sensor, a positive-polarity event is detected. On the other hand, when the back side edge is captured within the sensing range of the event sensor, a negative-polarity event is detected. Therefore, by performing the estimation of the line segment position of the back side edge as described above based on the position information of the event generation pixels for the negative-polarity events, the line segment of the back side edge can be appropriately estimated.

[0107] Furthermore, in the information processing apparatus according to the embodiment, the movement trajectory estimation unit uses information indicating the relative speed between the event sensor and the target object in the estimation of the movement trajectory. As a result, when estimating the inclination of the edge position movement trajectory, it becomes possible to consider the relative speed between the event sensor and the target object, and it becomes possible to improve the estimation accuracy of the movement trajectory against false detection of events. By improving the estimation accuracy of the edge position movement trajectory, the estimation accuracy of the reference-time edge position can be improved.

[0108] Furthermore, in the information processing apparatus according to an embodiment, it is configured to be able to input detection information by an event sensor attached to a hand part (same 10a) of a robot arm (same 10) that picks up a target object, and based on the positional relationship between the reference edge position estimated by the edge position estimation unit and the target position (same Pt) defined in the pixel coordinate system of the event sensor, it includes a timing determination unit (same F3) that determines the arrival of the pickup timing of the target object. Thereby, it is possible to appropriately determine the arrival of the pickup timing of the target object by the robot arm based on the information on the edge position of the target object estimated from the event detection information in the past period by the event sensor.

[0109] Also, in the information processing apparatus according to an embodiment, the target object has a rectangular shape in a top view, the movement trajectory estimation unit estimates the movement trajectories of line segments for two opposite side edges among the four side edges of the target object, the edge position estimation unit estimates the positions of these two side edges as the reference edge positions for each of the two side edges based on the movement trajectories of the line segments estimated for the two side edges, the timing determination unit calculates the central position between the reference edge positions estimated for each of the two side edges, and determines the arrival of the pickup timing based on the positional relationship between the central position and the target position. Thereby, in a situation where the target object is displaced within the sensing range of the event sensor, it becomes possible to determine the arrival of the pickup timing based on the central position in the displacement direction of the target object. Therefore, it is possible to appropriately press the suction hand against the center of the target object and stably pick up the target object.

[0110] Furthermore, in the information processing apparatus according to an embodiment, it is configured to include an event sensor. Thereby, the information processing apparatus is configured as an integrated apparatus including an event sensor and a signal processing unit that estimates the edge position of the target object based on the detection information by the event sensor. By configuring the event sensor and the signal processing unit for estimating the edge position as an integrated device, it becomes unnecessary to perform inter-device wiring between the event sensor and the signal processing unit, and it is possible to reduce the mounting occupation area when mounting on a device to be mounted such as a robot arm, etc., and it is possible to improve the mountability to the device to be mounted.

[0111] Also, as an information processing method according to an embodiment, when the positional relationship between the event sensor and the target object changes such that the target object is displaced within the sensing range of the event sensor, the information processing apparatus estimates the movement locus of the edge position of the target object based on the position information of the event occurrence pixels detected by the event sensor within a past period before the reference time point, and based on the estimated movement locus of the edge position, estimates the edge position of the target object at the reference time point as the reference-time edge position. Also, by such an information processing method according to an embodiment, the same operations and effects as those of the information processing apparatus according to the above-described embodiment can be obtained.

[0112] Here, as an embodiment, it is possible to consider a program that causes a device such as a CPU, a DSP (Digital Signal Processor), or the like, or a device including these, to execute the processing of the control unit 4 described in FIG. 11 and the like above. That is, the program according to the embodiment is a program readable by a computer device, and based on the position information of the event occurrence pixels detected by the event sensor within a past period before the reference time point while the positional relationship between the event sensor and the target object changes such that the target object is displaced within the sensing range of the event sensor, it has a movement locus estimation function for estimating the movement locus of the edge position of the target object, and based on the movement locus of the edge position estimated by the movement locus estimation function, an edge position estimation function for estimating the edge position of the target object at the reference time point as the reference-time edge position, and is a program for causing a computer device to realize these functions. By such a program, the functions as the above-described control unit 4 can be realized by software processing by a computer device.

[0113] The program as described above can be pre-recorded in an HDD (Hard Disk Drive) as a recording medium built into devices such as computer devices, or in a ROM or the like in a microcomputer having a CPU. Alternatively, it can be temporarily or permanently stored (recorded) in a removable recording medium such as a flexible disk, CD-ROM (Compact Disc Read Only Memory), MO (Magneto Optical) disk, DVD (Digital Versatile Disc), Blu-ray Disc (registered trademark), magnetic disk, semiconductor memory, memory card, etc. Such a removable recording medium can be provided as so-called package software. In addition, such a program can be installed from a removable recording medium into a personal computer or the like, or can also be downloaded from a download site via a network such as a LAN (Local Area Network) or the Internet.

[0114] Also, according to such a program, it is suitable for wide provision of the information processing apparatus 1 as an embodiment. For example, by downloading the program to a personal computer, portable information processing apparatus, mobile phone, game apparatus, video apparatus, PDA (Personal Digital Assistant), etc., the personal computer or the like can be made to function as an apparatus that realizes the processing as the information processing apparatus 1 of the present disclosure.

[0115] Note that the effects described in this specification are merely examples and are not limited, and there may be other effects.

[0116] <7. The present technology> Note that the present technology can also adopt the following configuration. (1) A movement trajectory estimation unit that estimates a movement trajectory of an edge position of the target object based on position information of event occurrence pixels detected by the event sensor within a past period before a reference time while the positional relationship between the event sensor and the target object changes such that the target object is displaced within a sensing range of the event sensor; An edge position estimation unit that estimates an edge position of the target object at the reference time as a reference-time edge position based on the movement trajectory of the edge position estimated by the movement trajectory estimation unit, and an information processing apparatus. Information processing apparatus. (2) The target object has a shape with side edges in a top view, The movement trajectory estimation unit estimates a movement trajectory of a line segment corresponding to a side edge of the target object, The edge position estimation unit estimates an edge position as the side edge as the reference-time edge position based on the movement trajectory of the line segment. The information processing apparatus according to (1) above. (3) The target object has a rectangular shape in a top view, The movement trajectory estimation unit estimates movement trajectories of line segments for two opposite side edges among the four side edges in the top view of the target object, The edge position estimation unit estimates positions of the two side edges as the reference-time edge positions for each of the two side edges based on the movement trajectories of the line segments estimated for the two side edges. The information processing apparatus according to (2) above. (4) When, among the two side edges, a side edge on the side that is captured in the sensing range in advance as the target object is displaced is defined as a front-side edge and a side edge opposite to the front-side edge is defined as a back-side edge, The movement trajectory estimation unit performs estimation of the line segment corresponding to the back-side edge based on position information of event occurrence pixels detected within a pixel region where a separation distance in the back-side direction from an estimated position of the line segment corresponding to the front-side edge is within a predetermined distance range. The information processing apparatus according to (3) above. (5) The event sensor is configured to be able to detect both a positive-polarity event, which is an increase-side change in the received light amount equal to or greater than a predetermined amount, and a negative-polarity event, which is a decrease-side change in the received light amount equal to or greater than a predetermined amount. The movement trajectory estimation unit estimates the line segment corresponding to the back-side end side based on the position information of the event-occurring pixels for the negative-polarity event. The information processing apparatus according to (3) or (4) above. (6) In estimating the movement trajectory, the movement trajectory estimation unit uses information indicating the relative speed between the event sensor and the target object. The information processing apparatus according to any one of (1) to (5) above. (7) It is configured to be able to input detection information from the event sensor attached to the hand part of the robot arm that picks up the target object. A timing determination unit that determines the arrival of the pickup timing of the target object based on the positional relationship between the reference-time edge position estimated by the edge position estimation unit and the target position defined in the pixel coordinate system of the event sensor. The information processing apparatus according to any one of (1) to (6) above. (8) The target object has a rectangular shape in a top view. The movement trajectory estimation unit estimates the movement trajectories of line segments for two opposite edges out of the four edges of the target object. Based on the movement trajectories of the line segments estimated for the two edges, the edge position estimation unit estimates the positions of those two edges as the reference-time edge positions for each of the two edges. The timing determination unit calculates the central position between the reference-time edge positions estimated for each of the two edges, and determines the arrival of the pickup timing based on the positional relationship between the central position and the target position. The information processing apparatus according to (7) above. (9) Equipped with the event sensor The information processing apparatus according to any one of (1) to (8) above. (10) An information processing apparatus estimates the movement trajectory of the edge position of the target object based on the position information of event occurrence pixels detected by the event sensor within a past period before a reference time point while the positional relationship between the event sensor and the target object changes such that the target object is displaced within the sensing range of the event sensor, and estimates the edge position of the target object at the reference time point as a reference-time edge position based on the estimated movement trajectory of the edge position. Information processing method. (11) A program readable by a computer device, wherein the computer device is caused to realize a movement trajectory estimation function that estimates the movement trajectory of the edge position of the target object based on the position information of event occurrence pixels detected by the event sensor within a past period before a reference time point while the positional relationship between the event sensor and the target object changes such that the target object is displaced within the sensing range of the event sensor, and an edge position estimation function that estimates the edge position of the target object at the reference time point as a reference-time edge position based on the movement trajectory of the edge position estimated by the movement trajectory estimation function. Program.

Explanation of Signs

[0117] 1 Information processing apparatus 2 EVS 3 Memory 4 Control unit 10 Robot arm 10a Hand part 11 Arm control device ob Target object As Sensing range Pt Target position 20 Pixel 21 Pixel array part 22 X arbiter 23 Y arbiter 24 Event processing circuit 25 Output I / F Xrq, Yrq request signal Xac, Yac response signal Ip positive electrode side event signal Im negative electrode side event signal 31 Logarithmic conversion unit 32 Buffer 33 Event detection circuit 34 Subtractor 35 Quantizer 36 Output control and reset circuit PD Photodiode Q1 to Q5, Q7 to Q12 Transistors C1, C2 Capacitors SWr Reset switch Vop Positive electrode side output voltage Vom Negative electrode side output voltage 37p Positive electrode side memory 37m Negative electrode side memory 38 Output circuit 39 OR circuit 40 Delay unit F1 Moving trajectory estimation unit F2 Edge position estimation unit F3 Timing determination unit Ls Line segment

Claims

1. A movement trajectory estimation unit that estimates a movement trajectory of an edge position of the target object based on position information of event occurrence pixels detected by the event sensor within a past period before a reference time while the positional relationship between the event sensor and the target object changes such that the target object is displaced within a sensing range of the event sensor; An edge position estimation unit that estimates an edge position of the target object at the reference time as a reference time edge position based on the movement trajectory of the edge position estimated by the movement trajectory estimation unit. An information processing apparatus comprising: An information processing apparatus.

2. The target object has a shape with side edges in a top view; The movement trajectory estimation unit estimates a movement trajectory of a line segment corresponding to a side edge of the target object; The edge position estimation unit estimates an edge position as the side edge as the reference time edge position based on the movement trajectory of the line segment. The information processing apparatus according to claim 1. The information processing apparatus according to claim 1.

3. The target object has a rectangular shape in a top view; The movement trajectory estimation unit estimates movement trajectories of line segments for two opposite side edges out of four side edges in the top view of the target object; The edge position estimation unit estimates the positions of the two side edges as the reference time edge positions for each of the two side edges based on the movement trajectories of the line segments estimated for the two side edges. The information processing apparatus according to claim 2. The information processing apparatus according to claim 2.

4. When, out of the two side edges, a side edge that is captured in advance within the sensing range as the target object is displaced is defined as a front side edge, and a side edge opposite to the front side edge is defined as a rear side edge; The movement trajectory estimation unit performs estimation of the line segment corresponding to the rear side edge based on the position information of the event occurrence pixels detected within a pixel region where a separation distance in the rear side direction from the estimated position of the line segment corresponding to the front side edge is within a predetermined distance range. The information processing apparatus according to claim 3. The information processing apparatus according to claim 3.

5. The event sensor is configured to be able to detect both a positive polarity event, which is an increase-side change in the amount of received light by a predetermined amount or more, and a negative polarity event, which is a decrease-side change in the amount of received light by a predetermined amount or more; The movement trajectory estimation unit performs estimation of the line segment corresponding to the rear side edge based on the position information of the event occurrence pixels for the negative polarity event. The information processing apparatus according to claim 3. The information processing apparatus according to claim 3.

6. The movement trajectory estimation unit uses information indicating a relative speed between the event sensor and the target object in the estimation of the movement trajectory. The information processing apparatus according to claim 1.

7. configured to be capable of inputting detection information by the event sensor attached to the hand portion of the robot arm that picks up the target object, including a timing determination unit that determines the arrival of the pickup timing of the target object based on the positional relationship between the reference-edge position estimated by the edge position estimation unit and the target position defined in the pixel coordinate system of the event sensor The information processing apparatus according to claim 1.

8. the target object has a rectangular shape in a top view, the movement trajectory estimation unit estimates the movement trajectories of line segments for two opposite edges out of the four side edges of the target object, the edge position estimation unit estimates the positions of the two edges as the reference-edge positions for each of the two edges based on the movement trajectories of the line segments estimated for the two edges, the timing determination unit calculates the central position between the reference-edge positions estimated for each of the two edges, and determines the arrival of the pickup timing based on the positional relationship between the central position and the target position The information processing apparatus according to claim 7.

9. including the event sensor The information processing apparatus according to claim 1.

10. An information processing apparatus, estimates the movement trajectory of the edge position of the target object based on the position information of event-occurring pixels detected by the event sensor within a past period before a reference time while the positional relationship between the event sensor and the target object changes such that the target object is displaced within the sensing range of the event sensor, estimates the edge position of the target object at the reference time as the reference-edge position based on the estimated movement trajectory of the edge position An information processing method.

11. A program readable by a computer device, causing the computer device to realize a movement trajectory estimation function that estimates the movement trajectory of the edge position of the target object based on the position information of event-occurring pixels detected by the event sensor within a past period before a reference time while the positional relationship between the event sensor and the target object changes such that the target object is displaced within the sensing range of the event sensor, and an edge position estimation function that estimates the edge position of the target object at the reference time as the reference-edge position based on the movement trajectory of the edge position estimated by the movement trajectory estimation function A program. ​

Images