System for imaging a moving object relative to a background feature

Abstract

A system (10) for imaging a target object (12) moving relative to a background feature (14), the system (10) including an event-based vision sensor (16) operable to detect a change in a field of view (FOV) and generate an event signal in response to detecting the change, a mount (18) carrying the event-based vision sensor (16) associated with a displacement mechanism (20) operable to rotate the mount (18) about at least one axis (22, 24) to orient the FOV, and a controller configured to operate the event-based vision sensor (16) and the displacement mechanism (20) to displace the mount (18) at an intermediate tracking rate (r3) to generate an event signal to move the FOV and enable imaging of each of the target object (12) and background feature (14) moving through the FOV.

Description

[Technical Field]

[0001] The present disclosure relates generally to imaging of moving objects relative to background features, and more particularly to imaging of resident space objects moving relative to stars. [Background technology]

[0002] Space domain awareness involves monitoring resident space objects (RSOs) moving near Earth, typically satellites orbiting Earth. Monitoring space around Earth may make it possible to detect and track RSOs, catalog RSOs, and determine their position relative to other objects, such as another RSO. This makes it possible, for example, to predict collisions between RSOs or take evasive action, or to predict the re-entry of an RSO into Earth's atmosphere. Reliable and thorough space domain awareness has become increasingly important due to the significant increase in the number of man-made RSOs orbiting Earth in recent years.

[0003] Traditional space domain awareness is typically achieved by operating frame-based optical cameras coupled with telescopes to capture images of space. This approach has many drawbacks, including generating a significant amount of image data (much of which shows black sky and is useless) and capturing blurry or faint images of moving objects that are difficult to process to enable meaningful insights.

[0004] Imaging an RSO using a ground-based optical camera typically means that the star and planet are the only other features visible in the image. Because the RSO moves relative to the star, it is not possible to operate the optical system so that the RSO and star are static in the field of view, meaning one or the other is blurred. This can mean that it is difficult to identify a specific RSO, for example, by measuring its orbit. One approach to solving this is to first operate the imaging system to track the movement of the star so that the star is stationary in the field of view. This allows the actuators that move the optical camera to be calibrated. The imaging system is then operated to track the movement of the RSO so that the RSO is stationary in the field of view. This then allows the orbit of the RSO in space to be inferred. When using this approach, the accuracy of the orbit measurement depends on the quality of the calibration and the accuracy of the feedback provided by the actuators.

[0005] Any discussion of documents, acts, materials, devices, articles or the like which has been included in this specification should not be construed as an admission that any or all of that matter was common general knowledge in the art to which the present disclosure pertains by virtue of its existence prior to the priority date of each of the appended claims. Summary of the Invention

[0006] According to disclosed aspects, a system for imaging a target object moving relative to a background feature is provided, the system including: an event-based vision sensor operable to detect a change in a field of view (FOV) and generate an event signal in response to detecting the change; a mount carrying the event-based vision sensor, the mount associated with a displacement mechanism operable to rotate the mount about at least one axis to orient the FOV; and a controller configured to operate the event-based vision sensor and the displacement mechanism. The controller is configured to: determine an imaging duration (t) starting at a specific time (t0); determine a background tracking rate (r1) including at least one first vector component defining a rotation of the mount about at least one axis to keep background features stationary within the FOV; determine an object tracking rate (r2) including at least one second vector component defining a rotation of the mount about at least one axis to keep the target object stationary within the FOV during at least a portion of the imaging duration; determine an intermediate tracking rate (r3) including at least one third vector component different from the at least one first vector component and the at least one second vector component, the at least one third vector component defining a rotation of the mount about at least one axis to keep neither the object nor the background features stationary within the FOV; and, from t0, operate the displacement mechanism to displace the mount at the intermediate tracking rate (r3) during the imaging duration (t) to generate a first set of event signals to enable imaging of each of the target object and background features moving through the FOV and operate the event-based vision sensor.

[0007] The controller may be configured to determine r3 that balances r1 and r2 to generate a first set of event signals such that imaging the relative movement indicates that a velocity vector of an object moving through the FOV is substantially opposite and substantially equal in magnitude to a velocity vector of a background feature moving through the FOV.

[0008] The controller may be configured to determine r3 weighted towards r2 to generate a first set of event signals such that imaging the relative movement indicates that the velocity vector of the object moving through the FOV is substantially opposite to and greater than the velocity vector of a background feature moving through the FOV.

[0009] The controller may be configured to determine r3 weighted towards r1 to generate a first set of event signals such that imaging the relative movement indicates that the velocity vector of the object moving through the FOV is substantially opposite to and less than the velocity vector of a background feature moving through the FOV.

[0010] The controller may be configured to determine r3 as a function of time such that r3 is variable during the imaging duration. The controller may be configured to adjust r3 such that during a first portion of t, a background feature is moving through the FOV and during a second portion of t, an object is moving through the FOV.

[0011] The controller calculates the velocity of the target object within the FOV (v o ) and set the tracking coefficient (α) to v o and the controller is further configured to determine r3 based on the tracking coefficient.

[0012] The controller may be configured to determine r3 based on the position of the target object at the start of the imaging duration (t0) and to operate the displacement mechanism so that the target object is at the center of the FOV midway through the imaging duration (t0+t / 2).

[0013] The controller determines the velocity of the background feature (v1) in the FOV, the velocity of the target object (v2) in the FOV, and the velocity of the background feature (v r ), and estimate at least one of the relative velocities of the object with respect to v1, v2, and v rIn response to estimating at least one of v1, v2, or v r may be configured to determine at least one of r1 and r2 based on

[0014] The controller calculates the velocity of the target object (v2) and the background features (v r ), and periodically estimate one of the relative velocities of the target object with respect to v or v r whereby, in response to determining each r2 value, the controller may be configured to determine r3 and operate the displacement mechanism at r3.

[0015] The system may further include a processor communicatively coupled to the event-based vision sensor and configured to process the event signals to image the relative movement.

[0016] A processor may be configured to image the relative movement to show changes in the position of the object and background features over time as trails adjacent to the object and background features.

[0017] A processor may be configured to image the trail to define one or more of a color gradient and a palette of different colors defined by changes in position over time.

[0018] A processor may be configured to determine a relative position of a background feature within the FOV, and to determine a position of the target object relative to the relative position of the background feature.

[0019] The mount may be configured to be positioned at a stationary location on Earth and oriented toward the sky in a first orientation such that background features are defined by astronomical objects, and the controller is configured to determine r1 as a sidereal rate based on the stationary location and the first orientation.

[0020] The object may be a Resident Space Object (RSO) orbiting Earth, and the controller may be configured to determine r2 based on a defined orbit determination for at least a portion of the imaging duration.

[0021] A controller may be configured to determine the intermediate tracking rate based on the position of the target object, defined as right ascension (RA) and declination (Dec) coordinates, at the beginning (t0) of the imaging duration.

[0022] The mount may be configured to be located on a Resident Space Object (RSO) orbiting and directed toward the Earth, and the controller is configured to determine r1 based on the orbit rate of the RSO.

[0023] The mount may be configured to be disposed on a first resident space object (RSO) orbiting the Earth and directed toward a second RSO, and the controller is configured to determine r1 based on the orbit rate of the first RSO.

[0024] According to another disclosed aspect, a method is provided for imaging a target object moving relative to a background feature using an event-based vision sensor operable to detect changes in a field of view (FOV). The method includes: determining an imaging duration (t) starting at a specific time (t0); determining a background tracking rate (r1) including at least one first vector component defining a rotation of the sensor about at least one axis to keep background features stationary within the FOV; determining an object tracking rate (r2) including at least one second vector component defining a rotation of the sensor about at least one axis to keep the target object stationary within the FOV during at least a portion of the imaging duration; determining an intermediate tracking rate (r3) including at least one third vector component different from the at least one first vector component and the at least one second vector component, the at least one third vector component defining a rotation about at least one axis to keep neither the object nor the background features stationary within the FOV; and operating the event-based vision sensor to detect changes in the FOV during the imaging duration (t) to generate a first set of event signals to enable the sensor to move at the intermediate tracking rate (r3) to image each of the target object and background features moving through the FOV, from t0. It will be appreciated that the method may be embodied as computer instructions that may be encoded within an application, the instructions being configured to direct the operation of a system to image a target object as described in the previous paragraph.

[0025] Throughout this specification, the word "comprise" or variations thereof (e.g., "comprises" or "comprising") will be understood to mean the inclusion of a stated element, integer, or step, or group of elements, integers, or steps, but not the exclusion of any other element, integer, or step, or group of elements, integers, or steps.

[0026] It will be understood that embodiments may include the steps, features, and / or integers disclosed herein or indicated individually or collectively in the specification of this application, and any and all combinations of two or more of such steps or features.

[0027] Embodiments will now be described, by way of example only, with reference to the accompanying drawings, in which: [Brief explanation of the drawings]

[0028] [Figure 1] 1 is a schematic diagram of a first embodiment of a system for tracking an object moving relative to a background feature; [Figure 2] FIG. 2 is a schematic diagram of a second embodiment of a system for tracking an object moving relative to a background feature. [Figure 3] This is the first image of the RSO moving relative to the star. This image is rendered from data captured by the event-based vision sensor of the system shown in Figure 1. [Figure 4] FIG. 4 is a second image of the RSO and stars shown in FIG. 3, rendered from data captured by the event-based vision sensor of the system shown in FIG. 1 and processed to add annotations identifying the spatial relationships between the six stars and the RSO. [Figure 5] Image of another RSO moving relative to the star. This image was rendered from data captured by the event-based vision sensor of the system shown in Figure 1 and processed to add a trail to depict temporal changes. [Figure 6] A further image of an RSO moving relative to the star and rotating around its own axis, rendered from data captured by the event-based vision sensor of the system shown in Figure 1, and processed to add a trail depicting temporal changes. DETAILED DESCRIPTION OF THE INVENTION

[0029] In the drawings, reference numeral 10 generally designates a system 10 for imaging a target object 12 moving relative to a background feature 14. The system 10 includes an event-based vision sensor 16 operable to detect a change in a field of view (FOV) and generate an event signal in response to detecting the change, a mount 18 carrying the operable event-based vision sensor 16 and associated with a displacement mechanism 20 operable to rotate the mount 18 about at least one axis 22, 24 to orient the FOV, and a controller configured to operate the event-based vision sensor 16 and the displacement mechanism 20.

[0030] The controller determines an imaging duration (t) starting at a particular time (t0), determines a background tracking rate (r1) including at least one first vector component defining a rotation of the mount 18 about at least one axis 22, 24 to keep the background feature 14 stationary within the FOV, and determines an object tracking rate (r2) including at least one second vector component defining a rotation of the mount 18 about at least one axis 22, 24 to keep the target object 12 stationary within the FOV during at least a portion of the imaging duration, and determines at least one second vector component different from the at least one first vector component and the at least one second vector component. determining an intermediate tracking rate (r3) including at least one third vector component, the at least one third vector component defining a rotation of the mount 18 about at least one axis 22, 24 so that neither the object 12 nor the background feature 14 is stationary within the FOV; and, from t0, operating the displacement mechanism 20 to displace the mount 18 at the intermediate tracking rate (r3) to move the FOV and operate the event-based vision sensor 16 during the imaging duration (t) to generate a first set of event signals to enable imaging of each of the target object 12 and background feature 14 moving through the FOV.

[0031] FIG. 1 illustrates a first embodiment 100 of the system 10 in which the mount 18 is configured as a robotic altitude (also referred to as altitude-azimuth or azimuth-altitude) telescope mount 102 carrying multiple telescopes 104. An event-based vision sensor 16 is attached to one of the telescopes 104 such that the optics of the telescope 104 define the FOV of the sensor 16. The mount 102 defines two axes of rotation about which the mount 102, or a portion of the mount 102, can rotate. The first axis 102 is configured to be operatively vertical, and the second axis 104 is orthogonal to the first axis 22 so as to be operatively horizontal. Rotation about the first axis 22 allows the azimuth angle (bearing) of the pointing direction of the telescope 104 to be adjusted, thereby adjusting the azimuth angle of the direction of the FOV of the sensor 16. Rotation about the second axis 24 allows the elevation angle of the pointing direction of the telescope 104 to be adjusted, and consequently, the elevation angle of the direction of the FOV of the sensor 16. In some embodiments (not illustrated), the mount 102 is rotatable about only a single axis, or about three axes. For example, if the tracked object 12 moves along a path having a constant altitude, it may be necessary to pivot the FOV of the sensor 16 only about the first vertical axis 22. In other embodiments (not illustrated), the mount 18 is additionally or alternatively configured to provide linear displacement of the FOV, for example, by sliding the sensor 16 and associated telescope 104 along a track or rail.

[0032] The displacement mechanism 20 of the mount 102 includes a pair of drive motors (not shown) associated with axes 22, 24 and operable to rotate the mount 102 or a portion of the mount 102 about each of the axes 22, 24. The controller of the illustrated embodiment 100 is operable to precisely control the operation of each drive motor to rotate the mount 102, or a portion thereof, about each of the axes 22, 24 to freely orient the FOV of the sensor 16 over a wide range. The controller is typically configured to generate control signals to drive the motors in response to receiving or determining right ascension (RA) and declination (Dec) coordinates, such as those related to the desired direction in which to orient the FOV of the sensor 16. In this embodiment, the controller includes or is configured as an application executed by a processor onboard or proximate to the mount 102, such as an edge computing device. In other embodiments, the controller is remotely hosted or executed by one or more remote processors and controls the operation of the mount 102 by communicating instructions to the mount 102, such as via the internet. In a further embodiment, the controller is hosted or executed by a processor located on-board the mount 102 and remotely from the mount 102 in a distributed computing configuration. For example, in this configuration of the controller, the majority of the processing may be performed remotely by a powerful processor, with only a portion of the processing being performed locally by a basic processor.

[0033] The embodiment 100 of FIG. 1 is configured for use from a location on Earth 112, where the mount 102 is typically fixed in a static position, to enable the FOV of the sensor 16 to be directed toward the sky 114, as illustrated by arrow 116. In some embodiments, the mount 102 is fixed to or carried by a portable structure, such as a shipping container or vehicle. Use of such a system 100 enables imaging of a target object 12 moving across the sky and within the FOV of the sensor 16. As shown in FIG. 1 , the target object 12 may be a satellite 106 orbiting Earth 112 and moving across a known or predicted orbit 108 against a backdrop of stars 110. It will be understood that although the universe is expanding and therefore the stars are moving relative to one another, this is happening very slowly, so that to a human observer, the stars 110 define constant relative positions to one another, providing a static frame of reference for the movement of the satellite 106.

[0034] 2 illustrates an alternative embodiment 200 of system 10 configured for use from the sky or space 201, in which a mount (not shown) is carried by an airborne structure such as a drone or resident space object (RSO), which in this embodiment is a satellite 204 orbiting Earth 206. In this application, the FOV of sensor 16 can be directed toward Earth 206, as illustrated by arrow 208, to image a target object 12, e.g., a vehicle 210 or an animal, that is moving relative to other objects 14 on or near Earth 206 and has a static relative position, such as two or more buildings 212 and / or geographic landmarks, to define a constant frame of reference. Alternatively, the FOV of sensor 16 can be directed across space 201, as illustrated by arrow 214, to image another RSO, such as another satellite 216 orbiting Earth 206 against a background of stars 218.

[0035] It will be appreciated that in other embodiments (not illustrated), the system 10 can be configured to be located on the Earth 112 to image ground-based moving objects of interest 12, such as vehicles or animals. For example, in some embodiments, the system 10 can be configured to monitor the movement of insects, such as for agricultural purposes.

[0036] An event-based vision sensor 16 is a vision sensor operable to detect changes within its FOV. Event-based vision sensors 16 generally have high temporal resolution, are operable to image on the move, produce low data rates for sparsely populated scenes, and have high intra-frame dynamic range, making them useful for space domain recognition applications. It will be appreciated that event-based vision sensors 16 differ from conventional frame-based vision sensors that capture frames (images) based on the detection of light at a defined frequency. Event-based vision sensors 16 are nonlinear, i.e., they do not operate at a defined frequency, as described below, but instead generate a signal only when a change is detected.

[0037] An event-based vision sensor 16 typically includes many pixels, each operable independently of the others to act as a change detector. The sensor 16 may be configured such that each detected change causes the sensor 16 to generate an event signal if the pixel's generated photocurrent changes by more than a defined percentage from the level at which it last emitted a change event. An "on" event signals an increase in photocurrent, while an "off" event signals a decrease in photocurrent. These two types of events each have separate parameters that control the percentage change required to emit an event signal.

[0038] System 10 typically includes or is communicatively coupled to a processor (not shown) configured to process event signals received from sensors 16 to generate images. The processor may be further configured to generate video from the images. The processor may also be configured to annotate the images to add text and / or graphics, such as to identify or classify objects 12. Exemplary images are shown in Figures 3-6 and described in more detail below.

[0039] The system 10 can be configured to move the sensor 16 to enable tracking. Tracking involves manipulating the displacement mechanism 20 to pivot the sensor 16 about at least one axis 22, 24 to adjust the orientation of the sensor's 16 FOV. The tracking movement is typically performed at a tracking rate, defined as a vector having a rotational component that defines movement about at least one axis. In the illustrated embodiment 100, the tracking rate includes two vector components, one for each axis 22, 24, to define the pan and tilt values ​​of the telescope 104 and the sensor 16. The vector components are determined by an associated processor or by a controller and used to control the movement of the displacement mechanism 20. The tracking rate may be measured in degrees / second or arc-seconds / second.

[0040] The tracking motion provided by the displacement mechanism 20 may be configured to focus on the movement of the object 12, or a portion of the object 12, such as a feature defined by the object 12, within the FOV so as to maintain the object 12 within the FOV. This may involve moving the FOV at a tracking rate, referred to as the object tracking rate (r1), that matches the movement of the object 12 so that the object 12 remains stationary within the FOV. In embodiments 100 in which the target object 12 is a satellite 106, r1 may be derived from the orbit determination or orbital rate of the satellite 106. Orbit determinations for known satellites are typically available from online databases and are typically defined to be valid for a particular time or period. Tracking with r1 using an event-based vision sensor 16 means that background features 14 moving within the FOV are registered by the sensor 16 as changes, but the object 12 remains stationary within the FOV, thereby not registering its movement as a change by the sensor 16. Instead, the only detected changes associated with object 12 are due to atmospheric diffraction effects and / or vibrations caused by displacement mechanism 20. As a result, visibility of object 12 is generally intermittent and / or faint / indistinct in images generated from event signals generated by sensor 16. Such images may have limited usefulness for space situational awareness unless object 12 has a significant minimum brightness to allow it to be observed.

[0041] The tracking provided by the displacement mechanism 20 may alternatively be configured to focus on tracking the movement of the background feature 14. For example, if the system 100 is positioned on the Earth 112 and oriented toward a star 14, the tracking motion may be configured to compensate for the rotation of the Earth to keep the star 110 within the FOV. This may involve moving the FOV at a tracking rate, referred to as the background tracking rate (r2), that matches the movement of the background feature 14 so that the background feature 14 remains stationary within the FOV. In embodiments 100 in which the background feature is a star 110, r2 may be derived from or equivalent to the star tracking rate. The star rate may be calculated based on the static position of the mount 102 on the Earth 112 and the direction 116 in which the sensor 16 is pointed into the sky 114. Tracking with r2 using an event-based vision sensor 16 means that the object 12 moves within the FOV, which registers as changes by the sensor 16, but the background feature 14 remains stationary within the FOV, causing only detected changes due to atmospheric diffraction effects and / or vibrations of the displacement mechanism 20. As a result, the visibility of the background feature 14 is generally intermittent and / or faint / indistinct in images generated from the event signals generated by the sensor 16. Such images may have limited usefulness for space situational awareness unless the background feature 14 has a significant minimum brightness to allow it to be observed.

[0042] The tracking provided by the displacement mechanism 20 can alternatively be configured to be at an intermediate rate (r3) different from r1 and r2. Tracking at r3 means that neither the background feature 14 nor the object 12 is stationary within the FOV; instead, both the background feature 14 and the object 12 pass through the FOV during a period of time. Tracking at r3 using an event-based vision sensor 16 means that the background feature 14 and the object 12 move within the FOV, which is registered by the sensor 16 as a change. As a result, both are generally visible, even at low brightness, in images generated from the event signals generated by the sensor 16.

[0043] The mean tracking rate (r3) of a particular target object 12 and a particular background feature 14 may vary within an optimal range defined by the sensor 16. Motion of the sensor 16 outside the optimal range means that the object 12 and / or background feature 14 moves at a speed greater or less than the response time of the sensor 16, which means that the sensor 16 does not detect the change and does not generate an event signal.

[0044] The operation of the system 10 for enabling tracking and imaging of the target object 12 includes determining an imaging duration (t) starting at a particular time (t0), determining r1 for a particular background feature 14 such that the background tracking rate (r1) includes at least one first vector component defining a rotation of the sensor 16 about at least one axis 22, 24 to keep the background feature 14 stationary within the FOV, and determining an object tracking rate (r2) including at least one second vector component defining a rotation of the sensor 16 about at least one axis 22, 24 to keep the target object 12 stationary within the FOV during at least a portion of t, and determining an object tracking rate (r2) including at least one second vector component defining a rotation of the sensor 16 about at least one axis 22, 24 to keep the target object 12 stationary within the FOV, and determining the at least one first vector component and and determining an intermediate tracking rate (r3) including at least one third vector component different from the at least one second vector component, the at least one third vector component defining a rotation about at least one axis 22, 24 such that neither the object 12 nor the background feature 14 is stationary within the FOV during at least a portion of t, from t, moving the sensor 16 by r3 to move the FOV, operating the event-based vision sensor 16 to detect changes, and generating a first set of event signals through t to allow each of the target object 12 and the background feature 14 to pass through the FOV. It will be understood that these steps may be embodied as computer instructions and programmed into a processor-executable application, and thus may be performed by systems other than the system 10 described above.

[0045] In some embodiments of the system 10, the controller is configured to determine r3 that balances r1 and r2 to generate a first set of event signals such that imaging the relative movement indicates that the velocity vector of the object 12 moving through the FOV should be substantially opposite to, and of substantially equal magnitude to, the velocity vector of the background feature 14 moving through the FOV. This can result in an image in which the motion of the object 12 is clearly opposite to the motion of the background feature. This can enhance detection of the object 12.

[0046] In some embodiments of the system 10, the controller is configured to determine r3 weighted toward r2 to generate a first set of event signals such that imaging the relative movement indicates that the velocity vector of the object 12 moving through the FOV is substantially opposite to and greater than the velocity vector of the background feature 14 moving through the FOV. Conversely, the controller may be configured to determine r3 weighted toward r1 to generate a first set of event signals such that imaging the relative movement indicates that the velocity vector of the object 12 moving through the FOV is substantially opposite to and less than the velocity vector of the background feature 14 moving through the FOV.

[0047] The intermediate tracking rate (r3) may be determined by the controller over the imaging duration (t) such that r3 is constant throughout t or such that r3 varies during t. In some embodiments of the system 10, the controller is configured to optimize the duration of the target object 12 moving relative to a background feature 14 within the FOV to generate an event signal, thereby determining r3 as a function of the time to image the object 12 and the background feature 14. For example, if the object 12 is an RSO, such as the satellite 106 shown in FIG. 1, the satellite 106 may move at a velocity large enough that the r3 value does not move both the star 110 and the satellite 106 across the FOV within the optimal range of the sensor 16, thus requiring adjustment or dynamic variation of r3 to enable imaging of the star 110 and the satellite 106 by the sensor 16.

[0048] As described above, movement of the sensor 16 outside the optimal range, e.g., at a very high r3, means that the object 12 and / or background feature 14 move at a speed greater than the response time of the sensor 16, meaning that the sensor 16 does not detect the changes defined by the object 12 and / or background feature 14. As a result, the imaging shows only the object 12, only the background feature 14, or nothing at all. In this scenario, one approach to generating a useful image is for the controller to recalculate and adjust the intermediate tracking rate (r3) as a function of time during the imaging duration (t), so that for a portion of t, such as the beginning of the period and / or the end of the duration, the sensor 16 tracks at a first intermediate rate (r3b), so that the background feature 14 moves across the FOV within the optimal range, and for another portion of t, such as the middle period of the duration, the sensor 16 tracks at a second intermediate rate (r3t), so that the target object 12 moves across the FOV within the optimal range. Recalculating the r3 value is generally based on the known r1 and r2 values, as described above.

[0049] To enhance signal generation, and consequently, image quality, the controller may be further configured to transition the tracking rate from r3b to r3t (and potentially back to r3b). It will be appreciated that various functions of time provide a smooth transition between tracking rates r3b, r3t, such as a linear function or a function approximating a sigmoidal function. The approach of adjusting r3 during t allows for combining imagery captured in response to event signals generated at different portions of t to image a target object 12 moving relative to background features 14.

[0050] In some embodiments of the system 10, the controller determines the velocity (v o ) and set the tracking coefficient (α) to v o In such an embodiment, the controller is further configured to determine r3 based on α.

[0051] Configuring operation of the system 10 may require manually defining or automatically identifying or predicting the position of the target object 12 at t0. If the object 12 is an RSO, such as the satellite 106 shown in FIG. 1, the position may be defined as RA / Dec coordinates. Determining the position of the object 12 at t0 allows the displacement mechanism 20 to operate to orient the FOV of the sensor 16 to the position at t0 or within defined boundaries of the position at t0, ensuring that the object 12 and background features 14 pass through the FOV while moving the sensor 16 by r3 throughout the period t. This may also involve the controller being configured to determine r3 based on the target object's position at t0 and operating the displacement mechanism 20 such that the target object 12 is centered in the FOV midway through the imaging duration (t0 + t / 2).

[0052] In some applications of the system 10, the trajectory and / or velocity of the target object cannot be identified from a database or other data store; for example, the system 10 is configured to image an unidentified ground-based target object 12, such as a vehicle, or an unidentified airborne object, such as a ballistic missile. In these applications of the system 10, the controller may calculate the velocity of a background feature (v1) within the FOV, the velocity of the target object (v2) within the FOV, and the velocity of the background feature (v1) within the FOV. r ), the relative velocity of the object with respect to v1, v2, and v r In response to estimating at least one of v1, v2, or v r and determining at least one of r1 and r2 based on the

[0053] In these applications, the target object 12 may accelerate or decelerate, or otherwise move at a non-constant velocity. In such scenarios, the controller may be configured for dynamically adjusted tracking, where the controller adjusts the velocity of the target object (v2) and the background features (v r ), and periodically estimate one of the relative velocities of the target object 12 with respect to v or v r Determine r2 periodically based on 。 In response to determining each r2 value, the controller determines r3 and operates the displacement mechanism at r3, in such an embodiment, r3 varies based on periodic redetermination of r2.

[0054] In some embodiments of the system 10, the controller executes an algorithm that provides control of the displacement mechanism 20 to orient the FOV of the sensor 16. The algorithm may be configured to predict the position of the object 12 in RA / Dec coordinates at any point in time. The algorithm may be configured as follows: t is the current time, r is the target recording duration in seconds, α is a fraction of the velocity of the object 12, α=0 is background tracking, α=1 is object tracking, p(t) is the position of the object 12 in RA / Dec coordinates at time t.

[0055] Execution of this algorithm involves the controller performing the following steps: 1. The displacement of the displacement mechanism 20 is controlled and tracked according to the following equation:

number

number

number

number

[0056] In this configuration, an α value between 0 and 1 results in both the object 12 and the background features 14 moving within the FOV. In some applications, α=0.5 is preferred so that the background features 14 and the object 12 move at comparable speeds through the FOV.

[0057] During the recording period, the apparent visual velocity of the object 12 within the FOV is (1-α) times its visual velocity relative to the background feature 14. The apparent visual velocity of the background feature 14 within the FOV is α times the visual velocity of the object 12 relative to the background feature 14. The displacement mechanism 20

number

[0058] A small α value combined with a large r period may cause the object 12 to be outside the FOV of the sensor 12 at the beginning and / or end of the recording, especially if the FOV is small or if the object 12 is fast. However, the object 12 always crosses the FOV in the middle of the recording period.

[0059] In some applications, the α value may be adjusted during recording to effectively reduce the velocity of objects 12 or background features 14 within the FOV.

[0060] 3-6 show images of a target object 12, in these figures an RSO 300, moving relative to a background feature 14, in these figures a star 302. The images are generated from event signals generated by sensors 16 of system 100 operating through a time period t, according to the approach described above, and being moved by displacement mechanism 20 at an intermediate tracking rate r3. It should be understood that the same images could be generated by a system 200 guided across space 201, such as an orbiting satellite 216.

[0061] In FIG. 3 , due to the change in position of RSO 300 detected by sensor 16, RSO 300 is shown in the image as a blur extending vertically upward behind the object. Due to the change in position of stars 302 detected by sensor 16, stars 302 are shown in the image as a blur extending vertically downward behind each star 302. When multiple images produced in this manner are shown in sequence as video footage, it is apparent to an observer, or software configured to analyze the images, that RSO 300 is moving downward while stars 302 are moving upward. Furthermore, because RSO 300 is the only feature in the image moving in a different direction than stars 302, it is readily apparent to an observer or associated analysis software that RSO 300 is not star 302 and therefore requires further investigation to be able to identify RSO 300.

[0062] FIG. 4 illustrates the same RSO 300 shown in FIG. 3 at a later time during imaging period t. A processor associated with system 10 is configured to identify known positional relationships between stars 302 shown in the image based on event signals generated by sensors 16, such as by the processor consulting one or more databases. In this example, the processor has determined that some of the stars 302 in the image belong to known constellations and has added an annotation, a network of lines 304, to the image to illustrate the constellations. In this example, system 100 is configured to track the particular RSO 300 based on cataloged orbital data, typically obtained from a database, and thus identifies the RSO 300 as satellite "Beidou-3M1" of the Beidou Navigation Satellite System (BDS). Based on this data, the processor adds further annotations, including text 306 and additional lines 308, to the image to identify the RSO 300.

[0063] In some embodiments, system 100 may not be provided with information about RSO 300, or may receive this information but be configured to verify the information before adding further annotations 306, 308 to the image. In such embodiments, a processor associated with system 100 may be configured to use the constellation indicated by the network of lines 304 as a reference frame, with the processor consulting one or more databases of orbital determinations of known satellites to determine whether any satellites may be observed from the position of mount 102 orbiting beyond the constellation during the imaging period. In the example illustrated in FIG. 4 , the processor determines that a Beidou Navigation Satellite System (BDS) satellite belonging to the third generation (BDS-3) of satellites, specifically “Beidou-3M1,” is the known satellite most likely to be present in the imaged region, and consequently identifies RSO 300 as this satellite. This causes the processor to add further annotations 306, 308 to the image to identify RSO 300.

[0064] Figure 5 is an image of another RSO 400 moving relative to another star 402. In this image, the processor enhances the blur that extends behind each moving feature, displaying them as trails to better show temporal changes. In some applications, the trails are shown as a color gradient, or a palette of colors, that may include the full spectrum of visible light colors to enhance the communication of changes to the observer or associated analysis software. This may involve assigning different colors to different timestamps, e.g., red = t-0.2 seconds, and purple = t-1 seconds.

[0065] Figure 6 is an image of yet another RSO 500 moving relative to another star 502. In this image, the processor also enhances the motion blur to show a trail behind each moving feature. The trail associated with the RSO 500 is depicted as a dashed line, indicating that the change defined by this RSO 500 was detected intermittently. This may be because the RSO 500 is rotating on its own axis, which means that light is reflecting (sparkling) off the RSO 500 intermittently, allowing for only intermittent detection.

[0066] The system 10 involves moving the sensor 16 to adjust the orientation of the FOV while detecting changes within the FOV. The sensor 16 is moved at a defined tracking rate (r3) that is based on and different from the background tracking rate (r1) and the object tracking rate (r2). Tracking at (r3) means that the object 12 and background features 14 move across the FOV, causing changes to be detected and event signals to be generated. As a result, both the object 12 and the background features 14 are shown in the image derived from the event signal. Thus, the system 10 can enable rapid detection and potential identification of the object 12. Moving the sensor 16 during imaging can also enable scanning a large area in a fraction of time, such as searching for a moving RSO. Furthermore, because the sensor 16 only generates event signals in response to changes detected by pixels, the data generated by the system 10 is typically low, which can reduce latency or otherwise increase the efficiency of post-processing the event signals.

[0067] Those skilled in the art will appreciate that numerous variations and / or modifications may be made to the above-described embodiments without departing from the broad general scope of the present disclosure, and the present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive.

Claims

1. 1. A system for imaging a target object moving relative to a background feature, comprising: an event-based vision sensor operable to detect a change in a field of view (FOV) and to generate an event signal in response to detecting the change; a mount carrying the event-based vision sensor, the mount associated with a displacement mechanism operable to rotate the mount about at least one axis to orient the FOV; a controller configured to operate the event-based vision sensor and the displacement mechanism, At a specific time (t ０ ) determining an imaging duration (t) starting at a background tracking rate (r) including at least one first vector component defining a rotation of the mount about the at least one axis to keep the background feature stationary within the FOV; １ ) is determined, an object tracking rate (r) including at least one second vector component defining a rotation of the mount about the at least one axis to keep the object stationary within the FOV during at least a portion of the imaging duration; ２ ) is determined, an intermediate tracking rate (r) including at least one third vector component different from the at least one first vector component and the at least one second vector component; ３ ), wherein the at least one third vector component defines a rotation of the mount about the at least one axis so that neither the object nor the background feature is stationary within the FOV; t ０ from the intermediate tracking rate (r ３ and a controller configured to move the FOV and operate the event-based vision sensor during the imaging duration (t) to displace the FOV by a predetermined distance (t) and generate a first set of event signals to enable imaging each of the target object and the background feature moving through the FOV.

2. the controller generating the first set of event signals such that imaging the relative movement indicates that a velocity vector of the object moving through the FOV is substantially opposite and substantially equal in magnitude to a velocity vector of the background feature moving through the FOV. １ and ２ Striking a balance between ３ The system of claim 1 configured to determine:

3. and the controller generates the first set of event signals such that imaging the relative movement indicates that a velocity vector of the object moving through the FOV is substantially opposite to and greater than a velocity vector of the background feature moving through the FOV. ２ weighted towards r ３ The system of claim 1 configured to determine:

4. and the controller generates the first set of event signals such that imaging the relative movement indicates that a velocity vector of the object moving through the FOV is substantially opposite to and less than a velocity vector of the background feature moving through the FOV. １ weighted towards r ３ The system of claim 1 configured to determine:

5. The controller ３ is variable during the imaging duration, as a function of time. ３ The system of claim 1 configured to determine:

6. The controller calculates the target object velocity (v ｏ ) and the tracking coefficient (α) is determined by v ｏ and the controller is configured to derive the r based on the tracking coefficient. ３ 10. A system according to any one of the preceding claims, further configured to determine:

7. The controller determines the imaging duration (t ０ ) based on the position of the target object at the start of ３ and determining whether the target object is within the imaging duration (t ０ 10. The system of claim 9, wherein the system is configured to operate the displacement mechanism so that the FOV is centred midway through (t + t / 2).

8. The controller detects the background features (v １ ) in the FOV, ２ ) and the background feature (v ｒ estimating at least one of the relative velocities of the object with respect to v １ , v ２ , and v ｒ In response to estimating the at least one of １ , v ２ , or v ｒ Based on this, r １ and r ２ 10. A system according to any one of the preceding claims, configured to determine at least one of:

9. The controller controls the target object (v ２ ) and the background feature (v ｒ periodically estimating one of the relative velocities of the target object with respect to v ２ or v ｒ Based on this, ２ whereby each r ２ In response to determining the value of r ３ Determine r ３ The system of claim 8 , configured to operate the displacement mechanism with

10. 10. The system of any one of the preceding claims, further comprising a processor communicatively coupled to the event-based vision sensor and configured to process the event signals to image the relative movement.

11. The system of claim 10 , wherein the processor is configured to image the relative movement to show changes in the position of the object and background features over time as trails adjacent the object and background features.

12. The system of claim 11 , wherein the processor is configured to image the trail to define one or more of a color gradient and a palette of different colors defined by the change in position over time.

13. 13. The system of claim 10, wherein the processor is configured to determine a relative position of the background feature within the FOV, and to determine the position of the target object relative to the relative position of the background feature.

14. the mount is configured to be positioned at a stationary location on Earth and to orient the sky in a first direction such that the background feature is defined by an astronomical object, and the controller determines a time domain based on the stationary location and the first direction. １ 4. A system according to any one of the preceding claims, configured to determine as a stellar rate:

15. the object is a Resident Space Object (RSO) orbiting the Earth, and the controller determines, based on a defined orbit determination for at least a portion of the imaging duration, ２ The system of claim 14 configured to determine:

16. The controller determines whether the start of the imaging duration (t ０ 16. The system of claim 15, configured to determine the intermediate tracking rate based on a position of the target object defined as right ascension (RA) and declination (Dec) coordinates in a given direction.

17. The mount is configured to be disposed on a Resident Space Object (RSO) orbiting and directed toward the Earth, and the controller determines r based on an orbit rate of the RSO. １ The system of any one of claims 1 to 13, configured to determine:

18. The mount is configured to be positioned on a first Resident Space Object (RSO) orbiting the Earth and directed toward a second RSO, and the controller determines r based on an orbit rate of the first RSO. １ The system of any one of claims 1 to 13, configured to determine:

19. 1. A method for imaging a target object moving relative to a background feature using an event-based vision sensor operable to detect changes in a field of view (FOV), comprising: At a specific time (t ０ ) determining an imaging duration (t) starting at a background tracking rate (r) including at least one first vector component defining a rotation of the sensor about at least one axis to keep the background feature stationary within the FOV; １ ) and an object tracking rate (r) including at least one second vector component defining a rotation of the sensor about the at least one axis to keep the target object stationary within the FOV during at least a portion of the imaging duration; ２ ) and an intermediate tracking rate (r) including at least one third vector component different from the at least one first vector component and the at least one second vector component; ３ ), wherein the at least one third vector component defines a rotation about the at least one axis such that neither the object nor the background feature is stationary within the FOV; t ０ From the above, the sensor is adjusted to the intermediate tracking rate (r ３ and operating the event-based vision sensor to detect changes by moving the FOV during the imaging duration (t) to generate a first set of event signals to enable imaging each of the target object and background features moving through the FOV.

Images