Tracking camera, tracking camera system and operation thereof

Abstract

A tracking camera is disclosed, including a mirror assembly and an event camera, each communicatively coupled to a control unit. The control unit receives event data from the event camera and adjusts the mirror assembly based on the event data. The lens assembly includes at least one lens between the mirror assembly and the event camera.

Description

[0001] This application claims priority to European Patent Application EP 20208562.7, filed on November 19, 2020, entitled “Tracking camera, tracking camera system and operation thereof”, the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present disclosure relates to optical imaging devices, in particular cameras and camera assemblies for tracking objects. Examples relate to methods and apparatus for tracking objects. BACKGROUND

[0003] Cameras can be used to record optical images, including moving objects. Tracking cameras can record optical images and can also track the motion of objects in space. For example, ball tracking systems, such as ball tracking cameras, can be used for sports such as tennis and golf and can track the motion of a ball in space. Tracking moving objects with visual sensors such as cameras presents a number of technical challenges, such as with respect to responsiveness and optical resolution of the tracking system / method. There can be a need to design tracking devices and methods in order to be able to track fast moving / accelerating objects, in particular small and / or low optical contrast objects, and to improve optical images obtained with tracking cameras. SUMMARY

[0004] In view of the technical challenges in designing tracking devices and methods capable of tracking fast moving or accelerating objects, a tracking camera is disclosed herein.

[0005] A tracking camera is disclosed herein, comprising a mirror assembly and an event camera, each communicatively coupled to a control unit. The control unit receives event data from the event camera and adjusts the mirror assembly based on the event data. A tracking system comprising the tracking camera is disclosed.

[0006] A method of operating a tracking camera is disclosed herein, comprising receiving event data from an event camera, determining an adjustment signal based on the event data, transmitting the adjustment signal to a mirror assembly, and adjusting the mirror assembly according to the adjustment signal. A non-transitory computer readable medium computer program having program code for causing execution of the method when executed on a processor is disclosed. BRIEF DESCRIPTION OF DRAWINGS

[0007] Some examples of apparatus and / or methods will hereinafter be described by way of example only, and with reference to the accompanying drawings, in which:

[0008] Figure 1 A tracking camera according to embodiments described herein is shown;

[0009] Figure 2A tracking camera is shown in accordance with embodiments described herein;

[0010] Figure 3 A tracking camera is shown in accordance with embodiments described herein;

[0011] Figure 4 A block diagram of a system, method, or apparatus for tracking in accordance with embodiments described herein is shown;

[0012] Figure 5 A tracking camera is shown in accordance with embodiments described herein;

[0013] Figure 6 A method of initiating tracking in accordance with embodiments described herein is shown;

[0014] Figure 7 A block diagram of a system, method, or apparatus for tracking in accordance with embodiments described herein is shown;

[0015] Figure 8 A tracking camera is shown in accordance with embodiments described herein;

[0016] Figure 9 A tracking system in accordance with embodiments described herein is shown;

[0017] Figure 10 A tracking system in accordance with embodiments described herein is shown;

[0018] Figure 11 A tracking system in accordance with embodiments described herein is shown;

[0019] Figure 12 A hybrid image sensor in accordance with embodiments described herein is shown;

[0020] Figure 13 A tracking camera is shown in accordance with embodiments described herein;

[0021] Figure 14 A tracking camera is shown in accordance with embodiments described herein; and

[0022] Figure 15 A tracking camera is shown in accordance with embodiments described herein. DETAILED DESCRIPTION

[0023] Various examples will now be described more fully with reference to the accompanying drawings in which some examples are illustrated. The drawings are not necessarily to scale, the emphasis instead being placed on the principles of the various examples.

[0024] Figure 1A tracking camera 100 is shown in accordance with embodiments described herein, including those shown in other figures. The tracking camera 100 can track a fast moving object such as a car or a ball, among others, as a tracked object 190. The tracking camera 100 includes a mirror assembly 130 and an event camera 110, each communicatively coupled to a control unit 150. The control unit 150 receives event data 115 from the event camera 110 and adjusts the mirror assembly 130 based on the event data 115. The event camera 110 is capable of providing event data 115 to the processor 150 quickly, such that responsiveness of the tracking camera 100 is improved, particularly as compared to conventional / frame cameras. The rate at which the event camera 110 transmits event data 115, particularly asynchronous event data, to the control unit 150 can exceed the possible frame rate of a conventional camera, which can be advantageous for tracking the tracked object 190. The event camera can have a temporal resolution of hundreds of microseconds or even lower, which can greatly exceed the temporal resolution of conventional / frame cameras. The tracking camera 100 as described herein can allow recording images, measuring trajectories, angular velocities, object rotations, vibrations, and combinations thereof.

[0025] The mirror assembly 130 of the tracking camera 100 can direct light from the tracked object 190 to the event camera 110. The event camera 110 can be optionally fixedly mounted to a frame of the tracking camera 100.

[0026] The event camera 110 can provide one or more advantages over conventional / frame cameras, such as reducing the amount of data processed by the control unit 150. The tracked object 190 can cover only a small portion of an image frame of a conventional / frame camera. Since a large portion of the image frame can be occupied by a background, this can result in the conventional / frame camera imaging the tracked object 190 with poor resolution. As a result, much of the image frame data from the conventional camera can need to be filtered / ignored by a tracking algorithm. The conventional / frame camera, by transmitting the entire image frame multiple times per second, can burden the tracking algorithm with a large amount of data to filter / ignore.

[0027] The control unit 150 of the tracking camera 100 can be a circuit, a computing device, a CPU, a programmable field array, or the like. The control unit 150 can be programmed to perform some or all of the methods described herein, such as determining mirror adjustments of the mirror assembly 130 based on the event data 115 from the event camera 110, particularly enabling tracking of the tracked object 190.

[0028] Tracking can be improved with the event camera 110, for example by providing more relevant data to the control unit 150 to determine the position of the tracked object 190. Efficient use of computational power can also help to quickly and accurately determine optical, mechanical and / or digital adjustments for tracking the tracked object 190 and can reduce power requirements.

[0029] The event camera 110 can transmit event data 115 to the control unit 150, which is also communicatively coupled to the mirror assembly 130. The event data 115 can comprise at least one event data, which can comprise a pixel x position, a pixel y position, a polarity and a timestamp. The event data 115 can be generated asynchronously. Each individual asynchronous event can generate a set of event data comprising at least one of a pixel x position, a pixel y position, a polarity, a timestamp and any combination thereof. The term “event data” can refer to a plurality of sets of event data, for example each set of event data resulting from a respective individual (asynchronous) event. Each individual asynchronous event can be timestamped. For example, the event data 115 corresponding to a single event comprises a polarity indicating whether the event was in response to an increase or decrease in intensity; the event data also comprises a pixel position x, y and a timestamp of the event.

[0030] In one embodiment, the mirror assembly 130 has at least one actuable mirror 131. The actuable mirrors 131, 132 can be galvanometer mirrors. It is envisaged that the response time of the actuable mirrors can be less than 500 ps, 300 ps, 200 ps or 100 ps for movements of up to 0.5° or 0.2° or 0.1°. Fast responding mirrors can allow for faster tracking. In particular, a moveable mirror for tracking movement can allow for a faster response time compared to movements of the body 111 of the tracking camera, for example using a pan / tilt mechanism. Other types of mirrors for the actuable mirrors 131, 132 can be considered, for example micro electro-mechanical (system) mirrors (MEMs mirrors) and piezoelectric mirrors (PZ mirrors). Furthermore, low mass mirrors can also improve responsiveness by having lower inertia, thereby improving response speed. Optionally, the at least one actuable mirror has a maximum diameter of 0.25 cm, 0.5 cm, 1 cm, 1.5 cm or 2.5 cm. Micro mirrors are also possible.

[0031] A fast moving mirror (for example, with high bandwidth, short set-up time and / or low latency) can work in synergy with an event camera 110 (for example, with excellent temporal resolution and / or low latency) to provide fast tracking. For example, motion blur can be reduced. As described herein, in particular each of the working in synergy mirror assembly 130 and event camera 110 can allow for precise / fast bearing information of the tracked object 190.

[0032] In an embodiment, the tracking comprises a lens assembly 120 comprising at least one lens positioned between the mirror assembly 130 and the event camera 110. The at least one lens of the lens assembly 120 can be fixedly mounted with respect to the body 111 of the tracking camera. The body 111 of the tracking camera 100 can hold the event camera 110, the lens assembly 120 and the mirror assembly 130, and optionally the control unit 160. A fixed lens can provide mechanical stability. It is also contemplated that the lens assembly 120 comprises a liquid lens, which can allow for a fast adjustment of the focus. It is further contemplated that an autofocus mechanism, e.g. an event-based autofocus, is implemented.

[0033] In an embodiment, the pupil position 140 is close to or within the mirror assembly 130. This can increase the dynamic range of the tracking device. Some pupil positions 140 can provide a large change in the direction of the field of view by small mirror movements, in particular when the pupil position 140 is not in front of the lens assembly 120. As shown in Fig. 1 1, by placing the pupil position 140 close to or within the mirror assembly 130, e.g. between a pair of mirrors 131, 132 of the mirror assembly 130, the temporal resolution of the tracking camera can be increased. As mentioned above, smaller mirror movements can be performed faster / more accurately than larger mirror movements. It can be advantageous to have small, precise mirror movements that enable a significant change in the direction of the field of view and / or optical axis. Furthermore, when the mirror is placed close to the pupil position, the mirror can be smaller. Figure 2

[0034] In an embodiment, an aperture, such as a variable aperture, can be placed at the pupil position 140, such as within the mirror assembly 130, and / or between a pair of mirrors 131, 132 of the mirror assembly 130.

[0035] It is in particular contemplated to place the mirrors 131, 132 of the mirror assembly 130 (or the only mirror in case of a single mirror) at the pupil position 140 of the lens assembly 120. Alternatively / additionally, the pupil position 140 can be within 2 cm, 1 cm, 5 mm or 1 mm of the mirror assembly 130 and / or the mirrors 131, 132 of the mirror assembly 130. The response time, resolution and / or dynamic range of the tracking camera 100 can be enhanced.

[0036] ​In one embodiment, the lens assembly 120 can have a focal length of at least 70mm, 100mm, 120mm, 150mm, or 200mm, for example, particularly when the event camera is close to the size of a full format sensor (e.g., a 35mm "full frame" sensor); the focal length can be 4-10mm, for example, when the event camera is small, for example, close to the size of a mobile phone camera sensor (e.g., about 1cm wide). For example, a long focal length lens can give a narrower field of view and greater magnification (relative to a lens assembly with a shorter focal length), allowing the image of the tracked object 190 to fill the event camera 110 better for better resolution.

[0037] In embodiments, which can be combined with any other embodiments described herein, a lens assembly 120 with a large aperture and long focal length can provide a shallow depth of field. The optical combination, particularly the lens configuration of the lens assembly 120, the pupil position 140, and the pupil size, can be such that the tracked object is in focus while its environment is not. The shallow depth of field of this optical combination can allow for easier determination of the distance to the tracked object 190. When the depth of field is shallow, and the object is in focus (e.g., by auto focus), the distance (z) to the object can be determined more precisely. The x, y coordinates of the object can be determined based on the event data 115. The focus of the object can be used to determine z. Particularly when focus is used to determine the z position, 3D position information can be determined to be determined from the tracking camera 100 (e.g., a separate tracking camera 100) without the need for additional cameras.

[0038] The shallow depth of field can also effectively isolate the tracked object 190 from the foreground and background. This can simplify the data generated by the event camera 110, for example, by making the statistics of events occurring in the foreground / background uniform. For example, the shallow depth of field can allow the foreground / background events to appear as uniform background noise. The signal of events associated with the tracked object 190 (e.g., a subset of the event data 115) can be more easily identified by their higher contrast with the foreground / background events, which can be more uniformly distributed in the remainder of the event data 115 when the depth of field is shallow, for example, when the environment of the tracked object is out of focus.

[0039] Figure 2 Tracking cameras 100 according to embodiments described herein are shown, including those shown in other figures. The lens assembly 120 (e.g., 120a, 120b) can provide a focused image at the event camera 110. Various lens configurations / combinations can be considered, including Figure 2 lens configurations / combinations of Figure 1 and Figure 2 As shown in each of As shown in each ofFigure 2 As shown in FIG. 1, lens assembly 120 can include a collection lens 120a (e.g., a bi-convex collection lens 120a) on the opposite side of mirror assembly 130 from event camera 110. Lens configurations are disclosed herein, particularly in embodiments of Figure 2 As shown in FIG. 1, lens assembly 120 can include a collection lens 120a (e.g., a bi-convex collection lens 120a) on the opposite side of mirror assembly 130 from event camera 110. Lens configurations are disclosed herein, particularly in embodiments of

[0040] Figure 3 A tracking camera 300 is shown in accordance with embodiments described herein, including those shown in other figures. Figure 3 A lens assembly 320 is shown working in conjunction with mirror assembly 330 and event camera 310. As Figure 3 As shown in FIG. 1, lens assembly 120 can include a collection lens 120a (e.g., a bi-convex collection lens 120a) on the opposite side of mirror assembly 130 from event camera 110. Lens configurations are disclosed herein, particularly in embodiments of

[0041] As shown in FIG. 1, lens assembly 120 can include a collection lens 120a (e.g., a bi-convex collection lens 120a) on the opposite side of mirror assembly 130 from event camera 110. Lens configurations are disclosed herein, particularly in embodiments of Figure 3 As shown in FIG. 1, lens assembly 120 can include a collection lens 120a (e.g., a bi-convex collection lens 120a) on the opposite side of mirror assembly 130 from event camera 110. Lens configurations are disclosed herein, particularly in embodiments of Figure 3 An edge ray 351 (dashed line) is also shown to help illustrate the image circle 350. The image circle 350 can be a circle traced by the edge ray 351, which rotates around a portion of the optical axis of tracking camera 300 that extends in a straight line (in Figure 3 As shown in FIG. 1, lens assembly 120 can include a collection lens 120a (e.g., a bi-convex collection lens 120a) on the opposite side of mirror assembly 130 from event camera 110. Lens configurations are disclosed herein, particularly in embodiments of Figure 3 Overfilling of event camera 310 by the image circle 350 of lens assembly 320 is shown.

[0042] In particular, event cameras 110, 310 can have larger pixels and / or lower pixel density compared to conventional / frame cameras. The inventors have recognized that the typically smaller pixel density of event cameras can be compensated for by utilizing a smaller field of view (e.g., a smaller angle of view), which can improve the optical resolution of tracked object 190. In Figure 3 In the configuration shown in FIG. 3, image circle 350 overfills event camera 310 at the image plane. Compensating by overfilling event camera 310 at the image plane can improve the image resolution of tracked object 190. Lens assembly 320 can include a lens 321 (e.g., a lenticular lens) between event camera 310 and mirror assembly 330, a second lens 322 (e.g., a lenticular lens) on an opposite side of mirror assembly 330 from event camera 310, and a third lens 323 (e.g., a negative meniscus lens) on an opposite side of second lens 322 from mirror assembly 330.

[0043] As is apparent from the description herein, as disclosed herein, the lens configuration of lens assembly 120, 320 can work in concert with mirror assembly 130, 330 to provide better optical resolution of tracked object 190 while providing faster response times. Event camera 110, 310 can also similarly work in concert with lens assembly 120, 320 and / or mirror assembly 130, 330, particularly to improve responsiveness.

[0044] Figure 4 A block diagram of a system, method, or apparatus for tracking 400 according to embodiments described herein is shown, which can be combined with other embodiments described herein, particularly those shown in any other figure. An object of interest 490 can be in the line of sight and / or field of view of a tracking camera, particularly an event camera 410 and / or a mirror assembly 430 of the tracking camera. Mirror assembly 430 can direct light to event camera 410.

[0045] Control unit 450 can accumulate / receive event data 415 from event camera 410. Control unit 450 can determine an adjustment signal 460 based on event data 415, and can transmit adjustment signal 460 to mirror assembly 430 to adjust mirror assembly 430. Mirror assembly 430 can be adjusted so that tracked object 190 remains within the line of sight of event camera 410, e.g., so that an image of tracked object 190 overlaps and / or is within event camera 410, e.g., centered in event camera 410.

[0046] The control unit 450 can receive / accumulate event data 415 over a time interval. The control unit 450 can use the event data 415 to determine an adjustment signal 460. The control unit 450 can estimate a position array (x, x', x", t) based on the event data 415. The position array (x, x', x", t) can correspond to the tracked object 190, e.g., a position (x, y) of the tracked object 190. (Here, x can be a 1-dimensional, 2-dimensional, or even 3-dimensional position; x' and x" can be 1-dimensional, 2-dimensional, or 3-dimensional velocity and acceleration, respectively.) The position array can also include rate or speed information (e.g., x', y', |x' + y'|) and can alternatively / additionally include acceleration information (x"). The adjustment signal 460 can be based on the position array (x, x', x", t).

[0047] The position array optionally includes at least one element with respect to time, which can correspond to a prior time, a current time, or a future time. For example, the position array has elements corresponding to instantaneous coordinates (x, y, and possibly z) of the tracked object and a time of the corresponding instant / event. In another example, the position array has elements corresponding to a determined average position (e.g., x, y, and possibly z) and an average time of a plurality of events used to determine the corresponding average position.

[0048] It is conceivable that the position array is determined by the control unit 450 for a projected position of the tracked object 190 at a future time t, e.g., a future time when the mirror assembly 430 is moved to a position encoded with the adjustment signal 460. Such a determination can take into account an amount of time taken to adjust the mirror assembly 430 due to a finite response time of the mirror(s) of the mirror assembly (430). Alternatively / additionally, the position array can include at least one element with a time corresponding to a prior time. The position array can include at least one copy of a portion of the event data (e.g., x, y, t).

[0049] For example, the control unit 450 can (i) accumulate event data 415 over a time interval, then (ii) determine a common time within the time interval, and (iii) estimate a position array element (x, x', x", t) at the common time based on the accumulated event data 415. The control unit 450 can determine the adjustment signal 460 based on the position array element (x, x', x", t) at the common time c or some other time t0, in particular after the common time. The control unit 450 can determine the adjustment signal 460 based on the position array element (x, x', x", t cdetermined. Time to can correspond to a time at which the mirror is expected to reach the new position after transmission of the adjustment signal 460. Time to can be a time at which the control unit 450 transmits the adjustment signal 460 and / or receives the event data 415, which can be useful for cases in which the mirror adjustment is fast enough compared to the motion of the tracked object 190.

[0050] The above steps (i), (ii) and (iii) can be performed, and then (iv) the adjustment of the mirror assembly can be made; and steps i-iv can be repeated over subsequent time intervals. These steps can be repeated to track the object.

[0051] For example, the bearing array can be used to determine a trajectory of the tracked object 190. Alternatively / additionally, the bearing array can be used to determine the adjustment signal 460.

[0052] The control unit 450 can adjust the accumulated set of events from the event camera 410 so that the accumulated events have a common time t c The common time t c may be used for motion compensation, for example motion compensation that occurs over the duration of the data accumulation. For example, by accumulating the event data 415 (and / or mirror positions, for example using a history of adjustment signals 460 or feedback signals from the mirror assembly), a synthetic image frame can be synthesized at the common time t c .

[0053] In one embodiment, the asynchronous event data 415 from the event camera 410 can be processed asynchronously as the event data 415 is transmitted to the processor 450. For example, multiple estimates (e.g., bearing array or bearing estimates) at different points can be probabilistically combined, for example using a recursive filter such as an extended Kalman filter, which can also include a dynamic model of the object. Synchronous or asynchronous methods or control algorithms can be used to actuate the mirror (e.g., determine the adjustment signal 460).

[0054] The actuation and / or adjustment signal 460 can be based on a difference between an estimated position of the object in image coordinates (e.g., a position element corresponding to a bearing array element (e.g., x, y in a frame of reference of the event camera)) and a center of the event camera 410. The actuation and / or adjustment signal 460 can be determined so that the object (e.g., repeatedly) is centered in a field of view of the event camera 410.

[0055] The control unit 450 can determine a position array (x, x', x", t) in any coordinate system and / or any coordinate system system. The frame of reference of the event camera 410 can be particularly useful, however other coordinate systems can also be useful, particularly in tracking systems using multiple cameras. The position array can comprise coordinates related to the orientation of the mirrors of the mirror assembly 430, e.g. a mirror orientation array (0, t).

[0056] Alternatively / additionally, the adjustment signal 460 can be based on a mirror orientation array (0, t), which can be an array of previous and / or desired / target orientations of the mirrors of the mirror assembly 430. The mirror orientation array (0, t) can be determined from event data based on the event data 415 and / or other determinations such as the position array and / or feedback signals from the mirror assembly indicating mirror positions.

[0057] Alternative / additional implementations of the control system can aim to extract more information from the event camera 410 by generating patterns of rapid saccadic movements around the tracked object 190. For example, the control unit 450 can determine a plurality of adjustment signals 460 that are sequentially transmitted to the mirror assembly 430.

[0058] According to embodiments described herein, a tracking camera can be controlled / operated by a control unit. The operation can comprise receiving event data from an event camera; determining an adjustment signal based on the event data; transmitting the adjustment signal to a mirror assembly; and adjusting the mirror assembly according to the adjustment signal. The operation of the tracking camera can further comprise collecting light and forming an image of a tracked object on the event camera. The collected light can pass through a lens assembly, including a pupil position near or inside the mirror assembly (as explained herein).

[0059] The control unit 450 can estimate a position array (x, x', x", t) based on the event data 415. The position array (x, x', x", t) can correspond to the tracked object 190 (e.g. a position of the tracked object 190). The adjustment signal can be determined based on the position array.

[0060] The communication coupling between the control unit 450 and the mirror assembly 430 can be bidirectional. For example, the control unit 450 sends an adjustment signal 460 to the mirror assembly 430. Alternatively / additionally, the mirror assembly 430 and / or a directional feedback system 439 (which can be part of the mirror assembly 430) sends a mirror assembly state 435 to the control unit 450. The mirror assembly state 435 can be a position / orientation signal, e.g. data about the position and / or orientation of the mirrors of the mirror assembly 430, e.g. a mirror orientation array (0, t) and / or the mirror orientation array (0, ) of the mirror assembly.

[0061] In embodiments, which can be combined with any other embodiments described herein, the tracking camera 400 comprises a directional feedback system 439 communicatively coupled to the control unit 150 for communicating mirror assembly states, e.g. mirror positions / orientations, and / or the mirror orientation array (0, t) of the mirror assembly. The directional feedback system 439 can communicate mirror positions / orientations for at least one mirror of the mirror assembly, possibly up to each mirror of the mirror assembly, and optionally including time information such as a time stamp.

[0062] For example, the directional feedback system 439 can utilize capacitive sensing. The directional feedback system 439 can provide data to the control unit 450, which can be used to determine a position / direction of the tracked object 490, and / or a direction of an optical path leading to the tracking camera 400, and / or a position / direction of an illuminated point of the field of view 480.

[0063] Figure 5 A tracking camera 500 according to embodiments described herein is shown. Figure 5 Embodiments of the tracking camera 500 can be combined with other embodiments described herein, including embodiments shown in other figures disclosed herein. The tracking camera 500 comprises an event camera 510, a mirror assembly 530, and a control unit 550. The tracking camera 500 can have an optical axis 501 extending from the tracking camera 500 to a field of view. Based on movements of mirrors of the mirror assembly 530, the optical axis 501, in particular Figure 5 Parts of the tracking camera 500 outside the mirror assembly 530 shown in Fig. 5 are movable. In one embodiment, the optical axis 501 of the tracking camera 500 moves when the mirror assembly 530 is adjusted.

[0064] For example, the control unit 550 determines or effectively determines a target direction 502 and transmits an adjustment signal 460 so that the optical axis 501 moves towards the target direction 502. The target direction 502 can be directed at a target object 590. For example, the control unit 550 can transmit an adjustment signal 460 so that the optical axis 501 of the tracking camera 500 (which can extend to a center of the event camera 510) moves to the target direction 502 based on the orientation array (x, x', x", t) (e.g. while also extending to the center of the event camera 510). Alternatively / additionally, the adjustment signal 460 can be based on the target direction 502, which can be determined based on the event data 415.

[0065] The target direction 502 can be effectively determined such that the target object 590 overlaps or is within the field of view of the event camera 510, e.g. centered in the field of view. Alternatively / additionally, the control unit 550 can determine a plurality of target directions 502, e.g. in order to generate a pattern of rapid saccadic movements around the tracked object 590. For example, the control unit 550 can determine a plurality of adjustment signals 460, e.g. based on an accumulation of the event data 415, e.g. over the entire time interval. The plurality of adjustment signals 460 can be sequentially transmitted to the mirror assembly 530 such that the optical axis 501 is sequentially moved to each of the plurality of target directions 502.

[0066] The plurality of target directions 502 and / or the adjustment signals 460 can be such that with each movement of the optical axis 501 to each of the target directions 502, the target object 590 overlaps or is within the field of view of the event camera 510.

[0067] In one embodiment, the control unit 550 determines the plurality of target directions 502 and / or the adjustment signals 460 asynchronously from the event data 415. The optical axis 501 is effectively moved away from the target object 590, e.g. intermittently away from the target object 590, e.g. around the target object 590. In one embodiment, the optical axis 501 is effectively moved alternately away from the target object 590 and towards the target object 590. Such movements can allow to also take environmental data into account for the movement of the tracked object 590, e.g. anticipated collisions.

[0068] Returning to Figure 4 In one embodiment, the adjustment signals 460 are transmitted from the control unit 450 to the mirror assembly 430 at regular time intervals. The adjustment signals 460 can be determined by the control unit 450 based on the event data 415 accumulated between the adjustments.

[0069] Figure 6 A method of initiating tracking 600 according to embodiments described herein is shown, which can be combined with other embodiments described herein, in particular those shown in any other figure. At least part of the method 600 can be performed by a control unit as described herein. A tracking system, which can comprise at least one tracking camera as described herein, can collect light from an area 610, e.g. by scanning. The collection area, in which light is collected, can optionally be a field of view, e.g. a field of view taken by a regular / frame camera. The scanning can be digital and / or mechanical, e.g. using a mirror assembly 130.

[0070] The control unit checks for a target object 620, e.g. a moving target object, an illuminated target object, e.g. a ball, an animal and / or a vehicle. The object for tracking can be identified by the check for the target object 620. As Figure 6If the check for the target object 620 does not detect / recognize the tracked object, scanning 610 can be repeated / continued, as shown in the middle. When the tracked object is detected (e.g., at step 620), object tracking 630 can be started. For example, the control unit has a start mode that performs start tracking 600 and a tracking mode that performs tracking method 400.

[0071] In one embodiment, initial position information can be estimated / determined based on user input. In another embodiment, which can be combined with any other embodiment, the tracking system can include at least one regular / frame camera, e.g., to help collect light from the area that can identify the tracked object (e.g., step 610).

[0072] Figure 7 A block diagram of a system, method, or apparatus for tracking 400 according to embodiments described herein is shown, which can be combined with other embodiments shown in any other figure. Figure 7 Similarly to Figure 4 As Figure 4 As shown in the middle, Figure 7 Embodiments of the system, method, or apparatus for tracking 400 according to embodiments described herein are shown, which can be combined with other embodiments shown in any other figure.

[0073] In Figure 7 In embodiments of the system, method, or apparatus for tracking 400 according to embodiments described herein, the tracked object 790 can be illuminated by a light source 712, such as a laser. The light source 712 can be communicatively coupled to the control unit 750, which can control the modulation of the light source. The light source can be directed by the mirror assembly 730, e.g., the same mirror that directs the light of the field of view into the event camera 710. Alternatively, the light source 712 can be directed to the tracked object 790 by an auxiliary system, such as a mirror, that is not part of the mechanism that directs the light into the event camera 710, or a second mirror assembly (not shown). Illuminating the tracked object 790 can increase the signal-to-noise ratio and allow for more accurate tracking. Illuminating the tracked object 790 can allow tracking to occur in dark conditions. Alternatively / additionally, illuminating the tracked object 790 can allow the tracked object to be picked out from the environment more easily, especially if the light source intensity is modulated to trigger the event camera 710.

[0074] Figure 8 A tracking camera 800 according to embodiments described herein is shown, including those shown in other figures. Figure 8 A tracked object 890, a mirror assembly 830, a lens assembly 820, an event camera 810, and a control unit 850 are shown. Figure 8The tracking camera 800 shown in the middle also has a light source 812, which can be a laser, that can be used to illuminate / light the tracked object 890. The light beam from the light source 812 can be directed to the tracked object 890 through a beam splitter 813 and / or a mirror assembly 830. The light source 812 can be communicatively coupled to the control unit. Illuminating the tracked object 890 can allow tracking to occur in dark conditions. Alternatively / additionally, illuminating the tracked object 890 can allow the tracked object to be more easily picked out from the environment, especially if the light source intensity is modulated to trigger the event camera 810. The illuminated tracked object 890 can be more easily identified and / or more easily tracked.

[0075] For example, the event camera 810 can record events caused by changing illumination conditions. The modulated light source 812 can generate changing illumination conditions to trigger events of the event camera 810. Alternatively / additionally, the intensity of the light source 812 reflected from the tracked object 890 can be significantly brighter than the environment. If any portion of the light from the light source 812 misses the tracked object 890 and passes to the background, and is reflected to the event camera, that portion of the intensity can be much lower, as the brightness will be inversely proportional to the square of the distance. Points from further away in the image that are focused onto the event camera 810 can trigger fewer events than points closer to the event camera, such as those from the tracked object 890. An intensity modulated (e.g., flashing) light source can be used to illuminate the tracked object 890 and highlight the tracked object on the background / environment to facilitate identification / detection / tracking of the tracked object 890.

[0076] The tracking system can include at least one tracking camera, such as any combination of tracking cameras according to any of the embodiments described herein, such as those shown in other figures disclosed herein. Multiple tracking cameras, which can have different perspectives of a scene including a tracked object, can help determine an array of bearings, particularly 3D bearing information.

[0077] Figure 9 A tracking system 900 according to embodiments described herein is shown. The tracking system 900 can include at least one tracking camera 901, such as a tracking camera according to embodiments described herein, and an illumination assembly 902. The illumination assembly 902 can include a light source 903, such as a laser. The illumination assembly 902 can also include a mirror assembly 933 for directing the light source 903 to a tracked object 990. The tracking system 900 is similar to the tracking system 800 shown in the middle, except that the tracking system 900 does not include a beam splitter 813. Figure 8The tracking camera 800 is equivalent to the tracking camera 901. In the tracking system 900, the illumination assembly 902 can allow tracking in dark conditions. Alternatively / additionally, illuminating the tracked object 890 can allow easier picking out of the tracked object from the environment, especially if the light source intensity is modulated to trigger the event camera of the tracking camera 901. The illuminated tracked object 990 can be more easily identified and / or more easily tracked.

[0078] In an embodiment, the tracking system 900 can be such that the relative positions of the tracking cameras 901 and / or the illumination assembly 902 are known. Furthermore, the tracking cameras 901 and / or the illumination assembly 902 can be time-synchronized. The known positions and / or time-synchronization can be used to determine position information, for example a position array representing the tracked object 990 and / or an absolute position of the tracked object 990 in a coordinate system, for example a coordinate system common to each of the tracking cameras 901 and / or the illumination assembly 902.

[0079] When the relative positions of the tracking cameras 901 and / or the illumination assembly 902 are known, it can also be used to determine the positions (for example, positions in a common coordinate system and absolute positions) of features of the field of view that are not part of the tracked object 990, for example, static points and environmental points.

[0080] Figure 10 A tracking system 1000 according to embodiments described herein is shown. The tracking system 1000 can comprise at least one tracking camera 1001 according to embodiments described herein, for example those shown in other figures. The tracking camera 1001 of the tracking system 1000 can comprise an event camera 1010, a control unit 1050, a lens assembly 1020 and a mirror assembly 1030. As Figure 10 As shown in the middle, a beamsplitter 1075 (shown to be in the optical path, for example, intersecting the optical path between the tracking camera 1001 and the tracked object 1090) can allow the tracking camera 1001 and a second camera 1070 to collect light from the tracked object 1090 and / or the common field of view 1080. The beamsplitter 1075 can split light from the tracked object 1090 such that the light reaches both the tracking camera 1001 and the second camera 1070.

[0081] For example, the second camera 1070 can be a regular / frame camera (like a wide-angle camera) that can facilitate object recognition by collecting light from a region, in particular a wide region or a variable region (see, for example, Fig. 6, in particular step 610 of Fig. 6). Figure 6 The second camera 1070 can be a tracking camera comprising an event camera, which can also be used to track the tracked object 1090. The use of the tracking camera 1001 in the tracking system 1000 can facilitate the determination of 3D position information of the tracked object 1090.

[0082] Figure 11 A tracking system 1100 according to embodiments described herein is shown. The tracking system 1100 can comprise at least one tracking camera 1101, 1102, each comprising an event camera. Alternatively / additionally, the tracking system 1100 can comprise at least one regular / frame camera 1170, 1171, e.g. a wide-angle camera. The regular / frame cameras 1170, 1170 can be positioned at known relative positions. They can also be synchronized. One or more, e.g. all, cameras 1101, 1102, 1170, 1171 of the tracking system 1110 can be communicatively coupled to a control unit 1150. It is in particular contemplated to have at least one regular / frame camera 1170, 1170 with a wider field of view than the tracking cameras 1101, 1102. This can make it easier to identify a tracked object 1190, e.g. during start of tracking (see e.g. description of Figure 6 The multiple cameras can also help to facilitate 3D tracking, in particular cameras with known relative positions and / or synchronization capabilities. The synchronization of the cameras can be done, e.g., by the control unit 1150.

[0083] The regular / frame cameras 1170, 1171 can also help to estimate / determine changes in lighting conditions, which can trigger the event cameras. The control unit 1150 can process image data from the regular / frame cameras 1170, 1171 to compensate for the effect of non-uniform lighting on tracking of the tracked object 1190, e.g. trajectory estimation and / or position array (x, x', x", t) determination.

[0084] Here, in any embodiment, tracking cameras and regular / frame cameras can be used to generate images, e.g. of a tracked object. For example, event camera output can be used for tracking and / or generating images of an object, in particular with high temporal resolution. The images can be generated by intensity reconstruction. The images can have high dynamic range.

[0085] Figure 12 A hybrid image sensor 1200 according to embodiments described herein is shown. Any tracking camera described herein can comprise a hybrid image sensor 1200, which comprises an event camera 1230 and a regular camera 1240. The hybrid image sensor 1200 can comprise regular pixels 1210 for measuring light intensity of the regular camera 1240, and event pixels 1220, which are sensitive to intensity changes. The event pixels 1220 can be configured for generating event data for transmission to a control unit. The regular pixels 1210 can be configured for generating image data for transmission of image frames to a control unit.

[0086] Figure 12A coordinate system x, y, z that can be associated with the orientation arrays and / or orientation information described herein is shown, without regard to the sensor type (event camera, frame camera, or hybrid). The coordinates x and y can correspond to the rows and columns of the sensor 1200 and / or the x and y directions of the field of view of the tracked object. The coordinate z can be perpendicular to the plane of the sensor 1200, e.g., connecting the center of the camera sensor (e.g., its center) to the center of the tracked object and / or the field of view. The distance to the object can be measured along z. The coordinates x, y, and / or z can move relative to another reference frame when the mirror moves.

[0087] Figure 13 A tracking camera 1300 according to embodiments described herein is shown, including those shown with other figures, in particular Figures 1 to 12 The tracking camera 1300 includes a mirror assembly 1330 and an event camera 1310, each communicatively coupled to a control unit 1350. Figure 13 An embodiment is shown in which a pupil location 1340 is located within the lens assembly 1320. Light rays can converge at the pupil location 1340, which can be on the optical axis. As Figure 13 An aperture 1344, e.g., an aperture stop, can be placed at the pupil location 1340, as shown in the middle. In one embodiment, the mirror assembly 1330 includes a smaller mirror 1331 close to the pupil location 1340 and a larger mirror 1332 further away from the pupil location 1340. The smaller mirror 1331 can be adjusted faster than the larger mirror. The smaller mirror 1331 can have 2-axis adjustment; the larger mirror 1332 can also have 2-axis adjustment.

[0088] Figure 14 A tracking camera 1400 according to embodiments described herein is shown, including those shown with other figures, in particular Figures 1 to 12 The tracking camera 1400 includes a mirror assembly 1430 and an event camera 1410, each communicatively coupled to a control unit 1450. Figure 14 An embodiment is shown in which a pupil location 1440 is located within the lens assembly 1420. Alternatively / additionally, a pupil location 1445 can be within the mirror assembly 1430, e.g., at one of the mirrors 1431. In particular at the first pupil location 1440, there can be an aperture 1444, e.g., an aperture stop.

[0089] Light rays can converge at any of the pupil locations 1440, 1445, which can be on the optical axis. As Figure 14As shown in the middle, an aperture 1444 can be placed at the pupil location 1440. Alternatively / additionally, a mirror 1431 (such as a baffle mirror) can be placed at the pupil location 1445. The baffle can help block scattered light from reaching the detector.

[0090] In one embodiment, the mirror assembly 1430 includes a smaller mirror 1431 close to the event camera 1410 and a larger mirror 1432 further away from the event camera 1410. The smaller mirror 1431 can be adjusted faster than the larger mirror, e.g. due to smaller mass. This can improve responsiveness. At least one mirror, in particular at least the smaller mirror 1431, can have 2-axis adjustment. This can allow fast tracking in at least two dimensions.

[0091] Figure 15 A tracking camera 1500 according to embodiments described herein is shown, including with other figures, in particular Figures 1 to 12 The tracking camera 1500 includes a mirror assembly 1530 and an event camera 1510, each communicatively coupled to a control unit 1550. Figure 15 An embodiment is shown in which a pupil location 1540 is located within the lens assembly 1520 and the mirror assembly 1530. The pupil location 1540 can be within the mirror assembly 1530, e.g. at a mirror, in particular at a baffle mirror. The baffle can help block scattered light from reaching the detector.

[0092] In an embodiment, which can be combined with any other embodiment described herein, light rays can converge at a pupil location 1540, which can be on the optical axis; alternatively / additionally, the mirror assembly 1530 has at least one mirror that can be adjusted along at least 2 axes, e.g. along x and y axes. This can allow fast tracking in at least two dimensions.

[0093] Examples:

[0094] Note that the present technology can also be configured as described below.

[0095] (1) A tracking camera, comprising:

[0096] a mirror assembly communicatively coupled to the control unit; and

[0097] an event camera communicatively coupled to the control unit;

[0098] wherein,

[0099] the control unit is configured to:

[0100] receive event data from the event camera, and

[0101] Adjusting the mirror assembly based on the event data.

[0102] (2) The tracking camera of (1), further comprising:

[0103] a lens assembly comprising at least one lens between the mirror assembly and the event camera.

[0104] (3) The tracking camera of (1) or (2), wherein:

[0105] a pupil position of the lens assembly is:

[0106] within 2 cm, 1 cm, 5 mm, or 1 mm of the mirror assembly.

[0107] (4) The tracking camera of any one of (1) to (3), wherein:

[0108] the mirror assembly comprises an adjustable mirror configured for adjustment based on an adjustment signal from the control unit.

[0109] (5) The tracking camera of any one of (1) to (4), wherein:

[0110] the actuable mirror of the mirror assembly is a galvanometer mirror, a MEM mirror, or a PZ mirror.

[0111] (6) The tracking camera of any one of (1) to (5), further comprising

[0112] a directional feedback system communicatively coupled to the control unit to communicate mirror assembly status.

[0113] (7) The tracking camera of any one of (4) to (6), wherein:

[0114] the control unit is configured to:

[0115] determine the adjustment signal based on the event data, and

[0116] transmit the adjustment signal to the mirror assembly to adjust the mirror assembly.

[0117] (8) The tracking camera of (7), wherein:

[0118] the control unit is configured to:

[0119] receive the event data over a time interval, and

[0120] estimate a bearing array corresponding to the tracked object based on the event data; wherein:

[0121] the adjustment signal is based on the bearing array.

[0122] (9) The tracking camera according to (8), wherein

[0123] the control unit is configured to:

[0124] transmit an adjustment signal such that an optical axis of the tracking camera extending out of the tracking camera is moved based on the orientation array towards the target direction.

[0125] (10) The tracking camera according to any one of (1) to (9), further comprising:

[0126] a hybrid image sensor comprising an event camera and a regular camera, the hybrid image sensor comprising regular pixels for measuring light intensity of the regular camera and event pixels sensitive to intensity changes, the event pixels being configured for generating data for events for transmission to the control unit.

[0127] (11) A tracking system comprising:

[0128] a tracking camera according to any one of (1) to (10), and

[0129] a regular camera, and

[0130] a beam splitter configured to split light from a field of view to each of the regular camera and the event camera.

[0131] (12) A tracking system comprising:

[0132] a tracking camera according to any one of (1) to (10), and

[0133] a plurality of regular cameras, each regular camera being configured to be communicatively coupled to the controller and synchronized.

[0134] (13) A tracking system comprising

[0135] a plurality of tracking cameras, each tracking camera being as defined in any one of (1) to (10).

[0136] (14) A tracking system comprising

[0137] a tracking camera according to any one of (1) to (10), and

[0138] a light source, e.g. a laser, the light source being configured to illuminate a portion of a field of view of the tracking camera, e.g. for an object being tracked therein.

[0139] (15) The tracking system according to (14), further comprising:

[0140] a beam splitter intersecting an optical axis of the tracking camera and being configured to direct light from the light source out of the tracking camera, e.g. extending through a mirror assembly and along an externally positioned portion of the optical axis from the beam splitter.

[0141] (16) A method of operating a tracking camera, comprising:

[0142] receiving event data from an event camera;

[0143] determining an adjustment signal based on the event data;

[0144] transmitting the adjustment signal to a mirror assembly;

[0145] adjusting the mirror assembly according to the adjustment signal.

[0146] (17) The method of operation according to (16), further comprising:

[0147] collecting light from a pupil position within 2 cm of the mirror assembly, forming an image of the tracked object on the event camera.

[0148] (18) The method of operating a tracking camera according to any one of (16) or (17), further comprising:

[0149] estimating a bearing array based on the event data, the bearing array corresponding to the tracked object, wherein,

[0150] determining the adjustment signal is based on the bearing array.

[0151] (19) The method of operating a tracking camera according to (18), further comprising:

[0152] accumulating the event data over a time interval;

[0153] determining a common time within the time interval; wherein,

[0154] the bearing array comprises elements at the common time based on the accumulated event data.

[0155] (20) A non-transitory computer readable medium computer program having program code for causing execution of the method according to any one of (16) to (19) when executed on a processor.

[0156] Aspects and features described and recited can also be combined with one or more other examples, either to replace similar features of the other examples, or to additionally introduce the feature to the other examples. Figure 1 Aspects and features described and recited can also be combined with one or more other examples, either to replace similar features of the other examples, or to additionally introduce the feature to the other examples.

[0157] Here, flowcharts, flow diagrams, state transition diagrams, pseudocode, and the like can represent various processes, operations, or steps, which may, for example, be substantially represented in a transitory and / or non-transitory machine-readable medium (e.g., floppy disks, DVDs, Blu-ray, CDs, ROMs, PROMs, and EPROMs, EEPROMs, or flash memory) and executable by a processor or programmable hardware, whether or not such a processor or programmable hardware is explicitly shown. Methods disclosed in the specification or claims can be implemented by a device having means for performing the individual actions of these methods.

[0158] A non-transitory computer readable medium computer program can have program code which, when executed on a processor, causes the execution of any of the methods described herein.

[0159] It should be understood that the disclosure of a number of actions, processes, operations, steps, or functions in the specification or claims disclosed in the specification or claims should not be interpreted as limiting unless otherwise explicitly or implicitly specified. In some examples, the described actions, functions, processes, operations, or steps can include or can be broken down into a plurality of dependent actions, functions, processes, operations, and / or steps.

[0160] Reference signs are used to help understanding, not for limiting.

[0161] It should be understood that when a feature is referred to as being “connected” or “coupled” to another feature, that the feature can be directly connected or coupled or via one or more intervening features.

[0162] Here, an “event camera” can refer to a sensor responsive to local changes in luminance, e.g., an imaging sensor, optoelectronic circuitry for determining and / or transmitting event signals, and / or a light reactive area thereof. Here, e.g., “responsiveness” can be used to indicate response time. An increase in responsiveness can be interpreted to mean a faster response time, and vice versa. Here, the term “illumination” can be used interchangeably with “lighting”. Here, “actuatable” and “adjustable” can be used interchangeably; e.g., an adjustable mirror can be an actuatable mirror. Here, e.g., a “control unit” can be at least one processor, e.g., a computer processor, a computing device (like a programmable electronic device), etc., possibly including a network of computing devices. “Controller” and “control unit” can be used interchangeably. Here, an “azimuth array” can be a single element array, a two element array, or a larger array; e.g., an azimuth array can be x and y coordinates corresponding to a position of a tracked object in an image plane formed by a lens assembly. Here, a “conventional camera” can be used interchangeably with a “frame camera”. A conventional camera can be a high speed camera. Here, a conventional camera is different from an event camera. Here, “tracked object” and “target object” can be used interchangeably.

[0163] Here, coordinates x, y, z, e.g., as used in orientation information and / or orientation arrays, can be in a reference frame of a camera, e.g., of an event camera thereof. Coordinates x and y can be associated with x, y dimensions of a camera and / or a camera sensor, e.g., of an event camera. The z direction can be associated with a distance in front of a camera, e.g., of a tracking camera, and can be associated with a portion of an optical axis extending out of the tracking camera. In another coordinate system, e.g., an “absolute” coordinate system, e.g., of a user / observer or a second camera, the coordinates can change orientation. For example, the z direction can change in response to a change in orientation of a mirror of the tracking camera.

[0164] Here, an “aperture” can be an opening, a hole, an iris, a field stop, an aperture stop, a spatial filter, a pupil, etc. An aperture and / or a pupil can block some light and allow other light to pass. An aperture and / or a pupil can limit light passing along / near an optical axis. For example, an aperture and / or a pupil on an optical axis of a tracking camera, e.g., within a lens assembly and / or a mirror assembly, can allow light near the optical axis to pass and block light further from the optical axis, e.g., off-axis. Here, a “pupil location” can be a location where an aperture is placed, e.g., a lens aperture stop. A pupil location can be a location where light converges on an optical axis. A pupil location can be at a location of an intermediate image within a tracking camera. Here, pupil location and pupil point can be used interchangeably.

[0165] Here, one or more of the indicated alternatives is optional. Thus, for example, “one or more mirrors” means “one or more mirrors,” “at least one mirror,” or “one mirror and optional multiple mirrors.” Here, a slash “ / ” indicates “and / or,” meaning “and” or “or.” Thus, “A / B” means “A and / or B,” and likewise, “A / B / C” means “at least one of A, B, and C.”

[0166] The specification and drawings are illustrative. The specification is intended to be construed to be illustrative only and not to limit the subject matter defined in the appended claims.

Claims

1. A tracking camera, comprising: The reflector assembly is communicatively coupled to the control unit; as well as The event camera is communicatively coupled to the control unit; The mirror assembly is configured to direct light from the tracked object to the event camera; and The control unit is configured as follows: Receive event data from the event camera, and The reflector assembly is adjusted based on the event data to keep the tracked object within the line of sight of the event camera.

2. The tracking camera according to claim 1, further comprising: A lens assembly, including at least one lens between the mirror assembly and the event camera.

3. The tracking camera according to claim 2, wherein, The pupil is located within 2 cm of the reflector assembly.

4. The tracking camera according to claim 1, wherein, The mirror assembly includes an actuable mirror configured to be adjusted based on an adjustment signal from the control unit.

5. The tracking camera according to claim 4, wherein, The actuable mirror of the mirror assembly is a galvanometer mirror, a microelectromechanical mirror, or a piezoelectric mirror.

6. The tracking camera according to claim 1, further comprising: A directional feedback system is communicatively coupled to the control unit to transmit the status of the reflector assembly.

7. The tracking camera according to claim 4, wherein, The control unit is configured to: The adjustment signal is determined based on the event data, and The adjustment signal is transmitted to the mirror assembly to adjust the mirror assembly.

8. The tracking camera according to claim 7, wherein, The control unit is configured to: Receive event data within the time interval, and Based on the event data, an azimuth array corresponding to the tracked object is estimated; wherein... The adjustment signal is based on the azimuth array.

9. The tracking camera according to claim 8, wherein, The control unit is configured to: The adjustment signal is transmitted so that the optical axis of the tracking camera, which extends from the tracking camera, moves toward the target direction based on the azimuth array.

10. The tracking camera according to claim 1, further comprising: A hybrid image sensor, including the event camera and the frame camera, the hybrid image sensor including a plurality of regular pixels for measuring the light intensity of the frame camera and a plurality of event pixels sensitive to intensity changes, the event pixels being configured to generate the event data for transmission to the control unit.

11. A tracking system, comprising: The tracking camera according to claim 1, and Frame camera, and A beam splitter is configured to split the light from the field of view into each of the frame camera and the event camera.

12. A tracking system, comprising: The tracking camera according to claim 1, and Multiple frame cameras, each configured to be communicatively coupled to the controller and synchronized.

13. A tracking system, comprising Multiple tracking cameras, each of which is defined as in claim 1.

14. A tracking system, comprising The tracking camera according to claim 1, and A light source is configured to illuminate at least a portion of the field of view of the tracking camera.

15. The tracking system according to claim 14, further comprising: A beam splitter intersects the optical axis of the tracking camera, and the beam splitter is configured to guide light from the light source out of the tracking camera, extend from the beam splitter through the mirror assembly, and along the outer positioning portion of the optical axis.

16. A method of operating a tracking camera, comprising: The light from the tracked object is directed to the event camera of the tracking camera via the mirror assembly of the tracking camera; Receive event data from the event camera; The adjustment signal is determined based on the event data; The adjustment signal is transmitted to the mirror assembly; The mirror assembly is adjusted according to the adjustment signal so that the tracked object remains within the line of sight of the event camera.

17. The method of operating a tracking camera according to claim 16, further comprising: Light is collected from the pupil position within 2 cm of the reflector assembly. An image of the tracked object is formed on the event camera.

18. The method of operating a tracking camera according to claim 17, further comprising: An azimuth array is estimated based on the event data, the azimuth array corresponding to the tracked object, wherein... The adjustment signal is determined based on the azimuth array.

19. The method of operating a tracking camera according to claim 18, further comprising: Accumulate the event data within the time interval; Determine the common time within the time interval; wherein, The azimuth array includes elements based on accumulated event data during the public time.

20. A non-transitory computer-readable medium storing a computer program having program code for causing execution of the method according to claim 16 when executed on a processor.

Images