A high-precision calibration method and system for event cameras for space situational awareness

Abstract

The present invention discloses a high-precision calibration method for an event camera used for space situational awareness. The method includes: establishing a combined calibration framework of a star simulator, a high-precision turntable, and an event camera; controlling the rotation of the high-precision turntable to collect event flow information generated by relatively moving point light sources and obtain the grayscale values ​​of a grid-like trajectory image; extracting connected domains, counting the number of overlapping pixels and the horizontal and vertical pixel overlap areas, and calculating the coarse centroid position; determining a spatial window and a temporal window, determining the centroid position based on a time-interpolated grayscale centroid positioning algorithm, and performing linear fitting on the centroid position to determine the trajectory equation; calculating the coarse centroid position of all intersections and the trajectory equation of each spatial window, and calculating the sub-pixel centroid position based on the vertical trajectory equation; and optimizing the internal and external parameters of the event camera based on the corresponding relationship between the sub-pixel centroid position and the turntable angle. A corresponding system, electronic device, and computer-readable storage medium are also disclosed.

Description

Technical Field

[0001] The present invention relates to the technical field of space situational awareness and high-precision calibration methods for event cameras, and in particular to a high-precision calibration method and system for event cameras used for space situational awareness. Background Art

[0002] Space situational awareness (SSA) is a comprehensive process encompassing the acquisition and processing of space target information, situational awareness, and comprehensive monitoring of the space environment. It is fundamental to ensuring space security and guaranteeing the smooth execution of various space missions. With the rapid development and widespread application of space technology, the space environment is becoming increasingly crowded, and competition and confrontation are significantly increasing. The rise of large-scale low-orbit constellations, the increasing sophistication of space system functions, and the frequent occurrence of space events have made the space security environment more complex, volatile, and full of uncertainty. Consequently, higher requirements are being placed on tasks such as the rapid and accurate monitoring and identification of dynamic targets in space.

[0003] Event cameras, as sensors inspired by biological vision mechanisms, abandon the traditional camera method of capturing images at a fixed frame rate. Their core output is a series of event streams. The event information includes the timestamp of the brightness change, the spatial location, and the direction of the change (increase or decrease). Compared with traditional cameras, event cameras have significant advantages, including high temporal resolution, high dynamic range, low latency, and low energy consumption. These features effectively make up for the shortcomings of traditional cameras in specific application scenarios (such as space situational awareness (SSA)). To achieve accurate recognition, rapid positioning, and continuous monitoring of space targets, a core challenge lies in the precise calibration of camera parameters.

[0004] Currently, there are two main methods for event camera calibration. One involves using an LED screen of known size to dynamically generate a flickering checkerboard pattern or a synchronously flickering LED calibration plate. These dynamic visual stimuli are effectively captured by the event camera, which then identifies feature points similar to those used in traditional camera calibration, thereby achieving event camera calibration. Another strategy involves keeping the calibration target stationary and moving the event camera to collect the event stream generated by the target pattern. This strategy then employs image grayscale reconstruction techniques based on the event stream, or directly extracts features efficiently from the event stream data. Once target-related feature information has been successfully extracted from the event stream, traditional camera calibration methods can be used to complete the event camera calibration.

[0005] There is a certain gap between the calibration accuracy of the above methods and the space surveillance performance requirements of traditional space-based visible light sensors. Therefore, in order to ensure that the event camera can achieve the same measurement accuracy as traditional cameras in space situational awareness applications, a new calibration method needs to be proposed. Summary of the Invention

[0006] The present invention aims to provide a high-precision calibration method and system for event cameras used in space situational awareness. Unlike existing calibration methods that use a checkerboard or similar target, this method utilizes a star simulator and a high-precision turntable calibration framework to meet the requirements of accurate and rapid monitoring of space targets for event cameras in space situational awareness. First, a dynamic calibration trajectory acquisition strategy is adopted based on the asynchronous imaging characteristics of event cameras, enabling the efficient and concise acquisition of a large number of calibration points. Subsequently, the sub-pixel centroid positions of the calibration points are determined in three steps. First, the coarse centroid positions of the calibration points are determined based on the overlapping pixels of the grid-like trajectory intersections. Second, the coarse centroid positions are used to determine the spatial and temporal windows. Within the temporal window, a temporal interpolation grayscale centroid location algorithm is used to determine the centroid position of each time step. Within the spatial window, a linear fitting technique is used to determine the trajectory equation. Third, the sub-pixel positions of the intersection points are calculated based on the mutually perpendicular trajectory equations, which serve as the sub-pixel positions of the calibration points. Finally, the internal and external parameters are optimized based on the one-to-one correspondence between the calibration points and the turntable angles. Comprehensive experimental results show that the calibration accuracy of the proposed method is comparable to that of traditional cameras, and is significantly improved compared to the current event camera calibration method, and can meet the requirements of space target detection accuracy.

[0007] A first aspect of the present invention is to provide a high-precision calibration method for an event camera for space situational awareness, comprising:

[0008] S1, establish a combined calibration framework of the star simulator, high-precision turntable and event camera;

[0009] S2, controls the rotation of the high-precision turntable, collects the event stream information generated by the relative motion point light source, performs time aggregation on the event stream, and obtains the grayscale value of the grid trajectory image;

[0010] S3, extracting the connected domain where the grid-shaped trajectory is located, counting the intersecting overlapping pixels in the grid-shaped trajectory and the pixel overlapping areas in the horizontal and vertical directions, and calculating the rough centroid position of the calibration point based on the pixel overlapping areas;

[0011] S4, determining a spatial window based on the position of the coarse centroid, determining a time window within the spatial window, dividing time into multiple time steps with equal steps within the time window, determining the centroid position of each time step based on a time interpolation grayscale centroid positioning algorithm, and performing a linear fit on the centroid position within the spatial window to determine a trajectory equation for the set of centroid coordinates;

[0012] S5, calculating the coarse centroid positions of all intersections in the grid trajectory and the trajectory equation in each spatial window, and then calculating the positions of the intersections based on the mutually perpendicular trajectory equations, and using them as the sub-pixel centroid positions of the calibration points;

[0013] S6, optimizing the internal and external parameters of the event camera based on the one-to-one correspondence between the sub-pixel centroid positions of the calibration points and the turntable angles.

[0014] Preferably, the combined calibration framework of S1 includes a hardware system and an imaging system, wherein:

[0015] The core components of the hardware system include a stellar simulator, a high-precision turntable, and an event camera. The stellar simulator is fixed to a marble platform and emits a collimated point light source. The event camera is mounted on the inner frame of the high-precision turntable to sense this light source. The middle and outer frames of the high-precision turntable rotate at a constant speed to adjust their angles, changing the position of the point light source on the event camera's image plane. There is a unique correspondence between the point light source's centroid position and the angles of the middle and outer frames of the high-precision turntable.

[0016] The imaging system includes an imaging mechanism and an imaging model, wherein:

[0017] (1) The imaging mechanism includes: In the event camera, each pixel continuously monitors the logarithm of the light intensity. Once the change exceeds the preset threshold C, the corresponding pixel An event will be triggered immediately Specifically as shown in formula (1):

[0018]

[0019] Where, Δt k is pixel x k The time since the last event, polarity p k ∈{+1, -1} is the sign of brightness change;

[0020] (2) The imaging model includes: the propagation process of the parallel light vector emitted by the star simulator through the event camera lens on the image plane, and the vector V is the turntable coordinate system X r -Y r -Z r The parallel light vector emitted by the star simulator under is expressed as shown in Equation (2), where α and β are the pitch and yaw angles of the vector V;

[0021]

[0022] Rotation matrix R rot Obtained by synthesizing different rotation angles of the turntable; R re is the mounting matrix between the event camera and the turntable, the event camera coordinate system X e -Y e -Z eThe expression of the parallel light vector V′ under the condition is shown in formula (3), where: and They are the rotation angles from the turntable coordinate system to the event camera coordinate system, θ x ,θ y These are the angles at which the turntable rotates the middle and outer frames;

[0023]

[0024] According to the pinhole imaging model, the projection position (u′, v′) of the parallel light vector on the pixel coordinate system uv is shown in Equation (4), where s is the scale factor, F is the lens focal length, d is the pixel size, u0 and v0 are the principal point positions of the pixel coordinate system respectively;

[0025]

[0026] On this basis, lens distortion is added, where q1 and q2 are radial distortion coefficients, p1 and p2 are tangential distortion coefficients; du and dv are the distortion magnitudes of the lens along the u and v directions of the pixel coordinate system, respectively. The final projection position (u, v) is shown in formula (5).

[0027]

[0028] By integrating formulas (2)-(5), the pixel coordinate position (u, v) and the rotation angle (θ x ,θ y ) have a one-to-one correspondence between them; based on this clear correspondence, a unified calibration algorithm for internal and external parameters is used to calibrate the 13 key parameters F, d, u0, v0, q1, q1, p1, p2, α, β, Perform optimization solution; the logic of the unified calibration algorithm for internal and external parameters is shown in formula (6):

[0029]

[0030] Preferably, the S2 includes:

[0031] S21, driving the high-precision turntable to rotate the outer frame and the middle frame respectively;

[0032] S22, the event camera obtains the event flow information generated by the relative motion point light source based on the asynchronous imaging principle, counts the number of positive events generated by each pixel, uses this as the intensity value of the pixel, and obtains the grayscale value of the grid trajectory image.

[0033] Preferably, the S3 includes:

[0034] S31, performing noise filtering on the grid trajectory image, and calculating the maximum connected domain in the image plane based on a nearest connected domain algorithm to extract the connected domain where the grid trajectory is located;

[0035] S32, acquiring pixels associated with the motion trajectory of the point light source, counting the intersecting overlapping pixels in the grid-like trajectory, and counting the pixel overlapping areas in both the horizontal and vertical motion trajectories;

[0036] S33, calculating the centroid position of the pixel overlap area based on the geometric center method, and using it as the rough centroid position of the calibration point.

[0037] Preferably, the S4 includes:

[0038] S41, selecting a fixed spatial window according to the vicinity of the pixel where the rough centroid is located;

[0039] S42, sorting the positive events generated by all pixels in the window according to the time axis, determining the start time and the end time as the time window, and dividing the time period into multiple time steps;

[0040] S43, determining the centroid position of each time step based on a time interpolation grayscale centroid positioning algorithm;

[0041] S44, performing linear fitting on the center of mass position within the spatial window to determine the trajectory equation of the group of center of mass coordinates.

[0042] Preferably, the S5 includes:

[0043] S51, calculating the coarse centroid positions of all intersections in the grid trajectory and the trajectory equation in each spatial window;

[0044] S52, solving a linear equation of two variables based on the equations of the mutually perpendicular trajectories, and calculating the coordinates as the positions of the intersection points, which are used as the sub-pixel centroid positions of the calibration points corresponding to each set of mutually perpendicular trajectories.

[0045] Preferably, the S6 includes:

[0046] S61, making a one-to-one correspondence between the sub-pixel centroid position of the calibration point and the position of the turntable outer frame and the middle frame;

[0047] S62, solving the internal and external parameter coefficients of the event camera based on the unified internal and external parameter calibration method, wherein the internal and external parameters of the event camera include: internal parameters including the focal length, principal point and distortion coefficient of the event camera, and external parameters including the installation matrix of the event camera coordinate system and the turntable coordinate system and the stellar simulator emission star vector in the turntable coordinate system.

[0048] A second aspect of the present invention further provides a high-precision calibration system for event cameras for space situational awareness, which is used to implement the method of the first aspect, comprising:

[0049] Combined calibration framework establishment module, used to establish a combined calibration framework for the star simulator, high-precision turntable and event camera;

[0050] The grid trajectory acquisition module is used to control the rotation of the high-precision turntable, collect event stream information generated by the relative motion point light source, perform time aggregation on the event stream, and obtain the grayscale value of the grid trajectory image;

[0051] a coarse centroid position extraction module, configured to extract the connected domain of the grid-like trajectory, count the intersecting overlapping pixels and the pixel overlap areas in the horizontal and vertical directions in the grid-like trajectory, and calculate the coarse centroid position of the calibration point based on the pixel overlap areas;

[0052] a trajectory equation determination module, configured to determine a spatial window based on the position of the coarse centroid, determine a time window within the spatial window, divide time into multiple time steps with equal steps within the time window, determine the centroid position of each time step based on a time interpolation grayscale centroid positioning algorithm, and perform linear fitting on the centroid position within the spatial window to determine the trajectory equation of the set of centroid coordinates;

[0053] a sub-pixel centroid position extraction module, configured to calculate the coarse centroid positions of all intersections in the grid trajectory and the trajectory equation within each spatial window, and then calculate the positions of the intersections based on the mutually perpendicular trajectory equations, and use them as the sub-pixel centroid positions of the calibration points;

[0054] The parameter optimization solution and high-precision calibration module is used to optimize the internal and external parameters of the event camera based on the one-to-one correspondence between the sub-pixel centroid position of the calibration point and the turntable angle.

[0055] A third aspect of the present invention provides an electronic device, comprising a processor and a memory, wherein the memory stores a plurality of instructions, and the processor is configured to read the instructions and execute the method described in the first aspect.

[0056] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a plurality of instructions, and the plurality of instructions can be read by a processor to execute the method described in the first aspect.

[0057] Beneficial effects of the method and system of the present invention:

[0058] (1) A calibration framework based on a star simulator and a high-precision turntable is used to meet the task of accurately and quickly monitoring space targets in space situational awareness by event cameras.

[0059] (2) According to the characteristics of asynchronous imaging of event cameras, a dynamic calibration trajectory acquisition strategy is adopted, which can obtain a large number of calibration points in a simple and efficient manner.

[0060] (3) The calibration accuracy of the method is comparable to that of traditional cameras, which is significantly improved compared to the current event camera calibration method and can meet the requirements of space target detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in related technologies, the following briefly introduces the drawings required for use in the specific embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0062] Figure 1 (a) is a schematic diagram of the process flow of a high-precision calibration method for an event camera for space situational awareness according to an embodiment of the present invention;

[0063] Figure 1 (b) is a schematic diagram of the software process of the high-precision calibration method for event cameras for space situational awareness provided by an embodiment of the present invention, excluding the part of building a combined calibration framework;

[0064] Figure 2 A diagram showing the hardware system structure of a calibration framework according to an embodiment of the present invention;

[0065] Figure 3 The event camera imaging model in the imaging system provided according to an embodiment of the present invention represents the imaging process of the event camera receiving a parallel light vector in a world coordinate system and transforming it into a pixel coordinate system;

[0066] Figure 4 A schematic diagram of a calibration trajectory acquisition strategy provided according to an embodiment of the present invention;

[0067] Figure 5 A schematic diagram of a rough extraction process of calibration points according to an embodiment of the present invention;

[0068] Figure 6 A schematic diagram of the principle of an event-based time linear interpolation grayscale centroid positioning algorithm provided according to an embodiment of the present invention;

[0069] Figure 7A schematic diagram of sub-pixel coordinate extraction according to an embodiment of the present invention. The shaded area represents the pixel where the coarse centroid is located, the gray area represents the valid pixels, the horizontal and vertical dots represent the sub-pixel centroid positions at different times within the horizontal and vertical spatial windows, respectively, and the straight line represents the linear fit trajectory.

[0070] Figure 8 A schematic diagram of a calibration grid trajectory provided according to an embodiment of the present invention;

[0071] Figure 9 A schematic diagram of the centroid position of the calibration points provided according to an embodiment of the present invention;

[0072] Figure 10 A schematic diagram of residual results of calibration points provided according to an embodiment of the present invention;

[0073] Figure 11 A schematic diagram of residual results of test points provided according to an embodiment of the present invention;

[0074] Figure 12 A structural diagram of an electronic device provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0075] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0076] In the description of the present invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limitations on the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0077] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed, detachable, or integral connections; mechanical or electrical connections; direct or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0078] Example 1

[0079] like Figure 1 As shown, this embodiment provides a high-precision calibration method for an event camera for space situational awareness, including:

[0080] S1, establish a combined calibration framework of the star simulator, high-precision turntable and event camera;

[0081] S2, controls the rotation of the high-precision turntable, collects the event stream information generated by the relative motion point light source, performs time aggregation on the event stream, and obtains the grayscale value of the grid trajectory image;

[0082] S3, extracting the connected domain where the grid-shaped trajectory is located, counting the intersecting overlapping pixels in the grid-shaped trajectory and the pixel overlapping areas in the horizontal and vertical directions, and calculating the rough centroid position of the calibration point based on the pixel overlapping areas;

[0083] S4, determining a spatial window based on the position of the coarse centroid, determining a time window within the spatial window, dividing time into multiple time steps with equal steps within the time window, determining the centroid position of each time step based on a time interpolation grayscale centroid positioning algorithm, and performing a linear fit on the centroid position within the spatial window to determine a trajectory equation for the set of centroid coordinates;

[0084] S5, calculating the coarse centroid positions of all intersections in the grid trajectory and the trajectory equation in each spatial window, and then calculating the positions of the intersections based on the mutually perpendicular trajectory equations, and using them as the sub-pixel centroid positions of the calibration points;

[0085] S6, optimizing the internal and external parameters of the event camera based on the one-to-one correspondence between the sub-pixel centroid positions of the calibration points and the turntable angles.

[0086] Implementation principles of steps S1-S6:

[0087] Due to the unique imaging mechanism of the event camera, the core of its high-precision calibration method lies in two parts: calibration data acquisition and sub-pixel calibration point centroid positioning. This embodiment uses a high-precision turntable with a mature framework of a star simulator to collect calibration data. In traditional camera calibration methods, the camera usually performs timed exposure sampling of the light source while the turntable and the star simulator remain relatively stationary. However, without taking other measures to change the brightness of the light source, the event camera will not be able to output event information for the light source. Therefore, in order to integrate event camera calibration into this framework, the present invention proposes for the first time a simple and efficient high-precision turntable dynamic calibration trajectory acquisition strategy, which completes the extraction of the centroid position of the calibration point through coarse extraction and local linear sub-pixel extraction. Among them, the original time interpolation grayscale centroid positioning algorithm can calculate the centroid position of point light sources at different times by processing event time information.

[0088] As a preferred embodiment, the combined calibration framework of S1 includes a hardware system and an imaging system, wherein the imaging system includes an imaging mechanism and an imaging model.

[0089] In this embodiment, Figure 2 The figure shows the hardware system in the calibration framework adopted by the present invention. The core components of the hardware system include a stellar simulator, a high-precision turntable and an event camera. The stellar simulator is fixed on a marble platform and emits a collimated point light source. The event camera is installed on the inner frame of the high-precision turntable to sense the light source. The design accuracy of the high-precision turntable reaches 0.35″ (3σ), in which the frame and the outer frame can be rotated at a constant speed to adjust the angle, thereby changing the position of the point light source on the image plane of the event camera. There is a unique correspondence between the center of mass position of the point light source and the angle of the middle frame and the outer frame of the turntable, which provides a reference benchmark for the accuracy of subsequent calibration experiments.

[0090] In this embodiment, the imaging system includes an imaging mechanism and an imaging model, wherein:

[0091] (1) Imaging mechanism

[0092] An event camera is a bio-inspired visual sensor that works very differently from a standard traditional camera. In an event camera, each pixel acts as an independent photodetector, continuously monitoring the logarithm of light intensity. Once this change exceeds the preset threshold C, the corresponding pixel An event will be triggered immediately The details are shown in formula (1).

[0093]

[0094] Where, Δt k is pixel x k The time since the last event, polarity p k ∈{+1, -1} is the sign of brightness change.

[0095] It is particularly worth noting that, given that the research background of this invention is set under the extremely dark conditions of the space environment, this special environment delays the detection of negative events (i.e., brightness reduction events) to a level that is difficult to ignore. According to experimental observations, for the same pixel, in the event stream generated by the motion of a 2Mv star, the time interval between two negative events is more than a hundred times that between two positive events. There is currently no relevant literature to explain this phenomenon. Therefore, to ensure the accuracy of the experimental data and the effectiveness of the analysis, the research scope of this invention focuses on the observation and analysis of positive events (i.e., brightness increase events), temporarily ignoring the impact of changes in negative events.

[0096] (2) Imaging model

[0097] like Figure 3 The figure shows the event camera imaging model in the imaging system. The event camera imaging model represents the imaging process of the event camera receiving the parallel light vector in the world coordinate system to the pixel coordinate system.

[0098] This embodiment constructs the propagation process of the coordinate position of the parallel light vector emitted by the star simulator through the event camera lens on the image plane. Among them, the conversion relationship between the coordinate systems is given by Figure 3 Description. Vector V is the turntable coordinate system X r -Y r -Z r The parallel light vector emitted by the star simulator under is expressed as shown in Equation (2), where α and β are the pitch and yaw angles of the vector V.

[0099]

[0100] Rotation matrix R rot It can be obtained by synthesizing different rotation angles of the turntable. re is the mounting matrix between the event camera and the turntable, the event camera coordinate system X e -Y e -Z e The expression of the parallel light vector V′ under the condition is shown in formula (3). and are the rotation angles from the turntable coordinate system to the event camera coordinate system. x ,θ y These are the angles at which the turntable rotates the middle frame and outer frame respectively.

[0101]

[0102] According to the pinhole imaging model, the projection position (u′, v′) of the parallel light vector on the pixel coordinate system uv is shown in Equation (4), where s is the scale factor, F is the lens focal length, d is the pixel size, and u0 and v0 are the principal point positions of the pixel coordinate system.

[0103]

[0104] In addition, lens distortion needs to be added to this, where q1 and q2 are radial distortion coefficients, and p1 and p2 are tangential distortion coefficients. du and dv are the lens distortion magnitudes along the u and v directions of the pixel coordinate system, respectively. The final projected position (u, v) is shown in Equation (5).

[0105]

[0106] By integrating formulas (2)-(5), the pixel coordinate position (u, v) and the rotation angle (θ x ,θ y ) have a one-to-one correspondence between them. Then, based on this clear correspondence, a unified calibration algorithm of internal and external parameters is used to calibrate the 13 key parameters F, d, u0, v0, q1, q1, p1, p2, α, β, The logic of the unified calibration algorithm for internal and external parameters is shown in formula (6).

[0107]

[0108] As a preferred embodiment, the S2 includes:

[0109] S21, driving the high-precision turntable to rotate the outer frame and the middle frame respectively;

[0110] S22, the event camera obtains the event flow information generated by the relative motion point light source based on the asynchronous imaging principle, counts the number of positive events generated by each pixel, uses this as the intensity value of the pixel, and obtains the grayscale value of the grid trajectory image.

[0111] In this embodiment, the middle frame and the inner frame of the turntable are kept stationary, and the outer frame of the turntable is started and controlled to rotate at a constant speed at a preset fixed angular velocity. The parallel light source emitted by the star simulator forms a continuous and stable movement on the imaging plane of the event camera, thereby drawing a trajectory that is approximately a straight line. Similarly, in order to obtain the calibration trajectory in the other direction (vertical direction), the middle frame of the turntable needs to be set as the rotation object while keeping the outer frame and the inner frame stationary. The middle frame also rotates at a preset fixed angular velocity to generate a new trajectory perpendicular to the previous trajectory, as shown in the schematic diagram. Figure 4 To construct a dense calibration grid, the calibration trajectory acquisition process described above is repeated. By adjusting the rotation starting position or angle, M horizontal and N vertical trajectories are generated at equal intervals within the imaging area of ​​the event camera. These trajectories intersect to form a total of M×N precise calibration intersection points. Each calibration point corresponds to a unique turntable angle value, providing rich data support for subsequent camera parameter calibration. This acquisition method is simple and efficient. As the number of trajectories in different directions increases, the number of calibration points collected increases by O(MN), greatly improving calibration efficiency.

[0112] As a preferred embodiment, the S3 includes:

[0113] S31, performing noise filtering on the grid trajectory image, and calculating the maximum connected domain in the image plane based on a nearest connected domain algorithm to extract the connected domain where the grid trajectory is located;

[0114] S32, acquiring pixels associated with the motion trajectory of the point light source, counting the intersecting overlapping pixels in the grid-like trajectory, and counting the pixel overlapping areas in both the horizontal and vertical motion trajectories;

[0115] S33, calculating the centroid position of the pixel overlap area based on the geometric center method, and using it as the rough centroid position of the calibration point.

[0116] In this embodiment, step S3 involves a rough extraction step in the calibration point extraction, such as Figure 5 The flowchart of the rough extraction process of the calibration point shown in the figure first performs time aggregation on the event stream generated by the moving point light source, that is, counts the number of positive events generated by each pixel, and uses this as the intensity value of the pixel. In actual tests, there is often a large amount of background noise. In the embodiment of the present invention, the pixels with a counted number of positive events not exceeding 2 are regarded as events caused by background noise, and the event stream stimulated by the movement of the point light source is retained. Subsequently, in order to further determine the pixels passed by the point light source, the nearest connected domain algorithm is used to extract the maximum connected domain of the image, and pixels closely related to the motion trajectory of the point light source are obtained. Finally, the pixel overlap area in the horizontal and vertical motion trajectories is counted, and the center of mass position (u, v) of the area is calculated using the geometric center method, and it is used as the preliminary center of mass position of the calibration point.

[0117] As a preferred embodiment, the S4 includes:

[0118] S41, selecting a fixed spatial window according to the vicinity of the pixel where the rough centroid is located;

[0119] S42, sorting the positive events generated by all pixels in the window according to the time axis, determining the start time and the end time as the time window, and dividing the time period into multiple time steps;

[0120] S43, determining the centroid position of each time step based on a time interpolation grayscale centroid positioning algorithm;

[0121] S44, performing linear fitting on the center of mass position within the spatial window to determine the trajectory equation of the group of center of mass coordinates.

[0122] In this embodiment, Figure 6 The figure shows the principle diagram of the time interpolation grayscale centroid positioning algorithm. From theoretical analysis and experimental tests, it is known that when a point light source enters and leaves a pixel, it will first generate a series of positive events and then a series of negative events. The moment when the last positive event and the first negative event are generated is the moment when the pixel is closest to the center plane of the point light source. According to this process, an event-based time interpolation grayscale centroid positioning algorithm is designed. Under extremely dark light conditions, the negative event has a very high delay, resulting in the unavailability of the negative event, making it difficult to determine the moment t when the energy center of the light spot coincides with the pixel center.mid Therefore, an assumption is made in the algorithm that for a positive event time series {t i ,i=1,2,...,k}, it is assumed that the last positive event occurs at time t k is the moment when the center plane of the point light source coincides with the center of the pixel, and it is assumed that the time of the last negative event is t k+1 =2t k -t1, for a certain time t id , when t k <t id <t k+1 When t id =2t k -t id Statistical time t id In the event sequence range of the pixels in the selected spatial window, determine whether the straight-line distance between these pixels is less than d. If the condition is met, record the pixel position {(u i ,v i ),i=1,2,...,l}.

[0123] Judgment time t id The position in the event sequence, if t i <t id ≤t i+1 , then id=i; if t id = t1, then id = 0. Here, it is assumed that the background noise intensity of all pixels is the same and is set to 1. According to the linear interpolation method, the time t id The logarithm of the pixel grayscale is L id (i), as shown in formula (7):

[0124]

[0125] Use formula (7) to calculate the grayscale values ​​of l pixels that meet the conditions, and calculate the time t according to the grayscale centroid algorithm id The center of mass position of the point light source (u id ,v id ), as shown in formula (8):

[0126]

[0127] As a preferred embodiment, the S5 includes:

[0128] S51, calculating the coarse centroid positions of all intersections in the grid trajectory and the trajectory equation in each spatial window;

[0129] S52, solving a linear equation of two variables based on the equations of the mutually perpendicular trajectories, and calculating the coordinates as the positions of the intersection points, which are used as the sub-pixel centroid positions of the calibration points corresponding to each set of mutually perpendicular trajectories.

[0130] In this embodiment, step S5 involves local linear sub-pixel extraction in the extraction of calibration points, and step S3 completes the preliminary extraction of the centroid of the calibration points. Based on this, the sub-pixel centroid position of the calibration points needs to be further calculated in step S5. To this end, step S4 of this embodiment uses a time interpolation grayscale centroid positioning algorithm to calculate the centroid position of the point light source motion trajectory at different times. Subsequently, a series of centroid positions are linearly fitted to solve the trajectory equation, and the sub-pixel centroid position of the calibration point is obtained by calculating the intersection of mutually perpendicular trajectory equations. However, lens distortion has a nonlinear effect on the motion trajectory of the point light source. Therefore, this embodiment selects a fixed spatial window in the vicinity of the pixel where the coarse centroid is located, and considers that the motion trajectory of the point light source within the window is approximately a straight line. Step S3 determines whether the pixel is a noise signal based on the positive event generated by the pixel, without considering whether the series of events generated by the pixel contains noise signals. Next, the event information generated by each pixel in the selected spatial window is analyzed, and judged one by one according to the time before and after the event occurs. A time threshold th is set as the judgment criterion. If the time interval between two adjacent positive events exceeds the threshold, the first event is considered to be inconsistent with the continuity requirement of the calibration trajectory and is removed. This process is iterated until a pair of adjacent events with a time interval less than the threshold th is found. These two positive events are considered to be the first two valid events excited by the motion of the point light source at that pixel.

[0131] Then, all positive events generated by all pixels after filtering noise in the spatial window are sorted according to the time axis, and the time when the first event occurs, t s and the last event generation time t e , and divide the time period into equal time periods to obtain the time series {t n}, the division method is shown in formula (9). Using the time interpolation grayscale centroid positioning algorithm time series {t n}, forming a set of centroid coordinates {(u n ,v n )}, and the trajectory equation v = ku + b of the group of centroid coordinates is obtained according to the linear fitting method.

[0132]

[0133] like Figure 7 As shown in the figure, a region with length and width of m and n pixels is selected in the horizontal and vertical directions as a spatial window, and the time series {t h}、{tv}, and then calculate the trajectory equations v=k in the horizontal and vertical directions h u+b h 、v=k v u+b v . This method is used to calculate the horizontal trajectory equation v=k in the M×N intersection adjacent space window h,i u+b h,i (i=1,2,...M×N) and vertical trajectory equation v=k v,i u+b v,i (i = 1, 2, ... M × N), the horizontal and vertical trajectory equations of the spatial window associated with each intersection point form a set of two-variable linear equations, with a total of M × N equations. By solving the equations, the sub-pixel centroid positions of all calibration points are obtained, as shown in Equation (10).

[0134]

[0135] As a preferred embodiment, the S6 includes:

[0136] S61, making a one-to-one correspondence between the sub-pixel centroid position of the calibration point and the position of the turntable outer frame and the middle frame;

[0137] S62, solving the internal and external parameter coefficients of the event camera based on the unified internal and external parameter calibration method, wherein the internal and external parameters of the event camera include: internal parameters including the focal length, principal point and distortion coefficient of the event camera, and external parameters including the installation matrix of the event camera coordinate system and the turntable coordinate system and the stellar simulator emission star vector in the turntable coordinate system.

[0138] In this embodiment, based on the sub-pixel coordinates of the calibration point (u i ,v i ) and the corresponding turntable angle The information is used to optimize the internal and external parameters of Equation (6) in a unified manner, and the internal parameters such as the focal length, principal point, and distortion coefficient of the event camera, the installation matrix between the event camera coordinate system and the turntable coordinate system, and the external parameters such as the stellar simulator launch star vector in the turntable coordinate system are solved.

[0139] Application Examples

[0140] In the calibration trajectory acquisition, the outer frame angle range is set to (5.5°, -4.5°), the span is 2°, and a total of 6 trajectories are collected; the turntable middle frame angle range is set to (-8.5°, 9.5°), the span is 3°, and a total of 7 trajectories are collected. Aggregate the 13 trajectories into one image as shown in the following figure. Figure 8 As shown, a total of 42 intersection points are obtained. The position of the centroid calculated using the method of this embodiment on the image plane is as follows Figure 9 As shown in Table 1, the internal and external parameters of the event camera are calibrated using the unified internal and external parameter calibration method.

[0141] Table 1 Calibration results

[0142]

[0143] Figure 10 The magnitude and direction of the residual error exhibit a remarkably random distribution across the focal plane. This observation directly points to an important conclusion: the systematic error has been effectively minimized by the calibration process. To quantify and evaluate the accuracy of the calibration process, reprojection error was introduced as a core evaluation metric. The calculated reprojection error values ​​are 0.0253 pixels in the x-axis and 0.0292 pixels in the y-axis, which translate to equivalent angular errors of 1.55 arc seconds and 1.79 arc seconds, respectively.

[0144] In order to verify the calibration results, another set of mesh trajectories was collected for testing. The outer frame angle was set to (5°, -4°), the span was 1.5°, and there were 7 trajectories in total. The turntable middle frame angle was set to (-6.5°, 7.5°), the span was 2°, and there were 8 trajectories in total, resulting in a total of 56 test points. Using the turntable position and the calibration parameters in Table 1, the center of mass position of the test point is calculated as a reference value. The test value of the center of mass position of the test point is obtained by the method of the first aspect. The accuracy of the calibration parameters is evaluated by the residual values ​​of the two sets of center of mass positions. The calculation results are as follows: the residual in the x-axis direction is 0.0250 pixels, and the residual in the y-axis direction is 0.0301 pixels. The equivalent angle errors of pitch and yaw are 1.53 arc seconds and 1.84 arc seconds, respectively. The residual direction and size of the specific test points are as follows. Figure 11 As shown in , the residual magnitude and direction are randomly distributed. The above analysis not only verifies the reliability of the calibration parameters, but also strongly demonstrates the accuracy and effectiveness of this method in the event camera calibration task.

[0145] Example 2

[0146] This embodiment provides a high-precision event camera calibration system for space situational awareness, which is used to implement the method of embodiment 1, including:

[0147] Combined calibration framework establishment module, used to establish a combined calibration framework for the star simulator, high-precision turntable and event camera;

[0148] The grid trajectory acquisition module is used to control the rotation of the high-precision turntable, collect event stream information generated by the relative motion point light source, perform time aggregation on the event stream, and obtain the grayscale value of the grid trajectory image;

[0149] a coarse centroid position extraction module, configured to extract the connected domain of the grid-like trajectory, count the intersecting overlapping pixels and the pixel overlap areas in the horizontal and vertical directions in the grid-like trajectory, and calculate the coarse centroid position of the calibration point based on the pixel overlap areas;

[0150] a trajectory equation determination module, configured to determine a spatial window based on the position of the coarse centroid, determine a time window within the spatial window, divide time into multiple time steps with equal steps within the time window, determine the centroid position of each time step based on a time interpolation grayscale centroid positioning algorithm, and perform linear fitting on the centroid position within the spatial window to determine the trajectory equation of the set of centroid coordinates;

[0151] a sub-pixel centroid position extraction module, configured to calculate the coarse centroid positions of all intersections in the grid trajectory and the trajectory equation within each spatial window, and then calculate the positions of the intersections based on the mutually perpendicular trajectory equations, and use them as the sub-pixel centroid positions of the calibration points;

[0152] The parameter optimization solution and high-precision calibration module is used to optimize the internal and external parameters of the event camera based on the one-to-one correspondence between the sub-pixel centroid position of the calibration point and the turntable angle.

[0153] The present invention also provides a memory storing a plurality of instructions, wherein the instructions are used to implement the method as in the first embodiment.

[0154] like Figure 12 As shown, the present invention also provides an electronic device, including a processor 301 and a memory 302 connected to the processor 301, the memory 302 stores multiple instructions, and the instructions can be loaded and executed by the processor to enable the processor to execute the method as in embodiment 1.

[0155] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A high-precision calibration method for event cameras for space situational awareness, characterized in that: include: S1, establish a combined calibration framework of the star simulator, high-precision turntable and event camera; S2, controls the rotation of the high-precision turntable, collects the event stream information generated by the relative motion point light source, performs time aggregation on the event stream, and obtains the grayscale value of the grid trajectory image; S3, extracting the connected domain where the grid-shaped trajectory is located, counting the intersecting overlapping pixels in the grid-shaped trajectory and the pixel overlapping areas in the horizontal and vertical directions, and calculating the rough centroid position of the calibration point based on the pixel overlapping areas; S4, determining a spatial window based on the position of the coarse centroid, determining a time window within the spatial window, dividing time into multiple time steps with equal steps within the time window, determining the centroid position of each time step based on a time interpolation grayscale centroid positioning algorithm, and performing a linear fit on the centroid position within the spatial window to determine a trajectory equation for the set of centroid coordinates; S5, calculating the coarse centroid positions of all intersections in the grid trajectory and the trajectory equation in each spatial window, and then calculating the positions of the intersections based on the mutually perpendicular trajectory equations, and using them as the sub-pixel centroid positions of the calibration points; S6, optimizing the internal and external parameters of the event camera based on the one-to-one correspondence between the sub-pixel centroid positions of the calibration points and the turntable angles.

2. The high-precision calibration method for event cameras for space situational awareness according to claim 1, characterized in that: The combined calibration framework of S1 includes a hardware system and an imaging system, wherein: The core components of the hardware system include a stellar simulator, a high-precision turntable, and an event camera. The stellar simulator is fixed to a marble platform and emits a collimated point light source. The event camera is mounted on the inner frame of the high-precision turntable to sense this light source. The middle and outer frames of the high-precision turntable rotate at a constant speed to adjust their angles, changing the position of the point light source on the event camera's image plane. There is a unique correspondence between the point light source's centroid position and the angles of the middle and outer frames of the high-precision turntable. The imaging system includes an imaging mechanism and an imaging model, wherein: (1) The imaging mechanism includes: In the event camera, each pixel continuously monitors the logarithm of the light intensity. Once the change exceeds the preset threshold C, the corresponding pixel An event will be triggered immediately The specific formula is as shown in formula (1); Where Δt k is pixel x k The time since the last event, polarity p k ∈{+1, -1} is the sign of brightness change; (2) The imaging model includes: the propagation process of the parallel light vector emitted by the star simulator through the event camera lens on the image plane, and the vector V is the turntable coordinate system X r -Y r -Z r The parallel light vector emitted by the star simulator under is expressed as shown in Equation (2), where α and β are the pitch and yaw angles of the vector V; Rotation matrix R rot Obtained by synthesizing different rotation angles of the turntable; R re is the mounting matrix between the event camera and the turntable, the event camera coordinate system X e -Y e -Z e The expression of the parallel light vector V′ under the condition is shown in formula (3), where: and They are the rotation angles from the turntable coordinate system to the event camera coordinate system, θ x ,θ y These are the angles at which the turntable rotates the middle and outer frames; According to the pinhole imaging model, the projection position (u′, v′) of the parallel light vector on the pixel coordinate system uv is shown in Equation (4), where s is the scale factor, F is the lens focal length, d is the pixel size, u0 and v0 are the principal point positions of the pixel coordinate system respectively; On this basis, lens distortion is added, where q1 and q2 are radial distortion coefficients, p1 and p2 are tangential distortion coefficients; du and dv are the distortion magnitudes of the lens along the u and v directions of the pixel coordinate system, respectively. The final projection position (u, v) is shown in formula (5). By integrating formulas (2)-(5), the pixel coordinate position (u, v) and the rotation angle (θ x ,θ y ) have a one-to-one correspondence between them; based on this clear correspondence, a unified calibration algorithm for internal and external parameters is used to calibrate the 13 key parameters F, d, u0, v0, q1, q1, p1, p2, α, β, Perform optimization solution; the logic of the unified calibration algorithm for internal and external parameters is shown in formula (6):

3. The high-precision calibration method for event cameras for space situational awareness according to claim 2, characterized in that: The S2 includes: S21, driving the high-precision turntable to rotate the outer frame and the middle frame respectively; S22, the event camera obtains the event flow information generated by the relative motion point light source based on the asynchronous imaging principle, counts the number of positive events generated by each pixel, uses this as the intensity value of the pixel, and obtains the grayscale value of the grid trajectory image.

4. The high-precision calibration method for event cameras for space situational awareness according to claim 3, characterized in that: The S3 includes: S31, performing noise filtering on the grid trajectory image, and calculating the maximum connected domain in the image plane based on a nearest connected domain algorithm to extract the connected domain where the grid trajectory is located; S32, acquiring pixels associated with the motion trajectory of the point light source, counting the intersecting overlapping pixels in the grid-like trajectory, and counting the pixel overlapping areas in both the horizontal and vertical motion trajectories; S33, calculating the centroid position of the pixel overlap area based on the geometric center method, and using it as the rough centroid position of the calibration point.

5. The high-precision calibration method for event cameras for space situational awareness according to claim 4, characterized in that: The S4 includes: S41, selecting a fixed spatial window according to the vicinity of the pixel where the rough centroid is located; S42, sorting the positive events generated by all pixels in the window according to the time axis, determining the start time and the end time as the time window, and dividing the time period into multiple time steps; S43, determining the centroid position of each time step based on a time interpolation grayscale centroid positioning algorithm; S44, performing linear fitting on the center of mass position within the spatial window to determine the trajectory equation of the group of center of mass coordinates.

6. The high-precision calibration method for event cameras for space situational awareness according to claim 5, characterized in that: The S5 includes: S51, calculating the coarse centroid positions of all intersections in the grid trajectory and the trajectory equation in each spatial window; S52, solving a linear equation of two variables based on the equations of the mutually perpendicular trajectories, and calculating the coordinates as the positions of the intersection points, which are used as the sub-pixel centroid positions of the calibration points corresponding to each set of mutually perpendicular trajectories.

7. The high-precision calibration method for event cameras for space situational awareness according to claim 6, characterized in that: The S6 includes: S61, making a one-to-one correspondence between the sub-pixel centroid position of the calibration point and the position of the turntable outer frame and the middle frame; S62, solving the internal and external parameter coefficients of the event camera based on the unified internal and external parameter calibration method, wherein the internal and external parameters of the event camera include: internal parameters including the focal length, principal point and distortion coefficient of the event camera, and external parameters including the installation matrix of the event camera coordinate system and the turntable coordinate system and the stellar simulator emission star vector in the turntable coordinate system.

8. A high-precision calibration system for event cameras used for space situational awareness, used to implement the method of any one of claims 1 to 7, characterized in that: include: Combined calibration framework establishment module, used to establish a combined calibration framework for the star simulator, high-precision turntable and event camera; The grid trajectory acquisition module is used to control the rotation of the high-precision turntable, collect event stream information generated by the relative motion point light source, perform time aggregation on the event stream, and obtain the grayscale value of the grid trajectory image; a coarse centroid position extraction module, configured to extract the connected domain of the grid-like trajectory, count the intersecting overlapping pixels and the pixel overlap areas in the horizontal and vertical directions in the grid-like trajectory, and calculate the coarse centroid position of the calibration point based on the pixel overlap areas; a trajectory equation determination module, configured to determine a spatial window based on the position of the coarse centroid, determine a time window within the spatial window, divide time into multiple time steps with equal steps within the time window, determine the centroid position of each time step based on a time interpolation grayscale centroid positioning algorithm, and perform linear fitting on the centroid position within the spatial window to determine the trajectory equation of the set of centroid coordinates; a sub-pixel centroid position extraction module, configured to calculate the coarse centroid positions of all intersections in the grid trajectory and the trajectory equation within each spatial window, and then calculate the positions of the intersections based on the mutually perpendicular trajectory equations, and use them as the sub-pixel centroid positions of the calibration points; The parameter optimization solution and high-precision calibration module is used to optimize the internal and external parameters of the event camera based on the one-to-one correspondence between the sub-pixel centroid position of the calibration point and the turntable angle.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a plurality of instructions, and the processor is configured to read the instructions and execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a plurality of instructions, and the plurality of instructions can be read by a processor to execute the method according to any one of claims 1 to 7.

Images