Dynamic space target detection method based on event information

Abstract

The invention relates to the technical field of space target detection, in particular to a dynamic space target detection method based on event information, and the method comprises the steps: reading a target event flow according to a certain time length, and obtaining an initial pseudo image frame; eliminating noise events based on the corresponding event density information by using a pre-established circular local sliding window, and judging whether a certain noise reduction condition is met or not; if not, eliminating noise events based on the corresponding event density information by using the self-adaptive filtering characteristic of a pre-established multi-neighborhood radius local sliding window, and calculating the centroid information of the detection space target; judging whether the centroid information meets a certain precision condition or not; and if not, correcting the motion track by using the data of the inertial measurement unit to obtain centroid information meeting a certain precision condition, and obtaining motion information of the detection space target. Therefore, the problems of low space target extraction precision, difficult tracking and the like in a high-dynamic maneuvering scene are solved.

Description

Technical Field

[0001] The present application relates to the technical field of space target detection, and in particular to a dynamic space target detection method based on event information. Background Art

[0002] Star sensors are the most accurate attitude measurement devices available, boasting drift-free operation and a long operating life. They typically operate during periods of stable flight. However, when operating in high-dynamic conditions, the movement of stars within the image sensor's photosensitive area during the integration time can reduce the image's signal-to-noise ratio and the accuracy of the star's centroid. Furthermore, errors in the centroid positioning of stars in the star map can easily lead to redundant and mismatched star recognition, reducing the speed and recognition rate of star recognition. Furthermore, significant variations in the positions of stars in star maps between frames can complicate the star sensor's tracking of target stars, leading to reduced star tracking success rates and efficiency.

[0003] As a new type of sensor, bio-inspired event cameras show significant potential for application in space exploration thanks to their advantages, including microsecond temporal resolution, high dynamic range, and low power consumption. Although existing event cameras lack the accuracy of traditional star sensors in space object detection, they significantly outperform conventional cameras in high-dynamic scenarios, providing a powerful solution for high-speed maneuvers.

[0004] In related technologies, event streams can be obtained from the output of event cameras, and noise reduction processing can be performed by comprehensively utilizing the inertial measurement unit information in the camera to detect high-dynamic target clusters; event stream data can also be obtained from event cameras, and the event streams can be converted into event images, and then convolutional neural networks can be used to obtain detection results with higher confidence.

[0005] However, in related technologies, event cameras on highly dynamic flight platforms have fewer adaptable scenes and low accuracy in extracting spatial targets, which urgently needs to be improved. Summary of the Invention

[0006] The present application provides a dynamic space target detection method based on event information to solve the problems of low space target extraction accuracy and tracking difficulties in high-dynamic maneuvering scenarios in related technologies.

[0007] The first aspect of the present application provides a method for detecting dynamic space targets based on event information, comprising the following steps: obtaining a target event stream containing a detected space target in a dynamic flight scene, and reading the target event stream according to a preset time duration to obtain an initial pseudo image frame containing the detected space target; using a pre-established circular local sliding window, based on corresponding event density information, removing noise events in the initial pseudo image frame to obtain a first pseudo image frame, and judging whether the first pseudo image frame meets a preset noise reduction condition; if the first pseudo image frame does not meet the preset noise reduction condition, using the adaptive filtering characteristics of a pre-established multiple neighborhood radius local sliding window, based on corresponding event density information, removing noise events in the first pseudo image frame to obtain a second pseudo image frame, and based on the second pseudo image frame, calculating the centroid information of the detected space target; if the first pseudo image frame meets the preset noise reduction condition, noise condition, then calculate the center of mass information of the detection space target based on the first pseudo image frame; judge whether the center of mass information in the second pseudo image frame meets the preset accuracy condition, and / or whether the center of mass information in the first pseudo image frame meets the preset accuracy condition; if the center of mass information in the second pseudo image frame does not meet the preset accuracy condition, and / or the center of mass information in the first pseudo image frame does not meet the preset accuracy condition, use the inertial measurement unit data to correct the motion trajectory of the detection space target to obtain the center of mass information that meets the preset accuracy condition, and obtain the motion information of the detection space target based on the center of mass information of the preset accuracy condition; if the center of mass information in the second pseudo image frame meets the preset accuracy condition, and the center of mass information in the first pseudo image frame meets the preset accuracy condition, obtain the motion information of the detection space target based on the center of mass information of the preset accuracy condition.

[0008] Optionally, in one embodiment of the present application, the method of using a pre-established circular local sliding window to eliminate noise events in the initial pseudo image frame based on corresponding event density information to obtain a first pseudo image frame includes: using the pre-established circular local sliding window to calculate the first event density of each event in the initial pseudo image frame; determining whether the first event density is greater than or equal to a first preset event density threshold; if the first event density is greater than or equal to the first preset event density threshold, determining that the event corresponding to the first event density is a target event, and retaining the target event in the initial pseudo image frame; if the first event density is less than the first preset event density threshold, determining that the event corresponding to the first event density is a noise event, and eliminating the noise event in the initial pseudo image frame; and obtaining the first pseudo image frame based on the target event retained in the initial pseudo image frame.

[0009] Optionally, in one embodiment of the present application, before utilizing the adaptive filtering characteristics of a pre-established multiple neighborhood radius local sliding window to eliminate noise events in the first pseudo image frame based on corresponding event density information, it also includes: determining at least two multiple neighborhood radii corresponding to the multiple neighborhood radius local sliding window based on the first pseudo image frame; and constructing the multiple neighborhood radius local sliding window based on the at least two multiple neighborhood radii.

[0010] Optionally, in one embodiment of the present application, the method of using the adaptive filtering characteristics of a pre-established multiple neighborhood radius local sliding window to eliminate noise events in the first pseudo image frame based on corresponding event density information to obtain a second pseudo image frame includes: using the pre-established multiple neighborhood radius local sliding window to calculate the second event density corresponding to each event in the first pseudo image frame at different multiple neighborhood radii; calculating a third event density for each event based on a pre-constructed multi-scale fusion function, the different multiple neighborhood radii and the second event density corresponding to the different multiple neighborhood radii; judging whether the third event density is greater than or equal to a second preset event density threshold; if the third event density is greater than or equal to the second preset event density threshold, determining that the event corresponding to the third event density is a target event, and retaining the target event in the first pseudo image frame; if the third event density is less than the second preset event density threshold, determining that the event corresponding to the third event density is a noise event, and eliminating the noise event in the first pseudo image frame; and obtaining the second pseudo image frame based on the target event retained in the first pseudo image frame.

[0011] Optionally, in one embodiment of the present application, the use of inertial measurement unit data to correct the motion trajectory of the detection space target to obtain center of mass information that meets the preset accuracy conditions includes: obtaining acceleration data and angular velocity data in the inertial measurement unit data; integrating the acceleration data to obtain speed information and position information of the camera corresponding to the inertial measurement unit; integrating the angular velocity data to obtain posture information of the camera corresponding to the inertial measurement unit; based on the speed information, the position information and / or the posture information, using a pre-constructed motion compensation model to correct the motion trajectory of the detection space target to obtain center of mass information that meets the preset accuracy conditions.

[0012] The second aspect of the present application provides a dynamic space target detection device based on event information, including: an acquisition module for acquiring a target event stream containing a detection space target in a dynamic flight scene, and reading the target event stream according to a preset time duration to obtain an initial pseudo image frame containing the detection space target; a first judgment module for using a pre-established circular local sliding window to eliminate noise events in the initial pseudo image frame based on corresponding event density information to obtain a first pseudo image frame, and to judge whether the first pseudo image frame meets a preset noise reduction condition; a first calculation module for using a pre-established adaptive filtering characteristic of a multiple neighborhood radius local sliding window to eliminate noise events in the first pseudo image frame based on corresponding event density information to obtain a second pseudo image frame when the first pseudo image frame does not meet the preset noise reduction condition, and to calculate the center of mass information of the detection space target based on the second pseudo image frame; a second calculation module for When the noise reduction condition is set, the center of mass information of the detection space target is calculated based on the first pseudo image frame; the second judgment module is used to judge whether the center of mass information in the second pseudo image frame meets the preset accuracy condition, and / or whether the center of mass information in the first pseudo image frame meets the preset accuracy condition; the first generation module is used to use the inertial measurement unit data to correct the motion trajectory of the detection space target when the center of mass information in the second pseudo image frame does not meet the preset accuracy condition, and / or the center of mass information in the first pseudo image frame does not meet the preset accuracy condition, so as to obtain the center of mass information that meets the preset accuracy condition, and obtain the motion information of the detection space target based on the center of mass information of the preset accuracy condition; the second generation module is used to obtain the motion information of the detection space target based on the center of mass information of the preset accuracy condition when the center of mass information in the second pseudo image frame meets the preset accuracy condition and the center of mass information in the first pseudo image frame meets the preset accuracy condition.

[0013] Optionally, in one embodiment of the present application, the first judgment module includes: a first calculation unit, used to calculate the first event density of each event in the initial pseudo image frame using the pre-established circular local sliding window; a first judgment unit, used to judge whether the first event density is greater than or equal to a first preset event density threshold; a first judgment unit, used to judge that the event corresponding to the first event density is a target event when the first event density is greater than or equal to the first preset event density threshold, and retain the target event in the initial pseudo image frame; a second judgment unit, used to judge that the event corresponding to the first event density is a noise event when the first event density is less than the first preset event density threshold, and eliminate the noise event in the initial pseudo image frame; a first generation unit, used to obtain the first pseudo image frame based on the target event retained in the initial pseudo image frame.

[0014] Optionally, in one embodiment of the present application, it also includes: a determination module for determining at least two multiple neighborhood radiuses corresponding to the multiple neighborhood radius local sliding window based on the first pseudo image frame before removing noise events in the first pseudo image frame based on corresponding event density information using the adaptive filtering characteristics of the pre-established multiple neighborhood radius local sliding window; and a construction module for constructing the multiple neighborhood radius local sliding window based on the at least two multiple neighborhood radii.

[0015] Optionally, in one embodiment of the present application, the first calculation module includes: a second calculation unit, used to calculate the second event density corresponding to each event in the first pseudo image frame at different multiple neighborhood radii using the pre-established multiple neighborhood radius local sliding window; a third calculation unit, used to calculate the third event density of each event based on the pre-constructed multi-scale fusion function, the different multiple neighborhood radii and the second event density corresponding to the different multiple neighborhood radii; a second judgment unit, used to judge whether the third event density is greater than or equal to a second preset event density threshold; a third judgment unit, used to judge that the event corresponding to the third event density is a target event when the third event density is greater than or equal to the second preset event density threshold, and retain the target event in the first pseudo image frame; a fourth judgment unit, used to judge that the event corresponding to the third event density is a noise event when the third event density is less than the second preset event density threshold, and eliminate the noise event in the first pseudo image frame; a second generation unit, used to obtain the second pseudo image frame based on the target event retained in the first pseudo image frame.

[0016] Optionally, in one embodiment of the present application, the first generation module includes: an acquisition unit for acquiring acceleration data and angular velocity data in the inertial measurement unit data; a first integration unit for integrating the acceleration data to obtain velocity information and position information of the camera corresponding to the inertial measurement unit; a second integration unit for integrating the angular velocity data to obtain posture information of the camera corresponding to the inertial measurement unit; and a correction unit for correcting the motion trajectory of the detection space target based on the velocity information, the position information and / or the posture information using a pre-constructed motion compensation model to obtain center of mass information that meets the preset accuracy conditions.

[0017] The third aspect of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the dynamic space target detection method based on event information as described in the above embodiment.

[0018] The fourth aspect of the present application provides a computer-readable storage medium, which stores a computer program. When the program is executed by a processor, it implements the above-mentioned dynamic space target detection method based on event information.

[0019] The fifth aspect of the present application provides a computer program product, including a computer program, which, when executed, implements the above-mentioned dynamic space target detection method based on event information.

[0020] The embodiment of the present application can read the target event stream containing the detected space target in a dynamic flight scene according to a certain time duration, thereby obtaining an initial pseudo image frame, and using a pre-established circular local sliding window to eliminate noise events to obtain a first pseudo image frame. If the first pseudo image frame does not meet certain noise reduction conditions, the noise events are further eliminated using a pre-established multiple neighborhood radius local sliding window to obtain a second pseudo image frame, and then the center of mass information of the detected space target is calculated and whether the center of mass information meets certain accuracy conditions is determined. If not, the inertial measurement unit data is used to make corrections, and then the motion information of the detected space target is determined; otherwise, the motion information of the detected space target is directly determined. This can significantly improve the spatiotemporal resolution of data acquisition, increase the dynamic range, and reduce power consumption, achieving good real-time detection and tracking effects, and laying a solid foundation for the navigation and control of hypersonic aircraft. This solves the problems of low space target extraction accuracy and tracking difficulties in high-dynamic maneuvering scenarios in related technologies.

[0021] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0023] Figure 1 This is a block diagram of a high-dynamic space target event stream acquisition device provided according to one embodiment of the present application;

[0024] Figure 2 This is a flowchart of a method for detecting dynamic space targets based on event information according to an embodiment of the present application;

[0025] Figure 3 A block diagram of a classical spatiotemporal correlation filter principle provided according to one embodiment of the present application;

[0026] Figure 4 A block diagram of a circular local sliding window traversal principle provided according to an embodiment of the present application;

[0027] Figure 5 A flowchart of a spatiotemporal correlation event stream filter based on neighborhood density according to one embodiment of the present application;

[0028] Figure 6 A denoising flow chart of a spatiotemporal correlation event stream filter based on multiple neighborhood densities according to one embodiment of the present application;

[0029] Figure 7 A flowchart of compensating for drift errors of an inertial measurement unit sensor in combination with event information according to one embodiment of the present application;

[0030] Figure 8 This is a general flow chart of the working principle of a dynamic space target detection method based on event information according to one embodiment of the present application;

[0031] Figure 9 This is an overall flow chart of a dynamic space target detection method based on event information according to another embodiment of the present application;

[0032] Figure 10 Schematic diagram of a block diagram of a dynamic space target detection device based on event information according to an embodiment of the present application;

[0033] Figure 11 A schematic diagram of the structure of an electronic device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0034] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0035] The following describes a method for detecting dynamic space targets based on event information according to an embodiment of the present application with reference to the accompanying drawings. In response to the problems of low extraction accuracy and tracking difficulty of space targets in highly dynamic maneuvering scenarios mentioned in the above background technology, the present application provides a method for detecting dynamic space targets based on event information. In this method, a target event stream containing a detected space target in a dynamic flight scenario can be read at a certain time duration to obtain an initial pseudo image frame. Noise events are removed using a pre-established circular local sliding window to obtain a first pseudo image frame. If the first pseudo image frame does not meet a certain noise reduction condition, noise events are further removed using a pre-established local sliding window with multiple neighborhood radii to obtain a second pseudo image frame. The center of mass information of the detected space target is then calculated and it is determined whether the center of mass information meets a certain accuracy condition. If not, the inertial measurement unit data is used to correct the information and determine the motion information of the detected space target. Otherwise, the motion information of the detected space target is directly determined. This method can significantly improve the spatiotemporal resolution of data acquisition, increase the dynamic range, and reduce power consumption, achieving good real-time detection and tracking effects, and laying a solid foundation for the navigation and control of hypersonic aircraft. This solves the problems in related technologies such as low accuracy in space target extraction and difficulty in tracking in high-dynamic maneuvering scenarios.

[0036] Before introducing the dynamic space target detection method based on event information proposed in the embodiment of the present application, the high-dynamic space target event stream acquisition device involved in the embodiment of the present application is first introduced.

[0037] Specifically, Figure 1 The present invention is a block diagram of a high-dynamic space target event stream acquisition device provided according to one embodiment of the present application.

[0038] Among them, Figure 1 As shown, the high-dynamic space target event stream acquisition device 10 includes: a star simulator 101 , a high-precision three-axis turntable 102 and an event camera 103 .

[0039] Among them, the star simulator 101 simulates the morphology and brightness of the space target and is composed of a point light source, a mask and a parallel light tube.

[0040] The high-precision three-axis turntable 102 enables the event camera 103 to obtain high dynamic characteristics when collecting data.

[0041] The event camera 103 is an optical device for collecting event streams.

[0042] Furthermore, the embodiment of the present application can use the high-dynamic space target event stream acquisition device 10 to simulate the process of obtaining space target event streams or experimental data on the ground or in the laboratory, and can also use other devices, equipment or systems. The specific settings can be made by technicians in this field according to actual conditions, and this application does not impose specific restrictions.

[0043] It should be noted that the event camera 103 is a bio-inspired visual sensor that operates completely differently from a standard camera. Rather than outputting intensity image frames at a constant rate, the event camera 103 only outputs information related to local pixel-level brightness changes. When these pixel-level brightness changes (or "events") exceed a certain threshold (which can be set by those skilled in the art based on actual conditions and is not specifically limited in this application), the event camera timestamps them with microsecond resolution and outputs an asynchronous event stream in an address-event format. Events can be represented, but are not limited to, as follows:

[0044] e=(x,y,t), (1)

[0045] Wherein, u=(x, y) represents a change event generated by a pixel located at on the camera sensor in response to a brightness change at time t.

[0046] Specifically, Figure 2 The present invention provides a flowchart of a method for detecting dynamic space targets based on event information according to an embodiment of the present application.

[0047] like Figure 2 As shown, the dynamic space target detection method based on event information includes the following steps:

[0048] In step S201, a target event stream containing a detected space target in a dynamic flight scene is obtained, and the target event stream is read according to a preset time duration to obtain an initial pseudo image frame containing the detected space target.

[0049] As a possible implementation, embodiments of the present application can obtain a target event stream containing a detected space target in a dynamic flight scenario and read the target event stream at a specific time duration to obtain an initial pseudo-image frame containing the detected space target. The specific time duration can be set by a skilled artisan based on actual circumstances and is not specifically limited by this application.

[0050] For example, the embodiment of the present application can be Figure 1The event camera capture includes detecting a target event stream of space targets (such as stars, satellites, etc., which are not specifically limited in this application), and then compressing the target event stream according to a certain time duration (such as 10ms, 20ms, 40ms, etc., which are not specifically limited in this application), thereby generating an initial pseudo-image frame. It can be understood that the embodiment of the present application can "cut" the target event stream according to a certain time duration, and each event in the target event stream has a timestamp, that is, there will only be one event for each timestamp. For example, assuming that a target event stream is 500ms, all events in the first 20ms of this target event stream can be placed on this initial pseudo-image frame by selecting a time interval of 20ms. The specific content can be:

[0051] The embodiment of the present application uses a certain time duration to sequentially read the target event stream to obtain the initial pseudo image frame N i , where the initial pseudo image frame N i Can contain a series of events, which can be represented by, but not limited to:

[0052]

[0053] Among them, N i is a pseudo image frame at a random moment; Δt is the time interval between pseudo image frames; e(x, y, t) is an event in the pseudo image frame, where e(x, y, t) includes event coordinates u = (x, y) and event timestamp t.

[0054] In step S202 , a pre-established circular local sliding window is used to remove noise events in the initial pseudo image frame based on corresponding event density information to obtain a first pseudo image frame, and it is determined whether the first pseudo image frame meets a preset noise reduction condition.

[0055] It is understandable that in the embodiment of the present application, the target event can be understood as an event caused by the change in light intensity caused by the relative motion between the spacecraft and the target, and the excited pixels are usually adjacent in a certain period of time. In addition, the difference between noise events and target events is that the noise lacks spatiotemporal correlation with events in the spatial neighborhood. Therefore, by utilizing this difference, the correlation technology can filter out noise by detecting events generated by pixels in the neighborhood, which is called a classical spatiotemporal correlation filter. Figure 3 As shown in , the principle of the classic spatiotemporal correlation filter is to store the timestamps of the events of the adjacent pixels around each pixel, and detect whether the time difference between the current timestamp t2 and the previous timestamp t1 is less than Δt. If so, the event is retained, otherwise the event is filtered out. This principle is shown in formula (3), and its expression can be, but is not limited to, pseudo:

[0056]

[0057] However, spatial targets usually have characteristics such as weak signals and low signal-to-noise ratios, making it difficult for classical spatiotemporal correlation filters to distinguish target events from noise events in a target event stream. Therefore, based on classical spatiotemporal correlation filters, embodiments of the present application can generate pseudo-image frames by compressing the event stream, setting a local sliding window on each frame to adjust the selection range of the spatial neighborhood, and judging whether it is a noise event based on the event neighborhood density information. This method can be called a spatiotemporal correlation event stream filter based on event neighborhood density.

[0058] In actual implementation, the embodiment of the present application can adjust the selection range of the spatial neighborhood by setting the radius of the circular local sliding window and then using the pre-established circular local sliding window on each frame of the image to determine whether it is a noise event based on the first event density, and then remove the noise event to obtain a first pseudo image frame, and then determine whether the first pseudo image frame meets certain noise reduction conditions. The certain noise reduction conditions can be set by those skilled in the art according to actual conditions and are not specifically limited by this application.

[0059] Optionally, in one embodiment of the present application, a pre-established circular local sliding window is used to eliminate noise events in the initial pseudo image frame based on corresponding event density information to obtain a first pseudo image frame, including: using a pre-established circular local sliding window to calculate the first event density of each event in the initial pseudo image frame; judging whether the first event density is greater than or equal to a first preset event density threshold; if the first event density is greater than or equal to the first preset event density threshold, judging the event corresponding to the first event density as a target event, and retaining the target event in the initial pseudo image frame; if the first event density is less than the first preset event density threshold, judging the event corresponding to the first event density as a noise event, and eliminating the noise event in the initial pseudo image frame; and obtaining the first pseudo image frame based on the target event retained in the initial pseudo image frame.

[0060] It is understandable that in the spatial event stream of the embodiment of the present application, the motion trajectory of the target signal usually presents a feature similar to a cylinder, and the local spatial area corresponding to it in the initial pseudo image frame is closer to a circle or an ellipse, rather than the rectangular window used by the classical spatiotemporal correlation filter. Therefore, the embodiment of the present application can be in the initial pseudo image frame N i A circular local sliding window H is used i (e) Traverse the neighborhood of each event in the initial pseudo image frame one by one. The circular local sliding window traversal principle is as follows: Figure 4 As shown. Assume that an event is e0, and its circular local sliding window H i (e0) can be expressed as, but not limited to:

[0061] H i(e0)={e j |e j ∈N i ,||(x j ,y j )-(x0,y0)|| <r}, (4)

[0062] Among them, r is the radius of the circular local sliding window; e0 is a reference event selected in a pseudo image frame, e j are the other events in the circular local sliding window radius r of event e0.

[0063] In some embodiments, the embodiments of the present application can utilize a pre-established circular local sliding window to remove noise events in the initial pseudo image frame based on the corresponding event density information, thereby obtaining a first pseudo image frame. Figure 5 As shown, the content of the first pseudo image frame generated in this embodiment of the application may be:

[0064] Step S501: Acquire an initial pseudo image frame.

[0065] Step S502: traverse the events in the initial pseudo image frame using a pre-established circular local sliding window.

[0066] Among them, the circular local sliding window traversal principle of the embodiment of the present application is as follows Figure 4 As shown, the circular local sliding window H i The expression of (e0) is shown in formula (4).

[0067] Step S503: Calculate the first event density.

[0068] Among them, the embodiment of the present application can count the first event density p(e0) of event e0. The event neighborhood density represents the correlation degree of events with different timestamps and the distribution of events in the neighborhood. Its calculation formula can be, but is not limited to, expressed as:

[0069] p(e0)=∑e j ,e j ∈H i (e), (5)

[0070] Wherein, p(e0) is the density value of the pseudo image frame statistical event e0.

[0071] It can be understood that, in the embodiment of the present application, at each event position, the first event density p(e0) accumulated within a fixed time interval Δt is used to describe the sparsity of the event e0.

[0072] Step S504: Determine and eliminate noise events.

[0073] In this embodiment, the first certain event density threshold d Th After removing noise events, the filtering result is represented by a binary function D(e0), which can be expressed as but not limited to:

[0074]

[0075] Where D(e0) is a binary function used to determine whether the initial pseudo image frame is a noise event, where D(e0) = 1 indicates a target event and D(e0) = 0 indicates a noise event; Th is the first certain event density threshold, which can be specifically set by those skilled in the art according to actual conditions, and is not specifically limited in this application.

[0076] Step S505: Generate a first pseudo image frame.

[0077] In this embodiment of the present application, noise events in the initial pseudo image frame can be eliminated, and target events in the initial pseudo image frame can be retained to obtain a first pseudo image frame.

[0078] In step S203, if the first pseudo image frame does not meet the preset noise reduction conditions, the adaptive filtering characteristics of the pre-established multiple neighborhood radius local sliding window are used to eliminate noise events in the first pseudo image frame based on the corresponding event density information to obtain a second pseudo image frame, and based on the second pseudo image frame, the center of mass information of the detected space target is calculated.

[0079] It can be understood that the embodiment of the present application can use the signal-to-noise ratio to evaluate whether the first pseudo image frame meets certain noise reduction conditions. If the signal-to-noise ratio reaches 4dB or above, it is determined that the pseudo image frame meets certain noise reduction conditions. The specific settings can be made by technicians in this field according to actual conditions, and this application does not impose any specific restrictions.

[0080] However, when the aircraft is in a highly dynamic maneuvering state, the spatial distribution of the target event stream will be "stretched" in a very short time. In this case, relying solely on a circular local sliding window for density calculation may retain noise events due to a larger neighborhood, significantly reducing the noise reduction effect.

[0081] Those skilled in the art will appreciate that, in the embodiment of the present application, when the first pseudo image frame does not meet certain noise reduction conditions, multiple neighborhood radii, such as {r1, r2, ..., r K} (arranged from small to large), this application does not make specific restrictions, in order to improve the noise reduction effect, and then use the pre-established multiple neighborhood radius local sliding window to perform neighborhood search and density calculation of each scale on the first pseudo image frame, and then fuse the results of each scale to determine whether to retain the target event or eliminate the noise event.

[0082] Furthermore, the embodiments of the present application can use the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm to realize the sub-pixel level centroid calculation of the detected spatial target, or other methods can be used. The specific settings can be made by technical personnel in this field according to actual conditions, and this application does not impose any specific restrictions.

[0083] Optionally, in one embodiment of the present application, before utilizing the adaptive filtering characteristics of a pre-established multiple neighborhood radius local sliding window to eliminate noise events in the first pseudo image frame based on corresponding event density information, it also includes: determining at least two multiple neighborhood radii corresponding to the multiple neighborhood radius local sliding window based on the first pseudo image frame; and constructing a multiple neighborhood radius local sliding window based on the at least two multiple neighborhood radii.

[0084] It is understandable that the embodiment of the present application can define a multiple neighborhood radius set R = {r1, r2, ..., r K}, k≥2, where 0 <r1<r2<…<r K For the first pseudo image frame N i For event e0 in the example, the corresponding local sliding windows are defined under different multiple neighborhood radii. It can be expressed as, but not limited to:

[0085]

[0086] In some embodiments, the embodiments of the present application may construct a multiple neighborhood radius local sliding window based on the first pseudo image frame using at least two multiple neighborhood radii.

[0087] Optionally, in one embodiment of the present application, the adaptive filtering characteristics of a pre-established local sliding window with multiple neighborhood radii are utilized to eliminate noise events in the first pseudo image frame based on corresponding event density information to obtain a second pseudo image frame, including: using a pre-established local sliding window with multiple neighborhood radii to calculate the second event density corresponding to each event in the first pseudo image frame at different multiple neighborhood radii; calculating a third event density for each event based on a pre-constructed multi-scale fusion function, different multiple neighborhood radii, and the second event density corresponding to different multiple neighborhood radii; determining whether the third event density is greater than or equal to a second preset event density threshold; if the third event density is greater than or equal to the second preset event density threshold, determining that the event corresponding to the third event density is a target event, and retaining the target event in the first pseudo image frame; if the third event density is less than the second preset event density threshold, determining that the event corresponding to the third event density is a noise event, and eliminating the noise event in the first pseudo image frame; and obtaining the second pseudo image frame based on the target event retained in the first pseudo image frame.

[0088] In some embodiments, the present invention can be combined with Figure 6 As shown, the adaptive filtering characteristics of the pre-established multiple neighborhood radius local sliding window are used to remove noise events in the first pseudo image frame based on the corresponding event density information, thereby obtaining a second pseudo image frame, the main contents of which can be:

[0089] Step S601: define a set of multiple neighborhood radii and construct a local sliding window of multiple neighborhood radii.

[0090] In the embodiment of the present application, the expression of the multiple neighborhood radius set can be, but is not limited to, R={r1, r2, ..., r K}, k≥2, where 0 <r1<r2<…<r K .

[0091] Step S602: Calculate the second event density.

[0092] In this embodiment of the present application, at each radius r K Under this condition, the local neighborhood density p of event e0 can be calculated k (e0), the calculation formula can be but not limited to:

[0093]

[0094] Among them, p k (e0) indicates how many events are close enough to e0 in the neighborhood of the k-th scale.

[0095] Step S603: Calculate the third event density.

[0096] In this embodiment, a multi-scale fusion function M(e0) is introduced to integrate density information at multiple scales to characterize the overall neighborhood density value of event e0 in neighborhoods of different radii. The overall neighborhood density can be understood as maintaining a certain density at all scales to identify it as a target event, thereby eliminating noise events. The expression of the multi-scale fusion function can be, but is not limited to,:

[0097]

[0098] Among them, M(e0) is expressed as a multi-scale fusion function, p k (e0) is the local neighborhood density of the event.

[0099] Step S604: Determine and eliminate noise events.

[0100] Among them, the embodiment of the present application can set an event overall density threshold M th , used to determine whether event e0 is a target event or a noise event. If the third event density is greater than or equal to the second first certain event density threshold, the event is determined to be a target event, retained, and recorded as 1; otherwise, the event is determined to be a noise event, eliminated, and recorded as 0. The second first certain event density threshold can be set by those skilled in the art according to actual circumstances and is not specifically limited in this application.

[0101] Step S605: Generate a second pseudo image frame.

[0102] In step S204 , if the first pseudo image frame meets the preset noise reduction condition, the centroid information of the detected space object is calculated based on the first pseudo image frame.

[0103] In some embodiments, the embodiments of the present application may use the DBSCAN algorithm to calculate the centroid information of the detected space target in the first pseudo image frame when the first pseudo image frame meets certain noise reduction conditions.

[0104] In the embodiment of the present application, the DBSCAN algorithm relies on two key parameters: the neighborhood radius and the minimum number of samples per event cluster. In the first pseudo image frame, the extraction of the centroid of the detection space object depends on the clustering effect of the DBSCAN algorithm, and its mathematical expression can be, but is not limited to, expressed as:

[0105]

[0106] Among them, the center point coordinates (x c ,y c) is the weighted average of all event spatial coordinates within each event cluster. Each centroid corresponds to the position of the detection space target in the current frame. By analyzing the changes in the centroid position across multiple consecutive frames, the motion trajectory of the detection space target can be tracked.

[0107] In step S205 , it is determined whether the centroid information in the second pseudo image frame meets a preset accuracy condition, and / or whether the centroid information in the first pseudo image frame meets a preset accuracy condition.

[0108] In some embodiments, embodiments of the present application can determine whether the centroid information in the second pseudo image frame meets a certain accuracy condition, and whether the centroid information in the first pseudo image frame meets a certain accuracy condition. The certain accuracy condition can be set by those skilled in the art based on actual conditions and is not specifically limited by this application.

[0109] In some embodiments, the embodiments of the present application may determine whether the centroid information in the second pseudo image frame meets certain accuracy conditions.

[0110] In some embodiments, the embodiments of the present application may determine whether the centroid information in the first pseudo image frame meets certain accuracy conditions.

[0111] In step S206, if the center of mass information in the second pseudo image frame does not meet the preset accuracy conditions, and / or the center of mass information in the first pseudo image frame does not meet the preset accuracy conditions, the motion trajectory of the detected space target is corrected using the inertial measurement unit data to obtain the center of mass information that meets the preset accuracy conditions, and the motion information of the detected space target is obtained based on the center of mass information of the preset accuracy conditions.

[0112] It is understandable that in some ultra-high dynamic environments, the target trajectory captured by the event camera in the embodiments of the present application is often distorted due to the extremely rapid movement and non-uniform speed changes of the camera, and the collected data shows significant discontinuity and large data gaps, which affects the accuracy of centroid extraction.

[0113] Therefore, to compensate for the trajectory distortion caused by the ultra-high-speed motion of the event camera and improve the accuracy of target center of mass extraction, the present embodiment introduces camera motion acceleration and angular velocity information provided by the inertial measurement unit to infer and correct the motion trajectory of the event camera. The motion information provided by the inertial measurement unit data in real time helps to build an accurate camera motion model, thereby calculating the position and posture of the camera at any time, achieving accurate compensation for the trajectory distortion generated during motion, and inferring and correcting the motion trajectory of the detected spatial target.

[0114] As a possible implementation method, embodiments of the present application can use inertial measurement unit data to correct the motion trajectory of the detected space target when the center of mass information (wherein the center of mass information may include the center of mass information calculated from the first pseudo image frame, or may include the center of mass information calculated from the second pseudo image frame) does not meet a certain accuracy condition, thereby obtaining center of mass information that meets the certain accuracy condition, thereby determining the motion information of the detected space target. The certain accuracy condition can be set by those skilled in the art based on actual conditions and is not specifically limited by this application.

[0115] For example, the embodiment of the present application evaluates the extraction accuracy of the centroid information, which can be compared with the calibration value of a traditional frame camera. For example, a star sensor generally has an error of no more than 0.5 pixels and is considered to meet certain accuracy conditions. The specific setting can be made by technicians in this field according to actual conditions, and this application does not impose any specific restrictions.

[0116] Optionally, in one embodiment of the present application, the motion trajectory of the detected space target is corrected using inertial measurement unit data to obtain center of mass information that meets preset accuracy conditions, including: obtaining acceleration data and angular velocity data in the inertial measurement unit data; integrating the acceleration data to obtain velocity information and position information of the camera corresponding to the inertial measurement unit; integrating the angular velocity data to obtain attitude information of the camera corresponding to the inertial measurement unit; based on the velocity information, position information and / or attitude information, using a pre-built motion compensation model to correct the motion trajectory of the detected space target to obtain center of mass information that meets the preset accuracy conditions.

[0117] It can be understood that in the embodiment of the present application, the movement of the event camera can be divided into two parts: translational movement and rotational movement. The acceleration data and angular velocity data provided by the inertial measurement unit can be used to describe these two parts of the movement, respectively, and then the motion trajectory of the detected space target is corrected using the pre-built motion compensation model to obtain the center of mass information that meets certain accuracy conditions.

[0118] In this embodiment of the present application, the acceleration provided by the inertial measurement unit can be measured by an accelerometer, and the acceleration expression can be a(t)=[a x (t),a y (t),a z (t)] T , and then calculate the speed and position of the camera. Assume that the initial speed of the camera is v0 and the position is P0. By integrating the acceleration, the speed v(t) and position P(t) of the camera at any time t can be obtained. The calculation formula can be, but is not limited to:

[0119]

[0120] Among them, v0 is the velocity of the camera at the initial moment; v(t) is the velocity of the camera at any time t.

[0121]

[0122] Where P0 is the initial position of the camera, and P(t) is the position of the camera at any time t. By double integrating the acceleration, the camera's position change in three-dimensional space is obtained.

[0123] Furthermore, in the embodiment of the present application, the angular velocity ω(t) provided by the gyroscope is [ω x (t),ω y (t),ω z (t)] T Describes the rotation rate of the camera on three axes. By integrating the angular velocity, the camera's posture (i.e., the rotation matrix R(t)) can be obtained. The calculation formula can be, but is not limited to,:

[0124]

[0125] Where R0 is the initial pose of the camera and ω(t′) is the angular velocity measured at time t′. The integral of the angular velocity provides the change in the camera's rotation in 3D space.

[0126] That is to say, in the embodiment of the present application, the acceleration data and angular velocity data provided by the inertial measurement unit and the event point data captured by the event camera have different sampling frequencies. The event camera records highly dynamically changing scene information, while the inertial measurement unit data is used to infer the motion state of the camera at each moment. In practical applications, the inertial measurement unit data can be adjusted to a moment consistent with the event camera sampling time by an interpolation method, or other methods can be used. The specific settings can be made by technicians in this field according to actual conditions, and this application does not impose specific restrictions. After time synchronization, the acceleration data and angular velocity data of the inertial measurement unit can be accurately aligned with the data of each event point of the event camera, thereby providing camera motion parameters for each event point.

[0127] By using the acceleration data and angular velocity data provided by the inertial measurement unit, the event point can be corrected. Let the event point at a certain moment captured by the event camera be e k (t)=(t,x k ,p k ), where t is the timestamp of the event, x2 is the position of the event point in the image, and p kis the polarity of the event (enhancement or weakening). Based on the acceleration data and angular velocity data provided by the inertial measurement unit, assuming that the motion state of the camera when capturing the event point is P(t) and R(t), the spatial coordinate compensation of the event point is performed using formula (14), and its expression can be but is not limited to:

[0128] x′ k =T(x k ,t;P(t),R(t)),(14)

[0129] Where x′ k is the position of the event point after compensation, T(x k ,t; P(t), R(t)) is the motion compensation model, P(t) and R(t) are the position and posture of the camera at time t respectively.

[0130] The above content has constructed a method for estimating camera motion using inertial measurement unit data to compensate for high-dynamic star point motion trajectories. However, the inertial measurement unit sensor itself will introduce high-frequency random noise under dynamic conditions, and may also accumulate offset and drift problems caused by factors such as zero bias and temperature changes during long-term use. In order to further improve the accuracy of target trajectory compensation, the embodiment of the present application can use a particle filter algorithm to correct the compensation error caused by noise or sensor offset. Among them, the particle filter algorithm is a state estimation algorithm based on the Monte Carlo method, which estimates the true state of the system by introducing multiple hypothetical states (particles) and assigning weights to each particle, thereby overcoming the limitations of traditional Kalman filtering in dealing with nonlinear and non-Gaussian noise. The specific content is as follows Figure 7 As shown:

[0131] Step S701: Initialize the particle swarm.

[0132] In the embodiment of the present application, the first step of the particle filter algorithm is to initialize a set of particles, each of which represents the possible state of the camera at a certain moment. Each particle represents the camera's position, speed, angle and other motion parameters. The positions of these particles can be initialized by the initial data of the inertial measurement unit, or randomly initialized according to the prior motion model. Set the state variable x of each particle k =[P k ,v k ,R k ] T , where P k is the position, v k is the speed, R k is the attitude (rotation matrix).

[0133] Step S702: performing particle prediction based on inertial measurement unit data.

[0134] In this embodiment of the present application, at each time step k, the particle can be measured based on the state at the previous moment and the acceleration a(t k ) and angular velocity ω(t k ) to predict the current state, which can be achieved through a motion model, wherein the expression of the model can be but is not limited to:

[0135]

[0136] Among them, a(t k ) is the acceleration measured by the inertial measurement unit sensor; Δt is the time interval measured by the inertial measurement unit sensor.

[0137] Step S703: performing particle update and resampling based on event data and observations.

[0138] Among them, in the embodiment of the present application, the key to the particle filter algorithm is to update the weight of the particle according to the observation data provided by the event camera. The weight of each particle is proportional to the matching degree between its predicted state and the actual observation data. For each particle i, its matching degree with the event camera data can be calculated to obtain the particle weight Particles with higher matching degree will get higher weight, and vice versa. The expressions can be, but are not limited to, pseudo-:

[0139]

[0140] Among them, z k is the observation data obtained from the event camera, is the match between the particle and the event data.

[0141] Furthermore, the particle filter algorithm resamples the particles according to their weights to obtain a new set of particles. Through resampling, the particle filter can eliminate the influence of low-weight particles and focus on particles that better match the actual observed data. Resampling helps improve the accuracy and robustness of the system.

[0142] Step S704: state estimation.

[0143] Among them, the embodiment of the present application can obtain the optimal estimate of the current moment by weighted averaging the state of the particles. The particle filter obtains the final motion state of the camera by weighted averaging all particles, and its expression can be but is not limited to:

[0144]

[0145] in, is the particle weight; is the state variable of the particle, expressed as in, For location, For speed, is the attitude (rotation matrix).

[0146] Step S705: Track repair and compensation.

[0147] The present embodiment can correct the star point trajectory captured by the event camera based on the optimal estimated state. The particle filter algorithm compensates for lost or distorted event points, corrects the deviation of the event camera trajectory, and restores the original target event trajectory.

[0148] In step S207, if the centroid information in the second pseudo image frame meets the preset accuracy condition, and the centroid information in the first pseudo image frame meets the preset accuracy condition, the motion information of the detected space target is obtained based on the centroid information of the preset accuracy condition.

[0149] During actual execution, in an embodiment of the present application, when the centroid information in the second pseudo image frame meets certain accuracy conditions and the centroid information in the first pseudo image frame meets certain accuracy conditions, the centroid information is used to confirm the motion information of the detected space target.

[0150] The working principle of the dynamic space target detection method based on event information proposed in the embodiments of the present application is introduced below in combination with multiple embodiments.

[0151] Example 1:

[0152] in, Figure 8 The present invention provides an overall flow chart of the working principle of a dynamic space target detection method based on event information according to an embodiment of the present application.

[0153] Step S801: target event stream processing.

[0154] Among them, the embodiment of the present application can read the target event stream containing the detected space target in the acquired dynamic flight scene according to a preset time duration, thereby generating an initial pseudo image frame, and using a pre-established circular local sliding window to eliminate noise events to obtain a first pseudo image frame.

[0155] Step S802: Noise reduction effect evaluation and reprocessing.

[0156] In particular, the embodiment of the present application can determine whether the first pseudo image frame meets certain noise reduction conditions, and if not, use a pre-established multiple neighborhood radius local sliding window to perform further elimination to obtain a second pseudo image frame.

[0157] Step S803: Calculate the centroid information of the detected space target.

[0158] In this embodiment of the present application, the DBSCAN algorithm may be used to calculate the centroid information of the detected space target in the first pseudo image frame, and / or the centroid information of the detected space target in the second pseudo image frame.

[0159] Step S804: Extracting precision compensation.

[0160] Among them, the embodiment of the present application can use the inertial measurement unit data to make corrections when the center of mass information does not meet certain accuracy conditions, and then determine the motion information of the detected space target.

[0161] Example 2:

[0162] in, Figure 9 The present invention provides an overall flow chart of a method for detecting dynamic space targets based on event information according to another embodiment of the present application.

[0163] Step S901: Acquire a target event stream containing detected space targets in a dynamic flight scene.

[0164] Step S902: Generate an initial pseudo image frame containing the detected space target.

[0165] Step S903: constructing a circular local sliding window.

[0166] Step S904: Calculate event density.

[0167] The event density may include, but is not limited to, a first event density calculated by a circular local sliding window, and a second event density and a third event density calculated by a local sliding window with multiple neighborhood radii.

[0168] Step S905: Determine whether the event density is greater than or equal to a first preset event density threshold.

[0169] If it is greater than or equal to, execute step S906; otherwise, execute step S907.

[0170] Step S907: Determine that the event is a noise event and remove it.

[0171] Step S908: Determine that the event is a target event and retain it.

[0172] Step S909: Generate a first dummy image frame or a second dummy image frame.

[0173] Step S910: determining whether the first dummy image frame or the second dummy image frame satisfies a certain noise reduction condition.

[0174] If not, execute step S911; otherwise, execute step S912.

[0175] Step S911: Construct a local sliding window with multiple neighborhood radii.

[0176] Among them, the embodiment of the present application can utilize the pre-established adaptive filtering characteristics of the multiple neighborhood radius local sliding window to eliminate noise events in the first pseudo image frame based on the corresponding event density information to obtain the second pseudo image frame.

[0177] Step S912: Calculate the centroid information of the detected space target using the DBSCAN algorithm.

[0178] Step S913: Determine whether the centroid information meets certain accuracy conditions.

[0179] In this embodiment of the present application, when the conditions are met, step S914 is executed; otherwise, step S915 is executed.

[0180] Step S914: Determine the motion information of the detected space target.

[0181] Step S915: Acquire inertial measurement unit data.

[0182] Step S916: Perform correction using inertial measurement unit data.

[0183] Among them, the content of the correction using the inertial measurement unit data in the embodiment of the present application is as follows Figure 7 shown.

[0184] According to the event information-based dynamic space target detection method proposed in the embodiment of the present application, the target event stream containing the detected space target in a dynamic flight scene can be read according to a certain time duration to obtain an initial pseudo image frame. Noise events are eliminated using a pre-established circular local sliding window to obtain a first pseudo image frame. If the first pseudo image frame does not meet certain noise reduction conditions, the noise events are further eliminated using a pre-established multiple neighborhood radius local sliding window to obtain a second pseudo image frame. The center of mass information of the detected space target is then calculated and whether the center of mass information meets certain accuracy conditions is determined. If not, the inertial measurement unit data is used to correct it and the motion information of the detected space target is determined. Otherwise, the motion information of the detected space target is directly determined. This can significantly improve the spatiotemporal resolution of data acquisition, increase the dynamic range, and reduce power consumption, achieving good real-time detection and tracking effects, laying a solid foundation for the navigation and control of hypersonic aircraft. This solves the problems of low space target extraction accuracy and tracking difficulties in high-dynamic maneuvering scenarios in related technologies.

[0185] Next, a dynamic space target detection device based on event information according to an embodiment of the present application will be described with reference to the accompanying drawings.

[0186] Figure 10Schematic diagram of a block diagram of a dynamic space target detection device based on event information according to an embodiment of the present application.

[0187] like Figure 10 As shown, the dynamic space target detection device 20 based on event information includes: an acquisition module 100, a first judgment module 200, a first calculation module 300, a second calculation module 400, a second judgment module 500, a first generation module 600 and a second generation module 700.

[0188] The acquisition module 100 is used to acquire a target event stream containing a detected space target in a dynamic flight scene, and read the target event stream according to a preset time duration to obtain an initial pseudo image frame containing the detected space target.

[0189] The first judgment module 200 is used to use a pre-established circular local sliding window to eliminate noise events in the initial pseudo image frame based on corresponding event density information to obtain a first pseudo image frame, and to determine whether the first pseudo image frame meets a preset noise reduction condition.

[0190] The first calculation module 300 is used to use the pre-established adaptive filtering characteristics of the local sliding window with multiple neighborhood radius to eliminate noise events in the first pseudo image frame based on the corresponding event density information when the first pseudo image frame does not meet the preset noise reduction conditions, so as to obtain a second pseudo image frame, and calculate the center of mass information of the detected space target based on the second pseudo image frame.

[0191] The second calculation module 400 is configured to calculate the centroid information of the detected space target based on the first pseudo image frame when the first pseudo image frame meets a preset noise reduction condition.

[0192] The second judgment module 500 is configured to judge whether the centroid information in the second pseudo image frame satisfies a preset accuracy condition, and / or whether the centroid information in the first pseudo image frame satisfies a preset accuracy condition.

[0193] The first generation module 600 is used to correct the motion trajectory of the detected space target using inertial measurement unit data when the center of mass information in the second pseudo image frame does not meet the preset accuracy condition and / or the center of mass information in the first pseudo image frame does not meet the preset accuracy condition to obtain the center of mass information that meets the preset accuracy condition, and obtain the motion information of the detected space target based on the center of mass information of the preset accuracy condition.

[0194] The second generation module 700 is used to obtain motion information of the detected space target based on the centroid information of the preset accuracy condition when the centroid information in the second pseudo image frame meets the preset accuracy condition and the centroid information in the first pseudo image frame meets the preset accuracy condition.

[0195] Optionally, in one embodiment of the present application, the first judgment module 200 includes: a first calculation unit, a first judgment unit, a first determination unit, a second determination unit and a first generation unit.

[0196] The first calculation unit is configured to calculate a first event density of each event in the initial pseudo image frame using a pre-established circular local sliding window.

[0197] The first judging unit is configured to judge whether the first event density is greater than or equal to a first preset event density threshold.

[0198] The first determination unit is configured to determine, when the first event density is greater than or equal to a first preset event density threshold, that the event corresponding to the first event density is a target event, and retain the target event in the initial pseudo image frame.

[0199] The second determination unit is configured to determine, when the first event density is less than a first preset event density threshold, that the event corresponding to the first event density is a noise event, and to remove the noise event from the initial pseudo image frame.

[0200] The first generating unit is configured to obtain a first pseudo image frame based on a target event retained in the initial pseudo image frame.

[0201] Optionally, in one embodiment of the present application, it further includes: a determination module and a construction module.

[0202] Among them, the determination module is used to determine at least two multiple neighborhood radii corresponding to the multiple neighborhood radius local sliding window based on the first pseudo image frame before using the adaptive filtering characteristics of the pre-established multiple neighborhood radius local sliding window to eliminate noise events in the first pseudo image frame based on the corresponding event density information.

[0203] A construction module is used to construct a multiple neighborhood radius local sliding window based on at least two multiple neighborhood radii.

[0204] Optionally, in one embodiment of the present application, the first calculation module 300 includes: a second calculation unit, a third calculation unit, a second judgment unit, a third judgment unit, a fourth judgment unit and a second generation unit.

[0205] The second calculation unit is configured to calculate a second event density corresponding to each event in the first pseudo image frame at different multiple neighborhood radii using a pre-established local sliding window with multiple neighborhood radii.

[0206] The third calculation unit is used to calculate the third event density of each event based on the pre-constructed multi-scale fusion function, different multiple neighborhood radii and the second event density corresponding to the different multiple neighborhood radii.

[0207] The second judging unit is configured to judge whether the third event density is greater than or equal to a second preset event density threshold.

[0208] The third determination unit is configured to determine, when the third event density is greater than or equal to a second preset event density threshold, that the event corresponding to the third event density is a target event, and retain the target event in the first pseudo image frame.

[0209] The fourth determination unit is configured to determine, when the third event density is less than a second preset event density threshold, that the event corresponding to the third event density is a noise event, and remove the noise event from the first pseudo image frame.

[0210] The second generating unit is configured to obtain a second pseudo image frame based on the target event retained in the first pseudo image frame.

[0211] Optionally, in one embodiment of the present application, the first generating module 600 includes: an acquiring unit, a first integrating unit, a second integrating unit, and a correcting unit.

[0212] The acquisition unit is used to acquire acceleration data and angular velocity data from the inertial measurement unit data.

[0213] The first integration unit is used to integrate the acceleration data to obtain velocity information and position information of the inertial measurement unit corresponding to the camera.

[0214] The second integration unit is used to integrate the angular velocity data to obtain the attitude information of the inertial measurement unit corresponding to the camera.

[0215] The correction unit is used to correct the motion trajectory of the detected space target based on the speed information, position information and / or posture information using a pre-built motion compensation model to obtain the center of mass information that meets the preset accuracy conditions.

[0216] It should be noted that the aforementioned explanation of the embodiment of the dynamic space target detection method based on event information is also applicable to the dynamic space target detection device based on event information of this embodiment, and will not be repeated here.

[0217] According to the event information-based dynamic space target detection device proposed in the embodiment of the present application, the target event stream containing the detected space target in a dynamic flight scene can be read according to a certain time duration to obtain an initial pseudo image frame. Noise events are eliminated using a pre-established circular local sliding window to obtain a first pseudo image frame. If the first pseudo image frame does not meet certain noise reduction conditions, the noise events are further eliminated using a pre-established multiple neighborhood radius local sliding window to obtain a second pseudo image frame. The center of mass information of the detected space target is then calculated and whether the center of mass information meets certain accuracy conditions is determined. If not, the inertial measurement unit data is used to correct it and the motion information of the detected space target is determined. Otherwise, the motion information of the detected space target is directly determined. This can significantly improve the spatiotemporal resolution of data acquisition, increase the dynamic range, and reduce power consumption, achieving good real-time detection and tracking effects, laying a solid foundation for the navigation and control of hypersonic aircraft. This solves the problems of low space target extraction accuracy and tracking difficulties in high-dynamic maneuvering scenarios in related technologies.

[0218] Figure 11 This is a schematic diagram of the structure of an electronic device provided according to an embodiment of the present application. The electronic device may include:

[0219] A memory 1101 , a processor 1102 , and a computer program stored in the memory 1101 and executable on the processor 1102 .

[0220] When the processor 1102 executes the program, the dynamic space target detection method based on event information provided in the above embodiment is implemented.

[0221] Furthermore, the electronic device further includes:

[0222] The communication interface 1103 is used for communication between the memory 1101 and the processor 1102 .

[0223] The memory 1101 is used to store computer programs that can be run on the processor 1102 .

[0224] The memory 1101 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0225] If the memory 1101, processor 1102, and communication interface 1103 are implemented independently, the communication interface 1103, memory 1101, and processor 1102 can be interconnected via a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 11 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0226] Optionally, in a specific implementation, if the memory 1101, the processor 1102 and the communication interface 1103 are integrated on a chip, the memory 1101, the processor 1102 and the communication interface 1103 can communicate with each other through an internal interface.

[0227] The processor 1102 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0228] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned dynamic space target detection method based on event information.

[0229] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed, implements the above-mentioned dynamic space target detection method based on event information.

[0230] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0231] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Thus, a feature specified as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of this application, "N" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0232] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing a custom logical function or process step, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed in a different order than shown or discussed, including performing functions in a substantially simultaneous manner or in a reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application pertain.

[0233] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or N wires (electronic devices), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program can be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing it in other suitable ways as necessary, and then storing it in a computer memory.

[0234] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiment, the N steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented using hardware, as in another embodiment, it can be implemented using any one or a combination of the following technologies known in the art: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0235] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0236] In addition, the functional units in the various embodiments of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into a module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0237] The storage medium mentioned above may be a read-only memory, a magnetic disk, or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application. Persons skilled in the art may make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.

Claims

1. A dynamic space target detection method based on event information, characterized in that: The following steps are involved: Acquire a target event stream containing a detected space target in a dynamic flight scene, and read the target event stream according to a preset time duration to obtain an initial pseudo image frame containing the detected space target; Using a pre-established circular local sliding window, noise events in the initial pseudo image frame are removed based on corresponding event density information to obtain a first pseudo image frame, and determining whether the first pseudo image frame meets a preset noise reduction condition; If the first pseudo image frame does not meet the preset noise reduction condition, using a pre-established adaptive filtering characteristic of a local sliding window with multiple neighborhood radii to remove noise events in the first pseudo image frame based on corresponding event density information to obtain a second pseudo image frame, and calculating the centroid information of the detection space target based on the second pseudo image frame; If the first pseudo image frame satisfies the preset noise reduction condition, calculating the centroid information of the detection space target based on the first pseudo image frame; Determining whether the centroid information in the second pseudo image frame satisfies a preset accuracy condition, and / or whether the centroid information in the first pseudo image frame satisfies the preset accuracy condition; If the centroid information in the second pseudo image frame does not meet the preset accuracy condition, and / or the centroid information in the first pseudo image frame does not meet the preset accuracy condition, correcting the motion trajectory of the detection space target using inertial measurement unit data to obtain centroid information that meets the preset accuracy condition, and obtaining the motion information of the detection space target based on the centroid information that meets the preset accuracy condition; If the centroid information in the second pseudo image frame meets the preset accuracy condition, and the centroid information in the first pseudo image frame meets the preset accuracy condition, the motion information of the detection space target is obtained based on the centroid information of the preset accuracy condition.

2. The method according to claim 1, characterized in that The method of using a pre-established circular local sliding window to remove noise events in the initial pseudo image frame based on corresponding event density information to obtain a first pseudo image frame includes: Calculating a first event density of each event in the initial pseudo image frame using the pre-established circular local sliding window; Determining whether the first event density is greater than or equal to a first preset event density threshold; If the first event density is greater than or equal to the first preset event density threshold, determining that the event corresponding to the first event density is a target event, and retaining the target event in the initial pseudo image frame; If the first event density is less than the first preset event density threshold, determining that the event corresponding to the first event density is a noise event, and removing the noise event from the initial pseudo image frame; The first pseudo image frame is obtained based on the target event retained in the initial pseudo image frame.

3. The method according to claim 1, characterized in that Before removing noise events in the first pseudo image frame based on corresponding event density information by using the pre-established adaptive filtering characteristics of the local sliding window with multiple neighborhood radii, the method further includes: determining, based on the first pseudo image frame, at least two multiple neighborhood radii corresponding to a multiple neighborhood radius local sliding window; Based on the at least two multiple neighborhood radii, the multiple neighborhood radius local sliding window is constructed.

4. The method according to claim 3, characterized in that The method of utilizing the pre-established adaptive filtering characteristics of the local sliding window with multiple neighborhood radii to remove noise events in the first pseudo image frame based on corresponding event density information to obtain a second pseudo image frame includes: Calculating a second event density corresponding to each event in the first pseudo image frame at different multiple neighborhood radii using the pre-established multiple neighborhood radius local sliding window; Calculating a third event density for each event based on a pre-constructed multi-scale fusion function, the different multiple neighborhood radii, and the second event densities corresponding to the different multiple neighborhood radii; Determining whether the third event density is greater than or equal to a second preset event density threshold; If the third event density is greater than or equal to the second preset event density threshold, determining that the event corresponding to the third event density is a target event, and retaining the target event in the first pseudo image frame; If the third event density is less than the second preset event density threshold, determining that the event corresponding to the third event density is a noise event, and removing the noise event from the first pseudo image frame; The second pseudo image frame is obtained based on the target event retained in the first pseudo image frame.

5. The method according to claim 1, wherein The method of using the inertial measurement unit data to correct the motion trajectory of the detected space target to obtain the center of mass information that meets the preset accuracy condition includes: Acquiring acceleration data and angular velocity data from the inertial measurement unit data; Integrating the acceleration data to obtain velocity information and position information of the inertial measurement unit corresponding to the camera; Integrating the angular velocity data to obtain attitude information of the camera corresponding to the inertial measurement unit; Based on the speed information, the position information and / or the posture information, the motion trajectory of the detection space target is corrected using a pre-built motion compensation model to obtain center of mass information that meets the preset accuracy condition.

6. A dynamic space target detection device based on event information, characterized in that: include: an acquisition module, configured to acquire a target event stream containing a detected space target in a dynamic flight scene, and read the target event stream according to a preset time duration to obtain an initial pseudo image frame containing the detected space target; a first determination module, configured to use a pre-established circular local sliding window to remove noise events from the initial pseudo image frame based on corresponding event density information to obtain a first pseudo image frame, and determine whether the first pseudo image frame meets a preset noise reduction condition; a first calculation module, configured to, when the first pseudo image frame does not meet the preset noise reduction condition, utilize a pre-established adaptive filtering characteristic of a local sliding window with multiple neighborhood radii to remove noise events in the first pseudo image frame based on corresponding event density information to obtain a second pseudo image frame, and calculate the centroid information of the detection space target based on the second pseudo image frame; a second calculation module, configured to calculate, based on the first pseudo image frame, the centroid information of the detection space target when the first pseudo image frame satisfies the preset noise reduction condition; a second determining module, configured to determine whether the centroid information in the second pseudo image frame satisfies a preset accuracy condition, and / or whether the centroid information in the first pseudo image frame satisfies the preset accuracy condition; a first generating module, configured to, when the center of mass information in the second pseudo image frame does not satisfy a preset accuracy condition and / or the center of mass information in the first pseudo image frame does not satisfy the preset accuracy condition, correct the motion trajectory of the detection space target using inertial measurement unit data to obtain center of mass information that satisfies the preset accuracy condition, and obtain motion information of the detection space target based on the center of mass information that satisfies the preset accuracy condition; The second generation module is used to obtain the motion information of the detection space target based on the centroid information of the preset accuracy condition when the centroid information in the second pseudo image frame meets the preset accuracy condition and the centroid information in the first pseudo image frame meets the preset accuracy condition.

7. The device according to claim 6, characterized in that The first judgment module includes: a first calculating unit, configured to calculate a first event density of each event in the initial pseudo image frame by using the pre-established circular local sliding window; a first determining unit, configured to determine whether the first event density is greater than or equal to a first preset event density threshold; a first determining unit, configured to determine, when the first event density is greater than or equal to the first preset event density threshold, that the event corresponding to the first event density is a target event, and retain the target event in the initial pseudo image frame; a second determining unit, configured to determine, when the first event density is less than the first preset event density threshold, that the event corresponding to the first event density is a noise event, and remove the noise event from the initial pseudo image frame; The first generating unit is configured to obtain the first pseudo image frame based on the target event retained in the initial pseudo image frame.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method for detecting dynamic space targets based on event information as described in any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the dynamic space target detection method based on event information as described in any one of claims 1 to 5.

10. A computer program product, characterized in that The invention comprises a computer program, which, when executed, is used to implement the dynamic space target detection method based on event information as described in any one of claims 1 to 5.

Images