A space-based infrared air target dynamic vision simulation system and method

Abstract

The present application relates to the field of infrared detection and infrared space technology, and particularly relates to a space-based infrared aerial target dynamic vision simulation system, which comprises a parameter configuration unit, a scene construction unit, an event simulation unit, and a data alignment and output unit. The parameter configuration unit obtains configuration information. The scene construction unit generates an infrared scene sequence according to the configuration information. The event simulation unit performs inter-frame difference processing and dynamic threshold polarity judgment on the infrared scene sequence, generates an asynchronous event stream with an event timestamp, and outputs an infrared image and records an image timestamp. The data alignment and output unit aligns the asynchronous event stream and the infrared image based on the image timestamp and the event timestamp, and outputs the infrared image. The present application also comprises a method. The present application realizes a full-process closed-loop output from parameter initialization, space-based detection scene construction, infrared dynamic vision data generation, to cross-modal alignment and data saving, and improves controllability and reproducibility of data generation.

Description

Technical Field

[0001] This invention relates to the fields of infrared detection and infrared space technology, and in particular to a space-based infrared aerial target dynamic visual simulation system and method. Background Technology

[0002] Dynamic vision sensors are a novel type of neuromorphic sensor capable of independently and asynchronously responding to brightness changes in individual pixels on an image surface. When the brightness change of a pixel exceeds a set threshold, it outputs polarized event data. Currently, infrared dynamic vision sensors are still in the early stages of research and development. High manufacturing costs, insufficient technological maturity, and an incomplete industrial chain result in their scarcity in the market, making it difficult to support large-scale applications. Existing systems are mainly concentrated in research institutions and a few innovative enterprises.

[0003] Currently, dynamic vision has made significant progress in detecting and tracking large, high-contrast targets such as those used in autonomous driving and drone obstacle avoidance. However, research in the observation of weak targets is clearly insufficient. For space-based reconnaissance missions, aerial targets often exhibit weak characteristics due to the combined effects of long distances, severe atmospheric attenuation, and the low detectability of new stealth aircraft. Research on using infrared dynamic vision for space-based target detection is still in the demonstration stage, and the scarcity of publicly available datasets restricts the development and verification of event-based processing algorithms.

[0004] Regarding existing technologies and the inventors' previous work, patent CN118552715 A mainly focuses on modeling and simulating asynchronous event flows, but it does not provide an integrated simulation link for scene construction and multimodal alignment of infrared images and events for space-based infrared aerial target detection missions. Therefore, it is difficult to directly use it for simulation verification and algorithm evaluation of space-based weak target detection. Patent CN119579917 A proposes an adaptive differential event detection method for infrared moving targets, which uses an adaptive threshold to trigger events. However, it does not clearly provide a specific method for adaptive threshold adjustment, and it relies on the user's own original infrared scene sequence, resulting in high costs for scene acquisition and annotation. Simulating time-varying backgrounds (platform jitter, noise), target trajectory generation, and sub-pixel point diffusion injection required for space-based aerial target detection is difficult to implement, leading to a high application threshold. Summary of the Invention

[0005] The purpose of this invention is to provide a space-based infrared aerial target dynamic visual simulation system and method, which mainly solves the problems existing in the prior art and provides key data support for the development of subsequent event stream processing algorithms.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is to provide a space-based infrared aerial target dynamic visual simulation system, characterized in that it includes a parameter configuration unit, a scene construction unit, an event simulation unit, and a data alignment and output unit.

[0007] The parameter configuration unit acquires the configuration information of the simulation task; the scene construction unit, connected to the parameter configuration unit, generates an infrared scene sequence containing moving targets and dynamic backgrounds according to the configuration information; the event simulation unit, connected to the scene construction unit, performs inter-frame difference processing on the infrared scene sequence, and combines a dynamic threshold to determine event polarity, generating an asynchronous event stream, and simultaneously outputs infrared images at a preset period and records image timestamps; wherein, each event in the asynchronous event stream contains an event timestamp; the data alignment and output unit, connected to the event simulation unit, performs cross-modal alignment of the asynchronous event stream and the infrared images based on the image timestamps and the event timestamps, and outputs the data.

[0008] Furthermore, the scene construction unit includes a background generation module, a trajectory generation module, and a target injection module;

[0009] The background generation module is connected to the parameter configuration unit and performs time-varying perturbation processing on the loaded background image according to the configuration information to generate a time-varying background sequence. The trajectory generation module is connected to the parameter configuration unit and generates a multi-target motion trajectory that satisfies the minimum spacing constraint according to the configuration information. The target injection module is connected to both the background generation module and the trajectory generation module and injects energy at the sub-pixel center position of the multi-target motion trajectory using a point spread function according to the configuration information, and superimposes it with the time-varying background sequence to obtain the infrared scene sequence.

[0010] Furthermore, the trajectory generation module includes a random walk control submodule, an escort formation control submodule, and a round-trip avoidance control submodule;

[0011] The random walk control submodule plans the trajectory of the random walk mode; when the configuration information is configured as random walk mode, the random walk control submodule is activated, and sets the initial position of each target in the field of view through uniform initialization, and rearranges the targets smaller than the spacing threshold along the connecting line direction; then in each frame, the position is updated based on the motion direction of the previous frame and random turning disturbance, and the normal reflection processing is performed on the targets that exceed the field of view boundary, and the minimum spacing constraint is applied through multiple rounds of iteration throughout the process to generate the random walk trajectory;

[0012] The escort formation control submodule is used to plan the trajectory of the escort formation mode. When the configuration information is configured as escort formation mode, the escort formation control submodule is activated. First, the leader aircraft trajectory is generated. Then, based on the leader aircraft speed direction, the formation template is mapped to the cell coordinate system through coordinate transformation. The horizontal and vertical spacing is superimposed to determine the position of each wingman and generate the escort formation trajectory.

[0013] The round-trip avoidance control submodule is used to plan the trajectory of the round-trip avoidance mode. When the round-trip avoidance mode is configured in the configuration information, the round-trip avoidance control submodule is activated and generates the round-trip motion trajectory of each target within the set motion range. When the target spacing is less than the minimum avoidance spacing, without changing the trajectory time step, collision avoidance adjustment is applied by small step size and multiple iterations to generate a round-trip avoidance trajectory that meets the minimum spacing constraint.

[0014] Furthermore, the target injection module includes a point spread function generation submodule and an energy minimum constraint submodule;

[0015] The point spread function generation submodule calculates the Gaussian kernel standard deviation based on the configuration information, target radius and energy concentration, and generates a normalized point spread function at the center of the sub-pixel.

[0016] The energy minimum constraint submodule is connected to the point spread function generation submodule, and determines the target total energy based on the preset target signal-to-noise ratio or grayscale peak value; wherein, when the grayscale peak value is used to determine the target intensity, the target intensity is not lower than the target minimum grayscale peak value; when the signal-to-noise ratio is used to determine the target total energy, the target total energy is not lower than the energy lower limit determined by the target minimum signal-to-noise ratio; and further applies a global energy minimum constraint, and superimposes the target energy that meets the constraint onto the time-varying background sequence.

[0017] Furthermore, the event simulation unit includes an inter-frame difference calculation module, an adaptive threshold decision module, a trigger rate feedback control module, an asynchronous event stream construction module, and a periodic output control module;

[0018] The inter-frame difference calculation module is connected to the scene construction unit and is used to calculate the frame difference between adjacent infrared scenes;

[0019] The adaptive threshold decision module is connected to the inter-frame difference calculation module and the trigger rate feedback control module. It is used to construct a noisy dynamic threshold and make an event polarity decision based on the comparison result between the frame difference and the noisy dynamic threshold, and output a positive event or a negative event. The noisy dynamic threshold is generated based on the feedback adjustment coefficient, the base threshold, the average frame difference, and the threshold noise term.

[0020] The trigger rate feedback control module calculates the current event trigger rate; when the current event trigger rate is higher than the preset trigger rate upper limit, it feeds back to the adaptive threshold decision module to increase the noisy dynamic threshold; when the current event trigger rate is lower than the preset trigger rate lower limit, it feeds back to the adaptive threshold decision module to decrease the noisy dynamic threshold.

[0021] The asynchronous event stream construction module is connected to the adaptive threshold decision module. Based on the absolute start time, it adds a trigger delay and quantizes the event according to a preset time resolution to generate the event timestamp. The event is then sorted based on the event timestamp and target identification information is added to form the asynchronous event stream. The trigger delay includes line readout delay and noise disturbance delay.

[0022] The periodic output control module is connected to the asynchronous event stream construction module and is used to output the infrared image according to a preset period and record the image timestamp.

[0023] Furthermore, the data alignment and output unit includes an infrared image output module, an event stream output module, an event frame output module, and a cross-modal alignment module;

[0024] The infrared image output module receives the infrared image from the event simulation unit and outputs the infrared image according to a preset period based on the image timestamp; the event stream output module receives and outputs the asynchronous event stream from the event simulation unit; the event frame output module receives the asynchronous event stream from the event simulation unit, segments it according to the event frame period based on the event timestamp, generates and outputs event frames; the cross-modal alignment module is connected to the infrared image output module, the event stream output module and the event frame output module respectively, and realizes the time alignment of the infrared image, the event frame and the asynchronous event stream through the image timestamp and the event timestamp, and saves the data and parameter configuration to a predetermined directory.

[0025] This invention also discloses a method for using the above-mentioned space-based infrared aerial target dynamic visual simulation system, characterized by comprising the following steps:

[0026] Step S100: Obtain the configuration information of the simulation task through the parameter configuration unit;

[0027] Step S200: The scene construction unit generates an infrared scene sequence containing moving targets and dynamic backgrounds according to the configuration information;

[0028] Step S300: The infrared scene sequence is processed by the event simulation unit through inter-frame differential processing, and the event polarity is determined by combining dynamic thresholds to generate an asynchronous event stream. At the same time, the infrared image is output according to a preset period.

[0029] Step S400: Using the data alignment and output unit, the asynchronous event stream is segmented according to the event frame period based on the event timestamp, and the event is projected to form the event frame.

[0030] Step S500: Using the data alignment and output unit, the event frame, the asynchronous event stream, and the infrared image are cross-modal aligned based on a unified time reference and then output.

[0031] Furthermore, step S200 includes the following sub-steps:

[0032] Step S210: The background generation module performs time-varying perturbation processing on the loaded background image according to the configuration information to generate a time-varying background sequence.

[0033] Step S220: The trajectory generation module generates a multi-target motion trajectory that satisfies the minimum spacing constraint based on the configuration information.

[0034] Step S230: According to the configuration information, the target injection module injects energy at the sub-pixel center position of the multi-target motion trajectory using a point spread function, and superimposes it with the time-varying background sequence to obtain the infrared scene sequence.

[0035] Furthermore, step S220 includes the following multiple trajectory generation modes;

[0036] When configured in random walk mode, the random walk control submodule is used to set the initial position of each target in the field of view through uniform initialization, and the targets smaller than the spacing threshold are rearranged along the connecting line direction; then in each frame, the position is updated based on the motion direction of the previous frame and random turning disturbance, the normal reflection processing is performed on the targets that exceed the field of view boundary, and the minimum spacing constraint is applied through multiple rounds of iteration throughout the process to generate a random walk trajectory;

[0037] When configured as escort formation mode, the escort formation control submodule first generates the lead aircraft trajectory, and then, based on the lead aircraft's speed direction, maps the formation template to the cell coordinate system through coordinate transformation, and superimposes the horizontal and vertical spacing to determine the position of each wingman, thereby generating the escort formation trajectory.

[0038] When configured in round-trip avoidance mode, the round-trip avoidance control submodule generates round-trip trajectories for each target within a set movement range. The target movement pattern is as follows: when a target reaches the boundary of the movement range, it moves in the opposite direction, and this cycle repeats. When the target spacing is less than the minimum avoidance spacing, without changing the time step of the trajectory, collision avoidance adjustments are applied through small step sizes and multiple iterations to generate a round-trip avoidance trajectory that satisfies the minimum spacing constraint.

[0039] Furthermore, step S300 includes the following sub-steps:

[0040] Step S310: Calculate the frame difference between adjacent infrared scenes using the inter-frame difference calculation module;

[0041] Step S320: A noisy dynamic threshold is generated by the adaptive threshold decision module; when the frame difference is higher than the positive value of the noisy dynamic threshold, a positive event is decided and output; when the frame difference is less than the negative value of the noisy dynamic threshold, a negative event is decided and output.

[0042] Step S330: The trigger rate feedback control module calculates the current event trigger rate and provides feedback information to the adaptive threshold decision module based on the statistical results;

[0043] Step S340: The adaptive threshold decision module adjusts the noisy dynamic threshold according to the feedback information to stabilize the event triggering rate within a preset range.

[0044] Step S350: Using the asynchronous event stream construction module as a reference, each event is superimposed with a trigger delay and then quantized according to a preset time resolution to form the corresponding event timestamp; the trigger delay includes line readout delay and noise disturbance delay;

[0045] Step S360: Sort all the events in ascending order of the timestamps, and attach target identity information to each event to form the asynchronous event stream.

[0046] In view of the above technical features, the space-based infrared aerial target dynamic visual simulation system and method of the present invention have significant advantages compared with the prior art: it realizes the closed-loop output of the whole process from parameter initialization, space-based detection scene construction, infrared dynamic visual data generation to cross-modal alignment and data storage, thereby improving the controllability and reproducibility of data generation. Attached Figure Description

[0047] Figure 1 This is a schematic diagram of a preferred embodiment of the space-based infrared aerial target dynamic visual simulation system of the present invention.

[0048] Figure 2 This is a flowchart of a preferred embodiment of the method of the present invention using a space-based infrared aerial target dynamic visual simulation system;

[0049] Figure 3 This is a comparison diagram of the original infrared image and the generated event frame in a preferred embodiment of the space-based infrared aerial target dynamic visual simulation system and method of the present invention.

[0050] In the diagram: 100 - Parameter configuration unit, 200 - Scene construction unit, 300 - Event simulation unit, 400 - Data alignment and output unit;

[0051] 210 - Background generation module, 220 - Trajectory generation module, 230 - Target injection module;

[0052] 221-Random Walk Control Submodule, 222-Escort Formation Control Submodule, 223-Round Trip Avoidance Control Submodule; 231-Point Diffusion Function Generation Submodule, 232-Energy Minimum Constraint Submodule;

[0053] 310 - Inter-frame differential calculation module, 320 - Adaptive threshold decision module, 330 - Trigger rate feedback control module, 340 - Asynchronous event stream construction module, 350 - Periodic output control module;

[0054] 410 - Infrared image output module, 420 - Event stream output module, 430 - Event frame output module, 440 - Cross-modal alignment module. Detailed Implementation

[0055] The present invention will be further described below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined by the appended claims.

[0056] Please see Figure 1 and Figure 3 This invention discloses a space-based infrared aerial target dynamic visual simulation system. As shown in the figure, a preferred embodiment includes a parameter configuration unit 100, a scene construction unit 200, an event simulation unit 300, and a data alignment and output unit 400.

[0057] The parameter configuration unit 100 provides a user graphical interface for obtaining configuration information for the simulation task. In this embodiment, the configuration information includes path parameters, size and time parameters, event threshold parameters, background jitter parameters, trajectory and avoidance parameters, and target parameters.

[0058] The path parameters define the paths to the data required for the simulation, such as the storage paths for background images and results. Size and time parameters include cropping size, cropping start point, scene frame count, temporal resolution, event frame period, infrared image period, and line readout delay.

[0059] The event threshold parameter is used to define the working threshold of each unit. For example, the event trigger rate range is set to 5% to 10%, and the threshold noise standard deviation is set to 1×10⁻. 5Background jitter parameters are used when generating dynamic backgrounds from static infrared images and include background grayscale offset, background noise parameters, and jitter coefficients. Trajectory and avoidance parameters configure the trajectory generation method for moving targets, including random walk mode, escort formation mode, and round-trip avoidance mode, as well as specific parameters for these modes. For example, in random walk and escort formation modes, parameters include at least the number of random groups, speed range, turning angle, initial minimum spacing, escort spacing, number of wingmen, initial coordinates of the leader aircraft, and leader aircraft speed. In round-trip avoidance mode, parameters include the number of round-trip targets, round-trip motion coordinate axes, round-trip speed, motion range, minimum avoidance spacing, avoidance step size per frame, and number of avoidance iterations per frame.

[0060] The scene construction unit 200 is connected to the parameter configuration unit 100. Based on the configuration information provided by the parameter configuration unit 100, it generates an infrared scene sequence containing moving targets and dynamic backgrounds. It includes a background generation module 210, a trajectory generation module 220, and a target injection module 230.

[0061] The background generation module 210 is connected to the parameter configuration unit 100. After obtaining configuration information from the parameter configuration unit 100, it performs time-varying perturbation processing on the loaded background image based on the configuration information to generate a time-varying background sequence. Specifically, in this embodiment, the background image "cloud.png" is loaded, and the region of interest is obtained according to the cropping size of 512×512 and the cropping starting point (100,100). For each frame of the image, a platform displacement controlled by a jitter coefficient of 0.5 pixels / frame is superimposed, and an additive perturbation controlled by the background grayscale bias and background noise parameters (Gaussian noise, with a mean of 0 and a variance of 1) is superimposed to form a time-varying background sequence.

[0062] The trajectory generation module 220 is connected to the parameter configuration unit 100 and generates multi-target motion trajectories that satisfy the minimum spacing constraint based on the configuration information provided by the parameter configuration unit 100. The trajectory generation module 220 further includes a random walk control submodule 221, an escort formation control submodule 222, and a round-trip avoidance control submodule 223.

[0063] This embodiment is configured in random walk mode, therefore the random walk control submodule 221 is activated. Its specific operation is as follows: First, at least one target's initial position is set within a 512×512 field of view using a uniform initialization method. This embodiment uses a low-difference sequence for uniform sampling. For targets with an initial spacing of less than 10 pixels, they are rearranged along their connecting lines until all target spacings meet the minimum spacing constraint. Subsequently, in each frame, the target position is updated based on the motion direction of the previous frame and random turning perturbations. When a target exceeds the field of view boundary, reflection processing is performed according to the corresponding boundary normal, allowing the target to continue moving within the field of view. Throughout the motion process, the minimum spacing constraint is applied through multiple iterations to ensure that the spacing between any two targets is not less than this threshold. Finally, a sufficient number of frames are generated to form a random walk trajectory.

[0064] In other embodiments, an escort formation mode can also be used. In this case, the escort formation control submodule 222 is activated. Its specific operation process is as follows:

[0065] First, the timing position of the lead aircraft is generated. The initial coordinates of the lead aircraft are set to (128, 128), its speed to 1 pixel / frame, and its initial direction of motion to 45°. The lead aircraft moves at a constant speed along this direction, while simultaneously adding small random perturbations (±5°) to simulate maneuvers in real flight. Then, using the lead aircraft's speed direction as a reference, the formation template is mapped to a pixel coordinate system through coordinate transformation. This embodiment uses a diamond formation with the lead aircraft in front. The specific coordinate transformation process is as follows: using the lead aircraft's position as the origin, the lead aircraft's speed direction as the positive vertical axis, and the vertical direction as the horizontal axis, the relative coordinates of the wingmen are transformed to the pixel coordinate system using a rotation matrix. Then, the absolute position of the lead aircraft is superimposed to obtain the precise sub-pixel coordinates of each wingman. Throughout the movement, minimum spacing constraints are applied through multiple iterations to ensure a safe distance is maintained between wingmen and between wingmen and the lead aircraft. Finally, a sufficient number of frames are generated to form the escort formation trajectory, in which each wingman always maintains a predetermined formation relative to the lead aircraft.

[0066] In another embodiment, a round-trip avoidance mode can also be used. In this case, the round-trip avoidance control submodule 223 is activated. Its specific operation process is as follows:

[0067] First, the reciprocating motion trajectory of each target is generated within a set motion range. In this embodiment, two reciprocating motion targets are set up, moving back and forth along the y-axis, with a motion range of 100 to 400 pixels and a reciprocating speed of 2 pixels / frame. The target motion pattern is as follows: when the target reaches the y=400 boundary, it moves in the opposite direction towards y=100; when the target reaches the y=100 boundary, it moves in the opposite direction towards y=400, and so on, repeating continuously.

[0068] Throughout the entire motion, a minimum spacing constraint is applied through multiple iterations. In this embodiment, the minimum avoidance spacing is set to 8 pixels. When two targets move towards each other and the spacing is less than this threshold, the avoidance mechanism is triggered. Without changing the time step of the trajectory, through small step sizes of 0.6 pixels per frame and 8 iterations, the two targets avoid each other while maintaining the overall motion trend. The avoidance direction is a lateral offset perpendicular to the motion direction to avoid target overlap or excessively small spacing. Finally, a sufficient number of frames are generated to form a round-trip avoidance trajectory, and the targets achieve smooth mutual avoidance during the round-trip motion.

[0069] The target injection module 230 is connected to both the background generation module 210 and the trajectory generation module 220. Based on configuration information, it injects energy at the sub-pixel center position of the multi-target motion trajectory using a point spread function, and superimposes this energy onto the time-varying background sequence generated by the background generation module 210 to obtain an infrared scene sequence. The target injection module 230 consists of a point spread function generation submodule 231 and an energy baseline constraint submodule 232.

[0070] The point spread function generation submodule 231 calculates the Gaussian kernel standard deviation σ based on the target radius (e.g., 4 pixels) and energy concentration (e.g., 0.95). The point spread function typically uses a Gaussian distribution model, and its standard deviation satisfies a specific relationship with the target radius and energy concentration. In this embodiment, σ is calculated to be 0.6 pixels, and small targets with a size of 9×9, a signal-to-noise ratio of 4, and an energy concentration of 0.95 are sequentially generated at the sub-pixel center position of each target.

[0071] The energy minimum constraint submodule 232 has two operating modes. One is based on the total target energy. Specifically, the energy minimum constraint submodule 232 first calculates the background noise level of the local area where the target is located, and then calculates the required total target energy based on the signal-to-noise ratio. Simultaneously, a global energy minimum constraint is applied: when the total target energy is determined using the signal-to-noise ratio, the total target energy is not lower than the energy lower limit determined by the minimum signal-to-noise ratio of the target. The other mode is based on grayscale peak value. In this mode, the energy minimum constraint submodule 232 ensures that the target intensity is not lower than the minimum grayscale peak value of the target. Regardless of the mode used, when the calculated total target energy or target intensity meets the requirements, the energy minimum constraint submodule 232 superimposes the energy onto the corresponding position of the time-varying background sequence according to the point spread function distribution. This process is repeated to inject energy into the target in each frame, ultimately obtaining the infrared scene sequence.

[0072] The event simulation unit 300 is connected to the scene construction unit 200. It simulates a real infrared camera, performs inter-frame difference processing on the infrared scene sequence, and combines dynamic thresholds to determine event polarity, generating an asynchronous event stream. Simultaneously, it outputs infrared images at preset periods and records image timestamps. The event simulation unit 300 includes an inter-frame difference calculation module 310, an adaptive threshold determination module 320, a trigger rate feedback control module 330, an asynchronous event stream construction module 340, and a periodic output control module 350.

[0073] The inter-frame difference calculation module 310 is connected to the scene construction unit 200 and is used to calculate the frame difference between adjacent infrared scenes. For each pixel position, the grayscale difference between the current frame and the previous frame is calculated.

[0074] The adaptive threshold decision module 320 is connected to the inter-frame difference calculation module 310 and the trigger rate feedback control module 330. It constructs a noisy dynamic threshold and determines the event polarity based on the comparison between the frame difference and the noisy dynamic threshold, outputting a positive or negative event. Specifically, the noisy dynamic threshold T = feedback adjustment coefficient × base threshold + average frame difference + threshold noise term. The threshold noise term simulates circuit noise in a real sensor. For each pixel, when the frame difference is higher than the positive value (T) of the noisy dynamic threshold, a positive event (representing increased brightness) is output; when the frame difference ΔL is lower than the negative value (-T) of the noisy dynamic threshold, a negative event (representing decreased brightness) is output.

[0075] The trigger rate feedback control module 330 calculates the current event trigger rate and provides feedback information to the adaptive threshold decision module 320 based on the statistical results. This iteratively adjusts the base threshold T to stabilize the event trigger rate within a preset range. In this embodiment, the preset trigger rate range is 5% to 10%. After processing each frame, the total number of events triggered within that frame is counted, and the current event trigger rate (number of events / total number of pixels) is calculated. If the current event trigger rate is higher than 10%, feedback is sent to the adaptive threshold decision module 320, increasing the base threshold T by 0.5; when it is lower than 5%, the base threshold T is decreased by 0.5. This closed-loop feedback control keeps the event trigger density stable.

[0076] The asynchronous event stream construction module 340 is connected to the adaptive threshold decision module 320. Based on the absolute start time, it generates the event timestamp by superimposing the trigger delay, and further sorts the events based on the event timestamp, adds target identity information, and forms an asynchronous event stream.

[0077] Specifically, for each triggered event, the timestamp is generated as follows: First, the event frame period in which the event occurs is determined. In this embodiment, the event frame period is 6 milliseconds. Within the k-th event frame period, the basic trigger time of the event is (k-1) × 6 milliseconds + line readout delay. The line readout delay simulates the line-by-line exposure characteristics of a rolling shutter. In this embodiment, the readout delay for each line increases linearly with the line number, with the first line delay being 0 and the last line delay being 10 microseconds. Random noise perturbation is then superimposed on this. In this embodiment, the noise perturbation follows a Gaussian distribution with a mean of 0 and a standard deviation of 0.05. Finally, quantization is performed at a time resolution of 1 microsecond to obtain a microsecond-level absolute timestamp. The data structure for each event includes: pixel horizontal coordinate x, pixel vertical coordinate y, event polarity p (+1 or -1), microsecond-level timestamp t, target label (1 indicates a moving target, 0 indicates background noise), and target number id (for moving targets, this identifies the target number to which they belong). In this embodiment, since background noise also generates a small number of events, it is necessary to add labels and numbers to the target events. After all events are generated, they are sorted in ascending order by timestamp to form an asynchronous event stream.

[0078] The periodic output control module 350 is connected to the asynchronous event stream construction module 340, outputting infrared images according to a preset period and recording the image timestamps. Specifically, the infrared image period is 50 milliseconds (corresponding to 20Hz). During the simulation, corresponding frames are extracted from the infrared scene sequence, output as infrared images, and the current absolute time is recorded as the image timestamp of the image.

[0079] The data alignment and output unit 400 is connected to the event simulation unit 300, and performs cross-modal alignment between the asynchronous event stream and the infrared image based on the image timestamp and event timestamp, and then outputs the data alignment and output unit 400. The data alignment and output unit 400 includes an infrared image output module 410, an event stream output module 420, an event frame output module 430, and a cross-modal alignment module 440.

[0080] The infrared image output module 410 receives infrared images from the event simulation unit 300 and continuously outputs infrared images according to a preset period based on the image timestamp. The image format is PNG or JPG, and a file is generated to record the absolute timestamp of each frame of the image.

[0081] The event stream output module 420 directly outputs the asynchronous event stream from the event simulation unit 300.

[0082] The event frame output module 430, based on the asynchronous event stream from the event simulation unit 300, reads the event timestamps contained therein, segments them according to the event frame period, merges and projects them to form event frames, and outputs them. Specifically, the event frame period is 6 milliseconds. The asynchronous event stream is sliced ​​into 6-millisecond time windows, and the events within each time window are projected onto a 512×512 image. The projection rules are: positive events are mapped to grayscale 255 (white), negative events are mapped to grayscale 0 (black), and areas with no events are mapped to grayscale 127 (gray). In this embodiment, while generating event frames, a file is also generated to record the time interval (start time - end time) corresponding to each event frame. Figure 3 (b) demonstrates the effect of the generated event frames, and... Figure 3 (a) Comparison of infrared images.

[0083] The cross-modal alignment module 440 is simultaneously connected to the infrared image output module 410, the event stream output module 420, and the event frame output module 430. It achieves time alignment of the infrared image, the event frame, and the asynchronous event stream using the image timestamp and the event timestamp, and saves the data and parameter configurations to a predetermined directory. Based on the predetermined directory output by the cross-modal alignment module 440, users can easily perform spatiotemporal alignment of the infrared image, event frame, and asynchronous event stream for subsequent algorithm training and verification. Figure 3 As shown, the differences between event frames and infrared images at the same moment can be easily and intuitively compared.

[0084] Please see Figure 1 and Figure 2 The present invention also discloses a method for performing dynamic visual simulation of space-based infrared aerial targets using the above-described system. A preferred embodiment includes the following steps:

[0085] Step S100: Parameter initialization.

[0086] The configuration information for the simulation task is obtained through the parameter configuration unit 100. Specifically, the configuration information includes path parameters, size and time parameters, event threshold parameters, background jitter parameters, trajectory and avoidance parameters, and target parameters, consistent with the device embodiment.

[0087] Step S200, Scene Construction.

[0088] The scene construction unit 200 generates an infrared scene sequence containing moving targets and a dynamic background based on the configuration information. This step includes the following sub-steps:

[0089] Step S210: Background generation.

[0090] The background generation module 210 performs time-varying perturbation processing on the loaded background image according to the configuration information to generate a time-varying background sequence. Specifically, after loading the background image, the region of interest is obtained according to the cropping size and cropping start point. For each frame image, a platform displacement controlled by the jitter coefficient and the frame is superimposed, and an additive perturbation controlled by the background grayscale offset and background noise parameters is superimposed to generate a time-varying background sequence.

[0091] Step S220: Trajectory generation.

[0092] The trajectory generation module 220 generates multi-target motion trajectories that satisfy the minimum spacing constraint based on the configuration information. This step includes the following three implementation methods:

[0093] The first method involves generating a random walk trajectory. When configured for random walk mode, the random walk control submodule 221 uses a uniform initialization method to set the initial positions of each target within the field of view. In this embodiment, a low-difference sequence is used to set the initial positions of multiple targets within the field of view, and targets smaller than the spacing threshold of 10 pixels are rearranged along the connecting line direction. Subsequently, in each frame, the positions are updated based on the motion direction of the previous frame and the random turning perturbation (±8°). Targets exceeding the field of view boundary undergo normal reflection processing, and a minimum spacing constraint of 6 pixels is applied through multiple iterations throughout the process to generate a random walk trajectory.

[0094] The second method involves generating escort formation trajectories. When configured in escort formation mode, the escort formation control submodule 222 first generates the lead aircraft trajectory. Then, using the lead aircraft's speed direction as a reference, the formation template is mapped to the pixel coordinate system through coordinate transformation, and lateral and longitudinal spacing is superimposed to determine the positions of each wingman, thus generating the escort formation trajectory. Throughout the entire movement, minimum spacing constraints are applied through multiple iterations to ensure safe distances are maintained between wingmen and between wingmen and the lead aircraft.

[0095] The third method is to generate a round-trip avoidance trajectory. When configured in round-trip avoidance mode, the round-trip avoidance control submodule 223 generates round-trip motion trajectories for each target within a set motion range. In this embodiment, two round-trip moving targets are set, moving back and forth along the x-axis, with a motion range of 100 to 400 pixels and a round-trip speed of 2 pixels per frame. The target movement pattern is as follows: when a target reaches the boundary, it moves in the opposite direction, and this cycle repeats. When the target spacing is less than the minimum avoidance spacing of 8 pixels, without changing the trajectory time step, collision avoidance adjustments are applied through small steps of 0.6 pixels per frame and 8 iterations, so that the targets avoid each other while maintaining the overall motion trend, generating a round-trip avoidance trajectory that satisfies the minimum spacing constraint.

[0096] Step S230, target injection.

[0097] According to the configuration information, the target injection module 230 injects energy at the center position of the sub-pixel of the multi-target motion trajectory through a point spread function, and superimposes it with the time-varying background sequence to obtain the infrared scene sequence.

[0098] Specifically, the point spread function generation submodule 231 generates a 9×9 normalized point spread function at the center of each sub-pixel of the target based on the target radius (e.g., 4 pixels) and energy concentration (e.g., 0.95). In this embodiment, a mode based on the total target energy is adopted. That is, the total target energy is determined by the energy minimum constraint submodule 232 according to the preset target signal-to-noise ratio. If the energy minimum constraint (not lower than the lower limit of energy determined by the minimum target signal-to-noise ratio) is met, the energy is distributed and superimposed on the corresponding position of the time-varying background sequence according to the point spread function. In other embodiments, the target intensity can also be determined by the grayscale peak value. In this case, the energy minimum constraint submodule 232 checks that the target intensity is not lower than the minimum grayscale peak value of the target. When the check passes, the energy is distributed and superimposed on the corresponding position of the time-varying background sequence according to the point spread function.

[0099] Step S300, event simulation.

[0100] The event simulation unit 300 performs inter-frame differential processing on the infrared scene sequence and combines it with a dynamic threshold to determine event polarity, generating an asynchronous event stream and simultaneously outputting infrared images at a preset period. This step includes the following sub-steps:

[0101] Step S310: Calculate the frame difference.

[0102] The frame difference between adjacent infrared scenes is calculated using the inter-frame difference calculation module 310. Then, each pixel position is traversed, and the grayscale difference between the current frame and the previous frame is calculated to obtain the frame difference. The frame difference can be a negative number.

[0103] Step S320, threshold decision.

[0104] The adaptive threshold decision module 320 generates a noisy dynamic threshold, which includes a base threshold and an injected threshold noise term, expressed as: Noisy dynamic threshold = Feedback adjustment coefficient × Base threshold + Average frame difference + Threshold noise term. The threshold noise term follows a standard deviation of 1 × 10⁻⁶, with a mean of 0 and a standard deviation of 1 × 10⁻⁶. -5 The data follows a Gaussian distribution. The initial value of the feedback adjustment coefficient is set to 0. The noisy dynamic threshold is a positive number that forms an interval centered at zero. When the frame difference falls within this interval, it is considered insufficient to trigger an event; otherwise, an event is triggered. Specifically, when the frame difference is higher than the positive value of the noisy dynamic threshold, a positive event is output. When the frame difference is less than the negative value of the noisy dynamic threshold, a negative event is output.

[0105] Step S330: Trigger rate statistics and feedback.

[0106] The trigger rate feedback control module 330 counts the current event trigger rate and then provides feedback information to the adaptive threshold decision module 320 based on the statistical results.

[0107] For example, in this embodiment, the preset trigger rate range is 5% to 10%. After each frame is processed, the trigger rate feedback control module 330 counts the total number of events triggered within that frame and calculates the current event trigger rate. If the current event trigger rate is not within the preset trigger rate range, feedback information is generated and sent to the adaptive threshold decision module 320, triggering it to adjust the noisy dynamic threshold, thereby changing the actual trigger rate of the next frame and forming a closed-loop control.

[0108] Step S340, threshold adjustment.

[0109] The adaptive threshold decision module 320 adjusts the final noisy dynamic threshold by adjusting the feedback adjustment coefficient based on the feedback information provided by the trigger rate feedback control module 330, so that the event trigger rate is stabilized within a preset range. For example, if the current event trigger rate is higher than 10%, the feedback adjustment coefficient is increased. When it is lower than 5%, the feedback adjustment coefficient is decreased. In this embodiment, the initial value of the feedback adjustment coefficient is set to 0, and the adjustment step size per frame is 0.5, achieving adaptive threshold control through frame-by-frame iteration.

[0110] Step S350: Timestamp generation.

[0111] The asynchronous event stream construction module 340 uses the absolute start time as a reference to add a trigger delay to each event, forming the event timestamp corresponding to that event. The trigger delay is the row readout delay plus the noise disturbance delay.

[0112] Specifically, the current system time is read as the absolute start time, i.e., the base time. Then, for each event, its event timestamp = base time + current event offset period + row readout delay + random noise perturbation, and quantized with a time resolution of 1 microsecond.

[0113] Step S360, Event Stream Construction.

[0114] The asynchronous event stream construction module 340 sorts all events in ascending order by their event timestamps and adds target identification information to each event to form an asynchronous event stream. The data structure of each event includes: pixel coordinates, event polarity, event timestamp, target label, and target number.

[0115] Step S370: Infrared image output.

[0116] The periodic output control module 350 outputs infrared images at a preset period (e.g., 50 milliseconds) and records the image timestamp.

[0117] Step S400: Event frame generation.

[0118] Using the data alignment and output unit 400, the asynchronous event stream is segmented according to the event frame period based on the event timestamp. Then, all the event frames in each segment are projected together to form an event frame.

[0119] In this embodiment, the event frame period is 6 milliseconds. The asynchronous event stream is sliced ​​into 6-millisecond time windows, and the events within each window are projected onto a 512×512 image. Positive events are mapped to grayscale 255, negative events are mapped to grayscale 0, and areas with no events are mapped to grayscale 127.

[0120] Step S500, cross-modal alignment and output.

[0121] Continuing with the data alignment and output unit 400, the event frame, asynchronous event stream, and infrared image are cross-modal aligned and output based on a unified time base. Specifically, the output directory includes the infrared image, event frame, asynchronous event stream, configuration file, random seed, and alignment log. Time alignment of the three types of data is achieved through timestamp logs.

[0122] The above description is merely a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A space-based infrared aerial target dynamic visual simulation system, characterized in that, It includes a parameter configuration unit, a scene construction unit, an event simulation unit, and a data alignment and output unit; The parameter configuration unit acquires the configuration information of the simulation task; the scene construction unit, connected to the parameter configuration unit, generates an infrared scene sequence containing moving targets and dynamic backgrounds according to the configuration information; the event simulation unit, connected to the scene construction unit, performs inter-frame difference processing on the infrared scene sequence, and combines a dynamic threshold to determine event polarity, generating an asynchronous event stream, and simultaneously outputs infrared images at a preset period and records image timestamps; wherein, each event in the asynchronous event stream contains an event timestamp; the data alignment and output unit, connected to the event simulation unit, performs cross-modal alignment of the asynchronous event stream and the infrared images based on the image timestamps and the event timestamps, and outputs the data.

2. The space-based infrared aerial target dynamic visual simulation system according to claim 1, characterized in that, The scene construction unit includes a background generation module, a trajectory generation module, and a target injection module; The background generation module is connected to the parameter configuration unit and performs time-varying perturbation processing on the loaded background image according to the configuration information to generate a time-varying background sequence. The trajectory generation module is connected to the parameter configuration unit and generates a multi-target motion trajectory that satisfies the minimum spacing constraint according to the configuration information. The target injection module is connected to both the background generation module and the trajectory generation module and injects energy at the sub-pixel center position of the multi-target motion trajectory using a point spread function according to the configuration information, and superimposes it with the time-varying background sequence to obtain the infrared scene sequence.

3. The space-based infrared aerial target dynamic visual simulation system according to claim 2, characterized in that, The trajectory generation module includes a random walk control submodule, an escort formation control submodule, and a round-trip evasion control submodule. The random walk control submodule plans the trajectory of the random walk mode; when the configuration information is configured as random walk mode, the random walk control submodule is activated, and the initial position of each target in the field of view is set by uniform initialization, and the targets smaller than the spacing threshold are rearranged along the connecting line direction. Subsequently, in each frame, the position is updated based on the motion direction and random turning perturbation of the previous frame. Normal reflection processing is performed on targets that exceed the field of view boundary. Throughout the process, the minimum spacing constraint is applied through multiple rounds of iteration to generate a random walk trajectory. The escort formation control submodule is used to plan the trajectory of the escort formation mode. When the configuration information is configured as escort formation mode, the escort formation control submodule is activated. First, the leader aircraft trajectory is generated. Then, based on the leader aircraft speed direction, the formation template is mapped to the cell coordinate system through coordinate transformation. The horizontal and vertical spacing is superimposed to determine the position of each wingman and generate the escort formation trajectory. The round-trip avoidance control submodule is used to plan the trajectory of the round-trip avoidance mode. When the round-trip avoidance mode is configured in the configuration information, the round-trip avoidance control submodule is activated and generates the round-trip motion trajectory of each target within the set motion range. When the target spacing is less than the minimum avoidance spacing, without changing the trajectory time step, collision avoidance adjustment is applied by small step size and multiple iterations to generate a round-trip avoidance trajectory that meets the minimum spacing constraint.

4. The space-based infrared aerial target dynamic visual simulation system according to claim 2, characterized in that, The target injection module includes a point spread function generation submodule and an energy minimum constraint submodule; The point spread function generation submodule calculates the Gaussian kernel standard deviation based on the configuration information, target radius and energy concentration, and generates a normalized point spread function at the center of the sub-pixel. The energy minimum constraint submodule is connected to the point spread function generation submodule, and determines the target total energy based on the preset target signal-to-noise ratio or grayscale peak value; wherein, when the grayscale peak value is used to determine the target intensity, the target intensity is not lower than the target minimum grayscale peak value; when the signal-to-noise ratio is used to determine the target total energy, the target total energy is not lower than the energy lower limit determined by the target minimum signal-to-noise ratio; and further applies a global energy minimum constraint, and superimposes the target energy that meets the constraint onto the time-varying background sequence.

5. The space-based infrared aerial target dynamic visual simulation system according to claim 1, characterized in that, The event simulation unit includes an inter-frame difference calculation module, an adaptive threshold decision module, a trigger rate feedback control module, an asynchronous event stream construction module, and a periodic output control module. The inter-frame difference calculation module is connected to the scene construction unit and is used to calculate the frame difference between adjacent infrared scenes; The adaptive threshold decision module is connected to the inter-frame difference calculation module and the trigger rate feedback control module. It is used to construct a noisy dynamic threshold and make an event polarity decision based on the comparison result between the frame difference and the noisy dynamic threshold, and output a positive event or a negative event. The noisy dynamic threshold is generated based on the feedback adjustment coefficient, the base threshold, the average frame difference, and the threshold noise term. The trigger rate feedback control module counts the current event trigger rate; when the current event trigger rate is higher than the preset trigger rate upper limit, it feeds back to the adaptive threshold decision module to increase the noisy dynamic threshold. When the current event trigger rate is lower than the preset trigger rate lower limit, feedback is sent to the adaptive threshold decision module to reduce the noisy dynamic threshold. The asynchronous event stream construction module is connected to the adaptive threshold decision module. Based on the absolute start time, it adds a trigger delay and quantizes the event according to a preset time resolution to generate the event timestamp. The event is then sorted based on the event timestamp and target identification information is added to form the asynchronous event stream. The trigger delay includes line readout delay and noise disturbance delay. The periodic output control module is connected to the asynchronous event stream construction module and is used to output the infrared image according to a preset period and record the image timestamp.

6. The space-based infrared aerial target dynamic visual simulation system according to claim 1, characterized in that, The data alignment and output unit includes an infrared image output module, an event stream output module, an event frame output module, and a cross-modal alignment module; The infrared image output module receives the infrared image from the event simulation unit and outputs the infrared image according to a preset period based on the image timestamp. The event stream output module receives the asynchronous event stream from the event simulation unit and outputs it. After receiving the asynchronous event stream from the event simulation unit, the event frame output module generates and outputs event frames based on the event timestamp and segmented according to the event frame period. The cross-modal alignment module is connected to the infrared image output module, the event stream output module, and the event frame output module, respectively. It realizes the time alignment of the infrared image, the event frame, and the asynchronous event stream through the image timestamp and the event timestamp, and saves the data and parameter configuration to a predetermined directory.

7. A method using the space-based infrared aerial target dynamic visual simulation system as described in any one of claims 1 to 6, characterized in that, Includes the following steps: Step S100: Obtain the configuration information of the simulation task through the parameter configuration unit; Step S200: The scene construction unit generates an infrared scene sequence containing moving targets and dynamic backgrounds according to the configuration information; Step S300: The infrared scene sequence is processed by the event simulation unit through inter-frame differential processing, and the event polarity is determined by combining dynamic thresholds to generate an asynchronous event stream. At the same time, the infrared image is output according to a preset period. Step S400: Using the data alignment and output unit, the asynchronous event stream is segmented according to the event frame period based on the event timestamp, and the event is projected to form the event frame. Step S500: Using the data alignment and output unit, the event frame, the asynchronous event stream, and the infrared image are cross-modal aligned based on a unified time reference and then output.

8. The method for using a space-based infrared aerial target dynamic visual simulation system according to claim 7, characterized in that, Step S200 includes the following sub-steps: Step S210: The background generation module performs time-varying perturbation processing on the loaded background image according to the configuration information to generate a time-varying background sequence. Step S220: The trajectory generation module generates a multi-target motion trajectory that satisfies the minimum spacing constraint based on the configuration information. Step S230: According to the configuration information, the target injection module injects energy at the sub-pixel center position of the multi-target motion trajectory using a point spread function, and superimposes it with the time-varying background sequence to obtain the infrared scene sequence.

9. The method for using a space-based infrared aerial target dynamic visual simulation system according to claim 8, characterized in that, Step S220 includes the following multiple trajectory generation modes; When configured in random walk mode, the random walk control submodule is used to set the initial position of each target in the field of view through uniform initialization, and targets smaller than the spacing threshold are rearranged along the connecting line direction. Subsequently, in each frame, the position is updated based on the motion direction and random turning perturbation of the previous frame. Normal reflection processing is performed on targets that exceed the field of view boundary. Throughout the process, the minimum spacing constraint is applied through multiple rounds of iteration to generate a random walk trajectory. When configured as escort formation mode, the escort formation control submodule first generates the lead aircraft trajectory, and then, based on the lead aircraft's speed direction, maps the formation template to the cell coordinate system through coordinate transformation, and superimposes the horizontal and vertical spacing to determine the position of each wingman, thereby generating the escort formation trajectory. When configured in round-trip avoidance mode, the round-trip avoidance control submodule generates round-trip trajectories for each target within a set movement range. The target movement pattern is as follows: when a target reaches the boundary of the movement range, it moves in the opposite direction, and this cycle repeats. When the target spacing is less than the minimum avoidance spacing, without changing the time step of the trajectory, collision avoidance adjustments are applied through small step sizes and multiple iterations to generate a round-trip avoidance trajectory that satisfies the minimum spacing constraint.

10. The method for using a space-based infrared aerial target dynamic visual simulation system according to claim 7, characterized in that, Step S300 includes the following sub-steps: Step S310: Calculate the frame difference between adjacent infrared scenes using the inter-frame difference calculation module; Step S320: A noisy dynamic threshold is generated by the adaptive threshold decision module; when the frame difference is higher than the positive value of the noisy dynamic threshold, a positive event is decided and output; when the frame difference is less than the negative value of the noisy dynamic threshold, a negative event is decided and output. Step S330: The trigger rate feedback control module calculates the current event trigger rate and provides feedback information to the adaptive threshold decision module based on the statistical results; Step S340: The adaptive threshold decision module adjusts the noisy dynamic threshold according to the feedback information to stabilize the event triggering rate within a preset range. Step S350: Using the asynchronous event stream construction module as a reference, each event is superimposed with a trigger delay and then quantized according to a preset time resolution to form the corresponding event timestamp; the trigger delay includes line readout delay and noise disturbance delay; Step S360: Sort all the events in ascending order of the timestamps, and attach target identity information to each event to form the asynchronous event stream.

Images