Dynamic platform multi-infrared sensor collaborative multi-target perception digital simulation platform

Abstract

This application belongs to the field of digital simulation technology for multi-infrared sensor multi-target cooperative sensing. This application provides a digital simulation platform for multi-infrared sensor cooperative sensing on a moving platform. The embodiments disclosed herein can comprehensively simulate the working process of distributed multi-infrared sensors in a virtual environment, providing accurate simulation of multi-infrared sensor multi-target sensing on a moving platform. This allows researchers to repeatedly test and verify the distributed multi-infrared sensor cooperative sensing platform under different operating conditions and environments, ensuring the scientific validity and feasibility of the design scheme and reducing uncontrollable factors and risks in real-world testing. Developers can systematically adjust parameters such as sensor configuration, sensing algorithms, and data fusion strategies to accurately evaluate the impact of different configurations on distributed multi-infrared sensor cooperative algorithms and schemes; enabling optimization of the platform's overall performance and reliability in the early stages, avoiding the waste of time and costs caused by repeated hardware adjustments.

Description

Technical Field

[0001] This disclosure relates to the field of digital simulation technology for multi-infrared sensor multi-target collaborative perception, and in particular to a dynamic platform multi-infrared sensor collaborative multi-target perception digital simulation platform. Background Technology

[0002] Infrared sensors do not actively emit electromagnetic waves, offering advantages such as good concealment, strong anti-interference capabilities, high detection accuracy, and long detection range. However, a single infrared sensor has limitations, including a small field of view and the ability to obtain only angular information of the target, failing to acquire distance and depth information. This prevents comprehensive and accurate target acquisition, hindering precise target tracking and positioning. A distributed multi-infrared sensor collaborative sensing system, composed of multiple infrared sensor nodes, effectively compensates for the shortcomings of a single sensor. Furthermore, with advancements in UAV technology, particularly in UAV swarm and collaborative combat techniques, at least two UAVs equipped with infrared sensors can measure the angle of the same target from different angles. Combining this with the position and attitude information of the UAVs themselves allows for real-time calculation of the target's position, velocity, and attitude, demonstrating significant application value.

[0003] In a multi-infrared sensor collaborative sensing system, multiple infrared sensor nodes mounted on UAV-based mobile platforms work together to not only cover a wider area and enhance target detection and recognition capabilities, but also improve spatial resolution and target positioning accuracy through multi-sensor collaboration, effectively reducing blind spots and enhancing target tracking and localization capabilities. Furthermore, the distributed system exhibits stronger robustness; the collaborative work of multiple sensors reduces the impact of single-point failures on the system, ensuring its reliability and stability. Compared to a single sensor, the distributed system utilizes the collaboration of multiple low-cost sensors, reducing deployment costs while improving system performance and flexibility.

[0004] However, existing digital simulation systems for infrared or photoelectric sensors are designed for single-target perception and image generation using a single platform or sensor. These systems fail to reflect the collaborative work and information sharing among multiple sensors, and cannot comprehensively and dynamically perceive targets. Furthermore, existing systems often neglect the dynamic characteristics of target motion and the impact of environmental changes. Most simulation systems use static or simple motion models of the target state, failing to fully consider the complexity of target motion, velocity changes, acceleration, and potential error accumulation. In addition, current infrared sensor digital simulation systems do not adequately consider error propagation models, such as measurement errors, sensor positioning errors, and target dynamic modeling errors. Due to error accumulation and the uncertainty of sensor performance, the reliability and accuracy of single-sensor systems in actual operation are often significantly affected. In multi-sensor collaborative systems, the establishment and correction of error models are particularly critical, requiring consideration of the error characteristics of each sensor and the error propagation of collaborative work to ensure system accuracy and stability. Finally, current infrared sensor simulation systems tend to focus on the performance evaluation of a single sensor or single target, lacking a comprehensive evaluation and optimization of the overall performance of multi-sensor, multi-target collaborative perception systems.

[0005] Therefore, it is necessary to improve one or more of the problems existing in the above-mentioned related technical solutions.

[0006] It should be noted that this section is intended to provide background or context for the technical solutions of this disclosure as set forth in the claims. The description herein does not constitute an admission that it is prior art simply because it is included in this section. Summary of the Invention

[0007] The purpose of this disclosure is to provide a digital simulation platform for multi-infrared sensor collaborative multi-target perception on a dynamic platform, thereby overcoming, to at least some extent, one or more problems caused by the limitations and defects of related technologies.

[0008] According to embodiments of this disclosure, a multi-infrared sensor collaborative multi-target perception digital simulation platform for a moving platform is provided, comprising:

[0009] The system comprises a data processing module, an error module, a data storage and visualization module, a measured database, and a model database; among which,

[0010] The error module is used to obtain the position error of the airborne moving platform, the missed detection error of the sensor, and the angle measurement error of the sensor.

[0011] The model database is used to generate motion data of the target, motion data of the airborne moving platform, and observation data of each sensor based on the kinematic model of the target being detected, the kinematic model of the airborne moving platform, and the two-dimensional imaging model of the sensor, and combined with the position error of the airborne moving platform, the missed detection error of the sensor, and the angle measurement error of the sensor.

[0012] The data processing module is used to process the infrared radiation intensity data of the target under typical working conditions and the infrared characteristic image of the sensor imaging plane to generate infrared radiation characteristic data and target infrared characteristic grayscale image of the target.

[0013] The measured database is used to store the infrared radiation characteristics data and target infrared characteristic grayscale images of the detected targets. Combined with the detection distance of each target relative to each sensor, the detection elevation and azimuth angle of each target relative to each sensor, the infrared characteristic data and images of each target are generated.

[0014] The data storage and visualization module is used to manage and store the infrared characteristic data and images of each target, and to analyze the infrared characteristic data and images of each target to generate the motion trajectory of each target and each sensor in the network coordinate system.

[0015] Furthermore, the model database includes:

[0016] The target motion data generation submodule is used to obtain the motion data of the target based on the kinematic model of the target being detected;

[0017] The airborne moving platform motion data generation submodule is used to obtain the motion data of the airborne moving platform based on the kinematic model of the airborne moving platform and the position error of the airborne moving platform.

[0018] The sensor observation data generation submodule is used to obtain the observation data of each sensor based on the sensor's two-dimensional imaging model, the sensor's missed detection error, and the sensor's angle measurement error.

[0019] Furthermore, the data processing module also includes:

[0020] The module includes a target infrared characteristic data interpolation submodule and a target-at-specified-distance image generation module; among which,

[0021] The target infrared characteristic data interpolation submodule is used to detect the target under several typical working conditions by using an infrared sensor at different elevation and azimuth angles, obtain the target infrared radiation intensity data, and perform interpolation processing on the target infrared radiation intensity data to obtain the infrared radiation characteristic data of the target.

[0022] The target-specified distance image generation module is used to detect targets by infrared sensors at different elevation and azimuth angles, obtain infrared characteristic images of the sensor imaging plane of the detected target, and scale the infrared characteristic images of the sensor imaging plane of the detected target to obtain a grayscale image of the infrared characteristics of the detected target.

[0023] Further, the interpolation process includes the following steps:

[0024] The azimuth angle in each elevation angle file is interpolated using a multivariable spline interpolation method with a single variable cubic spline interpolation.

[0025] After interpolation in the azimuth direction is completed, bivariate spline interpolation is performed on the interpolation data for different elevation angles;

[0026] After bivariate spline interpolation of the interpolation data at different pitch angles, trivariate spline interpolation under multiple working states is completed.

[0027] Further, the scaling process includes the following steps:

[0028] By taking into account the resolution of the detected target in the current infrared characteristic image of the sensor imaging plane, and combining the actual size of the detected target and the infrared sensor parameters, the actual distance between the detected target and the sensor in the current infrared characteristic image of the sensor imaging plane is calculated. The image scaling ratio at different distances is then calculated based on the different distances between the detected target and the sensor.

[0029] Based on the infrared characteristic image of the sensor imaging plane of the target being detected, and combined with the calculated image scaling ratio at different distances, grayscale images of the target's infrared characteristics in the infrared sensor imaging plane at different distances, positions, and detection elevation and azimuth angles are generated.

[0030] Furthermore, it also includes:

[0031] The time registration module and the spatial registration module; among them,

[0032] The time registration module is used to synchronize the timestamps of each sensor.

[0033] The spatial registration module is used to unify the coordinates of each sensor into the coordinate system of the multi-sensor network.

[0034] Furthermore, the data storage and visualization module includes:

[0035] Data storage submodule and visualization submodule; among which,

[0036] The data storage submodule is used to access motion data from different types of sensors, target motion data, and sensor observation data, and to classify and store the data according to the sensor ID number and target ID number.

[0037] The visualization submodule is used to display the motion trajectory of the target and sensors in the multi-sensor network coordinate system based on 3D graphics, so as to show the formation of the target and the airborne moving platform, and to display the imaging effect of the target in the sensor imaging plane based on 2D graphics.

[0038] Furthermore, it also includes:

[0039] The configuration module is used to obtain target items, platform items, error items, and simulation settings; among them,

[0040] The target items include the number and number of targets, the flight orientation of each target, the initial position of each target, and the motion model of each target;

[0041] The platform item includes the number and number of airborne moving platforms, the flight orientation of each airborne moving platform, the initial position of each airborne moving platform, the motion model of each airborne moving platform, and the rotation model of each sensor;

[0042] Error items include airborne moving platform position error, sensor angle measurement error, and sensor missed detection error;

[0043] The simulation settings include total simulation duration, simulation step size, and data save path.

[0044] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0045] In the embodiments of this disclosure, the aforementioned dynamic platform multi-infrared sensor collaborative multi-target perception digital simulation platform can comprehensively simulate the collaborative perception process of distributed multi-infrared sensors in a virtual environment, providing accurate simulation of dynamic platform multi-infrared sensor multi-target perception. This allows researchers to repeatedly test and verify the distributed multi-infrared sensor collaborative perception platform under different operating conditions and environments, ensuring the scientific validity and feasibility of the design scheme and reducing uncontrollable factors and risks in real-world testing. Through this platform, developers can systematically adjust parameters such as sensor configuration, perception algorithms, and data fusion strategies, accurately evaluating the impact of different configurations on distributed multi-infrared sensor collaborative algorithms and schemes. This enables the platform to optimize its overall performance and reliability in the early stages, avoiding the time and cost waste caused by repeated hardware adjustments in traditional experiments. By providing a flexible and customizable simulation environment, technical developers can quickly iterate and test new algorithms, new functions, and new designs. Without relying on physical hardware, it efficiently simulates various working conditions in real-world application scenarios, thereby accelerating the research and development and application deployment of new technologies and improving development efficiency. Attached Figure Description

[0046] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0047] Figure 1 This diagram illustrates the structure of a multi-infrared sensor collaborative multi-target perception digital simulation platform for a dynamic platform in an exemplary embodiment of this disclosure.

[0048] Figure 2 This diagram illustrates the infrared radiation characteristics of an aircraft target when detected by an infrared sensor at different azimuths with a fixed pitch angle of 30 degrees, according to an exemplary embodiment of the present disclosure.

[0049] Figure 3 This illustration shows a schematic diagram of the infrared characteristics of an aircraft target at a certain pitch and azimuth angle, selected from an infrared characteristic image of the sensor imaging plane in an exemplary embodiment of this disclosure.

[0050] Figure 4 This diagram illustrates the usage flow of the time soft synchronization method of the time registration module in an exemplary embodiment of this disclosure.

[0051] Figure 5 This diagram illustrates a method flow chart of the spatial registration module in an exemplary embodiment of this disclosure.

[0052] Figure 6 This diagram illustrates the usage flow of the multi-infrared sensor collaborative multi-target perception digital simulation platform in an exemplary embodiment of this disclosure.

[0053] Figure 7 This diagram illustrates a simulation environment of a moving platform with multiple infrared sensors for multi-target perception, in which two airborne platforms collaboratively observe two targets in an exemplary embodiment of this disclosure.

[0054] Figure 8 This illustrates the target data storage file generated by the data storage submodule and the data storage file format of the airborne moving platform equipped with sensors in an exemplary embodiment of this disclosure;

[0055] Figure 9 This diagram illustrates a multi-platform, multi-target three-dimensional spatial motion trajectory under a simulation environment configuration in an exemplary embodiment of this disclosure.

[0056] Figure 10 This illustration shows an infrared characteristic imaging diagram of two targets detected in a certain frame in the imaging plane of the airborne infrared sensor 1 mounted on the airborne platform 1 in an exemplary embodiment of the present disclosure.

[0057] Figure 11 This illustration shows an infrared characteristic imaging diagram of two targets detected in a certain frame in the imaging plane of the airborne infrared sensor 2 mounted on the airborne platform 2 in an exemplary embodiment of this disclosure. Detailed Implementation

[0058] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0059] Furthermore, the accompanying drawings are merely illustrative diagrams of embodiments of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities.

[0060] This example implementation provides a digital simulation platform for multi-target perception using multiple infrared sensors on a moving platform. (Reference) Figure 1 As shown, the multi-infrared sensor collaborative multi-target perception digital simulation platform of the dynamic platform may include:

[0061] The system comprises a data processing module, an error module, a data storage and visualization module, a measured database, and a model database; among which,

[0062] The error module is used to obtain the position error of the airborne moving platform, the missed detection error of the sensor, and the angle measurement error of the sensor.

[0063] The model database is used to generate motion data of the target, motion data of the airborne moving platform, and observation data of each sensor based on the kinematic model of the target being detected, the kinematic model of the airborne moving platform, and the two-dimensional imaging model of the sensor, and combined with the position error of the airborne moving platform, the missed detection error of the sensor, and the angle measurement error of the sensor.

[0064] The data processing module is used to process the infrared radiation intensity data of the target under typical working conditions and the infrared characteristic image of the sensor imaging plane to generate infrared radiation characteristic data and target infrared characteristic grayscale image of the target.

[0065] The measured database is used to store the infrared radiation characteristics data and target infrared characteristic grayscale images of the detected targets. Combined with the detection distance of each target relative to each sensor, the detection elevation and azimuth angle of each target relative to each sensor, the infrared characteristic data and images of each target are generated.

[0066] The data storage and visualization module is used to manage and store the infrared characteristic data and images of each target, and to analyze the infrared characteristic data and images of each target to generate the motion trajectory of each target and each sensor in the network coordinate system.

[0067] The aforementioned dynamic platform multi-infrared sensor collaborative multi-target perception digital simulation platform can comprehensively simulate the working process of distributed multi-infrared sensors in a virtual environment, providing accurate simulation of multi-target perception using dynamic platform multi-infrared sensors. This allows researchers to repeatedly test and verify the distributed multi-infrared sensor collaborative perception platform under different operating conditions and environments, ensuring the scientific validity and feasibility of the design and reducing uncontrollable factors and risks in real-world testing. Through this platform, developers can systematically adjust parameters such as sensor configuration, perception algorithms, and data fusion strategies, accurately evaluating the impact of different configurations on distributed multi-infrared sensor collaborative algorithms and schemes. This enables the platform to optimize its overall performance and reliability in the early stages, avoiding the time and cost waste caused by repeated hardware adjustments in traditional experiments. By providing a flexible and customizable simulation environment, it allows technology developers to quickly iterate and test new algorithms, new functions, and new designs. Without relying on physical hardware, it efficiently simulates various working conditions in real-world application scenarios, thereby accelerating the research and development and application deployment of new technologies and improving development efficiency.

[0068] Below, we will refer to Figures 1 to 11 The various parts of the above-described dynamic platform multi-infrared sensor collaborative multi-target perception digital simulation platform in this example embodiment will be described in more detail.

[0069] In one embodiment, a target infrared characteristic data and image library based on measured data (i.e., a measured database) is used to provide highly reliable infrared radiation characteristic data of the detected target and infrared characteristic images of the target's sensor imaging plane for the operation of the simulation system (i.e., a multi-infrared sensor collaborative multi-target perception digital simulation platform). The infrared radiation characteristic data of the detected target is obtained through simulated detection of an expert-approved target model or through actual detection of a physical target. The infrared radiation characteristic data is the target infrared radiation intensity data obtained by detecting the target at different pitch and azimuth angles using infrared sensors under several typical operating conditions (a certain flight altitude, a certain flight speed). The infrared characteristic images of the target's sensor imaging plane are obtained through simulated detection of an expert-approved target model or through actual detection of a physical target. They are grayscale images of the target's infrared characteristics obtained by detecting the target at different pitch and azimuth angles using infrared sensors. These images represent the two-dimensional image of the detected target in the infrared sensor imaging plane and are a visual representation of the infrared radiation intensity on the surface of the detected target. The larger the grayscale value in the image, the greater the infrared radiation intensity of the target area.

[0070] In one embodiment, the data processing module processes the infrared characteristic data of the target under several typical working conditions and the infrared radiation characteristic data of the target under several typical working conditions in the image library, as well as the infrared characteristic image of the target sensor imaging plane, based on the measured data, to obtain the target infrared characteristic data and image library based on the measured data for all working conditions, all positions, and all directions.

[0071] The target infrared characteristic data interpolation module (i.e., the target infrared characteristic data interpolation submodule) in the data processing module interpolates the target infrared radiation intensity data obtained by infrared sensors detecting the target at different pitch and azimuth angles under several typical operating conditions (a certain flight altitude, a certain flight speed). The interpolation method is as follows: using multivariate spline interpolation, firstly, univariate cubic spline interpolation is performed on the azimuth angle in each pitch angle file; secondly, after interpolation in the azimuth direction, bivariate spline interpolation is performed on the interpolated data for different pitch angles; finally, after bivariate spline interpolation of the interpolated data for different pitch angles, trivariate spline interpolation is completed for multiple operating conditions. Through the above steps, target infrared characteristic data based on measured data is obtained under all operating conditions and in all directions.

[0072] The target-specified distance image generation module in the data processing module scales the infrared characteristic images of the target sensor imaging plane obtained by detecting the target at different elevation and azimuth angles using an infrared sensor. The scaling method is as follows: The actual distance between the target and the sensor in the currently obtained infrared characteristic image is calculated based on the resolution occupied by the target image in the current image, combined with the actual size of the target and the infrared sensor parameters. Then, the image scaling ratio at different distances is calculated. Using the infrared characteristic image of the target sensor imaging plane as a reference, and combining the calculated image scaling ratio at different distances, infrared characteristic images of the target in the infrared sensor imaging plane at different distances, positions, and detection elevation and azimuth angles are generated. Through the above scaling process, grayscale images of the target's infrared characteristics under all operating conditions, positions, and omnidirectional orientations are obtained.

[0073] The target infrared characteristic data and image library based on measured data is obtained by processing the infrared radiation characteristic data of the detected target and the infrared characteristic images of the sensor imaging plane of the detected target under several typical working conditions through a data processing module.

[0074] In one embodiment, a model-based target platform motion, sensor platform motion, and (i.e., a model database) are used to generate motion data for the target, motion data for the airborne moving platform equipped with sensors, and sensor observation data. This database incorporates an error module to ensure that the generated data simulates real-world conditions as closely as possible. Specifically, the target motion data is obtained based on the kinematic model of the target being detected; the motion data for the airborne moving platform equipped with sensors is obtained based on the airborne moving platform's kinematic model combined with its position error; and the sensor observation data is obtained based on the airborne sensor's two-dimensional imaging model combined with the sensor's missed detection error and angular measurement error.

[0075] The error module consists of the position error of the airborne moving platform, the missed detection error of the sensor, and the angle measurement error of the sensor. The position error of the airborne moving platform follows a Gaussian distribution and is added to the three-dimensional position coordinates of the airborne moving platform as a deviation; the angle measurement error of the sensor follows a Gaussian distribution, and a random error is generated in any frame during the two-dimensional imaging process of the airborne sensor and added to the transformation matrix from the multi-sensor network coordinate system to the sensor coordinate system.

[0076] In one embodiment, the time registration module is used to synchronize the timestamps of multiple sensor nodes. The simulation system considers the observation errors caused by time asynchrony when multiple sensor nodes observe a target. Therefore, the time registration module performs time synchronization registration on the target data observed by multiple sensor nodes, including hard synchronization and soft synchronization methods. The hard synchronization method is as follows: data is acquired within the same simulation loop. The soft synchronization method is as follows: interpolation is performed by combining the known pixel coordinate data of the target in the sensors to interpolate the continuous motion trajectory of the target. Then, the pixel coordinate data of the target in the imaging planes of different sensors at the same timestamp are calculated by combining the interpolated continuous motion trajectory of the target, thereby achieving soft synchronization and data alignment.

[0077] In one embodiment, the spatial registration module is used to unify multiple sensor nodes into a network coordinate system. In a multi-sensor system, the sensor's own position is typically represented by latitude, longitude, and altitude coordinates obtained from the airborne navigation system of the airborne platform. The spatial registration module ensures that images acquired by different sensors are accurately aligned within the same coordinate system, thereby achieving accurate multi-sensor information fusion and target recognition. The spatial registration process is as follows: from the geodetic coordinate system to the geocentric coordinate system, and from the geocentric coordinate system to the multi-sensor network coordinate system.

[0078] In one embodiment, the data storage and visualization module is used to effectively manage, store, and display large amounts of text data and images collected by multiple sensors, while providing a user-friendly interface to view and analyze this data, including a data storage module (i.e., a data storage submodule) and a visualization module (i.e., a visualization submodule).

[0079] The data storage module can access motion data from different types of sensors and targets, as well as sensor observation data. It then categorizes and stores the data according to sensor IDs and target IDs. The categorization method is as follows: All sensor and target data are categorized and stored as .csv files based on the IDs of the airborne moving platform and the target. The target data storage file is named `target+target_number.csv`, with the number of targets equal to the total number of target files. Each target file stores the target's motion information, including its position coordinates and flight orientation for each frame within the multi-sensor network coordinate system. One line in the file represents one frame of data, and the time interval between each frame can be set by modifying the simulation system input. Similarly, the airborne moving platform data storage file is named `platform+airborne_moving_platform_number.csv`, with the number of airborne moving platforms equal to the total number of airborne moving platform data storage files. Each airborne moving platform file stores the platform's motion information, including its flight orientation, sensor orientation, pixel coordinates of each target within the sensor, and infrared radiation intensity information for each target. One line in the file represents one frame of data, and the time interval between each frame can be set by modifying the simulation system input.

[0080] The visualization module uses 3D graphics to display the motion trajectory of the target and sensor platform in the network coordinate system, showcasing the formation of the target and airborne sensor platform. It also uses 2D graphics to display the imaging effect of the target in the sensor imaging plane. The development method of the visualization module is as follows: by reading the motion data of the target and airborne platform from the target+target number.csv file that stores target data and the platform+airborne platform number.csv file that stores airborne moving platform data, the module draws the motion trajectory of the target and airborne moving platform in the multi-sensor network coordinate system. By reading the target position coordinates, target flight orientation, platform position coordinates, platform flight orientation, sensor orientation, and pixel coordinates of each target from the target+target number.csv file storing target data and the platform+airborne moving platform number.csv file storing airborne moving platform data, the target pitch azimuth angle and the distance between the target and the sensor are calculated by combining the platform orientation, sensor orientation, and target flight orientation with the spatial registration module. Then, by combining the target infrared characteristic data and image library based on measured data and the pixel coordinates of each target in the sensor, the scaled infrared characteristic images of each target are imaged on the specified pixel coordinate positions of the sensor's two-dimensional imaging plane.

[0081] In one embodiment, a data processing module, an error module, a data storage and visualization module, a time registration module, a spatial registration module, a measured database, a model database, and a configuration module constitute the dynamic platform multi-infrared sensor collaborative multi-target perception digital simulation platform of this application. After the simulation system is developed, the input of the simulation system is configured to simulate different dynamic platform multi-infrared sensor multi-target perception simulation environments. The input of the simulation system includes target items, platform items, error items, and simulation settings items. The target items include the number and number of targets, the flight orientation of each target, the initial position of each target, and the motion model of each target. The platform items include the number and number of platforms, the flight orientation of each platform, the initial position of each platform, the motion model of each platform, and the rotation model of each sensor. The error items include platform position error, sensor angle measurement error, and sensor missed detection error. The simulation settings items include the total simulation time, simulation step size, and data storage path.

[0082] After configuring different simulation systems, based on the model-based target and sensor platform motion and model database in the simulation system, and combined with the time registration module, spatial registration module, and error module, motion data of each platform and target, as well as observation data of each sensor, are generated. Based on the target infrared characteristic data and image library based on measured data in the simulation system, and combined with the detection distance of each target relative to each sensor and the detection elevation and azimuth angle of each target relative to each sensor, infrared characteristic data and images of each target are generated. The motion data of each platform and target, the observation data of each sensor, and the infrared characteristic data and images of each target are classified and stored through the data storage and visualization module. At the same time, the three-dimensional spatial motion trajectory of multiple platforms and multiple targets and the two-dimensional infrared imaging of multiple targets by each sensor are drawn.

[0083] In one specific embodiment, a digital simulation platform for multi-infrared sensor collaborative multi-target perception is provided, such as... Figure 1The diagram showing the structure of the multi-infrared sensor collaborative multi-target perception digital simulation platform for the moving platform illustrates that the entire simulation system consists of a database layer and a module layer. The database layer contains two databases: a target infrared characteristic data and image library based on measured data, and a model-based target, sensor platform motion and observation database. The target infrared characteristic data and image library based on measured data is constructed by processing the infrared radiation characteristic data of the detected target under several typical working conditions and the infrared characteristic images of the sensor imaging plane of the detected target, combined with the target infrared characteristic data interpolation module and the target specified distance image generation module in the data processing module. The model-based target, sensor platform motion and observation database is constructed by combining the kinematic model of the airborne moving platform, the kinematic model of the detected target, and the two-dimensional imaging model of the airborne sensor, while considering the airborne moving platform position error, sensor missed detection error, and sensor angle measurement error in the error module. In addition to the two databases and two modules mentioned above, the simulation system also includes a time registration module and a spatial registration module. The time registration module includes time soft synchronization and time hard synchronization, while the spatial registration module is the coordinate transformation process from a geodetic coordinate system to a multi-sensor network coordinate system. Combining all the aforementioned databases and modules, the text data generated during the fully digital simulation experiment in the moving platform multi-infrared sensor multi-target perception simulation environment is read by the data storage and visualization module, and then categorized, stored, and visualized. The following section will use an aircraft target as an example to introduce the entire process of constructing a moving platform multi-infrared sensor collaborative multi-target perception digital simulation platform for that aircraft target.

[0084] The first step is to construct a database of infrared characteristic data and images of targets based on measured data. For aircraft targets, infrared radiation characteristic data and sensor imaging plane infrared characteristic images are obtained under several typical operating conditions. This data and images are obtained through simulated detection of expert-approved aircraft target models using infrared sensors or through actual detection of physical aircraft targets using physical sensors. Since this process only considers how to combine the infrared radiation characteristic data of aircraft targets under several typical operating conditions and the sensor imaging plane infrared characteristic images to establish the database, it is assumed that the infrared radiation characteristic data and sensor imaging plane infrared characteristic images under several typical operating conditions have already been obtained through detection of physical aircraft targets.

[0085] The obtained infrared radiation characteristic data of the detected aircraft target is obtained by detecting the aircraft target under several different operating conditions (different flight altitudes and speeds of the aircraft target) at a specified detection distance, by changing the detection pitch angle and azimuth angle of the sensor relative to the detected aircraft target. The detection pitch angle ranges from -90 degrees to +90 degrees. A positive detection pitch angle indicates that the sensor is observing the aircraft target obliquely above the target, and a negative detection pitch angle indicates that the sensor is observing the aircraft target obliquely below the target. The detection azimuth angle represents the observation of the target from different positions along the circumference of the target. The detection azimuth angle ranges from 0 degrees to 360 degrees. An azimuth angle of 0 degrees and 360 degrees indicates that the sensor is observing the target along the head-on direction, and an azimuth angle of 180 degrees indicates that the sensor is observing the target from the tail direction. By setting different detection azimuth angles and detection pitch angle intervals, it is possible to obtain the infrared radiation intensity data of the aircraft target at different detection pitch angles and azimuth angles under certain flight altitudes and speeds.

[0086] Assuming an aircraft target flies at a speed of 100 m / s and an altitude of 5000 m, and the sensor is 500 m away from the target, by changing the sensor's pitch angle relative to the target at 2-degree intervals and the azimuth angle at 5-degree intervals, a total of 6497 omnidirectional infrared radiation intensity data points can be obtained for this flight condition. Similarly, by adjusting only the aircraft target's flight condition, omnidirectional infrared radiation characteristic data for different flight conditions can be obtained. Taking a target flight condition of 2 km to 6 km altitude and 50 m / s to 150 m / s speed as an example, by using 1 km altitude intervals and 20 m / s speed intervals, a total of 162425 infrared radiation characteristic data points for all 25 flight conditions can be obtained. Figure 2 The figure shows the infrared radiation characteristics of an aircraft target when detected by an infrared sensor at a fixed pitch angle of 30 degrees in different azimuths. The polar coordinates in the figure represent the detection azimuth angle of the sensor relative to the target, ranging from 0 degrees to 360 degrees. The curves represent the changes in infrared radiation intensity of the aircraft target under specified operating conditions and a specified detection pitch angle of 30 degrees at different detection azimuth angles.

[0087] Combining the above data, the target infrared characteristic data interpolation module in the data processing module performs multivariate spline interpolation to obtain more detailed infrared radiation characteristic data of aircraft targets under all operating conditions and in all directions, within the flight altitude range of 2km to 6km and flight speed range of 50m / s to 150m / s. The specific data intervals after interpolation can be freely configured; for example, it can be set as one interval for a target flight altitude of 500m, one interval for a target flight speed of 10m / s, one degree for the sensor's detection pitch angle relative to the target within the range of -90 degrees to +90 degrees, and one degree for the sensor's detection azimuth angle relative to the target within the range of 0 degrees to 360 degrees. Data interpolation processing simplifies the acquisition of data for targets under the above-mentioned flight altitude range of 2km to 6km and flight speed range of 50m / s to 150m / s. By acquiring infrared characteristic data of aircraft targets under only a few operating conditions, a more detailed infrared radiation characteristic database of aircraft targets under all operating conditions and in all directions can be constructed through interpolation methods. The interpolation method is as follows:

[0088] The first step is to perform univariate cubic spline interpolation on the azimuth angle, based on the azimuth angle data. and its corresponding infrared radiation intensity Through cubic spline functions Interpolate the data. The cubic spline function is as follows:

[0089]

[0090] in, Indicates the first Azimuth angle These are the first, second, third, and fourth coefficients to be solved. The four coefficients are solved using the following three conditions.

[0091] Interpolation condition: The spline function passes through the data points at the nodes, that is:

[0092]

[0093] in, Indicates the first Azimuth angle Indicates the first The infrared radiation intensity of the aircraft target corresponding to each azimuth angle.

[0094] Continuity condition: The spline function and its first and second derivatives are continuous at the nodes, that is:

[0095]

[0096] in, The first derivative of the spline function. It is the second derivative of the spline function.

[0097] Boundary condition: The second derivative is zero at the boundary, that is:

[0098]

[0099] in, This represents the total number of data points.

[0100] The second step is to perform bivariate spline interpolation on the interpolated data for different pitch angles, based on the interpolation in the azimuth direction. Assume the pitch angle is already known. and azimuth Infrared radiation intensity Through bivariate spline functions Interpolation is performed using the following formula:

[0101]

[0102] in, It is a spline basis function. These are interpolation coefficients. It is the number of pitch angles. This refers to the number of azimuth angles. The coefficients are determined using the following three conditions.

[0103] Interpolation condition: The spline function passes through the data points at the nodes, that is:

[0104]

[0105] in, Indicates the first A pitch angle, Indicates the first Azimuth angle Indicates the first The pitch angle and the first The infrared radiation intensity of the aircraft target corresponding to each azimuth angle.

[0106] Continuity condition: The spline function and its first and second partial derivatives must be continuous at every node, that is:

[0107] For continuity in the pitch direction:

[0108]

[0109] Regarding continuity in the azimuth direction:

[0110]

[0111] Continuity in the mixing direction:

[0112]

[0113] Boundary conditions:

[0114]

[0115] The third step involves performing bivariate spline interpolation on the interpolated data at different pitch angles, and then performing data interpolation under multiple operating conditions, assuming existing flight states. Pitch angle and azimuth Infrared radiation intensity Through three-dimensional variable spline functions Interpolation is performed using the following formula:

[0116]

[0117] in, It is a spline basis function. These are interpolation coefficients. It represents the number of working states. It is the number of pitch angles. This refers to the number of azimuth angles. The interpolation coefficients are determined using the following three conditions.

[0118] Interpolation conditions:

[0119]

[0120] in, Indicates the first The operating conditions of the aircraft target Indicates the first A pitch angle, Indicates the first Azimuth angle Indicates the first The operating conditions of the aircraft target, the first The pitch angle and the first The infrared radiation intensity of the aircraft target corresponding to each azimuth angle.

[0121] Continuity condition:

[0122] Regarding the continuity in the direction of flight state:

[0123]

[0124] For continuity in the pitch direction:

[0125]

[0126] Regarding continuity in the azimuth direction:

[0127]

[0128] Boundary conditions:

[0129]

[0130] By combining the existing infrared radiation characteristic data under several operating conditions, more detailed infrared radiation characteristic data of aircraft targets under all operating conditions and in all directions can be interpolated, thereby constructing a database of infrared radiation characteristics of the detected aircraft targets under all operating conditions and in all directions.

[0131] The obtained infrared image of the sensor imaging plane of the detected aircraft target is a grayscale image of the infrared characteristics of the aircraft target obtained by detecting the target at a relatively close detection distance by changing the detection pitch and azimuth angles of the sensor relative to the detected aircraft target. This image is a two-dimensional image of the detected aircraft target in the infrared sensor imaging plane, and is a visual representation of the infrared radiation intensity of the detected target surface. The larger the grayscale value in the image, the greater the infrared radiation intensity of the area representing the aircraft target. The detection pitch angle ranges from -90 degrees to +90 degrees. A positive detection pitch angle indicates that the sensor is observing the aircraft target obliquely above the target, and a negative detection pitch angle indicates that the sensor is observing the aircraft target obliquely below the target. The detection azimuth angle represents the observation of the target from different positions along the circumference of the target. The detection azimuth angle ranges from 0 degrees to 360 degrees. When the detection azimuth angle is 0 degrees and 360 degrees, it indicates that the sensor is observing the target along the head-on direction of the target. When the detection azimuth angle is 180 degrees, it indicates that the sensor is observing the target from the tail direction of the aircraft target. In close-range scenarios (detection distance between an unknown target and the sensor), by setting different detection azimuth and pitch angle intervals, infrared grayscale images of the target at different detection pitch and azimuth angles under a certain flight altitude and speed can be obtained. Assuming an aircraft target has a flight speed of 100 m / s and a flight altitude of 5000 m, and the sensor's detection distance to the aircraft target is relatively close (unknown), by changing the sensor's detection pitch angle relative to the target at 2-degree intervals and the sensor's detection azimuth angle relative to the target at 10-degree intervals, a total of 3293 omnidirectional infrared grayscale images of the target under this flight condition can be obtained. These 3293 images represent the infrared characteristic images of the aircraft target's sensor imaging plane. Figure 3 The image shown is a schematic diagram of the infrared grayscale image of an aircraft target detected at a certain pitch and azimuth angle, extracted from the infrared characteristic image of the sensor imaging plane. It includes infrared grayscale images of the target detected at pitch angles of -20 degrees and azimuth angles of 0 degrees to 360 degrees (10-degree intervals).

[0132] After obtaining the infrared characteristic image of the aircraft target's sensor imaging plane under the above-mentioned working condition, the target infrared characteristic grayscale image at a specified target distance from the sensor at a certain moment is calculated by combining the following scaling method in the target specified distance image generation module.

[0133] Let the length of the resolution dimension of the aircraft target image in the infrared characteristic image of the aircraft target sensor obtained under the above operating conditions be . Width is Height is The actual dimensions of the detected aircraft target are as follows: , width is Gao Wei Based on the theory of similar triangles transformed from the sensor coordinate system to the image coordinate system, the size of the aircraft target image and its actual size follow the following formula:

[0134]

[0135] in This indicates the distance between the target and the sensor. This indicates the focal length of the sensor. Since the pixel size along the horizontal and vertical axes of the sensor is equal, the focal length is [value missing] at a distance of [value missing]. The detection of aircraft targets involves measuring the resolution dimension of the image of the aircraft target within the sensor's imaging plane. ,width ,high The number of pixels its image occupies in the imaging plane can be used to determine this. We obtain the following formula:

[0136]

[0137] In the above formula, Indicates the pixel size of the sensor. This indicates the actual number of pixels occupied by the aircraft target in the width direction of the current image. This indicates the actual number of pixels occupied by the aircraft target in the current image along its length. This indicates the actual number of pixels occupied by the aircraft target in the current image along the altitude direction. The distance from the target to the sensor is then calculated. It can be calculated using the following formula:

[0138]

[0139] Combining the flight orientation of the airborne moving platform, the actual orientation of the sensor, and the flight orientation of the aircraft target, the target's azimuth and pitch angles relative to the sensor at a certain moment are calculated. Existing pitch and azimuth angles are searched in the infrared characteristic image of the sensor imaging plane of the detected aircraft target. The attitude of the aircraft target in the infrared sensor at the current moment is determined by the following judgment principles:

[0140]

[0141] in, This indicates the current pitch angle of the aircraft target relative to the sensor. This represents the pitch angle already present in the near-field image library. This indicates the current azimuth angle of the aircraft target relative to the sensor. This represents the azimuth angle already present in the near-field image library. If the elevation and azimuth detection intervals in the near-field infrared image library are refined to smaller degrees, the specific 1° and 5° in the criteria can be adjusted to other degrees.

[0142] After determining the current attitude of the aircraft target in the infrared sensor, the distance between the target and the sensor is determined by checking the near-field image database. Combined with the actual distance between the target and the sensor at the current moment Calculate image scaling ratio The scaling ratio is calculated using the following formula:

[0143]

[0144] After obtaining the above scaling ratio, the actual distance between the target and the sensor at the current moment is determined by the following formula. The actual imaging resolution of the target after disembarking from the aircraft in the sensor is calculated as follows:

[0145]

[0146] In the formula, This indicates the actual distance between the aircraft target and the sensor at the current moment. The lateral resolution of the aircraft target in the sensor imaging plane. This indicates the actual distance between the aircraft target and the sensor at the current moment. The longitudinal resolution of the aircraft target in the sensor imaging plane. This indicates the distance between the aircraft target and the sensor. The lateral resolution of the aircraft target in the sensor imaging plane. This indicates the distance between the aircraft target and the sensor. The horizontal resolution of the aircraft target in the sensor imaging plane.

[0147] Finally, combining the above , , , By scaling the infrared characteristic image of the sensor imaging plane of the target being detected, a grayscale image of the target's infrared characteristics at a specified distance from the sensor at a given moment can be generated. Based on the above method, a full-range, omnidirectional infrared characteristic image library of the detected aircraft target can be constructed.

[0148] The infrared radiation characteristics database of the detected aircraft target under all operating conditions and in all directions, as well as the infrared characteristic image library of the detected aircraft target at all distances and in all directions, constitute the infrared characteristic data and image library of the aircraft target based on measured data. The above process describes the method of constructing the infrared characteristic data and image library of the target based on measured data, taking the aircraft target as an example.

[0149] Secondly, the model-based database of target and sensor platform motion and observation is constructed. The relevant models include the dynamic model of the target being detected, the dynamic model of the airborne moving platform carrying sensors, and the two-dimensional imaging model of the airborne sensors. Specifically, the construction of the dynamic model of the airborne moving platform needs to incorporate the position error of the airborne moving platform from the error module; the construction of the two-dimensional imaging model of the airborne sensors needs to consider the sensor's missed detection error and angle measurement error. The following sections will describe the methods for constructing each model and the model-based database of target and sensor platform motion and observation.

[0150] ① Dynamic model of the detected target. In this embodiment, the detected target is an aircraft target. Its kinematic model can be selected according to different flight stages and mission requirements, including models of uniform linear motion, uniformly accelerated motion, uniform circular motion, parabolic motion, spiral trajectory, rotational motion, high-speed flight, etc. The following assumes that the aircraft target's motion model is along the coordinate system of the multi-sensor network. If the target aircraft moves at a constant velocity in the positive direction parallel to the axis, then the kinematic model of the target aircraft is as follows:

[0151]

[0152] in, These represent the position coordinates of the detected aircraft target in the multi-sensor network coordinate system. For time, A fixed, positive value greater than zero, representing the aircraft target along the multi-sensor network coordinate system. Velocity in the direction parallel to the axis.

[0153] ② Dynamic model of the airborne moving platform equipped with sensors. Assuming in this embodiment the airborne moving platform is an unmanned aerial vehicle (UAV) moving platform, its commonly used kinematic models include uniform linear motion, uniformly accelerated motion, three-dimensional spatial motion, rotational motion (such as Euler angles and quaternions), and physically constrained flight control models. Below, we assume the motion model of the UAV moving platform equipped with sensors is along the coordinate system of the multi-sensor network. If the UAV platform carrying the sensor moves at a constant velocity in the negative direction parallel to the axis, the kinematic model is as follows:

[0154]

[0155] in, These represent the position coordinates of the UAV platform in the multi-sensor network coordinate system. For time, A fixed, negative value less than zero, representing the UAV's moving platform along the multi-sensor network coordinate system. Velocity in the direction parallel to the axis.

[0156] When constructing the dynamic model of the airborne moving platform equipped with sensors, the position error of the airborne moving platform is considered. The position error is added to the position coordinates of the airborne moving platform in the multi-sensor network coordinate system at each moment with a fixed deviation. In this embodiment, the position error of the UAV moving platform is added in the following way:

[0157]

[0158] in,

[0159]

[0160] Among them, standard deviation , This indicates the position error of the UAV's moving platform. Adding "UAV moving platform position error" in this way means that the resulting random error has a 99.7% probability of occurring within the specified position error range. It is worth mentioning that the position error of the airborne moving platform is a random error generated at the initial moment, and this error remains unchanged at any subsequent moment. Therefore, even with this error, the motion law of the airborne moving platform still conforms to its own kinematic model.

[0161] ③ The two-dimensional imaging model of the airborne sensor represents a spatial target point located in the multi-sensor network coordinate system, which is transformed into the infrared sensor coordinate system through rotation and translation. Then, the pixel coordinates of the target point in the image coordinate system are obtained through the sensor intrinsic parameters. The following assumes an aircraft target located in the multi-sensor network coordinate system. The position coordinates are Its pixel coordinate position in the sensor imaging plane The calculation formula is as follows:

[0162]

[0163] In the above formula, the subscript is left-hand subscript. Indicates the coordinate system of a multi-sensor network, with the subscript at the left. Represents the image coordinate system of the sensor. They represent aircraft targets respectively. The horizontal and vertical pixel coordinates in the sensor's imaging plane Represents the target in the sensor coordinate system of Axis coordinates. Indicates the focal length of the sensor. Indicates the pixel size of the sensor. Indicates the coordinates of the center of the sensor's imaging plane. Indicates the translation of the sensor. This is the rotation matrix of the sensor coordinate system relative to the multi-sensor network coordinate system, which includes noise representing angular measurement error.

[0164] As mentioned above, This step involves adding a rotation matrix to the sensor coordinate system relative to the multi-sensor network coordinate system to represent noise indicating angular measurement error. The specific method for adding this noise is as follows:

[0165] The pixel coordinates in the sensor imaging plane are obtained by translating and transforming the target position in space through coordinate projection. During the coordinate projection transformation, the rotation matrix of the sensor coordinate system relative to the multi-sensor network coordinate system is... Add angle measurement error, specifically represented as follows:

[0166]

[0167] in,

[0168]

[0169] in These represent the sensor's pitch, yaw, and roll angles, respectively. These represent the random errors of the sensor in the pitch, yaw, and roll directions, respectively. The standard deviation , This indicates the angle measurement error of the infrared sensor, meaning that the generated noise error has a 99.7% probability of occurring at the specified angle measurement error. Within the range.

[0170] In addition, the missed detection error of the sensor must also be considered. The missed detection error of the infrared sensor follows a 0, 1 distribution, where the probability of being 1 is 90% and the probability of being 0 is 10%, which means that there is a 10% probability that the two-dimensional pixel coordinates in the sensor imaging plane cannot be obtained from the projection transformation of the spatial target coordinate position.

[0171] The model-based target and sensor platform motion and observation database is constructed through the above process. Various errors are considered during the construction process to ensure that the data generated by the simulation system based on this database is as close as possible to the actual situation.

[0172] Secondly, the time registration module includes hard synchronization and soft synchronization. Hard synchronization refers to directly connecting the clocks of multiple sensors physically in hardware or synchronizing them through a dedicated synchronization signal source. In the multi-infrared sensor collaborative multi-target perception digital simulation platform proposed in this embodiment, hard synchronization is achieved by acquiring data in real time within the same simulation loop. In addition, the time registration module also includes a soft synchronization method, which adjusts or corrects the timestamps of sensor data using algorithms to achieve time synchronization between multiple sensors. This embodiment achieves soft synchronization through interpolation, as detailed below. Figure 4 As shown, sensors 1 and 2 are used to observe moving targets in space. From the first to the fifth frame, due to time asynchrony, the aircraft targets observed by the sensors are... The targets do not overlap in space. By interpolating the pixel coordinates of the known aircraft target in the sensor, the continuous observed motion trajectory of the aircraft target is interpolated. Then, by combining the interpolated continuous observed motion trajectory of the aircraft target, the pixel coordinates of the target in the imaging planes of different sensors at the same time stamp are calculated, thereby achieving time soft synchronization and data alignment. The specific interpolation method is as follows:

[0173] Let's assume the objective is at time 0. The pixel coordinates in the sensor imaging plane are After that The target pixel coordinates observed at each time point are denoted as follows: Treating the above pixel coordinates as the independent variable time The dependent variable, which is located under the timestamp of the reference sensor, will be the above. Each moment is divided into Then use the cubic spline piecewise interpolation function for each interval. To interpolate the continuous trajectory of the aircraft target in the imaging plane, the interpolation function is as follows:

[0174]

[0175] In the formula, Indicates the first Within the interval, the first The pixel coordinates of the aircraft target at any given time. They represent the first The unknown coefficients of the cubic spline interpolation function for each interval, with four unknown coefficients for each interval. Given a range, the solution is required. One unknown coefficient. Simultaneously, the piecewise interpolation function within each spline interval is obtained. It is continuous and smooth at all known points, satisfying:

[0176]

[0177] The aircraft target is obtained by combining the cubic spline interpolation function equation. Continuous trajectory track in the imaging plane of sensor 1 Its trajectory equation is:

[0178]

[0179] In the formula In the imaging plane of sensor 1 The basis functions for each spline interval within the trajectory. In the imaging plane of sensor 1 The spline coefficients for each spline interval within the trajectory.

[0180] Similarly, by combining the cubic spline interpolation function equation, the target can be obtained. Continuous trajectory track in the imaging plane of sensor 2 Its trajectory equation is:

[0181]

[0182] In the formula In the imaging plane of sensor 2 The basis functions for each spline interval within the trajectory. In the imaging plane of sensor 2 The spline coefficients for each spline interval within the trajectory.

[0183] The above method is used to perform time soft synchronization on the data observed from the two sensors. If we want to obtain... time By processing or fusing the pixel coordinate data of the target in different sensors, the motion trajectory equation of the two targets can be continuously observed. and Bring in the moment After obtaining the soft synchronization of the completion time The pixel coordinates of the target in different sensors are used to eliminate observation bias after soft synchronization, ensuring that all observed points are in space. point.

[0184] Secondly, the process of the spatial registration module is as follows: Figure 5 As shown, the obtained latitude, longitude, and altitude coordinates are first transformed from the geodetic coordinate system to the geocentric-fixed coordinate system (GDC). The origin of the GDC is the Earth's center. Then, through coordinate transformation and translation, they are transformed to a multi-sensor network coordinate system to obtain the network coordinates. The origin of the multi-sensor network coordinate system is the zero-time position of a specific sensor. The purpose of this process is to ensure accurate comparison, fusion, and analysis of spatial data from different sensors, at different times, or from different perspectives, thereby reducing errors caused by differences in the perspectives of different sensors. Figure 5 The specific process of spatial registration is described as follows:

[0185] In practice, the position information of the UAV platform and sensors is usually obtained from latitude, longitude, and altitude coordinates by the onboard navigation system, which are represented in a geodetic coordinate system. This embodiment assumes that the centroid coordinates of the UAV platform coincide with the optical center coordinates of the sensor, and transforms the sensor's position coordinates into geodetic coordinates using the following coordinate transformation method:

[0186]

[0187] superscript Indicates a specific moment, subscript Represents the geocentric and Earth-fixed coordinate system. For the first The location of the unmanned aerial vehicle platform's center of mass is measured in longitude, latitude, and altitude coordinates at all times. These are the coefficients of the elliptic expression of the meridional elliptical plane where the UAV platform is located, representing the major axis radius and minor axis radius of the ellipse, respectively. For the first The coordinates of the center of mass of the unmanned aerial vehicle platform in the Earth's solid system at all times. To express coefficients through ellipse The first parameter to be solved is N, which is the second parameter obtained by combining the above parameters.

[0188] After obtaining the coordinates of the multi-sensor in the geocentric-ground-fixed coordinate system, all sensors are unified into the same multi-sensor network coordinate system by unifying the coordinate origin. The unified coordinate origin is the coordinate point of a certain sensor in the multi-sensor system at the initial position height of 0 at time zero in the geocentric-ground-fixed coordinate system.

[0189] Let the origin of the network coordinate system be... Then the other sensors In the Position in the multi-sensor network coordinate system at all times It can be obtained using the following formula:

[0190]

[0191] In the formula, For the first The sensor at the first Latitude and longitude coordinates of the moment For the first The sensor at the first The geocentric and geofixed coordinates at any given time. The coordinate system of a multi-sensor network follows the right-hand rule. The axis points east. The axis points north. The axis is perpendicular to the ground and points upwards.

[0192] Finally, combining all the aforementioned databases and modules, the text data generated during the fully digital simulation experiment in the multi-infrared sensor multi-target perception simulation environment of the moving platform is read by the data storage and visualization module. The data is then categorized, stored, and visualized. This module effectively manages, stores, and displays the large amount of text data and images collected by multiple sensors, while providing a user-friendly interface for viewing and analyzing this data. It includes a data storage module and a visualization module. The data storage module can access different types of sensor data and can categorize and store data according to sensor ID and target ID. The visualization module uses 3D graphics to display the motion trajectories of the target and sensor platform in the network coordinate system, showcasing the target and sensor airborne moving platform formation, and uses 2D graphics to display the infrared imaging effects of multiple targets in the sensor imaging plane.

[0193] The data storage module is based on logic written in Python, which stores all sensor and target data in .csv files according to the airborne platform ID and target ID.

[0194] The target data storage file is named target+target_number.csv. The number of targets equals the total number of target data storage files. The data stored in each target's storage file is the target's motion information, including the target's position coordinates and flight orientation for each frame in the multi-sensor network coordinate system. The data storage format is: simulation time, target's X-axis position coordinates, target's Y-axis position coordinates, target's Z-axis position coordinates, flight yaw angle, flight pitch angle, and flight roll angle. The unit for position coordinates is meters, and the unit for angles is degrees. Each line in the target+target_number.csv file represents one frame of data, and the time interval between each frame of data can be set by modifying the input of the simulation system.

[0195] The airborne motion platform data storage files are named `platform+airborne motion platform number.csv`. The number of airborne motion platforms equals the total number of final airborne motion platform data storage files. Each airborne motion platform's storage file contains motion information, including the platform's flight orientation, sensor orientation, pixel coordinates of targets within the sensors, and infrared radiation intensity information for each target. The data storage format is: simulation time, X-axis position coordinates of the airborne motion platform, Y-axis position coordinates of the airborne motion platform, Z-axis position coordinates of the airborne motion platform, flight yaw angle of the airborne motion platform, and flight distance of the airborne motion platform. The data includes pitch angle, roll angle of the airborne moving platform, pitch angle of the sensor, yaw angle of the sensor, x-coordinate of target 1 pixel within the sensor, y-coordinate of target 1 pixel within the sensor, infrared radiation intensity of target 1 within the sensor, x-coordinate of target 2 pixel within the sensor, y-coordinate of target 2 pixel within the sensor, infrared radiation intensity of target 2 within the sensor, ..., x-coordinate of target n pixel within the sensor, y-coordinate of target n pixel within the sensor, and infrared radiation intensity of target n within the sensor, where n represents the total number of targets. The unit for position coordinates is meters, the unit for angles is degrees, the unit for pixel coordinates is resolution, and the unit for infrared radiation intensity is watts per steradian. Each line in the file `platform+airborne moving platform number.csv` represents one frame of data, and the time interval between each frame can be set by modifying the input of the simulation system.

[0196] The visualization module is written in Python and uses the matplotlib library in Python to draw 2D and 3D graphics. It reads the motion data of the target and the airborne platform from the `target+target_number.csv` file (containing target data) and the `platform+airborne_platform_number.csv` file (containing airborne moving platform data), and plots the trajectories of the targets and airborne moving platforms in a multi-sensor network coordinate system. It reads the target position coordinates, target flight orientation, platform position coordinates, platform flight orientation, sensor orientation, and pixel coordinates of each target from the `target+target_number.csv` and `platform+airborne_platform_number.csv` files. Using the platform orientation, sensor orientation, and target flight orientation, combined with the spatial registration module, it calculates the target's pitch and azimuth angles relative to the sensors, as well as the target's distance from the sensors. Then, combining this with the target's infrared characteristic data based on measured data, an image library, and the pixel coordinates of each target in the sensors, it images the scaled infrared characteristic images of each target onto the specified pixel coordinate positions on the sensor's two-dimensional imaging plane.

[0197] The target infrared characteristic data and image library based on measured data, the target and sensor platform motion and observation database based on the model, the data processing module, the spatial registration module, the temporal registration module, the error module, and the data storage and visualization module constitute the motion platform multi-infrared sensor collaborative multi-target perception digital simulation platform described in this application.

[0198] The usage process of the multi-infrared sensor collaborative multi-target perception digital simulation platform is as follows: Figure 6 As shown in the figure, the simulation system is configured by setting different simulation system inputs. The simulation system inputs include target items, airborne moving platform items, error items, and simulation settings. The target items include the number and number of targets, the flight orientation of each target, the initial position of each target, and the motion model of each target. The airborne moving platform items include the number and number of platforms, the flight orientation of each platform, the initial position of each platform, the motion model of each platform, and the rotation model of each sensor. The error items include platform position error, sensor angle measurement error, and sensor missed detection error. The simulation settings include the total simulation time, simulation step size, and data save path. After configuring the different simulation system inputs, based on the base... The model combines the target and sensor platform motion and observation database with time registration, spatial registration, and error modules to generate motion data for each platform and target, as well as observation data for each sensor. Based on the target infrared characteristic data and image library based on measured data in the simulation system, the infrared characteristic data and images of each target are generated by combining the detection distance of each target relative to each sensor and the detection elevation and azimuth angle of each target relative to each sensor. The motion data of each platform and target, the observation data of each sensor, and the infrared characteristic data and images of each target are classified and stored through the data storage and visualization module. At the same time, the three-dimensional spatial motion trajectory of multiple platforms and multiple targets and the two-dimensional infrared imaging of multiple targets by each sensor are drawn.

[0199] In one specific embodiment, as follows Figure 7 The multi-infrared sensor multi-target collaborative perception scenario shown illustrates the use of a distributed multi-infrared sensor simulation system and its data visualization effects. Figure 7In the multi-sensor network coordinate system shown, there are two airborne moving platforms equipped with sensors and two targets. The two airborne moving platforms are 150m apart, and each platform moves at a constant speed along the negative X-axis at the same altitude. The sensors on each platform always point towards the geometric center of the target formation. The two targets are 100m apart, and each target moves at a constant speed along the positive X-axis at the same altitude. The initial geometric center of the two airborne moving platforms is 5000 meters horizontally and 500 meters vertically from the initial geometric center of the two targets. The target data storage file and airborne platform data storage file generated by the data storage module in this scenario are in the following formats: Figure 8 As shown, the target data storage files are target1.csv and target2.csv, and the airborne platform data storage files are platform1.csv and platform2.csv. The target data storage file includes seven columns of data: simulation time, target X-axis position coordinates, target Y-axis position coordinates, target Z-axis position coordinates, flight yaw angle, flight pitch angle, and flight roll angle. Each row of data represents one frame of data, and the time interval between each frame of data is 0.05 seconds. The airborne moving platform data storage file contains fifteen columns of data, namely: simulation time, X-axis position coordinates of the airborne moving platform, Y-axis position coordinates of the airborne moving platform, Z-axis position coordinates of the airborne moving platform, flight yaw angle of the airborne moving platform, flight pitch angle of the airborne moving platform, flight roll angle of the airborne moving platform, pitch angle of the sensor, yaw angle of the sensor, x-coordinate of target 1 pixel within the sensor, y-coordinate of target 1 pixel within the sensor, infrared radiation intensity of target 1 within the sensor, x-coordinate of target 2 pixel within the sensor, y-coordinate of target 2 pixel within the sensor, and infrared radiation intensity of target 2 within the sensor. Each row of data represents one frame of data, and the time interval between each frame of data is 0.05 seconds.

[0200] Combination such as Figure 8 The data shown is used to create the following visualizations using the data visualization module: Figure 9 The multi-platform, multi-target three-dimensional spatial motion trajectory shown, and as... Figure 10 and Figure 11 The images shown depict two-dimensional multi-target infrared imaging using various sensors. Figure 9The CCP has two targets and two airborne mobile platforms equipped with sensors. One airborne platform is 150 meters apart, and each platform moves at a constant speed along the negative X-axis at the same altitude. The sensors on each platform are always pointed towards the geometric center of the target formation. The other two targets are 100 meters apart, and each target moves at a constant speed along the positive X-axis at the same altitude. Figure 10 This shows the infrared characteristics of two targets detected in a certain frame on the imaging plane of the airborne infrared sensor 1 mounted on the airborne platform 1. Figure 11 The image shows the infrared characteristics of two targets detected in a certain frame in the imaging plane of the airborne infrared sensor 2 mounted on the airborne platform 2. There are two targets in the airborne sensor 1 and two targets in the airborne sensor 2. The total resolution of the images from all the sensors is 640*512. The pixel size of the target in the image and the infrared characteristics are calculated based on the target's distance from the sensor in the current frame, the detection pitch, and the azimuth.

[0201] The aforementioned dynamic platform multi-infrared sensor collaborative multi-target perception digital simulation platform can comprehensively simulate the working process of distributed multi-infrared sensors in a virtual environment, providing accurate simulation of multi-target perception using dynamic platform multi-infrared sensors. This allows researchers to repeatedly test and verify the distributed multi-infrared sensor collaborative perception platform under different operating conditions and environments, ensuring the scientific validity and feasibility of the design and reducing uncontrollable factors and risks in real-world testing. Through this platform, developers can systematically adjust parameters such as sensor configuration, perception algorithms, and data fusion strategies, accurately evaluating the impact of different configurations on distributed multi-infrared sensor collaborative algorithms and schemes. This enables the platform to optimize its overall performance and reliability in the early stages, avoiding the time and cost waste caused by repeated hardware adjustments in traditional experiments. By providing a flexible and customizable simulation environment, it allows technology developers to quickly iterate and test new algorithms, new functions, and new designs. Without relying on physical hardware, it efficiently simulates various working conditions in real-world application scenarios, thereby accelerating the research and development and application deployment of new technologies and improving development efficiency.

[0202] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. In addition, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.

[0203] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

Claims

1. A digital simulation platform for multi-target perception using multiple infrared sensors in a dynamic platform, characterized in that, include: The system comprises a data processing module, an error module, a data storage and visualization module, a measured database, and a model database; among which, The error module is used to obtain the position error of the airborne moving platform, the missed detection error of the sensor, and the angle measurement error of the sensor. The model database is used to generate motion data of the target, motion data of the airborne moving platform, and observation data of each sensor based on the kinematic model of the target being detected, the kinematic model of the airborne moving platform, and the two-dimensional imaging model of the sensor, and combined with the position error of the airborne moving platform, the missed detection error of the sensor, and the angle measurement error of the sensor. The data processing module is used to process the infrared radiation intensity data of the target under typical working conditions and the infrared characteristic image of the sensor imaging plane to generate infrared radiation characteristic data and target infrared characteristic grayscale image of the target. The measured database is used to store the infrared radiation characteristics data and target infrared characteristic grayscale images of the detected targets. Combined with the detection distance of each target relative to each sensor, the detection elevation and azimuth angle of each target relative to each sensor, the infrared characteristic data and images of each target are generated. The data storage and visualization module is used to manage and store the infrared characteristic data and images of each target, and to analyze the infrared characteristic data and images of each target to generate the motion trajectory of each target and each sensor in the network coordinate system. The data processing module includes a target infrared characteristic data interpolation submodule. This submodule is used to detect the target under several typical working conditions by using an infrared sensor at different elevation and azimuth angles, obtain the target infrared radiation intensity data, and interpolate the target infrared radiation intensity data to obtain the infrared radiation characteristic data of the target. The interpolation process includes the following steps: Using multivariate spline interpolation, univariate cubic spline interpolation is performed on the azimuth angle in each elevation angle file, based on the azimuth angle data. and its corresponding infrared radiation intensity The data is interpolated using a cubic spline function; After interpolation in the azimuth direction is completed, bivariate spline interpolation is performed on the interpolation data for different elevation angles, assuming the existing elevation angles are... and azimuth Infrared radiation intensity Interpolation is performed using a bivariate spline function; After performing bivariate spline interpolation on the interpolation data at different pitch angles, trivariate spline interpolation under multiple operating conditions is performed, assuming existing flight conditions. Pitch angle and azimuth Infrared radiation intensity Interpolation is performed using a three-dimensional variable spline function.

2. The multi-infrared sensor collaborative multi-target perception digital simulation platform for dynamic platforms according to claim 1, characterized in that, The model database includes: The target motion data generation submodule is used to obtain the motion data of the target based on the kinematic model of the target being detected; The airborne moving platform motion data generation submodule is used to obtain the motion data of the airborne moving platform based on the kinematic model of the airborne moving platform and the position error of the airborne moving platform. The sensor observation data generation submodule is used to obtain the observation data of each sensor based on the sensor's two-dimensional imaging model, the sensor's missed detection error, and the sensor's angle measurement error.

3. The multi-infrared sensor collaborative multi-target perception digital simulation platform for dynamic platforms according to claim 2, characterized in that, The data processing module also includes: Target-specified distance image generation module; where, The target-specified distance image generation module is used to detect targets by infrared sensors at different elevation and azimuth angles, obtain infrared characteristic images of the sensor imaging plane of the detected target, and scale the infrared characteristic images of the sensor imaging plane of the detected target to obtain a grayscale image of the infrared characteristics of the detected target.

4. The multi-infrared sensor collaborative multi-target perception digital simulation platform for dynamic platforms according to claim 3, characterized in that, The scaling process includes the following steps: By taking into account the resolution of the detected target in the current infrared characteristic image of the sensor imaging plane, and combining the actual size of the detected target and the infrared sensor parameters, the actual distance between the detected target and the sensor in the current infrared characteristic image of the sensor imaging plane is calculated. The image scaling ratio at different distances is then calculated based on the different distances between the detected target and the sensor. Based on the infrared characteristic image of the sensor imaging plane of the target being detected, and combined with the calculated image scaling ratio at different distances, grayscale images of the target's infrared characteristics in the infrared sensor imaging plane at different distances, positions, and detection elevation and azimuth angles are generated.

5. The multi-infrared sensor collaborative multi-target perception digital simulation platform for a moving platform according to claim 1, characterized in that, Also includes: The time registration module and the spatial registration module; among them, The time registration module is used to synchronize the timestamps of each sensor. The spatial registration module is used to unify the coordinates of each sensor into the coordinate system of the multi-sensor network.

6. The multi-infrared sensor collaborative multi-target perception digital simulation platform for a moving platform according to claim 5, characterized in that, The data storage and visualization module includes: Data storage submodule and visualization submodule; among which, The data storage submodule is used to access motion data from different types of sensors, target motion data, and sensor observation data, and to classify and store the data according to the sensor ID number and target ID number. The visualization submodule is used to display the motion trajectory of the target and sensors in the multi-sensor network coordinate system based on 3D graphics, so as to show the formation of the target and the airborne moving platform, and to display the imaging effect of the target in the sensor imaging plane based on 2D graphics.

7. The multi-infrared sensor collaborative multi-target perception digital simulation platform for a moving platform according to claim 1, characterized in that, Also includes: The configuration module is used to obtain target items, platform items, error items, and simulation settings; among them, The target items include the number and number of targets, the flight orientation of each target, the initial position of each target, and the motion model of each target; The platform item includes the number and number of airborne moving platforms, the flight orientation of each airborne moving platform, the initial position of each airborne moving platform, the motion model of each airborne moving platform, and the rotation model of each sensor; Error items include airborne moving platform position error, sensor angle measurement error, and sensor missed detection error; The simulation settings include total simulation duration, simulation step size, and data save path.

Images