Mobile platform multi-infrared sensor cooperative multi-target sensing digital simulation platform

Abstract

The invention belongs to the technical field of multi-infrared sensor multi-target collaborative sensing digital simulation. The invention provides a mobile platform multi-infrared sensor cooperative multi-target sensing digital simulation platform. According to the embodiment of the invention, the working process of the distributed multi-infrared sensor can be comprehensively simulated in a virtual environment, and accurate multi-target sensing simulation of the multi-infrared sensor of the moving platform is provided; therefore, researchers can repeatedly test and verify the distributed multi-infrared sensor collaborative sensing platform under different operation conditions and environments, so that the scientificity and feasibility of a design scheme are ensured, and uncontrollable factors and risks in a real test are reduced. Developers can systematically adjust parameters such as sensor configuration, a sensing algorithm and a data fusion strategy, and accurately evaluate the influence of different configurations on a distributed multi-infrared sensor collaborative algorithm and scheme; the overall performance and reliability of the platform can be optimized in the early stage, and time and cost waste caused by repeated adjustment of hardware is avoided.

Description

Technical Field

[0001] The disclosed embodiments relate to the technical field of digital simulation of multi-infrared sensor multi-target collaborative perception, and in particular to a digital simulation platform for multi-infrared sensor collaborative multi-target perception on a moving platform. Background Art

[0002] Infrared sensors do not actively emit electromagnetic waves, and have the advantages of good concealment, strong anti-interference ability, high detection accuracy, and long detection distance. However, a single infrared sensor has defects such as a small detection field of view, can only obtain the target's angle measurement information, and cannot obtain the target's distance and depth information. It cannot fully and accurately capture the target information, so a single infrared sensor cannot accurately track and locate the target. The distributed multi-infrared sensor collaborative perception system composed of multiple infrared sensor nodes can effectively make up for the shortcomings of the insufficient capabilities of a single sensor. With the advancement of drone technology, especially the development of drone clusters and collaborative combat technology, at least two drone-mounted dynamic platforms equipped with infrared sensors can measure the angle of the same target from different angles, and then combine the position and attitude information of the drone-mounted dynamic platform itself to obtain the position, speed, attitude and other information of the target in real time, which has great application value.

[0003] The infrared sensor nodes carried by multiple drone-mounted platforms in the multi-infrared sensor collaborative perception system work together, which can not only cover a wider area and enhance target detection and recognition capabilities, but also improve spatial resolution and target positioning accuracy through the collaboration of multiple sensors, effectively reduce blind spots, and enhance target tracking and positioning capabilities. In addition, the distributed system has stronger robustness, and the collaborative work of multiple sensors can reduce the impact of single point failures on the system and ensure the reliability and stability of the system. Compared with a single sensor, the distributed system utilizes the collaboration of multiple low-cost sensors, which not only reduces deployment costs, but also improves system performance and flexibility.

[0004] However, the existing digital simulation systems of infrared or photoelectric sensors are all for the perception and image generation of a single target on a single platform or a single sensor. Such systems cannot reflect the collaborative work and information sharing between multiple sensors, and cannot perceive the target comprehensively and dynamically. In addition, the existing systems often ignore the dynamic characteristics of target motion and the impact of environmental changes. The target state in most simulation systems is a static or simple motion model, which fails to fully consider factors such as the complexity of target motion, speed changes, acceleration, and possible error accumulation. In addition, the current digital simulation system of infrared sensors does not fully consider error propagation models, such as measurement errors, sensor positioning errors, and target dynamic modeling errors. Due to error accumulation and uncertainty in sensor performance, the reliability and accuracy of a single sensor system in actual operation are often greatly affected. In a multi-sensor collaborative system, the establishment and correction of the error model is particularly critical. The error characteristics of each sensor and the error propagation of collaborative work must be considered to ensure the accuracy and stability of the system. Finally, the current infrared sensor simulation system tends to evaluate the performance of a single sensor or a single target, and lacks comprehensive evaluation and optimization of the overall performance of a multi-sensor, multi-target collaborative perception system.

[0005] Therefore, it is necessary to improve one or more problems existing in the above-mentioned related technical solutions.

[0006] It should be noted that this section is intended to provide background or context for the technical solutions of the present disclosure stated in the claims. The description herein is not admitted to be prior art by virtue of being included in this section. Summary of the invention

[0007] The purpose of the embodiments of the present disclosure is to provide a digital simulation platform for multi-infrared sensors on a moving platform to coordinate multi-target perception, thereby overcoming one or more problems caused by the limitations and defects of related technologies at least to a certain extent.

[0008] According to an embodiment of the present disclosure, a digital simulation platform for multi-infrared sensors and multi-target perception on a moving platform is provided, comprising: Data processing module, error module, data storage and visualization module, measured database and model database; among them, The error module is used to obtain the position error of the airborne moving platform, the missed detection error of the sensor and the angle measurement error of the sensor; A model database is used to generate the motion data of the target, the motion data of the airborne moving platform and the observation data of each sensor based on the kinematic model of the detected target, the kinematic model of the airborne moving platform and the two-dimensional imaging model of the sensor, and in combination with the position error of the airborne moving platform, the missed detection error of the sensor and the angle measurement error of the sensor; A data processing module, used for processing target infrared radiation intensity data of the detected target and the infrared characteristic image of the sensor imaging plane under typical working conditions to generate infrared radiation characteristic data of the detected target and a target infrared characteristic grayscale image; The measured database is used to store the infrared radiation characteristic data of the detected target and the target infrared characteristic grayscale image, and generate the infrared characteristic data and image of each target by combining the detection distance of each target relative to each sensor, and the detection pitch and azimuth of each target relative to each sensor; The data storage and visualization module is used to manage and store the infrared characteristic data and images of each target, and analyze the infrared characteristic data and images of each target to generate the motion trajectory of each target and each sensor in the network coordinate system.

[0009] Furthermore, the model database includes: A target motion data generation submodule is used to obtain the motion data of the target based on the kinematic model of the detected target; The airborne moving platform motion data generation submodule is used to obtain the motion data of the airborne moving platform based on the kinematic model of the airborne moving platform and the position error of the airborne moving platform; The sensor observation data generation submodule is used to obtain the observation data of each sensor based on the two-dimensional imaging model of the sensor, the missed detection error of the sensor and the angle measurement error of the sensor.

[0010] Furthermore, the data processing module also includes: Target infrared characteristic data interpolation submodule and target specified distance image generation module; wherein, The target infrared characteristic data interpolation submodule is used to detect the detected target at different pitch angles and azimuth angles under several typical working conditions through the infrared sensor to obtain the target infrared radiation intensity data, and interpolate the target infrared radiation intensity data to obtain the infrared radiation characteristic data of the detected target; The target specified distance image generation module is used to detect the target at different pitch angles and azimuth angles through the infrared sensor to obtain the sensor imaging plane infrared characteristic image of the detected target, and to scale the sensor imaging plane infrared characteristic image of the detected target to obtain the target infrared characteristic grayscale image of the detected target.

[0011] Furthermore, the interpolation process includes: Using the multivariate spline interpolation method, the azimuth angle in each elevation angle file is interpolated using univariate cubic spline interpolation; After the interpolation in the azimuth direction is completed, bivariate spline interpolation is performed on the interpolation data at different elevation angles; After the interpolation data of different pitch angles are subjected to bivariate spline interpolation, the trivariate spline interpolation under multiple working states is completed.

[0012] Further, the steps of scaling processing include: The actual distance between the detected target in the infrared characteristic image of the current sensor imaging plane and the sensor is calculated by combining the resolution of the detected target image in the infrared characteristic image of the current sensor imaging plane, the actual size of the detected target and the infrared sensor parameters, and the image scaling ratio at different distances is calculated by combining the different distances between the detected target and the sensor; Taking the infrared characteristic image of the detected target in the sensor imaging plane as a reference and combining the calculated image scaling ratios at different distances, the target infrared characteristic grayscale images of the detected target in the infrared sensor imaging plane at different distances, different positions, and different detection pitch and azimuth angles are generated.

[0013] Furthermore, it also includes: Temporal registration module and spatial registration module; wherein, Time registration module, used to synchronize the timestamps of various sensors; The spatial registration module is used to unify the coordinates of each sensor into the multi-sensor network coordinate system.

[0014] Furthermore, the data storage and visualization module includes: Data storage submodule and visualization submodule; among them, The data storage submodule is used to access the motion data of different types of sensors, the motion data of targets and the sensor observation data, and classify and store the data according to the sensor ID number and the target ID number; The visualization submodule is used to display the motion trajectory of the target and the sensor in the multi-sensor network coordinate system based on 3D graphics to display the formation of the target and the airborne moving platform, and to display the imaging effect of the target in the sensor imaging plane based on 2D graphics.

[0015] Furthermore, it also includes: Configuration module, used to obtain target items, platform items, error items and simulation setting items; among them, The target items include the number and serial number of targets, the flight direction of each target, the initial position of each target, and the motion model of each target; The platform items include the number and serial number of airborne moving platforms, the flight direction of each airborne moving platform, the initial position of each airborne moving platform, the motion model of each airborne moving platform, and the rotation model of each sensor; The error terms include the position error of the airborne moving platform, the sensor angle measurement error, and the sensor missed detection error; The simulation settings include total simulation time, simulation step and data saving path.

[0016] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects: In the embodiments of the present disclosure, through the above-mentioned dynamic platform multi-infrared sensor collaborative multi-target perception digital simulation platform, the collaborative perception working process of distributed multi-infrared sensors can be fully simulated in a virtual environment, and accurate dynamic platform multi-infrared sensor multi-target perception simulation can be provided, so that researchers can repeatedly test and verify the distributed multi-infrared sensor collaborative perception platform under different operating conditions and environments to ensure the scientificity and feasibility of the design scheme and reduce the uncontrollable factors and risks in real tests. Through this platform, developers can systematically adjust parameters such as sensor configuration, perception algorithm and data fusion strategy, and accurately evaluate the impact of different configurations on distributed multi-infrared sensor collaborative algorithms and schemes; this enables the platform to optimize the overall performance and reliability of the platform in the early stage, avoiding the time and cost waste caused by repeated adjustment of hardware in traditional experiments. By providing a flexible and customizable simulation environment, technical developers can quickly iterate and test new algorithms, new functions and new designs. Without relying on physical hardware, efficient simulation of various working conditions in actual application scenarios can accelerate the research and development and application deployment of new technologies and improve development efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0018] Figure 1 A schematic diagram showing the structure of a digital simulation platform for multi-infrared sensors and multi-target perception in a mobile platform in an exemplary embodiment of the present disclosure is shown; Figure 2 A schematic diagram showing infrared radiation characteristic data of an aircraft target when an infrared sensor is used to detect at different azimuths at a fixed pitch angle of 30 degrees in an exemplary embodiment of the present disclosure; Figure 3 A schematic diagram of an infrared characteristic grayscale image of an aircraft target when detected at a certain pitch angle and azimuth angle is shown, which is selected from an infrared characteristic image of a sensor imaging plane in an exemplary embodiment of the present disclosure; Figure 4 A schematic diagram showing the use flow of a method for time soft synchronization of a time registration module in an exemplary embodiment of the present disclosure; Figure 5A schematic diagram showing a method flow of a spatial registration module in an exemplary embodiment of the present disclosure is shown; Figure 6 A schematic diagram showing the use process of a multi-infrared sensor collaborative multi-target perception digital simulation platform for a mobile platform in an exemplary embodiment of the present disclosure; Figure 7 A schematic diagram of a multi-target perception simulation environment of a moving platform with multiple infrared sensors in which two airborne platforms cooperate to observe two targets in an exemplary embodiment of the present disclosure is shown; Figure 8 The target data storage file and the data storage file format of the airborne moving platform equipped with the sensor generated by the data storage submodule in the exemplary embodiment of the present disclosure are shown; Fig. 9 A schematic diagram of a multi-platform and multi-target three-dimensional space motion trajectory under a simulation environment configuration in an exemplary embodiment of the present disclosure is shown; Fig.10 A schematic diagram of infrared characteristic imaging of two targets detected in a certain frame in an imaging plane of an airborne infrared sensor 1 carried by an airborne platform 1 in an exemplary embodiment of the present disclosure is shown; Fig.11 A schematic diagram of infrared characteristic imaging of two targets detected in a certain frame in an imaging plane of an airborne infrared sensor 2 carried by an airborne platform 2 in an exemplary embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0019] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0020] In addition, the accompanying drawings are only schematic illustrations of the embodiments of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated descriptions will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities.

[0021] This example embodiment provides a digital simulation platform for multi-infrared sensors on a moving platform to coordinate multi-target perception. Figure 1 As shown in , the digital simulation platform for multi-infrared sensors and multi-target perception on a mobile platform may include: Data processing module, error module, data storage and visualization module, measured database and model database; among them, The error module is used to obtain the position error of the airborne moving platform, the missed detection error of the sensor and the angle measurement error of the sensor; A model database is used to generate the motion data of the target, the motion data of the airborne moving platform and the observation data of each sensor based on the kinematic model of the detected target, the kinematic model of the airborne moving platform and the two-dimensional imaging model of the sensor, and in combination with the position error of the airborne moving platform, the missed detection error of the sensor and the angle measurement error of the sensor; A data processing module, used for processing target infrared radiation intensity data of the detected target and the infrared characteristic image of the sensor imaging plane under typical working conditions to generate infrared radiation characteristic data of the detected target and a target infrared characteristic grayscale image; The measured database is used to store the infrared radiation characteristic data of the detected target and the target infrared characteristic grayscale image, and generate the infrared characteristic data and image of each target by combining the detection distance of each target relative to each sensor, and the detection pitch and azimuth of each target relative to each sensor; The data storage and visualization module is used to manage and store the infrared characteristic data and images of each target, and analyze the infrared characteristic data and images of each target to generate the motion trajectory of each target and each sensor in the network coordinate system.

[0022] Through the above-mentioned dynamic platform multi-infrared sensor collaborative multi-target perception digital simulation platform, the working process of distributed multi-infrared sensors can be fully simulated in a virtual environment, providing accurate dynamic platform multi-infrared sensor multi-target perception simulation, so that researchers can repeatedly test and verify the distributed multi-infrared sensor collaborative perception platform under different operating conditions and environments to ensure the scientificity and feasibility of the design scheme and reduce the uncontrollable factors and risks in real tests. Through this platform, developers can systematically adjust parameters such as sensor configuration, perception algorithm and data fusion strategy, and accurately evaluate the impact of different configurations on distributed multi-infrared sensor collaborative algorithms and schemes; this enables the platform to optimize the overall performance and reliability of the platform in the early stage, avoiding the time and cost waste caused by repeated hardware adjustments in traditional experiments. By providing a flexible and customizable simulation environment, technical developers can quickly iterate and test new algorithms, new functions and new designs. Without relying on physical hardware, it can efficiently simulate a variety of working conditions in actual application scenarios, thereby accelerating the research and development and application deployment of new technologies and improving development efficiency.

[0023] Next, we will refer to Figures 1 to 11 The various parts of the above-mentioned moving platform multi-infrared sensor collaborative multi-target perception digital simulation platform in this example implementation are explained in more detail.

[0024] In one embodiment, the infrared characteristic data and image library of the target based on the measured data (i.e., the measured database) are used to provide the infrared radiation characteristic data of the detected target and the infrared characteristic image of the sensor imaging plane of the detected target with high credibility for the operation of the simulation system (i.e., the digital simulation platform for the coordinated multi-target perception of multiple infrared sensors on the moving platform). The infrared radiation characteristic data of the detected target is obtained by simulating the detection of the target model recognized by experts or by actually detecting the physical target. The infrared radiation characteristic data is the target infrared radiation intensity data obtained by detecting the target at different pitch angles and azimuth angles by the infrared sensor under several typical working conditions (a certain flight altitude, a certain flight speed); the infrared characteristic image of the sensor imaging plane of the detected target is obtained by simulating the detection of the target model recognized by experts or by actually detecting the physical target. It is the target infrared characteristic grayscale image obtained by detecting the target at different pitch angles and azimuth angles by the infrared sensor, which represents the two-dimensional imaging of the detected target in the infrared sensor imaging plane, and is a visual representation of the infrared radiation intensity of the detected target surface. The larger the grayscale value in the image, the greater the infrared radiation intensity representing the target area.

[0025] In one embodiment, the infrared characteristic data of the target based on the measured data and the infrared radiation characteristic data of the detected target under several typical working conditions in the image library and the infrared characteristic images of the detected target sensor imaging plane are processed by a data processing module to obtain the target infrared characteristic data and image library based on the measured data for all working conditions, all positions and all directions.

[0026] The target infrared characteristic data interpolation module (i.e., target infrared characteristic data interpolation submodule) in the data processing module interpolates the target infrared radiation intensity data obtained by detecting the target at different pitch angles and azimuth angles through the infrared sensor under several typical working conditions (a certain flight altitude, a certain flight speed). The interpolation processing method is as follows: through the multivariate spline interpolation method, firstly, the azimuth angle in each pitch angle file is interpolated by a single variable cubic spline; secondly, after the interpolation in the azimuth direction is completed, the interpolation data of different pitch angles is interpolated by a bivariate spline; finally, after the interpolation data of different pitch angles is interpolated by a bivariate spline, the trivariate spline interpolation under multiple working states is completed. Through the above steps, the target infrared characteristic data based on the measured data under all working conditions and all directions are obtained.

[0027] The infrared characteristic image of the sensor imaging plane of the detected target obtained by detecting the target at different pitch angles and azimuth angles by the infrared sensor is scaled by the target designated distance image generation module (i.e., the target designated distance image generation module) in the data processing module. The scaling method is as follows: the actual distance of the detected target from the sensor in the currently obtained infrared characteristic image is calculated by combining the resolution occupied by the target imaging in the currently obtained image, the actual size of the detected target and the infrared sensor parameters, and then the image scaling ratio at different distances is calculated in combination with the different distances of the target from the sensor. With the infrared characteristic image of the sensor imaging plane of the detected target as the reference, combined with the calculated image scaling ratio at different distances, the infrared characteristic image of the detected target in the infrared sensor imaging plane at different distances, different positions, and different detection pitch and azimuth angles is generated. The above-mentioned scaling process obtains the target infrared characteristic grayscale image of the target in all working conditions, all positions, and all directions.

[0028] The target infrared characteristic data and image library based on measured data are obtained by processing the infrared radiation characteristic data of the detected target under several typical working conditions and the infrared characteristic image of the detected target sensor imaging plane through the data processing module.

[0029] In one embodiment, the target platform motion and sensor platform motion based on the model (i.e., the model database) are used to generate the motion data of the target, the motion data of the airborne moving platform equipped with the sensor, and the observation data of the sensor, and the database is combined with an error module so that the generated data simulates the actual conditions as much as possible. Among them, the motion data of the target is obtained based on the kinematic model of the detected target, the motion data of the airborne moving platform equipped with the sensor is obtained based on the kinematic model of the airborne moving platform combined with the position error of the airborne moving platform, and the observation data of the sensor is obtained based on the two-dimensional imaging model of the airborne sensor combined with the missed detection error of the sensor and the angle measurement error of the sensor.

[0030] The error module is composed of the position error of the airborne moving platform, the missed detection error of the sensor, and the angle measurement error of the sensor. The position error of the airborne moving platform follows the Gaussian distribution and is added to the three-dimensional position coordinates of the airborne moving platform in the form of deviation; the angle measurement error of the sensor follows the Gaussian distribution, and any frame in the two-dimensional imaging process of the airborne sensor generates a random error and is added to the transformation matrix from the multi-sensor network coordinate system to the sensor coordinate system.

[0031] In one embodiment, the time alignment module is used to synchronize the timestamps of multiple sensor nodes. The simulation system takes into account that multiple sensor nodes may cause observation errors due to time asynchrony when observing a target. Therefore, the time alignment module is used to perform time synchronization alignment on the target data observed by multiple sensor nodes, including time hard synchronization and time soft synchronization methods. The time hard synchronization method is as follows: by implementing data acquisition in the same simulation cycle; the time soft synchronization method is as follows: by interpolating the pixel coordinate data of the known target in the sensor, and then interpolating the continuous motion trajectory of the target, and calculating the pixel coordinate data of the target in different sensor imaging planes at the same timestamp in combination with the interpolated target continuous motion trajectory, thereby achieving time soft synchronization and data alignment.

[0032] In one embodiment, the spatial registration module is used to unify multiple sensor nodes into a network coordinate system. The position of the sensor itself in the multi-sensor system is usually represented by the longitude and latitude coordinates obtained by the airborne navigation system of the airborne platform. The spatial registration module is used to ensure that the images collected by different sensors are accurately aligned in the same coordinate system, thereby achieving the accuracy of multi-sensor information fusion and target recognition. The spatial registration process is as follows: geodetic coordinate system to earth-centered earth-fixed coordinate system, and earth-centered earth-fixed coordinate system to multi-sensor network coordinate system.

[0033] In one embodiment, the data storage and visualization module is used to effectively manage, store and display large amounts of text data and images collected by multiple sensors, while providing a user-friendly interface to view and analyze these data, including a data storage module (i.e., a data storage submodule) and a visualization module (i.e., a visualization submodule).

[0034] The data storage module can access different types of sensors and target motion data and sensor observation data, and then classify and store the data according to the sensor ID number and the target ID number. The classification and storage method is as follows: All sensor and target data are classified and stored as .csv files according to the ID number of the airborne dynamic platform and the ID number of the target. The data storage file of the target is named target+target number.csv file, the number of targets is equal to the total number of the final target files, and the data stored in the storage file of each target is the motion information of the target, including the position coordinates and flight direction of the target in each frame in the multi-sensor network coordinate system. A line in the file represents a frame of data, and the time interval of each frame of data can be set by modifying the input of the simulation system; the data storage file of the airborne dynamic platform is named platform+airborne dynamic platform number.csv file, the number of airborne dynamic platforms is equal to the total number of the final airborne dynamic platform data storage files, and the data stored in the storage file of each airborne dynamic platform is the motion information of the platform, the flight direction of the platform, the direction of the sensor, the pixel coordinates of each target in the sensor, and the infrared radiation intensity information of each target. A line in the file represents a frame of data, and the time interval of each frame of data can be set by modifying the input of the simulation system.

[0035] The visualization module displays the motion trajectory of the target and the sensor platform in the networked coordinate system based on 3D graphics to display the formation of the target and the sensor airborne platform, and displays the imaging effect of the target in the sensor imaging plane based on 2D graphics. The development method of the visualization module is as follows: by reading the target+target number.csv file that stores the target data and the motion data of the target and the airborne moving platform in the platform+airborne moving platform number.csv file that stores the airborne moving platform data, the motion trajectory of the target and the airborne moving platform in the multi-sensor networked coordinate system is drawn. By reading the target position coordinate information, target flight direction, platform position coordinate information, platform flight direction, sensor direction, and pixel coordinate information of each target in the target+target number.csv file storing target data and the platform+airborne dynamic platform number.csv file storing airborne dynamic platform data, the detection pitch azimuth angle of the target relative to the sensor and the distance of the target from the sensor are calculated by combining the platform direction, sensor direction and target flight direction with the spatial registration module, and then combining the target infrared characteristic data based on the measured data and the image library as well as the pixel coordinates of each target in the sensor, the infrared characteristic image of each target is scaled and imaged at the specified pixel coordinate position of the two-dimensional imaging plane of the sensor.

[0036] In one embodiment, a data processing module, an error module, a data storage and visualization module, a time registration module, a space registration module, a measured database, a model database and a configuration module constitute the digital simulation platform for the collaborative multi-target perception of a moving platform multi-infrared sensor of the present application. After the simulation system is developed, the input of the simulation system is configured to simulate different simulation environments for multi-target perception of moving platforms multi-infrared sensors. The input of the simulation system includes target items, platform items, error items and simulation setting items, wherein the target items include the number and number of targets, the flight direction of each target, the initial position of each target, and the motion model of each target; the platform items include: the number and number of platforms, the flight direction of each platform, the initial position of each platform, the motion model of each platform, and the rotation model of each sensor; the error items include platform position error, sensor angle measurement error, and sensor missed detection error; the simulation setting items include the total simulation time, the simulation step length, and the data saving path.

[0037] After configuring the inputs of different simulation systems, based on the model-based target and sensor platform motion and model database in the simulation system, combined with the time registration module, space registration module and error module, the motion data of each platform and each target and the observation data of each sensor are generated; based on the target infrared characteristic data and image library based on measured data in the simulation system, combined with the detection distance of each target relative to each sensor and the detection pitch and azimuth of each target relative to each sensor, the infrared characteristic data and image of each target are generated; the motion data of each platform and each target, the observation data of each sensor and the infrared characteristic data and image of each target are classified and stored through the data storage and visualization module, and the three-dimensional spatial motion trajectory of multiple platforms and multiple targets and the two-dimensional multi-target infrared imaging of each sensor are drawn at the same time.

[0038] In a specific embodiment, a digital simulation platform for multi-infrared sensors on a moving platform to coordinate multi-target perception is provided. Figure 1In the structure diagram of the digital simulation platform for multi-infrared sensors of the moving platform for cooperative multi-target perception shown in the figure, the whole simulation system consists of a database layer and a module layer. The database layer contains two databases, namely, a target infrared characteristic data and image library based on measured data and a target, sensor platform motion and observation database based on the model; wherein, the target infrared characteristic data and image library based on measured data are constructed by combining the infrared radiation characteristic data of the detected target under several typical working conditions and the infrared characteristic image of the sensor imaging plane of the detected target with the target infrared characteristic data interpolation module and the target specified distance image generation module in the data processing module respectively; wherein, the target, sensor platform motion and observation database based on the model is constructed by combining the kinematic model of the airborne moving platform, the kinematic model of the detected target and the two-dimensional imaging model of the airborne sensor and taking into account the position error of the airborne moving platform in the error module, the missed detection error of the sensor and the angle measurement error of the sensor; in addition to the above two databases and two modules, the simulation system also includes a time registration module and a space registration module, wherein the time registration module includes time soft synchronization and time hard synchronization, and the space registration module is a coordinate transformation process from the geodetic coordinate system to the multi-sensor network coordinate system. Combining all the above databases and modules, the text data generated by the full digital simulation experiment in the dynamic platform multi-infrared sensor multi-target perception simulation environment is read by the data storage and visualization module, and then the data is classified, stored and visualized. The following will take an aircraft target as an example to introduce the entire process of building a dynamic platform multi-infrared sensor collaborative multi-target perception digital simulation platform for the aircraft target.

[0039] First, the target infrared characteristic data and image library are constructed based on measured data. For aircraft targets, the infrared radiation characteristic data and infrared characteristic images of the sensor imaging plane under several typical working conditions are obtained. The above data and images are obtained by simulating the detection of aircraft target models recognized by experts with infrared sensors or by actually detecting physical aircraft targets with physical sensors. Since this process only considers how to combine the infrared radiation characteristic data of aircraft targets under several typical working conditions and the infrared characteristic images of the sensor imaging plane to establish a data and image library, it is assumed that the infrared radiation characteristic data and infrared characteristic images of the sensor imaging plane under several typical working conditions have been obtained by detecting physical aircraft targets.

[0040] The infrared radiation characteristic data of the detected aircraft target obtained is the infrared radiation intensity data of the aircraft target obtained by changing the detection pitch angle and azimuth of the sensor relative to the detected aircraft target to detect the target under several different working conditions (different flight altitudes and flight speeds of the aircraft target) at a specified detection distance. The detection pitch angle range is from negative 90 degrees to positive 90 degrees. A positive detection pitch angle indicates that the sensor observes the aircraft target obliquely above the target, and a negative detection pitch angle indicates that the sensor observes the aircraft target obliquely below the target; the detection azimuth indicates that the sensor observes the target at different positions around the target circumference. The detection azimuth range is from 0 degrees to 360 degrees. When the detection azimuth is 0 degrees and 360 degrees, it indicates that the sensor observes the target in the head-on direction of the target, and when the detection azimuth is 180 degrees, it indicates that the sensor observes the target in the tail direction of the aircraft target. By setting different detection azimuths and detection pitch angle intervals, the infrared radiation intensity data of the aircraft target at different detection pitch angles and azimuths under certain flight altitudes and flight speeds can be obtained.

[0041] Assume that the flight speed of the aircraft target under one condition is 100m / s and the flight altitude is 5000m. When the detection distance between the sensor and the aircraft target is 500m, the detection pitch angle of the sensor relative to the target is changed at intervals of 2 degrees, and the detection azimuth angle of the sensor relative to the target is changed at intervals of 5 degrees. Under this setting, a total of 6497 omnidirectional infrared radiation intensity data of the target under this condition can be obtained. Similar to the above settings, only by adjusting the flight conditions of the aircraft target, the omnidirectional infrared radiation characteristic data of the target under different conditions can be obtained. Taking the target conditions of a flight altitude of 2km to 6km and a flight speed of 50m / s to 150m / s as an example, at a flight altitude of 1km and a flight speed of 20m / s, a total of 162,425 infrared radiation characteristic data of the aircraft target under all the above 25 conditions can be obtained, such as Figure 2 The figure shows the infrared radiation characteristic data of an aircraft target when an infrared sensor is used to detect an aircraft target at a fixed pitch angle of 30 degrees in different azimuths. The polar coordinates in the figure are the detection azimuth of the sensor relative to the target, ranging from 0 to 360 degrees. The curve represents the change law of the infrared radiation intensity of the aircraft target at different detection azimuths under specified working conditions and a specified detection pitch angle of 30 degrees.

[0042] Combined with the above data, the target infrared characteristic data interpolation module in the data processing module performs multivariate spline interpolation on the data to obtain more detailed infrared radiation characteristic data of the aircraft target in all working conditions and all directions under the conditions of flight altitude of 2km to 6km and flight speed of 50m / s to 150m / s. The specific data interval after interpolation can be freely configured, for example, it can be set to 1 interval of target flight altitude of 500m, 1 interval of target flight speed of 10m / s, 1 interval of the detection pitch angle of the sensor relative to the target in the range of negative 90 degrees to positive 90 degrees, and 1 interval of the detection azimuth angle of the sensor relative to the target in the range of 0 degrees to 360 degrees. Data interpolation processing simplifies the difficulty of obtaining the data of the above targets under the conditions of flight altitude of 2km to 6km and flight speed of 50m / s to 150m / s. Only by obtaining the infrared characteristic data of the aircraft target under several working conditions, a more detailed infrared radiation characteristic database of the aircraft target in all working conditions and all directions can be constructed through the interpolation method. The interpolation method is as follows: The first step is to perform univariate cubic spline interpolation on the azimuth angle based on the azimuth angle data. And its corresponding infrared radiation intensity , through the cubic spline function Interpolate the data. The cubic spline function is as follows:

[0043] in, Indicates Azimuth angle, They are the first coefficient, the second coefficient, the third coefficient and the fourth coefficient to be solved respectively, and the four coefficients are solved by the following three conditions.

[0044] Interpolation condition: The spline function passes through the data points at the nodes, that is:

[0045] in, Indicates Azimuth angle, Indicates The infrared radiation intensity of the aircraft target corresponding to the azimuth angle.

[0046] Continuity conditions: The spline function and its first and second order derivatives are continuous at the nodes, that is:

[0047] in, is the first-order derivative of the spline function, is the second-order derivative of the spline function.

[0048] Boundary conditions: The second-order derivative is zero at the boundary, that is:

[0049] in, is the total number of data points.

[0050] The second step is to perform bivariate spline interpolation on the interpolation data of different elevation angles based on the interpolation in the azimuth direction. and azimuth Infrared radiation intensity , through the bivariate spline function Interpolation is performed, and the interpolation formula is as follows:

[0051] in, is the spline basis function, is the interpolation coefficient, is the number of pitch angles, is the number of azimuth angles. The coefficients are also determined by the following three conditions.

[0052] Interpolation condition: The spline function passes through the data points at the nodes, that is:

[0053] in, Indicates Pitch angle, Indicates Azimuth angle, Indicates The pitch angle and The infrared radiation intensity of the aircraft target corresponding to the azimuth angle.

[0054] Continuity conditions: The spline function and its first-order and second-order partial derivatives must be continuous at each node, that is: For continuity in the pitch angle direction:

[0055] For continuity in the azimuthal direction:

[0056] Continuity in blending direction:

[0057] Boundary conditions:

[0058] The third step is to complete the data interpolation under multiple working states based on the bivariate spline interpolation of the interpolation data of different pitch angles. , Pitch angle and azimuth Infrared radiation intensity , through the three-dimensional variable spline function Interpolation is performed, and the interpolation formula is as follows:

[0059] in, is the spline basis function, is the interpolation coefficient, is the number of working states, is the number of pitch angles, is the number of azimuth angles. The interpolation coefficients are determined by the following three conditions.

[0060] Interpolation conditions:

[0061] in, Indicates The working condition of an aircraft target, Indicates Pitch angle, Indicates Azimuth angle, Indicates The working conditions of the aircraft target, The pitch angle and The infrared radiation intensity of the aircraft target corresponding to the azimuth angle.

[0062] Continuity conditions: For continuity in the flight state direction:

[0063] For continuity in the pitch angle direction:

[0064] For continuity in the azimuthal direction:

[0065] Boundary conditions:

[0066] Through the above steps, the infrared radiation characteristic data under several existing working conditions can be combined to interpolate more detailed infrared radiation characteristic data of aircraft targets under all working conditions and in all directions, thereby constructing a full-condition and all-directional infrared radiation characteristic database of the detected aircraft targets.

[0067] The obtained infrared image of the sensor imaging plane of the detected aircraft target is the infrared characteristic grayscale image of the aircraft target obtained by detecting the target by changing the detection pitch angle and azimuth of the sensor relative to the detected aircraft target at a relatively close detection distance. The image is a two-dimensional imaging of the detected aircraft target in the infrared sensor imaging plane, and is a visual representation of the infrared radiation intensity of the detection target surface. The greater the grayscale value in the image, the greater the infrared radiation intensity representing the aircraft target area. The detection pitch angle range is from negative 90 degrees to positive 90 degrees. A positive detection pitch angle indicates that the sensor observes the aircraft target obliquely above the target, and a negative detection pitch angle indicates that the sensor observes the aircraft target obliquely below the target; the detection azimuth indicates that the sensor observes the target at different positions around the target circumference. The detection azimuth range is from 0 degrees to 360 degrees. When the detection azimuth is 0 degrees and 360 degrees, it means that the sensor observes the target in the head-on direction of the target. When the detection azimuth is 180 degrees, it means that the sensor observes the target in the tail direction of the aircraft target. In close-range scenarios (detection distance between unknown target and sensor), by setting different detection azimuth and detection pitch angle intervals, it is possible to obtain infrared characteristic grayscale images of the target at different detection pitch and azimuth angles under certain flight altitude and flight speed conditions. Assume that the flight speed of an aircraft target under one flight condition is 100m / s and the flight altitude is 5000m. When the detection distance between the sensor and the aircraft target is relatively close (unknown), the detection pitch angle of the sensor relative to the target is changed at intervals of 2 degrees, and the detection azimuth angle of the sensor relative to the target is changed at intervals of 10 degrees. Under this setting, a total of 3293 omnidirectional infrared characteristic grayscale images of the target under this condition can be obtained. The above 3293 images are the infrared characteristic images of the sensor imaging plane of the aircraft target. Figure 3 Shown is a schematic diagram of an infrared characteristic grayscale image of an aircraft target when detected at a certain pitch angle and azimuth angle, which is extracted from the infrared characteristic image of the sensor imaging plane, including infrared characteristic grayscale images when detected at azimuth angles of 0 degrees to 360 degrees (10 degree intervals) at a pitch angle of -20 degrees.

[0068] After obtaining the infrared characteristic image of the sensor imaging plane of the aircraft target under the above working condition, the target infrared characteristic grayscale image at the specified target distance from the sensor at a certain moment is calculated by combining the following scaling method in the target specified distance image generation module.

[0069] Assume that the resolution size length of the aircraft target image in the infrared characteristic image of the aircraft target sensor imaging plane obtained under the above working conditions is , width is , height is ; The actual length of the detected aircraft target is , width is , Gao Wei . The similar triangle theory of transforming the sensor coordinate system to the image coordinate system can be obtained. The size of the aircraft target image and the actual size obey the following formula:

[0070] in That is, the distance of the target from the sensor. Represents the focal length of the sensor. Since the horizontal and vertical pixel sizes of the sensor are equal, When detecting the aircraft target, the image of the aircraft target in the sensor imaging plane occupies the resolution size length ,width ,high The number of pixels that its image occupies in the imaging plane Obtained, obey the following formula:

[0071] In the above formula, represents the pixel size of the sensor, Indicates the actual number of pixels occupied by the aircraft target in the width direction in the current image. Indicates the actual number of pixels occupied by the aircraft target in the length direction of the current image. Indicates the actual number of pixels occupied by the aircraft target in the height direction of the current image. The distance between the target and the sensor It can be calculated by the following formula:

[0072] Combined with the flight direction of the airborne moving platform, the actual direction of the sensor and the flight direction of the aircraft target, the detection azimuth and pitch angle of the target relative to the sensor at a certain moment are calculated, and the existing detection pitch angle and azimuth in the infrared characteristic image of the sensor imaging plane of the detected aircraft target are searched. The posture of the aircraft target in the infrared sensor at the current moment is determined by the following judgment principle:

[0073] in, Indicates the pitch angle of the current aircraft target relative to the sensor. represents the pitch angle already in the near-field image library, Indicates the azimuth of the current aircraft target relative to the sensor. Indicates the azimuth angle available in the near-field image library. If the pitch and azimuth angle detection intervals in the near-field infrared image library are refined to smaller degrees, the specific 1° and 5° in the judgment criteria can be adjusted to other degrees.

[0074] After determining the current posture of the aircraft target in the infrared sensor, the distance between the target and the sensor in the near-field image library is , combined with the actual distance between the target and the sensor at the current moment Calculate image scaling , the scaling ratio calculation formula is as follows:

[0075] After obtaining the above scaling ratio, the actual distance between the target and the sensor at the current moment is determined by the following formula: The actual imaging resolution of the target under the aircraft in the sensor is calculated as follows:

[0076] In the formula, Indicates that the actual distance between the aircraft target and the sensor at the current moment is The lateral resolution of the aircraft target in the sensor imaging plane is Indicates that the actual distance between the aircraft target and the sensor at the current moment is The vertical resolution of the aircraft target in the sensor imaging plane is Indicates the distance between the aircraft target and the sensor is The lateral resolution of the aircraft target in the sensor imaging plane is Indicates the distance between the aircraft target and the sensor is The lateral resolution of the aircraft target in the sensor imaging plane.

[0077] Finally, combined with the above , , , By scaling the infrared characteristic image of the sensor imaging plane of the detected target, a grayscale image of the infrared characteristic of the aircraft target at a specified target distance from the sensor at a certain moment can be generated. Based on the above method, a full-range and omnidirectional infrared characteristic image library of the sensor imaging plane of the detected aircraft target is constructed.

[0078] The full-condition, all-directional infrared radiation characteristics database of the detected aircraft target and the full-distance, all-directional sensor imaging plane infrared characteristics image library of the detected aircraft target constitute the aircraft target infrared characteristics data and image library based on measured data. The above process describes the method for constructing the target infrared characteristics data and image library based on measured data using the aircraft target as an example.

[0079] The second is the construction of the target and sensor platform motion and observation database based on the model. The relevant models include the dynamic model of the detected target, the dynamic model of the airborne moving platform equipped with the sensor, and the two-dimensional imaging model of the airborne sensor. The dynamic model of the airborne moving platform equipped with the sensor needs to be combined with the position error of the airborne moving platform in the error module; the sensor's missed detection error and angle measurement error need to be considered when constructing the two-dimensional imaging model of the airborne sensor. The following will explain each model and the construction method of the target and sensor platform motion and observation database based on the model.

[0080] ① Dynamic model of the detected target. In this embodiment, the detected target is an aircraft target, and its kinematic model can be selected according to different flight phases and mission requirements, including uniform linear motion, uniform accelerated motion, uniform circular motion, parabolic motion, spiral trajectory, rotational motion, high-speed flight and other models. It is assumed that the motion model of the aircraft target is along the multi-sensor network coordinate system. The kinematic model of the detected aircraft target is as follows:

[0081] in, They respectively represent the position coordinates of the detected aircraft target in the multi-sensor network coordinate system, For time, It is a fixed positive value greater than zero, indicating that the aircraft target is along the multi-sensor network coordinate system. Speed ​​in the direction parallel to the axis.

[0082] ② Dynamic model of the airborne mobile platform equipped with sensors. Assuming that the airborne mobile platform in this embodiment is an unmanned mobile platform, its commonly used kinematic models include uniform linear motion, uniformly accelerated motion, three-dimensional space motion, rotational motion (such as Euler angles and quaternions) and flight control models subject to physical constraints. The following assumes that the motion model of the unmanned mobile platform equipped with sensors is along the multi-sensor network coordinate system. The negative direction parallel to the axis is uniformly moving in a straight line, and the kinematic model of the unmanned aerial vehicle platform equipped with the sensor is as follows:

[0083] in, They represent the position coordinates of the UAV platform in the multi-sensor network coordinate system. For time, It is a fixed negative value less than zero, indicating that the UAV platform is moving along the multi-sensor network coordinate system. Speed ​​in the direction parallel to the axis.

[0084] When constructing the dynamic model of the airborne moving platform equipped with sensors, the position error of the airborne moving platform is considered, and its position error is added to the position coordinates of the airborne moving platform in the multi-sensor network coordinate system at each moment with a fixed deviation. In this embodiment, the position error of the unmanned moving platform is added as follows:

[0085] in,

[0086] , where the standard deviation , Indicates the position error of the unmanned platform. The meaning of adding the position error of the unmanned platform is that the random error generated has a 99.7% probability of appearing in the specified position error range. It is worth mentioning that the position error of the airborne moving platform is a random error generated at the initial moment. At any subsequent moment, the above error remains unchanged. Therefore, even if the error is added, the motion law of the airborne moving platform still conforms to its own kinematic model.

[0087] ③ The two-dimensional imaging model of the airborne sensor indicates that the spatial target point located in the multi-sensor network coordinate system is transformed into the infrared sensor coordinate system through rotation and translation, and then the pixel coordinate position of the target point in the image coordinate system is obtained through the sensor internal parameters. The position coordinates are , its pixel coordinate position in the sensor imaging plane The calculation formula is as follows:

[0088] In the above formula, the subscript is left subscript Indicates the multi-sensor network coordinate system, with the lower left represents the image coordinate system of the sensor, Represents aircraft targets The horizontal and vertical pixel coordinates in the sensor imaging plane, Represents the target in the sensor coordinate system of Axis coordinates. represents the focal length of the sensor, represents the pixel size of the sensor, represents the center coordinate of the sensor imaging plane, represents the translation of the sensor, It is the rotation matrix of the sensor coordinate system with the added noise representing the angle measurement error relative to the multi-sensor network coordinate system.

[0089] As mentioned above, The rotation matrix of the sensor coordinate system with the added noise representing the angle measurement error relative to the multi-sensor network coordinate system is obtained. This step is the method of adding the angle measurement error of the sensor. The specific adding method is as follows: The pixel coordinate position in the sensor imaging plane is obtained by transforming the target position in space through translation and coordinate projection. In the process of coordinate projection transformation, the rotation matrix of the sensor coordinate system relative to the multi-sensor network coordinate system is Add the angle measurement error in the following way:

[0090] in,

[0091] in Represent the pitch, yaw, and roll angles of the sensor respectively. Respectively represent the random errors of the sensor in pitch, yaw and roll directions, , where the standard deviation , Indicates the angle measurement error of the infrared sensor, which means that the noise error generated has a 99.7% probability of appearing in the specified angle measurement error Within the range.

[0092] In addition, the missed detection error of the sensor must also be considered. The missed detection error of the infrared sensor follows a 0, 1 distribution, where the probability of being 1 is 90% and the probability of being 0 is 10%, indicating that there is a 10% probability of not being able to obtain the two-dimensional pixel coordinates projected from the spatial target coordinate position to the sensor imaging plane.

[0093] The model-based target and sensor platform motion and observation database is constructed through the above process. Various types of errors are taken into account during the construction process so that the data generated by the simulation system based on the database is as close to the actual situation as possible.

[0094] Secondly, the time alignment module includes hard time synchronization and soft time synchronization, wherein hard time synchronization refers to directly physically connecting the clocks of multiple sensors through hardware or synchronizing through a dedicated synchronization signal source. In the digital simulation platform for collaborative multi-target perception of multiple infrared sensors on a moving platform proposed in this embodiment, hard synchronization is achieved by acquiring data in real time in the same simulation cycle. In addition, the time alignment module also includes a soft time synchronization method, that is, adjusting or correcting the timestamp of the sensor data through an algorithm to achieve time synchronization between multiple sensors. This embodiment achieves soft time synchronization through an interpolation method. The specific method is as follows. Figure 4As shown, the moving target in space is observed by sensor 1 and sensor 2. From the first frame to the fifth frame, due to the time asynchrony, the aircraft target observed by the sensor The positions in space do not overlap. By combining the pixel coordinate data of the known aircraft target in the sensor for interpolation, the continuous observation motion trajectory of the aircraft target is interpolated, and then the pixel coordinate data of the target in different sensor imaging planes at the same time stamp is calculated by combining the interpolated continuous observation motion trajectory of the aircraft target, thereby achieving time soft synchronization and data alignment. The specific interpolation method is as follows: Assuming that at time 0, the target The pixel coordinates in the sensor imaging plane are , thereafter The pixel coordinates of the target observed at each moment are recorded as , considering the above pixel coordinates as the independent variable time The dependent variable is at the time stamp of the reference sensor. The time is divided into intervals, and then use the cubic spline piecewise interpolation function To interpolate the continuous motion trajectory of the aircraft target in the imaging plane, the interpolation function is as follows:

[0095] In the formula, Indicates In the interval The pixel coordinates of the aircraft target at time, Respectively represent The unknown coefficients of the cubic spline interpolation function in intervals, each interval has 4 unknown coefficients, for interval, requiring solution unknown coefficients. At the same time, the obtained piecewise interpolation function in each spline interval It is continuous and smooth at each known value point, satisfying:

[0096] Combine the cubic spline interpolation function equation to get the aircraft target Continuous track in sensor 1 imaging plane , its trajectory equation is:

[0097] In the formula In the imaging plane of sensor 1 The basis functions for each spline interval within the trajectory, The imaging plane of sensor 1 Spline coefficients for each spline interval within the trajectory.

[0098] Similarly, combining the cubic spline interpolation function equation can get the target Continuous track in sensor 2 imaging plane , its trajectory equation is:

[0099] In the formula In the imaging plane of sensor 2 The basis functions for each spline interval within the trajectory, In the imaging plane of sensor 2 Spline coefficients for each spline interval within the trajectory.

[0100] The above method is used to perform time soft synchronization on the data observed in the two sensors. If we want to obtain time The pixel coordinate position data of the target in different sensors are processed or fused, and the motion trajectory equation can be continuously observed through the above two targets and Bring in the moment , after completing time soft synchronization The pixel coordinate position data of the target in different sensors. After soft synchronization, the data points observed by the two sensors eliminate the observation deviation, and the observed points are all in space. point.

[0101] Secondly, the process of the spatial registration module is as follows Figure 5 As shown, the obtained longitude and latitude coordinates are first transformed from the geodetic coordinate system to the geocentric earth-fixed coordinate system through coordinate transformation, the origin of which is the center of the earth, and then transformed to the multi-sensor network coordinate system through coordinate transformation and translation to obtain the coordinates of the network coordinate system, the origin of which is the zero time position of a certain sensor. The purpose of the above process is to ensure that spatial data from different sensors, at different times or from different perspectives can be accurately compared, fused or analyzed to reduce errors caused by differences in perspectives of different sensors. Combined with Figure 5 The specific process of describing spatial registration is as follows: In actual situations, the location information of the UAV platform and the sensor is usually obtained by the airborne navigation system. The coordinates are expressed in the form of coordinates in the geodetic coordinate system. This embodiment assumes that the coordinates of the center of mass of the UAV platform coincide with the coordinates of the optical center of the sensor, and the coordinates of the sensor position are transformed into coordinates in the geodetic fixed system by the following coordinate transformation method:

[0102] In the formula, superscript Indicates a moment in time, the subscript represents the Earth-centered Earth-fixed coordinate system, For the The longitude, latitude, and altitude coordinates of the center of mass of the UAV platform at this moment. are the coefficients of the ellipse expression of the meridian ellipse plane at the location of the UAV mobile platform, representing the major axis radius and minor axis radius of the ellipse respectively. For the The coordinates of the center of mass of the UAV platform at the earth's center fixed system at the moment, To express the coefficients through an ellipse is the first parameter solved, and N is the second parameter solved by the above parameter combination.

[0103] After obtaining the coordinates of the multi-sensor in the geocentric earth-fixed coordinate system, all sensors are unified into the same multi-sensor network coordinate system by unifying the coordinate origin. The unified coordinate origin is the coordinate point in the geocentric earth-fixed system where the initial position height of a sensor in the multi-sensor system is 0 at time zero.

[0104] Assume the origin of the network coordinate system is , then the remaining sensors In the The position at any moment in the multi-sensor network coordinate system It can be obtained by the following formula:

[0105] In the formula, For the The sensor in The latitude and longitude coordinates of the time, For the The sensor in The Earth-fixed coordinates of the time. The multi-sensor network coordinate system obeys the right-hand rule. The axis points east, The axis points north, The axis points upwards, perpendicular to the ground.

[0106] Finally, the text data generated by the full digital simulation experiment is completed in the multi-infrared sensor multi-target perception simulation environment of the moving platform by combining all the above databases and modules. The data storage and visualization module is used to effectively manage, store and display the large amount of text data and images collected by multiple sensors, and provide a user-friendly interface to view and analyze these data. It includes a data storage module and a visualization module, in which the data storage module can access different types of sensor data, and can also classify and store data according to the sensor ID and target ID; the visualization module displays the motion trajectory of the target and sensor platform in the network coordinate system based on 3D graphics to display the target and sensor airborne moving platform formation, and displays the infrared imaging effect of multiple targets in the sensor imaging plane based on 2D graphics.

[0107] The data storage module is based on the logic written in Python language, and the data of all sensors and targets are classified and stored as .csv files according to the airborne dynamic platform ID number and the target ID number.

[0108] The target data storage file is named target+target number.csv file, the number of targets is equal to the total number of target data storage files, and the data stored in the storage file of each target is the motion information of the target, including the position coordinates and flight direction of the target in each frame in the multi-sensor network coordinate system. The data storage format is: simulation time, target X-axis position coordinates, target Y-axis position coordinates, target Z-axis position coordinates, flight yaw angle, flight pitch angle, and flight roll angle, where the unit of the position coordinate is meter, and the unit of the angle is degree. Each line in the target+target number.csv file represents a frame of data, and the time interval of each frame of data can be set by modifying the input of the simulation system.

[0109] The data storage file of the airborne dynamic platform is named platform+airborne dynamic platform number.csv file. The number of airborne dynamic platforms is equal to the total number of the final airborne dynamic platform data storage files. The data stored in the storage file of each airborne dynamic platform is the motion information of the airborne dynamic platform, the flight direction of the airborne dynamic platform, the direction of the sensor, the pixel coordinates of each target in the sensor, and the infrared radiation intensity information of each target. The data storage format is: simulation time, X-axis position coordinates of the airborne dynamic platform, Y-axis position coordinates of the airborne dynamic platform, Z-axis position coordinates of the airborne dynamic platform, flight yaw angle of the airborne dynamic platform, and flight direction of the airborne dynamic platform. Pitch angle, flight roll angle of the airborne moving platform, pitch angle of the sensor, yaw angle of the sensor, horizontal coordinate of target 1 pixel in the sensor, vertical coordinate of target 1 pixel in the sensor, infrared radiation intensity of target 1 in the sensor, horizontal coordinate of target 2 pixel in the sensor, vertical coordinate of target 2 pixel in the sensor, infrared radiation intensity of target 2 in the sensor, ..., horizontal coordinate of target n pixel in the sensor, vertical coordinate of target n pixel in the sensor, infrared radiation intensity of target n in the sensor, n represents the total number of targets, where the unit of position coordinate is meter, the unit of angle is degree, the unit of pixel coordinate is resolution, and the unit of infrared radiation intensity is watt per steradian. Each line in the platform+airborne moving platform number.csv file represents a frame of data, and the time interval of each frame of data can be set by modifying the input of the simulation system.

[0110] The visualization module is based on Python language logic, and uses the Matplotlib library in Python to draw 2D and 3D graphics. By reading the target + target number .csv file storing target data and the target and airborne platform motion data in the platform + airborne platform number .csv file storing airborne platform data, the motion trajectory of the target and airborne platform in the multi-sensor network coordinate system is drawn. By reading the target position coordinate information, target flight direction, platform position coordinate information, platform flight direction, sensor direction, and pixel coordinate information of each target in the target + target number .csv file storing target data and the platform + airborne platform number .csv file storing airborne platform data, the platform direction, sensor direction, and target flight direction are combined with the spatial registration module to calculate the target detection pitch angle and azimuth angle relative to the sensor and the distance of the target from the sensor. Then, combined with the target infrared characteristic data based on the measured data and the image library and the pixel coordinates of each target in the sensor, the infrared characteristic images of each target that are scaled are imaged at the specified pixel coordinate position of the two-dimensional imaging plane of the sensor.

[0111] The target infrared characteristic data and image library based on measured data, the target and sensor platform motion and observation database based on the model, the data processing module, the spatial registration module, the temporal registration module, the error module, the data storage and visualization module constitute the moving platform multi-infrared sensor collaborative multi-target perception digital simulation platform described in this application.

[0112] The use process of the mobile platform multi-infrared sensor collaborative multi-target perception digital simulation platform is as follows Figure 6 As shown in the figure, the simulation system is configured by setting different simulation system inputs. The simulation system inputs include target items, airborne moving platform items, error items, and simulation setting items. The target items include the number and number of targets, the flight direction of each target, the initial position of each target, and the motion model of each target. The airborne moving platform items include: the number and number of platforms, the flight direction of each platform, the initial position of each platform, the motion model of each platform, and the rotation model of each sensor. The error items include platform position error, sensor angle measurement error, and sensor missed detection error. The simulation setting items include total simulation time, simulation step length, and data saving path. After configuring different simulation system inputs, based on the basic The target and sensor platform motion and observation database of the model is combined with the time alignment module, the space alignment module, and the error module to generate the motion data of each platform and each target and the observation data of each sensor; the infrared characteristic data and image of each target are generated based on the target infrared characteristic data and image library based on the measured data in the simulation system, combined with the detection distance of each target relative to each sensor and the detection pitch and azimuth of each target relative to each sensor; the motion data of each platform and each target, the observation data of each sensor, and the infrared characteristic data and image of each target are classified and stored through the data storage and visualization module, and the three-dimensional spatial motion trajectory of multiple platforms and multiple targets and the two-dimensional multi-target infrared imaging of each sensor are drawn.

[0113] In a specific embodiment, Figure 7 The multi-infrared sensor multi-target collaborative perception scenario shown specifically illustrates the use of the distributed multi-infrared sensor simulation system and the data visualization effect. Figure 7In the multi-sensor network coordinate system shown in , there are two airborne moving platforms equipped with sensors and two targets. The distance between the two airborne moving platforms is 150m, and each airborne platform moves at a uniform speed in a straight line along the negative direction of the X-axis. The speed and flight altitude of each airborne moving platform are the same, and the sensor carried by each airborne moving platform always points toward the geometric center of the target formation. The distance between the two targets is 100 meters, and each target moves at a uniform speed in a straight line along the positive direction of the X-axis. The movement speed and flight altitude of each target are the same. The horizontal distance between the geometric center of the initial position formation of the two airborne moving platforms and the geometric center of the initial position formation of the two targets is 5000 meters, and the vertical distance is 500 meters. In this scenario, the target data storage file and the airborne platform data storage file format generated by the data storage module are as follows: Figure 8 As shown, the target data storage files are target1.csv and target2.csv, and the airborne platform data storage files are platform1.csv and platform2.csv. The target data storage files include seven columns of data, namely: simulation time, target X-axis position coordinates, target Y-axis position coordinates, target Z-axis position coordinates, flight yaw angle, flight pitch angle, and flight roll angle. Each row of data represents a frame of data, and the time interval between each frame of data is 0.05 seconds. The airborne moving platform data storage file includes fifteen columns of data, namely: simulation time, X-axis position coordinate of the airborne moving platform, Y-axis position coordinate of the airborne moving platform, Z-axis position coordinate of the airborne moving platform, flight yaw angle of the airborne moving platform, flight pitch angle of the airborne moving platform, flight roll angle of the airborne moving platform, pitch angle of the sensor, yaw angle of the sensor, horizontal coordinate of target 1 pixel in the sensor, vertical coordinate of target 1 pixel in the sensor, infrared radiation intensity of target 1 in the sensor, horizontal coordinate of target 2 pixel in the sensor, vertical coordinate of target 2 pixel in the sensor, infrared radiation intensity of target 2 in the sensor, each row of data represents a frame of data, and the time interval of each frame of data is 0.05 seconds.

[0114] Combined with Figure 8 The data shown in the figure are plotted as follows through the data visualization module. Fig. 9 The multi-platform multi-target three-dimensional space motion trajectory shown in Fig.10 and Fig.11 The sensors shown are two-dimensional multi-target infrared imaging. Fig. 9There are two targets in total, two airborne moving platforms equipped with sensors. The two airborne moving platforms are 150 meters apart, and each airborne moving platform moves in a straight line at a uniform speed along the negative direction of the X-axis. Each airborne moving platform has the same speed and the same flight altitude. The sensors carried by each airborne moving platform always point toward the geometric center of the target formation. The two targets are 100 meters apart, and each target moves in a straight line at a uniform speed along the positive direction of the X-axis. Each target has the same speed and the same flight altitude. Fig.10 It shows the infrared characteristic imaging of two targets detected in a certain frame in the imaging plane of the airborne infrared sensor 1 carried by the airborne platform 1. Fig.11 It shows the infrared characteristic imaging of two targets detected in a certain frame in the imaging plane of the airborne infrared sensor 2 carried by the airborne platform 2. There are two targets in the airborne sensor 1 and two targets in the airborne sensor 2. The total image resolution of all sensors is 640*512. The pixel size occupied by the target in the image and the infrared characteristics are calculated based on the distance of the target from the sensor in the current frame, the detection pitch and azimuth.

[0115] Through the above-mentioned dynamic platform multi-infrared sensor collaborative multi-target perception digital simulation platform, the working process of distributed multi-infrared sensors can be fully simulated in a virtual environment, providing accurate dynamic platform multi-infrared sensor multi-target perception simulation, so that researchers can repeatedly test and verify the distributed multi-infrared sensor collaborative perception platform under different operating conditions and environments to ensure the scientificity and feasibility of the design scheme and reduce the uncontrollable factors and risks in real tests. Through this platform, developers can systematically adjust parameters such as sensor configuration, perception algorithm and data fusion strategy, and accurately evaluate the impact of different configurations on distributed multi-infrared sensor collaborative algorithms and schemes; this enables the platform to optimize the overall performance and reliability of the platform in the early stage, avoiding the time and cost waste caused by repeated hardware adjustments in traditional experiments. By providing a flexible and customizable simulation environment, technical developers can quickly iterate and test new algorithms, new functions and new designs. Without relying on physical hardware, it can efficiently simulate a variety of working conditions in actual application scenarios, thereby accelerating the research and development and application deployment of new technologies and improving development efficiency.

[0116] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification.

[0117] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the appended claims.

Claims

1. A digital simulation platform for multi-infrared sensors on a moving platform to coordinate multi-target perception, characterized in that: include: Data processing module, error module, data storage and visualization module, measured database and model database; among them, The error module is used to obtain the position error of the airborne moving platform, the missed detection error of the sensor and the angle measurement error of the sensor; A model database is used to generate the motion data of the target, the motion data of the airborne moving platform and the observation data of each sensor based on the kinematic model of the detected target, the kinematic model of the airborne moving platform and the two-dimensional imaging model of the sensor, and in combination with the position error of the airborne moving platform, the missed detection error of the sensor and the angle measurement error of the sensor; A data processing module, used for processing target infrared radiation intensity data of the detected target and the infrared characteristic image of the sensor imaging plane under typical working conditions to generate infrared radiation characteristic data of the detected target and a target infrared characteristic grayscale image; The measured database is used to store the infrared radiation characteristic data of the detected target and the target infrared characteristic grayscale image, and generate the infrared characteristic data and image of each target by combining the detection distance of each target relative to each sensor, and the detection pitch and azimuth of each target relative to each sensor; The data storage and visualization module is used to manage and store the infrared characteristic data and images of each target, and analyze the infrared characteristic data and images of each target to generate the motion trajectory of each target and each sensor in the network coordinate system.

2. According to claim 1, the dynamic platform multi-infrared sensor collaborative multi-target perception digital simulation platform is characterized in that: The model database includes: A target motion data generation submodule is used to obtain the motion data of the target based on the kinematic model of the detected target; The airborne moving platform motion data generation submodule is used to obtain the motion data of the airborne moving platform based on the kinematic model of the airborne moving platform and the position error of the airborne moving platform; The sensor observation data generation submodule is used to obtain the observation data of each sensor based on the two-dimensional imaging model of the sensor, the missed detection error of the sensor and the angle measurement error of the sensor.

3. According to claim 2, the dynamic platform multi-infrared sensor collaborative multi-target perception digital simulation platform is characterized in that: The data processing module also includes: Target infrared characteristic data interpolation submodule and target specified distance image generation module; wherein, The target infrared characteristic data interpolation submodule is used to detect the detected target at different pitch angles and azimuth angles under several typical working conditions through the infrared sensor to obtain the target infrared radiation intensity data, and interpolate the target infrared radiation intensity data to obtain the infrared radiation characteristic data of the detected target; The target specified distance image generation module is used to detect the target at different pitch angles and azimuth angles through the infrared sensor to obtain the sensor imaging plane infrared characteristic image of the detected target, and to scale the sensor imaging plane infrared characteristic image of the detected target to obtain the target infrared characteristic grayscale image of the detected target.

4. According to claim 3, the dynamic platform multi-infrared sensor collaborative multi-target perception digital simulation platform is characterized in that: The interpolation process includes: Using the multivariate spline interpolation method, the azimuth angle in each elevation angle file is interpolated using univariate cubic spline interpolation; After the interpolation in the azimuth direction is completed, bivariate spline interpolation is performed on the interpolation data at different elevation angles; After the interpolation data of different pitch angles are subjected to bivariate spline interpolation, the trivariate spline interpolation under multiple working states is completed.

5. According to claim 3, the dynamic platform multi-infrared sensor collaborative multi-target perception digital simulation platform is characterized in that: The steps of the scaling process include: The actual distance between the detected target in the infrared characteristic image of the current sensor imaging plane and the sensor is calculated by combining the resolution of the detected target image in the infrared characteristic image of the current sensor imaging plane, the actual size of the detected target and the infrared sensor parameters, and the image scaling ratio at different distances is calculated by combining the different distances between the detected target and the sensor; Taking the infrared characteristic image of the detected target in the sensor imaging plane as a reference and combining the calculated image scaling ratios at different distances, the target infrared characteristic grayscale images of the detected target in the infrared sensor imaging plane at different distances, different positions, and different detection pitch and azimuth angles are generated.

6. According to claim 1, the digital simulation platform for multi-infrared sensors in cooperation with multiple targets for dynamic platform perception is characterized in that: Also includes: Temporal registration module and spatial registration module; wherein, Time registration module, used to synchronize the timestamps of various sensors; The spatial registration module is used to unify the coordinates of each sensor into the multi-sensor network coordinate system.

7. The digital simulation platform for multi-infrared sensors in cooperation with multiple targets for dynamic platform perception according to claim 6 is characterized in that: The data storage and visualization modules include: Data storage submodule and visualization submodule; among them, The data storage submodule is used to access the motion data of different types of sensors, the motion data of targets and the sensor observation data, and classify and store the data according to the sensor ID number and the target ID number; The visualization submodule is used to display the motion trajectory of the target and the sensor in the multi-sensor network coordinate system based on 3D graphics to display the formation of the target and the airborne moving platform, and to display the imaging effect of the target in the sensor imaging plane based on 2D graphics.

8. According to claim 1, the dynamic platform multi-infrared sensor collaborative multi-target perception digital simulation platform is characterized in that: Also includes: Configuration module, used to obtain target items, platform items, error items and simulation setting items; among them, The target items include the number and serial number of targets, the flight direction of each target, the initial position of each target, and the motion model of each target; The platform items include the number and serial number of airborne moving platforms, the flight direction of each airborne moving platform, the initial position of each airborne moving platform, the motion model of each airborne moving platform, and the rotation model of each sensor; The error terms include the position error of the airborne moving platform, the sensor angle measurement error, and the sensor missed detection error; The simulation settings include total simulation time, simulation step and data saving path.

Images