Full-process Multi-domain Simulation and Real-time Comprehensive Management System for Aircraft

Abstract

The present invention relates to the field of aircraft simulation management, and discloses an aircraft full-process multi-domain simulation and real-time comprehensive management system, which is used to solve the problem that when performing aircraft simulation tasks, the normal changes of the aircraft may be misjudged as program failures. It includes: uniformly configuring the simulation scenario, generating a standardized configuration file, completing the three-dimensional scene construction and target dynamic modeling, obtaining image frames, making a preliminary abnormal judgment on the image frames according to the image frames. If it is judged that the image frames are abnormal, then making an actual abnormal judgment on the image frames according to the aircraft data. If it is judged that the image frames are abnormal, then marking the image frames and issuing a warning, dynamically adjusting the operating state of the aircraft according to the simulation control instructions, and transmitting the operating state of the aircraft to the visualization display module to provide a graphical display interface for image output and the operating state of the aircraft, effectively reducing the probability of false touch warnings, interrupting the simulation process, and outputting invalid samples, and improving the credibility and stability of the simulation results.

Description

Technical Field

[0001] The present invention relates to the field of aircraft simulation management, and more particularly to a full-process multi-field simulation and real-time comprehensive management system for aircraft. Background Art

[0002] Aircraft management is a key technical support means in the research, development, test and verification of high-end equipment such as aerospace, unmanned aerial vehicles, and missile weapon systems. It is widely used in multiple stages of aircraft aerodynamic design, structural analysis, flight control development, mission planning, environmental simulation, payload verification, etc., and involves multiple engineering fields such as aerodynamics, flight dynamics, control systems, navigation and guidance, and communication links.

[0003] During the aircraft R & D process, in order to achieve digital support for the full life cycle from initial scheme demonstration to prototype flight test, it is necessary to build a complex simulation system facing multiple disciplines, multiple models, and multiple granularities, and cooperate to complete tasks such as multi-field information coupling modeling, dynamic response calculation, and system behavior verification. At the same time, with the popularization of the concepts of "digital prototype" and "digital twin", the simulation platform is gradually evolving from an "offline analysis tool" to a "real-time simulation and mission execution support platform".

[0004] In recent years, multi-field joint simulation platforms and high-performance simulation engines have been greatly developed. In the prior art, the simulation-driven design of aircraft is usually initially realized through means such as model library construction, interface standardization, and process automation, and the real-time state of the aircraft is simulated by collecting flight parameter data (such as position, speed, attitude, timestamp, etc.).

[0005] For example, a parallel computing system and method for an aircraft simulation model disclosed in the invention patent announcement with the publication number CN112560184B. The system includes N computing service nodes, a communication network between the N computing service nodes, and parallel computing software installed on each computing service node; the computing service node includes a GPU, an RMDA fiber optic communication network card, and parallel computing software; the main control end of the parallel computing software runs on the first computing service node, and the node terminal of the parallel computing software runs on the GPU; the node terminals of the parallel computing software run on the GPUs of the remaining N - 1 computing service nodes respectively; the GPU provides parallel solution for the aircraft simulation models set as participating nodes; the RMDA fiber optic communication network card is used to build a communication network between the N computing service nodes and distribute data between the aircraft simulation models through the above communication network. The present invention solves the problems that currently the aircraft simulation models do not use GPU computing chips to perform parallel computing on the participating simulation nodes and the simulation data transmission efficiency is not high.

[0006] However, the above technologies have at least the following technical problems:

[0007] In complex flight simulation tasks, when rotating, jittering, or flickering occurs between image frames during the image output process, it is usually directly judged as a program failure or rendering failure. However, in actual applications, image fluctuations may also stem from drastic changes in the aircraft, which belong to the category of normal physical responses. If directly judged as a program failure or rendering failure, it will trigger error warnings, interrupt the simulation process, or output invalid samples, affecting the credibility and stability of the simulation results. Summary of the Invention

[0008] To overcome the above-mentioned defects of the prior art, the present invention provides an aircraft full-process multi-domain simulation and real-time comprehensive management system to solve the problems existing in the above-mentioned background technology.

[0009] To achieve the above object, the present invention provides the following technical solutions:

[0010] An aircraft full-process multi-domain simulation and real-time comprehensive management system, the system includes: a scenario editing and performing module, used for unified configuration of the simulation scenario to obtain unified configuration data, and the unified configuration is responsible for completing the visual configuration of the mission scenario, terrain environment, meteorological conditions, target deployment, channel settings, interference parameters, and sensor configuration parameters, generating a standardized configuration file, and transmitting the standardized configuration file to the scenario generation engine module; a scenario generation engine module, used for completing the three-dimensional scenario construction and target dynamic modeling according to the standardized configuration file to obtain a real-time dynamic flight scenario, obtaining image frames according to the real-time dynamic flight scenario, transmitting and distributing rendering tasks and image frames to the scenario generation node module through a distributed computing structure, and transmitting the image frames to the visualization display module; a simulation control and execution module, used for inputting and executing simulation control instructions, and the simulation control instructions are for the unified scheduling of the entire simulation process, and transmitting the simulation control instructions to the scenario generation node module; a scenario generation node module, used for executing image generation tasks according to the distributed rendering tasks and image frames, realizing real-time image generation, preview, and transmission, dynamically adjusting the aircraft operation state according to the simulation control instructions, and transmitting the aircraft operation state to the visualization display module; a visualization display module, used for providing a graphical display interface for image output and aircraft operation state, receiving the image frames from the scenario generation engine module, and combining the unified scheduling data of the entire simulation process of the simulation control and execution module to realize functions such as three-dimensional perspective switching, sensor situation preview, and parameter monitoring.

[0011] Preferably, the scene generation engine module includes a flight parameter processing unit, a three-dimensional scene construction unit, and a sensor data generation unit; the flight parameter processing unit is configured to receive real-time flight state data of the aircraft through channel settings in the unified configuration, and the flight state data includes the three-dimensional position coordinates, attitude angles, speed, and timestamp of the aircraft, and preprocess the flight state data to obtain preprocessed flight state data. The three-dimensional position coordinates are the three-dimensional center point coordinates of the aircraft, and the attitude angles include pitch angle, roll angle, and yaw angle. The preprocessed flight state data is transmitted to the three-dimensional scene construction unit and the sensor data generation unit; the three-dimensional scene construction unit is configured to construct a dynamic three-dimensional space scene according to the unified configuration data, perform scene target driving in combination with the preprocessed flight state data, generate a natural background in combination with time and environment, obtain a scene construction result, and transmit the scene construction result to the sensor data generation unit; the sensor data generation unit is configured to perform simulation generation of visible light, infrared, and depth field images according to the scene construction result and the sensor configuration parameters in the unified configuration data to obtain an image frame, acquire image frame-related data, and the image frame-related data is the pixel gray value and timestamp of each pixel point in the image frame. Calculate the jitter degree according to the image frame-related data, and perform a preliminary abnormal judgment on the image frame according to the jitter degree; if the preliminary abnormal judgment is that the image frame is normal, no marking is performed. If the preliminary abnormal judgment is that the image frame is abnormal, calculate a flight state influence index according to the preprocessed flight state data, and perform an actual abnormal judgment on the image frame according to the flight state influence index; if the actual abnormal judgment is that the image frame is abnormal, trigger an abnormal warning and mark the current image frame as an abnormal frame. If the actual abnormal judgment is that the image frame is normal, no marking is performed.

[0012] Preferably, the step of obtaining the jitter degree is as follows: convert each image frame into a grayscale image frame, acquire the pixel gray values of two adjacent grayscale image frames, and calculate the pixel difference between each pixel of the two adjacent grayscale image frames; calculate the variance of each pixel difference between the two adjacent grayscale image frames as the gray difference value according to the pixel difference between each pixel of the two adjacent grayscale image frames; identify the aircraft in two adjacent grayscale image frames, extract the aircraft contour, and calculate the structural similarity value according to the aircraft contour; obtain the minimum circumscribed rectangle according to the aircraft contour in two adjacent grayscale image frames, calculate the aircraft center point coordinates according to the minimum circumscribed rectangle, and calculate the Euclidean distance between the aircraft center points in two adjacent grayscale image frames, denoted as the center target offset value; normalize the gray difference value, the structural similarity value, and the center target offset value, and calculate the jitter degree according to the normalized gray difference value, the structural similarity value, and the center target offset value.

[0013] Preferably, the structural similarity value obtaining step is as follows: Use edge detection on two adjacent grayscale image frames to obtain a binary edge map; Use a contour detection algorithm to obtain the aircraft boundary region from the binary edge map, denoted as a binary contour image; Count the number of pixel intersections and the number of pixel unions of the two binary contour images, and calculate the ratio of the number of pixel intersections to the number of pixel unions of the two binary contour images to obtain the structural similarity value.

[0014] Preferably, the step of preliminarily determining image frame anomalies according to the jitter degree is as follows: Compare the jitter degree with the jitter threshold. If the jitter degree is greater than or equal to the jitter threshold, preliminarily determine that the image frame is abnormal; If the jitter degree is less than the jitter threshold, determine that the image frame is normal.

[0015] Preferably, the flight state influence index obtaining step is as follows: Obtain the timestamps of two adjacent image frames, and according to the timestamps of the two adjacent image frames, correspond them to the preprocessed flight state data to obtain the three-dimensional position coordinates of the aircraft corresponding to the timestamps of the two adjacent image frames. Calculate the position distance between the three-dimensional position coordinates of the aircraft corresponding to the two adjacent timestamps through Euclidean distance, and calculate the position influence coefficient according to the position distance; Obtain the attitude angles of the aircraft corresponding to the timestamps of two adjacent image frames, and use the quaternion rotation difference method to evaluate the attitude angles of the aircraft to obtain the attitude angle influence coefficient; Obtain the speeds of the aircraft corresponding to the timestamps of two adjacent image frames, and evaluate the speeds of the aircraft to obtain the speed influence coefficient; Normalize the position influence coefficient, the attitude angle influence coefficient, and the speed influence coefficient, and calculate the flight state influence index according to the normalized position influence coefficient, the attitude angle influence coefficient, and the speed influence coefficient. The specific obtaining steps are as follows: ; where represents the flight state influence index, represents the normalized position influence coefficient, represents the normalized attitude angle influence coefficient, represents the normalized speed influence coefficient, 、 、 represent the weight coefficients of the position influence coefficient, the weight coefficient of the attitude angle influence coefficient, and the weight coefficient of the speed influence coefficient.

[0016] Preferably, the attitude angle influence coefficient obtaining step is as follows: Obtain the attitude angles of the aircraft corresponding to the timestamps of two adjacent image frames; Convert the angle of the attitude angle to radian representation, and construct a quaternion according to the radian representation; Obtain the quaternions of two adjacent image frames according to the quaternion; Use the quaternion dot product formula to calculate the spherical angle rotation difference between two adjacent image frames. The specific obtaining steps are as follows: ; where Expressed as the spherical angle rotation difference, Expressed as the quaternion of the t-th image frame, Expressed as the quaternion of the (t - 1)-th image frame; the attitude angle influence coefficient is calculated based on the spherical angle rotation difference.

[0017] Preferably, the step of obtaining the speed influence coefficient is as follows: obtaining the aircraft speed corresponding to the timestamps of two adjacent image frames; calculating the speed change vector based on the aircraft speed, and calculating the magnitude of the speed change vector based on the speed change vector; calculating the acceleration between two adjacent image frames based on the magnitude of the speed change vector, and the specific obtaining steps are as follows: ; where, Expressed as the acceleration between two adjacent image frames, Expressed as the magnitude of the speed change vector, Expressed as the time interval between two adjacent image frames, Expressed as the time of the t-th image frame, Expressed as the time of the (t - 1)-th image frame; the speed influence coefficient is calculated based on the acceleration between two adjacent image frames.

[0018] Preferably, the step of actually judging the abnormality of the image frame according to the flight state influence index is as follows: comparing the flight state influence index with the influence threshold. If the flight state influence index is greater than or equal to the influence threshold, it is judged that the image frame is normal and no warning is issued; if the flight state influence index is less than the influence threshold, it is determined that the image frame is abnormal.

[0019] The technical effects and advantages of the present invention:

[0020] Perform unified configuration of the simulation scenario, generate a standardized configuration file, complete the three-dimensional scene construction and target dynamic modeling, obtain the image frame, perform preliminary abnormality judgment on the image frame according to the image frame. If it is judged that the image frame is abnormal, then perform actual abnormality judgment on the image frame according to the aircraft data. If it is judged that the image frame is abnormal, then mark the image frame and issue a warning, dynamically adjust the aircraft operation state according to the simulation control instruction, and transmit the aircraft operation state to the visualization display module to provide a graphical display interface for image output and aircraft operation state, effectively reducing the probability of false touch warning, interrupting the simulation process and outputting invalid samples, and improving the credibility and stability of the simulation result. Brief Description of the Drawings

[0021] Figure 1 It is the structure diagram of the aircraft full-process multi-domain simulation and real-time comprehensive management system provided by the embodiment of the present application;

[0022] Figure 2 It is the structure diagram of the scene generation engine module provided by the embodiment of the present application. Detailed Embodiment

[0023] Next, in combination with the accompanying drawings in the present invention, the technical solutions in the present invention will be clearly and completely described. In addition, the forms of each structure described in the following embodiments are merely examples, and the full-process multi-domain simulation and real-time comprehensive management system of the aircraft related to the present invention is not limited to the structures described in the following embodiments. All other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

[0024] The present invention provides a full-process multi-domain simulation and real-time comprehensive management system for an aircraft, as Figure 1 shown, the system includes:

[0025] A scenario editing module, which is used to perform unified configuration of the simulation scenario to obtain unified configuration data. The unified configuration is responsible for completing the visual configuration of elements such as mission scenarios, terrain environments, meteorological conditions, target deployments, channel settings, interference parameters, and sensor configuration parameters, generating a standardized configuration file, and transmitting the standardized configuration file to the scenario generation engine module;

[0026] In this embodiment, it should be specifically noted that the steps for unified configuration of the simulation scenario are as follows:

[0027] The user controls the interface through scenario editing operations, selects to create a new project or import an existing configuration file, enters the simulation task editing state, and provides an engineering basis for subsequent configurations. This operation ensures that all configuration information is relevant and can be saved and output uniformly;

[0028] The user sets the type of the simulation scenario (such as ground, water surface, air) and the corresponding area information in the scenario configuration interface. The area information includes longitude and latitude coordinates, scenario name, terrain description, and thumbnail, etc., which are used to define the scenario space reference framework and background landform environment;

[0029] In the environment configuration interface, configure the natural condition parameters that affect image generation and target visibility, including weather conditions (sunny, rainy, foggy, snowy, cloudy, etc.), wind speed and direction, atmospheric mode and transmittance, sea state, visibility, temperature, aerosol type, season, and time period, etc. These parameters will be used as inputs to the environment model and affect the performance of the simulation image such as brightness, occlusion, and blur;

[0030] In the target deployment interface, add aircraft targets, set their types, names, quantities, three-dimensional initial positions (longitude, latitude, altitude), and attitude angles (pitch, roll, heading), and specify the motion mode (such as straight line, hovering, fixed point) and the simulation data source method for each target;

[0031] In the channel configuration interface, the user sets the corresponding sensor channels for each target, including the channel name, sensor type (infrared, visible light, depth field, etc.), simulation data source (such as DDS, UDP, trajectory file, hovering in the air, etc.), data output method (local, DDS, Socket, etc.), output path, image format, time delay, etc. At the same time, the physical parameter configuration information of the detector is associated, such as resolution, field of view, image frequency, etc.;

[0032] In the interference information configuration interface, add interference objects such as smoke bombs, infrared decoy bombs, chaff, and corner reflectors, and set their types, release units, release times, release positions, and motion parameters, etc. The interference objects will interact with the targets in the same scene, affecting the sensor perception effect and enhancing the simulation authenticity;

[0033] The user edits the detailed technical parameters of various sensors in the detection resource management interface, including the band range, field of view, image resolution, frequency, image bit width, attitude offset, etc., which are used to drive the optical modeling process of the image generation engine;

[0034] After completing the configuration of various parameters, the system will automatically perform model reference verification and parameter integrity check. The user clicks the "Export Configuration" button to generate a standardized configuration file, which contains all the above-mentioned scene elements and is passed as input to the scene generation engine module and the image generation node;

[0035] After the configuration is completed, the user clicks "Start Instance" through the scene rehearsal control interface, and the system will synchronously send the configuration file and control instructions to each scene generation node module to officially start the execution stage of the simulation task.

[0036] The scene generation engine module is used to complete the three-dimensional scene construction and target dynamic modeling according to the standardized configuration file, obtain a real-time dynamic flight scene, obtain image frames according to the real-time dynamic flight scene, transmit and distribute rendering tasks and image frames to the scene generation node module through a distributed computing structure, and transmit the image frames to the visualization display module, such as Figure 2 shown, including a flight parameter processing unit, a three-dimensional scene construction unit, and a sensor data generation unit;

[0037] The distributed computing structure refers to a multi-node collaborative computing architecture built in the scene generation engine module to cope with the high computational load of large-scale three-dimensional scene rendering and multi-channel image generation. This structure decomposes and parallelizes the computational processes such as flight parameter parsing, three-dimensional model loading, and sensor image generation by deploying image generation subtasks among multiple computing nodes.

[0038] Distributing rendering tasks means that after the scene generation engine module constructs a real-time dynamic flight scene, according to the sensor channel configuration and image generation requirements, it divides the computing tasks related to image rendering into several subtasks and dispatches them to each scene generation node module for execution through a distributed computing structure. Each rendering task usually includes specified viewpoints, sensor parameters (such as wavelength bands, field of view angles, resolutions, etc.), target position and attitude information, environmental conditions, etc., which are used to guide the scene generation node module to independently complete image generation and output for the corresponding viewpoints. Through this mechanism, parallel generation of images with multiple viewpoints and multiple channels can be achieved, improving the overall image rendering efficiency and task response ability of the system.

[0039] The scene generation engine module constructs a three-dimensional simulation scene with physical environment characteristics and target dynamic behaviors by parsing a standardized configuration file, and performs image rendering in combination with sensor modeling parameters. It adopts a time-stepping mechanism to drive scene changes, generates image frames in real time at each time point, and outputs them to subsequent modules to achieve real-time acquisition of multi-channel and multi-type images, providing a basic image data source for image anomaly analysis.

[0040] The flight parameter processing unit is used to receive the real-time flight state data of the aircraft through the channel settings in the unified configuration. The flight state data includes the three-dimensional position coordinates, attitude angles, speed, and timestamp of the aircraft, etc., and preprocesses the flight state data to obtain the preprocessed flight state data. The preprocessing includes formatting, denoising, and caching processing. It should be noted that the three-dimensional position coordinates are the three-dimensional center point coordinates of the aircraft, and the attitude angles include the pitch angle, roll angle, and yaw angle. The preprocessed flight state data is transmitted to the three-dimensional scene construction unit and the sensor data generation unit;

[0041] The data channel refers to the communication link or data source method for receiving the real-time flight state data of the aircraft, and its type and parameters are set by the channel information configuration module in the scene choreography module. According to the document content, the data channel can include DDS, TCP, UDP based on network communication protocols, shared memory channels based on the shared memory mechanism, fiber optic reflective memory channels based on dedicated devices, and local trajectory data files for offline simulation, etc. The system establishes a corresponding data reception process according to the selected channel type, transfers the flight parameter data to the flight parameter processing unit, and uniformly parses it into a standard format including keyword fields such as position, attitude angle, speed, and timestamp for subsequent simulation driving and image generation processing.

[0042] The three-dimensional scene construction unit is used to construct a dynamic three-dimensional space scene according to the unified configuration data, drive the scene targets in combination with the preprocessed flight state data, and generate a natural background in combination with time and environment to obtain the scene construction result, and transmit the scene construction result to the sensor data generation unit;

[0043] Scene target driving means that during the construction of a 3D scene, according to the pre-processed flight state data, the spatial positions and motion states of each target model in the scene are dynamically updated so that their behaviors in the virtual environment are consistent with the actual flight state. Through scene target driving, the target can achieve real dynamic changes such as translation, rotation, acceleration, and orbit change in 3D space, thus ensuring that the generated image frames are consistent with the actual operating state of the aircraft in terms of time and space, and improving the realism and effectiveness of the simulation scene.

[0044] The sensor data generation unit is used to simulate and generate visible light, infrared, and depth field images according to the sensor configuration parameters in the scene construction result and the unified configuration data, obtain image frames, acquire image frame-related data. The image frame-related data are the pixel gray values and timestamps of each pixel point in the image frame. Calculate the jitter degree according to the image frame-related data, and perform a preliminary abnormal judgment on the image frame according to the jitter degree;

[0045] If the preliminary abnormal judgment determines that the image frame is normal, no marking is performed. If the preliminary abnormal judgment determines that the image frame is abnormal, calculate the flight state influence index according to the pre-processed flight state data, and perform an actual abnormal judgment on the image frame according to the flight state influence index;

[0046] If the actual abnormal judgment determines that the image frame is abnormal, trigger an abnormal warning and mark the current image frame as an abnormal frame. If the actual abnormal judgment determines that the image frame is normal, no marking is performed.

[0047] In this embodiment, it should be specifically noted that the steps for obtaining an image frame are as follows:

[0048] Receive the scene construction result and the unified configuration data. According to the sensor configuration parameters in the unified configuration data, initialize the corresponding virtual sensor model, and determine the observation position, orientation, and frustum in 3D space. Each simulation sensor model corresponds to an image channel, such as a visible light channel, an infrared channel, or a depth field channel. The initialization process also determines key parameters such as the output format, image size, and time step of each frame of image;

[0049] The sensor data generation unit calculates the imaging perspective and projection area of each sensor according to the current simulated scene state, that is, the target and background ranges "seen" by the sensor in the current frame. This area takes into account spatial relationships such as the position, orientation, target distance, and field of view occlusion of the sensor to ensure that the image generation has a real perspective;

[0050] The system calls the corresponding image simulation rendering algorithm to generate an image for the specified perspective area. For different types of sensors, different image generation methods are used. After the image rendering is completed, the system forms a complete frame of image data as the image frame.

[0051] In this embodiment, it should be specifically noted that the steps for obtaining the jitter degree are as follows:

[0052] Convert each image frame into a grayscale image frame, obtain the pixel grayscale values of two adjacent grayscale image frames, and calculate the difference of each pixel in the two adjacent grayscale image frames;

[0053] According to the difference of each pixel in the two adjacent grayscale image frames, calculate the variance of each pixel difference as the grayscale difference value;

[0054] Identify the aircraft in two adjacent grayscale image frames, extract the aircraft contour, and calculate the structural similarity value according to the aircraft contour;

[0055] According to the aircraft contour in two adjacent grayscale image frames, obtain the minimum circumscribed rectangle, calculate the coordinates of the aircraft center point according to the minimum circumscribed rectangle, and calculate the Euclidean distance between the aircraft center points in two adjacent grayscale image frames, denoted as the central target offset value. The Euclidean distance is a commonly used spatial distance metric for calculating the straight-line distance between two points in a two-dimensional or multi-dimensional space, and is used to measure the position change degree of the aircraft center point in two adjacent grayscale image frames;

[0056] Normalize the grayscale difference value, the structural similarity value, and the central target offset value, and calculate the jitter degree according to the normalized grayscale difference value, the structural similarity value, and the central target offset value. The specific obtaining steps are as follows: ;

[0057] In the formula, represents the jitter degree, represents the normalized structural similarity value, represents the normalized grayscale difference value, represents the normalized central target offset value. When the image changes smoothly, the grayscale difference value and the central target offset value are small, the structural similarity value is close to 1, the denominator is large, and the final jitter degree is small. When the image undergoes obvious jumps or the target suddenly shifts, the grayscale difference value and the central target offset value become larger, and at the same time the structural similarity value decreases, resulting in a significant increase in the jitter degree. Using the logarithmic function can avoid extreme amplification of the grayscale difference and control the non-linear growth. Using the square root function can make the influence of position offset more obvious under small jumps and tend to be gentle under extremely large offsets, which helps to alleviate the suppression of outliers on the result.

[0058] In this embodiment, it should be specifically noted that the steps for obtaining the structural similarity value are as follows:

[0059] The edge detection method is used to obtain a binary edge map for two adjacent grayscale image frames. The edge detection method is an image processing technique used to identify regions in an image with significant gray-level changes and extract object boundaries or contours. In this scenario, the edge detection method is used to process two adjacent grayscale image frames to generate corresponding binary edge maps. It usually includes steps such as image smoothing, gradient calculation, non-maximum suppression, and double-threshold edge connection, and can effectively extract the clear contours of objects in the image, serving as the basis for subsequent analysis of the structural similarity of the aircraft;

[0060] The contour detection algorithm is used to obtain the boundary region of the aircraft from the binary edge map, denoted as the binary contour image. The contour detection algorithm is an image analysis method used to extract continuous boundary information from a binary image and can identify and extract the external boundaries of objects in the image. It is used to analyze the binary image after edge detection processing to obtain the boundary region of the aircraft and generate a binary contour image representing its contour shape. By using contour tracking technology to find the set of edge points of the connected region and representing it as an ordered point set, it can be further used to calculate information such as contour area, bounding box, and center position, providing a data basis for structural similarity judgment and target motion analysis;

[0061] Count the number of pixel intersections and the number of pixel unions of the two binary contour images, and calculate the ratio of the number of pixel intersections of the two binary contour images to the number of pixel unions of the two binary contour images to obtain the structural similarity value.

[0062] In this embodiment, it should be specifically noted that the step of preliminary abnormal judgment of the image frame according to the jitter degree is as follows:

[0063] Compare the jitter degree with the jitter threshold. If the jitter degree is greater than or equal to the jitter threshold, the image frame is preliminarily judged to be abnormal; if the jitter degree is less than the jitter threshold, the image frame is judged to be normal. The jitter threshold is obtained by the adaptive threshold method. The adaptive threshold method is a method that automatically determines the judgment criterion according to the change characteristics of the current image data. It does not rely on a preset fixed value, but dynamically generates a threshold that is most suitable for the current scenario by analyzing the jitter degree between image frames over a period of time, such as calculating the average value, variance, or a certain proportion of high-order numbers of these jitter values. This allows the system to flexibly judge whether an image is abnormal under different flight states and different environmental complexities, improving the accuracy and stability of the judgment.

[0064] In this embodiment, it should be specifically noted that the step of obtaining the flight state influence index is as follows:

[0065] Obtain the timestamps of two adjacent image frames. According to the timestamps of two adjacent image frames, correspond them to the preprocessed flight state data, obtain the three-dimensional position coordinates of the aircraft corresponding to the timestamps of two adjacent image frames, calculate the position distance between the three-dimensional position coordinates of the aircraft corresponding to two adjacent timestamps through Euclidean distance, and calculate the position influence coefficient based on the position distance. The specific obtaining steps are as follows: ;

[0066] In the formula, is expressed as the position influence coefficient, is expressed as the position distance. By taking the natural logarithm after adding 1 to the Euclidean distance, a non-linear response is achieved: when the position of the aircraft changes slightly, the output value changes significantly, improving the detection sensitivity to micro-vibrations; while when the position changes greatly, the exponential increase is suppressed by the logarithmic function, preventing a small number of extreme values from dominating the overall judgment;

[0067] Obtain the attitude angles of the aircraft corresponding to the timestamps of two adjacent image frames, and use the quaternion rotation difference method to evaluate the attitude angles of the aircraft to obtain the attitude angle influence coefficient;

[0068] The quaternion rotation difference method is a calculation method used to evaluate the degree of spatial attitude change. By converting the attitude angles (pitch, roll, yaw) of the aircraft in adjacent image frames into quaternion form, the dot product between quaternions is used to calculate the rotation angle between two frames, thereby quantifying the amplitude of attitude change. It avoids the gimbal lock problem existing in the Euler angle representation and can more accurately reflect the overall rotation change of the aircraft in three-dimensional space. By comparing the attitude angles corresponding to the timestamps of two adjacent image frames, the quaternion rotation difference method is used to calculate the rotation angle of the attitude angle, and further construct the attitude angle influence coefficient to assist in judging the actual abnormal degree of the image frame.

[0069] Obtain the speed of the aircraft corresponding to the timestamps of two adjacent image frames, and evaluate the speed of the aircraft to obtain the speed influence coefficient;

[0070] Normalize the position influence coefficient, the attitude angle influence coefficient, and the speed influence coefficient, and calculate the flight state influence index based on the normalized position influence coefficient, the attitude angle influence coefficient, and the speed influence coefficient. The specific obtaining steps are as follows: ;

[0071] In the formula, is expressed as the flight state influence index, It is expressed as the normalized position influence coefficient. The more dramatic the spatial position change of the aircraft in adjacent image frames, the greater the impact on the image change. In actual scenes, when the aircraft undergoes rapid translation, jump transfer or discontinuous displacement, it often causes obvious changes in image content, such as sudden changes in perspective, background dislocation or target out of view. Therefore, the greater the position change, the stronger the flight state's ability to explain image jitter, and the greater its contribution to the flight state influence index, reflecting that image frame anomalies are more likely to be caused by physical movement rather than image generation anomalies. It is expressed as the normalized attitude angle influence coefficient. The more dramatic the change of the aircraft's attitude angle in adjacent image frames, the greater the impact on the change of image content. In actual flight scenarios, the rapid rotation or turning of the aircraft's attitude will cause significant changes in the target's perspective, angle, and composition in the image, and even cause the image to rotate, shake, or visually dislocate. Therefore, the greater the change in attitude angle, the more likely the image anomaly is caused by the aircraft's own movement rather than a system error, and thus occupies a higher proportion in the flight status influence index. Expressed as the normalized speed influence coefficient, the more obvious the speed change of the aircraft in adjacent image frames, the greater its potential impact on the image change. In some simulations or real scenes, the rapid increase or decrease of the aircraft speed may be accompanied by maneuvers, perspective switching or tracking instability, which may indirectly cause image blur, frame skipping or sudden changes in content between frames. Therefore, the more drastic the speed change, the more likely it is to cause image anomalies. , , It is expressed as the weight coefficient of the position influence coefficient, the weight coefficient of the attitude angle influence coefficient and the weight coefficient of the speed influence coefficient. , , , Obtained through the analytic hierarchy process, which is a commonly used multi-factor decision-making method used to establish a weight relationship of relative importance between multiple evaluation indicators. By constructing a pairwise comparison judgment matrix of factors, each factor is compared pairwise based on experience or actual importance to quantify the contribution of each factor to the overall goal. The eigenvector of the matrix is ​​extracted through mathematical calculation and normalized to obtain the weight value of each factor.

[0072] In this embodiment, it should be specifically explained that the steps for obtaining the attitude angle influence coefficient are:

[0073] Get the aircraft attitude angle corresponding to the timestamps of two adjacent image frames. The aircraft attitude angle is , , It is represented as the vehicle attitude angle of the t-th image frame, It is represented as the vehicle attitude angle of the t-1th image frame, , , represent the pitch angle, roll angle, and yaw angle of the aircraft corresponding to the t-th image frame, , , represent the pitch angle, roll angle, and yaw angle of the aircraft corresponding to the (t - 1)-th image frame;

[0074] Convert the angle of the attitude angle to radian representation. The specific steps are as follows:

[0075] ;

[0076] ;

[0077] ;

[0078] Construct a quaternion based on the radian representation. The specific steps are as follows:

[0079] ;

[0080] ;

[0081] ;

[0082] ;

[0083] Obtain the quaternions of two adjacent image frames according to the quaternion. Specifically, , , where represents the quaternion of the t-th image frame, represents the quaternion of the (t - 1)-th image frame, , , , respectively represent the four elements of the t-th image frame, , , , respectively represent the four elements of the (t - 1)-th image frame;

[0084] Use the quaternion dot product formula to calculate the spherical angle rotation difference between two adjacent image frames. The specific acquisition steps are as follows:

[0085] ;

[0086] where represents the spherical angle rotation difference, represents the quaternion of the t-th image frame, represents the quaternion of the (t - 1)-th image frame, where , the outermost layer of the formula is multiplied by 2, representing the complete spatial rotation angle;

[0087] The attitude angle influence coefficient is calculated based on the spherical angle rotation difference. The specific acquisition steps are:

[0088] ;

[0089] In the formula, Expressed as the attitude angle influence coefficient, It is expressed as the spherical angle rotation difference. The attitude angle influence coefficient is generated by solving the arc cosine of the three-dimensional rotation difference between adjacent frames of the aircraft and processing it through the square root function. The larger the value, the more drastic the aircraft rotation change is, and the greater the impact on the stability of the image frame; when the rotation change is small, the square root function can enhance the coefficient change caused by small rotation and improve the detection sensitivity of slight attitude jitter. The formula as a whole shows a nonlinear growth characteristic that is more sensitive to small disturbances and more stable to large rotations. The square root function has a strong low-value amplification effect, which can improve the responsiveness to slight attitude changes, while suppressing the expansion of abnormal values ​​under large rotation conditions and effectively controlling the output range.

[0090] In this embodiment, it should be specifically explained that the steps for obtaining the speed influence coefficient are:

[0091] Get the aircraft speed corresponding to the timestamps of two adjacent image frames. The aircraft speed includes the speed in the x, y, and z coordinate directions. The aircraft speed is , ,in Expressed as the aircraft speed in the tth image frame, Expressed as the aircraft speed in the t-1th image frame, , , It is expressed as the velocity of the aircraft in the x, y, and z coordinate directions of the t-th image frame, , , It is expressed as the velocity of the aircraft in the x, y, and z coordinate directions of the t-1th image frame;

[0092] The speed change vector is calculated based on the aircraft speed. The specific acquisition steps are:

[0093] ;

[0094] In the formula, Expressed as a velocity change vector, Expressed as the aircraft speed in the tth image frame, It is expressed as the aircraft speed of the t-1th image frame;

[0095] Calculate the modulus of the velocity change vector. The specific steps are as follows:

[0096] ;

[0097] In the formula, represents the magnitude of the velocity change vector;

[0098] Calculate the acceleration between two adjacent image frames according to the magnitude of the velocity change vector. The specific acquisition steps are as follows:

[0099] ;

[0100] In the formula, represents the acceleration between two adjacent image frames, represents the magnitude of the velocity change vector, represents the time interval between two adjacent image frames, represents the time of the t-th image frame, represents the time of the (t - 1)-th image frame;

[0101] Calculate the velocity influence coefficient according to the acceleration between two adjacent image frames. The specific acquisition steps are as follows: ;

[0102] In the formula, represents the velocity influence coefficient, represents the acceleration between two adjacent image frames. By calculating the three-dimensional velocity change and time interval between consecutive image frames of the aircraft, the magnitude of the acceleration is derived, and a square root function is introduced to construct the velocity influence coefficient. This coefficient remains at a low value in the low-speed stable state and has a strong response to sudden maneuvers, sudden stops, or changes in speed fluctuations, and can be effectively used to identify potential abnormal image frames due to speed jumps in the image.

[0103] In this embodiment, it should be specifically noted that the steps for actually judging the abnormality of the image frame according to the flight state influence index are as follows:

[0104] Compare the flight state influence index with the influence threshold. If the flight state influence index is greater than or equal to the influence threshold, it is determined that the image frame is normal and no warning is given; if the flight state influence index is less than the influence threshold, it is determined that the image frame is abnormal, and the influence threshold is obtained through an adaptive threshold.

[0105] The simulation control and execution module is used to input and execute simulation control instructions. The simulation control instructions are for the unified scheduling of the entire simulation process, such as providing control operation interfaces for start, pause, and reset, etc. At the same time, it monitors the simulation state, progress, and data consistency, and can link the scene generation engine module and the scene generation node module to execute cross-module instruction issuance, realize unified control logic, and transmit the simulation control instructions to the scene generation node module;

[0106] A scene generation node module, which is used to perform an image generation task according to the distributed rendering task and the image frame, realize real-time image generation, preview and transmission output, dynamically adjust the flight state of the aircraft according to the simulation control instruction, and transmit the flight state of the aircraft to the visualization display module;

[0107] A visualization display module, which is used to provide a graphical display interface for image output and the flight state of the aircraft, receive the image frame from the scene generation engine module, and combine the unified scheduling data of the entire simulation process of the simulation control and execution module to realize functions such as three-dimensional perspective switching, sensor situation preview, and parameter monitoring, and provide users with intuitive task perception and interaction capabilities.

[0108] Finally: The above are only the preferred embodiments of the present invention and are not used to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

[0109] The above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art within the technical scope disclosed in this application can easily think of changes or replacements, which should all be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

1. An aircraft full-process multi-domain simulation and real-time integrated management system, characterized in that, The system includes: A scenario programming module for performing unified configuration of the simulation scenario to obtain unified configuration data. The unified configuration is responsible for completing the visual configuration of the mission scenario, terrain environment, meteorological conditions, target deployment, channel settings, interference parameters, and sensor configuration parameters, generating a standardized configuration file, and transmitting the standardized configuration file to the scenario generation engine module; A scenario generation engine module for completing the construction of a three-dimensional scenario and target dynamic modeling according to the standardized configuration file to obtain a real-time dynamic flight scenario, acquiring image frames based on the real-time dynamic flight scenario, transmitting and distributing rendering tasks and image frames to the scenario generation node module through a distributed computing structure, and transmitting the image frames to the visualization display module; A simulation control and execution module for inputting and executing simulation control instructions. The simulation control instructions are for the unified scheduling of the entire simulation process, and transmitting the simulation control instructions to the scenario generation node module; A scenario generation node module for executing image generation tasks according to the distributed rendering tasks and image frames, realizing real-time image generation, preview, and transmission, dynamically adjusting the flight state of the aircraft according to the simulation control instructions, and transmitting the flight state of the aircraft to the visualization display module; A visualization display module for providing a graphical display interface for image output and the flight state of the aircraft, receiving the image frames from the scenario generation engine module, and combining the unified scheduling data of the entire simulation process of the simulation control and execution module to realize functions such as three-dimensional perspective switching, sensor situation preview, and parameter monitoring; The scenario generation engine module includes a flight parameter processing unit, a three-dimensional scenario construction unit, and a sensor data generation unit; The flight parameter processing unit is used to receive the real-time flight state data of the aircraft through the channel settings in the unified configuration. The flight state data includes the three-dimensional position coordinates, attitude angles, speed, and timestamp of the aircraft, and preprocess the flight state data to obtain the preprocessed flight state data. The three-dimensional position coordinates are the three-dimensional center point coordinates of the aircraft, and the attitude angles include the pitch angle, roll angle, and yaw angle. Transmit the preprocessed flight state data to the three-dimensional scenario construction unit and the sensor data generation unit; The three-dimensional scenario construction unit is used to construct a dynamic three-dimensional space scenario according to the unified configuration data, drive the scenario target in combination with the preprocessed flight state data, generate a natural background in combination with time and environment, obtain the scenario construction result, and transmit the scenario construction result to the sensor data generation unit; The sensor data generation unit is used to perform simulation generation of visible light, infrared, and depth field images according to the scenario construction result and the sensor configuration parameters in the unified configuration data to obtain image frames, acquire image frame-related data. The image frame-related data is the pixel gray value and timestamp of each pixel point in the image frame, calculate the jitter degree according to the image frame-related data, and perform preliminary abnormal judgment of the image frame according to the jitter degree; If the preliminary anomaly judgment determines that the image frame is normal, no marking is performed. If the preliminary anomaly judgment determines that the image frame is abnormal, the flight state impact index is calculated based on the preprocessed flight state data, and the actual anomaly judgment of the image frame is performed according to the flight state impact index; If the actual anomaly judgment determines that the image frame is abnormal, an anomaly warning is triggered, and the current image frame is marked as an abnormal frame. If the actual anomaly judgment determines that the image frame is normal, no marking is performed; The step of performing the actual anomaly judgment of the image frame according to the flight state impact index is as follows: Compare the flight state impact index with the impact threshold. If the flight state impact index is greater than or equal to the impact threshold, it is determined that the image frame is normal and no warning is issued. If the flight state impact index is less than the impact threshold, it is determined that the image frame is abnormal.

2. The full-process multi-domain simulation and real-time comprehensive management system for an aircraft according to claim 1, wherein The step of obtaining the jitter degree is as follows: Convert each image frame into a grayscale image frame, obtain the pixel gray values of two adjacent grayscale image frames, and calculate the difference of each pixel in the two adjacent grayscale image frames; According to the difference of each pixel in the two adjacent grayscale image frames, calculate the variance of each pixel difference as the gray difference value; Identify the aircraft in two adjacent grayscale image frames, extract the aircraft contour, and calculate the structural similarity value according to the aircraft contour; According to the aircraft contour in two adjacent grayscale image frames, obtain the minimum bounding rectangle, calculate the coordinates of the aircraft center point according to the minimum bounding rectangle, and calculate the Euclidean distance between the aircraft center points in two adjacent grayscale image frames, denoted as the center target offset value; Normalize the gray difference value, the structural similarity value, and the center target offset value, and calculate the jitter degree according to the normalized gray difference value, the structural similarity value, and the center target offset value.

3. The full-process multi-domain simulation and real-time comprehensive management system for an aircraft according to claim 2, wherein The step of obtaining the structural similarity value is as follows: Use the edge detection method on two adjacent grayscale image frames to obtain a binary edge map; Use the contour detection algorithm to obtain the aircraft boundary region of the binary edge map, denoted as the binary contour image; Count the number of pixel intersections and the number of pixel unions of the two binary contour images, and calculate the ratio of the number of pixel intersections and the number of pixel unions of the two binary contour images to obtain the structural similarity value.

4. The full-process multi-domain simulation and real-time comprehensive management system for an aircraft according to claim 1, characterized in that: The step of performing the preliminary anomaly judgment of the image frame according to the jitter degree is as follows: Compare the jitter degree with the jitter threshold. If the jitter degree is greater than or equal to the jitter threshold, it is preliminarily determined that the image frame is abnormal. If the jitter degree is less than the jitter threshold, it is determined that the image frame is normal.

5. The full-process multi-domain simulation and real-time comprehensive management system for an aircraft according to claim 1, wherein: The step of obtaining the flight state impact index is as follows: Obtain the timestamps of two adjacent image frames, correspond the timestamps of the two adjacent image frames with the preprocessed flight state data, obtain the three-dimensional position coordinates of the aircraft corresponding to the timestamps of the two adjacent image frames, calculate the position distance between the three-dimensional position coordinates of the aircraft corresponding to the two adjacent timestamps through the Euclidean distance, and calculate the position impact coefficient according to the position distance; Obtain the attitude angles of the aircraft corresponding to the timestamps of two adjacent image frames, and use the quaternion rotation difference method to evaluate the attitude angles of the aircraft to obtain the attitude angle impact coefficient; Obtain the aircraft speed corresponding to the timestamps of two adjacent image frames, evaluate the aircraft speed, and obtain the speed influence coefficient; Normalize the position influence coefficient, attitude angle influence coefficient, and speed influence coefficient, and calculate the flight state influence index based on the normalized position influence coefficient, attitude angle influence coefficient, and speed influence coefficient. The specific acquisition steps are as follows: ; In the formula, represents the flight state influence index, represents the normalized position influence coefficient, represents the normalized attitude angle influence coefficient, represents the normalized speed influence coefficient, , , represent the weight coefficients of the position influence coefficient, the attitude angle influence coefficient, and the speed influence coefficient respectively.

6. The full-process multi-domain simulation and real-time comprehensive management system for an aircraft according to claim 5, characterized in that: The steps for obtaining the attitude angle influence coefficient are as follows: Obtain the aircraft attitude angle corresponding to the timestamps of two adjacent image frames; Convert the angle of the attitude angle to radian representation, and construct a quaternion based on the radian representation; Obtain the quaternions of two adjacent image frames according to the quaternion; Use the quaternion dot product formula to calculate the spherical angle rotation difference between two adjacent image frames. The specific acquisition steps are as follows: ； wherein is expressed as the spherical angle rotation difference, is expressed as the quaternion of the t-th image frame, is expressed as the quaternion of the (t - 1)-th image frame; Calculate the attitude angle influence coefficient based on the spherical angle rotation difference.

7. The full-process multi-domain simulation and real-time comprehensive management system for an aircraft according to claim 5, characterized in that: The steps for obtaining the speed influence coefficient are as follows: Obtain the aircraft speed corresponding to the timestamps of two adjacent image frames; Calculate the speed change vector based on the aircraft speed, and calculate the magnitude of the speed change vector according to the speed change vector; Calculate the acceleration between two adjacent image frames based on the magnitude of the speed change vector. The specific acquisition steps are as follows: ； Wherein, represents the acceleration between two adjacent image frames, represents the magnitude of the velocity change vector, represents the time interval between two adjacent image frames, represents the time of the t-th image frame, represents the time of the (t - 1)-th image frame; Calculate the speed influence coefficient based on the acceleration between two adjacent image frames.

Images