Aviation virtual-real joint platform construction system, method and platform based on digital twinning

Abstract

The invention discloses an aviation virtual-real joint platform construction system and method based on digital twinning and a platform, and relates to the field of aviation virtual-real joint platform construction. The aviation virtual-real combined platform construction system comprises a virtual model construction module, a physical attribute simulation and rule setting module, a platform visualization module, a virtual mapping module, an operation state monitoring model construction module, a performance evaluation module and a state prediction module. The aviation virtual-real joint platform comprises a perception transmission layer, a data layer, a service layer and an application layer. The sensing transmission layer collects data through various sensors and transmits the data to the data layer through big data, the Internet of Things and cloud service; the data layer comprises a static database. According to the aviation virtual-real combined platform established by the invention, potential faults and performance reduction can be found in advance through real-time monitoring and predictive analysis, so that preventive measures are taken, and the performance and reliability of an aircraft are improved.

Description

Technical Field

[0001] The present invention relates to the field of aviation virtual-reality joint platform construction, and in particular to an aviation virtual-reality joint platform construction system, method and platform based on digital twins. Background Art

[0002] Digital twin technology is a method to simulate, predict and optimize real-world systems through real-time data connection between virtual models and the physical world. It uses technologies such as the Internet of Things, big data, and artificial intelligence to map real-world objects, systems, or processes into virtual space, and achieve information exchange and collaboration between entities and virtual bodies.

[0003] In the aviation field, the application of digital twin technology can significantly improve the performance and reliability of aircraft. Due to the changeable operating environment of aircraft, its performance is closely related to the design, manufacturing and other processes. With the development of control systems and sensor technology, the amount of monitoring data has increased dramatically, and the data presents typical characteristics such as high speed, multi-source, and heterogeneity. The current technology is difficult to meet the high-precision requirements of real-time evaluation and prediction of the status of aircraft in dynamic and changing environments. Digital twin technology, with its multi-scale, high-fidelity model and full life cycle data management capabilities, synchronously reflects the status of the aircraft in the virtual space. Through visualization technologies such as three-dimensional models, digital twins can intuitively and vividly display the operating status of the aircraft. Summary of the invention

[0004] In order to solve the shortcomings of the above-mentioned prior art, the present invention proposes a system, method and platform for building an aviation virtual-reality joint platform based on digital twins, so as to further improve the performance and reliability of the aircraft, reduce maintenance costs, and extend its service life through the combination of virtual and real.

[0005] Specifically, on the one hand, the present invention provides an aviation virtual-real joint platform construction system based on digital twin, which includes a virtual model construction module, a physical property simulation and rule setting module, a platform visualization module, a virtual mapping module, an operation status monitoring model construction module, a performance evaluation module, a status prediction module and a maintenance report generation module;

[0006] The virtual model building module is used to build a virtual model of an aircraft in 3ds Max and input it into Unity3D. The virtual model building module includes a data collection unit, a model shape construction unit, an initial aircraft model structure construction unit, an aircraft material selection unit, a model input unit and an aviation environment simulation unit. The data collection unit is used to collect aircraft drawings and parameter data; the model shape construction unit is used to use basic geometric body tools in 3ds Max to create the basic shape of the model, and adjust the size of the basic geometric body according to the parameter data in the aircraft drawing; the initial aircraft model structure construction unit is used to use the editing tools in 3ds Max on the basis of the basic geometric body, adjust the model based on the aircraft drawing, and build the initial aircraft model structure; the aircraft material selection unit is used to use the material editor in 3ds Max to create and apply the materials corresponding to each component of the aircraft; the model input unit is used to convert 3ds Max into a virtual model of an aircraft. The initial aircraft model structure created in Max is exported to FBX format and imported into Unity3D; the aviation environment simulation unit is used to set up a scene in Unity3D to simulate the real aviation environment. The scene includes a skybox, ground, background and environmental elements. The sky background is simulated using Unity's Skybox material; in the Hierarchy panel, select the Terrain component under 3D Object to create a terrain object, generate a default terrain model, and edit it according to needs through the Terrain component; create a directional light to simulate sunlight, and adjust the position of the directional light to simulate different time lighting effects;

[0007] The physical property simulation and rule setting module is used to simulate the physical properties and rules of the aircraft itself using Unity3D to perform physical property simulation and rule setting, so as to realize the embodiment of physical entities in the virtual environment; specifically, it includes gravity setting, physical material setting and camera system setting. The gravity setting is specifically to modify the physical setting of the project in the Unity editor to adjust the global gravity; the physical material setting is specifically to define the physical material through Physics Material, and the physical material includes dynamic friction, static friction and elastic properties; the parameter range of dynamic friction is set to between 0 and 1, the parameter range of static friction is set to between 0 and 1, and the parameter range of elastic property is set to between 0.1 and 0.2; the camera system includes an anthropomorphic prefabricated body and a main camera for controlling the forward direction;

[0008] The platform visualization module is used to build a visualization page of the twin platform;

[0009] The virtual mapping module is used to bind and debug the interaction and mapping relationship of various aviation aircraft components, design interactive functions and realize virtual mapping;

[0010] The operation status monitoring model building module is used to collect multi-source data of the component to be predicted, and train the model based on the Informer model to predict the operation status of the component;

[0011] The performance evaluation module is used to perform performance evaluation on the training results of the prediction model;

[0012] The state prediction module is used to save the prediction model and then write a dynamic link library, and call the dynamic link library to realize the state prediction of the aircraft when predicting the platform failure;

[0013] The maintenance report generation module is used to automatically generate a guiding maintenance report after completing the state prediction. The guiding maintenance report can clearly mark the specific area where the fault occurs by combining the 3D components of the aircraft.

[0014] Preferably, the platform visualization module creates a UI component in Unity3D for displaying data and realizing user interaction. The UI component is Text or Button, and a preset visualization interface is obtained by adjusting the position, size and style of the UI element.

[0015] Preferably, the virtual mapping module controls the behavior and interaction logic of the UI component through a C# script in Unity3D. First, the UI component is created and configured in the Unity editor. Then, a C# script file is created and a public method is defined therein to update the content of the UI component. Then, the script is attached to the aviation component object containing the UI component, and the public fields of the script are set in the Inspector panel to ensure that the script can correctly reference the UI component. Finally, the functionality of the script is verified by running and testing to ensure that the UI component can respond to interactions as expected.

[0016] Preferably, the operation status monitoring model building module adopts the Informer model to predict the signal of the component to be predicted and diagnose the prediction result. The Informer model includes an encoder and a decoder. The encoder is used to extract features from the input historical time series data, and the decoder is used to map the high-dimensional features generated by the encoder to the target time series to generate future time series predictions.

[0017] Preferably, the performance evaluation module can evaluate the accuracy of the prediction results through the prediction evaluation index, and the performance evaluation module uses the mean absolute error MAE, the root mean square error RMSE and the determination coefficient R 2 Assess the accuracy of forecast results;

[0018] The mean absolute error (MAE) measures the size of the error by calculating the average of the absolute values ​​of the differences between the predicted values ​​and the actual observed values. The calculation formula of MAE is as follows:

[0019]

[0020] Among them, y i is the ith observation value, is the average value of all observations, f(x i ) is the model prediction value of the i-th observation, and m is the length of the time series;

[0021] The root mean square error RMSE is the standard deviation of the difference between the observed value and the model predicted value. The calculation formula of RMSE is as follows:

[0022]

[0023] R 2 R is the ratio of actual observations to total observations that can be explained by the model. 2 The value of is determined by three parameters: SSR, SST and SSE. SSR is the sum of squares of the difference between the predicted data and the mean of the true data, SST is the sum of squares of the difference between the true data and its mean, and SSE is the sum of squares of the difference between the true data and the predicted data. SST = SSR + SSE. The calculation formulas for the three parameters SSR, SST and SSE are as follows:

[0024]

[0025] Coefficient of determination R 2 The calculation formula is as follows:

[0026]

[0027] R 2 The closer the value is to 1, the better the model fits the data.

[0028] Preferably, the specific steps of generating a maintenance report are:

[0029] Create a fault annotation, add a red box or arrow at the fault location, and the annotation starts flashing when a component is predicted to fail;

[0030] Define a maintenance report template, which includes fault description, maintenance steps and precautions;

[0031] Use Unity's ScreenCapture.CaptureScreenshot method to capture Figure 3 Fault annotation view of the D model;

[0032] Fill the operation status monitoring results, prediction results and fault annotation views of the 3D model obtained from the condition monitoring system into the maintenance report template;

[0033] Use iTextSharp to integrate the filled maintenance report content and screenshots to generate a maintenance report in PDF format.

[0034] Preferably, on the other hand, the present invention provides a method for constructing an aviation virtual-real joint platform based on digital twins, which comprises the following steps:

[0035] S1. Create a virtual model of the aircraft in 3ds Max and import it into Unity3D;

[0036] S2. Use Unity3D to simulate the physical properties and rules of the aircraft itself and set rules to realize the embodiment of physical entities in the virtual environment;

[0037] S3. Build a visualization page for the twin platform;

[0038] S4. Bind and debug the interaction and mapping relationships of various aircraft components, design interactive functions and implement virtual mapping;

[0039] S5. Collect multi-source data of the component to be predicted, train the model based on the Informer model, and predict the operating status of the component;

[0040] S6. Performing performance evaluation on the training results of the prediction model;

[0041] S7, after saving the prediction model, compile a dynamic link library and call the dynamic link library to realize the state prediction of the aircraft when predicting the platform fault;

[0042] S8. After the status prediction is completed, a guiding maintenance report is automatically generated. The guiding maintenance report can clearly mark the specific area where the fault occurs by combining the 3D components of the aircraft.

[0043] Preferably, step S1 specifically includes the following sub-steps:

[0044] S11. Collect relevant drawings and data of aviation aircraft;

[0045] S12. In 3ds Max, use basic geometry tools to create and adjust the basic shapes of the model, including cubes, spheres, or cylinders;

[0046] S13. Based on the basic geometry, use the editing tools in 3ds Max to modify and adjust the model and build the model structure. The modifications and adjustments include stretching, rotating and chamfering.

[0047] S14. Use the material editor in 3ds Max to create a variety of different materials and increase model details through maps and textures;

[0048] S15. Export the model created in 3ds Max to FBX format and import it into Unity3D;

[0049] S16. Set up a scene in Unity3D to simulate a real aviation environment. The scene includes ground, background and environmental elements.

[0050] Preferably, step S2 specifically includes the following sub-steps:

[0051] S21. Gravity settings: In the Unity editor, adjust the global gravity by modifying the project's physics settings. Access it through the following path: Edit → Project Settings → Physics to find the Gravity setting and modify its value to adjust the global gravity effect. The initial value of gravity is -9.81.

[0052] S22. Physical material settings: define physical materials through Physics Material, which includes dynamic friction, static friction and elastic properties;

[0053] S23. Camera system: The camera system includes an anthropomorphic preform that controls the forward direction and a main camera. The main camera is placed on the upper end of the preform and the angle is adjusted accordingly to imitate the perspective of human eyes.

[0054] In the third aspect, the present invention provides an aviation virtual-reality joint platform based on digital twins, which includes a perception and transmission layer, a data layer, a service layer and an application layer. The perception and transmission layer collects data through various sensors and transmits data to the data layer through big data, the Internet of Things and cloud services; the data layer includes a static database, a dynamic database and knowledge documents. The static database includes configuration information, historical data, business data and expert experience; the dynamic database includes airborne monitoring data, airborne operation and maintenance data, integrated data and emergency data; the knowledge documents include fault dictionaries, product technical specifications, diagnostic methods, guidelines, cases, maintenance manuals, regulations and cases; the service layer includes a data storage unit, a reading unit, a visualization unit, a file download unit, a diagnostic unit, a prediction unit, a report generation unit, a data query unit, an information publishing unit, an update synchronization unit and an information browsing unit. The visualization management of the application layer is divided into PC display, mobile display and large-screen display.

[0055] Compared with the prior art, the present invention has the following beneficial effects:

[0056] (1) The aviation virtual-reality joint platform constructed by the present invention can detect potential failures and performance degradations in advance through real-time monitoring and predictive analysis, so as to take preventive measures and improve the performance and reliability of the aircraft.

[0057] (2) The aviation virtual-reality joint platform built by the present invention utilizes digital twin technology to make maintenance work more accurate, reduce unnecessary inspections and repairs, and thus reduce maintenance costs and operating expenses.

[0058] (3) The data and analysis results provided by the aviation virtual-reality joint platform built by the present invention can help decision makers make more scientific and accurate decisions and improve operational efficiency.

[0059] (4) The aviation virtual-reality joint platform constructed by the present invention can simulate different environmental conditions and help aircraft better adapt to changing operating environments.

[0060] (5) The construction method of the present invention realizes real-time monitoring and predictive maintenance of aircraft performance by integrating sensor technology, big data analysis and artificial intelligence algorithms, thereby significantly improving the operational efficiency and safety of aircraft. At the same time, after completing the status prediction, the platform can automatically generate a guiding maintenance report, which can clearly mark the specific area where the fault occurs by combining the 3D components of the aircraft, so that maintenance personnel can intuitively understand the fault location and provide detailed precautions and maintenance steps for specific fault conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 A schematic block diagram of a system for constructing an aviation virtual-reality joint platform of the present invention;

[0062] Figure 2 It is a flow chart of the method for constructing an aviation virtual-reality joint platform of the present invention;

[0063] Figure 3 It is a hierarchical architecture diagram of the digital twin platform of an aircraft of the present invention;

[0064] Figure 4 It is a model structure diagram of the present invention;

[0065] Figure 5 is a first-view reference diagram of the preform of the present invention;

[0066] Figure 6 It is the structure of the Informer model of the present invention;

[0067] Figure 7 This is the state prediction process of the present invention. DETAILED DESCRIPTION

[0068] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0069] On the one hand, the present invention provides a system for constructing an aviation virtual-real joint platform based on digital twins, such as Figure 1As shown, it includes a virtual model building module 1, a physical property simulation and rule setting module 2, a platform visualization module 3, a virtual mapping module 4, an operation status monitoring model building module 5, a performance evaluation module 6, a status prediction module 7 and a maintenance report generation module 8.

[0070] The virtual model building module 1 is used to establish a virtual model of an aircraft in 3ds Max and input it into Unity3D. The virtual model building module includes a data collection unit, a model shape construction unit, an initial aircraft model structure construction unit, an aircraft material selection unit, a model input unit and an aviation environment simulation unit. The data collection unit is used to collect aircraft drawings and parameter data; the model shape construction unit is used to use basic geometric tools in 3ds Max to create the basic shape of the model, and adjust the size of the basic geometric body according to the parameter data in the aircraft drawing; the initial aircraft model structure construction unit is used to use the editing tools in 3ds Max on the basis of the basic geometric body, adjust the model based on the aircraft drawing, and build the initial aircraft model structure; the aircraft material selection unit is used to use the material editor in 3ds Max to create and apply the materials corresponding to each component of the aircraft; the model input unit is used to convert 3ds Max into a virtual model of an aircraft. The initial aircraft model structure created in Max is exported to FBX format and imported into Unity3D; the aviation environment simulation unit is used to set up a scene in Unity3D to simulate the real aviation environment. The scene includes a skybox, ground, background, and environmental elements. The sky background is simulated using Unity's Skybox material; in the Hierarchy panel, select the Terrain component in the 3D Object module to create a terrain object, generate a default terrain model, and edit it according to needs through the Terrain component; create a directional light to simulate sunlight, and adjust the directional light position to simulate different time lighting effects. Use Unity's Skybox material to simulate the sky background, download the skybox material, and apply it to the camera in the Scene.

[0071] Physical property simulation and rule setting module 2 is used to simulate the physical properties and rules of the aircraft itself using Unity3D to set the rules, so as to realize the embodiment of physical entities in the virtual environment; specifically, it includes gravity setting, physical material setting and camera system setting. Gravity setting is to modify the physical settings of the project in the Unity editor to adjust the global gravity. The initial value of gravity is -9.81, which means the standard gravity acceleration. Physical material setting is to define the physical material through PhysicsMaterial. The physical material includes dynamic friction, static friction and elastic properties. Dynamic friction is the friction used when the object is already moving, and the parameter range is set between 0 and 1. The default setting is 0.6, which needs to be adjusted according to the aircraft material and the expected physical behavior.

[0072] Static Friction is the amount of friction used when the object is at rest on a surface and can be set between 0 and 1. The default setting is 0.6 and needs to be adjusted depending on the aircraft material and expected physical behavior.

[0073] The elasticity attribute defines the elasticity of the surface. A value of 0 indicates that the surface is not elastic, and a value of 1 indicates that the surface is very elastic. In this embodiment, the parameter range of the elasticity attribute is set between 0.1 and 0.2, preferably set to 0.1, simulating a slight rebound when the aircraft lands.

[0074] The above operations and settings simulate the actual operating state and environment of the aircraft to restore the aviation state that is closest to the real physical entity. It can achieve the visual effect of roaming in the scene when controlling the movement of the prefab.

[0075] Camera system settings: The first-person camera system consists of an anthropomorphic prefab with controllable forward direction and a main camera. The main camera is placed on the top of the prefab and the angle is adjusted accordingly to imitate the human eye perspective. The visual effect of roaming in the scene can be achieved when controlling the movement of the prefab.

[0076] The platform visualization module 3 is used to build the visualization page of the twin platform; specifically: create UI components such as Text, Button, etc. in Unity3D to display data and realize user interaction. Adjust the position, size and style of UI elements to meet design requirements.

[0077] In the Unity editor, right-click the Hierarchy panel and select UI→Canvas. Create a Canvas object as a container for all UI elements.

[0078] Right-click the Canvas object and select UI→Text to create a Text object under the Canvas. In the Text component, adjust the font, font size, color, and alignment.

[0079] Right-click the Canvas object, select UI→Button, and create a Button object under the Canvas.

[0080] Adjust the position, size, and style of UI elements to meet the design requirements of aviation aircraft digital twin visualization panels.

[0081] The virtual mapping module 4 is used to bind and debug the interaction and mapping relationship of each aviation aircraft component, design interactive functions and implement virtual mapping; specifically: in Unity3D, the behavior and interaction logic of UI components are controlled by C# scripts. First, create and configure UI components such as UI Text in the Unity editor. Next, create a C# script file and define public methods in it to update the content of UI components. Then, attach the script to the aviation component object containing the UI component and set the public fields of the script in the Inspector panel to ensure that the script can correctly reference the UI component. Finally, verify the functionality of the script by running and testing to ensure that the UI component can respond to the interaction as expected.

[0082] The operating status monitoring model 5 construction module is used to collect multi-source data of the component to be predicted, and train the model based on the Informer model to predict the operating status of the component; specifically: the Informer model is used to predict the signal of the component to be predicted, and the prediction result is diagnosed to achieve the goal of predictive diagnosis of the future status of the component. The structure of the Informer model is as follows Figure 4 As shown, the left frame is the encoder, which is used to extract features from the input historical time series data, and the right frame is the decoder, which is used to map the high-dimensional features generated by the encoder to the target time series to generate future time series forecasts.

[0083] The performance evaluation module 6 is used to perform performance evaluation on the training results of the prediction model.

[0084] The performance evaluation module 6 can evaluate the accuracy of the prediction results through the prediction evaluation index. The performance evaluation module uses the mean absolute error MAE, the root mean square error RMSE and the determination coefficient R 2 Assess the accuracy of the prediction results.

[0085] In specific applications, the mean absolute error (MAE) measures the size of the error by calculating the average of the absolute values ​​of the differences between the predicted values ​​and the actual observed values. The calculation formula of MAE is as follows:

[0086]

[0087] Among them, y i is the ith observation value, is the average value of all observations, f(x i ) is the model prediction value of the ith observation, and m is the length of the time series.

[0088] The root mean square error RMSE is the standard deviation of the difference between the observed value and the model predicted value. The calculation formula of RMSE is as follows:

[0089]

[0090] R 2 R is the ratio of actual observations to total observations that can be explained by the model. 2 The value of is determined by three parameters: SSR, SST and SSE. SSR is the sum of squares of the difference between the predicted data and the mean of the true data, SST is the sum of squares of the difference between the true data and its mean, and SSE is the sum of squares of the difference between the true data and the predicted data. SST = SSR + SSE. The calculation formulas for the three parameters SSR, SST and SSE are as follows:

[0091]

[0092] Coefficient of determination R 2 The calculation formula is as follows:

[0093]

[0094] R 2 The closer the value is to 1, the better the model fits the data.

[0095] The state prediction module 7 is used to save the prediction model and then compile a dynamic link library, and call the dynamic link library to realize the state prediction of the aircraft when predicting the platform failure.

[0096] The maintenance report generation module 8 is used to automatically generate a guidance maintenance report after completing the state prediction. The guidance maintenance report can clearly mark the specific area where the fault occurs by combining the 3D components of the aircraft. The steps for generating the maintenance report are as follows:

[0097] Write a C# script to create a fault annotation and add a red box or arrow at the fault location. When a component is predicted to fail, the annotation starts flashing.

[0098] Define a maintenance report template that includes key information such as fault description, maintenance steps, and precautions.

[0099] Write a C# script to use Unity's ScreenCapture.CaptureScreenshot method to capture Figure 3 A fault-annotated view of the 3D model. Save the screenshot as an image file for later inclusion in the report.

[0100] The report template is filled in based on the operating status monitoring results and prediction results obtained from the condition monitoring system combined with expert experience and the fault annotation view of the 3D model.

[0101] Write a C# script and use iTextSharp to integrate the filled report content and screenshots to generate a maintenance report in PDF format.

[0102] Preferably, on the other hand, the present invention provides a method for constructing an aviation virtual-real joint platform based on digital twins, such as Figure 2 As shown, it includes the following steps:

[0103] S1. Create a virtual model of the aircraft in 3ds Max and import it into Unity3D. S11. Collect relevant drawings and data of the aircraft.

[0104] S12. In 3ds Max, use basic geometry tools to create and adjust the basic shapes of the model, including cubes, spheres, or cylinders.

[0105] S13. Based on the basic geometry, use the editing tools in 3ds Max to modify and adjust the model and build the model structure. The modifications and adjustments include stretching, rotating and chamfering.

[0106] S14. Use the material editor in 3ds Max to create a variety of different materials, and increase model details through maps and textures.

[0107] S15. Export the model created in 3ds Max to FBX format and import it into Unity3D.

[0108] S16. Set up a scene in Unity3D to simulate a real aviation environment. The scene includes ground, background and environmental elements.

[0109] S2. Use Unity3D to simulate the physical properties and rules of the aircraft itself and set the rules to realize the embodiment of the physical entity in the virtual environment. Step S2 specifically includes the following sub-steps:

[0110] S21. Gravity settings: In the Unity editor, adjust the global gravity by modifying the project's physical settings. Access it through the following path: Edit→Project Settings→Physics to find the Gravity setting, modify its value to adjust the global gravity effect. The initial value of gravity is -9.81.

[0111] S22. Physical material settings: Define physical materials through Physics Material. Physical materials include dynamic friction, static friction and elastic properties.

[0112] S23. Camera system: The camera system includes an anthropomorphic preform that controls the forward direction and a main camera. The main camera is placed on the upper end of the preform and the angle is adjusted accordingly to imitate the perspective of human eyes.

[0113] S3. Build a visualization page for the twin platform.

[0114] S4. Bind and debug the interaction and mapping relationships of various aircraft components, design interactive functions and implement virtual mapping.

[0115] S5. Collect multi-source data of the component to be predicted, train the model based on the Informer model, and predict the operating status of the component.

[0116] S6. Perform performance evaluation on the training results of the prediction model.

[0117] S7. After saving the prediction model, a dynamic link library is compiled and called during platform fault prediction to realize aircraft status prediction.

[0118] S8. After completing the status prediction, a guidance maintenance report is automatically generated. The guidance maintenance report can clearly mark the specific area where the fault occurs by combining the 3D components of the aircraft.

[0119] In the third aspect, the present invention provides an aviation virtual-reality joint platform based on digital twins, which includes a perception and transmission layer, a data layer, a service layer and an application layer. The perception and transmission layer collects data through various sensors and transmits data to the data layer through big data, the Internet of Things and cloud services; the data layer includes a static database, a dynamic database and knowledge documents. The static database includes configuration information, historical data, business data and expert experience; the dynamic database includes airborne monitoring data, airborne operation and maintenance data, integrated data and emergency data; the knowledge documents include fault dictionaries, product technical specifications, diagnostic methods, guidelines, cases, maintenance manuals, regulations and cases; the service layer includes a data storage unit, a reading unit, a visualization unit, a file download unit, a diagnostic unit, a prediction unit, a report generation unit, a data query unit, an information publishing unit, an update synchronization unit and an information browsing unit. The visualization management of the application layer is divided into PC display, mobile display and large-screen display. Specific embodiments

[0121] This embodiment provides a specific method for building an aviation virtual reality platform, which includes the following steps:

[0122] S1, such as Figure 3 and Figure 4 As shown, collect the design drawings and related data of the aircraft, use 3ds Max to create basic 3D geometric shapes, and adjust the parameters to meet the design requirements. Use the editing tools of 3ds Max to refine the model structure, add materials and textures to enhance the details, and export it in FBX format. Import the model in Unity3D and set up the scene environment, such as the ground and background, to simulate the real aviation environment.

[0123] S2, such as Figure 3 and Figure 5 As shown in the figure, adjust the global gravity settings in Unity to simulate the standard gravity acceleration. Add Rigidbody and Collider components to the object to achieve the elastic simulation of the aircraft landing. Set the physical material properties through Physics Material to simulate friction and elasticity. And add a first-person camera system.

[0124] S3, such as Figure 3 As shown in the figure, create UI components in Unity3D, such as Text and Button, to display data and implement user interaction, and adjust these elements to meet design requirements.

[0125] S4, such as Figure 3 As shown, write C# scripts to control the behavior and interaction logic of UI components, attach the scripts to aviation component objects, and ensure that the scripts can correctly reference UI components.

[0126] S5, such as Figure 6 As shown, the Informer model is used to perform predictive diagnosis on the signal, which extracts features through the encoder and generates future time series forecasts through the decoder.

[0127] S6. The accuracy of the prediction results can be evaluated through prediction evaluation indicators. The mean absolute error MAE, root mean square error RMSE and determination coefficient R 2 To evaluate the accuracy of the prediction results.

[0128] S7, such as Figure 7 As shown, after saving the prediction model, a dynamic link library (DLL) is written and the DLL is called in the platform fault prediction function to realize the state prediction of the aviation aircraft.

[0129] S8. To improve the efficiency of maintenance work and shorten the maintenance cycle, the platform can automatically generate a guiding maintenance report after completing the status prediction. The report can clearly mark the specific area where the fault occurred by combining the 3D components of the aircraft, so that maintenance personnel can intuitively understand the fault location and provide detailed precautions and maintenance steps for specific fault conditions.

[0130] The embodiments described above are only descriptions of the preferred implementation modes of the present invention, and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A digital twin-based aviation virtual-reality joint platform construction system, characterized by: It includes a virtual model building module, a physical property simulation and rule setting module, a platform visualization module, a virtual mapping module, an operation status monitoring model building module, a performance evaluation module, a status prediction module, and a maintenance report generation module; The virtual model building module is used to build a virtual model of an aircraft in 3ds Max and input it into Unity3D. The virtual model building module includes a data collection unit, a model shape construction unit, an initial aircraft model structure construction unit, an aircraft material selection unit, a model input unit, and an aviation environment simulation unit. The data collection unit is used to collect aircraft drawings and parameter data; the model shape construction unit is used to create the basic shape of the model in 3ds Max using a basic geometry tool, and adjust the size of the basic geometry according to the parameter data in the aircraft drawings; The initial aircraft model structure construction unit is used to adjust the model based on the aircraft drawings using the editing tools in 3ds Max on the basis of basic geometric bodies to construct the initial aircraft model structure; the aircraft material selection unit is used to create and apply materials corresponding to each component of the aircraft using the material editor in 3ds Max; the model input unit is used to export the initial aircraft model structure created in 3ds Max into FBX format and import it into Unity3D; the aviation environment simulation unit is used to set up a scene in Unity3D to simulate a real aviation environment, the scene includes a sky box, ground, background and environmental elements, and the sky background is simulated using Unity's Skybox material; in the Hierarchy panel, select the Terrain component under 3D Object to create a terrain object, generate a default terrain model, and edit it according to needs through the Terrain component; create directional light to simulate sunlight, and adjust the position of the directional light to simulate different time lighting effects; The physical property simulation and rule setting module is used to simulate the physical properties and rules of the aircraft itself using Unity3D and set rules to realize the embodiment of physical entities in the virtual environment; specifically, it includes gravity setting, physical material setting and camera system setting. The gravity setting is specifically to modify the physical settings of the project in the Unity editor to adjust the global gravity; the physical material setting is specifically to define the physical material through Physics Material, and the physical material includes dynamic friction, static friction and elastic properties; the parameter range of dynamic friction is set to between 0 and 1, the parameter range of static friction is set to between 0 and 1, and the parameter range of elastic property is set to between 0.1 and 0.2; The camera system includes an anthropomorphic prefab and a main camera that controls the forward direction; The platform visualization module is used to build a visualization page of the twin platform; The virtual mapping module is used to bind and debug the interaction and mapping relationship of various aviation aircraft components, design interactive functions and realize virtual mapping; The operation status monitoring model building module is used to collect multi-source data of the component to be predicted, and train the model based on the Informer model to predict the operation status of the component; The performance evaluation module is used to perform performance evaluation on the training results of the prediction model; The state prediction module is used to save the prediction model and then write a dynamic link library, and call the dynamic link library to realize the state prediction of the aircraft when predicting the platform failure; The maintenance report generation module is used to automatically generate a guiding maintenance report after completing the state prediction. The guiding maintenance report can clearly mark the specific area where the fault occurs by combining the 3D components of the aircraft.

2. The digital twin-based aviation virtual-reality joint platform construction system according to claim 1 is characterized by: The platform visualization module creates UI components in Unity3D for displaying data and realizing user interaction. The UI components are Text or Button. The preset visualization interface is obtained by adjusting the position, size and style of the UI elements.

3. The digital twin-based aviation virtual-reality joint platform construction system according to claim 1 is characterized by: The virtual mapping module controls the behavior and interaction logic of UI components through C# scripts in Unity3D. First, create and configure UI components in the Unity editor. Then, create a C# script file and define public methods in it to update the content of UI components. Then, attach the script to the aviation component object containing the UI component and set the public fields of the script in the Inspector panel to ensure that the script can correctly reference the UI component. Finally, verify the functionality of the script by running and testing to ensure that the UI component can respond to interactions as expected.

4. The digital twin-based aviation virtual-reality joint platform construction system according to claim 1 is characterized by: The operating status monitoring model building module adopts the Informer model to predict the signal of the component to be predicted and diagnose the prediction result. The Informer model includes an encoder and a decoder. The encoder is used to extract features from the input historical time series data, and the decoder is used to map the high-dimensional features generated by the encoder to the target time series to generate future time series predictions.

5. The digital twin-based aviation virtual-reality joint platform construction system according to claim 1 is characterized by: The performance evaluation module can evaluate the accuracy of the prediction results through the prediction evaluation index. The performance evaluation module uses the mean absolute error MAE, the root mean square error RMSE and the determination coefficient R 2 Assess the accuracy of forecast results; The mean absolute error (MAE) measures the size of the error by calculating the average of the absolute values ​​of the differences between the predicted values ​​and the actual observed values. The calculation formula of MAE is as follows: Among them, y i is the ith observation value, is the average value of all observations, f(x i ) is the model prediction value of the ith observation, and m is the length of the time series; The root mean square error RMSE is the standard deviation of the difference between the observed value and the model predicted value. The calculation formula of RMSE is as follows: R 2 R is the ratio of actual observations to total observations that can be explained by the model. 2 The value of is determined by three parameters: SSR, SST and SSE. SSR is the sum of squares of the difference between the predicted data and the mean of the true data, SST is the sum of squares of the difference between the true data and its mean, and SSE is the sum of squares of the difference between the true data and the predicted data. SST = SSR + SSE. The calculation formulas for the three parameters SSR, SST and SSE are as follows: Coefficient of determination R 2 The calculation formula is as follows: R 2 The closer the value is to 1, the better the model fits the data.

6. The digital twin-based aviation virtual-reality joint platform construction system according to claim 5 is characterized by: The specific steps to generate a maintenance report are: Create a fault annotation, add a red box or arrow at the fault location, and the annotation starts flashing when a component is predicted to fail; Define a maintenance report template, which includes fault description, maintenance steps and precautions; Use Unity's ScreenCapture.CaptureScreenshot method to take a screenshot of the fault-annotated view of the 3D model; Fill the operation status monitoring results, prediction results and fault annotation views of the 3D model obtained from the condition monitoring system into the maintenance report template; Use iTextSharp to integrate the filled maintenance report content and screenshots to generate a maintenance report in PDF format.

7. A method for constructing an aviation virtual-reality joint platform based on the digital twin-based aviation virtual-reality joint platform construction system of claim 1, characterized in that: It includes the following steps: S1. Create a virtual model of the aircraft in 3ds Max and import it into Unity3D; S2. Use Unity3D to simulate the physical properties and rules of the aircraft itself and set rules to realize the embodiment of physical entities in the virtual environment; S3. Build a visualization page for the twin platform; S4. Bind and debug the interaction and mapping relationships of various aircraft components, design interactive functions and implement virtual mapping; S5. Collect multi-source data of the component to be predicted, train the model based on the Informer model, and predict the operating status of the component; S6. Performing performance evaluation on the training results of the prediction model; S7, after saving the prediction model, compile a dynamic link library and call the dynamic link library to realize the state prediction of the aircraft when predicting the platform fault; S8. After the status prediction is completed, a guiding maintenance report is automatically generated. The guiding maintenance report can clearly mark the specific area where the fault occurs by combining the 3D components of the aircraft.

8. The method for constructing an aviation virtual-reality joint platform according to claim 7, characterized in that: Step S1 specifically includes the following sub-steps: S11. Collect relevant drawings and data of aviation aircraft; S12. In 3ds Max, use basic geometry tools to create and adjust the basic shapes of the model, including cubes, spheres, or cylinders; S13. Based on the basic geometry, use the editing tools in 3ds Max to modify and adjust the model and build the model structure. The modifications and adjustments include stretching, rotating and chamfering. S14. Use the material editor in 3ds Max to create a variety of different materials and increase model details through maps and textures; S15. Export the model created in 3ds Max to FBX format and import it into Unity3D; S16. Set up a scene in Unity3D to simulate a real aviation environment. The scene includes ground, background and environmental elements.

9. The method for constructing an aviation virtual-reality joint platform according to claim 7, characterized in that: Step S2 specifically includes the following sub-steps: S21. Gravity settings: In the Unity editor, adjust the global gravity by modifying the project's physics settings. Access it through the following path: Edit → Project Settings → Physics to find the Gravity setting and modify its value to adjust the global gravity effect. The initial value of gravity is -9.

81. S22. Physical material settings: define physical materials through Physics Material, which includes dynamic friction, static friction and elastic properties; S23. Camera system: The camera system includes an anthropomorphic preform that controls the forward direction and a main camera. The main camera is placed on the upper end of the preform and the angle is adjusted accordingly to imitate the perspective of human eyes.

10. An aviation virtual-reality combined platform constructed by using the aviation virtual-reality combined platform construction method of claim 7, characterized in that: The aviation virtual-reality joint platform includes a perception and transmission layer, a data layer, a service layer and an application layer. The perception and transmission layer collects data through various sensors and transmits data to the data layer through big data, the Internet of Things and cloud services; the data layer includes static databases, dynamic databases and knowledge documents. The static database includes configuration information, historical data, business data and expert experience; the dynamic database includes airborne monitoring data, airborne operation and maintenance data, integrated data and emergency data; the knowledge documents include fault dictionaries, product technical specifications, diagnostic methods, guidelines, cases, maintenance manuals, regulations and cases; the service layer includes data storage units, reading units, visualization units, file download units, diagnostic units, prediction units, report generation units, data query units, information publishing units, update synchronization units and information browsing units. The visualization management of the application layer is divided into PC display, mobile display and large-screen display.

Images