Shipboard aircraft maintenance process simulation method and system based on virtual reality

Abstract

The invention discloses a shipboard aircraft maintenance process simulation method and system based on virtual reality, and relates to the technical field of analogue simulation. According to the method, the environment sensing data packets of the shipboard aircraft and the maintenance environment are collected through the multi-modal sensor array, the shipboard aircraft maintenance simulation space is constructed based on the environment sensing data packets, and the regional maintenance step association network under different maintenance events is established according to the maintenance event records; mapping the regional maintenance step associated networks to the surface of a shipboard aircraft maintenance simulation space, and matching the regional maintenance step associated networks with one another to obtain a maintenance step associated network; and a user performs communication interaction with the shipboard aircraft maintenance simulation space through the VR equipment, collects a user interaction process through the VR equipment, inputs a user interaction action into the maintenance step association network, and further outputs a corresponding special reference scheme to judge whether the user interaction process is compliant or not.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of simulation and specifically relates to a carrier-based aircraft maintenance process simulation method and system based on virtual reality. BACKGROUND

[0002] As the core equipment of naval aviation combat, the maintenance and support of carrier-based aircraft is crucial to maintaining combat capability. Traditional carrier-based aircraft maintenance training mainly relies on physical equipment and actual operation training, but this approach has many limitations. On the one hand, the number of physical equipment is limited, making it difficult to meet the needs of large-scale training, and frequent actual operation will cause some degree of damage to the equipment, increasing maintenance costs. On the other hand, the actual maintenance scene is complex and variable, making it difficult to simulate various possible faults and maintenance situations, so that maintenance personnel lack enough experience when facing some complex faults.

[0003] At the same time, during the maintenance process, maintenance personnel often need to operate based on a large number of maintenance manuals and experience, but due to the large amount of manual content, it is not convenient to find and refer to in actual operation, which can easily lead to operational errors. Moreover, the operation habits and skill levels of different maintenance personnel differ, making it difficult to ensure the standardization and normalization of the maintenance process.

[0004] With the development of virtual reality technology, it has shown great application potential in the fields of training and simulation. However, the virtual reality simulation technology for carrier-based aircraft maintenance processes is not perfect at present, lacking a systematic, comprehensive and efficient simulation method, which cannot accurately simulate the real scene and complex process of carrier-based aircraft maintenance, and cannot well meet the needs of carrier-based aircraft maintenance training and actual maintenance guidance. Therefore, a carrier-based aircraft maintenance process simulation method and system based on virtual reality are provided. SUMMARY

[0005] The purpose of the present application is to provide a carrier-based aircraft maintenance process simulation method and system based on virtual reality to solve the problems in the background art.

[0006] In order to achieve the above-mentioned purpose, the present application provides the following technical solutions: A carrier-based aircraft maintenance process simulation method based on virtual reality, comprising the following steps: Step S1, acquiring environmental perception data packets of the carrier-based aircraft and the maintenance environment through a multi-modal sensor array, and constructing a carrier-based aircraft maintenance simulation space based on the environmental perception data packets; Step S2, obtaining maintenance event records, establishing a regional maintenance step association network under different maintenance events according to the maintenance event records, mapping the regional maintenance step association network on the surface of the carrier-based aircraft maintenance simulation space, and then matching each regional maintenance step association network to obtain a maintenance step association network; Step S3, the user communicates and interacts with the carrier-based aircraft maintenance simulation space through the VR device, collects the user interaction process through the VR device, inputs the user interaction process into the maintenance step association network, and then outputs the corresponding special reference scheme to determine whether the user interaction process is compliant.

[0007] Further, the process of collecting the environmental perception data packet of the carrier-based aircraft and the maintenance environment through the multi-modal sensor and constructing the carrier-based aircraft maintenance simulation space includes: A plurality of multi-modal sensor arrays are installed on the carrier-based aircraft and the maintenance environment, and the environmental perception data packet of the carrier-based aircraft and the maintenance environment is collected from different angles through the multi-modal sensor array, wherein the maintenance environment includes maintenance tools and support equipment; The environmental perception data packet includes point cloud data and device image data; The data in the environmental perception data packet is spatially aligned, and then the carrier-based aircraft maintenance simulation space is constructed according to the spatial alignment result, and a plurality of uniformly distributed simulation data nodes are set in the carrier-based aircraft maintenance simulation space, each simulation data node is associated with a part of the carrier-based aircraft maintenance simulation space, and the carrier-based aircraft maintenance simulation space is composed of a plurality of carrier-based aircraft region models and maintenance models.

[0008] Further, the process of establishing the maintenance step association network according to the maintenance event record includes: The maintenance event record includes the maintenance event name, the maintenance region, the calling sequence and the model name of the maintenance tool and the support equipment, and the maintenance process video; The maintenance dynamic model is established according to the maintenance process video in the maintenance event record, the maintenance dynamic model is divided into a plurality of maintenance step models in the calling sequence of the maintenance tool and the support equipment, the maintenance step models corresponding to the same maintenance region but different subsequent called maintenance tools or support equipment are matched and connected with each other, and then the regional maintenance step association network corresponding to the maintenance event name is obtained; According to the maintenance region corresponding to each regional maintenance step association network, the regional maintenance step association network is mapped on the maintenance dynamic model, the regional maintenance step association networks associated with the same position on the maintenance dynamic model are matched with each other, and the maintenance step association network is obtained according to the matching result.

[0009] Further, the process of the user communicating and interacting with the carrier-based aircraft maintenance simulation space through the VR device includes: The VR device includes visual VR devices and interactive VR devices, and visual data nodes and interactive data nodes are set for various VR devices; After the user is connected with the carrier aircraft maintenance simulation space through the VR device, the visual data node first acquires the visual interaction range of the visual VR device, and is connected with the simulation data node in the visual interaction range, and then the simulation data node shares the model region associated therewith to the visual data node and the interaction data intersection.

[0010] Further, the process of collecting the user interaction process through the VR device includes: The interactive VR device is mapped in the form of an animation model in the carrier aircraft maintenance simulation space, and is recorded as an interactive simulation model. The interactive simulation model is set as the center, and an interaction association range is set for the interactive simulation model. During the movement of the interactive VR device by the user, the interactive data intersection acquires the dynamic spatial position of the interactive VR device, and transmits it to the nearest interactive data intersection, and then dynamically updates the interactive simulation model in the carrier aircraft maintenance simulation space. Meanwhile, it is judged whether the interactive association range of the interactive simulation model exists the maintenance model, and according to the judgment result, the interaction action between the corresponding maintenance model and the interactive simulation model is set, and then according to the gesture trajectory of the interactive VR device, the user interaction process between the maintenance model and the carrier aircraft region model is acquired.

[0011] Further, the process of inputting the user interaction action into the maintenance step association network includes: According to the type name and the maintenance region corresponding to the user interaction action corresponding to the maintenance model and the carrier aircraft region model, the corresponding region maintenance step association network is matched in the maintenance step association network, and a special reference scheme is generated according to the subsequent maintenance step model of the region maintenance step association network; It is judged whether the user interaction process is compliant through the special reference scheme, and corresponding correction video is generated according to the judgment, and at the same time, when the user forms an interaction action with a new maintenance model, the special reference scheme is dynamically updated until the user completes the entire maintenance event.

[0012] Further, the generation process of the special reference scheme includes: A plurality of maintenance event records corresponding to the maintenance step model are called, and a maintenance interaction action animation is established based on the maintenance event records, and the maintenance interaction action animation is composed of the maintenance model, the carrier aircraft region model and the interactive simulation model; A plurality of interaction recognition points are set at the interaction position of the maintenance model, the carrier aircraft region model and the interactive simulation model, and according to the change trajectory of each model in the maintenance interaction action animation, the recognition point trajectory corresponding to the interaction recognition point is generated; Map the recognition point trajectories corresponding to different maintenance event records generated in the same two-dimensional coordinate system, set a plurality of equal-length time segments, normally distribute the recognition point trajectory segments under each time segment, and obtain the corresponding standard trajectory interval according to the normal distribution result; Connect the standard recognition point trajectories under each time segment in sequence to obtain a standard recognition point trajectory set corresponding to the maintenance step model, and then integrate the maintenance step model and the corresponding standard recognition point trajectory set into a special reference scheme.

[0013] In the above technical solution, the technical effects and advantages provided by the present application are: 1、The present application establishes a maintenance step association network and generates a special reference scheme, thereby realizing real-time judgment of whether the operation of the maintenance personnel is compliant during the maintenance process, and timely generating a correction video to help the maintenance personnel correct the wrong operation, ensuring the standardization and normalization of the maintenance process, and improving the maintenance quality and reliability; 2、The present application maps the regional maintenance step association network on the maintenance dynamic model, and matches the regional maintenance step association networks associated with the same position to obtain a complete maintenance step association network, realizing effective association of local maintenance steps and the whole maintenance process, enabling the maintenance personnel to grasp the maintenance process as a whole, while also paying attention to the specific maintenance requirements of each maintenance area, thereby improving the maintenance efficiency and accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed in the embodiments. Obviously, the drawings described below are only some embodiments described in the present application, and other drawings can also be obtained by those skilled in the art based on these drawings.

[0015] Figure 1 The method flowchart of the aircraft maintenance process simulation method based on virtual reality.

[0016] Figure 2 The system block diagram of the aircraft maintenance process simulation system based on virtual reality. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0018] Please see Figure 1 As shown, a simulation method for shipborne aircraft maintenance based on virtual reality includes the following steps: Step S1: Collect environmental perception data packets of the carrier-based aircraft and maintenance environment through a multimodal sensor array, and construct a carrier-based aircraft maintenance simulation space based on the environmental perception data packets; Step S2: Obtain maintenance event records, establish regional maintenance step association networks under different maintenance events based on maintenance event records, map the regional maintenance step association networks onto the surface of the carrier-based aircraft maintenance simulation space, and then match the various regional maintenance step association networks to obtain the maintenance step association network. Step S3: The user communicates and interacts with the carrier-based aircraft maintenance simulation space through VR equipment, and the user interaction process is collected through VR equipment. The user interaction process is input into the maintenance step association network, and then the corresponding special reference solution is output to determine whether the user interaction process is compliant.

[0019] Furthermore, step S1 is implemented through the following process: Step S101: Collect environmental awareness data packets of the carrier-based aircraft and the maintenance environment through multimodal sensors. The specific process includes: In the maintenance hangar or deck simulation area, deploy n high-precision lidars and m depth cameras to form a multimodal sensor array, where n and m are natural numbers greater than 5; The multimodal sensor array scans the maintenance area of ​​the target carrier-based aircraft (such as fighter jets and helicopters) from different angles, and at the same time collects point cloud data and equipment video data of maintenance tools, support equipment and surrounding environment in the maintenance environment, and integrates them to generate corresponding environmental perception data packets. It should be noted that virtual boundary beacons are set up around the area to be repaired (such as the engine compartment and avionics compartment) to define the effective range of sensor data acquisition and to assist in calibrating the sensor attitude. The distance from the virtual boundary beacon to the core repair area defines the safe operating space of the area to be repaired.

[0020] Step S102: Construct a carrier-based aircraft maintenance simulation space based on environmental awareness data packets. The specific process includes: Point cloud data from different perspectives and sensors are aligned to the same coordinate system. Meanwhile, the device video data is divided into several device image data by frame. Then, according to the camera calibration parameters of the depth camera, the device image data is mapped to the corresponding position of the point cloud data, thereby completing the spatial alignment between the point cloud data and the device video data. Semantic information such as the names of maintenance tools and support equipment in the maintenance environment are bound to the spatially aligned point cloud data as attribute data. The spatially aligned point cloud data is converted into a polygonal mesh model, and the equipment image data is overlaid on the surface of the polygonal mesh model in the form of texture according to the spatial correspondence, thereby generating a carrier-based aircraft area model (such as engine compartment, landing gear compartment, avionics compartment) and a maintenance model (such as tools, support equipment). Based on the spatial position of the carrier-based aircraft area model and the maintenance model in the polygonal mesh model, all carrier-based aircraft area models and maintenance models are stitched together to obtain the carrier-based aircraft maintenance simulation space. Set physical properties such as rigid body and collision body for the carrier-based aircraft area model and maintenance model. For example, set weight and collision boundary for the wrench model included in the maintenance model, and set hinge joints for the hatch model in the carrier-based aircraft area model so that it can simulate real opening and closing actions and physical interactions. Using a spatial grid, several simulation data nodes are evenly set up in the carrier-based aircraft maintenance simulation space. Each simulation data node is responsible for managing the loading, unloading, status updates, and interactive communication of all models within its associated spatial area.

[0021] Furthermore, step S2 is implemented through the following process: Step S201: Obtain maintenance event records. The specific process includes: Based on the target carrier-based aircraft model, several maintenance event records are obtained via the Internet. The maintenance event records include the maintenance event name, maintenance area, order and model name of maintenance tools and support equipment, and video of the maintenance process. The maintenance process videos include every action of the maintenance personnel, how to use maintenance tools, and the complete process of disassembling and assembling carrier-based aircraft components.

[0022] Step S202: Establish a maintenance step association network based on the maintenance event records. The specific process includes: A carrier-based aircraft area model and a maintenance model are established based on maintenance process videos. At the same time, the movement trajectory of the maintenance personnel's hands is identified in the maintenance process videos, and a corresponding maintenance dynamic model is constructed based on the time series. The maintenance dynamic model includes a carrier-based aircraft area model, a maintenance model, and a motion trajectory. The order in which maintenance tools and support equipment are called is used as the basis for division. When the video of the maintenance process detects that the previous maintenance tool or support equipment has been put down and a new maintenance tool or support equipment has been picked up and the operation of the maintenance tool or support equipment has begun, such as switching from a wrench to a multimeter to measure voltage; This means that a critical event node is identified, and the corresponding maintenance dynamic model is automatically divided into two maintenance step models based on the corresponding time segment. Each maintenance step model encapsulates the action sequence of all models within the critical event node (e.g., the entire process of measuring voltage with a multimeter) and its duration. Each of the defined maintenance steps is marked as a network node. The network node stores the associated maintenance tools or support equipment, maintenance area, and expected time (obtained through duration). The maintenance step models that correspond to the same maintenance area but use different maintenance tools or support equipment are matched and connected to each other. For example, after checking the line, it may branch to two subsequent steps using different tools, namely replacing the cable or tightening the interface, based on the inspection results. This constitutes a local regional maintenance step association network with decision branches. The regional maintenance step association network is used to describe a directed graph of all possible maintenance paths in each maintenance area. Based on the maintenance area corresponding to each maintenance step association network, the network is mapped onto the maintenance dynamic model in the form of a texture. For network nodes that are spatially adjacent or functionally dependent on each other on the maintenance dynamic model, such as disassembling the intake duct liner as a prerequisite for inspecting the compressor blades, the maintenance step association networks of the same location are matched with each other and connected by directed edges. Through the above matching process, all regional maintenance step association networks are integrated into a maintenance step association network.

[0023] Furthermore, step S3 is implemented through the following process: Step S301: Communicate and interact with the carrier-based aircraft maintenance simulation space through VR equipment. The specific process includes: The VR devices include visual VR devices (mainly VR head-mounted displays) and interactive VR devices (mainly VR controllers and data gloves). Visual data nodes are created for the visual VR devices. The visual data nodes are used to manage the user's field of vision data, rendering instructions and graphics load. Interactive data intersections are created for each interactive VR device. The interactive data intersections are used to continuously track the interactive data such as the spatial pose (six degrees of freedom), button status, and gesture information of the interactive VR devices. When a user connects to the carrier-based aircraft maintenance simulation space via VR device, the visual data node obtains the user's current visual interaction range in real time. The visual interaction range is a visual cone with a certain opening angle and subtraction angle, starting from the user's eyes, and is used to represent the spatial area that the user can associate with at the current moment. Starting from the spatial coordinates of the visual VR device projected in the carrier aircraft maintenance simulation space, the visual interaction range is projected into the carrier aircraft maintenance simulation space. Then, the visual data node broadcasts a communication to all simulation data nodes and visual data nodes within the visual interaction range. The simulation data node that receives the broadcast signal will perform collision detection to determine whether its associated model area is within the visual interaction range. If the simulation data node determines that the model region it is associated with is within the visual interaction range, it immediately establishes a high-speed data channel with the visual data node and the interaction data node; otherwise, it does not perform any operation. Through high-speed data channels, simulation data nodes send all the data contained in their associated model regions to visual data nodes and interactive data intersections. Then, based on the received data, visual data nodes and interactive data intersections establish a local carrier-based aircraft maintenance simulation space in the VR device. This ensures that the device's computing resources are only used to render the area within the user's field of vision, greatly optimizing performance and thus supporting the smooth operation of ultra-large-scale scenes to a certain extent.

[0024] Step S302: Collect user interaction data. The specific process includes: Each interactive VR device is instantiated as a corresponding high-fidelity 3D animation model in the local carrier-based aircraft maintenance simulation space, and is denoted as an interactive simulation model. For example, the VR controller is mapped as a virtual 3D animation model of a hand with joint structure. During the user's mobile interaction with the VR device, the corresponding interaction data intersection point collects the dynamic spatial position (X, Y, Z spatial coordinates) and rotational attitude (pitch, yaw, roll angle) of the interactive VR device, and dynamically updates the interactive simulation model based on the dynamic spatial position and rotational attitude, so that the interactive simulation model and the interactive VR device make synchronized movements. Centered on an interactive simulation model (such as a virtual hand), an invisible and tiny interactive association range is set for it. The interactive association range represents the operating force field or sensitive area of ​​the maintenance tool or support equipment. Its shape and size can be configured according to different tools. For example, it may be a sphere around the hand when grasping, or a tiny cylinder with a pointed tip when using a screwdriver. Continuously perform physical collision detection to determine whether the interaction association range of the interactive simulation model intersects with any maintenance model (such as a wrench or multimeter) or carrier-based aircraft area model (such as a bolt that needs to be tightened or a hatch that needs to be opened). If an intersection is determined, the operation will not be triggered immediately, and the final interaction intention will be confirmed. The confirmation signal comes from a specific input of the interactive VR device, such as the user pressing the grip button or the data glove recognizing a specific grip gesture. After the interaction intention is confirmed, an interactive action is established between the interactive simulation model and the target maintenance model. For example, the virtual hand grabs the virtual wrench, and the movement of the wrench will completely follow the movement of the hand. Once the interactive actions between the interactive simulation model and the target maintenance model are established, the gesture trajectory (including movement path, speed, and acceleration) of the interactive simulation model and the interactive actions of all models bound to it are recorded through the intersection of interactive data. The user interaction process is obtained by using gesture trajectories and the spatial relationship between the interactive simulation model and the target maintenance model. The user interaction process includes the operation object (interactive simulation model and target maintenance model), the maintenance area, and the action type. For example, a user, using a gripping motion and a 12-gauge box wrench, performs a 35-degree clockwise rotation on the engine's mounting bolt #3.

[0025] Step S303: Input the user interaction action into the maintenance step association network. The specific process includes: The user interaction process is input into the maintenance step association network. Based on the model name and maintenance area associated with the maintenance model and the carrier-based aircraft area model corresponding to the user interaction action, the corresponding regional maintenance step association network is matched in the maintenance step association network. Then, a special reference scheme is generated based on the subsequent maintenance step model of the regional maintenance step association network. The process of generating the specific reference scheme includes: Retrieve several maintenance event records corresponding to the maintenance procedure model, and establish maintenance interactive animation based on the maintenance event records. The maintenance interactive animation consists of a maintenance model, a carrier-based aircraft area model, and an interactive simulation model. Several interactive recognition points are set at the interactive location composed of the maintenance model, the carrier-based aircraft area model, and the interactive simulation model. Based on the change trajectory of each model in the maintenance interactive action animation, the recognition point trajectory corresponding to the interactive recognition point is generated. Map the generated identification point trajectories corresponding to different maintenance event records onto the same two-dimensional coordinate system, set multiple time intervals of equal length, perform normal distribution on the identification point trajectory segments under each time interval, and obtain the corresponding standard trajectory interval based on the normal distribution results; By sequentially connecting the various time segments, a set of standard identification point trajectories for retrieving maintenance steps is obtained. Then, the maintenance step model and the corresponding set of standard identification point trajectories are integrated into a special reference scheme.

[0026] Step S304: Determine whether the user interaction process is compliant through a specific reference solution. The specific process includes: The standard identification point trajectory set in the special reference scheme is compared with the real-time collected user interaction process trajectory in the same spatiotemporal coordinate system at the millisecond level. If the user's interaction trajectory is within the standard trajectory range, the VR device will provide positive feedback (such as a green highlight on the visual VR device or an auditory confirmation sound) and prepare to move on to the next step. If the user interaction process trajectory is not within the standard trajectory range, the sequence is incorrect, or the operation object is incorrect, it is immediately determined to be non-compliant, and the visual VR device will automatically trigger the playback of the correction video. The correction videos are usually presented in the form of picture-in-picture, semi-transparent overlay, or highlighted outline animation, which intuitively demonstrate the correct repair interaction animation of the current step in the non-interference area of ​​the visual interaction range, so as to realize hands-on teaching. When it is detected that the user has completed the current step (such as successfully disassembling an old part) and established an interaction relationship with the next new repair model (such as a new part or installation tool), the special reference plan is immediately and dynamically updated. The evaluation and guidance standards are switched to the new repair steps according to the special reference plan until the user completes the entire repair event independently and in a standardized manner.

[0027] Please see Figure 2 As shown, a virtual reality-based simulation system for shipborne aircraft maintenance includes a simulation space construction module, a maintenance data analysis module, and a user interaction module. The simulation space construction module is used to collect environmental perception data packets of the carrier-based aircraft and maintenance environment through a multimodal sensor array, and to construct a carrier-based aircraft maintenance simulation space based on the environmental perception data packets; The maintenance data analysis module is used to establish a regional maintenance step association network under different maintenance events based on maintenance event records, map the regional maintenance step association network onto the surface of the carrier-based aircraft maintenance simulation space, and then match the various regional maintenance step association networks to obtain the maintenance step association network. The user interaction module is used to communicate and interact with the carrier-based aircraft maintenance simulation space through VR devices, collect user interaction processes through VR devices, input user interactions into the maintenance step association network, and then output a special reference solution to determine whether the user interaction process is compliant.

[0028] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A simulation method for shipborne aircraft maintenance process based on virtual reality, characterized in that, Includes the following steps: Step S1: Collect environmental perception data packets of the carrier-based aircraft and maintenance environment through a multimodal sensor array, and construct a carrier-based aircraft maintenance simulation space based on the environmental perception data packets; Step S2: Obtain maintenance event records, establish regional maintenance step association networks under different maintenance events based on maintenance event records, map the regional maintenance step association networks onto the surface of the carrier-based aircraft maintenance simulation space, and then match the various regional maintenance step association networks to obtain the maintenance step association network. Step S3: The user communicates and interacts with the carrier-based aircraft maintenance simulation space through VR equipment, and the user interaction process is collected through VR equipment. The user interaction process is input into the maintenance step association network, and then the corresponding special reference solution is output to determine whether the user interaction process is compliant.

2. The simulation method for carrier-based aircraft maintenance process based on virtual reality according to claim 1, characterized in that, The process of acquiring environmental awareness data packets of the carrier-based aircraft and its maintenance environment through multimodal sensors and constructing a carrier-based aircraft maintenance simulation space includes: Several multimodal sensor arrays are installed in the carrier-based aircraft and maintenance environment. Environmental awareness data packets of the carrier-based aircraft and maintenance environment are collected from different angles through the multimodal sensor arrays. The maintenance environment includes maintenance tools and support equipment. The environmental perception data packet includes point cloud data and device image data; The data in the environmental awareness data packet is spatially aligned, and then a carrier-based aircraft maintenance simulation space is constructed based on the spatial alignment results. Several uniformly distributed simulation data nodes are set in the carrier-based aircraft maintenance simulation space, and each simulation data node is associated with a part of the carrier-based aircraft maintenance simulation space. The carrier-based aircraft maintenance simulation space consists of a carrier-based aircraft area model and a maintenance model.

3. The simulation method for shipborne aircraft maintenance process based on virtual reality according to claim 2, characterized in that, The process of establishing a network of relationships between maintenance steps based on maintenance event records includes: The maintenance event record includes the maintenance event name, maintenance area, order and model name of maintenance tools and support equipment, and video of the maintenance process; A maintenance dynamic model is established based on the maintenance process video in the maintenance event record. The maintenance dynamic model is divided into several maintenance step models according to the order of calling maintenance tools and support equipment. Maintenance step models that correspond to the same maintenance area but have different maintenance tools or support equipment called later are matched and connected to each other, thereby obtaining the regional maintenance step association network under the corresponding maintenance event name. Based on the maintenance regions corresponding to the maintenance step association networks of each region, the regional maintenance step association networks are mapped onto the maintenance dynamic model. The regional maintenance step association networks that are associated with the same location on the maintenance dynamic model are matched with each other, and the maintenance step association network is obtained based on the matching results.

4. The simulation method for carrier-based aircraft maintenance process based on virtual reality according to claim 3, characterized in that, The process of users communicating and interacting with the carrier-based aircraft maintenance simulation space through VR devices includes: The VR devices include visual VR devices and interactive VR devices, and visual data nodes and interactive data nodes are set for various VR devices; After the user directly communicates with the carrier-based aircraft maintenance simulation space through VR devices, the visual data node first obtains the visual interaction range of the visual VR device and communicates with the simulation data node within the visual interaction range. Then, the simulation data node shares its associated model area with the intersection of the visual data node and the interaction data point.

5. The simulation method for carrier-based aircraft maintenance process based on virtual reality according to claim 4, characterized in that, The process of collecting user interaction data through VR devices includes: Interactive VR devices are mapped onto the carrier-based aircraft maintenance simulation space in the form of animated models, and are denoted as interactive simulation models. Centered on the interactive simulation model, the interactive association range is set for the interactive simulation model. During the user's movement of the interactive VR device, the interactive data intersection point obtains the dynamic spatial position of the interactive VR device and transmits it to the nearest interactive data intersection point, thereby dynamically updating the interactive simulation model in the carrier aircraft maintenance simulation space. Simultaneously, it is determined whether the maintenance model exists within the interactive simulation model's interaction association range. Based on the determination result, interactive actions are set between the corresponding maintenance model and the interactive simulation model. Then, based on the gesture trajectory of the interactive VR device, the user interaction process between the maintenance model and the carrier-based aircraft area model is obtained.

6. The simulation method for carrier-based aircraft maintenance process based on virtual reality according to claim 5, characterized in that, The process of inputting user interaction actions into the maintenance procedure association network includes: Based on the maintenance model corresponding to the user interaction action and the model name and maintenance area corresponding to the carrier-based aircraft area model, the corresponding regional maintenance step association network is matched in the maintenance step association network, and a special reference scheme is generated based on the subsequent maintenance step model of the regional maintenance step association network. The system uses a specific reference solution to determine whether the user interaction process is compliant. Based on the determination, it generates corresponding correction videos. At the same time, when the user interacts with the new repair model, the specific reference solution is dynamically updated until the user completes the entire repair event.

7. The simulation method for shipborne aircraft maintenance process based on virtual reality according to claim 6, characterized in that, The process of generating the specific reference scheme includes: Retrieve several maintenance event records corresponding to the maintenance step model, and establish maintenance interactive animation based on the maintenance event records. The maintenance interactive animation consists of a maintenance model, a carrier-based aircraft area model, and an interactive simulation model. Several interactive recognition points are set at the interactive location composed of the maintenance model, the carrier-based aircraft area model, and the interactive simulation model. Based on the change trajectory of each model in the maintenance interactive action animation, the recognition point trajectory corresponding to the interactive recognition point is generated. Map the generated identification point trajectories corresponding to different maintenance event records onto the same two-dimensional coordinate system, set multiple time intervals of equal length, perform normal distribution on the identification point trajectory segments under each time interval, and obtain the corresponding standard trajectory interval based on the normal distribution results; By sequentially connecting the various time segments, a set of standard identification point trajectories for retrieving the corresponding maintenance step model is obtained. Then, the maintenance step model and the corresponding set of standard identification point trajectories are integrated into a special reference scheme.

8. A virtual reality-based simulation system for carrier-based aircraft maintenance processes, used to implement the virtual reality-based simulation method for carrier-based aircraft maintenance processes as described in any one of claims 1-7, characterized in that, It includes a simulation space construction module, a maintenance data analysis module, and a user interaction module; The simulation space construction module is used to collect environmental perception data packets of the carrier-based aircraft and maintenance environment through a multimodal sensor array, and to construct a carrier-based aircraft maintenance simulation space based on the environmental perception data packets; The maintenance data analysis module is used to establish a regional maintenance step association network under different maintenance events based on maintenance event records, map the regional maintenance step association network onto the surface of the carrier-based aircraft maintenance simulation space, and then match the various regional maintenance step association networks to obtain the maintenance step association network. The user interaction module is used to communicate and interact with the carrier-based aircraft maintenance simulation space through VR devices, collect user interaction processes through VR devices, input user interactions into the maintenance step association network, and then output a special reference solution to determine whether the user interaction process is compliant.

Images