Aircraft refueling virtual-real combination practical training method and platform based on AR and scrapped equipment

Abstract

The invention provides an aircraft refueling virtual-real combined practical training method and platform based on AR and scrapped equipment, and relates to the technical field of aircraft refueling training, and the method comprises the steps that a processing module loads digital twin scene data according to a training subject; the AR display module carries out space registration and superposition on the virtual airplane model, the virtual instrument interface and real scrapped refueling equipment; collecting operation action data of the trainee in real time; inputting the operation action data into the digital twin model, updating the state information of the virtual refueling operation, and sending the state information to an AR (Augmented Reality) display module; the AR display module displays the state information in an overlapping manner; when the butt joint operation is carried out, the relative position and posture of the physical joint and the physical refueling panel are monitored in real time, the correctness of the operation is judged, and the result is displayed through the AR display module; and the processing module records, analyzes and generates a quantitative evaluation report. The method can effectively solve the problems that pure virtual training lacks sense of reality, and pure physical training is high in cost and high in risk.

Description

Technical Field

[0001] This invention relates to the field of aircraft refueling training technology, specifically to a virtual-real combined training method and platform for aircraft refueling based on AR and scrapped equipment. Background Technology

[0002] Currently, in the field of aviation ground support training, the skills training of aircraft refueling operators mainly relies on two modes: First, using real aircraft, refueling trucks and aviation fuel for hands-on training. Although this mode can ensure the realism of the operation, it has the problems of high cost, huge safety risks and serious resource consumption. Second, using pure computer virtual simulation software for training. Although this mode reduces costs and risks, it lacks the real physical operation feel, and the training immersion and effect are not good.

[0003] However, all of the aforementioned existing technologies have significant limitations. Purely hands-on training is difficult to conduct frequently and cannot safely simulate various malfunctions and emergencies; while purely virtual training is disconnected from the real working environment, and trainees cannot acquire muscle memory for operating real equipment.

[0004] Therefore, there is an urgent need for a new type of practical training program that can balance the realism of hands-on practice with the depth of training while ensuring safety and controllability. Summary of the Invention

[0005] To address the technical problems in related technologies, this invention provides a virtual-real combined training method and platform for aircraft refueling based on AR and scrapped equipment.

[0006] To achieve the above objectives, the technical solution adopted by the present invention includes: According to a first aspect of the present invention, a virtual-real combined training method for aircraft refueling based on AR and decommissioned equipment is provided, comprising: Step S1: The processing module loads the corresponding digital twin scene data according to the selected training subject and sends it to the AR display module; the AR display module recognizes the visual markers set on the physical entity module and spatially registers and overlays the virtual aircraft model, virtual instrument interface and real scrapped refueling equipment to form a training scene that combines virtual and real elements. Step S2: By setting up sensor groups on key operating components of the physical entity module, the system collects real-time data on the student's actions on the actual physical components and sends the data to the processing module. Step S3: The processing module inputs the received operation data into the digital twin model for calculation, updates the status information of the virtual refueling operation, and sends the status information to the AR display module in real time; the AR display module overlays the status information on the real scene, and the status information includes virtual instrument readings, virtual fuel flow effects, and virtual equipment actions; Step S4: When performing the docking operation, the relative position and attitude of the physical joint and the physical refueling panel are monitored in real time by the pose sensor. The processing module compares the relative position and attitude with the preset virtual docking data in the digital twin model, determines the correctness of the operation, and displays the result through the AR display module. Step S5: The processing module records and analyzes all operational action data and docking operation judgment results collected throughout the training process, and generates a quantitative evaluation report.

[0007] Optionally, step S4 may further include: When an emergency response is performed, the AR display module presents the corresponding virtual fault phenomena and alarm information. The pose sensor monitors the relative position and attitude of the corresponding real physical components in real time. The processing module compares the relative position and attitude with the virtual emergency response data preset in the digital twin model, determines the correctness of the operation, and displays the result through the AR display module.

[0008] Optionally, the processing module receives instructions from the instructor or emergency response procedures automatically triggered according to preset logic, and records the emergency response data performed by the trainee on the actual physical components.

[0009] Optionally, in step S5, the emergency response operation includes simulating one or more of the following: fuel leak, abnormal pressure, electrostatic alarm, and connector failure.

[0010] Optionally, before step S1, the process may include a step of harmlessly treating and adapting the scrapped aircraft refueling equipment, wherein the adaptation includes installing the sensor group and visual markers.

[0011] Optionally, in step S3, the visual feedback also includes presenting step-by-step operation guidance information through the AR display module. The operation guidance information includes any one or more of three-dimensional arrows, highlighted borders, and text prompts.

[0012] According to a second aspect of the present invention, an AR-based virtual-real combined training platform for aircraft refueling with scrapped equipment is also provided, for implementing the AR-based virtual-real combined training method for aircraft refueling with scrapped equipment as described in any of the technical solutions of the first aspect of the present invention, wherein the AR-based virtual-real combined training platform for aircraft refueling with scrapped equipment includes: The physical entity module includes partially decommissioned aircraft refueling equipment that has undergone harmless treatment and adaptive modification. The aircraft refueling equipment includes a decommissioned refueling vehicle body and a decommissioned aircraft refueling panel, which are equipped with sensor groups and visual markers. AR display module is used to display scenes that combine virtual and real elements, operation guidance, and feedback information; The processing module is communicatively connected to the physical entity module and the AR display module. It internally constructs a digital twin model of the aircraft refueling operation and is used to process sensor data, drive the virtual scene, execute operation logic, and generate an evaluation report. The sensor group includes motion sensors mounted on the valves, joints, and reels of the physical entity module, as well as pose sensors for monitoring spatial position.

[0013] Optionally, the AR-based virtual and real-world training platform for aircraft refueling with scrapped equipment also includes a coach terminal for selecting training subjects, triggering fault simulations, and viewing evaluation reports.

[0014] Optionally, the motion sensor is a rotary encoder or an inertial measurement unit, and the pose sensor is set as an ultra-wideband (UWB) positioning module or a visual tracking camera.

[0015] Optionally, the visual marker is a QR code or an ARUco marker.

[0016] Beneficial effects: 1. Through the above technical solution, firstly, this invention can achieve the construction of a low-cost, highly immersive training environment. Specifically, this invention creates a composite training scenario that includes both real physical entities and virtual extended elements by spatially registering and overlaying virtual aircraft models, virtual instrument interfaces, and real decommissioned refueling equipment. This not only effectively reduces costs (using decommissioned equipment as the physical core significantly reduces the enormous costs of purchasing and maintaining a complete set of operational real equipment), but also effectively enhances realism and immersion (compared to pure virtual simulation, trainees face and operate real refueling truck mechanical structures, real connectors and valves, gaining real tactile feedback, force feedback, and spatial awareness, effectively solving the problem of the disconnect between pure virtual training and the real working environment). Furthermore, by loading different digital twin scenario data, the same set of decommissioned physical equipment can simulate service for different aircraft models (only the virtual aircraft model and instrument interface need to be changed), increasing the coverage of training, which is difficult to achieve with pure physical training.

[0017] Secondly, this invention enables operation-driven, dynamically visualized process simulation. Steps S2 and S3 form a real-time closed loop of "physical operation - data acquisition - model calculation - virtual feedback." This method effectively ensures safety and environmental friendliness. The trainee's operation on real physical components (such as turning a handwheel) is collected by sensors and fed into a digital twin model for calculation, ultimately presented as virtual fuel flow effects and virtual instrument readings. This means the entire refueling process (including high-risk core aspects such as fuel flow and pressure changes) is safely conducted in a virtual environment, achieving "dry-expansion" training and completely eliminating the safety risks and environmental pollution caused by real fuel. Moreover, this method also enhances training depth; the trainee's operations immediately trigger changes in the virtual world's state. This instantaneous, visualized causal feedback allows trainees to deeply understand the relationship between operation and system state, improving training depth and comprehension, far exceeding the effects of static drawing instruction or screen operation detached from physical objects.

[0018] Third, the method of this invention enables objective and precise evaluation of key operational steps. Specifically, for the most critical docking operation in aircraft refueling, step S4 provides an automated detection and evaluation mechanism. In this way, the method of this invention can effectively improve training standards and accuracy. By comparing real-time monitored posture data with preset virtual docking data, the system can objectively and quantitatively determine the correctness of the operation (such as whether the docking angle and position meet the standards), avoiding the potential biases that may exist in the subjective judgment of instructors in traditional teaching. Moreover, the instant feedback generated by the method of this invention can promote skill formation. The judgment results are fed back to the trainee in real time through the AR display module, enabling them to immediately recognize and correct errors, accelerating the formation of correct operational muscle memory.

[0019] Fourth, the method of this invention can also achieve digital and quantifiable evaluation of the training process and results. Specifically, the method of this invention records all operational action data and docking results, and generates a quantitative evaluation report based on this. This provides objective and consistent data support for evaluating trainees' skill levels, overcoming the limitations of traditional evaluation methods that rely on the coach's personal experience. At the same time, the quantitative data can clearly reveal trainees' operational weaknesses (such as excessive time spent on a certain step or low docking success rate), facilitating targeted reinforcement training.

[0020] In summary, the method of this invention, through the organic combination of five steps, can solve the problem of the lack of realism in purely virtual training by utilizing decommissioned equipment, while simultaneously addressing the issues of high cost and risk associated with purely physical training through AR and digital twin technologies. Ultimately, at the methodological level, it constructs a new paradigm for aircraft refueling training that is safe, low-cost, highly immersive, and quantifiable.

[0021] 2. Other beneficial effects or advantages of the present invention will be described in detail in the specific embodiments. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] in: Figure 1 This is a flowchart illustrating the steps of an exemplary embodiment of the present invention for a virtual-real combined training method for aircraft refueling based on AR and decommissioned equipment; Figure 2 This is a schematic diagram of the layout of an AR-based virtual and real training platform for aircraft refueling with scrapped equipment, provided in an exemplary embodiment of the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0025] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0026] Furthermore, the terms "comprising" and "having," and any variations thereof, used in the description of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices. It should also be noted that in embodiments of this invention, the words "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in embodiments of this invention should not be construed as preferred or advantageous over other embodiments or designs. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0027] To facilitate a clearer and more accurate understanding of the technical solutions of this invention by those skilled in the art, the existing related technologies and their technical problems will be described in more detail below.

[0028] In the aviation ground support system, the operational skills and emergency response capabilities of aircraft refueling personnel are directly related to aviation operational safety. Currently, skills training in this field mainly relies on the following two technical models, but each has its own insurmountable limitations.

[0029] Firstly, a purely hands-on training model.

[0030] This mode is conducted entirely in a real-world operational environment. For example, the training unit needs to deploy a real A320 passenger plane, a working aircraft refueling truck, and hundreds to thousands of liters of aviation kerosene. In this environment, trainees operate the real aircraft refueling panel, connect heavy physical refueling connectors, and manipulate the valves and instruments on the refueling truck to complete the entire pressure refueling process.

[0031] Its advantage lies in the fact that this mode can provide 100% realism in operation, and trainees can obtain complete physical feedback including the sense of power and touch, which helps to form deep muscle memory.

[0032] Its disadvantages are: high cost (the need to use real aircraft and refueling trucks would disrupt normal operations or maintenance schedules, resulting in huge opportunity costs. At the same time, aviation fuel consumption is also a significant direct expense), huge safety risks (aviation fuel is a highly hazardous chemical; any minor mistake in practice, such as improper static electricity handling or loose connections, could lead to fuel leaks or even fires and explosions, posing a serious threat to personnel, aircraft, and ground facilities), severe resource consumption (the aviation fuel and equipment wear required for training are one-time consumptions, which does not conform to the concept of green and sustainable training), and limited training scenarios (for absolute safety reasons, it is almost impossible to simulate various malfunctions and emergencies in this model. For example, instructors would never artificially create a real fuel leak accident for training purposes, leaving trainees lacking experience in handling critical emergencies).

[0033] Secondly, the pure virtual simulation training mode.

[0034] This mode takes place entirely in a computer-generated virtual environment. Students sit in front of a computer and use a mouse and keyboard (or a simple joystick) to control 3D virtual refueling truck and airplane models on the screen. For example, students "turn" virtual valves by clicking the mouse and "connect" virtual refueling connectors by dragging.

[0035] Its advantages are that this mode has extremely low cost and risk (no real equipment and fuel required, unlimited training sessions, and the ability to safely simulate any fault, such as simulating a virtual scenario of "leakage caused by a broken refueling connector seal" in the software) and is flexibly configurable (different aircraft models can be quickly switched through the software, such as switching from a Boeing 737 to an Airbus A380, and training subjects).

[0036] Its disadvantages lie in the lack of realistic physical operation feel (trainees cannot feel the weight of a real refueling connector, the force required to connect it, or the sound when locking it, nor can they experience the torque required to turn a large valve handwheel. This disconnect from the real physical world makes it impossible to form effective muscle memory and operational feel during training) and poor training immersion and effectiveness (because the operating interface is a two-dimensional screen and the interaction method is indirect mouse clicks, the immersion of training is greatly reduced. Trainees are more like playing a "computer game," and the transfer rate of training effects to real work scenarios is low).

[0037] In summary, existing technologies face a core contradiction: purely physical training is realistic but unsafe and uneconomical, while purely virtual training is safe but unrealistic and has limited effectiveness. These two models are like two ends of a technological scale, making it difficult to simultaneously ensure the physical realism of operations and the depth and breadth of training content while maintaining safety and controllable costs. Therefore, the aviation training field urgently needs a new practical training solution that can break this deadlock. This solution must provide physical operation training of real equipment in a controlled environment, while safely and flexibly covering various complex operating conditions, including malfunctions.

[0038] In view of this, the present invention provides a novel solution: a virtual-real integrated training method for aircraft refueling based on AR and decommissioned equipment. The technical concept of this invention lies in constructing a virtual-real integrated training environment by combining augmented reality (AR) technology with adaptively modified decommissioned physical equipment. This invention creatively utilizes decommissioned equipment to provide a realistic physical operational foundation, addressing the pain point of purely virtual training lacking tactile feedback and muscle memory. Simultaneously, AR technology digitally and virtually reproduces high-cost, high-risk elements (such as fuel, complete aircraft, complex instruments, and malfunction phenomena), enabling safe, low-cost, and highly realistic full-process operation and emergency response training in a "dry" (without actual fuel) state. Essentially, this invention seamlessly integrates real physical interaction with virtual dynamic simulation into a unified training system, thereby ensuring safety and reducing costs while combining the immersive experience of physical operation with the flexibility and depth of virtual training.

[0039] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings.

[0040] like Figure 1As shown, according to a first aspect of the present invention, a virtual-real combined training method for aircraft refueling based on AR and decommissioned equipment is provided, comprising: Step S1: The processing module loads the corresponding digital twin scene data according to the selected training subject and sends it to the AR display module; the AR display module recognizes the visual markers set on the physical entity module and spatially registers and overlays the virtual aircraft model, virtual instrument interface and real scrapped refueling equipment to form a training scene that combines virtual and real elements. Step S2: By setting up sensor groups on key operating components of the physical entity module, the system collects real-time data on the student's actions on the actual physical components and sends the data to the processing module. Step S3: The processing module inputs the received operation data into the digital twin model for calculation, updates the status information of the virtual refueling operation, and sends the status information to the AR display module in real time; the AR display module overlays the status information on the real scene, and the status information includes virtual instrument readings, virtual fuel flow effects, and virtual equipment actions; Step S4: When performing the docking operation, the relative position and attitude of the physical joint and the physical refueling panel are monitored in real time by the pose sensor. The processing module compares the relative position and attitude with the preset virtual docking data in the digital twin model, determines the correctness of the operation, and displays the result through the AR display module. Step S5: The processing module records and analyzes all operational action data and docking operation judgment results collected throughout the training process, and generates a quantitative evaluation report.

[0041] Through the above technical solution, firstly, this invention can achieve the construction of a low-cost, highly immersive training environment. Specifically, this invention creates a composite training scenario that includes both real physical entities and virtual extended elements by spatially registering and overlaying virtual aircraft models, virtual instrument interfaces, and real decommissioned refueling equipment. This not only effectively reduces costs (using decommissioned equipment as the physical core significantly reduces the enormous costs of purchasing and maintaining a complete set of operational real equipment), but also effectively enhances realism and immersion (compared to pure virtual simulation, trainees face and operate real refueling truck mechanical structures, real connectors and valves, gaining real tactile feedback, force feedback, and spatial awareness, effectively solving the problem of the disconnect between pure virtual training and the real working environment). Furthermore, by loading different digital twin scenario data, the same set of decommissioned physical equipment can simulate service for different aircraft models (only the virtual aircraft model and instrument interface need to be changed), increasing the coverage of training, which is difficult to achieve with purely physical training.

[0042] Secondly, this invention enables operation-driven, dynamically visualized process simulation. Steps S2 and S3 form a real-time closed loop of "physical operation - data acquisition - model calculation - virtual feedback." This method effectively ensures safety and environmental friendliness. The trainee's operation on real physical components (such as turning a handwheel) is collected by sensors and fed into a digital twin model for calculation, ultimately presented as virtual fuel flow effects and virtual instrument readings. This means the entire refueling process (including high-risk core aspects such as fuel flow and pressure changes) is safely conducted in a virtual environment, achieving "dry-expansion" training and completely eliminating the safety risks and environmental pollution caused by real fuel. Moreover, this method also enhances training depth; the trainee's operations immediately trigger changes in the virtual world's state. This instantaneous, visualized causal feedback allows trainees to deeply understand the relationship between operation and system state, improving training depth and comprehension, far exceeding the effects of static drawing instruction or screen operation detached from physical objects.

[0043] Third, the method of this invention enables objective and precise evaluation of key operational steps. Specifically, for the most critical docking operation in aircraft refueling, step S4 provides an automated detection and evaluation mechanism. In this way, the method of this invention can effectively improve training standards and accuracy. By comparing real-time monitored posture data with preset virtual docking data, the system can objectively and quantitatively determine the correctness of the operation (such as whether the docking angle and position meet the standards), avoiding the potential biases that may exist in the subjective judgment of instructors in traditional teaching. Moreover, the instant feedback generated by the method of this invention can promote skill formation. The judgment results are fed back to the trainee in real time through the AR display module, enabling them to immediately recognize and correct errors, accelerating the formation of correct operational muscle memory.

[0044] Fourth, the method of this invention can also achieve digital and quantifiable evaluation of the training process and results. Specifically, the method of this invention records all operational action data and docking results, and generates a quantitative evaluation report based on this. This provides objective and consistent data support for evaluating trainees' skill levels, overcoming the limitations of traditional evaluation methods that rely on the coach's personal experience. At the same time, the quantitative data can clearly reveal trainees' operational weaknesses (such as excessive time spent on a certain step or low docking success rate), facilitating targeted reinforcement training.

[0045] In summary, the method of this invention, through the organic combination of five steps, can solve the problem of the lack of realism in purely virtual training by utilizing decommissioned equipment, while simultaneously addressing the issues of high cost and risk associated with purely physical training through AR and digital twin technologies. Ultimately, at the methodological level, it constructs a new paradigm for aircraft refueling training that is safe, low-cost, highly immersive, and quantifiable.

[0046] In one embodiment of the present invention, step S4 may further include: when performing an emergency response operation, the AR display module presents the corresponding virtual fault phenomenon and alarm information, the pose sensor monitors the relative position and attitude of the corresponding real physical component in real time, the processing module compares the relative position and attitude with the preset virtual emergency response data in the digital twin model, determines the correctness of the operation, and displays the result through the AR display module.

[0047] Firstly, this implementation method enables high-risk emergency response training under absolutely safe conditions. This allows trainees to face visually impactful and urgency-inducing virtual hazards in a "dry" (without actual hazardous materials) physical environment. This fundamentally eliminates the significant safety and equipment damage risks associated with simulating real-world malfunctions in traditional physical training.

[0048] Secondly, this implementation method can provide objective and precise assessment standards for emergency response procedures. Specifically, it transforms complex operations like emergency response, which typically rely on instructors' subjective experience, into a quantifiable data comparison process. For example, the system can accurately determine whether a trainee rotated a real valve to the correct physically closed position during a simulated valve-closing operation, or whether they moved a real connector to a safe distance during a simulated disconnection operation. This objective judgment based on sensor data and pre-set logic effectively improves the accuracy and fairness of the assessment, providing a technical basis for high-standard emergency skills certification.

[0049] Third, compared to handling malfunctions by clicking a mouse in purely virtual simulation software, this method requires trainees to perform emergency operations on real, heavy physical components. This physical operation driven by realistic virtual emergencies not only trains trainees' thinking, judgment, and process execution abilities, but also more effectively simulates the psychological pressure under accident conditions, trains trainees' fine motor control and operational feel under tension, thereby cultivating their stable psychological qualities and reliable muscle memory.

[0050] In one embodiment of the present invention, the emergency response operation process of the present invention can be triggered by the processing module receiving instructions from the instructor, or it can be automatically triggered according to preset logic. At the same time, the processing module records the emergency response operation data performed by the trainee on the real physical components.

[0051] This implementation method achieves flexibility and controllability in training modes and teaching strategies. Specifically, regarding instructor-side command triggering, it grants instructors a high degree of control over instruction. Instructors can intervene and trigger faults at the most appropriate moment (e.g., when an instructor's operation becomes slightly lax) based on the trainee's actual level and training progress, thereby conducting targeted reinforcement training or stress testing. This human controllability is crucial for implementing tiered and personalized advanced teaching strategies. For pre-set logic automatic triggering, the training system can operate independently, simulating the randomness and suddenness of faults in real-world operations. This avoids the psychological preparation required for trainees to anticipate the timing of faults, better testing their instinctive emergency response capabilities and operational proficiency, greatly enhancing the realism and challenge of the training.

[0052] In one embodiment of the present invention, in step S5, the emergency response operation includes simulating one or more of the following: fuel leak, abnormal pressure, electrostatic alarm, and connector failure.

[0053] In this way, this implementation method can achieve comprehensive training on various mechanisms and handling procedures, from fluid leakage (fuel leakage), abnormal system parameters (pressure abnormality), safety procedure triggering (static electricity alarm) to mechanical connection failure (connector failure), so that trainees can receive differentiated and professional training on different fault mechanisms.

[0054] In this implementation, it should be noted that training for fuel leaks focuses on quickly cutting off the source (closing valves) and controlling the spread. Training for abnormal pressure focuses on monitoring instruments to identify risks and depressurizing or adjusting the system. Training for electrostatic alarms focuses on immediately stopping operations and checking grounding lines, among other safety procedures. Training for connector failure focuses on the mechanical operation of safely disconnecting and replacing the equipment.

[0055] In one embodiment of the present invention, before step S1, there is a step of harmlessly treating and adapting the scrapped aircraft refueling equipment, the adaptation including installing sensor groups and visual markers.

[0056] In one embodiment of the present invention, in step S3, the visual feedback further includes presenting step-by-step operation guidance information through the AR display module. The operation guidance information includes any one or more of three-dimensional arrows, highlighted borders, and text prompts.

[0057] This approach not only enables immersive and intuitive operational guidance, significantly lowering the learning threshold (3D arrows and highlighted borders precisely draw trainees' visual attention to specific operational components of the real physical equipment, while text prompts, such as "rotate 90 degrees counterclockwise" or "lift to the locked position," convey precise operational actions and quantifiable requirements), but also establishes a reinforced training mechanism for standard operating procedures, ensuring the standardization and consistency of operations (the system can guide trainees step-by-step according to the optimal operational sequence. For example, after a trainee completes the grounding operation, the system automatically highlights the next key component, such as the refueling connector, and displays an arrow for guidance. This ensures that every trainee receives uniform and standardized training content, fundamentally improving the overall quality and standardization of training and reducing operational deviations caused by differences in individual instructors).

[0058] According to a second aspect of the invention, such as Figure 2 As shown, an AR-based virtual-real combined training platform for aircraft refueling with scrapped equipment is also provided, which is used to implement the AR-based virtual-real combined training method for aircraft refueling with scrapped equipment according to any of the technical solutions in the first aspect of the present invention. The AR-based virtual-real combined training platform for aircraft refueling with scrapped equipment includes a physical entity module, an AR display module, a processing module, and a sensor group.

[0059] The physical entity module includes partially decommissioned aircraft refueling equipment that has undergone harmless treatment and adaptive modifications. The aircraft refueling equipment comprises a decommissioned refueling vehicle body and a decommissioned aircraft refueling panel, equipped with sensor arrays and visual markers. The AR display module is used to display a hybrid virtual-real scene, operation guidance, and feedback information. The processing module communicates with both the physical entity module and the AR display module. Internally, it constructs a digital twin model of the aircraft refueling operation and is used to process sensor data, drive the virtual scene, execute operational logic, and generate evaluation reports. The sensor array includes motion sensors mounted on valves, connectors, and reels within the physical entity module, as well as pose sensors for monitoring spatial position.

[0060] In this way, a complete system solution for virtual-real combined training on aircraft refueling can be constructed at the device level. It is not simply a matter of piecing together existing equipment, but rather a deep integration of low-cost scrapped hardware, advanced AR display technology, sensor networks, and intelligent digital twin software through an innovative modular architecture. This creates a dedicated training platform with clear functional division, significant cost advantages, highly flexible training content, and an interactive closed loop.

[0061] In one embodiment of the present invention, the motion sensor can be configured as a rotary encoder or an inertial measurement unit, and the pose sensor can be configured as an ultra-wideband (UWB) positioning module or a visual tracking camera.

[0062] In this embodiment, rotary encoders are particularly suitable for measuring rotational motion. When used to modify components such as valve handwheels and reel handles, they can directly and accurately measure the angle, number of rotations, and speed. This measurement method has strong anti-interference capabilities, provides direct and high-precision data, and can provide the most critical flow and pressure control input parameters for digital twin models, ensuring the accuracy and realism of virtual simulation responses. Inertial Measurement Units (IMUs), integrating gyroscopes and accelerometers, excel at measuring an object's attitude angles, angular velocities, and linear accelerations. When used for components requiring monitoring of movement and attitude in three-dimensional space (such as handheld refueling connectors), they can effectively track complex movements such as waving, lifting, and lowering, compensating for the shortcomings of pure rotational measurements. Ultra-wideband (UWB) positioning modules offer advantages such as strong penetration, good anti-multipath interference capabilities, and high positioning accuracy. Even in situations where visual obstruction, lighting changes, or AR marker damage may occur in the training environment, UWB can still provide stable and reliable centimeter-level positioning data. This ensures the continuity and reliability of docking operation decisions, greatly improving the system's robustness. The technological advantage of visual tracking cameras lies in their extremely high precision and ability to directly acquire rich visual information (such as posture and images). Working in conjunction with the camera built into AR glasses, they can achieve marker-based or markerless tracking, providing extremely accurate posture and angle information, making them ideal for docking guidance and judgment scenarios where absolute accuracy is required.

[0063] In one embodiment of the invention, the visual marker is a QR code or an ARUco marker. QR codes and ARUco markers have unique, high-contrast geometric patterns with very distinct corner and boundary features. This allows the AR glasses' camera and algorithms to calculate the marker's position and orientation (i.e., six degrees of freedom pose) relative to the camera with extremely high precision. This sub-centimeter and sub-degree-of-freedom positioning accuracy is the technical foundation for ensuring that virtual aircraft models and virtual instruments can stably "fit" onto real, decommissioned equipment, effectively avoiding problems such as virtual object drift and jitter, thereby guaranteeing the realism and credibility of the immersive training experience.

[0064] The present invention will be further described below with reference to an exemplary embodiment.

[0065] Example Title: Practical Training on Handling Pressure Refueling Faults of A320 Aircraft Based on AR and Decommissioned Equipment Platform preparation: The physical module consists of a decommissioned aircraft refueling truck that has undergone thorough cleaning (complete cleaning of fuel lines and removal of all flammable materials) (retaining the basic cockpit control structure, reel, and hoses) and a real refueling panel assembly (including the refueling port cap and connector) removed from a decommissioned A320 wing. Rotary encoders (motion sensors) are installed on the valve handwheels and reel handles of the refueling truck, UWB positioning tags (attitude sensors) are installed around the refueling connector and the aircraft refueling port, and ARUco markings (visual markings) are affixed to the equipment surfaces.

[0066] AR display module: Students wear Microsoft HoloLens 2 AR glasses.

[0067] Processing module: A local high-performance server, in which a digital twin model of A320 aircraft pressure refueling is built. This model includes a virtual A320 fuselage, virtual fuel, virtual pipeline pressure / flow data, and a fault database (containing fault logic such as "fuel leak").

[0068] Coach terminal: Coaches use a tablet computer to select subjects, trigger faults, and view reports.

[0069] Practical training execution process: Step S1: System Initialization and Scene Loading The instructor selects the training subject, "A320 Aircraft Standard Pressure Refueling and Troubleshooting," via a tablet computer (instructor terminal). The processing module then loads the corresponding digital twin scene data and sends it to the trainee's AR glasses.

[0070] Trainees, wearing AR glasses, focused on the refueling panels of a refueling truck and an aircraft. The AR glasses' camera recognized the ARUco logo on them, completing spatial registration. Subsequently, in the trainees' field of vision, a complete virtual A320 aircraft wing model was precisely overlaid on the real, decommissioned aircraft's refueling panel, as if a complete aircraft were right in front of them. Simultaneously, a virtual, interactive instrument panel and button interface were overlaid on the real decommissioned refueling truck's control panel. A "virtual-real hybrid training scenario" was thus constructed.

[0071] Step S2: Real-time operation data acquisition Following the procedures, the trainees approached the refueling truck and began operation. They manually turned the actual valve handwheel. A rotary encoder mounted on the handwheel shaft collected real-time rotation angle data (e.g., the valve handwheel rotated 270 degrees).

[0072] The operation data (rotation 270 degrees) is sent to the processing module in real time.

[0073] Step S3: Digital Twin Computation and Virtual-Real Interconnected Visual Feedback The processing module inputs the received "valve rotates 270 degrees" data into the digital twin model for calculation.

[0074] Example of calculation process: The digital twin model presets the maximum valve opening (900 degrees) to correspond to the maximum flow rate (1000 liters / minute). According to the formula: Current flow rate = (Current angle / Maximum angle) * Maximum flow rate, the current flow rate is calculated to be (270 / 900) * 1000 = 300 liters / minute. The model updates the pressure and flow status in the virtual pipeline accordingly, causing the virtual fuel gauge reading to begin to rise.

[0075] The updated status information (including the virtual instrument reading changing to 300L / min, the animation of the virtual fuel starting to flow, and the animation of the virtual valve rotating synchronously) is sent back to the AR glasses in real time.

[0076] The AR display module overlays this information onto the real-world scene. Meanwhile, to guide trainees to the next step, the AR glasses point to the refueling connector with a 3D arrow and a highlighted border (operation guidance information), and display the text: "Please lift the refueling connector to align with the aircraft's refueling port."

[0077] Step S4: Critical Operation Determination (Door-to-Door and Emergency Response) Step S4.1: Dock Operation Determination The trainee brought out a real refueling connector. The UWB pose sensor continuously monitors the relative position and attitude between the connector and the aircraft refueling port (e.g., returning three-dimensional coordinates (x,y,z) and pitch / yaw angles).

[0078] The processing module compares this real-time data with the preset virtual docking data (i.e., the correct docking area spatial range and attitude tolerance) in the digital twin model.

[0079] Example of calculation process: The preset conditions for successful docking are: the distance d between the center point of the connector and the center point of the filler port is less than 5cm, and the attitude angle deviation θ is less than 10 degrees. The system calculates that d=3cm and θ=5 degrees in real time, which meets the conditions, and determines that "docking is successful". A green "docking successful" prompt and a virtual oil pipe connection animation are displayed in the AR field of view.

[0080] Step S4.2: Emergency Response Operation Judgment During the simulated refueling process, the instructor manually triggered a "fuel leak" fault simulation through the instructor's terminal.

[0081] Upon receiving the instruction, the processing module immediately simulates the fault in the digital twin model and sends the instruction to the AR glasses.

[0082] The AR display module immediately presents the corresponding virtual fault phenomenon (virtual fuel spraying out from the connector) and alarm information (flashing red warning light and alarm sound).

[0083] Trainees need to follow the emergency plan: immediately close the refueling valve. The processing module monitors whether the valve has been rotated in the reverse direction to the closed position (0 degrees) via a rotary encoder, and monitors whether the connector has been removed from the refueling port via a UWB sensor (the pose sensor monitors the relative position and attitude of the corresponding physical component in real time).

[0084] The system compares the trainees' operational data with the preset virtual emergency response data in the digital twin model (the correct steps should be: first close the valve, then disconnect the connector) to determine the correctness and timeliness of their operations, and displays the results (such as "valve closing operation: correct, time taken 2.1 seconds") through the AR display module.

[0085] Step S5: Training Data Recording and Performance Evaluation Throughout the training process, the processing module continuously records all operational data (such as valve rotation angle and timestamp), docking operation judgment results (such as docking success / failure, time consumed), and every step of the emergency response.

[0086] After training, the processing module analyzes the data and generates a quantitative evaluation report. The report includes: Total training time: 8 minutes and 35 seconds.

[0087] Key steps assessment: The docking operation was successful on the first attempt, taking 4.2 seconds (excellent); the response time to the fuel leak fault was 2.1 seconds (good); the sequence of the handling procedures was correct.

[0088] Deficiency: The initial valve opening speed is too fast.

[0089] The report is sent to the coach's terminal for coaches to conduct precise debriefing and guidance.

[0090] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A training method for aircraft refueling based on AR and scrapped equipment, characterized in that, include: Step S1: The processing module loads the corresponding digital twin scene data according to the selected training subject and sends it to the AR display module; The AR display module identifies visual markers set on physical entity modules and spatially registers and overlays virtual aircraft models, virtual instrument interfaces, and real scrapped refueling equipment to form a training scenario that combines virtual and real elements. Step S2: By setting up sensor groups on key operating components of the physical entity module, the system collects real-time data on the student's actions on the actual physical components and sends the data to the processing module. Step S3: The processing module inputs the received operation data into the digital twin model for calculation, updates the status information of the virtual refueling operation, and sends the status information to the AR display module in real time; the AR display module overlays the status information on the real scene, and the status information includes virtual instrument readings, virtual fuel flow effects, and virtual equipment actions; Step S4: When performing the docking operation, the relative position and attitude of the physical joint and the physical refueling panel are monitored in real time by the pose sensor. The processing module compares the relative position and attitude with the preset virtual docking data in the digital twin model, determines the correctness of the operation, and displays the result through the AR display module. Step S5: The processing module records and analyzes all operational action data and docking operation judgment results collected throughout the training process, and generates a quantitative evaluation report.

2. The AR-based virtual-real combined training method for aircraft refueling with scrapped equipment according to claim 1, characterized in that, Step S4 further includes: When an emergency response is performed, the AR display module presents the corresponding virtual fault phenomena and alarm information. The pose sensor monitors the relative position and attitude of the corresponding real physical components in real time. The processing module compares the relative position and attitude with the virtual emergency response data preset in the digital twin model, determines the correctness of the operation, and displays the result through the AR display module.

3. The AR-based virtual-real combined training method for aircraft refueling with scrapped equipment according to claim 2, characterized in that, The processing module receives instructions from the instructor or emergency response procedures automatically triggered according to preset logic, and records the emergency response data performed by the trainee on the actual physical components.

4. The AR-based virtual-real combined training method for aircraft refueling with scrapped equipment according to claim 2, characterized in that, In step S5, the emergency response operation includes simulating one or more of the following: fuel leak, abnormal pressure, electrostatic alarm, and connector failure.

5. The AR-based virtual-real combined training method for aircraft refueling with scrapped equipment according to claim 1, characterized in that, Before step S1, the process also includes a step of harmlessly treating and adapting the scrapped aircraft refueling equipment, wherein the adaptation includes installing the sensor group and visual markers.

6. The AR-based virtual-real combined training method for aircraft refueling with scrapped equipment according to claim 1, characterized in that, In step S3, the visual feedback also includes presenting step-by-step operation guidance information through the AR display module. The operation guidance information includes any one or more of the following: three-dimensional arrows, highlighted borders, and text prompts.

7. A virtual-real combined training platform for aircraft refueling based on AR and scrapped equipment, characterized in that: For implementing the AR-based virtual reality training method for aircraft refueling with scrapped equipment as described in any one of claims 1-6, the AR-based virtual reality training platform for aircraft refueling with scrapped equipment includes: The physical entity module includes partially decommissioned aircraft refueling equipment that has undergone harmless treatment and adaptive modification. The aircraft refueling equipment includes a decommissioned refueling vehicle body and a decommissioned aircraft refueling panel, which are equipped with sensor groups and visual markers. AR display module is used to display scenes that combine virtual and real elements, operation guidance, and feedback information; The processing module is communicatively connected to the physical entity module and the AR display module. It internally constructs a digital twin model of the aircraft refueling operation and is used to process sensor data, drive the virtual scene, execute operation logic, and generate an evaluation report. The sensor group includes motion sensors mounted on the valves, joints, and reels of the physical entity module, as well as pose sensors for monitoring spatial position.

8. The AR-based virtual and real-world training platform for aircraft refueling using scrapped equipment as described in claim 7, characterized in that, The AR-based virtual and real-world training platform for aircraft refueling using scrapped equipment also includes an instructor terminal for selecting training subjects, triggering fault simulations, and viewing evaluation reports.

9. The AR-based virtual and real-world training platform for aircraft refueling using scrapped equipment as described in claim 7, characterized in that, The motion sensor is a rotary encoder or an inertial measurement unit, and the pose sensor is set as an ultra-wideband (UWB) positioning module or a visual tracking camera.

10. The AR-based virtual and real-world training platform for aircraft refueling with scrapped equipment as described in claim 7, characterized in that, The visual marker is a QR code or an ARUco marker.

Images